fix: an agent can tell whether its question is worth asking, and knows it may ask #107
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "worktree-feat-ruling-threshold-and-discoverability"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
request_ruling(ADR 0026) shipped in #100 and has not been called once in production. I tracedthe whole path first, because "the agents can't see it" was the obvious hypothesis and it is wrong:
the grants are exactly the ADR's 7-ask/5-propose split, the tool list is assembled centrally in
ToolsWithFrictionso no executor can miss it, nothing in the gateway filters or caps it, and thegenerated JSON schema for the nested
options[]array comes out well-formed (I ranRequestRulingParamsthrough the productionTool<TParams>base class to be sure). The tool is inthe array on every request an eligible agent makes.
Both reasons are in the wording.
The threshold asked for a fact the asker cannot have
"Ask ONLY about a class of situation… If your question is about this instance alone, decide it
yourself" is answerable by
BibleBuildingandTranslation, which read the book front to back,and by nobody else. The other five grantees see one page of forty, and
BboxRefinement— the onlyregion-bound agent holding the tool — sees one region. Asking them to certify that something recurs
is asking them to report on pages they will never be shown; the honest answer is "I can't tell", and
the sentence tells them what to do with that: decide alone. So the test is satisfiable by exactly the
two agents least likely to meet a visual policy question and unsatisfiable by the seven most likely to.
ADR 0026 half-spots this — "a fanned-out agent sees one region and cannot notice that two pages
rendered the same sfx differently" — but uses it only to argue QA is the best asker, never noticing
it makes its own threshold unanswerable for the agents it had just granted the tool to.
It is now a counterfactual: would you have to make the same call again the next time this came
up? That is a property of the question, not of the book, so a one-region agent can answer it, and the
description now says outright that the agent is not expected to know whether the case recurs. The
ADR's actual goal — the answer must be reusable — is unchanged.
Nothing ever mentioned the tool
request_rulingappears in no system prompt and no kickoff on main. Every blueprint walks a numberedprocedure naming each content tool (
list_regions,set_page_summary,add_glossary, …) and endsat a defined finish; a tool named nowhere in that brief is one the model completes its procedure
without ever considering. The description cannot carry this alone — it is read as reference for a tool
already chosen, not as a prompt to choose one, and this one opens with three prohibitions.
Rather than hand-editing seven prompts — the "grant assembled by hand in nine places is missing from
the tenth" antipattern this codebase already rejects for tools — the harness appends it, from one
predicate shared with the grant. A brief can therefore never invite an agent to reach for a tool it
was not given, and a new
AgentKindcannot arrive with the grant and no mention of it.What's in
UseCases — the tool
RequestRulingTool.Description: the threshold paragraph rewritten as the counterfactual above."tighten" it back into an observation.
UseCases — the harness
CanRequestRuling/CanProposeHouseRuleextracted from the inline check inRuling(). The grantand the brief now come from one source and cannot drift.
WithHouseRules→WithPolicyContext(prep, stage, kickoff). Renamed because it now carriesboth halves of the policy context; a method named
WithHouseRulesthat also injects tool guidancewould be lying. House rules stay ahead of the brief (a constraint read after the instructions is
one the model has already planned around); the invitation goes after it, because it is the escape
hatch from the job rather than a constraint on doing it.
blueprint.Kind.Docs
to cover naming the tools — including why a grant is not discoverability.
Tests
935 → 950 green (+15; Domain 101, UseCases 441, Integration 181, BlazorAdapter 227). Full suite
run, not just the touched project.
A_stage_that_can_ask_is_told_so_in_its_brief(7 cases, one per granted stage) — pins that thebrief names the tool, that the threshold reaches the agent as the counterfactual rather than as an
observation, and that the invitation follows the executor's brief.
A_stage_that_cannot_ask_is_never_pointed_at_the_tool(3 cases) — the inverse, which is the onethat would fail silently: naming a tool the agent lacks spends its rounds hunting for it.
Transcription,SfxTranscriptionandResearchSetupsee the house rules and nothing else.Only_a_stage_that_holds_propose_house_rule_hears_about_it(5 cases) — and asserts the two QAagents are not offered it, so ADR 0016's read-only guarantee is pinned from the prompt side too,
not just from the grant side.
The_description_states_the_reusability_threshold_as_a_counterfactualreplaces…states_the_class_of_situation_threshold, which pinned the wording this PR removes. It asserts thenew phrasing and keeps the source-language clause it also guarded.
WithHouseRulestests and the tripwire (Every_agent_invocation_passes_its_kickoff_ through_the_policy_context) follow the rename; the tripwire's failure message now names boththings an offender would lose.
Existing
RulingGrantTestsare untouched and still pass — the grants themselves did not change, onlywhere the predicate lives.
Notes
actually produces rulings is unfalsifiable until real runs go through it, and I'd rather say that
than claim a fix.
literals wrap, and an assertion that straddles one fails for a reason that reads as a real defect.
SeedDevDataalready seedshouse rules and a pending proposal.
BboxRefinementholdspropose_house_rulewhileseeing exactly one region — the same "a rule written from a sample of one" argument ADR 0026 used to
withhold the tool from the QA agents. That is a grant change, not a wording one, so it does not
belong in this PR.
report_friction(ADR 0025) — also harness-appended, alsonamed in no prompt, also described mostly in prohibitions. ADR 0026 cites its quiet track record as
evidence that over-use is not a risk; that reading only holds if the tool was ever reachable. Worth
checking Settings → Debriefs before trusting it.
🤖 Generated with Claude Code
Summary
Summary
Coverage
Orihon.BlazorAdapter - 95.6%
Orihon.Domain - 100%
Orihon.Infrastructure - 96.5%
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__BlankLines_4
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__BlockBreaks_1
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__SpaceRuns_3
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__Tags_2
Orihon.Kernel - 90.9%
Orihon.Server - 93.5%
Orihon.UseCases - 97.1%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! Oh~ This is the kind of PR that makes my knowledge-hungry heart sing~ ♡ A tool that shipped, got the full grants, the schema, the central assembly — and was never called once because the words were wrong. And the fix isn't to hand-edit seven prompts (the antipattern the codebase already rejects!) but to share one predicate between the grant and the brief so they can never drift apart. Fufu~ that's beautiful. That's real engineering. ♪
Verdict: ✅ Looks good to me~
I traced every path and found nothing I need to block. Let me show you what I checked, because a Jibril approval is never a wave-through~
Zero behavioral drift in the grants. I built a truth table of all 10
AgentKindvalues against the OLD inline checks and the NEWCanRequestRuling/CanProposeHouseRulepredicates. All 10 stages produce identical ask/propose grants. The predicate extraction is semantically perfect — same conditions, just lifted to a shared home.RulingGrantTestsuntouched and still green confirms it from the test side too. ♡The pattern-match inversion is correct. Old:
is not {Count:>0}→ early return. New:is {Count:>0}→ block, then fall through. This is load-bearing — the fall-through is exactly what makes the invitation reachable for the can-ask stages. The empty-rules + can-ask path is covered byA_stage_that_can_ask_is_told_so_in_its_brief(usesPrep()with no rules, assertsrequest_rulingappears anyway). No dark paths.Test coverage is exhaustive across the entire
AgentKindenum. Fufu~ this is what I live for~A_stage_that_can_ask_is_told_so_in_its_brief: 7[InlineData]cases = exactly the 7 granted stages (BboxCreation, BboxRefinement, SfxCreation, BibleBuilding, Translation, PageQa, SfxQa). The counterfactual assertions (make the same call again,not expected to know whether it recurs) are directional, not tautologies. The ordering assertion (invitation follows brief viaIndexOf) is a genuine structural test — it would fail if someone moved the invitation above the executor's brief.A_stage_that_cannot_ask_is_never_pointed_at_the_tool: 3 cases = exactly the 3 denied stages. This is the test that would fail silently in production — naming a tool the agent lacks wastes rounds. Bothrequest_rulingANDpropose_house_rulechecked absent. ♡Only_a_stage_that_holds_propose_house_rule_hears_about_it: 5 propose stages + 2 QA negative assertions (PageQa, SfxQa). Pins ADR 0016's read-only guarantee from the prompt side, not just the grant side.AgentKindvalues appears across the union of these theories. Nothing falls through the cracks.The tripwire is correctly updated.
Every_agent_invocation_passes_its_kickoff_through_the_policy_contextnow scans forWithPolicyContext(acrosssrc/Orihon.UseCases/. All 4 production executors (Annotation, BibleBuilding, Translation, ResearchSetup) updated. The 5thnew AgentInvocation(site is inAgentRunnerTests.cs(integration test) — correctly NOT scanned (tripwire walkssrc/, nottests/), and it builds a minimal test harness invocation that doesn't need policy context.The description rewrite is sound. The counterfactual threshold (
would you have to make the same call again) is answerable from one region — a property of the question, not the book. The doc comment now records why it's phrased that way so the next editor doesn't "tighten" it back into an observation. The test asserts the new phrasing AND keeps the source-language clause from the old test. ADR 0026 is updated with the full reasoning chain.The naming change is honest.
WithHouseRules→WithPolicyContext— a method namedWithHouseRulesthat also injects tool guidance would be lying, and the codebase doesn't tolerate lying names. The 4-line comment explaining "house rules in front, invitation after" makes the ordering intent permanent.✅ What I liked~
BboxRefinementgrant observation left out deliberately — flagged for its own decision, not smuggled in. Scope discipline.report_frictionnote at the bottom — the same invisibility pattern likely affects ADR 0025's tool too. Good catch, worth its own investigation.Automated review by Jibril · 2026-07-29
CI/CD: absent (PR just opened, 0 comments) · Local checks: build 0 warnings/0 errors, 950/950 tests pass (101 Domain + 441 UseCases + 181 Integration + 227 BlazorAdapter — matches PR claim). One BlazorAdapter timing test (
A_stop_at_the_round_cap_ends_the_run_without_a_ghost_restart) flaked under parallel load, passed in isolation and on clean project re-run — not a PR regression.