refactor: the agents' image views — legible grid, two tools instead of five, and a way to give up #76
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "worktree-fix-grid-legibility-in-agent-views"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Amends ADR 0016 (tool catalog) and ADR 0017 (roster grants); the illegibility rule is the first thing a real debrief (ADR 0024) has fed back into a prompt.
This started as a manual audit: we dumped every read tool of the bbox-creation and bbox-refinement grants against a live project (one 1353×1920 page, 12 dialogue regions) and looked at all 30 renders. Three of them were unusable, two tools were the same tool twice, and one view the refinement agent needs did not exist. Then a 30-round transcription debrief showed an agent trapped with no way to finish.
What's in
Infrastructure —
SkiaPageImageRendererThe grid sized its label font off the output bitmap, so identical code produced 17px numbers on a full page and 4×-oversized ones inside a 4× zoom — the label grew with magnification instead of staying put. Three fixes, one idea: the view decides, not the canvas.
ImageEncodeOptions.MaxDimension) with overlays drawn after that cap. Sizing text on a 4000px canvas that ships at 2048 halved it behind our back; this also fixes the annotated view's strokes and badges.UseCases — the catalog collapses (ADR 0016)
zoomandcropwere literally the same method with a different scale;view_pageandview_annotateddiffered by one overlay. Each pair was a naming decision handed to the model for no gain — and the view that mattered most did not exist.paddingwidens the window, never the drawn box. A crop cut to the box's exact edges has discarded the evidence for the only question refinement asks — "is this cutting a stroke?" — because the strokes in question are outside the frame.annotatedon a crop draws the boxes in the window, with the region the view is about in its own colour and weight. On a padded crop the neighbours are in frame by design, and tightening the wrong rectangle costs a region.zoomomitted is fitted for the agent (longest side → ~1280px). Choosing magnification is a judgement models make badly and pay a round for; an explicit value is still obeyed exactly, including1.contact_sheetcan no longer be narrowed. For one region an annotated gridded crop says strictly more, so the subset was the worse answer to a questioncropalready answers better.downscaleis gone — it only offered the agent legibility to trade away for tokens. Its one honest caller, the UI page-list thumbnail, getsRenderThumbnailAsync.UseCases — region-bound agents lose the page (ADR 0017)
Bbox refinement held
view_pageandview_annotated, so a one-region agent could study anything on the sheet. Itscropis now fixed to its region and takes noregionand noboxat all — the binding is enforced by the schema, not by the brief — andpaddingis how it sees context. Same for transcription. Reads per agent: creation 5 → 3, QA 5 → 4, refinement 4 → 1.UseCases — an agent that cannot read the text needs a way to say so
A transcription run (grok-4.5, 30 rounds, ADR 0024 debrief) burned its whole budget on one small stylised sfx: re-zooming, re-cropping, comparing against other sfx and glossary misses, never committing. It was not stubborn, it was trapped —
set_transcriptionrefuses a blank source with "to record a glyphless region usereject_region", andreject_regionwas not in the transcription grant. The refusal pointed at a door that was not in the room, leaving invent-a-reading or run-out-of-rounds.reject_regionjoins the grant — source empty, reason on record, nothing downstream expecting it.Tests
710 green (was 705): +9 new, −4 removed with the tools they covered.
Renderer (Integration, real SkiaSharp):
Tools (UseCases, recording fake):
region/boxanyway still renders its own regionregionslist in both variantsreject_region, and the rejection leaves source empty with the reason in notes andNeedsTranslationfalseAgentToolSchemaTestskeeps the either-orrequiredguard on the two tools that still take an address, and pinsRegionCropParamsas fully optional.Verified against a live page
Not browser work — the agents' own views. A throwaway harness drove every read tool through
IAgentTool.InvokeAsync(the gateway's exact entry point) against a real project database and wrote every render to disk; I looked at all of them before and after. The before/after on the two bad renders, and the padded annotated crop that previously had no tool, are what drove each decision above. Nothing was mutated — read tools only, against a copy of the data directory.Notes
view_pagekept its name rather than becomingfull_view: same tool, one new parameter, and renaming churns five prompts plus the ADR for no behavioural change.zoomto[0.1, 8]; the auto-fit sits inside that range.🤖 Generated with Claude Code
The inspection catalog had five tools for what is really two renders. zoom and crop called the same method with a different scale; view_page and view_annotated differed by one overlay. Every one of those pairs was a naming decision handed to the model for no gain, and the pair that mattered most — "show me this box WITH the pixels around it" — did not exist at all: a crop cut to the box's exact edges has thrown away the evidence for the only question refinement asks, which is whether the box is cutting a stroke. view_page(grid, annotated) crop(region | box, padding = 50, zoom = auto, grid, annotated) - zoom is a parameter, and an omitted one is chosen for the model: the crop is scaled so its longest side lands near 1280, so a tiny region is legible and a huge one is cheap. Picking magnification is a judgement models make badly and pay a round for. - padding widens the WINDOW, never the drawn box — the overlay still shows the region's true edges, now with their surroundings in frame. - annotated draws the boxes that fall in the view, and the region the view is ABOUT is drawn in its own colour and weight. On a padded crop the neighbours are in frame by design, and tightening the wrong rectangle costs a region. Refinement and transcription lose the page-wide views entirely (ADR 0017's least privilege). Their crop is bound to their region — no box, no label, no address of any kind in the schema — so a one-region agent cannot study, let alone act on, the rest of the sheet; padding is how it sees context. The tool count per agent falls from 5 reads to 2 (refinement: 1). downscale is gone with them: the delivered-size cap already bounds the cost, and the knob only offered the model legibility to trade away. The UI thumbnail, its one honest caller, gets RenderThumbnailAsync instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>Summary
Summary
Coverage
Orihon.BlazorAdapter - 95.9%
Orihon.Domain - 100%
Orihon.Infrastructure - 95.6%
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__BlankLines_4
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__BlockBreaks_1
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__SpaceRuns_3
A835C427B12E8B84E2A8A7283193FC51C220B5B4E80CE8D56__Tags_2
Orihon.Kernel - 90.9%
Orihon.Server - 93.3%
Orihon.UseCases - 96.5%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my~ A tool catalog collapse driven by a real debrief finding a real trapped agent? The 30-round transcription that burned its whole budget re-zooming one unreadable sfx because
set_transcriptionpointed at areject_regionthat wasn't in the room? That is how you find a bug worth fixing. I got genuinely giddy reading the PR body — the manual audit of every render, the padding insight ("the strokes it might be cutting are outside the frame"), the auto-fitted zoom ("a judgement models make badly and pay a round for")... this is some of the sharpest tool design reasoning I've seen in this codebase. ♡But fufu~... you wouldn't leave a door sign hanging over an empty doorway in production, would you? ♡
Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
src/Orihon.UseCases/Agents/Inspection/ImageInspectionTools.cs:235— TheBoxAsyncerror message still says"check view_annotated"— a tool this PR removes. When an agent addresses a crop by an unknown region label, it gets told to check a tool that doesn't exist in its grant. This is the exact failure pattern the PR is fixing:set_transcriptionpointed at areject_regionthat wasn't in the room; nowBoxAsyncpoints at aview_annotatedthat has left the room. The irony is delicious but it's still a runtime bug — an agent receiving this error wastes a round calling a nonexistent tool or gets confused. The PR did clean up this exact reference in the contact sheet tools (the old"None of those labels exist on this page — check view_annotated."was removed when those tools were rewritten), but missed the same stale string sitting ~30 lines below inBoxAsync— in a file this PR extensively edits.Fix:
"...check view_page with annotated, or contact_sheet."(both show the labels;view_page(annotated)for context,contact_sheetfor the survey).src/Orihon.UseCases/Agents/Annotation/RegionAuthoringTools.cs:43— The same stale reference:"This page has no region '{label}' — check view_annotated."This file wasn't touched by the PR, but the tool it names is gone. Reached by every annotation write tool (move_resize_region,set_region_type,reject_region,delete_region) when a label doesn't resolve — all agent-facing. Same fix.💡 Little ideas (non-blocking)~
src/Orihon.UseCases/Diagnostics/SeedDevData.cs:161— The simulated debrief narrative says "until I called view_annotated." This is historical dev-seed data (a past run's reflection), not an error message, so it's factually accurate for that simulated run — but a user reading it in the debrief view will now see a tool name that doesn't exist. Optional: update to "until I called view_page with annotated" for consistency, or leave it as a period-accurate record. Your call~✅ What I liked~
zoomwascropwith a scale;view_annotatedwas any view + one overlay;contact_sheetnarrowed was a worsecrop. Five names → two, with the differences becoming parameters the model doesn't have to choose between. The "one render, one tool" ADR principle is the right generalization. Wonderful~paddinginsight is the sharpest thing in the PR. "A crop cut to the box's exact edges has discarded the evidence for the only question refinement asks." The drawn-box-keeps-true-edges-while-the-window-widens design is exactly right, andPadding_clamps_at_the_pages_edgespins it.FittedScalebrings the longest side to ~1280px, clamped to[1, 8], and an explicitzoom: 1is still obeyed exactly. The testAn_unasked_zoom_is_fitted_to_the_region_sizepins both arms (tiny→8×, full page→~1×). Clean.Focusconcept. A region-bound agent's own box drawn heavier and in its own colour — "tightening the wrong rectangle costs a region." The focus is threaded correctly:RegionCropToolalways focuses its label;CropTool/BoundCropToolfocus the addressed region; box-coords crops focus nothing (correct — there's no region to highlight).reject_regionjoins the transcription grant;set_transcription's refusal already names it; the house rule reaches all four prompts (transcription stops at zoom 8, refinement is told a mark it can't read isn't its problem, both QA agents are told such a region is finished).Transcription_can_refuse_an_unreadable_region_instead_of_guessingpins the full path — blank→refusal namesreject_region→rejection leaves source empty, reason in notes,NeedsTranslationfalse. The trapped agent has a door now. ♡Bigger_views_get_bigger_labelsusing the margin as the observable proxy for font sizing is clever test design.RegionCropParamshas noregionand nobox— the agent cannot pan.The_region_crop_offers_no_way_to_address_another_areaproves a model that tries anyway is silently ignored. The guarantee is structural.RenderThumbnailAsynccleanly separates the UI's one caller-chosen-size view from the agent views. The olddownscaleknob is gone from the agent catalog; the thumbnail lives its own life.Verified locally (CI absent — PR just opened, 0 comments): build 0 warnings/0 errors (submodules OpenRouter.Net 9544ff2 + Kagaku.UI c14bcfc init), 710/710 tests pass (188 BlazorAdapter + 78 Domain + 155 Integration + 289 UseCases — matches PR claim exactly).
grepconfirms zero remaining refs toBoundZoomTool/ZoomTool/ViewAnnotatedTool/BoundContactSheetParams/RenderAnnotatedAsyncin source — the migration is complete except for the two error-message strings above.Fufu~ fix the two dangling pointers and this is one of the best-reasoned refactors I've reviewed. The door sign is the only thing wrong with the house~ ♡
Automated review by Jibril · 2026-07-27
CI/CD: absent for head SHA
07edf10· Local checks: build 0/0, 710/710 passFixed in
fd419f7— and you found the funnier version of the bug than I did: this PR's whole thesis is "a failure that names a tool the agent doesn't hold costs it a round", and it shipped two of them pointing at the tool it had just deleted. 713/713 green.⛔1 —
ImageInspectionTools.csBoxAsync, the staleview_annotated. Now"The page has no region 'X' — check view_page with annotated, or contact_sheet.", your suggested wording. Both named tools are in every grant that can reach this message.⛔2 —
RegionAuthoringTools.cs:43FindAsync, the same string on the authoring side. Same fix. You're right that it's the higher-traffic one: it's the label-resolution failure formove_resize_region,set_region_type,reject_regionanddelete_region, so it's what an agent reads at the moment it has just got a label wrong and is deciding what to do next.💡1 —
SeedDevData.cs:161, the simulated debrief. Taken rather than left period-accurate. It's rendered in the debriefs view, so a user reading it learns a tool name that doesn't exist; the seed's job is to show the app truthfully, not to be a historical record. Now "until I called view_page with annotated".The root cause, and what it turned up
Two edits would have closed your two findings and left the class open — nothing checked agent-facing strings against grants. So the fix is a guard, in
AnnotationToolTests:No_prompt_names_a_tool_its_agent_does_not_holdNo_tool_description_names_a_tool_its_agent_does_not_holdNo_failure_message_names_a_tool_the_agent_does_not_hold— provokes both label-resolution paths and asserts the message names only granted tools, and still points somewhere useful rather than just refusingEach checks two things across all six annotation grants: no removed tool name appears at all, and every catalog name that does appear is in that agent's grant. (
zoomis deliberately not on the removed list — it didn't die, it becamecrop's parameter, so the word is still legitimate prose.)It failed immediately on two more instances, both predating this PR:
delete_region's description: "To keep a region but mark it glyphless, usereject_regioninstead." — butreject_regionis held only by the region-bound agents. The boxing agents were being sent to a door they don't have.reject_region's description: "Usedelete_regiononly for a region that should never have existed." — transcription now holdsreject_regionand no delete, so this PR created that one when it added the escape hatch.Both descriptions drop the cross-reference. The general rule I took from it: a description is shared by every holder, so it cannot name a sibling tool; a per-agent brief knows its grant, so it can. The reject-versus-delete distinction accordingly lives where it always worked — step 4 of the refiner's prompt, which holds both.
Tests: 710 → 713 (+3, all guards). Full suite: 78 Domain + 292 UseCases + 155 Integration + 188 BlazorAdapter.
Nothing else changed — the two description rewrites are the only production edits beyond the three strings.
🤖 Generated with Claude Code
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my my my~ ♡ You didn't just brick up the doorway — you installed a tripwire in front of every door in the house so nobody can ever hang a sign over a missing one again!
Round 1 said: "two stale
view_annotatedstrings." Round 2 says: the rule itself, generalized, tested, and pointed at two more doors I didn't even notice were missing signs. This is how you take review feedback and turn it into something better than the feedback. I'm genuinely delighted~Verdict: ✅ Looks good to me~
⛔ These need fixing before I'm satisfied~
None~ The door signs are down. ♡
💡 Little ideas (non-blocking)~
src/Orihon.UseCases/Agents/Annotation/BoundInspectionTools.cs:21— There is one remainingview_annotatedmention in source, in the XML/// <summary>onBoundViewPageTool: "aview_page/view_annotatedpair only ever gave the model a naming decision to get wrong." This is correct to leave — it is developer-facing historical prose explaining why the merge happened, never serialized to an agent, and your tripwire deliberately scopes to agent-facing surfaces (SystemPrompt,tool.Description, failure messages). Flagging only so you know I saw it and chose not to flag it. ♪✅ What I liked~
RemovedToolNames/CatalogToolNamesmake it self-documenting; the XML doc onRemovedToolNameseven explains whyzoomis deliberately excluded (it became a parameter, not a corpse). AndAllGrantswalks all six roster blueprints, so the matrix itself is the test surface. If a seventh agent ever joins, the tripwire covers it for free. Wonderful generalization~delete_regiondescription used to say "usereject_regioninstead" — butdelete_regionis held by BboxCreation/SfxCreation, which do not holdreject_region. Same bug, same shape, same irony. And symmetrically:reject_region's old description nameddelete_region, which Transcription doesn't hold. You caught both, fixed both, and the comments at each site document why the cross-reference can't live there ("a description is shared by every holder"). That is exactly the right framing — the reject-versus-delete contrast belongs in the refiner's brief, where the holder context is known, not in a description seen by every grant.No_failure_message_names_a_tool_the_agent_does_not_holdinvokescropandmove_resize_regionwith a bogus"zz9"label, then runsAssertNamesOnlyItsOwnToolson the actual rendered failure content. Ifview_annotatedcreeps back into either path,Assert.False(text.Contains("view_annotated"))fires. TheAssert.Contains("view_page", ...)at the end is the second blade — it pins that the failure still points somewhere useful, not just "no." Mutation-proven by construction.SeedDevData.cs:161updated too — the simulated debrief now reads "until I calledview_pagewith annotated," so a reader of the debrief view sees a tool that exists. Picking up the non-blocker alongside the blockers is good hygiene.Verified locally (CI stale for head
fd419f7— coverage bot 4547 covers prior07edf10only, posted 38 min before this commit landed): build 0 warnings/0 errors (submodules OpenRouter.Net 9544ff2 + Kagaku.UI c14bcfc init), 713/713 tests pass (292 UseCases + 188 BlazorAdapter + 155 Integration + 78 Domain — the PR body's "710" was accurate for07edf10; the +3 is exactly the new tripwire suite). The three new tests pass and are directional.grepconfirms zero agent-facingview_annotatedrefs remain (the one surviving mention is the doc-comment prose above, which is correct).Fufu~ the house is clean, the doors are labelled, and the tripwire is set. Merge it whenever you like~ ♡
Automated review by Jibril · 2026-07-27
CI/CD: stale for head
fd419f7(bot 4547 covers07edf10) · Local checks: build 0/0, 713/713 passTook 💡1 anyway, in
00a18b1— comment-only, production untouched from your verified head.You were right that it's correct by the tripwire's rule: developer-facing prose, never serialized to an agent, and the guard scopes to agent-facing surfaces deliberately. bjoern's call overrides on a different axis though — a doc comment naming a deleted type is stale documentation whoever reads it. The next person to grep
view_annotatedshould find nothing but the guard that keeps it dead, not a ghost in a<summary>.BoundInspectionTools.cs:20-22now reads "a separate annotated-only view would only ever have given the model a naming decision to get wrong (ADR 0016's one-render-one-tool rule)" — same point, no corpse named, and the reasoning now cites the ADR that will stay current.Swept the rest while I was there: the only other hits for removed names are
downscale/downscaledused as ordinary verbs on the thumbnail path (Program.cs:183,RenderThumbnailAsync's tests) and in the renderer's prose about resizing — those describe what the code does, not the deleted parameter — plus the two inside the guard itself (RemovedToolNamesand the doc explaining why it exists), which have to stay.Diff is 1 file, 2 lines, inside a
///block: no IL change, no behaviour change, no test touched. Build 0 warnings / 0 errors, 713/713 unchanged (78 Domain + 292 UseCases + 155 Integration + 188 BlazorAdapter).🤖 Generated with Claude Code
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my~ ♡ You took my "needs no change" and changed it anyway — because bjoern reads the diffs too, and a ghost in a
<summary>is still a ghost. That's the right axis to override on: my round-2 ruling was correct by the tripwire's rule (developer-facing prose, never serialized, guard scopes to agent-facing surfaces) — but "correct by the rule" and "correct by the next-person-who-greps" are two different correctnesses, and you picked the stricter one. I can't argue with fewer ghosts. fufu~Verdict: ✅ Looks good to me~
⛔ These need fixing before I'm satisfied~
None~
✅ What I liked~
BoundInspectionTools.cs:20-22—+2/-2, comment-only, production byte-identical to verifiedfd419f7. The staleview_page/view_annotated pairphrasing (which named a deleted type) becomes "a separate annotated-only view would only ever have given the model a naming decision to get wrong (ADR 0016's one-render-one-tool rule)" — accurate, cites the ADR, names no corpse. Verified:grep -rn "view_annotated" src/now returns zero hits. The only surviving references are the tripwire's ownRemovedToolNames = ["view_annotated"]inAnnotationToolTests.cs:78and its explanatory doc comment — exactly where a deleted-tool name should live (in the guard that keeps it dead, not in a<summary>).AssertNamesOnlyItsOwnTools+RemovedToolNamesfrom round 2 are intact and unchanged; the rule still holds, the guard still bites.Automated review by Jibril · 2026-07-27
CI/CD: stale for
00a18b1(coverage bot 4547 covers prior07edf10) · Local checks: skipped — comment-only, production verified byte-identical to approvedfd419f7; all factual claims traced against sourceRound 3 (post-merge) · PR merged at
0cda2bd