feat: assistant history budget and cache-stable system prompt #76
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/assistant-context-budget"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What & why
The assistant's persistent history grew without bound: huge tool results accumulated until the model's context overflowed, after which every turn failed identically and clearing the conversation was the only escape. The system prompt also embedded volatile state (date, route, memories), invalidating the OpenRouter prompt-cache prefix on every turn.
History compaction (
agent_service.dart)compactHistorypass runs before each turn and after each sync. Over budget, it first swaps old tool-result contents (never the newest exchange's) for short[result trimmed: <tool>, N chars]placeholders, then drops whole oldest exchanges; a single note at the front keeps the running dropped-turn count across repeated compactions.DOUJIN_MANAGER_AGENT_HISTORY_BUDGET_CHARS(default 200000 chars, documented in the README). Inline images count a flat estimate instead of their base64 length.reasoningContent/ serializedreasoningDetails) counts toward the estimate too — history replays re-send it, so ignoring it let reasoning models exceed the budget.Cache-stable system prompt (
system_prompt.dart)The system prompt is now static per session. Volatile state (date, current route, viewed doujin, memories) moved into a per-turn context message that is replaced each turn instead of accumulating, so the provider's prompt-cache prefix survives across turns.
Review fix: reflection starvation
The self-reflection gate compared
_history.lengthagainst a watermark taken at the last reflection, but compaction makes the length non-monotonic — once exchanges start dropping, the delta never reaches the threshold again and reflection starves permanently in exactly the long-conversation regime the budget targets. The gate now compares a virtual monotonic length (live messages plus everything compaction ever dropped). Regression test verified to fail against the pre-fix gate.Testing
flutter analyze: cleanflutter test test/agent_compaction_test.dart test/agent_history_test.dart test/agent_reflection_test.dart: 29 tests, all passing (compaction passes, note folding, protected newest exchange, reasoning estimate, cache-stable prompt, reflection-after-compaction)🤖 Generated with Claude Code
Flutter Coverage
Total: 75.1% (6016 of 8010)
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my~ A history compactor with a cache-stable prompt split AND a self-aware fix for the starvation your own compaction would have caused? You found the bug your feature creates and fixed it before I could sink my teeth into it. I'm equal parts delighted and deeply disappointed I didn't get to catch it myself~ ♡
Verdict: ✅ Looks good to me~
I traced every line of the +652/−66, cross-checked the vendored
openrouter_dartmessage shapes, and ran everything locally (CI hasn't reported for5ec393cyet). No blockers. Here's the evidence:Verified locally:
flutter analyze→ No issues found. Targeted suites 29/29 pass (matches the PR body exactly). Full suite 505/505 pass. And because green tests prove nothing on their own — I ran a mutation probe: reverted the reflection gate to the raw_history.lengthcomparison, andreflection keeps firing after compaction drops exchangesgoes red withExpected: non-empty, Actual: []. The regression test is genuinely directional, not a tautology. Restored afterwards, tree clean.Traced, not trusted:
[-prefix turn-opener convention is load-bearing, so I checked every syntheticUserMessagethe vendored agent can inject: the leaked-tool-call corrective ([System] ..., agent.dart:57), tool-image injections (multimodal →text == null), plus your own context message and dropped-note. All four correctly fail_isTurnOpener. The convention holds._runTurn, post-run in_syncHistory) — I traced the counters for double-accounting: the pre-request pass records drops against pre-compaction length, the sync pass re-compacts already-compacted content (no-op). No double-counting of_droppedByCompaction, and the note-folding test pins idempotence across repeated compactions._virtualHistoryLengthgrows +2/turn → delta 8 reacheskReflectionThresholdafter four turns. The test drives exactly this regime withhistoryBudgetOverride = 1500.estimateMessageCharsmatches whattoJsonactually re-sends: reasoning content AND serializedreasoningDetails(encrypted blobs included) both counted — that's the correct wire-format view, and the flatkImageBudgetCharsper image avoids evicting a conversation over a couple of base64 covers. Both pinned by tests.agent_history_test.dart—isNot(contains('Current page'))plusmessagesOf(0).first['content'] == messagesOf(1).first['content']. Exactly ONE context message per request, replaced not accumulated. That's the claim, proven.clearConversationresets_droppedByCompaction— no stale virtual length leaking into a fresh conversation. ♪💡 Little ideas (non-blocking)~
'assistant-${_history.length}'is now non-monotonic (compaction shrinks the length), so two text entries can theoretically share an id across turns. I verified nothing consumes it for identity (the reducer id-matches only tool/approval entries; noValueKeyon panel entries), so it's purely cosmetic — a_messageCounterwould make it forever-unique if you ever care._isTurnOpener— a user message that *starts with[(say a pasted[info] ...) escapes opener detection, so its exchange loses pass-1 protection and its tool results could be placeholder-trimmed one turn early. Mild degradation, no protocol break (the placeholder keeps thetoolCallIdpairing valid), and the bracket convention is established in the vendored agent — but if you ever want it airtight, marking synthetic messages with a dedicated field instead of a text prefix would remove the collision class entirely.✅ What I liked~
[Conversation note: N earlier turn(s)...]instead of stacked notes) is exactly the kind of small detail that keeps long conversations clean.estimateMessageCharspinned againstjsonEncode(detail.toJson()).length(the actual serialized form!), the under-budget test assertingcompacted == historyidentity, the budget-parse junk arms. Nothing is a coverage-tick.Fufu~ merge it when CI agrees with me~ ♡
Automated review by Jibril · 2026-08-15
CI/CD: absent for head SHA
5ec393c· Local checks: analyze 0 issues, 29/29 targeted, 505/505 full suite, mutation probe verifiedVerification follow-up since the ✅ (and the 505/505 local run) predate the four merges that landed on
maintoday (#72–#75):mainsince the merge-base — which is whymergeable=trueholds.feat/assistant-context-budget⊕ currentmain, includes #75's widenedDoujinRepository.listDoujins— the interface this PR's agent tests fake around) and ran it:flutter analyze— cleanflutter testfull suite — 566/566 (your 505-era baseline + the tests #74/#75 added + this PR's net-new)Nothing on the branch changed; head remains
5ec393c. Ready to merge as-is.