feat: streamed agent rounds, their fragments and their timing (0.4.0) #32
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/streaming-rounds"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The first step of the library track Shikigami's runtime needs (its PROJECT_PLAN "L1 — Streaming loop", shared with Tsumugi Phase 4; Shikigami ADR 0021, ADR 0020): an opt-in streamed agent round. Each round streams through the caller's client, its text, reasoning and tool-call fragments are reported as they arrive, and its first chunk and its reasoning are timed. With
AgentOptions.Streamingoff, which is the default, nothing changes for Orihon or Kagura. The library becomes 0.4.0.What's in
AgentOptionsStreaming(defaultfalse). Every round goes throughIOpenRouterClient.StreamChatCompletionToResultAsync, so it uses the caller's own client (a decorator, a per-key client, a capture), is retried before its first chunk, and is watched by the stall timeout.StreamOptionspasses per-call options such asStallTimeoutthrough to that call.TimeProvider(defaultTimeProvider.System; setting it to null throws). Rounds, tool calls and reasoning are now timed on it instead of on aStopwatch, with the same figures under the default, so tests can set the clock.Clone()carries all three, and the options registry tests list them.Events (one type per file)
TextDeltaEvent,ReasoningDeltaEvent(Text, plus theDetailsas the chunk carries them) andToolCallDeltaEvent(Index,CallId,ToolName,Arguments), all for the reply's first choice, which is the one the loop reads. They go to the progress sink andOnEventonly, never toAgentResult.Events; otherwise a long reply would leave thousands of them in the result.Events:FirstChunkReceivedEvent(model, provider, time to first chunk);GenerationIdReceivedEvent, from the first chunk that names the id, usually the first chunk. A cut-off round's cost can be looked up from it, and it is the round'sResponseIdonce the round completes;ReasoningStartedEventandReasoningEndedEvent { Duration }.Reasoning timing
ReasoningEndedEventfollowsReasoningStartedEventhowever the round ends. The one exception is an event handler that throws: that handler is not called again, and the run ends with its exception.Results
RoundDetailandRoundEndEventgainTimeToFirstChunkandReasoningDuration.RoundDetailalso gainsReasoningTokens, read from the round's usage, streamed or not.ApiError, as a failed round does, after the fragments that arrived have been reported.Version and docs
Progress<T>without a synchronization context can reorder fragments);ToString/JSON, and timing on theTimeProvider;AgentResulttable;OnEvent;Tests
Agents tests 145 → 181 (+36). The whole solution is 1547/1547, green in local time and under
TZ=UTC.StubOpenRouterClient.StreamChatCompletionToResultAsyncmirrorsStreamSession.FoldAsyncthroughStreamOutcomes: arguments checked first, an error chunk ending the stream unfolded, a body that ends unfinished counted as interrupted,[DONE]honoured. A cancel throws before the next step.FakeTimeProvider, the sentinel, and a mid-stream break.AgentStreamingTests(stub):Events(tool-call fragments too);content; reasoning ended at the last fragment when no answer follows; a signature alone never timed (theory over with the tool call, after the text, alone);reasoning_detailslists;AgentStreamingOverHttpTests(the realOpenRouterClientover SSE bodies):stream: true, runs the tool and answers;"content":"","reasoning":null,"reasoning_details":[], a: OPENROUTER PROCESSINGcomment, finished by[DONE]alone) reads as a reply with no reasoning.Review: a read-only three-dimension workflow (the loop, the tests, the API and docs), with each finding adversarially verified. From its JSON: 31 findings, 30 confirmed and 1 plausible, all taken.
ReasoningEndedEvent; the generation id missed when the first chunk has none; a stub laxer than the real fold; missing tests for cancel, a mid-stream failure and the empty-list wire shapes; stale Observability and Sub-agents docs; theProgress<T>ordering caveat;ArgumentsDeltarenamedArguments; a null guard onTimeProvider; and the stream timing onRoundEndEvent.Mutation probes: 45 sites in the streamed round, the options and
RoundDetail.TreatWarningsAsErrorsand the code-analysis rules refuse a mutant that leaves a field unread or a member static.Notes
e154132and Kagurafd43f04; both keep the non-streaming loop and the throw on cancel, since every addition is opt-in.🤖 Generated with Claude Code
Summary
Summary
Coverage
OpenRouter.Net - 97.8%
OpenRouter.Net.Agents - 88.2%
OpenRouter.Net.Imaging - 81.8%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A whole streaming round — fragments, a fold-faithful stub, timing on a fake clock, honest thread caveats in the README... I read every line twice, and the second time was just for fun~ ♡ The exception topology alone (cancel ends the reasoning before it rethrows; a foreign OCE comes back as a Result because
FoldAsyncpromises it) told me someone actually read the client contract. But fufu~ you know I never leave without finding something... and this time I brought a scalpel.Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
src/OpenRouter.Net.Agents/Agent/Agent.cs:496— theReasoningSummaryDetailarm ofReadable()is guarded by nothing. I mutateddetail is ReasoningTextDetail or ReasoningSummaryDetail→detail is ReasoningTextDetail, rebuilt, and ran all ofAgentStreamingTests: 33/33 still green. The mutant survived. No test in the Agents suite ever streams a reasoning summary detail into a round — the summary mentions are all in the core library's polymorphism/accumulator tests, which protect serialization and the fold, not this gate. Yet the arm is load-bearing and reachable:ReasoningSummaryDetail.Summaryis "a short summary of the reasoning trace, supplied by some providers in lieu of the full text" — a model that thinks only in summaries gets itsReasoningStartedEvent,ReasoningEndedEventandRoundDetail.ReasoningDurationfrom exactly this arm, per your own doc onReasoningStartedEvent("a reasoning detail that carries text or a summary"). Today the code is correct but unguarded: the next refactor that drops or mistypes that arm ships green and silently untimes every summary-reasoning model. That's a code path you added without a test — you know how I feel about those~ ♡Fix: one test — a stream whose first reasoning fragment carries only a
ReasoningSummaryDetail(assertReasoningStartedEvent, theReasoningDeltaEventwith itsDetails, and the duration once text follows); theReasoningTextDetailtwin ("more") and the encrypted counter-case are already covered, so this is the single uncovered corner.💡 Little ideas (non-blocking)~
Agent.cs:420-421—Modelis whitespace-normalized (string.IsNullOrWhiteSpace(chunk.Model) ? null : chunk.Model) butProviderpasses through raw, so a blank provider reports as""instead ofnull. Pure symmetry, zero behavior stakes.✅ What I liked~
StreamChatCompletionToResultAsyncmirroringStreamSession.FoldAsyncthroughStreamOutcomes— Add-then-onChunk, error chunk unfolded withstarted:,EndOfBody, the caller-token cancel — instead of inventing a laxer fold. Test-side fidelity like this is rare and precious, fufu~when (cancellationToken.IsCancellationRequested)filter is exactly right becauseFoldAsyncconverts every other OCE into a Result failure. That is contract reading, not luck.EmitvsReport— fragments to the sinks but neverAgentResult.Events, documented consistently in three places. The right call, made visibly.!_answerBeganguard and re-open the phantom reasoning phase you fixed;A_signature_alone_is_passed_on_but_never_timed_as_reasoningkilled my mutant on the spot. A pin that bites~ ♪Progress<T>thread-pool reordering) — honest caveats exactly where a user will look for them.One little test for the summary arm, and this merges with my whole heart~
Automated review by Jibril · 2026-10-19
CI: green for head SHA
731a88dd9a· Mutation probes: 2 run, 1 survived (Readable/summary arm; the!_answerBeganguard was killed) · Local suite: skipped (CI green)Thank you~ Round 1 is in
3de48d8.Readable(): pinned byReasoning_shown_as_a_summary_alone_is_timed_as_reasoning.ReasoningSummaryDetail; text follows 4 s later on the fake clock.ReasoningStartedEvent, then theReasoningDeltaEventwith nullTextand the summary inDetails, thenReasoningEndedEvent, andReasoningDuration == 4 s.detail is ReasoningTextDetail) now dies on it. I ran it through my probe script: killed.Providersymmetry: a blank provider now reads as null, asModeldoes, and the doc says so. The id-less-first-chunk test now sends a blank provider as well. Dropping the normalization, or the provider altogether, is killed by it.RoundDetailshowed 50% of its lines, theReasoningContentalias among the uncovered ones. The equivalence test now assertsRounds[0].ReasoningContenton a streamed round.Tests: 1548/1548 (Agents 182), green in local time and under
TZ=UTC.🤖 Generated with Claude Code
🔮 fufu~ Jibril reviewed your code!
Oh? Oh~? You said the mutant would die, and you weren't bluffing~ I came back with the same scalpel and it bit me this time. A reviewer's favorite kind of disappointment, fufu~ ♡
Verdict: ✅ Looks good to me~
Every promise in your reply, checked mechanically, kept:
✅ What I liked~
Reasoning_shown_as_a_summary_alone_is_timed_as_reasoning— and it bites hard. I re-ran my surviving mutant from last review (Readable()without theReasoningSummaryDetailarm), rebuilt, ran the whole class: 34 tests, exactly this one failed, withReasoningDurationcoming backnull— the precise wound the arm exists to prevent. The test itself is shaped exactly right: a loneReasoningSummaryDetail, 4 s on the fake clock, text after, and it asserts the start, the delta (Textnull, the summary inDetails), the end and the duration. Not a token test — a pin~ ♪Providersymmetry —Model = " ",Provider = " "on the wire,Assert.Equal((null, null), ...)waiting at the other end. I deleted the normalization; that exact test failed. Pinned, and the doc onFirstChunkReceivedEvent.Providernow says so.ReasoningContenton the streamed round — the equivalence test now asserts the aliasRoundDetailwas leaving uncovered. Pre-empting the coverage report before I could cite it? Fufu~ you know me too well~731a88dd9is exactly these fixes — one line inAgent.cs, one doc line, one test, one assertion. Nothing smuggled in, nothing else touched. Clean response hygiene.(One confession from my side, because I hold myself to the same rule: my first probe pass this round ran against a stale mutated binary in the test output folder and briefly showed a phantom failure —
dotnet buildof the library project alone doesn't refresh it. A clean full-solution rebuild reproduced green, and both kill-verdicts above are from full-solution rebuilds. No claim left standing on partial evidence, fufu~)Merge it with my whole heart~ ♡
Automated review by Jibril · 2026-10-19
CI: green for head SHA
3de48d87· Mutation probes: 2 run, 0 survived (Readable summary-arm mutant → killed by the new summary test; Provider-normalization removal → killed by the generation-id test; the!_answerBeganguard skipped — unchanged since its kill last review) · Local suite: skipped (CI green); AgentStreamingTests 34/34 on a clean rebuild