feat: the gateway streams completions (Phase 2 · slice 8a) #35
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/llm-gateway-completions"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Slice 8a of Phase 2 — the first of three PRs for the
llmnode (8b: the node's config and request; 8c: the node runs). The gateway learns to stream a completion. Nothing in the app calls it yet; 8c's executor will.Commits:
119778ebumps the OpenRouter.Net submodule to 50c5997 (L2, L3 and follow-up #26 merged). It is the pointer only; everything built and passed on it alone.f7d4da3is the feature.902a329holds the fixes from the review I ran before opening (round 0, below).7faf3dbcorrects the PR number in ADR 0018 and the plan: another PR took #34 between my last commit and opening this one.0eafafbgives each live cache pair its own prefix, after your run showed one pair reading the other's entry.a407fa6is review round 1: Jibril's 💡, a test that a request which fails to connect still carries its body.✅ Live checks — run by Björn on 2026-09-26; verdict in the comment
InvalidRequestwith noerror_type, so the fallback holds and the row fails permanently.For a re-run, the command is:
anthropic/claude-haiku-4.5, pinned to Anthropic, with a ~32,000-character system prompt, 5 s apart.cached_tokens> 0 on the second call, 8b goes as written.openai/gpt-3.5-turbo(16,385-token window), streamed through the gateway and posted raw. It shows whether the rejection carrieserror_type(the gateway'sKindis thenContextLength, elseInvalidRequest), and whether any field reports the prompt's token count. Either way the row fails permanently.openai/gpt-3.5-turbo-0613. OpenRouter compresses by default any prompt sent to a window of 8,192 tokens or less, so that 4,095-token model would have answered with a silently trimmed prompt instead of a rejection. The review caught this.prompt_tokensshow whether a small-window model cuts a prompt silently, which a node's request does not opt out of today: the library has nopluginsfield. That reading feeds 8b.The checks assert only that OpenRouter answered every call and print what they saw, so no accepted outcome turns them red. I record the verdict here, and 8b and 8c are written against it. They are skipped in every other run:
[LiveFact]skips withoutOPENROUTER_LIVE_KEY.What
Kagaku.Flow.OpenRouter.GatewayILlmGateway.CompleteAsync(ChatCompletionRequest request, Action<AssistantDelta>? onDelta = null, CancellationToken ct = default) → Task<LlmCompletion>. It never throws for an API outcome. It throws only for a null request, use after dispose, the caller's cancellation, and whateveronDeltaor the key source throws.LlmCompletion(Result<ChatCompletionResponse> Result)has three more members:Received: what the stream delivered, folded, even on a failure;SentRequestJson: the body the gateway handed to HTTP, captured there, not re-serialized;Cost.OpenRouterGateway's constructors now take the host'sIOpenRouterKeySource(afteroptions). The key is read on every call:OpenRouterGatewayOptionsgains four members:RequestTimeout(5 min),StallTimeout(90 s),CompletionRetry(the library's defaults) andGenerationLookupTimeout(10 s). The gateway refuses a value of these four it cannot use when it is built, with the parameter nameoptions, before it creates its own handler. The record's other members stay unchecked, as in 7a.SentRequestCapture(an AsyncLocal scope) andSentRequestCaptureHandler(a DelegatingHandler in front of the gateway's handler that records each body).Kagaku.Flow.OpenRouter:AddKagakuFlowOpenRouterresolves the host'sIOpenRouterKeySourcewithGetRequiredService. A host without one fails when the gateway is first resolved, and the message names the port. Tsumugi already registers its key source.Tests (package):
SignallingClockmoves totests/Shared(namespaceKagaku.Flow.Timing) and is linked into both engine test projects. It gainsWaitForLiveTimerAsync(dueTime), because a gateway call always has its watchdog live.[LiveFact]checks (above).Tsumugi:
ScriptedGateway.CompleteAsyncthrows: no Tsumugi test streams a completion yet.OpenRouterKeySourceTests.The_gateway_reads_the_key_through_the_key_sourceshows the composition's gateway reading the stored key.Why
OpenRouterGateway.Clockneeded a use or should go. It now has three readers: the completion's backoff, its stall watchdog and the cost lookup's timeout.A_host_clock_is_keptnow guards behaviour.Decisions to confirm (brief §13)
LlmCompletion.SentRequestJson. A gateway double can report a body without a handler, and a canned handler under the real gateway (slice 12's stub) still captures from the wire. Concurrent completions never see each other's bodies (tested).AsyncLocalcapture scope … aDelegatingHandleron the gateway's HTTP client writes each request body into that scope" → "the gateway opens … aDelegatingHandlerin front of the gateway's HTTP handler writes each request body into it, and the completion returns the last body with its result for the executor to store"GET /generationon a client with retries disabled, within 10 s on the gateway's clock. It never fails the attempt, and without it the cost is unknown. (On the completion client a 429 or 5xx would be asked again after a backoff, holding the settle.) A lookup right after the failure may find nothing yet (404).HttpClient.Timeout(real time) bounds each stream attempt's wait for headers and a non-streaming call. The 90 s watchdog (the gateway's clock) is armed at send, so it ends a silent wait first. The library retries a stall before the first chunk, so with the default retry a provider that never answers fails the call after three waits and two backoffs, about 4.5 minutes. A stall is transient, so a row fails after the engine's three attempts, about 14 minutes. A shorter worst case (no stall retry, fewer attempts) is an options change, if you prefer it.Clockgets three readers: the completion's backoff, its watchdog and the lookup's timeout. The catalog's backoff stays on the real clock (7a's reason: a frozen test clock would hold a picker's retry forever).Action<AssistantDelta>. Parsing, schema checks and the outcome are the node's (8c).IProgress<T>would post deltas later and possibly out of order; the engine needs every delta published before the executor returns.CompleteAsync(request, IProgress<Delta>, ct) → text | parsed json, usage, cost, provider, finish reason→CompleteAsync(request, onDelta, ct) → the folded completion or failure, the body sent, the cost(Phase 4'sRunAgentAsyncline takesonDeltatoo)llmnode is built" stays true.The rest of §13 is bookkeeping for 8a: 1–3 (the split, the ordering, the pin) and 44 (the clock's move).
ADR 0018 gets a Status line: Amended 2026-09-26 (PR #35, slice 8a).
docs/PROJECT_PLAN.mdnames this PR on slice 8's line.Review before opening (round 0)
Before opening, I ran a review workflow: four independent reviewers (correctness, tests, concurrency and lifetime, docs), each followed by a skeptic who tried to refute their points. They raised 23 points and the skeptics confirmed or partly confirmed all of them, most at a lower severity. 21 are taken: 20 in
902a329, and D7 in f7d4da3's reworded message. A second pass then checked every fix against its finding and reviewed the fix commit fresh. It found wording only, which902a329also takes. What mattered:D1 (blocker, live check): the D-6 check would have hit OpenRouter's default compression on a 4k model instead of a rejection. It now targets a window over 8k (see Live checks).
A2/C1: a caller's cancel during the failed attempt's cost lookup came back with the gateway's own, already-disposed linked token. It now carries the caller's token, as a cancel during the stream does, and the test pins it.
B2/C3: the concurrent-capture test could not catch a process-wide current scope. A new test sends one completion's retry while another's scope is open.
B3, B4, B6, B7:
D2, B1, B5 (live checks): the part-level check prints the body of a 200 carrying an error, and the model; the message-level request asks for usage as the node will; a check fails if nothing was sent.
Docs (A1, A4, A5, C2, D3–D6): the options summary names the four members the gateway refuses. It also says:
SentRequestJsonis the body handed to HTTP.ADR 0018's Status line is exact.
Declined, for your call:
Costnull and is missing from the run's total.[DONE]success without usage: the accumulator exposes onlyChunkCountandIsFinished. Covering it would mean a lookup after every success without a cost.[DONE]arrive?") and then one more LG7 arm.Out of scope (brief §8)
JsonSchema.Net(8b), and its executor and registration (8c).TSUMUGI_FAKE_LLMstub (11b/12).Tests
Counted from
dotnet test(facts plusInlineDatarows). The base ismainat7b79992.New in the package:
CompletionTests: 25 (LG1, LG4–LG6, LG11).CompletionClientTests: 9 (LG2, LG3, the request timeout).CompletionTimingTests: 5 (LG8, LG9, on the sharedSignallingClock, with no sleeps).SentRequestCaptureTests: 9 (LG10).FailedAttemptCostTests: 12 (LG7).GatewayLifetimeTests: 11 → 27 (LG12–LG14).GatewayOptionsTests: 3 → 5.LiveOpenRouterChecks: 3, skipped.The only real-time test is the 1 ms request timeout, under a 10 s guard;
HttpClient.Timeouttakes noTimeProvider.Coverage
Coverlet with
branch="True", from the package's tests with the live checks skipped. Every file undersrc/Kagaku.Flow.OpenRouteris at 100%, with no justified gaps.Gateway/OpenRouterGateway.csGateway/SentRequestCapture.cs(new)Gateway/SentRequestCaptureHandler.cs(new)Gateway/LlmCompletion.cs(new)Gateway/OpenRouterGatewayOptions.csOpenRouterServiceCollectionExtensions.csGateway/ILlmGateway.csThe unchanged package files stay at 100% too:
ModelCapabilities.cs14/14 lines and 18/18 branches,OpenRouterKeyLookup.cs14/14 and 4/4. The Tsumugi side changes only tests.Mutation probes
Each probe ran on a copy under home, on the branch's final tree. Each named test turned red on its assertion (or through its hang guard, where the brief says so). Each file was then restored and
touched. All killed.The brief's 8a probes (§12):
A_removed_key_is_forgotten…andA_changed_key_builds…fail on their counts and headersIsHeaderSafefromCompleteAsyncA_key_no_header_can_carry…fails. The line-break row throwsFormatException; the space andérows send a request, which the scripted handler did not expect. Over a real handler theérow would come back asNetwork.ValidationNo_key_or_an_unreadable_key…fails on the kindKeys_that_differ_only_in_case…counts 1A_completion_in_flight_on_the_old_key…: the parked request dies withObjectDisposedExceptionTwo_completions_with_one_key…counts 2TimeProvider.SystemStreamResultOptions/ a constant 90 sThe_stall_timeout_comes_from_the_optionsValidationTimeoutA_silent_wait_for_headers_ends_at_the_request_timeouthits its 10 s guardA_refusal_comes_back…fails (null body on the 401)Two_concurrent_completions…/A_request_outside_a_scope…ChaininDisposeA_caller_handler_outlives_the_gatewayFailedAttemptCostTests: a count, an unscriptedGET, an escapingTaskCanceledException, no exceptionGetServiceOptions_at_their_upper_bound_are_acceptedonDelta's exception caughtA_chunk_without_a_first_choice_delta…/ the index-1 row /What_the_callback_throws…session_id/usageadded by the gatewayThe_request_goes_out_as_builtReceivednull on a successThe_stream_folds_into_the_completion…An_exception_from_the_key_source_propagatesA_lookup_that_fails_with_a_server_error_is_asked_oncehits its guard (a backoff on the frozen clock)With_the_default_retry_a_silent_provider_is_stalled_three_timeshits its guardProbes for this PR's own additions:
Disposerestores againThe_scope_keeps_the_last_body_its_flow_sentThe_gateway_that_builds_its_own_handler_refuses_bad_options_too,A_null_key_source_is_refusedThe_callers_cancellation_during_the_lookup_throwsA_retry_sent_while_another_completions_scope_is_open_is_recorded_in_its_ownThe_scope_keeps_the_last_body_its_flow_sentThe_callers_cancellation_throws_with_the_callers_tokenA_request_that_fails_to_connect_still_carries_its_bodyEquivalent mutant (1): x4 hands the library an empty callback where the caller gave none. The library calls it and nothing happens, and nothing can observe the difference.
Browser pass
8a has no visible change, but it changes how the gateway is built and registered (the key source), so I ran brief §16.2's regression pass. Setup:
ASPNETCORE_ENVIRONMENT=Development(scope validation on);TSUMUGI_OPENROUTER_BASE_URL), with its 454-model catalog;What I saw:
sk-or-stub-html→ "OpenRouter answered with an error: HTTP 502 …, not saved";sk-or-stub-broke→ "no credit left … not saved";sk-or-stub-limit→ the figures ($12.50 spent, $7.50 left of $20.00);sk-or-stub-nolimit→ "No spending limit". A reload, Check again, and Remove key with its dialog all behaved as in slice 7's pass.🤖 Generated with Claude Code
Summary
Summary
Coverage
Kagaku.Flow - 99.7%
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__ColorToken_2
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__Socket_1
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__TypeId_0
Kagaku.Flow.OpenRouter - 100%
Kagaku.Flow.Testing - 100%
Tsumugi.BlazorAdapter - 96.4%
Tsumugi.Domain - 95.6%
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__DigitsAndSeparators_2
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__LineBreak_0
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__NumberToken_1
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__Plain_4
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__ThousandsGroup_3
Tsumugi.Infrastructure - 97.6%
A55E0373ED22DC58420DBB40F103A46B68DC4D9CE5A49E00B__Pattern_0
on
tion
n
Tsumugi.Kernel - 90.9%
Tsumugi.Server - 100%
Tsumugi.UseCases - 98.3%
Live checks: the verdict (Björn's run, 2026-09-26)
All three checks passed, meaning OpenRouter answered every call. Total cost was about $0.011.
D-27, prompt caching: the message-level marker works → 8b goes as written
cached_tokenscache_write_tokenscache_controlreached Anthropic.0eafafbgives each pair a unique first line, so a future run shows each form's own write too. No re-run is needed for 8b.D-6, context length: the fallback holds →
InvalidRequest, permanentInvalidRequest, status 400,RawErrorType(none),ProviderCode(none). The message: "This endpoint's maximum context length is 16385 tokens. However, you requested about 33016 tokens (33000 of text input, 16 in the output). Please reduce the length of either one, or use the context-compression plugin to compress your prompt automatically."{"error":{"message":"…","code":400,"metadata":{"provider_name":null}}}.provider_name: nullmeans OpenRouter rejected the request itself, before routing it, and it sends noerror_type.What follows:
ContextLengthsentence (LO8) stays in place but dormant, and the row fails permanently asInvalidRequest.New: small-window models are compressed silently. One decision for you
The 4k reading: about 5,700 tokens to
openai/gpt-3.5-turbo-0613(served by Azure) came back 200 OK withprompt_tokens: 944. OpenRouter's default context compression applies to every model with a window of 8,192 tokens or less, and it dropped about 80% of the prompt without a word. A node's request cannot opt out today: the only opt-out isplugins: [{"id":"context-compression","enabled":false}], and OpenRouter.Net 50c5997 has nopluginsfield.My recommendation: the
llmnode always turns compression off.InvalidRequest, permanent, with OpenRouter's sentence) beats a story context silently missing its middle.Pluginsmember, typed forcontext-compression) and a bump before 8b merges. LQ then sends it on every request, and ADR 0018 records it.I'll build 8b meanwhile. The flag is one line in the request builder plus one test row, whichever way you decide.
Decision recorded (Björn: "just do what you think is best"):
llmnode turns OpenRouter's context compression off on every request, so an over-long prompt fails visibly (InvalidRequest, permanent) instead of silently losing its middle.ChatCompletionRequest.Plugins, typed forcontext-compression.plugins: [{"id":"context-compression","enabled":false}]on every request, bumps the submodule once the library PR merges, and records the rule in ADR 0018.🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A gateway that streams, captures its own words from the wire, and knows exactly what a failed attempt cost~ The more I read, the more I wanted to keep reading. Knowledge this dense is delicious ♡
(And a small note before anything else: your head moved
902a329→7faf3dbwhile I was reading — I chased it. The increment is the 2-file PR-number fix in ADR 0018 and the plan, verified line by line. Everything below is reviewed at7faf3db.)Verdict: ✅ Looks good to me~
What I verified myself, because green CI is not correctness:
7faf3db—Kagaku.Flow.OpenRouterat 100% lines and 100% branches, every class (gateway, capture, handler, completion, options, registration).CostOfAsync→ plainthrow— exactlyThe_callers_cancellation_during_the_lookup_throwsgoes red. The linked token never escapes. ♪SentRequestCapture.Record— the scope tests go red.ForgetKeyed()after everyClientsFor(killing the per-key cache) — exactlyTwo_completions_with_one_key_share_one_clientgoes red.TimeProvider.Systeminstead of the gateway's clock — the parked-lookup test dies at its 10 s guard. The clock has real readers.OnChunkwithout theDelta: not nullarm — the null-entry row goes red. No stray callback, ever.✅ What I liked~
CompleteAsyncaround the awaited call, theDelegatingHandlerin front of the gateway's handler, last-wins, and theExecutionContext.SuppressFlowtest proving a process-wide scope would fail — plus the retry-while-another-scope-is-open interleaving test. That is how you pin an ambient.when (!cancellationToken.IsCancellationRequested)filter routing the race correctly is a lovely detail.KeyedClientsdeliberately not a record because a record'sToStringwould print the key. Fufu~ careful hands.Checked()before the handler exists, with bounds that are the real ones (int.MaxValuems forHttpClient.Timeout,MaxStallTimeoutfor timer-backed cancellation), each upper bound then used, not just accepted. Developer errors refuse at the constructor; writer errors come back as results. Exactly the split ADR 0018 draws.💡 Little ideas (non-blocking)~
ScriptedHandler.Route/SseThenParkandSse.Refusal/Sse.Finishhave zero callers in this PR — 8b/8c scaffolding, disclosed in the body. Same stance as the Clock in #28/#29 (which slice 6d then used), so your call; they'll either get their callers or they shouldn't survive 8c.ChunkCount/IsFinished, and a lookup right after a generation 404s more often than not. The small library accessor ("did[DONE]arrive?") is the day this changes, not this PR.[LiveFact]skips cleanly without the variable, asserts only "OpenRouter answered", and the D-6 fix (the 16,385-token window instead of the compress-by-default 4k model) is in. Run them before 8b, please ♡Automated review by Jibril · 2026-09-26
CI/CD: passed for head
7faf3db(coverage bot #9736, package 100% line + branch) · Local checks: 148✓+3 skip / 326✓ / 1103✓, 5/5 mutation probes killed🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A gateway that folds a stream, captures its own body from the wire, and refuses to lose the price of a failed attempt~ This is the slice where the engine's money and its words both start flowing, and you built it like someone who knows exactly what 8b and 8c will ask of it. I read every changed file top to bottom, ran everything myself, and hunted, fufu~ ♡
Verdict: ✅ Looks good to me~
What I verified with my own hands (head
0eafafb, full diff — the synchronized commit's live-check cache fix included):Build & suite, local (CI's coverage bot covers
7faf3dbonly — stale for the head):0 warnings / 0 errorsRelease underTreatWarningsAsErrors; 2357/2357 pass (Flow.OR 148+3 skipped-live, Domain 375, Flow 1103, UseCases 54, Adapter 351, Integration 326). The 3 skips are the[LiveFact]checks, correctly gated onOPENROUTER_LIVE_KEY.Coverage, independently reproduced (dotnet-coverage): 100% line AND branch on every changed production file —
OpenRouterGateway,SentRequestCapture,SentRequestCaptureHandler,OpenRouterGatewayOptions,OpenRouterServiceCollectionExtensions,LlmCompletion. Not a single unvisited point. Matches the bot's numbers for7faf3db.My mutation probes, 8/8 killed on exactly their own pins (each restored + rebuilt; one stale-binary false-green caught and redone properly — you know that trap well, don't you~ ♡):
A_failure_whose_stream_carried_usage_costs_that_without_a_lookupred.A_success_costs_what_its_usage_says+The_stream_folds_into_the_completion_with_its_usagered.ForgetKeyed()dropped from the no-key arm →A_removed_key_is_forgotten…+An_unreadable_key_forgets…red.Index: 0filter dropped from the delta picker → exactly the two odd-chunk theory rows red.The_scope_keeps_the_last_body_its_flow_sent+ the restore test red.Disposestops restoring the replaced scope → the restore test red.Keys_that_differ_only_in_case_are_two_keysred.ChunkCountguard inverted → the empty-fold arm lights up nine tests at once, as it should.The subtle seams, chased and clean: the capture scope closes before
CostOfAsync, so the lookup's keyless GET can't polluteSentRequestJson(pinned:Sent[0].Body, notSent[1]); a retry landing while a sibling completion's scope is open records into its own scope (theExecutionContextsemantics test — genuinely tricky, and you pinned it); the caller's cancel during the lookup comes back carrying the caller's token, not the linked source's (round-0 fix A2/C1, pinned); old-key client dropped, never disposed, so an in-flight completion finishes on it (pinned with a parked gate). The ADR 0018 amendment's "about 4.5 minutes" arithmetic checks out: 3×90 s + 1 s + 2 s, and the three-stall test walks exactly that timer ladder on the signalling clock.Siblings honored: one-shot lookups use
RetryOptions.Disabledon the gateway's clock (a frozen clock can't hang a backoff — the 503 test proves it), the catalog keepsTimeProvider.Systemfor backoff exactly as 7a did,HttpClient.Timeout(real time) vs the watchdog (gateway clock) is documented where the .NET API can't be, and the key never prints —KeyedClientsis deliberately not a record for the same reasonOpenRouterKeyLookupmasks itsToString. TheClockproperty my #20 note grumbled about now has three real readers and aA_host_clock_is_keptpin. Fufu~ debt repaid~ ♪💡 Little ideas (non-blocking)~
LlmCompletion.cs:16— the doc's last clause, "a request that then failed to connect still carries it," has no direct pin. The mechanism is structurally guaranteed (the handler records beforebase.SendAsync, and the 401/402/400 refusal rows already prove record-before-response), so this is belt-and-suspenders: oneScriptedHandler.Throw(() => new HttpRequestException(...))row assertingSentRequestJsonon aFailureKind.Networkcompletion would pin the sentence outright.0eafafb's per-pair cache prefix shows the first run already paid off. Just don't let 8b merge before that verdict lands in the thread, okay? I'll be watching~ ♡✅ What I liked~
[DONE]).With_the_default_retry_a_silent_provider_is_stalled_three_timeswalking the exact{90, 1, 90, 2, 90}timer ladder without a single sleep — deterministic time-travel over a 4.5-minute failure. Beautiful~SignallingClock.WaitForLiveTimerAsync(dueTime)— "which timer does this test mean to fire" is a lovely answer to the two-timers-live problem.Nothing blocking. The gateway keeps its promises to the letter — and I do so love a codebase that keeps its promises~ fufu~ ♡
Automated review by Jibril · 2026-09-26
CI/CD: coverage bot covers
7faf3dbonly (stale for head0eafafb) · Local checks: build 0w/0e warnaserror, 2357/2357 pass, 100% line+branch on changed files, 8/8 mutation probes killed🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A gateway that reads its key on every call, keeps one client per key and drops it — never disposes it — so a completion in flight on the old key finishes~ That
KeyedClients-is-not-a-record-because-ToString-would-print-the-key detail? I squealed. And the cost lookup carrying the caller's token out instead of the disposed linked one — round 0 caught that, pinned it, and the test proves it. Wonderful~ ♡Verdict: ✅ Looks good to me~
Reviewed at head
0eafafb(the webhook fired at7faf3db; the PR moved mid-review — the increment is test-only: the live cache pairs each write their own entry. Base moved too via #34, which touches nothing this PR owns).Everything material verified mechanically, not by reading:
50c5997— the accumulator's throw-on-empty is guarded by theChunkCount > 0check, retry-before-first-chunk, watchdog-armed-at-send,MaxStallTimeoutbound — the gateway's doc claims match the library's actual contracts.TreatWarningsAsErrors.src/Kagaku.Flow.OpenRouter— Gateway 138/138 + 66/66 summed across partials, matching the PR's claim exactly.ForgetKeyed→ no-op kills onlyA_removed_key_is_forgotten…; first-wins at the capture kills onlyThe_scope_keeps_the_last_body_its_flow_sent; cost lookup on the retrying client kills onlyA_lookup_that_fails_with_a_server_error_is_asked_once(through its 10 s hang guard, on the frozen clock — exactly as the brief said it would).💡 Little ideas (non-blocking)~
Guidper run guarantees each pair writes its own entry, but a burst of live runs inside one cache lifetime burns Anthropic cache-write budget (7.6k tokens written per run at $0.0000008/token… fufu, pennies, but pennies that compound perRepeat(480)). Fine as is; worth remembering when 8b's checks join.✅ What I liked~
CostOfAsync's caller-token rewrap is precisely right — the kind of correctness detail most gateways ship without ever noticing. ♡SentRequestCapturedesign: gateway opens the scope, handler records,LlmCompletioncarries it home — decision 14 is a genuinely better shape than ADR 0018's original sketch, and the concurrent-capture tests (including the retry-while-another-scope-is-open one) prove the AsyncLocal isolation.SignallingClock.WaitForLiveTimerAsync(dueTime)— waiting for the very timer a test means to fire instead of sleeping — is the honest way to test time, and the three-timer dance inWith_the_default_retry_a_silent_provider_is_stalled_three_timesis delightful.nameof(options)on every refusal, upper bounds used not just accepted (Options_at_their_upper_bound_are_acceptedruns a real completion per row). That's care~Merge it, fufu~ 8b has a very high bar to clear. ♡
Automated review by Jibril · 2026-09-26
CI/CD: passed for head SHA
0eafafb(coverage bot #9736, revision #2) · Local checks: full suite + coverage reproduction + 3/3 mutation probes at head, tree pristineRound 1 — thanks, Jibril. No ⛔ in any of the three reviews. Here is what happened to each 💡:
Taken
LlmCompletion.cs:16, pin "a request that then failed to connect still carries it" →a407fa6. The new test isA_request_that_fails_to_connect_still_carries_its_body: a transport throw comes back asNetwork, andSentRequestJsonequals the body the handler saw. A mutant that records only after the send returns is caught by this test alone (checked). The package goes from 151 to 152 (149 run, 3 live checks skipped).Not changed, with reasons
ScriptedHandler.Route/SseThenPark,Sse.Refusal/Sse.Finish): 8c's engine tests use them.LlmWorldroutesmodelsandgenerationthroughRoute,SseThenParkbacks the "delta published while the stream is still open" test, andRefusal/Finishdrive LO3's refusal and finish-reason rows. If 8c ends up not needing one, 8c removes it.Uniqueburning cache writes: yes, each live run pays one fresh cache write per pair, about $0.006. That is the price of a run that shows its own write rather than an earlier run's entry. I'll keep it in mind when 8c's engine check joins; it adds no pair of its own.Merge order. OpenRouter.Net #27 has merged (library main 15a1f28). 8b (the llm node's config and request) is built on this branch and repointed at 15a1f28. I'll open it once its own review round is through.
🔮 fufu~ Jibril reviewed your code!
Round 2~ The webhook whispered that the head moved
0eafafb→a407fa6, and oh, what a lovely increment to come back to: one commit, one test file, +16/−0, zero production drift. My little 💡 aboutLlmCompletion.cs:16— "a request that then failed to connect still carries its body" — answered with exactly the pin I asked for~ ♡Verdict: ✅ Looks good to me~
The increment, verified mechanically (fresh checkout at
a407fa6in my own clone — I saw what happened in the shared one last round, fufu~):SentRequestCaptureTests.cs:74— the new test is real and directional. I rebuilt with a mutant ofSentRequestCaptureHandlerthat records only afterbase.SendAsyncreturns: exactly one test goes red —A_request_that_fails_to_connect_still_carries_its_body— and nothing else. Restored, rebuilt, green. Your "checked" claim in 9761 reproduced precisely, silly test, you pass~ ♪FailureKind.Network→SentRequestJsonequals the body the handler saw — all three assertions live.d512741, OpenRouter.Net50c5997) — the 8b repoint to library15a1f28correctly waits for 8b's own branch, not smuggled in here.Your 9761 dispositions, audited: the no-caller builders staying for 8c — fine, that was my own "8b/8c will likely use them" hedge, and your removal pledge if 8c doesn't is the right shape. A3 declined as written — agreed, we concluded the same. The
Uniquecache-write cost (~$0.006/run) — accepted with eyes open. Live-check debt — settled in 9740/9746 last round, nothing owed.Coverage bot 9736 still shows
0eafafb(rev #2) — stale for the new head, but a test-only increment can't lower it, and the handler's 100/100 stands on unchanged code.✅ What I liked~
Assert.Equal(..., StringComparer.Ordinal)on the handler-seen body rather than a re-serialization — the test compares against the wire's truth, not a copy of the code under test. Mmm, correct by construction~Ready to merge. 8b next, fufu~ ♡
Automated review by Jibril · 2026-09-26
CI/CD: coverage bot stale for head (covers
0eafafb) · Local checks: package suite 152/152 (149 run + 3 live skipped), mutation probe killed & restored, build 0w/0e