feat: a run can ask to stay with one provider, and the catalog says what its cache costs #14
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/session-and-catalog-caching-fields"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The first of the library changes Tsumugi's Phase 2 engine waits on (Tsumugi ADR 0018 §"Prompt caching and stickiness", §"Library work"). Purely additive: no existing member changes shape, and nothing is renamed.
Sticky sessions
A prompt cache lives at one provider, so a follow-up request only hits it when it is routed to the same provider again. OpenRouter takes
session_idas the sticky routing key for exactly that; the request had no way to send one — no field, no header seam, no extension bag.ChatCompletionRequest.SessionId→session_id. Checked against the live API reference on 2026-09-23: a string of at most 256 characters (MaxSessionIdLength), and the body wins over anx-session-idheader. Null omits it (WhenWritingNull); a blank value is a validation error because it would group every blank-id request together rather than none; an over-long one is refused rather than sent. As for every other validation error, the non-streaming path returnsFailurewith no request and the streaming path throwsOpenRouterExceptionbefore sending — both pinned.AgentOptions.SessionIdrides on every round (BuildRequest), read per request likeModel, so a change applies from the next round;Clonecarries it, so a sub-agent joins its parent's session unless the configure callback gives it another id. The clone guard's fixture sets it.GenerationInfo.SessionIdreports the id a generation carried — how a caller confirms afterwards that stickiness took.Catalog fields
Checked against a live
GET /api/v1/models(454 models):ModelPricing.InputCacheRead/InputCacheWrite(input_cache_read,input_cache_write; USD strings like the other prices).ProviderEndpoint.Pricingshares the type, so the endpoints listing reads them too (tested on that path).ModelInfo.DefaultParameters→ newModelDefaultParametersrecord (temperature,top_p,top_k,frequency_penalty,presence_penalty,repetition_penalty, each nullable). Every live entry sends the object, 176 of them as{}, many with all membersnull; 51 sendtemperatureas a JSON integer, which reads as the same double.ModelInfo.CanonicalSlug— the dated version slug an id currently serves (anthropic/claude-opus-5.5→…-20260921); a catalog variant shares its base's slug; for a~…-latestalias it is the alias itself.ModelInfo.AliasTarget→ newModelAliasTarget { Slug, Name }(both required, as the reference requires them): the concrete model behind an alias.top_kstaysint?, as strict ascontext_length: one catalog entry with a fractionaltop_kfails the whole listing, exactly as a fractionalcontext_lengthwould today. A test pins that, so a later tolerance change is a deliberate one.Known and not modelled (the verbatim fixtures prove they are skipped, not fatal):
input_cache_write_1h,input_audio_cache,discount, pricingoverrides(an array of tiered prices),audio_output,image_output;expiration_date,knowledge_cutoff,hugging_face_id,links,benchmarks,supported_voices.README
Prompt caching gains the
SessionIdparagraph and snippet; Account & metadata gains the catalog snippet, the note that-1means "the price varies" (router models — read as unknown, never as a negative price) and the alias/canonical-slug distinction; Auto prompt caching and Sub-agents each gain a sentence. The staleEffort = "high"snippet under Reasoning now readsReasoningEffort.High. The version stays0.2.0— #12 did not bump it either.Tests
af9fd7b)New
ModelCatalogTests(10): every new field present / absent / null /{}/ partial, integer temperature, the fractional-top_kpin, an alias, the endpoints path, and two entries copied verbatim from the live catalog throughListModelsAsync(~anthropic/claude-opus-latestwithalias_target,input_cache_write_1h,links,supported_voices, a nullknowledge_cutoff;openai/gpt-6-lunawithbenchmarksand pricingoverrides). Serialization and omission, the 256/257 and blank boundaries, the wire, both validation paths (the streaming one drained insideThrowsAsync, then no request asserted), the generation record, and four agent tests (every round, changed between turns, none, sub-agent inherits).Coverage (
dotnet test --coverage --coverage-output-format cobertura): every added line is hit. The four uncovered lines inChatCompletionRequest.Validate(the null fallback entry,TopP,MaxTokens) predate this PR.Mutation probes, each restored and touched before the next build — all killed:
BuildRequestwithoutSessionId(3 red);Clonewithout it (2); the clone fixture without it (the guard itself, 1);session_idrenamed (2);>→>=(1); cap 255 / 257 (1 each);IsNullOrWhiteSpace→IsNullOrEmpty(1); the blank check dropped (2); each ofinput_cache_read,input_cache_write,default_parameters,canonical_slug,alias_target,top_krenamed (2–6 each);GenerationInfo'ssession_idrenamed (1);Temperaturemaderequired(2). Two die at compile time rather than in a test:Temperatureasint?, and droppingAliasTarget.Out of scope
parallel_tool_calls(with concurrent tools, deferred);metadata, anx-session-idheader seam or an extension bag;User/Route/Transforms/IncludeonAgentOptions; tolerant integer parsing for catalog fields; every streaming change (tool-call deltas, the usage-only final chunk, typed failure kinds — the next PR); structured-output helpers.🤖 Generated with Claude Code
A prompt cache lives at one provider, so the second request of a conversation only hits it when it is routed to the same provider again. OpenRouter takes a `session_id` as the sticky routing key for exactly that, and the request had no way to send one. `ChatCompletionRequest.SessionId` sends it; `AgentOptions.SessionId` rides on every round, is read per request like `Model`, and is carried by `Clone`, so a sub-agent joins its parent's session unless the configure callback gives it another. The live API caps the id at 256 characters; a blank one would group every blank-id request together, so both are validation errors rather than something sent. `GenerationInfo.SessionId` reports the id a generation carried, which is how a caller confirms stickiness took. The catalog grew fields a cache-aware caller needs. `ModelPricing` reads `input_cache_read` and `input_cache_write` (the endpoints listing shares the type, so it gets them too). `ModelInfo` reads `default_parameters` — present on every live entry and often `{}`, so a new record with every member nullable, an integer `temperature` reading as a double — plus `canonical_slug`, the dated version an id serves, and `alias_target`, the concrete model behind a `~…-latest` alias, whose own canonical slug is the alias itself. `top_k` stays as strict as every other integer catalog field, and a test pins that a fractional value fails the listing, so loosening it is a deliberate change. Two entries copied verbatim from the live catalog run through `ListModelsAsync`, carrying every field this library still does not model (`input_cache_write_1h`, pricing `overrides` as an array, `benchmarks`, `links`, `supported_voices`, …), to prove they are skipped rather than failing the listing. All additions; no existing member changes shape. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>Summary
Summary
Coverage
OpenRouter.Net - 84.4%
OpenRouter.Net.Agents - 80%
OpenRouter.Net.Imaging - 81.8%
🔮 fufu~ Jibril reviewed your code!
Oh? Sticky sessions so the cache stays warm, AND the catalog finally tells you what that cache costs? A prompt cache lives at one provider and a follow-up request only hits it when it lands on that same provider again — this is exactly the seam that was missing. I read every changed line, every unchanged neighbor, and both verbatim live fixtures~ ♡
Verdict: ✅ Looks good to me~
✅ What I liked~
Every_option_survives_the_clone_a_sub_agent_inheritsasserts the whole property surface by reflection, and the "guard the guard" check (fixture must set every option, no nulls allowed) means the next forgottenClone()line fails loudly before a sub-agent silently loses it.SessionId = "run-1"sits in the fixture — the exact bug this PR's design makes possible is the one the test makes impossible. Fufu~, that's craftsmanship~WhenWritingNull), blank is refused with the reason in the doc comment (a blank id would group every blank-id request together rather than none — subtle and correct), 256 accepted / 257 refused with both boundaries pinned. TheMaxSessionIdLengthdoc even says "UTF-16 code units" instead of hand-waving "characters." ♪FailurewithStatusCode == nullANDhandler.Requestsempty; streaming →OpenRouterExceptionthrown inside the enumeration (with the lazy-iterator drain comment — correct, the body runs on firstMoveNextAsync) AND no request sent. Many PRs test one arm and hope for the other~~anthropic/claude-opus-latestcarryingalias_target,input_cache_write_1h,links,supported_voices, nullknowledge_cutoff;gpt-6-lunacarryingbenchmarksand pricingoverridesas an array — every unmodeled field must be skipped, not fatal, and now that's law. The fractional-top_kpin matchingcontext_length's strictness is a deliberate decision written down as a test, which is where decisions belong.top_kstayingint?— consistency over convenience, and the pin makes loosening it a visible act rather than silent drift.Effort = ReasoningEffort.Highfix for the stale snippet, and the-1-means-varies warning next to the parse advice — small, kind.Verified by me, not just read: CI green for head
1fd0dce(build + test, run #32); locally the full suite is 301/301 (231 + 63 + 7 — your table's arithmetic is exact). And because green CI only proves tests run, I mutation-probed the new pins at head: droppingSessionIdfromBuildRequest→ exactly the 3 agent session tests red; renaming the wire key tosessionId→ exactly the wire + serialization tests red. Both restored, tree pristine, rebuilt, 301/301 again~💡 Little ideas (non-blocking)~
A_session_id_one_past_the_maximum_is_refusedhardcodes257in the constructor while its sibling usesMaxSessionIdLength. Keeping 256/257 in the message asserts (pinning the wire constant) is right; the input could beMaxSessionIdLength + 1so a future cap change moves the whole test in one place. Pure taste, ignore freely~Fufu~ purely additive, every branch owned by a test that provably bites, sibling patterns honored (
ProviderEndpoint.Pricingshares the type, validation error shapes match both paths,Clonemirrors its neighbors). Tsumugi's Phase 2 engine may proceed~ ♡Automated review by Jibril · 2026-09-23
CI/CD: passed for head SHA (run #32, build + test) · Local checks: full suite 301/301, mutation probes 2/2 killed, tree restored pristine