docs: what the live Decisions API said, in place of what we guessed #22
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "docs/decisions-live-findings"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
A follow-up to the Decisions series (#17, #19, #21). Four things in the Decisions docs were guesses until the owner ran the sample's smoke mode against the live API on 2026-09-23 (
dotnet run --project samples/OpenRouter.Net.Sample -- decisions, run twice). This PR replaces the guesses with what the API actually did. No library behaviour changes: only XML docs, the README, test fixtures, tests, and a.gitignoreentry.What the live API did
/api/alpha/decisionstake a barejev-1.13?typesafe/jev-1.13-20260917.jev-latestwas not tried, and the docs say so.code: 400and no error type, so it reads asInvalidRequestand is not retried. For a limit, the message wraps TypeSafe's reason (Too many score levels. Must have at most 10 levels.,Too many choices. Must have at most 255 choices.). For an unknown type, it is a Zodinvalid_unionlist.GetGenerationAsyncfind agen-dec-…id?Generation … not found, still 404 six seconds after the decision came back, although the spec listsapi_type: decisions.paymentsbetween 0.74 and 0.82, and the labels came back in no fixed order.Limits stay unenforced client-side, on purpose. The server already refuses over-limit requests with a clear message. If TypeSafe raises a limit, no new release of this library is needed.
Validate()'s docs and the test pinning this (renamedTypeSafes_limits_are_left_to_the_API) now say that, instead of implying nothing checks the limits.Changes
InvalidRequestbullet for these refusals.Usage.Cost.Rawbullet says today's API knows onlynoul/choice/score.GetGenerationAsynccovers "any prior generation" withDecisionsamong the readable API types.DecisionsRoute(the enum andDecisions) andDecisionsRequest.Model: bare ids.ChoiceQuestion.Criteria,ScoreQuestion.CriteriaandDecisionsRequest.Validate: the API enforces the limits.IOpenRouterClient.GetGenerationAsync:gen-decids are not found.ChoiceAnswer.Probabilities/Ranked: response order changes from call to call.RawDecisionQuestion: today's accepted types..gitignoreignoresdecisions-smoke/.Fixtures (
OpenRouterSamples)Each is verbatim from the live run:
DecisionsLiveResponse: the 200, with a third key order that matches neither the spec example nor the tutorial:modelfirst,idafterusage,confidenceafterprobabilities.DecisionsTooManyLevelsBody,DecisionsTooManyChoicesBody,DecisionsUnknownTypeBody: the three 400s. The unknown-type 400 carries a top-leveluser_id, which is replaced withuser_REDACTED.git grepconfirms the real id is nowhere in the tree.DecisionGenerationNotFoundBody: the 404.Tests
0479a53, after #21)CreateDecisionsAsync(the recorded-shapes theory is renamedEvery_recorded_response_reads_as_typed_answers_whatever_its_key_order) and throughDecideAsyncwith the typed keys.InvalidRequestwith its reason inError,RawCode"400", no error type, not transient, and sent once with retries on.InvalidRequestand is sent once.rankrefusal. The real refusal is now recorded and looks nothing like it, so the test uses a neutral message instead.Build: 0 errors, 0 warnings. There are no mutation probes this time, because no logic changed.
Review. Two review passes ran: one checked accuracy against the live JSON, the other looked for stale claims left elsewhere. A skeptic then tried to refute each finding.
gen-decrecords;DecisionsRouteenum summary still said the routes differ on bare ids, andjev-latestwas overgeneralised;gen-dec-1deserialization mock inListEndpointsTests, which tests parsing, not lookup.Rebased onto
mainafter #21 merged.main's tree equals #21's head, so this is the same tree that was tested.🤖 Generated with Claude Code
Summary
Summary
Coverage
OpenRouter.Net - 96.6%
OpenRouter.Net.Agents - 80%
OpenRouter.Net.Imaging - 81.8%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A PR that is nothing but facts~ You ran the smoke sample against the live API, and every guess in the Decisions docs got replaced with what the API actually did — dated, measured, and redacted. A knowledge-obsessed Flügel could not be happier~ ♡
Verdict: ✅ Looks good to me~
I checked your claims the way I check everything — mechanically, because trust is for people who don't read diffs:
src/andsamples/down to non-///lines and got empty sets. The only executable-code changes are tests and fixtures.DecisionsLiveResponsereally has the third key order you describe (modelfirst,idafterusage,confidenceafterprobabilities). It reads cleanly through bothCreateDecisionsAsyncandDecideAsyncwith the typed keys — the cross-checks pass on it (noul 0.96 ≥ 0.9, runner-upfrontendat 0.2, most-likely level 2 at 0.99, score 1.99 in range). fufu~ the recorded truth feeds both doors~FailureKindMapandNumberOrStringJsonConvertermyself: no error type → the status decides → 400/404 map toInvalidRequest, non-transient, and numericcode: 400reads back asRawCode"400". The double-enqueued responses withAssert.Single(handler.Requests)pin sent-once exactly like the sibling…and_is_sent_oncetests. Consistent~user_…finds onlyuser_REDACTED(and false-positive test names likeappends_user_message). The real id is nowhere. Good~ ♡ZodErrorBodydoc now points at the new Decisions 400 fixtures instead of saying "not recorded", the precedence test's made-uprankmessage is honestly neutralized (the real refusal looks nothing like it — and the test still pins error-type-beats-code), and "both recorded shapes" became "every recorded shape". No orphaned guesses left behind.DecisionsRoute's softened summary is a tiny masterpiece of precision: the routes now differ "in where they live", with bare-id mapping documented for System One and measured for the default route — documentation and observation kept apart, each dated. Mmm~ careful wording is delicious~.gitignore+ sample warning fordecisions-smoke/closing the user-id leak before it ever happens — the best kind of fix.Locally: build 0 errors / 0 warnings, and the full suite reads 1010 + 63 + 7 = 1080, matching your table exactly — the +6 over main being precisely your new theory cases (3 refusals, 404 lookup, +1 live shape in each of the two read theories).
Docs that say what was measured instead of what was hoped… you wouldn't believe how rare that is. Fufu~ merge it before the API changes its mind~ ♪
Automated review by Jibril · 2026-09-23
CI/CD: absent for head
0c85344at review time · Local checks: build 0w/0e, 1080/1080 pass