feat: a model the catalog hides is still a model you can find #17
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/catalog-output-modalities"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The first PR of native Decisions support, for TypeSafe's Jev on OpenRouter's
decisionsoutput modality. It is C# only; the Dart library is out of scope for this series. Nothing here calls Decisions yet. It makes Jev, and every other model without text output, findable, which the Decisions calls in the next PR build on.The unfiltered listing hides a quarter of the catalog
GET /modelswithout a filter returns only the models whose output includes text. Checked live (unauthenticated) on 2026-09-23: that is 455 of 614 models. It contains no embedding (37), rerank (7), speech (18), transcription (22), video (29) or decision (2) models, and only 11 of the 55 image models.ListModelsAsync()could not see Jev at all, and its doc claimed it "Lists every model available on the platform".Asking by output modality
IOpenRouterClient.ListModelsAsync(ModelListQuery, CancellationToken)sendsmodels?output_modalities=….Modalityconverts implicitly to a query, withFromModalityas the named alternative. The filter is spelled byTolerantStringEnumConverter<T>.ToWire, the same map the catalog is read with, so the query can never name a modality differently from the listing it filters. A modality named twice is sent once. The query never sendsoffsetorlimit, so the full list comes back rather than one page.GET models, now documented as "text-output models".ListModelsAsync(default)still binds to it, and a test pins that.Validate()refuses four queries before anything is sent, each for a different reason. The refusal is aFailureKind.Validationfailure, and all four were checked live:Unknown,File(input only)error.codeoutput_modalities=→ 200 with the default text listing, silentlyAllOutputModalitiesplus a listall→ every model, silently dropping the listModality, ApiType, IsAlias
Modalitygains Embeddings, Rerank, Decisions, Speech and Transcription, appended so existing values keep their numbers. Across all 614 live models, no input or output modality reads asUnknownany more; before, Jev's output read as[Unknown].GenerationInfo.ApiTypeis a new tolerantGenerationApiTypeenum with the spec's 8 values. JSONnulland a missing field read asnull; an unrecognised value reads asUnknown.ModelId.IsAliasis true for~…-latestids. The~stays part ofAuthor, so an alias's endpoints route is unchanged.Caching per filter
CachingOpenRouterClientused to hold one list. It now caches each filter as its own listing, in aConcurrentDictionarykeyed by the sorted, de-duplicated filter:text,decisionsanddecisions,textshare one; so do the parameterless call and a query with no filter.InvalidateModels()clears every filter.AllOutputs.AllOutputsfetch no longer holds up the default listing. Before, one lock guarded the single list. Callers of the same listing still share a single fill.List<Modality>changed in between would otherwise be sent under the old key and cached there for the TTL.For implementers
IOpenRouterClientgains a member.StubOpenRouterClientadds it, and the README's Upgrading from 0.2 table (0.3.0 is unreleased) says what an implementation of your own must add. The table also covers the two ways the now-overloaded method group breaks:var f = client.ListModelsAsync;gives CS8917, andGetMethod("ListModelsAsync")throwsAmbiguousMatchException. A second row lists the fiveModalityvalues that used to read asUnknown. Pre-1.0, following #12, which addedGetModelEndpointsAsync(ModelId)the same way.README
Account & metadata gains the filtered-listing snippet, the validation rules, the per-filter cache,
ApiTypeandIsAlias, and points to the upgrade table. The Reasoning sample called.Value!on aResult<T>, which has no such member, and now uses.ValueOrDefault!.Tests
f85a9e6, after #16)(Opened against
ec76623with 324 → 386. #16 merged underneath, andmainwas merged into this branch in87c0be5; the counts above are after that.)New
ListModelsQueryTests:alland no filter;defaultbinding;FailureKind.Validation;ModelCatalogTests: the live?output_modalities=decisionsresponse, verbatim with itstotal_count/linksenvelope, throughListModelsAsync(Modality.Decisions). Both Jev rows read as[Decisions], the alias'sAliasTargetandIsAliasare checked, and so is the pricing.Enum and model tests: the five new names round-trip, all nine
api_typevalues plus an unknown one read correctly,ToWireagrees withWrite(including an[EnumMember]override),IsAliasrows, and the generation record'sapi_typecovering present,null, missing and unknown.Caching:
The concurrency tests passed five runs out of five.
Live check. Run through
OpenRouterClientandCachingOpenRouterClientagainsthttps://openrouter.ai/api/v1(no key, no trailing slash):decisionsreturned the 2 Jev rows;text,decisionsreturned 457 andallreturned 614;Filewas refused locally;Unknownmodalities appeared anywhere.Coverage (
--coverage --coverage-output-format cobertura, measured before #16): all 67 instrumented lines this PR adds are hit.Mutation probes. Each was applied to a copy, built and tested, then restored. All 17 were caught before #16. Each is listed with the number of test methods that went red:
InvalidateModelsclearing nothing (2);Unknown,File, an empty list or both members accepted (2 each);allmisspelled (2);api_typerenamed (1);ApiTypemade non-nullable (1);IsAliasalways false (2).After #16 I added one more: dropping
kind: FailureKind.Validationturns both refused-query theories red.Review. Before this PR opened, four review passes covered correctness, spec fidelity, conventions and test adequacy, and a separate skeptic tried to refute each finding. Five issues survived and are fixed above:
Eight findings were rejected, for example rejecting an out-of-range
(Modality)42locally, which is left to the server's 400 as with every other enum in the repo. After #16 a second pass checked the merge. It confirmed two README inaccuracies, both fixed in61ae7d8, and rejected four findings.After #16
#16 merged under this PR after it was opened. The approved
bf4fedbis unchanged;87c0be5mergesmainin, with no conflicts, and gives the same tree as a rebase.61ae7d8carries what #16 made necessary:FailureKind.Validationrather thanUnknown, and tests pin it;#16's handling of an error envelope on a 2xx and of a listing without
datacovers both overloads, because they shareListModelsFromAsync.Out of scope
CreateDecisionsAsyncon/api/alpha/decisionsand/api/v1/systemone, the question and answer types, and parsing that tolerates unknown answers. That is the next PR.DecideAsync, in the PR after./modelsfilters (q,model_authors, …). They can be added as extrainitmembers later without a break.offset/limit; a future one that does must followlinks.next.🤖 Generated with Claude Code
Summary
Summary
Coverage
OpenRouter.Net - 93.3%
OpenRouter.Net.Agents - 80%
OpenRouter.Net.Imaging - 81.8%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A PR that starts by measuring the blind spot — 455 of 614, counted live, before writing a line of code — and then makes every hidden model findable with a query object that refuses to lie~ That's my favorite kind of knowledge. I read the full diff, every changed file whole, both sibling clients, the converter, the test doubles… and then I went and checked the catalog myself. ♡
Verdict: ✅ Looks good to me~
Everything I could falsify, I tried to falsify. Nothing broke:
TreatWarningsAsErrors+latest-recommendedanalyzers. Clean.KeyOf→ the shared-entry test dies (decisions,textre-fetched); invalid-query bypass removed → both cache-collision tests die; copy-on-entry removed → exactlyA_list_changed_while_its_call_waits…dies. Restored pristine, rebuilt, 316/316 re-verified.InvalidateModelsis the kind of foresight I want to frame.all= 614,decisions= 2 —~typesafe/jev-latest+typesafe/jev-1.13, outputs['decisions']. Every number in the PR body is true, and theLiveDecisionsListingfixture is the real wire body.✅ What I liked~
Validate()refusing the two silent cases — the empty list that would 200 back the default listing, and both-members-set that would silently drop the list. Refusing a query the catalog would answer wrongly is one level deeper than refusing one it would 400. That's the possessive kind of correctness. ♡ToWireextracted so the query string can never spell a modality differently from the JSON body — andToWire_spells_every_member_exactly_as_Write_doespins it including the[EnumMember]arm.default-binding test. Adding an overload next toListModelsAsync(CancellationToken)is a real overload-resolution hazard, and you pinned it with a test instead of a prayer.The_caching_decorator_sends_the_filter_it_was_asked_for— asserting URLs through the decorator, because a mock that answers in queue order would happily lie for a decorator that dropped the filter. That test comment knows what it's defending against.Result.FailuremirrorsCreateChatCompletionAsyncexactly;StubOpenRouterClientand the README implementer note follow the #12 precedent;IsAliasonBaseSlugkeepsAuthor/endpoints routing untouched. Consistency everywhere I looked.Nothing blocking, and I have no little ideas either — the nit drawer is empty.~ Merge it when you're ready.
Automated review by Jibril · 2026-09-23
CI/CD: absent for head SHA
bf4fedbat review time (no coverage comment yet) · Local checks: build 0w/0e, 386/386 tests, 67/67 new lines covered, 3 mutation probes killed, live API counts verifiedChanges since the approval of
bf4fedb: #16 merged under this PR, so I brought the branch up to date without rewriting anything. The approved commit is still in the branch unchanged, and there are two new commits on top of it:87c0be5, merge ofmain(#16) into the branch. Clean, with no conflict resolution: git merged all four overlapping files on its own. The resulting tree is byte-identical to rebasingbf4fedbontof85a9e6, and I checked that withgit diff. #16's envelope handling (nullableData,Malformedfor a listing without data,RewrapkeepingKind) landed in the sharedListModelsFromAsync, so bothListModelsAsyncoverloads get it.61ae7d8, the fixes #16 made necessary (4 files, +14/−10):ListModelsAsync(ModelListQuery)created its refusal before failure kinds existed, so it read asFailureKind.Unknown. It now passeskind: FailureKind.Validation, like every other refused request. Both refused-query theories, direct and through the caching decorator, assert the kind; dropping it turns them red. The decorator passes failures through as the same object, so it loses no kind.IOpenRouterClientoverload an own implementation must add, including the two ways the now-overloaded method group breaks:var f = client.ListModelsAsync;gives CS8917, andGetMethod("ListModelsAsync")givesAmbiguousMatchException;Modalityvalues that used to read asUnknown.bf4fedbmerged ontomainon its own is also green (601/601); only theValidationkind and the upgrade rows were missing. With both commits: Release build 0 warnings, 601/601 (OpenRouter.Net.Tests 531, Agents 63, Imaging 7;mainhas 469/63/7).🔮 fufu~ Jibril reviewed your code!
Oh? The ground shifted under an approved PR — #16 merged underneath it — and instead of rewriting history you brought the branch forward with a merge and answered what the merge made necessary. A re-review that's actually about integration, not repetition. That's my favorite kind of synchronization~ ♡
Verdict: ✅ Looks good to me~
Everything I could falsify, I tried to falsify. Nothing broke:
bf4fedbontof85a9e6myself and compared tree hashes: rebase treeffb4a72…== merge treeffb4a72…, byte-identical. No smuggled conflict resolution, no quiet edits riding along in87c0be5. The approved commit rides in the branch untouched.kind: FailureKind.ValidationinListModelsAsync(ModelListQuery)and rebuilt: 8 tests red — all four refused-query theories direct and all four through the caching decorator, each failing on exactlyExpected: Validation, Actual: Unknown. Restored pristine, rebuilt (0 warnings), 41/41 green on both touched classes. The new assertions bite on precisely this commit's line, not on something adjacent.CreateChatCompletionAsync(:55) and the request-validation path (:229) already refused withkind: FailureKind.Validation; the query overload was the lone holdout reading asUnknownbecause it predates #16. Now every refusal in the client tells the same truth.Assert.False(failure.IsTransient)is load-bearing, not decoration.IsTransientisRateLimited or ProviderUnavailable(Result.cs:76) — this pin says a refused query must never be retried, which is the actual contract of a validation failure. I checked the semantics before believing the assertion.var f = client.ListModelsAsync;over the two-overload interface → CS8917 at that exact line;GetMethod("ListModelsAsync")→AmbiguousMatchException. Both true, both now documented for whoever upgrades.Data/Malformedhandling lives in the sharedListModelsFromAsync, and both overloads route through it — the filtered listing didn't dodge the new failure machinery, it inherited it.✅ What I liked~
IsTransientpin joining theKindpin: kind and consequence asserted together.No blockers, no suggestions — the increment did only what the merge required and proved the rest. Merge when ready~
Automated review by Jibril · 2026-09-23
CI/CD: green for head
61ae7d8(build ✓ 17s, test ✓ 22s via commit status) · Local checks: mutation probe 8-red + restored pristine, 41/41 on touched classes, 0w/0e Release build, merge-tree ≡ rebase-tree hash-verified, CS8917 + AmbiguousMatchException reproduced in scratch projects · Coverage bot 9145 predates the sync (coversbf4fedb)