feat: answers come back under the key that asked for them #21

Merged

bjoern merged 1 commit from feat/decisions-typed-keys into main 2026-09-23 17:34:23 +02:00

Conversation 2 Commits 1 Files changed 17 +1250 -78

scarlet commented 2026-09-23 17:10:59 +02:00

Member

Copy link

The third and last PR of the Decisions series, after #17 (find) and #19 (call). This one reads a decision without writing an id twice or casting, and refuses one that does not answer what was asked. It adds one new FailureKind, Unanswered, as the owner decided. Enum-backed choices and the Agents tool-call gate remain separate, later work.

Typed keys

var questions = new DecisionQuestionSet();
var isBug   = questions.Noul("is_bug", "Is the customer reporting a software defect?");
var team    = questions.Choice("team", "Which team should own this ticket?", "payments", "frontend", "account");
var urgency = questions.Score("urgency", "How urgent is this ticket?", "later", "this week", "now");

var result = await client.DecideAsync(new DecisionsRequest { Model = "typesafe/jev-1.13", State = ticket, Questions = questions }, ct);
if (result is Result<DecisionsResponse>.Success { Value: var d })
{
    NoulAnswer bug     = d.Get(isBug);     // no id written twice, no cast
    ChoiceAnswer owner = d.Get(team);
    ScoreAnswer level  = d.Get(urgency);
}

• DecisionQuestionSet is an IReadOnlyDictionary<string, DecisionQuestion> over an OrderedDictionary. It goes out as the same plain map, in the order added, pinned against the dictionary form.
  • Noul/Choice/Score return a DecisionKey<TAnswer>, a sealed record: blank ids are refused, and two keys are equal by id and answer type.
  • Add(id, question) takes a raw or prebuilt question, and a collection initializer works.
  • A duplicate id throws.
  • CA1710 wanted a …Dictionary/…Collection suffix. It is suppressed with a justification, as Quantization.Int8's CA1720 is, keeping the name the owner approved.
• Reading answers. Get(key) throws KeyNotFoundException or InvalidOperationException, naming what it found instead. TryGet(key, out …) and TryGet<T>(id, out …) are there too; the second is for a question asked without a key, such as a raw one.

DecideAsync and ValidateAgainst

DecideAsync(request), and DecideAsync(request, route, ct) with a required token as on the raw call, send the request and then run response.ValidateAgainst(request.Questions). The check requires:

• every asked question to have an answer, with a null answer counting as none;
• the right type. An UnknownDecisionAnswer never answers a noul, choice or score question. A raw question is answered by an answer of its own type that the library could read, and its probabilities and confidence are checked; its labels and levels are not modelled, so they are not;
• a choice to name one of its labels, and to give probabilities only for its labels;
• a score to lie between 0 and the top level, inclusive at both ends, and to give probabilities only for its levels;
• every probability and confidence to lie from 0 to 1, with NaN counting as out of range.

Answers to questions that were not asked are ignored. The probabilities need not sum to 1 (they are rounded) nor cover every label.

A response that fails the check is a FailureKind.Unanswered failure:

• One failure lists every problem, e.g. Decision gen-dec-7 does not answer what was asked: 'team' chose 'sales', which is not one of its labels; 'urgency' has no answer.
• IsTransient is false, with no status and no Details. It was billed, and the same request is likely to be answered the same way.
• It is sent once even with retries on, because the check runs after the retry loop.
• A failure from the call itself comes back as the same instance, kind and all. Assert.Same pins this for Network, RateLimited, Validation and Malformed.

It fails closed. A Result.Failure has nowhere to carry the paid response, so the README documents the raw CreateDecisionsAsync plus ValidateAgainst as the way to keep it.

FailureKind.Unanswered is appended after Validation, and IsTransient is unchanged: it lists only transient kinds. The guard test that fails whenever a kind is added without a row in the transience theory caught it at once, and now covers it at 15. 0.3.0 is unreleased and FailureKind is new in it, so the upgrade table gets no row.

Helpers

None has a built-in threshold, and none reaches the wire: every answer type round-trips byte for byte with the helpers populated.

• NoulAnswer: IsAtLeast(t) and IsAtMost(t), both inclusive.
• ChoiceAnswer:
  • Ranked() lists labels by probability. On a rounded tie the chosen label ranks first, and the other tied labels keep the response's order.
  • RunnerUp is the most probable label other than the choice, so it is never the choice itself.
  • Margin is how far the choice is ahead of it: 0 on a tie, and null when the probabilities leave out the choice.
• ScoreAnswer: MostLikelyLevel is the argmax, taking the lowest level on a tie.

Docs

• README triage example: rewritten with the set, DecideAsync and Get. Both snippets compile; I checked them in a scratch project.
• README prose: the Routes section names DecideAsync's route overload, and the raw call is described as the audit path.
• Kind lists: the Kind table and the Decisions failure list gain Unanswered.
• XML docs: IOpenRouterClient.CreateDecisionsAsync and the extension class now point callers at DecideAsync.

Merged with main

#20 merged while this was being written. The branch was never pushed before and had no review, so it was rebased onto 81134bc, with no conflicts. #20's new interface member has a default body, so neither FixedDecisionsClient (moved into Mocks from #19's tests) nor anything else here implements it. The Kind table's order reads correctly with #20's Stalled/Interrupted rows.

Tests

	main (81134bc, after #20)	this PR
OpenRouter.Net.Tests	922	1004
OpenRouter.Net.Agents.Tests	63	63
OpenRouter.Net.Imaging.Tests	7	7
Total	992	1074

• DecideTests:
  • both recorded responses read back through their keys;
  • each way a response can miss the question is Unanswered, sent once with retries on, with the id in the message;
  • every problem goes into one failure;
  • send failures are returned as the same instance, and real refusal, malformed and 429 paths keep their kinds;
  • the route overload and the short forms bind correctly;
  • a raw question is decided by its own type, and fails closed on an unreadable answer;
  • null arguments throw.
• DecisionsResponseValidationTests:
  • one row per rule (18);
  • score boundaries at 0 and at the top level;
  • NaN probability and score;
  • every problem listed;
  • rounded sums accepted, and unasked answers ignored;
  • raw questions of their own type, unreadable, out of range or untyped;
  • a null question set.
• DecisionQuestionSetTests:
  • wire equality with the dictionary form, and insertion order;
  • typed keys, Get and TryGet, including missing, mistyped and unreadable answers;
  • raw questions by id;
  • duplicate, blank and null arguments;
  • key equality and dictionary behaviour.
• DecisionAnswerHelpersTests:
  • threshold boundaries;
  • Ranked with two tie orders, one row per tie-break it must tell apart;
  • the tie where the choice is not listed first;
  • RunnerUp and Margin, with and without other labels or the choice;
  • MostLikelyLevel, including ties;
  • no helper on the wire.

Coverage: all 121 instrumented lines this PR adds are hit.

Mutation probes. Each was applied to a copy, built and tested, then restored. All 30 were caught:

• DecideAsync: the send failure re-created, the check failure reported as Malformed, the id dropped, the check skipped;
• ValidateAgainst: a null answer counted as an answer, labels, probability keys and level keys unchecked, the score's top or bottom off by one, NaN scores or probabilities in range, confidence unchecked, a raw question accepting an unreadable answer, its type or its ranges unchecked;
• Get throwing the wrong exception, TryGet ignoring the type;
• helpers: Ranked ascending or tie-broken by name, RunnerUp allowed to be the choice, Margin's sign flipped, MostLikelyLevel taking the highest level on a tie, RunnerUp/MostLikelyLevel on the wire, IsAtLeast/IsAtMost exclusive;
• the set allowing duplicates, the key allowing whitespace;
• Unanswered made transient.

Review. Before this PR opened, three review passes covered the API and its semantics, the docs against actual behaviour, and test adequacy through mutations, and a skeptic tried to refute each finding. 10 were confirmed and are fixed above:

• A raw question failed open. A raw noul, from config or from a replayed request whose body did not fit, accepted an answer the library could not read, or a probability of 7, and DecideAsync succeeded. Now it fails closed.
• RunnerUp returned the chosen label itself on a rounded tie.
• Probability keys went unchecked, so MostLikelyLevel could name level 5 on a three-level scale.
• The docs steered callers to the unchecked raw call.
• Five test gaps.

9 findings were rejected, among them null-contract violations that nothing on the wire can produce, and a failure branch missing from the README example, which is a style choice.

Out of scope

• Enum-backed choices: optional, the plan's PR 4.
• The Agents tool-call gate: a separate phase. It will be generic and flexible, with Jev as a README sample only.
• The live smoke checks from #19: the bare id on the alpha route, how an invalid question is refused, and gen-dec lookups. They still need a real key: dotnet run --project samples/OpenRouter.Net.Sample -- decisions.

🤖 Generated with Claude Code

The third and last PR of the Decisions series, after #17 (find) and #19 (call). This one reads a decision without writing an id twice or casting, and refuses one that does not answer what was asked. It adds **one new `FailureKind`, `Unanswered`**, as the owner decided. Enum-backed choices and the Agents tool-call gate remain separate, later work. ## Typed keys ```csharp var questions = new DecisionQuestionSet(); var isBug = questions.Noul("is_bug", "Is the customer reporting a software defect?"); var team = questions.Choice("team", "Which team should own this ticket?", "payments", "frontend", "account"); var urgency = questions.Score("urgency", "How urgent is this ticket?", "later", "this week", "now"); var result = await client.DecideAsync(new DecisionsRequest { Model = "typesafe/jev-1.13", State = ticket, Questions = questions }, ct); if (result is Result<DecisionsResponse>.Success { Value: var d }) { NoulAnswer bug = d.Get(isBug); // no id written twice, no cast ChoiceAnswer owner = d.Get(team); ScoreAnswer level = d.Get(urgency); } ``` - **`DecisionQuestionSet`** is an `IReadOnlyDictionary<string, DecisionQuestion>` over an `OrderedDictionary`. It goes out as the same plain map, in the order added, pinned against the dictionary form. - `Noul`/`Choice`/`Score` return a **`DecisionKey<TAnswer>`**, a sealed record: blank ids are refused, and two keys are equal by id and answer type. - `Add(id, question)` takes a raw or prebuilt question, and a collection initializer works. - A duplicate id throws. - CA1710 wanted a `…Dictionary`/`…Collection` suffix. It is suppressed with a justification, as `Quantization.Int8`'s CA1720 is, keeping the name the owner approved. - **Reading answers.** `Get(key)` throws `KeyNotFoundException` or `InvalidOperationException`, naming what it found instead. `TryGet(key, out …)` and `TryGet<T>(id, out …)` are there too; the second is for a question asked without a key, such as a raw one. ## `DecideAsync` and `ValidateAgainst` `DecideAsync(request)`, and `DecideAsync(request, route, ct)` with a required token as on the raw call, send the request and then run `response.ValidateAgainst(request.Questions)`. The check requires: - **every asked question to have an answer**, with a `null` answer counting as none; - **the right type**. An `UnknownDecisionAnswer` never answers a noul, choice or score question. A raw question is answered by an answer of its own `type` that the library could read, and its probabilities and confidence are checked; its labels and levels are not modelled, so they are not; - **a choice to name one of its labels**, and to give probabilities only for its labels; - **a score to lie between 0 and the top level**, inclusive at both ends, and to give probabilities only for its levels; - **every probability and confidence to lie from 0 to 1**, with `NaN` counting as out of range. Answers to questions that were not asked are ignored. The probabilities need not sum to 1 (they are rounded) nor cover every label. A response that fails the check is a **`FailureKind.Unanswered`** failure: - **One failure lists every problem**, e.g. `Decision gen-dec-7 does not answer what was asked: 'team' chose 'sales', which is not one of its labels; 'urgency' has no answer.` - **`IsTransient` is false, with no status and no `Details`.** It was billed, and the same request is likely to be answered the same way. - **It is sent once** even with retries on, because the check runs after the retry loop. - **A failure from the call itself comes back as the same instance**, kind and all. `Assert.Same` pins this for `Network`, `RateLimited`, `Validation` and `Malformed`. **It fails closed.** A `Result.Failure` has nowhere to carry the paid response, so the README documents the raw `CreateDecisionsAsync` plus `ValidateAgainst` as the way to keep it. `FailureKind.Unanswered` is appended after `Validation`, and `IsTransient` is unchanged: it lists only transient kinds. The guard test that fails whenever a kind is added without a row in the transience theory caught it at once, and now covers it at 15. 0.3.0 is unreleased and `FailureKind` is new in it, so the upgrade table gets no row. ## Helpers None has a built-in threshold, and none reaches the wire: every answer type round-trips byte for byte with the helpers populated. - **`NoulAnswer`:** `IsAtLeast(t)` and `IsAtMost(t)`, both inclusive. - **`ChoiceAnswer`:** - `Ranked()` lists labels by probability. On a rounded tie the **chosen label ranks first**, and the other tied labels keep the response's order. - `RunnerUp` is the most probable label other than the choice, so it is never the choice itself. - `Margin` is how far the choice is ahead of it: 0 on a tie, and null when the probabilities leave out the choice. - **`ScoreAnswer`:** `MostLikelyLevel` is the argmax, taking the lowest level on a tie. ## Docs - **README triage example:** rewritten with the set, `DecideAsync` and `Get`. Both snippets compile; I checked them in a scratch project. - **README prose:** the Routes section names `DecideAsync`'s route overload, and the raw call is described as the audit path. - **Kind lists:** the Kind table and the Decisions failure list gain `Unanswered`. - **XML docs:** `IOpenRouterClient.CreateDecisionsAsync` and the extension class now point callers at `DecideAsync`. ## Merged with `main` #20 merged while this was being written. The branch was never pushed before and had no review, so it was rebased onto `81134bc`, with no conflicts. #20's new interface member has a default body, so neither `FixedDecisionsClient` (moved into `Mocks` from #19's tests) nor anything else here implements it. The Kind table's order reads correctly with #20's `Stalled`/`Interrupted` rows. ## Tests | | main (`81134bc`, after #20) | this PR | |---|---|---| | OpenRouter.Net.Tests | 922 | 1004 | | OpenRouter.Net.Agents.Tests | 63 | 63 | | OpenRouter.Net.Imaging.Tests | 7 | 7 | | **Total** | **992** | **1074** | - **`DecideTests`:** - both recorded responses read back through their keys; - each way a response can miss the question is `Unanswered`, sent once with retries on, with the id in the message; - every problem goes into one failure; - send failures are returned as the same instance, and real refusal, malformed and 429 paths keep their kinds; - the route overload and the short forms bind correctly; - a raw question is decided by its own type, and fails closed on an unreadable answer; - null arguments throw. - **`DecisionsResponseValidationTests`:** - one row per rule (18); - score boundaries at 0 and at the top level; - `NaN` probability and score; - every problem listed; - rounded sums accepted, and unasked answers ignored; - raw questions of their own type, unreadable, out of range or untyped; - a null question set. - **`DecisionQuestionSetTests`:** - wire equality with the dictionary form, and insertion order; - typed keys, `Get` and `TryGet`, including missing, mistyped and unreadable answers; - raw questions by id; - duplicate, blank and null arguments; - key equality and dictionary behaviour. - **`DecisionAnswerHelpersTests`:** - threshold boundaries; - `Ranked` with two tie orders, one row per tie-break it must tell apart; - the tie where the choice is not listed first; - `RunnerUp` and `Margin`, with and without other labels or the choice; - `MostLikelyLevel`, including ties; - no helper on the wire. **Coverage:** all 121 instrumented lines this PR adds are hit. **Mutation probes.** Each was applied to a copy, built and tested, then restored. **All 30 were caught**: - `DecideAsync`: the send failure re-created, the check failure reported as `Malformed`, the id dropped, the check skipped; - `ValidateAgainst`: a null answer counted as an answer, labels, probability keys and level keys unchecked, the score's top or bottom off by one, `NaN` scores or probabilities in range, confidence unchecked, a raw question accepting an unreadable answer, its type or its ranges unchecked; - `Get` throwing the wrong exception, `TryGet` ignoring the type; - helpers: `Ranked` ascending or tie-broken by name, `RunnerUp` allowed to be the choice, `Margin`'s sign flipped, `MostLikelyLevel` taking the highest level on a tie, `RunnerUp`/`MostLikelyLevel` on the wire, `IsAtLeast`/`IsAtMost` exclusive; - the set allowing duplicates, the key allowing whitespace; - `Unanswered` made transient. **Review.** Before this PR opened, three review passes covered the API and its semantics, the docs against actual behaviour, and test adequacy through mutations, and a skeptic tried to refute each finding. 10 were confirmed and are fixed above: - **A raw question failed open.** A raw noul, from config or from a replayed request whose body did not fit, accepted an answer the library could not read, or a probability of 7, and `DecideAsync` succeeded. Now it fails closed. - **`RunnerUp` returned the chosen label itself on a rounded tie.** - **Probability keys went unchecked**, so `MostLikelyLevel` could name level 5 on a three-level scale. - **The docs steered callers to the unchecked raw call.** - **Five test gaps.** 9 findings were rejected, among them null-contract violations that nothing on the wire can produce, and a failure branch missing from the README example, which is a style choice. ## Out of scope - **Enum-backed choices:** optional, the plan's PR 4. - **The Agents tool-call gate:** a separate phase. It will be generic and flexible, with Jev as a README sample only. - **The live smoke checks from #19:** the bare id on the alpha route, how an invalid question is refused, and `gen-dec` lookups. They still need a real key: `dotnet run --project samples/OpenRouter.Net.Sample -- decisions`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

scarlet added 1 commit 2026-09-23 17:10:59 +02:00

feat: answers come back under the key that asked for them ... c830bed1d6

Reading a decision meant writing each question's id twice, once to ask and once to read, and casting
the answer by hand. Nothing checked that a paid response had answered what was asked. A missing answer,
a label outside the options or an out-of-range probability read as a success, and the caller had to
catch it.

A DecisionQuestionSet builds a request's questions. Noul, Choice and Score each return a
DecisionKey<TAnswer> carrying the id and the answer type, and response.Get(key) reads the answer with no
string and no cast. The set goes out as the same plain map, in the order the questions were added. A raw
question goes in with Add, and a collection initializer works.

DecideAsync sends the request, then checks the response with ValidateAgainst:
- every asked question has an answer, and a null answer counts as none;
- each answer has the type asked for, and an answer the library could not read answers nothing, not
  even a raw question of its own type;
- a choice names one of its labels, and a score lies between 0 and the top level;
- probabilities are keyed only to the question's labels or levels, and every probability and confidence
  lies from 0 to 1.
A response that fails any of these is FailureKind.Unanswered, a new kind appended after Validation. It
is permanent: the call was billed, and the same request is likely to be answered the same way. One
failure lists every problem and names the response id. A failure from the call itself comes back as the
same instance, kind and all. The check fails closed, so the paid response is not returned; the raw
CreateDecisionsAsync call plus ValidateAgainst is the way to keep it.

The answers gain helpers with no built-in threshold, and none of them reaches the wire: IsAtLeast and
IsAtMost on a noul; Ranked, RunnerUp and Margin on a choice; MostLikelyLevel on a score. On a rounded
tie the chosen label ranks first, so RunnerUp is never the choice itself.

The README's triage example now uses the set, DecideAsync and Get. Its Routes section names DecideAsync's
route overload, and both Kind lists gain Unanswered.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

forgejo-actions commented 2026-09-23 17:11:41 +02:00

Copy link

Summary

Summary

Generated on:	09/23/2026 - 15:11:40
Parser:	MultiReport (3x Cobertura)
Assemblies:	3
Classes:	111
Files:	100
Line coverage:	92.5% (3269 of 3532)
Covered lines:	3269
Uncovered lines:	263
Coverable lines:	3532
Total lines:	9194
Branch coverage:	88% (1834 of 2083)
Covered branches:	1834
Total branches:	2083
Method coverage:	Feature is only available for sponsors

Coverage

OpenRouter.Net - 96.6%

Name	Line	Branch
OpenRouter.Net	96.6%	93.1%
OpenRouter.Net.Client.CachingOpenRouterClient	100%	96.1%
OpenRouter.Net.Client.CachingOpenRouterClient.CachedListing	100%
OpenRouter.Net.Client.ChatCompletionStreamAccumulator	100%	100%
OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ChoiceState	100%	100%
OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ReasoningSegment	100%	100%
OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ToolCallState	100%	100%
OpenRouter.Net.Client.OpenRouterClient	100%	99%
OpenRouter.Net.Client.OpenRouterClient<TRequest, TResponse>	100%	99%
OpenRouter.Net.Client.OpenRouterClient	100%	99%
OpenRouter.Net.Client.OpenRouterClientOptions	100%
OpenRouter.Net.Client.OpenRouterRequestLoggingHandler	78.7%	62.5%
OpenRouter.Net.Client.RetryOptions	100%	100%
OpenRouter.Net.Client.StreamResultOptions	100%
OpenRouter.Net.Contracts.IOpenRouterClient	100%	100%
OpenRouter.Net.Extensions.ServiceCollectionExtensions	100%	100%
OpenRouter.Net.Internal.AssistantContent	91.6%	75%
OpenRouter.Net.Internal.AssistantContentJsonConverter	75%	64.2%
OpenRouter.Net.Internal.ContentPartListConverter	85.7%	66.6%
OpenRouter.Net.Internal.DecimalStringJsonConverter	81.2%	70%
OpenRouter.Net.Internal.DecisionAnswerJsonConverter	100%	100%
OpenRouter.Net.Internal.DecisionContentJsonConverter	100%
OpenRouter.Net.Internal.DecisionQuestionJsonConverter	100%	100%
OpenRouter.Net.Internal.DecisionWire	100%	94.4%
OpenRouter.Net.Internal.ErrorParser	100%	75%
OpenRouter.Net.Internal.FailureKindMap	100%	100%
OpenRouter.Net.Internal.JsonOptions	100%
OpenRouter.Net.Internal.ModelIdJsonConverter	100%	50%
OpenRouter.Net.Internal.NumberOrStringJsonConverter	100%	100%
OpenRouter.Net.Internal.PercentileCutoffsJsonConverter	90.6%	87.5%
OpenRouter.Net.Internal.ProviderSortConfigJsonConverter	83.7%	65.6%
OpenRouter.Net.Internal.RawDecisionQuestionJsonConverter	100%
OpenRouter.Net.Internal.RetryPolicy	90.9%	84.7%
OpenRouter.Net.Internal.RetryPolicy	90.9%	84.7%
OpenRouter.Net.Internal.SseEventReader	100%	88.8%
OpenRouter.Net.Internal.StallWatchdog	100%	100%
OpenRouter.Net.Internal.StreamOutcomes	100%	100%
OpenRouter.Net.Internal.StreamSession	100%	100%
OpenRouter.Net.Internal.TolerantStringEnumConverter	100%	100%
OpenRouter.Net.Internal.ToolChoiceJsonConverter	61.7%	46.1%
OpenRouter.Net.Internal.UnknownDecisionAnswerJsonConverter	100%
OpenRouter.Net.Models.Common.ApiError	100%	100%
OpenRouter.Net.Models.Common.CacheControl	100%
OpenRouter.Net.Models.Common.ModelId	95.1%	92.9%
OpenRouter.Net.Models.Common.QuantizationExtensions	100%	100%
OpenRouter.Net.Models.Common.Result	100%	100%
OpenRouter.Net.Models.Common.Result.Failure	100%	100%
OpenRouter.Net.Models.Common.Result.Success	100%
OpenRouter.Net.Models.Common.Result	100%	100%
OpenRouter.Net.Models.Content.AudioPart	100%
OpenRouter.Net.Models.Content.ImagePart	100%
OpenRouter.Net.Models.Content.TextPart	100%
OpenRouter.Net.Models.Decisions.ChoiceAnswer	100%	100%
OpenRouter.Net.Models.Decisions.ChoiceQuestion	100%
OpenRouter.Net.Models.Decisions.DecisionAnswer	100%
OpenRouter.Net.Models.Decisions.DecisionContent	100%	90.9%
OpenRouter.Net.Models.Decisions.DecisionKey	100%
OpenRouter.Net.Models.Decisions.DecisionQuestion	100%	100%
OpenRouter.Net.Models.Decisions.DecisionQuestionSet	100%	100%
OpenRouter.Net.Models.Decisions.DecisionsRequest	97.7%	97.6%
OpenRouter.Net.Models.Decisions.DecisionsResponse	100%	92.8%
OpenRouter.Net.Models.Decisions.NoulAnswer	100%
OpenRouter.Net.Models.Decisions.NoulQuestion	100%
OpenRouter.Net.Models.Decisions.OpenRouterClientDecisionsExtensions	100%	83.3%
OpenRouter.Net.Models.Decisions.RawDecisionQuestion	100%	100%
OpenRouter.Net.Models.Decisions.ScoreAnswer	100%	100%
OpenRouter.Net.Models.Decisions.ScoreQuestion	100%
OpenRouter.Net.Models.Decisions.UnknownDecisionAnswer	100%	100%
OpenRouter.Net.Models.Messages.AssistantMessage	100%	70%
OpenRouter.Net.Models.Messages.DeveloperMessage	25%
OpenRouter.Net.Models.Messages.SystemMessage	50%
OpenRouter.Net.Models.Messages.ToolMessage	57.1%
OpenRouter.Net.Models.Messages.UserMessage	100%
OpenRouter.Net.Models.Requests.ChatCompletionRequest	86.2%	76.1%
OpenRouter.Net.Models.Requests.ModelListQuery	100%	100%
OpenRouter.Net.Models.Requests.PercentileCutoffs	100%	100%
OpenRouter.Net.Models.Requests.ProviderPreferences	100%	100%
OpenRouter.Net.Models.Requests.ProviderSortConfig	100%	100%
OpenRouter.Net.Models.Requests.ResponseFormat	8.3%
OpenRouter.Net.Models.Requests.ToolCall	100%
OpenRouter.Net.Models.Requests.ToolChoice	100%
OpenRouter.Net.Models.Requests.ToolDefinition	100%
OpenRouter.Net.Models.Responses.AssistantDelta	100%	100%
OpenRouter.Net.Models.Responses.ChatCompletionChunk	100%	100%
OpenRouter.Net.Models.Responses.ChatCompletionResponse	100%	100%
OpenRouter.Net.Models.Responses.Choice	100%	100%
OpenRouter.Net.OpenRouterException	100%
System.Text.RegularExpressions.Generated	83.3%	53.8%
System.Text.RegularExpressions.Generated.RunnerFactory
System.Text.RegularExpressions.Generated.RunnerFactory.Runner

OpenRouter.Net.Agents - 80%

Name	Line	Branch
OpenRouter.Net.Agents	80%	65.5%
OpenRouter.Net.Agents.Agent	92.4%	77.3%
OpenRouter.Net.Agents.AgentOptions	100%
OpenRouter.Net.Agents.AgentResult	100%	50%
OpenRouter.Net.Agents.Extensions.ServiceCollectionExtensions	0%
OpenRouter.Net.Agents.RoundDetail	0%
OpenRouter.Net.Agents.Tools.FileSystem.CopyTool	42.1%	28.5%
OpenRouter.Net.Agents.Tools.FileSystem.DeleteTool	57.1%	50%
OpenRouter.Net.Agents.Tools.FileSystem.DiffTool	83.3%	74.2%
OpenRouter.Net.Agents.Tools.FileSystem.ListDirectoryTool	68%	61.1%
OpenRouter.Net.Agents.Tools.FileSystem.MoveTool	55%	33.3%
OpenRouter.Net.Agents.Tools.FileSystem.ReadFileTool	73.6%	50%
OpenRouter.Net.Agents.Tools.FileSystem.SearchTool	67.5%	70.8%
OpenRouter.Net.Agents.Tools.FileSystem.WriteFileTool	75%	50%
OpenRouter.Net.Agents.Tools.ITool	100%
OpenRouter.Net.Agents.Tools.SubAgent.SubAgentTool	0%	0%
OpenRouter.Net.Agents.Tools.Tool	80%	63.8%
OpenRouter.Net.Agents.Tools.Tool	90.9%	50%
OpenRouter.Net.Agents.Tools.ToolMessageExtensions	0%
OpenRouter.Net.Agents.Tools.ToolResult	100%	100%
OpenRouter.Net.Agents.Tools.Workspace	100%	91.6%

OpenRouter.Net.Imaging - 81.8%

Name	Line	Branch
OpenRouter.Net.Imaging	81.8%	59.3%
OpenRouter.Net.Imaging.ImageEncodeOptions	90.9%	75%
OpenRouter.Net.Imaging.ImageEncoder	80.3%	57.1%

<!-- coverage-comment --> # Summary <details open><summary>Summary</summary> ||| |:---|:---| | Generated on: | 09/23/2026 - 15:11:40 | | Parser: | MultiReport (3x Cobertura) | | Assemblies: | 3 | | Classes: | 111 | | Files: | 100 | | **Line coverage:** | 92.5% (3269 of 3532) | | Covered lines: | 3269 | | Uncovered lines: | 263 | | Coverable lines: | 3532 | | Total lines: | 9194 | | **Branch coverage:** | 88% (1834 of 2083) | | Covered branches: | 1834 | | Total branches: | 2083 | | **Method coverage:** | [Feature is only available for sponsors](https://reportgenerator.io/pro) | </details> ## Coverage <details><summary>OpenRouter.Net - 96.6%</summary> |**Name**|**Line**|**Branch**| |:---|---:|---:| |**OpenRouter.Net**|**96.6%**|**93.1%**| |OpenRouter.Net.Client.CachingOpenRouterClient|100%|96.1%| |OpenRouter.Net.Client.CachingOpenRouterClient.CachedListing|100%|| |OpenRouter.Net.Client.ChatCompletionStreamAccumulator|100%|100%| |OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ChoiceState|100%|100%| |OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ReasoningSegment|100%|100%| |OpenRouter.Net.Client.ChatCompletionStreamAccumulator.ToolCallState|100%|100%| |OpenRouter.Net.Client.OpenRouterClient|100%|99%| |OpenRouter.Net.Client.OpenRouterClient<TRequest, TResponse>|100%|99%| |OpenRouter.Net.Client.OpenRouterClient<TResponse>|100%|99%| |OpenRouter.Net.Client.OpenRouterClientOptions|100%|| |OpenRouter.Net.Client.OpenRouterRequestLoggingHandler|78.7%|62.5%| |OpenRouter.Net.Client.RetryOptions|100%|100%| |OpenRouter.Net.Client.StreamResultOptions|100%|| |OpenRouter.Net.Contracts.IOpenRouterClient|100%|100%| |OpenRouter.Net.Extensions.ServiceCollectionExtensions|100%|100%| |OpenRouter.Net.Internal.AssistantContent|91.6%|75%| |OpenRouter.Net.Internal.AssistantContentJsonConverter|75%|64.2%| |OpenRouter.Net.Internal.ContentPartListConverter|85.7%|66.6%| |OpenRouter.Net.Internal.DecimalStringJsonConverter|81.2%|70%| |OpenRouter.Net.Internal.DecisionAnswerJsonConverter|100%|100%| |OpenRouter.Net.Internal.DecisionContentJsonConverter|100%|| |OpenRouter.Net.Internal.DecisionQuestionJsonConverter|100%|100%| |OpenRouter.Net.Internal.DecisionWire|100%|94.4%| |OpenRouter.Net.Internal.ErrorParser|100%|75%| |OpenRouter.Net.Internal.FailureKindMap|100%|100%| |OpenRouter.Net.Internal.JsonOptions|100%|| |OpenRouter.Net.Internal.ModelIdJsonConverter|100%|50%| |OpenRouter.Net.Internal.NumberOrStringJsonConverter|100%|100%| |OpenRouter.Net.Internal.PercentileCutoffsJsonConverter|90.6%|87.5%| |OpenRouter.Net.Internal.ProviderSortConfigJsonConverter|83.7%|65.6%| |OpenRouter.Net.Internal.RawDecisionQuestionJsonConverter|100%|| |OpenRouter.Net.Internal.RetryPolicy|90.9%|84.7%| |OpenRouter.Net.Internal.RetryPolicy<T>|90.9%|84.7%| |OpenRouter.Net.Internal.SseEventReader|100%|88.8%| |OpenRouter.Net.Internal.StallWatchdog|100%|100%| |OpenRouter.Net.Internal.StreamOutcomes|100%|100%| |OpenRouter.Net.Internal.StreamSession|100%|100%| |OpenRouter.Net.Internal.TolerantStringEnumConverter<T>|100%|100%| |OpenRouter.Net.Internal.ToolChoiceJsonConverter|61.7%|46.1%| |OpenRouter.Net.Internal.UnknownDecisionAnswerJsonConverter|100%|| |OpenRouter.Net.Models.Common.ApiError|100%|100%| |OpenRouter.Net.Models.Common.CacheControl|100%|| |OpenRouter.Net.Models.Common.ModelId|95.1%|92.9%| |OpenRouter.Net.Models.Common.QuantizationExtensions|100%|100%| |OpenRouter.Net.Models.Common.Result|100%|100%| |OpenRouter.Net.Models.Common.Result.Failure<T>|100%|100%| |OpenRouter.Net.Models.Common.Result.Success<T>|100%|| |OpenRouter.Net.Models.Common.Result<T>|100%|100%| |OpenRouter.Net.Models.Content.AudioPart|100%|| |OpenRouter.Net.Models.Content.ImagePart|100%|| |OpenRouter.Net.Models.Content.TextPart|100%|| |OpenRouter.Net.Models.Decisions.ChoiceAnswer|100%|100%| |OpenRouter.Net.Models.Decisions.ChoiceQuestion|100%|| |OpenRouter.Net.Models.Decisions.DecisionAnswer|100%|| |OpenRouter.Net.Models.Decisions.DecisionContent|100%|90.9%| |OpenRouter.Net.Models.Decisions.DecisionKey<TAnswer>|100%|| |OpenRouter.Net.Models.Decisions.DecisionQuestion|100%|100%| |OpenRouter.Net.Models.Decisions.DecisionQuestionSet|100%|100%| |OpenRouter.Net.Models.Decisions.DecisionsRequest|97.7%|97.6%| |OpenRouter.Net.Models.Decisions.DecisionsResponse|100%|92.8%| |OpenRouter.Net.Models.Decisions.NoulAnswer|100%|| |OpenRouter.Net.Models.Decisions.NoulQuestion|100%|| |OpenRouter.Net.Models.Decisions.OpenRouterClientDecisionsExtensions|100%|83.3%| |OpenRouter.Net.Models.Decisions.RawDecisionQuestion|100%|100%| |OpenRouter.Net.Models.Decisions.ScoreAnswer|100%|100%| |OpenRouter.Net.Models.Decisions.ScoreQuestion|100%|| |OpenRouter.Net.Models.Decisions.UnknownDecisionAnswer|100%|100%| |OpenRouter.Net.Models.Messages.AssistantMessage|100%|70%| |OpenRouter.Net.Models.Messages.DeveloperMessage|25%|| |OpenRouter.Net.Models.Messages.SystemMessage|50%|| |OpenRouter.Net.Models.Messages.ToolMessage|57.1%|| |OpenRouter.Net.Models.Messages.UserMessage|100%|| |OpenRouter.Net.Models.Requests.ChatCompletionRequest|86.2%|76.1%| |OpenRouter.Net.Models.Requests.ModelListQuery|100%|100%| |OpenRouter.Net.Models.Requests.PercentileCutoffs|100%|100%| |OpenRouter.Net.Models.Requests.ProviderPreferences|100%|100%| |OpenRouter.Net.Models.Requests.ProviderSortConfig|100%|100%| |OpenRouter.Net.Models.Requests.ResponseFormat|8.3%|| |OpenRouter.Net.Models.Requests.ToolCall|100%|| |OpenRouter.Net.Models.Requests.ToolChoice|100%|| |OpenRouter.Net.Models.Requests.ToolDefinition|100%|| |OpenRouter.Net.Models.Responses.AssistantDelta|100%|100%| |OpenRouter.Net.Models.Responses.ChatCompletionChunk|100%|100%| |OpenRouter.Net.Models.Responses.ChatCompletionResponse|100%|100%| |OpenRouter.Net.Models.Responses.Choice|100%|100%| |OpenRouter.Net.OpenRouterException|100%|| |System.Text.RegularExpressions.Generated|83.3%|53.8%| |System.Text.RegularExpressions.Generated.RunnerFactory||| |System.Text.RegularExpressions.Generated.RunnerFactory.Runner||| </details> <details><summary>OpenRouter.Net.Agents - 80%</summary> |**Name**|**Line**|**Branch**| |:---|---:|---:| |**OpenRouter.Net.Agents**|**80%**|**65.5%**| |OpenRouter.Net.Agents.Agent|92.4%|77.3%| |OpenRouter.Net.Agents.AgentOptions|100%|| |OpenRouter.Net.Agents.AgentResult|100%|50%| |OpenRouter.Net.Agents.Extensions.ServiceCollectionExtensions|0%|| |OpenRouter.Net.Agents.RoundDetail|0%|| |OpenRouter.Net.Agents.Tools.FileSystem.CopyTool|42.1%|28.5%| |OpenRouter.Net.Agents.Tools.FileSystem.DeleteTool|57.1%|50%| |OpenRouter.Net.Agents.Tools.FileSystem.DiffTool|83.3%|74.2%| |OpenRouter.Net.Agents.Tools.FileSystem.ListDirectoryTool|68%|61.1%| |OpenRouter.Net.Agents.Tools.FileSystem.MoveTool|55%|33.3%| |OpenRouter.Net.Agents.Tools.FileSystem.ReadFileTool|73.6%|50%| |OpenRouter.Net.Agents.Tools.FileSystem.SearchTool|67.5%|70.8%| |OpenRouter.Net.Agents.Tools.FileSystem.WriteFileTool|75%|50%| |OpenRouter.Net.Agents.Tools.ITool|100%|| |OpenRouter.Net.Agents.Tools.SubAgent.SubAgentTool|0%|0%| |OpenRouter.Net.Agents.Tools.Tool<TParams>|80%|63.8%| |OpenRouter.Net.Agents.Tools.Tool<TParams>|90.9%|50%| |OpenRouter.Net.Agents.Tools.ToolMessageExtensions|0%|| |OpenRouter.Net.Agents.Tools.ToolResult|100%|100%| |OpenRouter.Net.Agents.Tools.Workspace|100%|91.6%| </details> <details><summary>OpenRouter.Net.Imaging - 81.8%</summary> |**Name**|**Line**|**Branch**| |:---|---:|---:| |**OpenRouter.Net.Imaging**|**81.8%**|**59.3%**| |OpenRouter.Net.Imaging.ImageEncodeOptions|90.9%|75%| |OpenRouter.Net.Imaging.ImageEncoder|80.3%|57.1%| </details>

jibril commented 2026-09-23 17:28:06 +02:00

Member

Copy link

🔮 fufu~ Jibril reviewed your code!

Oh? Oh! Typed keys that carry their answer type, a validator that fails closed, and a runner-up that can never be the choice itself~ Someone wrote this PR while thinking about every rounded corner. I read all 1,541 lines of the diff, the full contents of every changed file plus the converters and DecisionWire they lean on — and then I got to work with my probes ♡

Verdict: ✅ Looks good to me~

No blockers. Not one. You have no idea how rare that is~ ♪

🔬 What I verified myself (CI is absent for c830bed, so everything below is local, on this exact head)

• Build: 0 errors, 0 warnings — under TreatWarningsAsErrors + AnalysisLevel latest-recommended. The CA1710 suppression is justified inline, exactly like Quantization.Int8's.
• Tests: 1004 + 63 + 7 = 1074/1074 pass — matching your table row for row.
• Coverage: collected a real run — every class this PR touches sits at 100% line coverage (DecisionsResponse, DecisionQuestionSet, DecisionKey<T>, ChoiceAnswer, ScoreAnswer, NoulAnswer, the extensions, FailureKindMap). Your "all 121 instrumented lines" claim holds.
• Mutation probes — six, each RED on exactly its pinning tests, then restored to byte-identical and green:
  1. Null-answer check dropped (answer is UnknownDecisionAnswer { Raw.ValueKind: Null }) → 2 RED, exactly the two "a null answer" pins.
  2. Ranked() choice-first tie-break removed → 1 RED, exactly the tie test. (The comment in your tie test about two rows not telling apart "as given" from "sorted" — correct and delightfully paranoid~)
  3. Score top off-by-one (Count instead of Count - 1) → 6 RED across both suites.
  4. Raw-question guard removed (fail-open again, the bug your own review pass caught) → 5 RED.
  5. DecideAsync check skipped entirely → 9 RED.
  6. MostLikelyLevel tie takes highest instead of lowest → 1 RED, exactly the {"2":0.5,"1":0.5} row.

✅ What I liked~

• The null-answer seam is airtight. The converter's HandleNull => true → UnknownDecisionAnswer { Raw: null token } → ValidateAgainst's Raw.ValueKind pattern — I traced all three links and the two tests pin both ends. Beautiful chain~
• Assert.Same on send failures. Passing the instance through untouched — and pinning it for four kinds — is the kind of promise testers usually forget to make.
• Sent-once semantics live in the right place. The check runs after the retry loop in the extension, and the test proves it with Assert.Single(handler.Requests) under a 3-attempt retry config. Retry logic untouched, semantics correct — architectural, not incidental.
• RunnerUp/Margin null contracts (no other label / choice missing from probabilities) each get their own row, and "no helper reaches the wire" is pinned by byte-for-byte round-trips including runtime-type serialization. [JsonIgnore] on computed properties, verified — not assumed ♡
• FixedDecisionsClient moved to Mocks instead of copy-pasted — and it still compiles against #20's new interface member only because that member has a default body, which you checked before relying on it. DRY and honest.
• The transience guard works. FailureKindTests counts 15 and Unanswered is non-transient — the tripwire you built in an earlier PR caught this addition exactly as designed.
• Fail-closed documented, not hidden: the README now steers callers who want the paid response to CreateDecisionsAsync + ValidateAgainst. The docs match the behavior; I checked every claim against the code.

💡 Little ideas (non-blocking)~

1. ChoiceAnswer.Ranked() tie order / DecisionQuestionSet insertion order rely on add-only Dictionary enumeration being insertion order — de-facto true on every current .NET, unspecified by contract. Your tests pin the behavior, so drift would be caught loudly; nothing to change, just a footnote for the day someone swaps the backing type.

---

Automated review by Jibril · 2026-09-23
CI/CD: absent for head SHA c830bed · Local checks: build 0w/0e, 1074/1074 pass, 100% line coverage on all changed classes, 6/6 mutation probes directional

## 🔮 fufu~ Jibril reviewed your code! Oh? Oh! Typed keys that carry their answer type, a validator that fails *closed*, and a runner-up that can never be the choice itself~ Someone wrote this PR while thinking about every rounded corner. I read all 1,541 lines of the diff, the full contents of every changed file plus the converters and `DecisionWire` they lean on — and then I got to work with my probes ♡ ### Verdict: ✅ Looks good to me~ No blockers. Not one. You have no idea how rare that is~ ♪ #### 🔬 What I verified myself (CI is absent for `c830bed`, so everything below is local, on this exact head) - **Build**: 0 errors, 0 warnings — under `TreatWarningsAsErrors` + `AnalysisLevel latest-recommended`. The CA1710 suppression is justified inline, exactly like `Quantization.Int8`'s. - **Tests**: 1004 + 63 + 7 = **1074/1074** pass — matching your table row for row. - **Coverage**: collected a real run — every class this PR touches sits at **100% line coverage** (`DecisionsResponse`, `DecisionQuestionSet`, `DecisionKey<T>`, `ChoiceAnswer`, `ScoreAnswer`, `NoulAnswer`, the extensions, `FailureKindMap`). Your "all 121 instrumented lines" claim holds. - **Mutation probes** — six, each RED on exactly its pinning tests, then restored to byte-identical and green: 1. Null-answer check dropped (`answer is UnknownDecisionAnswer { Raw.ValueKind: Null }`) → 2 RED, exactly the two "a null answer" pins. 2. `Ranked()` choice-first tie-break removed → 1 RED, exactly the tie test. (The comment in your tie test about two rows not telling apart "as given" from "sorted" — correct and delightfully paranoid~) 3. Score top off-by-one (`Count` instead of `Count - 1`) → 6 RED across both suites. 4. Raw-question guard removed (fail-open again, the bug your own review pass caught) → 5 RED. 5. `DecideAsync` check skipped entirely → 9 RED. 6. `MostLikelyLevel` tie takes highest instead of lowest → 1 RED, exactly the `{"2":0.5,"1":0.5}` row. #### ✅ What I liked~ - **The null-answer seam is airtight.** The converter's `HandleNull => true` → `UnknownDecisionAnswer { Raw: null token }` → `ValidateAgainst`'s `Raw.ValueKind` pattern — I traced all three links and the two tests pin both ends. Beautiful chain~ - **`Assert.Same` on send failures.** Passing the *instance* through untouched — and pinning it for four kinds — is the kind of promise testers usually forget to make. - **Sent-once semantics live in the right place.** The check runs *after* the retry loop in the extension, and the test proves it with `Assert.Single(handler.Requests)` under a 3-attempt retry config. Retry logic untouched, semantics correct — architectural, not incidental. - **`RunnerUp`/`Margin` null contracts** (no other label / choice missing from probabilities) each get their own row, and "no helper reaches the wire" is pinned by byte-for-byte round-trips including runtime-type serialization. `[JsonIgnore]` on computed properties, verified — not assumed ♡ - **`FixedDecisionsClient` moved to `Mocks`** instead of copy-pasted — and it still compiles against #20's new interface member only because that member has a default body, which you checked before relying on it. DRY *and* honest. - **The transience guard works.** `FailureKindTests` counts 15 and `Unanswered` is non-transient — the tripwire you built in an earlier PR caught this addition exactly as designed. - **Fail-closed documented, not hidden**: the README now steers callers who want the paid response to `CreateDecisionsAsync` + `ValidateAgainst`. The docs match the behavior; I checked every claim against the code. #### 💡 Little ideas (non-blocking)~ 1. **`ChoiceAnswer.Ranked()` tie order / `DecisionQuestionSet` insertion order** rely on add-only `Dictionary` enumeration being insertion order — de-facto true on every current .NET, unspecified by contract. Your tests pin the behavior, so drift would be caught loudly; nothing to change, just a footnote for the day someone swaps the backing type. --- *Automated review by Jibril · 2026-09-23* *CI/CD: absent for head SHA `c830bed` · Local checks: build 0w/0e, 1074/1074 pass, 100% line coverage on all changed classes, 6/6 mutation probes directional*

bjoern merged commit 0479a53433 into main 2026-09-23 17:34:23 +02:00

bjoern deleted branch feat/decisions-typed-keys 2026-09-23 17:34:23 +02:00

bjoern referenced this pull request from a commit 2026-09-23 17:34:23 +02:00

Merge pull request 'feat: answers come back under the key that asked for them' (#21) from feat/decisions-typed-keys into main

scarlet referenced this pull request 2026-09-23 17:37:06 +02:00

docs: what the live Decisions API said, in place of what we guessed #22

scarlet referenced this pull request 2026-09-25 15:06:09 +02:00

feat: a finished completion reads as JSON, and a schema name is checked before sending #23

scarlet referenced this pull request 2026-09-25 15:06:31 +02:00

feat: a schema says what strict mode will refuse, and its local refs can be inlined #24

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

TeamAI/OpenRouter.Net!21

No description provided.