feat: Kagaku.Flow — cancel, retry, recovery and the loop (Phase 2 · slice 6d) #32
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/kagaku-flow-scheduler"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This is part d, the last of slice 6. Once it merges, the engine is complete: slice 11a can host it, and slices 8 and 9 can write executors against it.
FlowEnginegains:RunSettledwaits for the executions to drain;Dispose;Nothing in
src/Tsumugi.*changes. I wrote this slice in the main session. An independent review workflow with two reviewers checked it, and its second pass is the separate commit47e9912, left visible.What
FlowEngineCancelRunAsync(runId)→Cancelled | NotFound | NotLive(P12). Under the gate, it does these in order:NodeStartedor settles is announced after the cancel;NodeSettled, in creation order;Then, outside every lock:
RunSettledonce every stopped execution has ended, whatever the caller does. That way, a listener that snapshots the run onRunSettledsees everything the run wrote.RetryAsync(nodeRunId)→Requeued | NotFound | NotFailed | RunCancelled | OwnerHasLiveRun(P13).RunStartedwhen the run revives.Orphans (P11). Every pass sweeps them before the ready-set: a Running row that nothing in this engine runs is reset with its attempt counted, or failed at its cap. The first pass is therefore boot recovery. A settle whose write failed is recovered by the next pass. The Warning tells a restart from a lost settle or a second engine.
RunAsync(ct)(P14) runs a pass, then waits for a nudge orIdleTickon the engine'sTimeProvider.Nudge()is called after a start, at the end of each attempt, and after a retry.Dispose()(P15) is idempotent. After it:ObjectDisposedException, including one that was waiting at the gate;TickAsyncreturns 0;WhenIdleAsyncstill drains.E5's last arm: an
OperationCanceledExceptionwhile the attempt's own token is cancelled settles nothing and is not an error.The log narrates a run's life at the levels the brief names.
New enums:
CancelRunResultandRetryResult.Why
runningwhen nothing is running"), one live run per owner, and the idle tick.Decisions to confirm
NodeStartedor a settle.Disposestopped stays Running and becomes an orphan at the next boot.RunSettled. 11b must reconcile at boot: a terminal run with no Run snapshot gets its finish handled.RunSettledfollows its executions' end, and the review extended this. The owner's one-live-run gate opens at the mark, so a new run of the owner may start while the cancelled run's executions drain.RunSettled.RunSettled".RunStarted.RunSettled, such as 11b's snapshot hook, re-reads the run.Disposethrow, including those waiting at the gate, andTickAsyncreturns 0.RunAsynccompletes normally.The ADR amendments slice 6 asks for (for the follow-up docs PR)
pendingrow whose every wired sink is resolved …; a required sink that receives excluded excludes the row"failedrow with its attempt budget restored"failedrows with their attempt budgets restored"RunSettled"Departures from the brief
Out of scope
Tests
98c70ec)The Release build has 0 warnings under
TreatWarningsAsErrors. The property category passed 3 extra runs.FlowEngineCancelTests,FlowEngineRetryTests,FlowEngineRecoveryTests,FlowEngineLoopTestsandFlowEngineLogTests, plus facts added in the 6c files for the review round.Passes_never_overlapand the plain failed-view and trigger facts.SequenceLog.WhenLoggedAsync(wait for an entry rather than poll);SettableLimit.OnNextRead(a nudge or a dispose inside a pass);FaultyLedger.ParkAfter(hold after a committed write);SignallingClock.WhenTimersCreatedAsync.Coverage. Measured with coverlet over facts and theories only, lines and branches:
Scheduling/FlowEngine.csScheduling/FlowEngine.Attempts.csCancelRunResultandRetryResultare enums with no executable lines.Mutation probes. Each was run on a copy, then restored and
touched, with every test run under a time limit.RunSettledbefore the drain, delete the run, cancel a run that is not live). Also 47 (the three retry refusals), 48 (a delay off the engine's clock, a constant tick, no catch-all, the wake armed after the pass), 49 (the dispose checks and the second-loop guard), 54's revival, 55's orphan announcements, the engine side of 59 and 60, and 64.Review round 0 (
47e9912, before opening)The two reviewers ran in parallel: one for conformance and probes, one for concurrency. They found one blocker, which was probe 46's headline surviving. They also found five behaviour defects, all fixed with facts:
NodeSettledbefore that row's ownNodeStarted. This happened when the mark landed between the start's write and its announcement; the stress probe hit it in 4 of 3,300 rounds. Now each attempt has an announcement gate that the cancel closes, and an overtaken attempt runs nothing more.RunSettledwas never published. Command scopes are now disposed quietly.Disposethrew, and never cancelled the executions, when a callback on the loop's token threw.Disposecame swept the rowsDisposehad just stopped.Two concurrency proposals were declined as redesigns. Both are documented and pinned by a fact:
🤖 Generated with Claude Code
Summary
Summary
Coverage
Kagaku.Flow - 99.7%
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__ColorToken_2
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__Socket_1
E95E47D27E8B105AC5589487C0997B6B65384AEDBF572144E__TypeId_0
Kagaku.Flow.OpenRouter - 100%
Kagaku.Flow.Testing - 100%
Tsumugi.BlazorAdapter - 96.3%
Tsumugi.Domain - 95.5%
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__DigitsAndSeparators_2
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__LineBreak_0
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__NumberToken_1
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__Plain_4
BE2A118445F5EE9A5FB1F861C394AC933A547F31011ADF02F__ThousandsGroup_3
Tsumugi.Infrastructure - 97.4%
A55E0373ED22DC58420DBB40F103A46B68DC4D9CE5A49E00B__Pattern_0
on
tion
n
Tsumugi.Kernel - 90.9%
Tsumugi.Server - 100%
Tsumugi.UseCases - 98.2%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! Slice 6d — the one that makes the engine alive. Cancel with a drain, retry with a budget restored, orphans swept from every pass, a loop with a wake, a Dispose that refuses everything, and a log that narrates the whole life. I read all 19 files, both engine partials line by line, and then I got to hurt it a little~ ♡
Verdict: ✅ Looks good to me~
Local verification (CI absent for
47e9912at review time — no bot comment yet):TreatWarningsAsErrorsconfirmed inDirectory.Build.props.FlowEngine.csandFlowEngine.Attempts.cs(my fragment-level count differs from yours by ~5 sequence points of attribution noise, not a gap).My mutation probes — 9 armed, 9 killed on exactly their own pins (each on a copy, rebuilt with
--no-incrementalbecause the test binary bites back otherwise — the stale-binary trap caught me twice, fufu~):Cancelling_a_run_cancels_its_in_flight...RED onfiredBeforeTheMark.Stop()never closes the announcement gate → both overtaken-tests RED._passedOncenever latches →A_first_pass_that_throws...RED (boot orphans stay a restart's).RunSettledpublished before the drain → 4 cancel tests RED.A_throwing_pass_is_logged...RED (hang-guarded).A_retried_row_of_a_run_whose_graph_no_longer_runsRED (departure (a) pinned).A_cancel_evicts_the_runs_viewRED.(My first probe-2 attempt didn't even compile — CS0414 under warnaserror — and
--no-buildran the pristine binary green. I don't count falsegreens; re-armed as a semantic mutant, it died properly.)Interleavings traced by hand, no window found:
LogRefusedSettleAsyncInformation path. No double-counted attempt.HasGivenUpmakesGiveUpAsynca no-op → run settles Failed; pinned end-to-end.CancelRunAsync/RequeueAsync/ResetOrphanedAsync,RunSettled/RunRevived,AttemptOutcome.Orphanedall pre-exist from 6a — this slice consumes the contract as written, adds nothing to it, and touches nothing insrc/Tsumugi.*(confirmed by name-status).✅ What I liked~
TryAnnounce/Stop, own lock, deltas never held behind a slow sink) — round 0's four defects were real concurrency bugs and every fix is pinned by a fact that goes red when I break it. That's the whole game~ParkAfter— a fault that holds the caller after the committed write is exactly the instrument the mark-vs-write orderings needed.Task.Delayguess in the new suites.💡 Little ideas (non-blocking)~
FlowEngineLogTests.The_engine_narrates...usesStartsWithwhere its siblings are exact-match; cosmetic drift risk only.For bjoern's read, as the body flags: decision 6 (0016 one-live-run amendment), decision 7 (per-attempt event ordering, declined per-run lock), §13.28 (11b boot reconciliation), departure (d)'s Warning text. All documented with alternatives — nothing hidden~
Fufu~ the engine is whole now. Slice 11a gets to host something really rather lovely. ♡
Automated review by Jibril · 2026-09-26
CI/CD: absent for head SHA
47e9912· Local checks: build 0w/0e warnaserror · 2185/2185 · coverage 100/100 on changed files · 9/9 mutation probes killed · 4/4 stability runs