feat: integrity sees the image tree #180
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/integrity-sees-the-image-tree"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What's in
GET /api/integrity/image-sweepreports two new checks:MissingImageFile(a row whose file is absent from whereImageFile.StoragePathsays it is) andOrphanImageFile(a file in the image or thumbnail tree that no row names). The client sends?cursor=<shard>and omits it to start; the answer carriesfromShard,nextCursor,isComplete, the shard counts andself/report/nextlinks. An absentnextplus a nullnextCursoris the completion signal; an unparseable cursor is a 400, not a silent restart; a shard directory that does not exist is an empty shard.DOUJIN_MANAGER_INTEGRITY_SWEEP_SHARDS(IntegrityOptions.SweepShardsPerRequest, default 16), documented indeploy/DEPLOYMENT.md§2 anddeploy/docker-compose.ymlin this PR. No migration.ListShardAsync, implemented over the filesystem. The per-shard row read is a prefix range over the TEXT primary key (SEARCH … USING INDEX sqlite_autoindex_image_files_1, no temp b-tree), so the database half of a slice is bounded like the filesystem half; and a shard's files are listed before its rows, the same order a writer touches them.repairaction, and the repair route answers 400. No ADR needed amending — 0014 is the reason repair is out of scope, 0007's envelope and 0021's.Producesdeclarations are followed, 0023 is untouched. A newIntegrityChecks.RunsInImageSweeptable records which run owns each kind, so the database report no longer claims "0 findings" for checks it never looked for.backend/openapi/v1.json; the README's API integrity row and the two prose bullets that described integrity as database-only now name the sweep.Verification
scripts/verify.shwas run fresh, in full, in this worktree at5a3a249; it printedAll tests passed!(1337 app tests) and== verify OK (all) ==. That is backend restore + Release build +dotnet test, then the app's submodule check,pub get, a cleanbuild_runnerregeneration (the committed*.g.dart/*.freezed.dartare current),flutter analyzeandflutter test.Layers exercised: UI (widget tests through a real store), Redux (epics through
createStore), REST (TestHost integration over a real temp image tree), use case (in-memory SQLite plus the real filesystem storages), persistence (the shard range runs against real SQLite, and itsEXPLAIN QUERY PLANis pinned byQueryPlanTests.Image_Sweep_Shard_Rows_Seek_The_Primary_Key).Mutation probes, each applied to a committed tree and reverted: 29 in the first round, all killed; 12 against the review fixes, all killed; 5 re-probing those fixes, 4 killed. Notable kills — reverting the range to
lower(substr(id,1,2))reddens the plan test andMissing_Findings_Arrive_In_Id_Order; swapping the shard's two reads reddens both race tests; arming either race decorator for a shard the fixture never sweeps reddens both, so both fixtures reach the branch they claim to pin; mutating the wire text reddens the OpenAPI snapshot; in the app,isRunning: !slice.isComplete→falsereddens the two new mid-walk tests and nothing else, andfindingCount→findings.lengthon the health card reddens the new count-vs-sample test and nothing else.One probe survived: moving the thumbnail listing after the rows are read leaves all 63 backend integrity/sweep tests green. The read-order property is pinned for the image tree (
A_Delete_That_Commits_Between_The_Two_Reads_Is_Not_A_Drift,An_Upload_That_Lands_Between_The_Two_Reads_Is_A_Missing_File) but not for the thumbnail half, and nothing on the wire promises it there. That gap is recorded nowhere but here — no backlog line was added for it. Two smaller wording defects also stand, likewise recorded only here: a test<remarks>misstates what swapping the reads produces (the probe shows it reports nothing, not an orphan), andSweepDescription's case split does not cover every overlap shape.Unverified:
Program.cs's parsing ofDOUJIN_MANAGER_INTEGRITY_SWEEP_SHARDS(reject non-integer, or outside 1..256) has no test — no test host runsProgram.cs, the pre-existing blind spot everyDOUJIN_MANAGER_*variable shares; the clamp the use case applies to whatever value arrives is tested, at both ends. Nothing was run in the real app against a live server: no Flutter app was launched, no backend was started, no sweep was driven by hand. Every claim above rests on automated tests and on reading the code.Follow-ups
docs/BACKLOG.md, sweep item: records the residual race — an upload whose write the slice's two reads fall inside is still anOrphanImageFile, and narrowing it further means an mtime grace period with a tunable of its own, worth designing alongside the quarantine item.docs/BACKLOG.md, repair-all item: its cross-reference now names the shipped sweep shape (GET /api/integrity/image-sweep,DOUJIN_MANAGER_INTEGRITY_SWEEP_SHARDS) as the pattern to copy instead of pointing at unbuilt work.🤖 Generated with Claude Code
Flutter Coverage
Total: 85.6% (10759 of 12574)
Summary
Summary
Coverage
DoujinManager.ApplicationCore - 96.7%
DoujinManager.Infrastructure - 97.6%
on
pshot
Series
E20E2ABA2099901FFAB4904C0513DF077BA6D6CA374__CoverImageHelper
E20E2ABA2099901FFAB4904C0513DF077BA6D6CA374__PrimaryTitleHelper
E20E2ABA2099901FFAB4904C0513DF077BA6D6CA374__RolledBackImageCleanup
E20E2ABA2099901FFAB4904C0513DF077BA6D6CA374__UploadChapterValidator
DoujinManager.RestAdapter - 99%
E5CB12CEAC5BBBEA65844E1C05F9ADA4DA38B848075C41304FC6A1ABF77__ValidationAttr
ibuteCache
DoujinManager.Server - 64.6%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my. A 47-file, +4061-line PR where the author mutation-probed their own work, confessed the one probe that survived, and wrote the residual race into the backlog with a design sketch for narrowing it~ ♡ I read the whole diff twice hunting for something to sharpen my knife on and instead kept finding comments that had already finished the argument I was starting. This is the kind of submission that makes a Flugel's wings flutter♪
Verdict: ✅ Looks good to me~
What I verified myself, because trust is earned:
dotnet test→ 876 passed, 0 failed (5 skips are the pre-existing root-user permission-theory skips inTempRootsTests/health checks — environmental, not code). Pristine-tree re-run of the whole integrity slice: 63/63 sweep+integrity+plan+enumeration, 4/4IntegrityRepairsTests.flutter analyze— No issues found;flutter test— 1337/1337, matching your verify.sh claim exactly.MissingImageFilesignature (6d00…0f, no file at 6d/…). The read-order pin is real, not decorative. ♡💡 Little ideas (non-blocking)~
<remarks>, andSweepDescription's incomplete case split are recorded nowhere in the repo. Under squash-merge the body survives in the commit message, but nobody greps commit messages six months from now —docs/BACKLOG.mdis where this codebase's residuals live, and the upload-inside-reads orphan already got its line. One short line for the trio would keep them findable. (For the record, I probed the two race-test<remarks>claims I could check — both accurate as written; I couldn't locate the misstated one from the diff alone, which is itself evidence of how small it is~)_foldSweepChecksfolds every unknown server kind into the singleunknownbucket — two future sweep kinds this client predates would merge their counts behind one expander. Today's server has exactly two sweep kinds, so this is theoretical; worth a sentence in the helper's doc if it ever matters.✅ What I liked~
ImageShards— you found a definition that lived in three places (ImageFile.Shard, bothGetFilePaths) and made it live in one, with the layout-agreement test (A_Saved_File_Is_Listed_By_The_Shard_The_Entity_Names) proving the collapse can't drift. That's an architectural improvement riding along a feature, my favorite kind♡FromSqlbecause the value converter leaves no LINQ shape for it, noORDER BYbecause the index walk is the order — and every one of those sentences pinned by a test (QueryPlanTests,The_Shard_Range_Reaches_Rows_In_Every_Corner_Of_The_Space,Missing_Findings_Arrive_In_Id_Order). Thesubstrscan you avoided would have been invisible from the outside. Fufu~ exquisite.IntegrityChecks.RunsInImageSweep— a falsifiable table enforced from both sides (report tests filter through it, sweep tests filter through it,Every_Check_Declares_Which_Run_Answers_Itwalks the enum). "0 findings from a run that did not look" being banned as a matter of contract is the kind of honesty I get possessive about~requestSeqgate mirrors the report sibling exactly.CommitsBetweenTheReads— a deterministic decorator instead of threads-and-sleeps for the interleaving tests, and the remarks state precisely what the order does and does not promise (the window between the reads, not the width of the write). The wire description carries the same honesty with "Re-run the sweep before acting on a finding."Path.Combine—?cursor=../../etcdies atTryParse, tested with six flavors of nonsense. ♪Automated review by Jibril · 2026-09-20
CI/CD: absent for head SHA
5a3a249(no coverage bot comment yet) · Local checks: backend 876/876 pass (5 root-skips) + integrity slice 63/63 + 4/4 pristine re-run; flutter analyze clean + 1337/1337; 2 independent mutation probes (image-swap red ×2 race tests, thumbnail-swap green ×20 — confession confirmed)