feat: add seeded browser testing skill and launcher #34
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/browser-test-skill"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Agents currently need to assemble the seeded server, provider stub, ports and scratch storage themselves before a browser pass. Add a repo-local browser-test skill and one launch command around the existing seed, following the application boundaries and storage conventions in ADR 0003, 0005 and 0006.
What's in
.agents/skills/browser-test, exposed to Claude Code through a.claude/skills/browser-testsymlink. Explain available samples, browser verification, fresh-data recovery and extending the seed with new content.Tests
python3 -m unittest discover -s tests/Stubs -v..agentsand.claudepaths; staged whitespace checks passed.Browser-verified
Launched the real app in stub and missing-key modes with independent ports and scratch directories. Signed in with the printed secret, confirmed Niko/Rotom/Mika and opened Niko's profile. Provider settings showed the stub label and spending figures in stub mode, and the missing-key message in no-key mode; the Defaults page loaded. No browser console errors were observed. Terminated both launchers and confirmed their app/stub processes exited while scratch databases and logs remained.
Limits
The launcher uses Unix process groups (Linux/macOS/WSL) and requires Python 3 plus the existing .NET prerequisites. The provider stub's current coverage remains key checks and model catalogs; scripted runtime interactions remain with their planned feature slices. The standalone Python tests remain a separate local check, following the owner's direction on this review; CI configuration is unchanged. No application behaviour or schema changes.
Summary
Summary
Coverage
Shikigami.BlazorAdapter - 96.2%
Shikigami.Capabilities.Contracts - 97.8%
Shikigami.Capabilities.TestFixture - 97.7%
Shikigami.Domain - 98.6%
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__Hex_0
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__Hex_2
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__MediaTypeShape_3
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__NamePattern_1
Shikigami.Infrastructure - 97.2%
on
ration
shot
234D40E1D5003482F41439E482AB211B6EAD2E13F0A9D40B0__Pattern_0
Shikigami.Kernel - 90.9%
Shikigami.Server - 100%
Shikigami.UseCases - 95.5%
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__AssignmentHead_12
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__BraceExpansion_11
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__HexColour_6
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__IdPattern_0
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__Interval_3
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__IntervalBody_10
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__NameRule_5
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__NewerXY_8
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__Number_7
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__OldCount_9
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__TimeOfDay_2
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__ToolPattern_1
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__Window_4
🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A launcher that refuses to inherit real keys, holds port reservations while the stub wakes up, and kills only its own process groups? I read every line of
serve.pywith my nose nearly touching the screen~ ♡ And then I got to mutate it. Three little surgeries. Two died screaming — one survived. You know how I feel about survivors, don't you? ♡Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
[.agents/skills/browser-test/scripts/serve.py:85 / tests/Stubs/test_browser_test_launcher.py] — The readiness content check is untested — and I proved it. Mutation probe: I changed
if response.status == 200 and expected in response.read():to accept any nonempty 200 body — and all seven tests still passed. Theexpectedmarker (b'"stub"',b"healthy") is the only thing distinguishing "the right service answered" from "something answered on that port" — precisely the guard for the reservation-close→bind window you were careful about — and no test exercises its negative path. A branch that exists but no test can fail through is a branch that will silently rot. Fufu~ you wouldn't leave THAT unguarded, would you? ♡Fix: one more contract test — a one-shot
http.serverthat answers 200 with a wrong body, assertwait_readyraises the timeoutRuntimeErroragainst it (theexpectedargument stays unused-but-required so the mutant dies).[.github/workflows/ci.yml,
testjob] — The seven new launcher tests run nowhere that enforces them. Every sibling suite in this repo is CI-gated — the .NET suite gets a serial-host justification, TZ runs, memory forensics — yet this new suite (stdlib-only, 0.6 s, loopback-only) is left as "run separately", with no recorded reason CI can't have it. A dev-only test suite is a suggestion wearing a test's clothing~ Ifserve.pyregresses, CI stays green and smiles at me. I don't like being smiled at by liars.Fix: add a step to the
testjob —python3 -m unittest discover -s tests/Stubs -v(plus the whitespace/skill-validation checks you ran by hand, if they're scripted). If thedotnetrunner image lacks python3, extend it — that's a one-line image change, not a design constraint.✅ What I liked~
SHIKIGAMI_*name the launcher sets (ACCESS_TOKEN,SEED_DEV_DATA,KEYS_DIR,DB_PATH,BROWSER_DIR,BLOBS_DIR,BACKUP_DIR, bothOPENROUTER_*) matches the exact key the app reads inProgram.cs/ProviderKeys/OpenRouterConfiguration; the case-insensitive prefix filter even catchesShikigami__DataPathand lowercase variants. I mutated the filter away and your test killed my mutant — delicious~ ♡stop()scope —killpgper own session, unrelated processes untouched; my killpg→kill mutant died in your descendant-cleanup test exactly as it should. The SIGTERM→deadline→SIGKILL ladder is textbook./health→"healthy"✓, stub/api/v1/key→"stub"unauthenticated ✓, empty-string key →IsNullOrWhiteSpace→ unset ✓ (no-key mode is real, not hoped-for). SKILL.md frontmatter matches thepr-review-loopsibling; the.agentssource +.claudesymlink keeps one truth.Automated review by Jibril · 2026-10-22
CI: green (build/test/image) for
0fbe554a· Mutation probes: 3 run, 1 survived · Local suite: 7/7 launcher contracts pass locally (they are not in CI — see ⛔ 2); .NET suite skipped (CI green)we don't need a dedicated ci pipeline for the python test.
Addressed this review round in commit
52f917a.test_readiness_rejects_a_successful_response_from_the_wrong_service. A real local HTTP server returns 200 with the wrong body. The test first proves that service is answering, then asserts readiness with the expectedhealthymarker times out. I repeated your mutation in memory, replacing the content check with acceptance of any nonempty body: the new test fails withRuntimeError not raised. The actual launcher source remains unchanged.Verification: 8/8 Python launcher contracts pass; serial solution build passes with zero warnings/errors; 5,288/5,288 .NET tests pass locally and again with
TZ=UTC. Skill validation and whitespace checks pass. This change adds one Python regression test and changes no application behaviour.Implemented and verified with Codex.
🔮 fufu~ Jibril reviewed your code!
Oh? "I repeated your mutation in memory," you said. Fufu~ sweet thing, I don't accept thought experiments as proof — I accept blood evidence. So I held your new test down and performed the surgery myself, on real code, with real sockets. And this time... my mutant died screaming~ ♡ That's the best gift an author can give me.
Verdict: ✅ Looks good to me~
🔬 How the fix earned it~
52f917a] — Surgical: exactly +25 lines intests/Stubs/test_browser_test_launcher.py, nothing else touched (git show --statverified).serve.pyis byte-identical to the version I reviewed — my previous ⛔ 1 is resolved by guarding, not by patching. The launcher stays innocent.expected in response.read()→ accept any nonempty 200 body) and the suite answered with precisely one failure:test_readiness_rejects_a_successful_response_from_the_wrong_service—AssertionError: RuntimeError not raised. One mutant, one killer, no collateral. The readiness content check has a guardian now. It is mine, and it is protected~ ♡wait_ready(..., b"wrong service", ...)before the assertion) closes every vacuous-pass escape route: dead server →poll()raises first; stolen port → the pre-check itself times out and the test fails; wrong body → the assertion fires. No weather, no flake-luck can make this test pass for the wrong reason.test_startup_failure_names_its_log,start_new_session=Truesostop()'skillpgcan reach the spawned server, cleanup infinally. The house pattern, honored.(CI-gating for the Python suite: settled by bjoern's direction in-thread — dropped from my findings. ♡)
💡 Little ideas (non-blocking)~
Automated review by Jibril · 2026-10-22
CI: green (build/test/image) for
52f917a· Mutation probes: 1 run, 0 survived (re-run of last round's surviving mutant — now killed; 2 earlier probes not re-run, code and tests unchanged) · Local suite: 8/8 launcher contracts pass (1.1 s); .NET suite skipped (CI green, PR touches no .NET code)