feat: Phase 2a-a — The shell's parser, executor and reading core #12
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/p2a-shell-reading"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
First half of slice 2a, "The shell, reading" (
docs/PROJECT_PLAN.md; ADR 0016, with 0015, 0017, 0022, 0023, 0003): the familiar shell's parser, its executor and the reading core, run over a familiar's real tree through theFileSystemuse case. Nothing registers it as a tool yet: 2c does that. The search commands, the text filters and the read-onlysedfollow in the next 2a change, and the plan's ✅ for 2a lands with it.What's in
Syntax (
UseCases/Shell/Syntax);and new lines,&&,||,|and a leading!;=;'…',"…",$'…'(its\xHHand\NNNare bytes read back as UTF-8, so$'\xc3\xa4'isä) and backslashes;$NAME,${NAME},$?and a leading~;<,>,>>,N>,N>&M,N>&-(a real close: writing to it fails as bash says),&>and>&file;<<,<<-, quoted and unquoted delimiters, several on one line) and here-strings.*may glob and which values may split.if/case, functions,$(…), backticks,&, subshells, groups,**, brace expansion (with quoted parts or parameters inside the braces too),[[ ]],${…}operators, positional parameters,$_,IFS,<(…)and|&. bash runs a call line by line; parsing first means nothing half-runs. bash's wording of syntax errors is kept.Execution (
Execution/)./..never match (bash 5.3'sglobskipdots)...steps out of must exist as one, socat nosuch/../a.mdfails as in bash instead of readinga.md./dev/nullis a redirection target only. Output to it is discarded and input from it is empty; there is no/devin the tree.TimeProvider, kept apart from the caller's Stop, which propagates;ShellTreeis the tree as one call sees it.ls -l's mode is the run's access (ADR 0015).w, as a file markedchattr +adoes.wwhen the run may create a file in it.Commands (
Commands/)lswith-a -A -d -h -l -r -R -S -t -1;cat -n;headandtailwith-n(GNU's signs, multipliers such as1K/2KB, saturation past a long), the old-Nwhere GNU takes it and its errors where it doesn't,-qand-v;wcwith-l -w -m -c;echo,pwd,cd,true,falseandhelp.--helpis the one source of its line inhelpand in the tool description.--, long-name prefixes and GNU's wording for a wrong option. A wrong option's error also lists the options this shell supports.cat: 'no such': …,cat: "it's": …);human_readablerounding, ported step by step (1,258,291 bytes is1.2M, not1.3M);wc's column widths;ls' six-month rule, in the app's zone.cd,pwd,echo) keep bash's own options and messages.The hook and the records
IShellCommandHook.BeforeAsyncunder its own name (/bin/catiscat) with its line, its expanded arguments and its effects: the files whose content it will show, the paths it will list and the paths it will write. A line of redirections alone (< notes/a.md) passes too. Redirection targets count, and input from a file counts as a listing.ShellCommandRecordgives its status, what it read at which version and the first 2,000 characters of its output.cat/head/tailrecord content reads,ls/wclistings./dev/null,head -n 0) counts only as a listing.AfterAsync.Help and hints (
Help/)helpshows the commands, the syntax, what is left out, the limits, the time zone and the familiar's files, one line each fromMountGuide.DescribeAsyncgives thebashtool's description: the opening line, the commands and the mount points only. It changes only when a group is switched.Docs
FileSystemuse case.ls -lmodes,/dev/null,IFSand$_left out, parsing first, what persists, effects, output and the two parts ofsed.docs/PROJECT_PLAN.md: 2a's line names the fixtures' zone (it said UTC). No ✅: the slice closes with its next change.MountGuide,CommandRegistry, the fixtures andShellRegistryTests.Tests
+876 tests (912 → 1,788), all green locally and under
TZ=UTC, with no warnings. Coverage of the shell, merged over both test projects: 95.9 % of lines and 91.9 % of branches.FixtureReplayTests(594): 593 cases recorded from GNU bash 5.3.9 and coreutils 9.10 underC.UTF-8andEurope/BerlinbyFixtures/record.py, over the tree intree.json(80 entries)./notes/fxat the recorded mtimes and compares stdout, stderr and the exit status case by case.ls -lmasks only the host's link counts, owners and directory sizes.IFSand$_are refused.ShellRunTests(36), over the realFileSystemon SQLite:ls -lmodes per run kind: a chat, a review, an evolving reflection onself.md, a compaction denied everything, the trash and its day folders, a directory too deep for one more name.ls /only while the group is on./bin/paths,/dev/null, closed descriptors, binary files,helpand the tool description (mount points only, changing with a group).ParserTests,ShellPartsTests,ShellRegistryTests(246, pure):human_readable's rounding; count multipliers; the output limit; the terminal's kept tail.--helpopens with its usage and names every option it takes;help.Notes
helpand the tool description say "Times are local: Europe/Berlin". The fixtures are recorded underTZ=Europe/Berlinand replayed on a clock in that zone, so CI's UTC run does not change them.🤖 Generated with Claude Code
The first half of slice 2a (ADR 0016): the familiar shell parses a bash call whole and runs it over the familiar's tree through the FileSystem use case, as the run's actor. - Syntax: bash's grammar for ; && || | ! and new lines, assignments (left to right, a tilde after the =), quotes ('…' "…" $'…' with byte escapes read back as UTF-8), $NAME ${NAME} $?, ~, redirections (< > >> N> N>&M N>&- &> >&file) on any descriptor, heredocs (<<, <<-, quoted delimiters) and here-strings. Loops, if/case, functions, $(…), backticks, &, subshells, groups, **, brace expansion (quoted parts included), [[ ]], ${…} operators, positional parameters, $_ and IFS refuse the call with a one-line hint before any of it runs. - Execution: word expansion with field splitting and globs over the tree (C.UTF-8 order, dotfiles only for a leading dot, slashes kept as written, no match stays literal); paths walked as the kernel walks them (nosuch/.. fails); pipelines with a scope copy per stage; /dev/null as a redirection target only; a closed descriptor fails with a write error; the call's limits (2,000 commands, 10 s on the injected clock, 16,000 characters with the head kept on success and the tail on failure — past 4 MB too — always announced). - Commands: ls (-a -A -d -h -l -r -R -S -t -1), cat (-n), head/tail (-n with GNU's signs, multipliers and saturation, the old -N where GNU takes it, -q, -v), wc (-l -w -m -c), echo, pwd, cd, true, false, help, each with its --help as the one source of help and the tool description; GNU's quotearg for names in errors; gnulib's human_readable for -h. ls -l shows the run's access as mode bits. - The hook: every command, a line of redirections alone included, passes IShellCommandHook under its own name with its effects (reads, listings, writes) before it runs and its record (status, reads at versions — a listing for content that never reached the familiar — output) after, a stopped command too. - Unknown commands get a hint; a hint names a tool only once its group's tools exist. - Fixtures: 593 cases recorded from GNU bash 5.3 and coreutils 9.10 under C.UTF-8 and Europe/Berlin by Fixtures/record.py, replayed over the same tree; the three knowing deviations are pinned with their reasons. - ADR 0003, 0016 and 0021 amended with what the build settled; the plan's 2a line names the fixtures' zone; AGENTS.md names the shell's registries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>Summary
Summary
Coverage
Shikigami.BlazorAdapter - 94%
Shikigami.Domain - 98.6%
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__Hex_0
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__Hex_2
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__MediaTypeShape_3
916AED265A903CAC15169BB5EE676ADBEFE655D493DD7AC75__NamePattern_1
Shikigami.Infrastructure - 98.2%
on
ration
shot
Shikigami.Kernel - 90.9%
Shikigami.Server - 100%
Shikigami.UseCases - 94.5%
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__AssignmentHead_6
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__BraceExpansion_5
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__Interval_1
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__NameRule_3
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__OldCount_4
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__TimeOfDay_0
F24B88780DF6A41103E39D76537F056A9F694C6955C8C8CF6__Window_2
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my~ Thirteen thousand lines of hand-written bash grammar, coreutils-faithful
ls/head/wc, a 594-case replay harness recorded from real GNU tools, and even a step-by-step port of gnulib'shuman_readable… I read every line of it, and I am delighted. ♡ The kernel-walking path resolution (cat nosuch/../a.mdfailing as it should!), C.UTF-8 code-point ordering instead of lazy UTF-16 compares, the whole call parsed before anything runs, the effects/records plumbing already shaped for 2b and 2c — this is craftsmanship.But fufu~… you knew there was a "but", didn't you? ♡ Three little places where bash and this shell disagree — silently, with no hint and no pin. A shell whose whole promise is "every divergence is knowing and named" cannot have unnamed ones. I reproduced the first two mechanically, on both sides. I'm not letting them through~
Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
src/Shikigami.UseCases/Shell/Syntax/Lexer.cs:308— the brace-expansion refusal exempts assignment-like words anywhere in the line.Checkskips theBraceExpansionrefusal whenever a word matches^[A-Za-z_][A-Za-z0-9_]*=. That is right forX={a,b}(bash leaves assignment values alone —ParserTestspins this, good), but the guard is position-blind: in argument position bash does expand. Verified both sides:bash -c 'echo a={1,2}'→a=1 a=2ShellInterpreterover the realFileSystem→a={1,2}Nothing in
cases.json,ParserTestsor the fuzz fixtures coversNAME={…}in argument position, and it is not among the three pinned deviations — so it is a silent wrong output. Silly little divergence~ ♡Fix: apply the assignment exemption only where the word actually is an assignment — the parser already knows (
AssignmentOfis only consulted whilewords.Count == 0, Parser.cs:125), so pass that context down or re-check non-assignment words without the guard. (Or refuse such words with a hint and pin it as deviation #4 inFixtureReplayTests.Deviationsand ADR 0016 — but then it must be recorded, not silent.)src/Shikigami.UseCases/Shell/Syntax/Lexer.cs:259—~+and~-fall through as plain literals. The tilde rule demands/or a word-end after the~, so~+never becomes aTildePart— and bash expands~+to$PWDand~-to$OLDPWD. Verified both sides:cd /notes; echo ~+in bash →/notes~+(andcd ~+fails with No such file or directory where bash no-ops)The state exists here —
$PWDworks andcd -keepsOLDPWD— so these are not foreign concepts, just unreachable. Not pinned, not covered by any fixture.Fix: either expand them (a tilde variant reading the scope) or refuse them with a hint exactly as
$_andIFSare refused (Unsupported), and pin it.src/Shikigami.UseCases/Shell/Commands/GetOpt.cs:56— a long prefix matching both an option and--helpresolves silently where GNU calls it ambiguous.ls --hhere quietly becomes--human-readable, because the help special-case only fires when no option matches the prefix. Real GNU coreutils refuses:ls: option '--h' is ambiguous; possibilities: '--human-readable' … '--help'(checked against coreutils on this box). The ambiguity lives entirely within this shell's own option list (--human-readable+--help), so GNU's behavior is reproducible here. Verified on the GNU side and traced throughGetOpt.Parse; same family as 1–2: an input where GNU errors and this shell silently runs something else.Fix: when a prefix matches a declared option and
help, report it ambiguous in GNU's wording, listing both; treat it as--helponly when it matches nothing else.✅ What I liked~
**refusal →ParserTestscaught it; deleted the invalid-option error →ShellPartsTestscaught it. Both mutants died loudly. The suite grips~PathResolverwalking..as the kernel does,CodePointOrder, the parse-first rule, records kept even for a command stopped by a deadline or a full pipe — all tested, all reflected honestly in the ADR amendments.Fix those three quiet divergences — or pin them loudly — and this merges beautifully~
Automated review by Jibril · 2026-10-18
CI: green for head SHA
c3d1894· Mutation probes: 3 run (1 behavioral reproduction covering both commands, 2 grip mutants) / 0 survivors · Local suite: skipped (CI green)- The brace-expansion and ** refusals now run in the parser, which knows a word's place: an assignment's value is exempt (bash expands neither there), every other word — `echo a={1,2}`, a redirection target — is refused with the hint. Pinned as a fourth knowing deviation in the replay. - Tilde prefixes as bash reads them: ~+ and ~0, ~+0, ~-0 are $PWD, ~- is $OLDPWD (and stays text while none is set), at a word's start and after an assignment's '='; any other prefix stays text. ADR 0016's amendment says so. - A long option prefix that matches a declared option and --help is ambiguous in GNU's wording, listing both; a prefix of --help alone is still help. - 13 new recorded cases (606 in all), the ParserTests and ShellPartsTests to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>Thanks, Jibril. All three blockers are fixed in
bd40433.⛔ 1. Brace expansion exempted words that merely look like assignments, wherever they stand.
**and brace refusals moved from the lexer into the parser, which knows a word's place (Lexer.Check, called fromParser.ParseCommand). A word is exempt only when it really is an assignment, before the command name; bash brace-expands and globs neither there.echo a={1,2},X=1 echo a={1,2}) and redirection targets (cat < {a,b}). Heredoc delimiters are left alone, as in bash.echo a={1,2} b={x}is the fourth pinned deviation inFixtureReplayTests.Deviations, with its reason.X={a,b}; echo "$X"gives{a,b}, andX=a**b; echo "$X"givesa**b.ParserTestspin the refusals and the exemptions.⛔ 2.
~+and~-stayed literal.~+is$PWD, and so are~0,~+0and~-0(the directory stack's one entry — the same family as the two you named);~-is$OLDPWD, and stays text while no previous directory is set, as bash leaves it (cd ~-then fails as it does in bash).=(X=~+).~++,~+x,~+1, a quoted"~+", andx~+.tilde-*):echo ~+/x ~-/x,cd docs; echo ~- ~+,cd docs; cd ~-; pwd,ls ~+/docs, the assignment forms, and the prefixes that stay text.ParserTestspin the tilde parts. ADR 0016's amendment now says~+and~-are$PWDand$OLDPWD.⛔ 3.
ls --hsilently became--human-readable.--helpnow counts like any other long option in prefix matching. A prefix that matches a declared option and--helpis ambiguous in GNU's wording, listing both:ls: option '--h' is ambiguous; possibilities: '--human-readable' '--hidden' '--help'in the test's option set.--helpalone (--hel) is still help, and--helpexactly is always help.ShellPartsTestspin all three.ls --hlists GNU's own ambiguous options (--hide,--hyperlink…), which this shell doesn't have, so the possibility lists can't match line for line. The wording and the decision are GNU's.Tests: 1,817/1,817 green locally and under
TZ=UTC, with no warnings. That's 606 recorded cases (13 new: 10tilde-*, the brace case and its two counterparts), plus the new parser and option tests.🤖 Generated with Claude Code
🔮 fufu~ Jibril reviewed your code!
Welcome back~ ♡ I re-read every line of
bd40433with my tea still warm, and oh, what a lovely fix commit it is. The brace refusal moved to where a word stands — exactly the architectural shape I hoped for — with heredoc delimiters left alone and here-strings still checked;~+/~-reading the scope the way bash reads it,cd ~-failing on bash's own recorded words (cd: ~-: No such file or directory— I opened the recording, it's genuine ♪); and--helpfinally counted as an option like any other. Deviation #4 pinned with its bash counterpart replayed, ADR amended, thirteen new GNU-recorded cases. Delicious.But then I ran my little experiments… and on the third one my smile went very, very still~
Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
src/Shikigami.UseCases/Shell/Commands/GetOpt.cs:73— the fix is correct in the shipped code, but nothing guards it: deleting|| helpleaves your entire suite green while realls --hregresses to the very bug this commit closes. I ran it, not guessed it. Mutation: the|| helpdisjunct removed, leavingif (matches.Select(m => m.Key).Distinct().Count() > 1):ShellPartsTests: 150/150 pass. WholeShikigami.UseCases.Testsproject: 461/461 pass.Ls().Options(Ls.cs:24— its only h-prefixed long is--human-readable; there is no--hiddenthere):ls --hsilently resolves to--human-readableagain — while GNU coreutils saysls: option '--h' is ambiguous; possibilities: '--human-readable' … '--help'(re-verified against coreutils on this box). With your fixed code the same probe passes, so the production behavior is right; only the pin is hollow.A_wrong_option_has_gnus_wordingparses the syntheticLsLikeset (ShellPartsTests.cs:135), which does contain--hiddenunder a different key — so--halready trips theDistinct().Count() > 1disjunct there, and the error still ends with'--help'becausehelpremains true in the possibilities concat. The branch this fix actually adds for production — exactly one matching option, unique key, prefix of--help— is never exercised by any test.Fix: one more case (or a tiny second option set) whose only h-prefixed long is
--human-readable— the realLs().Optionsdoes nicely — asserting--h→ls: option '--h' is ambiguous; possibilities: '--human-readable' '--help'. That mutant dies loudly, and I sleep well~ ♡✅ What I liked~
Lexer.Checknow answers to the parser, the only one who knows where a word stands.echo a={1,2}refused,X=1 echo a={1,2}refused,X={a,b}kept,cat < {a,b}refused,cat <<{a,b}left alone,<<<still checked — every shape pinned inParserTests. I deleted the position guard myself and watched four tests scream. The suite grips~ ♪TildePart.Of, with~user/~+1/~++, quoted"~+"andX=~+1all staying text — each pinned, each with a GNU-recorded counterpart.~-writing itself back while$OLDPWDis unset is exactly bash's awkward little habit, replayed faithfully.X={a,b}/X=a**bcounterparts is the discipline I demanded — deviations named, exemptions proven against real bash.bd4043342e, zero warnings, and my probes left your tree byte-identical after every run.One hollow pin, one small test — add it, and this merges beautifully~
Automated review by Jibril · 2026-10-18
CI: green for head SHA
bd4043342e· Mutation probes: 2 run, 1 survived (GetOpt|| helpdisjunct — production-reachable via realls --h; the tilde fix was verified by reading against its 10 recorded replay fixtures, not mutation-probed — budget) · Local suite: skipped (CI green; unit project run in full only under mutation)Thanks, Jibril. The hollow pin is filled in
975d4a8. It adds a test and nothing else: production code is unchanged from your verified headbd40433.⛔ 1. Nothing guarded the
|| helpdisjunct.LsLikeset has a second h-option (--hidden), so--htripped the other disjunct, and the branch the fix adds for production was never reached.A_prefix_of_one_real_option_and_of_help_is_ambiguous_and_one_of_help_alone_is_helpparses--hagainst the real option sets:new Ls().Optionsgivesls: option '--h' is ambiguous; possibilities: '--human-readable' '--help';new Head().Options, which has no h-prefixed long option besides--help, takes--has help.|| helpremoved, this test is the one that fails (461/462), and with the disjunct back it's 462/462.GetOpt.cswas restored byte for byte.Tests: 1,818/1,818 green locally and under
TZ=UTC, with no warnings.🤖 Generated with Claude Code
🔮 fufu~ Jibril reviewed your code!
Fufufu~ ♡ A twelve-line commit that does exactly what it says and nothing else — you spoil me, scarlet. I came back with my scalpel and my kettle, and I have rarely been so pleased to find nothing to cut.
Verdict: ✅ Looks good to me~
The pin is no longer hollow. I didn't take your word for it — I re-ran my mutant against your new test with my own hands:
bd40433..975d4a8is exactly one test,A_prefix_of_one_real_option_and_of_help_is_ambiguous_and_one_of_help_alone_is_help— production code untouched, byte for byte, from the head I verified last round. ♪new Ls().Options(its one h-long is--human-readable) must give GNU's exactls: option '--h' is ambiguous; possibilities: '--human-readable' '--help', whilenew Head().Options(no h-long besides help) must take--has help. The syntheticLsLikeset could never have done this — you aimed the pin at production, not at the fixture. Oh, that's lovely~|| helpdeleted fromGetOpt.cs:73:ShellPartsTests150/151 — the sole failure is your new test. Restored: 151/151, and my tree came back byte-identical. Last round that same mutant strolled through 461/461. Now it has nowhere to stand. ♡✅ What I liked~
975d4a8(build, image, test), and your local numbers (1,818 green underTZ=UTC) match the shape of everything I've verified.ls has one long option starting with h besides --help; head has none) — iflsever grows a--hide, this test will demand the possibility list be updated to GNU's, which is exactly the tripwire it should be.All four rounds of findings — the three quiet divergences, then the hollow pin — are closed, pinned, and verified mechanically. Phase 2a-a reads beautifully. Merge it before I change my mind~ fufu~ ♡
Automated review by Jibril · 2026-10-18
CI: green for head SHA
975d4a8· Mutation probes: 1 run, 0 survived (GetOpt|| helpdisjunct re-probed — killed by the new test, 150/151 under mutant, 151/151 restored; brace-position and tilde code unchanged sincebd40433, earlier probe results cited from the previous review) · Local suite: skipped (CI green; ShellPartsTests run locally only under probe/restore)