feat(tts): scene lines speak — per-line generation, auto-regen, playback #148
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/tts-scene-lines"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The scene slice of stories/audio-tts.md — on #144's providers and #145's Voice tab. After this, only player playback remains of the audio arc.
What's in here
Domain —
SceneStep.VoiceFileName(nullable, its own migration), with the staleness rule in the domain:Applyclears it whenever the spoken material changes — text, speaker, or kind — because the clip spoke the old words.SetVoicerecords generation's result through an unjournaled store path (the sprite-reference rule).The generator (
SceneStepAudioGenerator, kindscene-step-audio) — the sibling-generator shape, with one deliberate distinction: silence is not failure. Narration, no speaker, no text, TTS off, and speaker-without-a-voice all resolve toOk(false)in the precondition scope, before any job exists — the queue only ever carries lines that can actually speak. The job re-resolves the step at run time, so the newest text wins; clone handles persist to the character exactly as in previews (sharedBuildConfig).Auto-regeneration — the effect compares the pre-save DTO: only a real change to the spoken material dispatches generation. Re-saving unchanged text or editing the stage burns nothing (pinned by test).
The row UI — a speaker-state button per dialogue line: generating (spinner) / ready (plays via the new gated
/projects/{slug}/scene-steps/{id}/audioendpoint throughkaguraAudio) / failed (reason in the tooltip, click retries) / silent (click voices it), plus a regenerate button once a clip exists. Narration rows carry nothing.Pocket-TTS clip caching (the deferral flagged in #145) — an optional App URL field on the pocket descriptor plus an anonymous capability endpoint
/tts-clips/{characterId}/{token}/{fileName}: the token is per-upload and unguessable (rotation doubles as sidecar cache invalidation — a new clip is a new URL is a fresh embedding), and a mismatched token or file name reads as absent. Unset, clips fall back to per-request upload. Integration finding baked into the contract: the sidecar only accepts voice URLs ending in a recognizable audio extension (live-verified: 200 with.wav, 500 without) — hence the file name as the URL's last segment.Folded in from the parallel review thread (already in this tree, tested): uploads clear a lingering preset
VoiceId; the preset combobox gains "(none — use the cloned clip)"; OGG/MIME aliasing + extension fallback on upload; per-extension serving content types.Tests (1,603 green)
SceneStepAudioGenerationTests, sibling-generator shape): clip on the real filesystem under_audio/<scene-id>/, file name on the step, served byOpenSceneStepAudiowith the right content type; the three silence rules with nothing on the wire; TTS-off silent-not-failed; provider 402 rides the job error; text edit clears the reference. Plus the capability endpoint (right token+name serves, wrong anything reads absent) and the pocket capability-URL shape.SceneLineAudioTests): all four row states; narration carries no buttons; text edit regenerates / unchanged re-save does not; manual regenerate; failure tooltip + retry clearing.Live verification (Playwright + the real umbrel sidecar)
The full flagship chain, on real infrastructure:
(The first live attempt failed with the sidecar's 500 — which is how the URL-extension requirement was discovered; the failure state rendered exactly as designed, reason in the tooltip.)
Notes for review
localhostonly) means the capability URL needsASPNETCORE_URLS=http://0.0.0.0:5295locally; deployed on umbrel both containers see each other — the App URL would be the LAN address of the Kagura container.SceneStepDtogainedVoiceFileName; the transient generating/failed state lives in a per-step dictionary onSceneStepsState, returning to the durable truth when an attempt ends.🤖 Generated with Claude Code
The audio-tts story's scene slice (ADR 0033). Every dialogue line can carry a generated voice clip; editing the spoken material — text, speaker, kind — invalidates it in the domain (Apply clears VoiceFileName: the clip spoke the old words) and the editor auto-regenerates through the queue (kind scene-step-audio). The regeneration discipline lives in the effect: only a real change to the spoken material burns a generation — re-saves and stage edits do not. Each script row carries a speaker-state button: generating (spinner), ready (plays via the gated audio endpoint + kaguraAudio), failed (the reason in the tooltip; click retries), silent (click voices it) — plus a regenerate button once a clip exists. Narration rows carry nothing. Silence is not failure: the generator resolves narration/no-speaker/no-text/TTS-off/no-voice to Ok(false) before any job exists. Clips land at assets/_audio/<scene-id>/<step-id>.<ext> (ADR 0005 as amended), replaced per step; served by /projects/{slug}/scene-steps/{id}/audio (gated, cache-busted). The step's VoiceFileName is unjournaled (the sprite-ref rule). Pocket-TTS clip caching (the PR #145 deferral): an optional "App URL" field + an anonymous capability endpoint /tts-clips/{charId}/{token}/{fileName} — the token is per-upload (rotation doubles as sidecar cache invalidation), the file name is the last segment because the sidecar only accepts voice URLs ending in a recognizable audio extension (found live: 200 with .wav, 500 without). Unset falls back to per-request clip upload. Also folded in from a parallel review thread: uploads clear a lingering preset VoiceId; the preset combobox gains "(none — use the cloned clip)"; OGG/MIME aliasing + extension fallback on clip upload; per-extension serving content types. Verified live end-to-end against the umbrel sidecar: generate on a seeded line -> capability URL fetched by the sidecar (token-authenticated, anonymous) -> clone synthesized -> play (endpoint 200) -> text edit -> auto-regen through the cached embedding -> ready again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>Summary
Summary
Coverage
Kagura.BlazorAdapter - 88.7%
Kagura.Domain - 95.4%
D04ADFED3A21D401C2764A1D17367E35BEB556CBB3B4B0B74__NonSlugChars_0
Kagura.Infrastructure - 97.5%
n
on
C689CA5BBE6D1849B2CA79581BBB07D40636D85790C0E51A9__ArtifactTimestampRegex_2
C689CA5BBE6D1849B2CA79581BBB07D40636D85790C0E51A9__ProjectRoute_0
C689CA5BBE6D1849B2CA79581BBB07D40636D85790C0E51A9__TabQuery_1
Kagura.Kernel - 90%
Kagura.Server - 82.8%
Kagura.UI - 94.8%
Kagura.UseCases - 96.2%
🔮 fufu~ Jibril reviewed your code!
Oh? Oh my~ This is the scene-line TTS story, the big one, isn't it? Per-line generation, auto-regen, the capability-URL caching for Pocket-TTS, silence-is-not-failure... fufu~ Jibril has been waiting for this! Every piece carefully threaded through the activity queue, the staleness rule living in the domain where it belongs, the
BuildConfigextraction killing the duplication between the two generators. Let me look very, very closely~ ♡Verdict: ✅ Looks good to me~
I read the full diff, then the full contents of
SceneStepAudioGenerator,VoicePreviewGenerator,EfSceneStepStore,OpenSceneStepAudio,OpenVoiceClipByToken,Character.cs(theMergeVoicesandSetVoiceClippaths),PocketTtsProvider,Program.cs, and the test files. I checked every concern against the siblingVoicePreviewGeneratorandCharacterImageGeneratorpatterns. Here's what I found:✅ What I liked~
The sibling-generator shape is faithful to a fault.
SceneStepAudioGeneratormirrorsVoicePreviewGeneratorexactly — precondition scope,Activity<Unit>with a fresh-scopeRunAsync, the same error-surfacing pattern. The one deliberate distinction (silence-is-not-failure returningOk(false)before enqueueing) is called out clearly in the doc comment and tested three ways. Wonderful~BuildConfigextraction — the DRY elimination between the preview generator and the scene-step generator is exactly right.VoicePreviewGeneratornow callsSceneStepAudioGenerator.BuildConfigand the two can never drift. TheClipTokenthreaded throughVoiceReferenceClip(with a default of""for backward compatibility) is clean. ♪The staleness rule in the domain —
SceneStep.ApplyclearsVoiceFileNamewhenever text, speaker, or kind changes. The testEditing_the_text_clears_the_stale_clip_referencepins it. This is the right place for it — the clip spoke the old words. Putting it in the domain means no path can bypass it.SuppressJournalinginSetVoiceAsync— matches theEfCharacterStore.SetVoiceHandleAsyncsibling pattern exactly (try/finally reset). A generated clip reference is a derived artifact, not authored content. Correct.MergeVoicespreservesClipToken— I checked:ClipToken = existing?.ClipToken ?? ""is in the merge path alongsideClipFileName, so a profile update from the character editor won't blow away the token. The image-file-name rule is applied consistently.ClipTokenonCharacterVoiceneeds no migration —VoiceConfigsis stored as JSON (AsJsonList()), so the new field just appears. TheSceneStepVoiceFileNamemigration adds only the nullable column. Correct.Path traversal on
/tts-clips/{characterId}/{token}/{fileName}— thefileNameis compared with strict equality againstvoice.ClipFileName(which is sanitized at upload asproviderId + extension). A malicious../path fails the equality check and returnsnull. Safe. ✓Capability URL security — token is
Guid.CreateVersion7().ToString("N")(unguessable), rotates on every upload (invalidating the sidecar cache), and a mismatched token or file name reads as absent. TheOpenVoiceClipByTokentest covers right-token+serves, wrong-token+null, empty-token+null, mismatched-name+null. Good.Auto-regen comparison — the effect compares the pre-save DTO (
before) against the post-save result, only dispatching generation when the spoken material actually changed.Resaving_unchanged_material_burns_no_generationpins this. Thebeforeis captured before the save patches the Fluxor state list — correct ordering.Coverage — CI's coverage comment shows 1,603 tests green, 95.7% overall.
SceneStepAudioGeneratorat 92.2% / 72.2% branch,OpenVoiceClipByTokenat 92.3% / 91.6% branch. The new code paths are genuinely exercised, not just compiled.💡 Little ideas (non-blocking)~
OpenSceneStepAudioat 77.7% / 50% branch — the uncovered branch is the slug/project mismatch guard (project.Id != step.ProjectId). It's the same defensive pattern as every sibling image endpoint, so it's not a concern — but if you wanted to pin it, a test passing a wrong slug with the right stepId would do it in one line. Entirely optional~ ♡IProjectStore.ListAsyncscan for slug — bothSceneStepAudioGenerator.RunAsyncandOpenVoiceClipByTokendoprojects.FirstOrDefault(p => p.Id == ...)over a list. The comments say "single-user scale makes the list scan a fine lookup" and that's true today. If aFindByIdAsyncever lands onIProjectStore, both sites would benefit — but this is pre-existing in the codebase's style, not new debt.Automated review by Jibril · 2026-07-15
CI/CD: passed (forgejo-actions coverage comment, 1,603 tests green, 95.7% line coverage for head SHA
cde51df) · Local checks: skipped (CI green)