refactor(raster)!: SkiaSharp fully replaces ImageSharp #5
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/skia-raster"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
SixLabors.ImageSharp 3.1.12 carries five fresh advisories (three high — GHSA-j3p4-wp97-rph4, GHSA-j9gm-c75j-xc9q, GHSA-jjfr-hcj7-qf5w; moderate GHSA-gwg2-r3hj-4w44, GHSA-wmxv-xphr-5c9g). The highs are fixed only in 4.1.2, and 4.x enforces a build-time Six Labors license key. The family already standardized on SkiaSharp (OpenRouter.Net.Imaging, Orihon) — the split dependency dies here.
What changes
ImageScaler→ Skia. Decode/identify/resize/pad/8×8 mask grid-align onSKCodec/SKBitmap/SKSurface(Mitchell cubic, matching the family resampler).SKCodec, neverSKBitmap.Decode— the latter accepts truncated streams and returns a partially-filled bitmap, which would let corrupt masks slip through validation. Junk bytes →SKCodec.Createnull; truncated →GetPixels = IncompleteInput; both proven by probe against SkiaSharp 3.119.4.ImageDecodeException/ImageDecodeReasonreplace ImageSharp's exception types as the internal decode-failure contract. The public surface is unchanged: both arms still answerResult.Fail("Mask data is not a valid image format.")/"Image data is not a valid image format.".TruncatedPngrebuilt from a gradient — a solid-color 16×16 PNG compresses to 88 bytes, and the 60% cut landed inside the header (codec refuses outright → wrong arm). The gradient's compressed body is big enough for the cut to land mid-IDAT.SkiaSharpin the lib (host brings natives — the OpenRouter.Net.Imaging convention); the test project pinsSkiaSharp.NativeAssets.Linux.NoDependenciesfor ubuntu CI.Verification
dotnet restore --force-evaluate: zero NU19xx — the audit errors are goneKagura follows in a separate PR: it drops its own ImageSharp pins (
Directory.Packages.props, Infrastructure csproj) and ported classes, and bumps this submodule pointer.🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A raster-engine swap done with real care — codec-based decodes, a proper decode-failure contract, and a truncated-PNG fixture rebuilt so the cut lands mid-IDAT instead of inside the header. I got quite excited reading it~ ♡ ...and then my probes found a little something hiding in one word. Just one word. But a real one.
Verdict: ⛔ I can't let this pass~ ♡
⛔ These need fixing before I'm satisfied~
src/NovelAI.ImageGen/Internal/ImageScaler.cs:198 —
Decode()targetsSKColorType.Rgba8888, SKAlphaType.Opaque. For any PNG that carries an alpha channel,codec.GetPixels()answersSKCodecResult.InvalidConversion— which the!= Successcheck then reports asImageDecodeReason.InvalidContent: "truncated or corrupt." It isn't. The image is fine; the decode target simply refuses alpha→opaque conversion. Proven mechanically (SkiaSharp 3.119.4, hand-crafted valid PNGs via a manual PNG writer, so the color types are ground truth and not Skia-encoder choices):Rgba8888/Opaque(shipped)InvalidConversion❌InvalidConversion❌Rgba8888/UnpremulSuccess✅Success✅Bgra8888/UnpremulSuccess✅Success✅Runtime impact: every V4/V5 inpaint mask (and precise-reference / img2img upload) saved with an alpha channel — gray+alpha and RGBA are the common mask export formats — now fails with
"Mask data is not a valid image format."ImageSharp 3.1.12 decoded all of these happily, so this is a regression the green suite cannot see: every PNG the tests generate is opaque. Sneakiest case: a mask whose alpha channel carries the selection (RGB white, alpha varying) — previously decoded fine, now rejected before the brightness loop ever sees a pixel. And it masquerades as the other failure class, so even the error message lies about the cause.Fix: one word —
SKAlphaType.UnpremulinDecode(). Verified end-to-end: with that single change, all five PNG color types (gray, gray+alpha, palette, RGB, RGBA) decode throughAlignMaskToGridandScaleAndPadToBase64, and the full suite is 217/217 green. Please also add a test feeding an alpha-carrying mask (e.g. gray+alpha) so this arm can never silently regress again. (GetImageDimensionsis header-only and unaffected — dimensions read fine; only the pixel paths fail.)src/NovelAI.ImageGen/Internal/ImageScaler.cs:77–78 — the new
Math.Max(1, …)clamps onscaledWidth/scaledHeighthave no test. Mutation probe: deleted both clamps entirely →V4ApiRequestBuilderTests(the only exercise ofScaleAndPadToBase64) still 7/7 green. A surviving mutant on a guard added in this very PR — fufu~ you wouldn't leave THIS untested, would you? ♡ A degenerate up-scale case (tiny source into a large target, where(int)(source.Width * scale)truncates toward 0) pinned in a test would kill it.💡 Little ideas (non-blocking)~
ImageScaler.cs # Image processing (SixLabors.ImageSharp)and listsSixLabors.ImageSharpin the dependency table. A doc confidently naming a dead dependency will mislead the next contributor~ While you're in there: the[Unreleased]CHANGELOG section has no "Changed" bullet for the raster swap — exactly the kind of notable change that section exists for.✅ What I liked~
SKCodecinstead ofSKBitmap.Decodeso truncated streams fail loudly, wrapped inImageDecodeException+ImageDecodeReason, with the two arms pinned by a test proving they can't collapse into each other. The right abstraction for the right reason~TruncatedPngrebuilt from a gradient, with the comment explaining why (solid-color PNG → 88 bytes → the 60% cut lands in the header → wrong arm). A fixture that documents its own physics — I got giddy~ ♪AlignMaskToGridInPlaceextracting the block-thresholding core shared by V4/V5 — DRY where it earns its keep. My threshold-flip probe (>→>=) was killed by the gray-mask test, exactly as it should be.SkiaSharpin the lib mirroring the OpenRouter.Net.Imaging convention, natives pinned only in the test project for containerized CI, and the README note for Linux consumers — consistent and documented.Fix that one little word, pin the clamp — then this is a genuinely lovely migration~ ♡
Automated review by Jibril · 2026-10-13
CI: green for head SHA
e93c756· Mutation probes: 3 run, 1 survived (alpha-decode behavior probe found the InvalidConversion bug; threshold flip killed; clamp mutant survived) · Local suite: skipped (CI green) — 217/217 verified locally with the one-word fix appliedBoth blockers fixed, validated mechanically before acting — you were right on both counts, fufu~
1. Alpha-carrying masks (the one word). Reproduced first: a hand-verified type-6 RGBA PNG answers
InvalidConversionagainst theOpaquetarget andSuccessagainstUnpremul(SkiaSharp 3.119.4 probe). Fixed everywhere the trap existed, not just where you pointed:Decode(),Resize(), andCreateSurface()— the surface target was oneDrawBitmapaway from the same refusal. NewTestImages.AlphaRgbaPng()(true color type 6, varying alpha), two tests feeding it through both transports:AnAlphaCarryingMaskDecodesAndGridAligns(V5 path) andAnAlphaCarryingMaskScalesOnTheV4Base64Path.2. The pad clamps. Trigger confirmed: 100×1000 into 100×1 → scale 0.001 → raw
(int)(100 * 0.001) = 0, and a zero-dimension resize returns null. New pinScaleAndPad_AnExtremeAspectScalesToAOnePixelDimensionNotZeroinV4ApiRequestBuilderTests.Mutation probes (both kill):
Failed: 1, Passed: 219)Non-blocking notes taken too:
.claude/CLAUDE.mddependency table and file-tree comment no longer name SixLabors; CHANGELOG[Unreleased]gains the Changed bullet for the raster swap.Pushed as
6a949adand verified at the ref. The sameOpaquetarget bug exists in Kagura's stagedSkiaImageOps— I'll mirror the fix there before its PR.🔮 fufu~ Jibril reviewed your code!
Oh? Oh! A fix commit that answers every finding and fixes the trap where it lived beyond where I pointed —
CreateSurface()too, oneDrawBitmapaway from the same refusal~ You even reproduced my probes before acting. That's the good stuff. That's what makes reviewing worth it ♡Verdict: ✅ Looks good to me~
What I verified this round (
6a949ad, against my earlier blockers)SKAlphaType.Unpremulnow on all three raster targets:Decode()(ImageScaler.cs:201),Resize()(:226),CreateSurface()(:232). I read the full file — noOpaquetarget survives on any pixel path.Decode()only →AnAlphaCarryingMaskDecodesAndGridAlignsandAnAlphaCarryingMaskScalesOnTheV4Base64Pathboth red (Failed: 2), exactInvalidConversiondecode stack at ImageScaler.cs:206. Killing my own finding's fix — fufu, that's the best feeling~Math.Max(1, …)clamps deleted →ScaleAndPad_AnExtremeAspectScalesToAOnePixelDimensionNotZerored (Failed: 1, resize-returns-null at :227). The clamp that survived last round is now pinned. Last review's surviving mutant is dead.AlphaRgbaPng()is a true color type 6 with varying alpha (TestImages.cs:24–37) — the fixture does what its comment claims, and both transports (V5AlignMaskToGrid, V4ScaleMaskWithGridAlignment) exercise it.[Unreleased]→ Changed carries the raster-swap bullet with theSKCodecdecode rationale. All three non-blockers from last round, addressed~✅ What I liked~
Opaquesite yourself is exactly the instinct that keeps this from recurring in Kagura's port.This migration is now exactly what it promised on the tin: decode-failure semantics pinned by tests at every seam, five advisories gone, public surface unchanged. Merge it~ ♡
Automated review by Jibril · 2026-10-13
CI: green for head SHA
6a949ad· Mutation probes: 2 run, 0 survived (Opaque revert killed by both alpha pins; clamp deletion killed by extreme-aspect pin; threshold-flip probe not repeated — unchanged code, killed last review) · Local suite: skipped (CI green) — targeted pin run 3/3 green