Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
feat(video-frames): --decontam auto by default in video-frames and video-set; the video fit keeps a small key-coloured part's rim and the material carried to the edge; one warning line
Soohong Kimb3d48dd49d53
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e6346c5..546b8e1 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,114 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## Unreleased++**Upgrading from 2.45.3: Release 2.46.0 changes defaults for video frames.**++Outputs that change bytes are frames from `video-frames` and `video-set` on a green or magenta key.+These changes occur within 6 px inside and 3 px outside the keyed silhouette.+Video frames keyed with explicit `--decontam palette` also change bytes where a new guard applies.++Outputs that do not change bytes include runs on a white key, subjects with no interior, and runs with `--alpha-mode brightness`.+Stills (`cutout`, `gen`) and row extraction (`extract`, `inspect`) do not change bytes.+Runs with `--decontam off` also give the 2.45.3 bytes back exactly.++To get the 2.45.3 frames back, pass:+`--decontam off`++- **Default change: `video-frames` and `video-set` now default to `--decontam auto` (before: `off`).**+ Release 2.46.0 updates these defaults to clean chroma-bleed edges automatically.+ On a chroma key, H.264 4:2:0 chroma mixes key colour into strands one to three pixels wide.+ With decontamination off, a red hair strand on green keeps an olive rim.+ The `auto` pass restores the subject's own colour to those edges.++ The video default stayed `off` until now out of caution rather than any known defect.+ The pass was benchmarked on a green key only.+ The video fit is less decisive than the still fit, leaving 10.8 % residual contamination against 0.03 % on the original benchmark.+ Earlier fixes resolved the Grok-magenta regression through the key-hue cap and corrected the frame-edge interaction.+ No blocking defect remains known.++ On synthetic diagnosis clips (CRF 18, `--spill small`, 6 frames), `auto` reduced key contamination on red strands on painted green from 23.8 % to 8.5 %, and on pure green from 27.5 % to 9.3 %.+ Halo composite error on dark and white backgrounds improved from 11.3/11.9 to 9.1/9.8, while strand recall rose from 90.9 % to 97.2 %.+ Key contamination dropped from 3.4 % to 0.5 % on magenta, and from 5.4 % to 2.2 % on Grok darker magenta.+ Contamination dropped from 20.4 % to 7.1 % on a translucent glow scene, and from 23.4 % to 10.3 % on a small green charm scene.+ Outline ink CIELAB error on the two magenta keys shifted from 19.7 to 21.8 and 21.9 to 25.5.+ Changed pixels make up about 3 to 4 % of each frame (7 % in translucent scenes), all within 6 px inside or 3 px outside the silhouette.+ Interior pixels remain byte-identical.++ Keying costs more processing time and memory.+ On an Intel i5-7500 with a 1080x1920 clip, keying took 1.10 s a frame with `off` and 3.32 s with `auto` (3.14 s for the pass without guards).+ Peak memory rose from 389 MB to 938 MB, adding about 265 MB per megapixel.+ Because `video-set` keys three items at once by default, peak memory can rise by about 1.6 GB.++ Stills (`cutout`, `gen`) and row extraction (`extract`, `inspect`) still default to `off`.+ The row extractor frames are a derived cache that `heal` re-derives, so changing their default would silently rewrite existing runs.+ Stills remain as they were.++- **Guards: Two new guards protect key-coloured details and material behind edges in the video fit.**+ The still fit output is unchanged, but explicit `--decontam palette` runs in video also use these guards.++ The key-coloured parts guard protects small details that share the key colour.+ A subject can own a key-coloured part too small for the palette to keep, such as a green charm on red hair under a green key.+ When such a part covers under 0.5 % of the interior, the palette learns key-free colours.+ Without the guard, the pass repainted the rim in the nearest key-free colour, turning the green charm's rim brown.+ Now, when the palette is key-free, band and flank pixels within 6 px of such a part keep the matte's bytes.+ This applies to pixels near parts that are opaque, deeper than 6 px, and have key hue above 8.+ The key-hue cap leaves those pixels alone.+ On synthetic tests, of the charm's key-coloured edge pixels, the share that each lost at least half of its key hue or of its coverage was 0.77 with `off`, 1.00 without the guard, and 0.82 with it.+ Colour error was 49.7 with the guard, 41.5 with `off`, and 64.9 without the guard.+ The guards cost the red-strand scene 8.2 % to 8.5 % key contamination, with no measurable change in halo.++ The material-behind guard preserves edges where subject material continues to the silhouette boundary.+ A band pixel keeps the matte's bytes when its colour, observed directly or at its own luma, falls within the opaque margin of the mean colour of subject pixels right behind it.+ This check reads subject pixels deeper than 2 px within a 3 px box radius.+ Such pixels are treated as material continued to the edge rather than blends or tints.+ The key-hue cap still applies to them.+ Unlike the still fit, this guard does not require the pixel to be free of the key's hue, because decoded chroma blur gives every edge some key colour.++- **Diagnostics: A one-line stderr warning and new report fields track skipped frames and guard activity.**+ `video-frames` prints a single line on stderr after its JSON summary when edge decontamination warrants attention.+ The line starts with `video-frames: warning: decontam auto ...` and ends by naming `--decontam off` as the way to keep the matte as it is.++ The warning prints when `auto` ran on no frames, when `auto` skipped some frames, or when the subject owns key-coloured material so the palette is not key-free.+ On magenta keys, the red hair's blue channel sits more than 8 above its green, making the hair key-coloured material (12 to 16 % of the interior) and triggering this warning.+ The warning also prints when the key-coloured part guard kept edge pixels, reporting how many and noting that thin details farther out may be repainted.+ A few key-hued pixels deep inside from codec noise guard no edge and raise no warning.+ Brightness alpha raises no warning.+ `video-set` prefixes this warning line with the item name and records it under `warnings` in the set report.++ The JSON report adds `decontam.reason` for the first skip reason and `decontam.skipped_frames` when frames are skipped.+ Per-frame stats and clip totals in the video fit add `key_part_kept_px` and `behind_kept_px`.+ Where `auto` cannot run, frames remain exactly as the matte keyed them.+ The report records "the white matte has no key colour to remove" for a white key.+ It records "no interior to learn the subject's colours from" when no pixel lies deeper than 6 px inside the silhouette.+ Under `--alpha-mode brightness`, `auto` defaults to off, recording `{"mode": "auto", "applied_frames": 0, "reason": "brightness alpha has no key colour to remove"}`.+ Explicit `--decontam palette` with brightness alpha remains refused.++ Known limits apply to this release.+ A thin feature whose colour is absent from the interior can be read as a palette colour plus key.+ For example, a thin yellow line beside red hair with a pink highlight turned pink (colour error 25.8 off to 48.3 auto).+ `--decontam off` keeps that line.+ A thin key-coloured detail farther than 6 px from any key-coloured interior is repainted in a palette colour.+ The thinnest strands retain some olive because decoders share chroma across aligned 2x2 blocks, leaving a one-pixel strand with half the key's chroma.+ A block-aware fit is planned separately.+ Outside a dark outline, the flank can still leave a few faint pixels in the subject's colour.++- **Tests**: `tests/frames/test_chroma_decontam.py` checks that both video verbs default to `auto` and that still verbs and the row extractor do not.+ It checks that `off` gives the engine's original bytes.+ It checks that `video-set` hands its value to `video-frames` and lists the warning.+ It checks that a white key, a subject with no interior, and brightness alpha give the same bytes as `off` with the reason.++ On red strands 1 to 3 px wide on green, with chroma shared across aligned 2x2 blocks as a decoder does, the default halves key contamination and does not raise halo or lower strand recall by more than 0.05.+ The rim of a small green charm keeps the matte's pixels and loses at most 0.1 more than `off` while strands away from it are cleaned.+ On magenta and Grok magenta, halo and alpha error do not rise.++ A head 6 px under the top frame edge passes the edge gate under the default.+ A few key-hued pixels deep inside raise no warning.+ `video-frames` prints the warning line on stderr.++ `tests/video/test_gait_guard_anchor_wide.py` has its stand-in for `run_frames` take `decontam` and checks that the set's `auto` reaches it.+## v2.45.3 - The seam check reads the frames that play when the body anchor spreads a sideways slide; such a cut passes again, as it did in 2.44.0**Upgrading from 2.45.2: Every cut 2.45.2 passed, and every `--anchor motion-auto` run, gives the same bytes as 2.45.2.** Only `--anchor body` cuts that 2.45.2 refused because of the walk sliding sideways before the anchor can now pass, read on the frames that play. Refused cuts print the same error line as before, the report's `producer` field names the new version, and there are no restoration schema changes.diff --git a/docs/chroma-alpha.md b/docs/chroma-alpha.mdindex bd63628..f97c979 100644--- a/docs/chroma-alpha.md+++ b/docs/chroma-alpha.md@@ -60,7 +60,7 @@ When unsure, let `--chroma-key auto` sample the base. It excludes the detected f2. **Soft-alpha unmix** — key-tinted blends near the keyed region are separated into despilled RGB + **partial alpha**: the blend model `observed = (1-k)·subject + k·key` is solved from the key-tint score, so antialiased silhouettes keep their coverage ramp instead of collapsing into a binary staircase. In-band blends (inside `--fringe-key-threshold`) are eligible only at key-distance `<= 2`; out-of-band blends use `--fringe-unmix-reach` (default 4). This preserves deeper key-tinted material while fixing trapped blend pockets between hair strands.3. **Trapped-spill despill** — a small connected cluster of key-tinted pixels (≤ `--spill-max-fraction` of the subject, default 0.005) containing at least one strongly tinted pixel is generator spill, wherever it sits: its color is corrected in place with alpha kept (no pinholes). Large key-tinted regions are intentional material and are never touched; marginally warm subject colors (skin) never qualify.-An opt-in fourth pass, edge decontamination (`decontam`, below), then re-explains the edge with the subject's own colours.+A fourth pass, edge decontamination (`decontam`, below; on by default in `video-frames` and `video-set`, off elsewhere), then re-explains the edge with the subject's own colours.The unmix/spill tunables live in the run's `sprite-request.json` under `chroma` (`unmix_reach`, `spill_max_fraction`) — the extractor reads them from there, CLI flags override, and the effective values are written back to the request so every run records what produced it. Then it extracts connected components and writes fresh transparent cells. The pixel-unfake path is unaffected: it binarizes alpha downstream (α ≥ 128 → opaque), so soft-alpha input degrades gracefully. This is intentionally closer to hatch-pet than to simple `magick -transparent`.@@ -94,6 +94,8 @@ B is the key background measured locally (mean of keyed pixels within 12 px). P| no palette colour explains it (a colour the interior never shows), or, in the still fit, the explanation misses its luma (outline ink darker than any blend) | engine bytes || closer to a palette colour than the blend explains it (margin 10, or 3 noise sigmas of the background): as observed, or, in the still fit, at its own luma within the subject's own spread (below), which then also raises the margin | engine bytes, except where the still fit finds that the matte changed it although it carries no key hue: then its own colour and coverage come back. With a key-free palette, a leftover key hue (every keyed channel above every other by more than 8) loses the excess beyond 8 on its keyed channels, which never raises a channel and leaves colours without the key hue alone (a red or a skin tone under magenta is not magenta) || still fit: deeper than a still's antialiased edge (2 px, the engine's in-band unmix depth) without the key's hue, or on that edge without it and with the colour (at its own luma) of the material right behind it | the subject's own colour, never a blend or a tint; its own colour and coverage come back where the matte changed it |+| video fit: with a key-free palette, a band or flank pixel within 6 px of a key-coloured part (opaque, deeper than 6 px, key hue above 8) | keeps the matte's bytes, and the key-hue cap leaves it alone |+| video fit: a band pixel whose colour, as observed or at its own luma, is within the opaque margin of the mean colour of the subject right behind it (subject pixels deeper than 2 px, within a 3 px box radius) | that material continued to the edge, not a blend or a tint: keeps the matte's bytes (the key-hue cap still applies) || pushed toward the key clearly more than the fit misses by, but not proven partial coverage (in the still fit, not when a shade of a palette colour explains it within the spread) | colour only || a blend within the unmix reach (`chroma.unmix_reach`, default 4) | alpha and colour || a blend deeper in the 6 px band | colour only, so a key reflection on a solid surface is recoloured rather than made see-through |@@ -116,7 +118,7 @@ Three modes: `palette` runs the pass and fails loud where it cannot (no key hue,|---|---|---|| `sprite-gen cutout IMAGE` | `off` | `--decontam off\|auto\|palette` (chroma routes; on the white matte `auto` records that it has no key colour and `palette` is refused) || `sprite-gen gen … --transparent` | `off` | `--decontam off\|auto\|palette` (chroma strategy; `auto` leaves native alpha alone, `palette` is refused before the provider is called when it cannot apply) |-| `sprite-gen video-frames`, `video-set` | `off` | `--decontam off\|auto\|palette` |+| `sprite-gen video-frames`, `video-set` | `auto` | `--decontam off\|auto\|palette` (where `auto` cannot run it leaves frames as keyed and records the reason; with brightness alpha `auto` means off and `palette` is refused) || row extractor (`extract`, `inspect`) | `off` | request `chroma.decontam`, or `--decontam` to override; written back to the request once in play |Reports record what the pass did under `decontam`: mode, whether it applied (and why not), fit, the palette and its material spread, and changed / refit / tint / recovered / unexplained / restored pixel counts.@@ -134,7 +136,18 @@ Measured on a procedural ground-truth set (red strands 0.5–3 px wide, outlineThe two still-fit rows are v2.10.2's; v2.10.0 measured 0.07 % · 1.14 / 1.14 and 0.16 % · 1.15 / 1.14 on the same set, with the same recall. The video-fit row is unchanged.-**Defaults.** Every entry point defaults to `off`, which returns the engine's output byte for byte; pass `--decontam auto` (or `palette`) to use the pass. Stills defaulted to `auto` for a while during development. The full test suite then caught a regression on Grok's darker magenta: the key-hue cap greyed the warm colours of the palette, so a hard-edged brown square lost its opaque edge. The cap is fixed and tested, but the measurements above use a green key, so the default stays `off`. The row extractor stays `off` in any case: its frames are a derived cache that `heal` re-derives from raw, so a changed default would silently rewrite every existing run, and pixel-art rows binarize alpha downstream anyway. `video-frames` / `video-set` stay `off` (see the video numbers above).+**Defaults.** In release 2.46.0, `sprite-gen video-frames` and `video-set` default to `--decontam auto` (before: `off`). Stills (`cutout`, `gen`) stay `off`: they defaulted to `auto` during development until a regression on Grok's darker magenta greyed warm palette colours; that cap and the frame-edge interaction were fixed, but the still measurements use a green key, so stills remain `off`. The row extractor (`extract`, `inspect`) stays `off` because its frames are a derived cache that `heal` re-derives from raw, so a changed default would silently rewrite every existing run, and pixel-art rows binarize alpha downstream anyway. Video defaulted to `off` until now out of caution rather than a known defect, because the pass was benchmarked on a green key only and the video fit is less decisive than the still fit (10.8 % residual key contamination against 0.03 % on the original benchmark); the Grok-magenta regression and the frame-edge interaction were fixed earlier, and no blocking defect is known. Video defaults to `auto` now because on a chroma key, H.264 4:2:0 chroma mixes the key into every strand one to three pixels wide, leaving an olive rim on red hair over green with the pass off, and the pass gives such edges the subject's own colour back. Where `auto` cannot run, it leaves frames exactly as the matte keyed them and records the reason in the report (`decontam.reason`, and per frame in `rows[].decontam`): a white key ("the white matte has no key colour to remove") or a subject with no pixel deeper than 6 px inside the keyed silhouette ("no interior to learn the subject's colours from"). With `--alpha-mode brightness` (emitted light on black, no chroma key), `auto` simply means off and records `decontam: {"mode": "auto", "applied_frames": 0, "reason": "brightness alpha has no key colour to remove"}`, while an explicit `--decontam palette` is refused. To keep the matte as it is, `--decontam off` returns the 2.45.3 bytes exactly. On stderr after its JSON summary, `video-frames` prints a one-line warning starting `video-frames: warning: decontam auto ...` when `auto` did not run on any frame (naming the reason and stating that frames match `--decontam off`), when `auto` skipped some frames, when the subject owns key-coloured material so the palette is not key-free (edges may be repainted in the key colour too), or when the key-coloured part guard kept edge pixels beside a small key-coloured part (stating how many, and that a thin detail of that colour farther out may be repainted); codec noise deep inside the subject guards no edge and raises no warning, brightness alpha prints no warning, and the line ends by naming `--decontam off` as the way to keep the matte as it is. `video-set` lists the same line prefixed by the item name under `warnings` in its set report. On an Intel i5-7500 (1080x1920 synthetic clip, CRF 18, `--spill small`, one process per run), keying took 1.10 s a frame with `off` and 3.32 s with `auto` (3.14 s for the pass without the two guards), and peak memory rose from 389 MB to 938 MB, about 265 MB more per megapixel of frame; `video-set` keys three items at once by default, so at that size it can use about 1.6 GB more at a moment. Changed pixels are about 3 to 4 % of each frame (7 % in translucent scenes), all within 6 px inside or 3 px outside the silhouette with byte-identical interiors, and the guards cost the red-strand scene 8.2 % to 8.5 % key contamination and nothing measurable in halo.++| scene | key contamination | halo dark / white | strand recall |+|---|---|---|---|+| red strands 1 to 3 px on painted green | 23.8 % to 8.5 % | 11.3/11.9 to 9.1/9.8 | 90.9 % to 97.2 % |+| the same on pure #00FF00 | 27.5 % to 9.3 % | 12.0/11.8 to 9.8/9.7 | 92.4 % to 97.7 % |+| red hair on magenta | 3.4 % to 0.5 % | 14.9/19.5 to 11.8/15.3 | 86.4 % to 97.7 % |+| red hair on Grok's darker magenta | 5.4 % to 2.2 % | 21.1/32.3 to 17.1/24.7 | 72.8 % to 91.2 % |+| a translucent glow and a cyan wisp | 20.4 % to 7.1 % | 12.7/9.3 to 9.2/8.0 | 85.2 % to 97.9 % |+| a small green charm (0.04 % of the interior) | 23.4 % to 10.3 % | 11.6/12.7 to 9.5/10.6 | 90.2 % to 97.0 % |++On the small green charm, the share of key-coloured edge losing half its key hue or coverage was 0.77 with `off`, 1.00 without the guard, and 0.82 with it (colour error 41.5 to 64.9 to 49.7). On the two magenta keys, red hair counts as key-coloured material (12 to 16 % of the interior, since blue sits above green) so the palette is not key-free, the warning line appears, and outline ink CIELAB error increases slightly from 19.7 to 21.8 and 21.9 to 25.5; a white key and a subject with no interior produce identical bytes to `off`, with the reason recorded. Reports add `decontam.reason` (first skip reason), `decontam.skipped_frames` when frames are skipped, and two counters in per-frame stats and clip totals: `key_part_kept_px` and `behind_kept_px`.Limitations:@@ -143,6 +156,11 @@ Limitations:- A subject that owns key-coloured material keeps it in its palette, and its edges are repainted with whatever explains them best, key colour included. Keep the default when that material matters.- The pixel-unfake path binarizes alpha downstream, so on pixel-art rows only edge colours change.- The measurements are on a green key. On magenta the pass is covered by synthetic tests (teal, skin-toned and hard-edged warm subjects, the painted key and Grok's darker one), not by a benchmark.+- A thin feature whose colour the interior never shows can be read as a palette colour plus key and take that colour: a thin yellow line beside red hair with a pink highlight came out pink (colour error 25.8 off to 48.3 auto). `--decontam off` keeps it.+- A thin key-coloured detail farther than 6 px from any key-coloured interior (the far end of a green ribbon) is repainted in a palette colour.+- Even with the pass, the thinnest strands keep some olive: the decoder shares chroma across each aligned 2x2 block, so a one-pixel strand has the strand's luma but half the key's chroma, which no single blend line explains; the pass reads it as opaque and keeps it. This is most of what remains. A block-aware fit is planned separately.+- Outside a dark outline, the flank can give a few faint pixels in the subject's colour (already described in the frame-edge paragraph).+- Time and memory go up: keying on an Intel i5-7500 took 1.10 s a frame with `off` and 3.32 s with `auto` at 1080x1920, and peak memory rose from 389 MB to 938 MB (about 265 MB more per megapixel of frame, or about 1.6 GB more for `video-set` running three concurrent jobs).## Relateddiff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 5fede81..b52898f 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -688,19 +688,11 @@ a time. A window over 6 megapixels is keyed on every n-th pixel each way insteadreference costs more than that to judge. The report records `reference_size`,`reference_window` and `reference_stride` (1 = every pixel of the window).-### Edges: `--decontam palette`--`--spill` fixes key colour painted *into* the subject. The strands and outlines at its-edge are a different problem. H.264 4:2:0 has already averaged the key into the chroma of-anything one or two pixels wide, and despilling what is left turns thin red strands-orange. `video-frames --decontam palette` (also `video-set --decontam palette`) re-explains-each edge pixel as a blend of the local key background with one colour the subject owns,-and writes that colour at the pixel's observed luma. It uses the video fit and one palette-per clip, learned on the first frame, so edge colours cannot flicker between palettes. Within-the edge band the edge gate reads, it gives no coverage to a pixel the matte left transparent,-so it adds no edge contact: a clip with none under `off` has none under `palette`. The-default `off` keeps frames byte-identical. Method, guards and measurements:-[chroma-alpha.md](chroma-alpha.md#decontam--give-the-edge-the-subjects-own-colour-back).+### Edges: `--decontam`++`--spill` fixes key colour painted into the subject, but strands and outlines at its edge are a different problem: H.264 4:2:0 chroma mixes the key into anything one to three pixels wide, leaving an olive rim on red hair or turning it orange under despill. In release 2.46.0, `video-frames` and `video-set` default to `--decontam auto` (before: `off`), which re-explains each edge pixel as a blend of the local key background with colours the subject owns and writes that colour at the pixel's observed luma, using the video fit and one palette per clip learned on the first frame. Where `auto` cannot run (a white key or a subject with no interior deeper than 6 px) and under `--alpha-mode brightness`, it leaves frames exactly as keyed and records the reason in the report. On stderr, `video-frames` prints a one-line warning (and `video-set` lists it under `warnings`) when `auto` did not run on any frame, skipped frames, found key-coloured subject material, or triggered the key-coloured part guard.++Two video guards keep the matte's bytes for edge pixels beside small key-coloured parts and for pixels matching the material right behind them. Pass `--decontam off` to keep the matte as it is and return 2.45.3 bytes exactly. Keying time roughly triples (1.10 s to 3.32 s a frame on an Intel i5-7500 at 1080x1920) and peak memory rises by about 265 MB per megapixel of frame. Method, guards and measurements: [chroma-alpha.md](chroma-alpha.md#decontam--give-the-edge-the-subjects-own-colour-back).## 3b. Canvas shape for raised limbs and wide costumesdiff --git a/sprite_gen/frames/decontam.py b/sprite_gen/frames/decontam.pyindex 53f2ea4..1df5a27 100644--- a/sprite_gen/frames/decontam.py+++ b/sprite_gen/frames/decontam.py@@ -61,6 +61,20 @@ Guards, in order of the decision they make:red or blue magenta), so it unmixes gold as if it were a blend. The video fitdoes none of this, since a decoded frame's chroma is blurred and an edge pixel'scolour is no evidence of what the subject is made of there.+- *Key-coloured parts (video fit).* A subject can own a key-coloured part too small+ for the palette to keep (a green charm under a green key, under+ `KEY_MATERIAL_MAX_SHARE` of the interior). A key-free palette then has no colour that+ explains the part's edge, and the fit repainted it in the nearest key-free colour: a+ green charm's rim came out brown. Band and flank pixels within `KEY_PART_GUARD_PX` of+ such a part (opaque, deeper than `BAND_PX`, key-hued past `KEY_HUE_TINT`) keep the+ engine's bytes, and the key-hue cap leaves them alone.+- *Material behind (video fit).* A band pixel that has the colour, as it is or at its+ own luma, of the subject right behind it (the mean of subject pixels deeper than+ `_IN_BAND_UNMIX_KEY_DEPTH` within `VIDEO_BEHIND_PX`), to within the opaque margin, is+ that material carried on to the edge, not a blend or a tint: it keeps the engine's+ bytes, the key-hue cap still applying. Unlike the still fit's test it does not ask+ the pixel to be free of the key's hue, which a decoded frame's blurred chroma gives+ every edge a little of.- *Tint.* Between the two: a pixel pushed from its colour toward the key by atleast `TINT_SHIFT`, and `TINT_RATIO` times more than the fit misses by, isrecoloured with its coverage left alone (a faint cast, not proven coverage).@@ -118,6 +132,8 @@ LUMA_SEPARATION = 24.0 # video fit: luma alpha only where P and B differ this mLUMA_MISS_SLACK = 12.0 # still fit: a blend must reproduce the observed luma this closely (plus 3 sigma)TINT_SHIFT = 8.0 # a tint moves the colour at least this far toward the key ...TINT_RATIO = 2.0 # ... and at least this many times farther than the fit misses by+KEY_PART_GUARD_PX = BAND_PX # video fit: edge pixels this close to a key-coloured part keep the engine's bytes+VIDEO_BEHIND_PX = 3 # video fit: the material behind an edge pixel is the subject's mean within this box radiusLUMA = np.array([0.299, 0.587, 0.114])_CHUNK = 1 << 16@@ -371,15 +387,28 @@ def decontaminate(source_rgb: np.ndarray, keyed: np.ndarray, keyed_mask: np.ndar# the subject's plain material right behind the edge: deeper than it and without the key's huesource_int = np.asarray(source_rgb)[..., :3].astype(np.int32)behind = subject & (depth > edge) & (_key_excess_field(source_int, keyed_channels, unkeyed_channels) <= 0)- behind_weight = _box_sum_at(behind.astype(np.float64), edge, rows, cols)- behind_mean = np.stack([_box_sum_at(rgb[..., channel] * behind, edge, rows, cols) for channel in range(3)],- axis=-1) / np.maximum(behind_weight, 1.0)[:, None]+ reach = edge+ else:+ # *Material behind*: a decoded frame's edge carries the key's blurred hue, so no hue test+ behind = subject & (depth > edge)+ reach = VIDEO_BEHIND_PX+ behind_weight = _box_sum_at(behind.astype(np.float64), reach, rows, cols)+ behind_mean = np.stack([_box_sum_at(rgb[..., channel] * behind, reach, rows, cols) for channel in range(3)],+ axis=-1) / np.maximum(behind_weight, 1.0)[:, None]+ guarded = np.zeros(len(rows), dtype=bool)+ if fit == "video" and keyfree:+ # *Key-coloured parts*: the palette left them out, so no colour of it explains their edge+ excess = _key_excess_field(out[..., :3].astype(np.int32), keyed_channels, unkeyed_channels)+ part = subject & (depth > BAND_PX) & (out[..., 3] == 255) & (excess > KEY_HUE_TINT)+ if part.any():+ guarded = (_ring_depth(part, KEY_PART_GUARD_PX) <= KEY_PART_GUARD_PX)[rows, cols]alpha_new = np.zeros(len(rows))colour_new = np.zeros((len(rows), 3))blend = np.zeros(len(rows), dtype=bool)tinted = np.zeros(len(rows), dtype=bool)unexplained = np.zeros(len(rows), dtype=bool)own = np.zeros(len(rows), dtype=bool)+ behind_kept = np.zeros(len(rows), dtype=bool)for start in range(0, len(rows), _CHUNK):r = rows[start:start + _CHUNK]c = cols[start:start + _CHUNK]@@ -427,25 +456,27 @@ def decontaminate(source_rgb: np.ndarray, keyed: np.ndarray, keyed_mask: np.ndar# pixel darker than the line allows (outline ink the interior never shows) is not a blendexplained &= np.abs(residual @ LUMA) <= LUMA_MISS_SLACK + NOISE_SIGMAS * sigma_lumain_flank = flank[r, c]+ mean = behind_mean[start:start + len(r)]+ behind_distance = np.minimum(np.linalg.norm(obs - mean, axis=-1),+ np.linalg.norm(obs - recolor_to_luma(mean, obs @ LUMA), axis=-1))+ behind_near = (behind_weight[start:start + len(r)] > 0) & (behind_distance <= margin)if material_test:# Deeper than a still's antialiased edge, a key share shows only as the key's hue: a pixel# without it is the subject's own colour. On the edge, so is a pixel without it that has# the colour (at its own luma) of the plain material right behind it.hueless = ~in_flank & (_key_excess_field(obs, keyed_channels, unkeyed_channels) <= 0)deep = depth[r, c] > edge- mean = behind_mean[start:start + len(r)]- behind_distance = np.minimum(np.linalg.norm(obs - mean, axis=-1),- np.linalg.norm(obs - recolor_to_luma(mean, obs @ LUMA), axis=-1))- settled = hueless & (deep | ((behind_weight[start:start + len(r)] > 0) & (behind_distance <= margin)))+ settled = hueless & (deep | behind_near)else:- settled = np.zeros(len(r), dtype=bool)- is_blend = explained & (alpha < 1.0) & (in_flank | (miss + margin < material)) & ~settled & ~held[r, c]+ settled = ~in_flank & behind_near+ kept_as_is = settled | guarded[start:start + len(r)] | held[r, c]+ is_blend = explained & (alpha < 1.0) & (in_flank | (miss + margin < material)) & ~kept_as_is# a tint: displaced from its colour toward the key clearly more than the fit misses by, but# not by enough to prove partial coverage, and by more than shading any palette colour# accounts for. Recoloured, coverage untouched.shift = (1.0 - alpha) * np.sqrt((line * line).sum(-1))is_tint = (explained & ~is_blend & ~in_flank & (shift >= TINT_SHIFT) & (shift >= TINT_RATIO * miss)- & ~own_colour & ~settled & ~held[r, c])+ & ~own_colour & ~kept_as_is)# coverage: refit within the unmix reach and on the flank; colour only deeper in the bandrefit = (in_flank | (depth[r, c] <= alpha_depth)) & ~is_tintfloor = np.maximum(MIN_RECOVERED_ALPHA, NOISE_SIGMAS * sigma_luma / np.maximum(np.abs((chosen - bg) @ LUMA), 1e-6))@@ -461,6 +492,8 @@ def decontaminate(source_rgb: np.ndarray, keyed: np.ndarray, keyed_mask: np.ndarblend[start:start + len(r)] = is_blend | is_tinttinted[start:start + len(r)] = is_tintunexplained[start:start + len(r)] = ~explained+ if not material_test:+ behind_kept[start:start + len(r)] = settledown[start:start + len(r)] = (material_test & ~is_blend & ~is_tint & ~in_flank & ~held[r, c]& ((material <= margin) | settled))@@ -480,7 +513,7 @@ def decontaminate(source_rgb: np.ndarray, keyed: np.ndarray, keyed_mask: np.ndarrestored = int(np.count_nonzero(back))clamped = 0if keyfree:- kept = ~blend & band[rows, cols] & (out[rows, cols, 3] > 0)+ kept = ~blend & ~guarded & band[rows, cols] & (out[rows, cols, 3] > 0)kr, kc = rows[kept], cols[kept]kept_rgb = out[kr, kc, :3].astype(np.int32)hued = _key_excess_field(kept_rgb, keyed_channels, unkeyed_channels) > KEY_HUE_TINT@@ -519,4 +552,7 @@ def decontaminate(source_rgb: np.ndarray, keyed: np.ndarray, keyed_mask: np.ndar"restored_px": restored,"key_hue_capped_px": clamped,}+ if fit == "video":+ stats["key_part_kept_px"] = int(np.count_nonzero(guarded))+ stats["behind_kept_px"] = int(np.count_nonzero(behind_kept))return out, statsdiff --git a/sprite_gen/video/batch.py b/sprite_gen/video/batch.pyindex 68b6544..a410342 100644--- a/sprite_gen/video/batch.py+++ b/sprite_gen/video/batch.py@@ -710,7 +710,7 @@ def run_item(prepare_side: Callable[[Path], tuple[Path, dict]] | None = None,body_height: int | None = None,fit: str = "state",- decontam: str = "off",+ decontam: str = frames_mod.DECONTAM_DEFAULT,walk_start: str = "redraw",redraw_runner: Callable[..., int] = run_redraw_cli,still_provider: str | None = None,@@ -839,14 +839,16 @@ def run_item(generated["end_frame_sha256"] = end_digestatomic_write_text(clip_report, json.dumps(generated, ensure_ascii=False, indent=2) + "\n")- # a tight frame is clipped on purpose; leftover key background still fails. The default- # call keeps its pre-decontam signature, so a stand-in for run_frames still fits- extra = {} if decontam == "off" else {"decontam": decontam}- if lifetime:- extra["alpha_mode"] = "brightness"- fr = frames_mod.run_frames(clip, item_dir / "frames", key=key, allow_edge_contact=False, report_path=item_dir / "frames.report.json", spill=spill, reference=canvas_png, allow_subject_edge_contact=fit == "tight", **extra)+ # a tight frame is clipped on purpose; leftover key background still fails. `decontam` is passed+ # whatever its value: run_frames defaults to auto, so leaving it out would not mean off+ extra = {"alpha_mode": "brightness"} if lifetime else {}+ fr = frames_mod.run_frames(clip, item_dir / "frames", key=key, allow_edge_contact=False, report_path=item_dir / "frames.report.json", spill=spill, reference=canvas_png, allow_subject_edge_contact=fit == "tight", decontam=decontam, **extra)result["frames"] = {k: fr[k] for k in ("fps", "frames", "alpha_zero_pct_min", "alpha_zero_pct_max")}result["frames"]["spill"] = fr.get("spill", {}).get("mode")+ decontam_note = frames_mod.decontam_warning(fr.get("decontam") or {})+ if decontam_note:+ # the set report lists it under `warnings`+ result["frames"]["decontam_warning"] = decontam_notelp = loop_mod.run_loop(Path(fr["keyed_dir"]), item_dir / "loop", fps=float(fr["fps"]), state=state, min_len=None, max_len=None, n_out=None, seam_max=loop_mod.SEAM_RATIO_MAX, name=item, report_path=item_dir / "loop.report.json", anchor=anchor, body_height=body_height,cycle_mode=cycle_for(state, direction, play, pinned=pins),facing=facing if direction in handed_mod.LATERAL_VIEWS else "right")@@ -1066,7 +1068,7 @@ def run_set(facing_fix: str = "none",body_height: int | None = None,fit: str = "state",- decontam: str = "off",+ decontam: str = frames_mod.DECONTAM_DEFAULT,align_cycles: str = "auto",align_between: str = align_mod.DEFAULT_BETWEEN,interpolate: Any = None,@@ -1160,6 +1162,8 @@ def run_set(f"(the loops under `suspects` in {a['report']}, docs/loop-repair.md section 4)"for st, a in aligned.items() if a.get("reason") == "cycle-suspects"]warnings += [f"cycle-align:{st}: {line}" for st, a in aligned.items() for line in a.get("warnings", [])]+ warnings += [f"{r['item']}: {r['frames']['decontam_warning']}" for r in results+ if r.get("ok") and "decontam_warning" in r.get("frames", {})]warnings += [f"cycle-align:{st}: --align-foot {item}={foot} not applied ({a.get('why') or a.get('error') or 'the item failed'})"for st, a in aligned.items() for item, foot in a.get("feet_unused", {}).items()]# A start still `gen` asked about instead of drawing (codex cannot run, and the choice is to ask): the set ends@@ -1256,7 +1260,7 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:parser.add_argument("--spill", choices=frames_mod.SPILL_MODES, default="auto", help="auto: judge key reflections from each item's canvas still (default); small / full: force")parser.add_argument("--fit", choices=canvas_mod.FITS, default="state", help="state: each state's room for the motion (default); tight: no room, the subject fills the frame and a motion that leaves it is clipped (use with --body-height at a low --resolution)")parser.add_argument("--body-height", type=int, default=None, help="scale every state's loop so the standing height is this many px (video-loop --body-height): one value for the whole set keeps the character the same size across states")- parser.add_argument("--decontam", choices=frames_mod.DECONTAM_MODES, default="off", help="passed to video-frames: palette re-explains key-tinted edges with the subject's own colours (default off)")+ parser.add_argument("--decontam", choices=frames_mod.DECONTAM_MODES, default=frames_mod.DECONTAM_DEFAULT, help="passed to video-frames: auto (default) re-explains key-tinted edges with the subject's own colours where it applies; palette demands it; off keeps the matte as it is")parser.add_argument("--walk-start", choices=WALK_START_MODES, default="redraw", help="redraw (default): a front or back walk films from its base still redrawn mid-step (one image generation, `sprite-gen gen --ref base`, kept as walk-start.png) — from a standing still the clip model walks askew; as-given: film from the base still itself")parser.add_argument("--still-provider", help="image provider for the mid-step redraw (default: sprite-gen gen's own default; ""a codex that cannot run there asks, and the set stops with the question)")@@ -1280,7 +1284,7 @@ def run(**kwargs: object) -> int:facing=str(kwargs.get("facing") or "right"), facing_fix=str(kwargs.get("facing_fix") or "none"),shape=(str(kwargs["shape"]) if kwargs.get("shape") else None), anchor=str(kwargs.get("anchor") or "none"), spill=str(kwargs.get("spill") or "auto"),body_height=(int(kwargs["body_height"]) if kwargs.get("body_height") else None), fit=str(kwargs.get("fit") or "state"),- decontam=str(kwargs.get("decontam") or "off"), align_cycles=str(kwargs.get("align_cycles") or "auto"),+ decontam=str(kwargs.get("decontam") or frames_mod.DECONTAM_DEFAULT), align_cycles=str(kwargs.get("align_cycles") or "auto"),align_between=str(kwargs.get("align_between") or align_mod.DEFAULT_BETWEEN),walk_start=str(kwargs.get("walk_start") or "redraw"), still_provider=kwargs.get("still_provider"), # type: ignore[arg-type]handed=handed_mod.parse_all(list(kwargs.get("handed") or [])) or None, # type: ignore[arg-type]diff --git a/sprite_gen/video/frames.py b/sprite_gen/video/frames.pyindex c3e4e55..1368f9a 100644--- a/sprite_gen/video/frames.py+++ b/sprite_gen/video/frames.py@@ -45,6 +45,7 @@ import jsonimport mathimport shutilimport subprocess+import sysfrom pathlib import Pathfrom typing import Any@@ -87,9 +88,16 @@ SPECK_APART = 0.1# Edge decontamination (`sprite_gen.frames.decontam`). A decoded frame's chroma is blurred by# 4:2:0 while its luma is not, so frames use the video fit. One palette serves the whole clip,# learned on the first frame, so the colours an edge may take cannot change between frames.+# `auto` is the default: a thin strand on a chroma key comes out of the codec mixed with the key+# (olive on green), and the pass gives it the subject's colour back. Where it cannot run (a white+# key, a subject with no interior) the frames keep the matte's bytes and the report says why;+# brightness alpha has no key, so `auto` is off there.DECONTAM_MODES = ("off", "auto", "palette")+DECONTAM_DEFAULT = "auto"DECONTAM_FIT = "video"-DECONTAM_STAT_KEYS = ("changed_px", "refit_px", "tint_px", "recovered_px", "unexplained_px", "restored_px", "key_hue_capped_px")+DECONTAM_STAT_KEYS = ("changed_px", "refit_px", "tint_px", "recovered_px", "unexplained_px", "restored_px", "key_hue_capped_px",+ "key_part_kept_px", "behind_kept_px")+BRIGHTNESS_NO_KEY = "brightness alpha has no key colour to remove"def _require(binary: str) -> str:@@ -336,16 +344,19 @@ def key_frames(check_edges: bool = True,spill: str = "small",allow_subject: bool = False,- decontam: str = "off",+ decontam: str = DECONTAM_DEFAULT,alpha_mode: str = "chroma",) -> dict[str, Any]:from . import effectif alpha_mode not in effect.ALPHA_MODES:raise SystemExit(f"video-frames: unknown --alpha-mode {alpha_mode!r}")if alpha_mode == "brightness":- if decontam != "off":+ if decontam == "palette":raise SystemExit("video-frames: brightness alpha does not use --decontam")- return effect.key_frames(raw_files, keyed_dir)+ report = effect.key_frames(raw_files, keyed_dir)+ if decontam == "auto":+ report["decontam"] = {"mode": "auto", "applied_frames": 0, "reason": BRIGHTNESS_NO_KEY}+ return reportif spill not in ("small", "full"):raise SystemExit(f"video-frames: key_frames takes a resolved spill mode (small|full), got {spill!r}")if decontam not in DECONTAM_MODES:@@ -354,6 +365,7 @@ def key_frames(decontam_first: dict[str, Any] | None = Nonedecontam_totals = {name: 0 for name in DECONTAM_STAT_KEYS}decontam_applied = 0+ decontam_skipped: list[str] = [] # why `auto` left a frame as the matte keyed it, frame by frame# Full correction lowers the tint threshold as well as lifting the size cap.spill_max = SPILL_FULL_FRACTION if spill == "full" else Nonespill_tint = SPILL_FULL_MIN_TINT if spill == "full" else None@@ -386,6 +398,7 @@ def key_frames(decontam_applied += 1else:row["decontam"] = {"applied": False, "reason": done.get("reason")}+ decontam_skipped.append(str(done.get("reason")))if check_edges:contact = edge_contact(image)if any(v > EDGE_MAX_PIXELS for v in contact.values()):@@ -420,8 +433,12 @@ def key_frames("applied_frames": decontam_applied,"totals": decontam_totals,}+ if decontam_skipped:+ report["decontam"] |= {"skipped_frames": len(decontam_skipped), "reason": decontam_skipped[0]}else:report["decontam"] = {"mode": decontam, "applied_frames": 0}+ if decontam_skipped:+ report["decontam"]["reason"] = decontam_skipped[0]if contacts and check_edges:if not allow_subject:raise SystemExit(_edge_contact_message(contacts))@@ -435,6 +452,34 @@ def key_frames(return report+def decontam_warning(decontam: dict[str, Any]) -> str | None:+ """The line to read when the edge pass did less than the default promises, or met the key's+ own colour: `auto` left frames as the matte keyed them, the subject owns key-coloured material,+ or the guard kept the edge beside a small key-coloured part. None when there is nothing to look+ at (the pass off, or brightness alpha)."""+ mode = decontam.get("mode")+ if mode in (None, "off") or decontam.get("reason") == BRIGHTNESS_NO_KEY:+ return None+ if not decontam.get("applied_frames"):+ return (f"decontam {mode} did not run ({decontam.get('reason')}): the frames are the matte's, "+ "the same as --decontam off")+ notes = []+ if decontam.get("skipped_frames"):+ notes.append(f"it left {decontam['skipped_frames']} frame(s) as the matte keyed them ({decontam['reason']})")+ share = float(decontam.get("key_material_share") or 0.0)+ kept = int((decontam.get("totals") or {}).get("key_part_kept_px") or 0)+ if not decontam.get("palette_keyfree", True):+ notes.append(f"the subject owns key-coloured material ({share:.1%} of its interior), "+ "so an edge may be repainted in the key's colour too")+ elif kept:+ # a few key-hued interior pixels deep inside (codec noise) guard no edge and are not worth a line+ notes.append(f"the subject has small key-coloured parts: {kept} edge pixel(s) beside them kept as the matte "+ "keyed them, and a thin detail of that colour farther out may be repainted")+ if not notes:+ return None+ return f"decontam {mode}: {'; '.join(notes)} — look at those edges; --decontam off keeps the matte as it is"++def _worst(contacts: list[dict[str, Any]]) -> tuple[dict[str, Any], str]:worst = max(contacts, key=lambda c: c["top"] + c["left"] + c["right"])return worst, f"(worst {worst['frame']}: top {worst['top']} / left {worst['left']} / right {worst['right']} px)"@@ -468,11 +513,11 @@ def _edge_contact_message(contacts: list[dict[str, Any]]) -> str:return message-def run_frames(clip: Path, out_dir: Path, *, key: str, allow_edge_contact: bool, report_path: Path | None, spill: str = "small", reference: Path | None = None, allow_subject_edge_contact: bool = False, decontam: str = "off", source_manifest: Path | None = None, alpha_mode: str = "chroma") -> dict[str, Any]:+def run_frames(clip: Path, out_dir: Path, *, key: str, allow_edge_contact: bool, report_path: Path | None, spill: str = "small", reference: Path | None = None, allow_subject_edge_contact: bool = False, decontam: str = DECONTAM_DEFAULT, source_manifest: Path | None = None, alpha_mode: str = "chroma") -> dict[str, Any]:from . import effectif alpha_mode not in effect.ALPHA_MODES:raise SystemExit(f"video-frames: unknown --alpha-mode {alpha_mode!r}")- if alpha_mode == "brightness" and decontam != "off":+ if alpha_mode == "brightness" and decontam == "palette":raise SystemExit("video-frames: brightness alpha does not use --decontam")if source_manifest is not None and reference is None:raise SystemExit("video-frames: --source-manifest needs --reference (the source canvas)")@@ -528,7 +573,7 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:parser.add_argument("--source-manifest", type=Path, help="write exact clip/canvas/keyed-file digests and source timestamps (requires --reference)")parser.add_argument("--spill", choices=SPILL_MODES, default="small", help="small: correct only small key-tinted clusters (default); full: every key tint in the subject is spill; auto: decide from --reference")parser.add_argument("--reference", type=Path, help="the still the clip was made from (required by --spill auto)")- parser.add_argument("--decontam", choices=DECONTAM_MODES, default="off", help="off: the matte as is (default); palette: re-explain key-tinted edges with the subject's own colours (one palette per clip, video fit); auto: the same where it applies, reasons recorded where not")+ parser.add_argument("--decontam", choices=DECONTAM_MODES, default=DECONTAM_DEFAULT, help="auto (default): re-explain key-tinted edges with the subject's own colours (one palette per clip, video fit) where it applies, the reason recorded where not (a white key, a subject with no interior; brightness alpha is off); palette: the same, failing where it cannot run; off: the matte as is")def run(**kwargs: object) -> int:@@ -536,10 +581,13 @@ def run(**kwargs: object) -> int:allow_edge_contact=bool(kwargs.get("allow_edge_contact")), report_path=kwargs.get("report"), # type: ignore[arg-type]spill=str(kwargs.get("spill") or "small"), reference=kwargs.get("reference"), # type: ignore[arg-type]allow_subject_edge_contact=bool(kwargs.get("allow_subject_edge_contact")),- decontam=str(kwargs.get("decontam") or "off"), source_manifest=kwargs.get("source_manifest"),+ decontam=str(kwargs.get("decontam") or DECONTAM_DEFAULT), source_manifest=kwargs.get("source_manifest"),alpha_mode=str(kwargs.get("alpha_mode") or "chroma")) # type: ignore[arg-type]summary = {k: payload[k] for k in ("clip", "keyed_dir", "width", "height", "fps", "frames", "codec", "pix_fmt", "audio_streams", "alpha_zero_pct_min", "alpha_zero_pct_max", "edge_contacts", "edge_policy", "spill", "report")}print(json.dumps(summary, ensure_ascii=False, indent=2))+ line = decontam_warning(payload["decontam"])+ if line:+ print(f"video-frames: warning: {line}", file=sys.stderr)return 0