Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

docs(decontam): the video fit's hold, settle and band-only examination; the scene numbers redone with them

Soohong Kimc976f315093b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 546b8e1..4f1804e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,8 +7,8 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
**Upgrading from 2.45.3: Release 2.46.0 changes defaults for video frames.**
Outputs that change bytes are frames from `video-frames` and `video-set` on a green or magenta key.
-These changes occur within 6 px inside and 3 px outside the keyed silhouette.
-Video frames keyed with explicit `--decontam palette` also change bytes where a new guard applies.
+These changes occur within 6 px inside the keyed silhouette only.
+Video frames keyed with explicit `--decontam palette` also change bytes, because they use the same video fit.
Outputs that do not change bytes include runs on a white key, subjects with no interior, and runs with `--alpha-mode brightness`.
Stills (`cutout`, `gen`) and row extraction (`extract`, `inspect`) do not change bytes.
@@ -19,46 +19,108 @@ To get the 2.45.3 frames back, pass:
- **Default change: `video-frames` and `video-set` now default to `--decontam auto` (before: `off`).**
Release 2.46.0 updates these defaults to clean chroma-bleed edges automatically.
+
On a chroma key, H.264 4:2:0 chroma mixes key colour into strands one to three pixels wide.
With decontamination off, a red hair strand on green keeps an olive rim.
The `auto` pass restores the subject's own colour to those edges.
The video default stayed `off` until now out of caution rather than any known defect.
The pass was benchmarked on a green key only.
- The video fit is less decisive than the still fit, leaving 10.8 % residual contamination against 0.03 % on the original benchmark.
+ The video fit is less decisive than the still fit.
+ In the documentation's benchmark table, the video-fit row (10.8 % residual contamination, 9.38/8.73 halo, 91.4 % recall) was measured in September 2026 with flank recovery.
+ It has not been remeasured since flank recovery was dropped, because those benchmark inputs and tools are not at hand.
+ The diagnosis table in `docs/chroma-alpha.md` (its `decontam` section) has the current video fit's numbers.
+ Still-fit rows in that table are unchanged.
+
Earlier fixes resolved the Grok-magenta regression through the key-hue cap and corrected the frame-edge interaction.
No blocking defect remains known.
- On synthetic diagnosis clips (CRF 18, `--spill small`, 6 frames), `auto` reduced key contamination on red strands on painted green from 23.8 % to 8.5 %, and on pure green from 27.5 % to 9.3 %.
- Halo composite error on dark and white backgrounds improved from 11.3/11.9 to 9.1/9.8, while strand recall rose from 90.9 % to 97.2 %.
- Key contamination dropped from 3.4 % to 0.5 % on magenta, and from 5.4 % to 2.2 % on Grok darker magenta.
- Contamination dropped from 20.4 % to 7.1 % on a translucent glow scene, and from 23.4 % to 10.3 % on a small green charm scene.
- Outline ink CIELAB error on the two magenta keys shifted from 19.7 to 21.8 and 21.9 to 25.5.
- Changed pixels make up about 3 to 4 % of each frame (7 % in translucent scenes), all within 6 px inside or 3 px outside the silhouette.
- Interior pixels remain byte-identical.
+ On synthetic diagnosis clips (CRF 18, `--spill small`, 6 frames, one machine), `auto` reduced key contamination on red strands on painted green from 24.2 % to 9.2 %, and on pure green from 27.4 % to 9.3 %.
+ Halo composite error on dark and white backgrounds shifted from 11.6/12.1 to 9.5/10.1 on painted green, and from 12.0/11.9 to 9.8/9.7 on pure green.
+ Strand recall stayed at 91.4 % on painted green and 92.4 % on pure green.
+ Strand recall no longer rises with the pass, because that gain came from the flank that the video fit no longer recovers.
+ Halo gains are smaller than before for the same reason.
+
+ Key contamination dropped from 3.4 % to 0.5 % on magenta, and from 5.4 % to 2.0 % on Grok darker magenta.
+ Halo shifted from 15.0/19.6 to 12.5/16.3 on magenta, and from 20.7/31.5 to 19.4/30.0 on Grok darker magenta (compared to 17.1/24.7 when flank recovery was present).
+ Strand recall stayed at 87.0 % on magenta and 73.3 % on Grok darker magenta.
+ Outline ink CIELAB error shifted from 19.3 to 20.8 on magenta, and from 21.7 to 24.1 on Grok darker magenta.
+ Red hair counts as key-coloured material there, so the palette is not key-free and the warning line appears.
+
+ Contamination dropped from 20.6 % to 7.8 % on a translucent glow scene (halo 12.6/9.4 to 11.5/8.4, strand recall 85.7 %).
+ It dropped from 23.8 % to 10.4 % on a small green charm scene (halo 11.8/12.9 to 9.9/11.1, strand recall 90.7 %).
- Keying costs more processing time and memory.
- On an Intel i5-7500 with a 1080x1920 clip, keying took 1.10 s a frame with `off` and 3.32 s with `auto` (3.14 s for the pass without guards).
- Peak memory rose from 389 MB to 938 MB, adding about 265 MB per megapixel.
- Because `video-set` keys three items at once by default, peak memory can rise by about 1.6 GB.
+ Changed pixels make up about 2 % of each frame (4.5 % in translucent scenes).
+ All changed pixels lie within 6 px inside the silhouette.
+ Nothing outside the silhouette changes any more.
+ Interior pixels remain byte-identical.
Stills (`cutout`, `gen`) and row extraction (`extract`, `inspect`) still default to `off`.
The row extractor frames are a derived cache that `heal` re-derives, so changing their default would silently rewrite existing runs.
Stills remain as they were.
+- **Temporal fit: The video fit holds edge decisions across frames, settles still pixels, and drops flank recovery.**
+ With default `auto`, the edge of a subject standing still sparkled from frame to frame where `--decontam off` remained calm.
+ Codec noise pushed edge pixels across the pass's threshold tests from frame to frame.
+ These tests decide blends, tints, material behind, explained pixels, palette colours, and flank coverage.
+ Each crossing changed the written colour by far more than the noise itself did.
+ The raw matte with `off` follows codec noise smoothly, staying calm despite keeping an olive rim.
+
+ `video-frames` and `video-set` now maintain a clip history in clip order.
+ This history passes to each frame through `decontaminate` as `history` (`decontam_history` in `cutout` and `remove_chroma_background`).
+
+ Each test now holds its previous outcome at that pixel until current evidence passes the bar the other way by 16 (an RGB distance, `VIDEO_HOLD`).
+ A palette colour picked in the previous frame stays unless another colour's line passes closer by 16.
+ Moving pixels change their evidence far beyond 16 and are decided afresh.
+ Only near-threshold calls stick.
+
+ Still pixels settle toward the current fit.
+ When a pixel was written in both frames in the same palette colour, and its source moved by at most 8 in any channel across its 3x3 box (`VIDEO_STILL`), it settles.
+ It moves a tenth of the way (`VIDEO_SETTLE` 0.1) from its previous value toward this frame's fit, capped at 12 per channel (`VIDEO_SETTLE_REACH`).
+ This dampens codec noise while following real slow changes within 12 levels.
+
+ The video fit also drops flank recovery.
+ It no longer gives coverage back to keyed pixels (hard cut background within 3 px of the subject).
+ Decoded chroma blur created faint blends on keyed pixels, and codec noise carried them back and forth over the noise floor.
+ This filled dark gaps between strands with a faint smear that came and went across frames.
+ The video fit now examines the band only (subject pixels within 6 px of the keyed background).
+ The still fit keeps its flank recovery unchanged.
+
+ With history enabled, frame bytes depend strictly on previous frames in clip order and remain deterministic.
+ A single video-fit call without history gives the same band bytes as before, leaving keyed flank pixels as the matte keyed them.
+
+ On a synthetic still scene under decoder noise (red strands 1 to 3 px wide on painted green, 8 frames, sigma 2):
+ Mean edge change between frames on a dark background was 2.34 with `off` (99th percentile 8.3).
+ Before this fix, the pass gave a mean change of 2.75 (99th percentile 16.0).
+ With this fix, the mean change dropped to 2.27 (99th percentile 8.3).
+ Key contamination on that scene dropped from 6.5 % with `off` to 2.3 % now.
+
+ Per-frame stats and clip totals add `held_px` and `settled_px`.
+ `held_px` counts pixels where a held outcome or colour differed from an independent fit.
+ `settled_px` counts pixels settled toward the current fit.
+ In video, `recovered_px` is now 0.
+
+ Keying cost was remeasured on an Intel i5-7500 with a 1080x1920 synthetic clip, one process per run, in the same session.
+ Keying took 1.21 s a frame with `off` and 3.42 s with `auto` (the pass before this fix took 3.65 s).
+ Peak memory rose from 396 MB to 908 MB, adding about 247 MB per megapixel.
+ Because `video-set` keys three items at once by default, peak memory can rise by about 1.5 GB at that size.
+ These numbers replace the earlier 2.46.0 numbers.
+
- **Guards: Two new guards protect key-coloured details and material behind edges in the video fit.**
- The still fit output is unchanged, but explicit `--decontam palette` runs in video also use these guards.
+ The still fit output is unchanged byte for byte, but explicit `--decontam palette` runs in video also use these guards.
The key-coloured parts guard protects small details that share the key colour.
A subject can own a key-coloured part too small for the palette to keep, such as a green charm on red hair under a green key.
When such a part covers under 0.5 % of the interior, the palette learns key-free colours.
Without the guard, the pass repainted the rim in the nearest key-free colour, turning the green charm's rim brown.
- Now, when the palette is key-free, band and flank pixels within 6 px of such a part keep the matte's bytes.
+
+ Now, when the palette is key-free, band pixels within 6 px of such a part keep the matte's bytes.
+ There is no flank in video any more, so this guard covers band pixels only.
This applies to pixels near parts that are opaque, deeper than 6 px, and have key hue above 8.
The key-hue cap leaves those pixels alone.
- On synthetic tests, of the charm's key-coloured edge pixels, the share that each lost at least half of its key hue or of its coverage was 0.77 with `off`, 1.00 without the guard, and 0.82 with it.
- Colour error was 49.7 with the guard, 41.5 with `off`, and 64.9 without the guard.
- The guards cost the red-strand scene 8.2 % to 8.5 % key contamination, with no measurable change in halo.
+
+ On synthetic tests, the share of the charm's key-coloured edge pixels that lost half their key hue or coverage was 0.77 with `off`, 0.82 now (the diagnosis measured 1.00 without the guard).
+ Its colour error was 41.7 with `off`, 46.2 now (64.9 without the guard in the diagnosis).
The material-behind guard preserves edges where subject material continues to the silhouette boundary.
A band pixel keeps the matte's bytes when its colour, observed directly or at its own luma, falls within the opaque margin of the mean colour of subject pixels right behind it.
@@ -72,14 +134,14 @@ To get the 2.45.3 frames back, pass:
The line starts with `video-frames: warning: decontam auto ...` and ends by naming `--decontam off` as the way to keep the matte as it is.
The warning prints when `auto` ran on no frames, when `auto` skipped some frames, or when the subject owns key-coloured material so the palette is not key-free.
- On magenta keys, the red hair's blue channel sits more than 8 above its green, making the hair key-coloured material (12 to 16 % of the interior) and triggering this warning.
+ On magenta keys, the red hair's blue channel sits more than 8 above its green, making the hair key-coloured material (9.6 % of the interior on magenta and 6.6 % on Grok's darker magenta) and triggering this warning.
The warning also prints when the key-coloured part guard kept edge pixels, reporting how many and noting that thin details farther out may be repainted.
A few key-hued pixels deep inside from codec noise guard no edge and raise no warning.
Brightness alpha raises no warning.
`video-set` prefixes this warning line with the item name and records it under `warnings` in the set report.
The JSON report adds `decontam.reason` for the first skip reason and `decontam.skipped_frames` when frames are skipped.
- Per-frame stats and clip totals in the video fit add `key_part_kept_px` and `behind_kept_px`.
+ Per-frame stats and clip totals in the video fit add `key_part_kept_px`, `behind_kept_px`, `held_px`, and `settled_px`.
Where `auto` cannot run, frames remain exactly as the matte keyed them.
The report records "the white matte has no key colour to remove" for a white key.
It records "no interior to learn the subject's colours from" when no pixel lies deeper than 6 px inside the silhouette.
@@ -87,24 +149,50 @@ To get the 2.45.3 frames back, pass:
Explicit `--decontam palette` with brightness alpha remains refused.
Known limits apply to this release.
+
+ The hold works on pixels that stay put.
+ A strand that moves is decided afresh in every frame.
+ A moving edge still changes from frame to frame as its colour does, so a red rim moving over a dark background changes more than the olive rim `off` leaves.
+
+ The settle follows a slow change within 12 levels per channel.
+
+ The video fit gives back no coverage the hard cut keyed.
+ Outer pixels of a soft edge that the cut erased stay erased.
+ Only on stills, outside a dark outline the flank can give a few faint pixels in the subject's colour.
+
A thin feature whose colour is absent from the interior can be read as a palette colour plus key.
- For example, a thin yellow line beside red hair with a pink highlight turned pink (colour error 25.8 off to 48.3 auto).
+ For example, a thin yellow line beside red hair with a pink highlight turned pink (colour error 26.2 with `off`, 48.0 now).
`--decontam off` keeps that line.
+
A thin key-coloured detail farther than 6 px from any key-coloured interior is repainted in a palette colour.
+
The thinnest strands retain some olive because decoders share chroma across aligned 2x2 blocks, leaving a one-pixel strand with half the key's chroma.
A block-aware fit is planned separately.
- Outside a dark outline, the flank can still leave a few faint pixels in the subject's colour.
+
+ Pixel-art rows binarizing alpha, the green-only benchmark, and increased processing time and memory remain known limits.
- **Tests**: `tests/frames/test_chroma_decontam.py` checks that both video verbs default to `auto` and that still verbs and the row extractor do not.
It checks that `off` gives the engine's original bytes.
It checks that `video-set` hands its value to `video-frames` and lists the warning.
It checks that a white key, a subject with no interior, and brightness alpha give the same bytes as `off` with the reason.
- On red strands 1 to 3 px wide on green, with chroma shared across aligned 2x2 blocks as a decoder does, the default halves key contamination and does not raise halo or lower strand recall by more than 0.05.
+ A new section tests the video fit over time on the red-strand scene standing still while decoder noise changes every frame (luma per pixel, chroma per aligned 2x2 block, eight frames).
+ The default's edge changes from one frame to the next no more than `off`'s, in mean and at the 99th percentile.
+ The edge is still cleaned, with key contamination at most half of `off`'s.
+ Clip totals count held and settled pixels, and the first frame has none.
+ A frame given twice comes out with the same bytes both times.
+
+ The video fit leaves every pixel the hard cut keyed transparent and recovers nothing.
+ The still fit still recovers its flank.
+
+ Changed tests reflect dropped flank recovery in video.
+ A head 6 px under the top frame edge passes the edge gate under the default and keeps the blur rows below the edge band as the matte keyed them.
+ For magenta skin tone in a 4:2:0 frame, the video fit's halo is read on the band where it acts (at most 0.9 of `off`'s), and the keyed flank keeps the matte's bytes.
+ The still fit keeps its halved halo.
+
The rim of a small green charm keeps the matte's pixels and loses at most 0.1 more than `off` while strands away from it are cleaned.
On magenta and Grok magenta, halo and alpha error do not rise.
- A head 6 px under the top frame edge passes the edge gate under the default.
A few key-hued pixels deep inside raise no warning.
`video-frames` prints the warning line on stderr.
diff --git a/docs/chroma-alpha.md b/docs/chroma-alpha.md
index af486aa..d32f0a9 100644
--- a/docs/chroma-alpha.md
+++ b/docs/chroma-alpha.md
@@ -83,7 +83,7 @@ Select with request `chroma.mode: "ycbcr"` or `--chroma-mode ycbcr`. **The defau
The unmix above separates a blend by its key-tint score, and that assumes the subject's own colour carries no tint (grey, white). A saturated subject breaks the assumption. A crimson hair strand blended a third of the way into green scores as barely tinted, so it stays opaque and olive; where the full spill pass does reach it, the `1/(1-k)` gain of `despill_color` turns it orange. Video adds a second loss: H.264 4:2:0 has already averaged the key into the chroma of every strand one or two pixels wide. Its luma survives at full resolution.
-`decontam: "palette"` re-explains the edge after the matte (either matte). It learns a 32-colour palette from the subject's confident interior (opaque, deeper than 6 px) and fits every subject pixel within 6 px of the keyed background, and every keyed pixel within 3 px of the subject, as
+`decontam: "palette"` re-explains the edge after the matte (either matte). It learns a 32-colour palette from the subject's confident interior (opaque, deeper than 6 px) and fits every subject pixel within 6 px of the keyed background (the still fit also fits every keyed pixel within 3 px of the subject; the video fit fits the subject pixels only), as
observed = alpha · P + (1 − alpha) · B
@@ -94,21 +94,24 @@ B is the key background measured locally (mean of keyed pixels within 12 px). P
| no palette colour explains it (a colour the interior never shows), or, in the still fit, the explanation misses its luma (outline ink darker than any blend) | engine bytes |
| closer to a palette colour than the blend explains it (margin 10, or 3 noise sigmas of the background): as observed, or, in the still fit, at its own luma within the subject's own spread (below), which then also raises the margin | engine bytes, except where the still fit finds that the matte changed it although it carries no key hue: then its own colour and coverage come back. With a key-free palette, a leftover key hue (every keyed channel above every other by more than 8) loses the excess beyond 8 on its keyed channels, which never raises a channel and leaves colours without the key hue alone (a red or a skin tone under magenta is not magenta) |
| still fit: deeper than a still's antialiased edge (2 px, the engine's in-band unmix depth) without the key's hue, or on that edge without it and with the colour (at its own luma) of the material right behind it | the subject's own colour, never a blend or a tint; its own colour and coverage come back where the matte changed it |
-| video fit: with a key-free palette, a band or flank pixel within 6 px of a key-coloured part (opaque, deeper than 6 px, key hue above 8) | keeps the matte's bytes, and the key-hue cap leaves it alone |
+| video fit: with a key-free palette, a band pixel within 6 px of a key-coloured part (opaque, deeper than 6 px, key hue above 8) | keeps the matte's bytes, and the key-hue cap leaves it alone |
| video fit: a band pixel whose colour, as observed or at its own luma, is within the opaque margin of the mean colour of the subject right behind it (subject pixels deeper than 2 px, within a 3 px box radius) | that material continued to the edge, not a blend or a tint: keeps the matte's bytes (the key-hue cap still applies) |
| pushed toward the key clearly more than the fit misses by, but not proven partial coverage (in the still fit, not when a shade of a palette colour explains it within the spread) | colour only |
| a blend within the unmix reach (`chroma.unmix_reach`, default 4) | alpha and colour |
| a blend deeper in the 6 px band | colour only, so a key reflection on a solid surface is recoloured rather than made see-through |
-| a keyed pixel within 3 px of the subject that fits a blend above the background's noise floor and chains to the subject | coverage recovered: the faint sides of a strand the hard cut erased |
+| still fit: a keyed pixel within 3 px of the subject that fits a blend above the background's noise floor and chains to the subject | coverage recovered: the faint sides of a strand the hard cut erased |
+| video fit: a keyed pixel within 3 px of the subject | stays transparent: the video fit recovers no flank coverage |
+| video fit in a clip: each test at that pixel, unless this frame's evidence passes the test's bar the other way by 16 (an RGB distance, `VIDEO_HOLD`); and the palette colour picked in the previous frame, unless another palette colour's line passes closer by 16 | hold: keeps the outcome and palette colour it had in the previous frame |
+| video fit in a clip: a pixel written in this frame and in the previous one in the same palette colour, whose source moved by at most 8 in any channel anywhere in its 3x3 box (`VIDEO_STILL`) | settle: moves a tenth of the way (`VIDEO_SETTLE` 0.1) from its previous value toward this frame's fit, and never farther than 12 per channel (`VIDEO_SETTLE_REACH`) from that fit |
| `video-frames`: within its edge band (4 px of the top, left and right edges), a pixel the matte left transparent | stays transparent: the pass adds no edge contact to a clip (below) |
Interior pixels are never touched, `off` returns the engine's output byte for byte, and everything is integer-exact or a fixed sequence of float64 operations (no random seed, no iteration-order dependence). The palette is a histogram k-means with a greedy weighted farthest-point seed and a fixed iteration count. There is no new dependency.
Licensing: the pass is plain numpy inside this Apache-2.0 repository and installs nothing. Learned matting was considered as an optional install: BiRefNet and ViTMatte (MIT) and MODNet (Apache-2.0) are compatible, while RMBG-2.0 (non-commercial) and Robust Video Matting (GPL-3.0) cannot be dependencies. None was adopted: on the ground-truth set below, the two that were measured (BiRefNet-lite and ViTMatte-S alphas with foreground estimation or this repaint) left more key contamination than this pass, and each needs a model download.
-**The frame edge.** A decoded frame's blur runs the subject on into the key as a smooth tail, and the video fit reads that tail as a faint blend at every distance, on the keyed ring just past the flank (background by the pass's own definition) as well as on the flank, so no floor tells a strand's side from it. In the interior that costs a few faint pixels in the subject's colour. Along the frame's top, left and right edges it decides a clip: `video-frames` refuses one where any visible pixel in its edge band has key colour in the source, as the key surviving the matte, so a head a few pixels under the frame edge would pass with the pass off and fail with it on. `video-frames` therefore hands the pass its edge band (`decontam_edge_band`, the gate's `EDGE_ROWS`), and there a pixel the matte left transparent stays transparent. Everywhere else the flank keeps what it recovers; on the ground-truth set below, the video fit's strand recall gain comes from it.
+**The frame edge.** A decoded frame's blur runs the subject on into the key as a smooth tail, and the pass reads that tail as a faint blend at every distance, on the keyed ring just past the flank as well as on the flank, so no floor tells a strand's side from it. The video fit now gives keyed pixels no coverage at all, so its frame-edge band holds by construction there. Along the frame's top, left and right edges it decides a clip: `video-frames` refuses one where any visible pixel in its edge band has key colour in the source, as the key surviving the matte, so a head a few pixels under the frame edge would pass with the pass off and fail with it on. `video-frames` therefore hands the pass its edge band (`decontam_edge_band`, the gate's `EDGE_ROWS`), and there a pixel the matte left transparent stays transparent. The still fit's flank keeps what it recovers.
-Two fits. `still` chooses P per pixel. `video` chooses P on a 3×3 mean of the observation, whose chroma is blurred anyway, and averages the projection alpha with a luma-only alpha where P and B differ in luma. `video-frames` uses the video fit and learns one palette per clip on the first frame, so the colours an edge may take cannot change from frame to frame.
+Two fits. `still` chooses P per pixel. `video` chooses P on a 3x3 mean of the observation, whose chroma is blurred anyway, and averages the projection alpha with a luma-only alpha where P and B differ in luma. `video-frames` uses the video fit and learns one palette per clip on the first frame, so the colours an edge may take cannot change from frame to frame. In a clip, `video-frames` keeps a history across frames, so a frame's bytes depend on the frames before it in the order they were given. This stops the edge of a still subject sparkling from frame to frame when codec noise shifts pixels across the pass's tests and palette choices. With hold, each test keeps the outcome it had at that pixel in the previous frame until this frame's evidence passes the test's bar the other way by 16 (an RGB distance, `VIDEO_HOLD`), and the palette colour picked in the previous frame stays unless another palette colour's line passes closer by 16. With settle, a pixel written in this frame and in the previous one in the same palette colour, whose source moved by at most 8 in any channel anywhere in its 3x3 box (`VIDEO_STILL`), moves a tenth of the way (`VIDEO_SETTLE` 0.1) from its previous value toward this frame's fit, and never farther than 12 per channel (`VIDEO_SETTLE_REACH`) from that fit.
**Gold and yellow.** Gold sits between red and green, so a gold a little darker or yellower than the palette's own also fits as a warmer palette gold with some green mixed in. Up to v2.10.1 the still fit took that reading, and gold edges and small gold drops came out orange and partly see-through. Since v2.10.2 the still fit weighs the blend against the subject's own colour at the pixel's own light (the freedom the written colour already has), and a blend has to win by more than the subject's own spread: the 99th percentile of how far its confident interior sits from the palette, large for a textured gold and small for flat colour. A still's antialiased edge is at most two pixels deep, so deeper in the band a key share has to show as the key's hue: a pixel without it is the subject's own colour, and so is an edge pixel without it that has the colour of the material right behind it. Where the matte changed such a pixel, the source colour and coverage come back. The RGB matte (`cutout`'s default) needs that: it scores key tint on the channel average, which calls yellow green, so it unmixes gold as if it were a blend; the YCbCr matte (`gen`) does not. The video fit keeps its decisions, because a decoded frame's chroma is blurred and an edge pixel's colour is no evidence of what the subject is made of there.
@@ -134,33 +137,36 @@ Measured on a procedural ground-truth set (red strands 0.5–3 px wide, outline
| H.264 frames | RGB engine + `--spill full` | 33.2 % | 12.79 / 12.43 | 82.1 % |
| H.264 frames | + `decontam` (video fit) | 10.8 % | 9.38 / 8.73 | 91.4 % |
-The two still-fit rows are v2.10.2's; v2.10.0 measured 0.07 % · 1.14 / 1.14 and 0.16 % · 1.15 / 1.14 on the same set, with the same recall. The video-fit row is unchanged.
+The two still-fit rows are v2.10.2's; v2.10.0 measured 0.07 % · 1.14 / 1.14 and 0.16 % · 1.15 / 1.14 on the same set, with the same recall. The video-fit row was measured in September 2026 with the video fit as it was then, which recovered the keyed flank, and has not been re-measured since flank recovery was dropped because that benchmark's inputs and tools are not at hand. The still-fit rows of that table are unchanged, and the scene table under Defaults has the current video fit's numbers.
-**Defaults.** In release 2.46.0, `sprite-gen video-frames` and `video-set` default to `--decontam auto` (before: `off`). Stills (`cutout`, `gen`) stay `off`: they defaulted to `auto` during development until a regression on Grok's darker magenta greyed warm palette colours; that cap and the frame-edge interaction were fixed, but the still measurements use a green key, so stills remain `off`. The row extractor (`extract`, `inspect`) stays `off` because its frames are a derived cache that `heal` re-derives from raw, so a changed default would silently rewrite every existing run, and pixel-art rows binarize alpha downstream anyway. Video defaulted to `off` until now out of caution rather than a known defect, because the pass was benchmarked on a green key only and the video fit is less decisive than the still fit (10.8 % residual key contamination against 0.03 % on the original benchmark); the Grok-magenta regression and the frame-edge interaction were fixed earlier, and no blocking defect is known. Video defaults to `auto` now because on a chroma key, H.264 4:2:0 chroma mixes the key into every strand one to three pixels wide, leaving an olive rim on red hair over green with the pass off, and the pass gives such edges the subject's own colour back. Where `auto` cannot run, it leaves frames exactly as the matte keyed them and records the reason in the report (`decontam.reason`, and per frame in `rows[].decontam`): a white key ("the white matte has no key colour to remove") or a subject with no pixel deeper than 6 px inside the keyed silhouette ("no interior to learn the subject's colours from"). With `--alpha-mode brightness` (emitted light on black, no chroma key), `auto` simply means off and records `decontam: {"mode": "auto", "applied_frames": 0, "reason": "brightness alpha has no key colour to remove"}`, while an explicit `--decontam palette` is refused. To keep the matte as it is, `--decontam off` returns the 2.45.3 bytes exactly. On stderr after its JSON summary, `video-frames` prints a one-line warning starting `video-frames: warning: decontam auto ...` when `auto` did not run on any frame (naming the reason and stating that frames match `--decontam off`), when `auto` skipped some frames, when the subject owns key-coloured material so the palette is not key-free (edges may be repainted in the key colour too), or when the key-coloured part guard kept edge pixels beside a small key-coloured part (stating how many, and that a thin detail of that colour farther out may be repainted); codec noise deep inside the subject guards no edge and raises no warning, brightness alpha prints no warning, and the line ends by naming `--decontam off` as the way to keep the matte as it is. `video-set` lists the same line prefixed by the item name under `warnings` in its set report. On an Intel i5-7500 (1080x1920 synthetic clip, CRF 18, `--spill small`, one process per run), keying took 1.10 s a frame with `off` and 3.32 s with `auto` (3.14 s for the pass without the two guards), and peak memory rose from 389 MB to 938 MB, about 265 MB more per megapixel of frame; `video-set` keys three items at once by default, so at that size it can use about 1.6 GB more at a moment. Changed pixels are about 3 to 4 % of each frame (7 % in translucent scenes), all within 6 px inside or 3 px outside the silhouette with byte-identical interiors, and the guards cost the red-strand scene 8.2 % to 8.5 % key contamination and nothing measurable in halo.
+**Defaults.** In release 2.46.0, `sprite-gen video-frames` and `video-set` default to `--decontam auto` (before: `off`). Stills (`cutout`, `gen`) stay `off`: they defaulted to `auto` during development until a regression on Grok's darker magenta greyed warm palette colours; that cap and the frame-edge interaction were fixed, but the still measurements use a green key, so stills remain `off`. The row extractor (`extract`, `inspect`) stays `off` because its frames are a derived cache that `heal` re-derives from raw, so a changed default would silently rewrite every existing run, and pixel-art rows binarize alpha downstream anyway. Video defaulted to `off` until now out of caution rather than a known defect, because the pass was benchmarked on a green key only and the video fit is less decisive than the still fit (10.8 % residual key contamination against 0.03 % on the original benchmark); the Grok-magenta regression and the frame-edge interaction were fixed earlier, and no blocking defect is known. Video defaults to `auto` now because on a chroma key, H.264 4:2:0 chroma mixes the key into every strand one to three pixels wide, leaving an olive rim on red hair over green with the pass off, and the pass gives such edges the subject's own colour back. The video fit drops flank recovery, examining only subject pixels within 6 px of the keyed background, and holds calls and settles pixels across frames so still edges do not sparkle: on a still scene under decoder noise (red strands on painted green), the mean change of each keyed edge pixel from one frame to the next on a dark background was 2.34 with `off` (99th percentile 8.3), 2.75 with the pass before this fix (16.0), and 2.27 now (8.3), while key contamination fell from 6.5 % to 2.3 %. Where `auto` cannot run, it leaves frames exactly as the matte keyed them and records the reason in the report (`decontam.reason`, and per frame in `rows[].decontam`): a white key ("the white matte has no key colour to remove") or a subject with no pixel deeper than 6 px inside the keyed silhouette ("no interior to learn the subject's colours from"). With `--alpha-mode brightness` (emitted light on black, no chroma key), `auto` simply means off and records `decontam: {"mode": "auto", "applied_frames": 0, "reason": "brightness alpha has no key colour to remove"}`, while an explicit `--decontam palette` is refused. To keep the matte as it is, `--decontam off` returns the 2.45.3 bytes exactly. On stderr after its JSON summary, `video-frames` prints a one-line warning starting `video-frames: warning: decontam auto ...` when `auto` did not run on any frame (naming the reason and stating that frames match `--decontam off`), when `auto` skipped some frames, when the subject owns key-coloured material so the palette is not key-free (edges may be repainted in the key colour too), or when the key-coloured part guard kept edge pixels beside a small key-coloured part (stating how many, and that a thin detail of that colour farther out may be repainted); codec noise deep inside the subject guards no edge and raises no warning, brightness alpha prints no warning, and the line ends by naming `--decontam off` as the way to keep the matte as it is. `video-set` lists the same line prefixed by the item name under `warnings` in its set report. On an Intel i5-7500 (1080x1920 synthetic clip, CRF 18, `--spill small`, one process per run), keying took 1.21 s a frame with `off` and 3.42 s with `auto` (the pass before this fix took 3.65 s there), and peak memory rose from 396 MB to 908 MB, about 247 MB more per megapixel of frame; `video-set` keys three items at once by default, so at that size it can use about 1.5 GB more at a moment. Changed pixels are about 2 % of each frame (4.5 % in the translucent scene), all within 6 px inside the silhouette; nothing outside it changes any more, and the interior is byte-identical.
| scene | key contamination | halo dark / white | strand recall |
|---|---|---|---|
-| red strands 1 to 3 px on painted green | 23.8 % to 8.5 % | 11.3/11.9 to 9.1/9.8 | 90.9 % to 97.2 % |
-| the same on pure #00FF00 | 27.5 % to 9.3 % | 12.0/11.8 to 9.8/9.7 | 92.4 % to 97.7 % |
-| red hair on magenta | 3.4 % to 0.5 % | 14.9/19.5 to 11.8/15.3 | 86.4 % to 97.7 % |
-| red hair on Grok's darker magenta | 5.4 % to 2.2 % | 21.1/32.3 to 17.1/24.7 | 72.8 % to 91.2 % |
-| a translucent glow and a cyan wisp | 20.4 % to 7.1 % | 12.7/9.3 to 9.2/8.0 | 85.2 % to 97.9 % |
-| a small green charm (0.04 % of the interior) | 23.4 % to 10.3 % | 11.6/12.7 to 9.5/10.6 | 90.2 % to 97.0 % |
+| red strands 1 to 3 px on painted green | 24.2 % to 9.2 % | 11.6/12.1 to 9.5/10.1 | 91.4 % to 91.4 % |
+| the same on pure #00FF00 | 27.4 % to 9.3 % | 12.0/11.9 to 9.8/9.7 | 92.4 % to 92.4 % |
+| red hair on magenta | 3.4 % to 0.5 % | 15.0/19.6 to 12.5/16.3 | 87.0 % to 87.0 % |
+| red hair on Grok's darker magenta | 5.4 % to 2.0 % | 20.7/31.5 to 19.4/30.0 | 73.3 % to 73.3 % |
+| a translucent glow and a cyan wisp | 20.6 % to 7.8 % | 12.6/9.4 to 11.5/8.4 | 85.7 % to 85.7 % |
+| a small green charm (0.04 % of the interior) | 23.8 % to 10.4 % | 11.8/12.9 to 9.9/11.1 | 90.7 % to 90.7 % |
-On the small green charm, the share of key-coloured edge losing half its key hue or coverage was 0.77 with `off`, 1.00 without the guard, and 0.82 with it (colour error 41.5 to 64.9 to 49.7). On the two magenta keys, red hair counts as key-coloured material (12 to 16 % of the interior, since blue sits above green) so the palette is not key-free, the warning line appears, and outline ink CIELAB error increases slightly from 19.7 to 21.8 and 21.9 to 25.5; a white key and a subject with no interior produce identical bytes to `off`, with the reason recorded. Reports add `decontam.reason` (first skip reason), `decontam.skipped_frames` when frames are skipped, and two counters in per-frame stats and clip totals: `key_part_kept_px` and `behind_kept_px`.
+Strand recall no longer rises with the pass because that gain came from the flank the video fit no longer recovers; halo gains are smaller for the same reason (on Grok's darker magenta, 19.4/30.0 now against 17.1/24.7 with the flank, off 20.7/31.5). On the small green charm, the share of key-coloured edge losing half its key hue or coverage was 0.77 with `off`, 1.00 without the guard in the diagnosis, and 0.82 now (colour error 41.7 with `off`, 64.9 without the guard, and 46.2 now). On the two magenta keys, red hair counts as key-coloured material so the palette is not key-free, the warning line appears, and outline ink CIELAB error goes from 19.3 to 20.8 on magenta and from 21.7 to 24.1 on Grok's darker magenta; a white key and a subject with no interior produce identical bytes to `off`, with the reason recorded. Reports add `decontam.reason` (first skip reason), `decontam.skipped_frames` when frames are skipped, and per-frame stats and clip totals add two counters, `held_px` (pixels where a held outcome or colour differed from what that frame alone would have given) and `settled_px` (pixels settled), alongside `key_part_kept_px` and `behind_kept_px`, while `recovered_px` is now 0 in video.
Limitations:
- An edge takes one of the subject's interior colours at its observed brightness. A thin feature whose colour appears nowhere inside the subject keeps the engine's pixels. On the RGB matte that includes the matte's reading of a thin yellow line or dot, with no pixel deeper than two pixels, as a green blend.
- In video a thin strand's chroma is gone before keying, so its colour is chosen from blurred chroma; strands come back in the hair's hue, sometimes a shade darker or paler than drawn.
+- The hold works on pixels that stay put. A strand that moves is decided afresh in every frame, so a moving edge still changes from frame to frame as its colour does; a red rim moving over a dark background changes more than the olive rim `off` leaves.
+- The settle follows a slow change within 12 levels per channel.
+- The video fit gives back no coverage the hard cut keyed: the outer pixels of a soft edge that the cut erased stay erased (the halo figures above).
- A subject that owns key-coloured material keeps it in its palette, and its edges are repainted with whatever explains them best, key colour included. Keep the default on stills, but pass `--decontam off` on video when that material matters.
- The pixel-unfake path binarizes alpha downstream, so on pixel-art rows only edge colours change.
- The measurements are on a green key. On magenta the pass is covered by synthetic tests (teal, skin-toned and hard-edged warm subjects, the painted key and Grok's darker one), not by a benchmark.
-- A thin feature whose colour the interior never shows can be read as a palette colour plus key and take that colour: a thin yellow line beside red hair with a pink highlight came out pink (colour error 25.8 off to 48.3 auto). `--decontam off` keeps it.
+- A thin feature whose colour the interior never shows can be read as a palette colour plus key and take that colour: a thin yellow line beside red hair with a pink highlight came out pink (colour error 26.2 off to 48.0 auto). `--decontam off` keeps it.
- A thin key-coloured detail farther than 6 px from any key-coloured interior (the far end of a green ribbon) is repainted in a palette colour.
- Even with the pass, the thinnest strands keep some olive: the decoder shares chroma across each aligned 2x2 block, so a one-pixel strand has the strand's luma but half the key's chroma, which no single blend line explains; the pass reads it as opaque and keeps it. This is most of what remains. A block-aware fit is planned separately.
-- Outside a dark outline, the flank can give a few faint pixels in the subject's colour (already described in the frame-edge paragraph).
-- Time and memory go up: keying on an Intel i5-7500 took 1.10 s a frame with `off` and 3.32 s with `auto` at 1080x1920, and peak memory rose from 389 MB to 938 MB (about 265 MB more per megapixel of frame, or about 1.6 GB more for `video-set` running three concurrent jobs).
+- Only on stills, outside a dark outline the flank can give a few faint pixels in the subject's colour.
+- Time and memory go up: keying on an Intel i5-7500 took 1.21 s a frame with `off` and 3.42 s with `auto` at 1080x1920, and peak memory rose from 396 MB to 908 MB (about 247 MB more per megapixel of frame, or about 1.5 GB more for `video-set` running three concurrent jobs).
## Related
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index b52898f..d12d655 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -690,9 +690,9 @@ reference costs more than that to judge. The report records `reference_size`,
### Edges: `--decontam`
-`--spill` fixes key colour painted into the subject, but strands and outlines at its edge are a different problem: H.264 4:2:0 chroma mixes the key into anything one to three pixels wide, leaving an olive rim on red hair or turning it orange under despill. In release 2.46.0, `video-frames` and `video-set` default to `--decontam auto` (before: `off`), which re-explains each edge pixel as a blend of the local key background with colours the subject owns and writes that colour at the pixel's observed luma, using the video fit and one palette per clip learned on the first frame. Where `auto` cannot run (a white key or a subject with no interior deeper than 6 px) and under `--alpha-mode brightness`, it leaves frames exactly as keyed and records the reason in the report. On stderr, `video-frames` prints a one-line warning (and `video-set` lists it under `warnings`) when `auto` did not run on any frame, skipped frames, found key-coloured subject material, or triggered the key-coloured part guard.
+`--spill` fixes key colour painted into the subject, but strands and outlines at its edge are a different problem: H.264 4:2:0 chroma mixes the key into anything one to three pixels wide, leaving an olive rim on red hair or turning it orange under despill. In release 2.46.0, `video-frames` and `video-set` default to `--decontam auto` (before: `off`), which re-explains each edge pixel as a blend of the local key background with colours the subject owns and writes that colour at the pixel's observed luma, using the video fit and one palette per clip learned on the first frame. The video fit recovers no coverage on keyed pixels, examining band pixels only; across frames in a clip, near-threshold choices are held and still pixels are settled toward the fit to eliminate edge sparkle. Where `auto` cannot run (a white key or a subject with no interior deeper than 6 px) and under `--alpha-mode brightness`, it leaves frames exactly as keyed and records the reason in the report. On stderr, `video-frames` prints a one-line warning (and `video-set` lists it under `warnings`) when `auto` did not run on any frame, skipped frames, found key-coloured subject material, or triggered the key-coloured part guard.
-Two video guards keep the matte's bytes for edge pixels beside small key-coloured parts and for pixels matching the material right behind them. Pass `--decontam off` to keep the matte as it is and return 2.45.3 bytes exactly. Keying time roughly triples (1.10 s to 3.32 s a frame on an Intel i5-7500 at 1080x1920) and peak memory rises by about 265 MB per megapixel of frame. Method, guards and measurements: [chroma-alpha.md](chroma-alpha.md#decontam--give-the-edge-the-subjects-own-colour-back).
+Two video guards keep the matte's bytes for edge pixels beside small key-coloured parts and for pixels matching the material right behind them. Pass `--decontam off` to keep the matte as it is and return 2.45.3 bytes exactly. Keying time roughly triples (1.21 s to 3.42 s a frame on an Intel i5-7500 at 1080x1920) and peak memory rises by about 247 MB per megapixel of frame. Method, guards and measurements: [chroma-alpha.md](chroma-alpha.md#decontam--give-the-edge-the-subjects-own-colour-back).
## 3b. Canvas shape for raised limbs and wide costumes