Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
docs: chroma keying follows the painted background colour; canvas normalizes the key
Soohong Kimbba2e2d528e8
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a8c72f9..7925a97 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,12 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## Unreleased (v2.1.1)++- Chroma keying measures the key distance from the background colour the generator actually painted (detected from the flat borders, `detect_background_key_rgb`) as well as from the pure key, in `cutout`, `extract`, `slice-sheet` and `video-frames`. Image models paint `#00FF00` as (8, 162, 24) and darker; that colour sat on the 96 radius from pure green and keyed half-and-half pixel by pixel. The radius is unchanged. Reports carry `chroma_key_painted`. Documented cost: key-family dark subject colours are erased on darker painted backgrounds — choose the key away from the subject's hues.+- `video-canvas` normalizes a green/magenta still's flat background to the exact declared key and pads with that key (`--key auto|green|magenta|white`; the repaint mask is the `cutout` matte's own alpha-0 set, the subject is byte-identical). The report records `key`, `key_painted`, `normalized_px`.+- `video-frames` tells leftover key background at the frame edge (`residual`, points at `video-canvas`) from a subject that was framed too tight (`subject`, points at a taller/wider canvas) instead of reporting both as "framed too tight".+## v2.1.0 - Guided requests and saved defaults- Added two user journeys: sprites (base-image provider, then GPT rows or Grok video) and ordinary images (provider). The read-only `workflow` command checks credentials, resolves request choices over saved defaults and returns the next questions and existing engine route.diff --git a/docs/chroma-alpha.md b/docs/chroma-alpha.mdindex e86a6e9..00e8058 100644--- a/docs/chroma-alpha.md+++ b/docs/chroma-alpha.md@@ -38,7 +38,7 @@ The close-up crops below show the edge detail behind the full-body comparisons.`prepare_sprite_run.py` chooses a chroma key by sampling the base image unless the request forces one. The generated character must not use the chroma color or chroma-adjacent colors.-**Choose the chroma away from the subject's dominant hue — do not blindly default to magenta.** The hard key erase is a color-distance ball around the key (`--key-threshold`, default 96): a subject color inside that radius is deleted no matter where it sits. Antialias fringe is no longer erased by a separate peel; key/subject boundary blends are **unmixed into despilled RGB + partial alpha** when they are close enough to the keyed region. In-band blends are limited to key-distance `<= 2`, out-of-band blends use `--fringe-unmix-reach` (default 4), and deeper key-tinted *interior* subject colors (hot pink / purple under a magenta key) survive byte-identical. Small strongly key-tinted clusters buried inside the subject (generator spill) are despilled in place — see below. Key choice still decides quality:+**Choose the chroma away from the subject's dominant hue — do not blindly default to magenta.** The hard key erase is a color-distance ball (`--key-threshold`, default 96) around the key **and** around the background colour the generator actually painted (detected from the flat borders, `detect_background_key_rgb`; since 2026-09-11 — generators paint `#00FF00` as (8, 162, 24) or darker and the pure-key ball alone missed it): a subject color inside either radius is deleted no matter where it sits, so the darker the generator painted the key, the deeper into the key's dark variants (olive under green, plum under magenta) the erase reaches. Antialias fringe is no longer erased by a separate peel; key/subject boundary blends are **unmixed into despilled RGB + partial alpha** when they are close enough to the keyed region. In-band blends are limited to key-distance `<= 2`, out-of-band blends use `--fringe-unmix-reach` (default 4), and deeper key-tinted *interior* subject colors (hot pink / purple under a magenta key) survive byte-identical. Small strongly key-tinted clusters buried inside the subject (generator spill) are despilled in place — see below. Key choice still decides quality:- Pink / purple / magenta-family subjects (꽃, 씨앗봉투) → use **green** `#00FF00`; their edge blend under a magenta key wastes real silhouette pixels.- Deep-red / crimson / wine hair or clothing is **magenta-adjacent** (both high R). Use **green** for red/crimson/warm subjects.@@ -54,7 +54,7 @@ When unsure, let `--chroma-key auto` sample the base. It excludes the detected f`extract_sprite_row_frames.py` owns alpha cleanup for sprite rows. Three passes, in order:-1. **Hard key cut** — pixels within `--key-threshold` of the key (and already-transparent input) are erased; alpha=0 pixels get their RGB cleared to `(0,0,0)` (no halo).+1. **Hard key cut** — pixels within `--key-threshold` of the key *or* of the painted background colour detected on the borders (`detect_background_key_rgb`; `background_key=` pins it explicitly) — and already-transparent input — are erased; alpha=0 pixels get their RGB cleared to `(0,0,0)` (no halo).2. **Soft-alpha unmix** — key-tinted blends near the keyed region are separated into despilled RGB + **partial alpha**: the blend model `observed = (1-k)·subject + k·key` is solved from the key-tint score, so antialiased silhouettes keep their coverage ramp instead of collapsing into a binary staircase. In-band blends (inside `--fringe-key-threshold`) are eligible only at key-distance `<= 2`; out-of-band blends use `--fringe-unmix-reach` (default 4). This preserves deeper key-tinted material while fixing trapped blend pockets between hair strands.3. **Trapped-spill despill** — a small connected cluster of key-tinted pixels (≤ `--spill-max-fraction` of the subject, default 0.005) containing at least one strongly tinted pixel is generator spill, wherever it sits: its color is corrected in place with alpha kept (no pinholes). Large key-tinted regions are intentional material and are never touched; marginally warm subject colors (skin) never qualify.diff --git a/docs/gen.md b/docs/gen.mdindex b50e2df..f0d14ef 100644--- a/docs/gen.md+++ b/docs/gen.md@@ -100,7 +100,7 @@ strategy it can execute, and `--alpha-mode auto` (the default) follows it:| Strategy | Who | What happens | Refused when ||---|---|---|---|| `native` | `codex` (**first choice**, 2026-09-08) | The transport prompt asks image_gen for a genuinely transparent background (the bundled `imagegen` skill honours "transparent background" and keeps the generated alpha; codex reports `transparentBackground: true` on the completed item). The decoded PNG's alpha is **measured**: no alpha band or `alpha_zero_pct: 0.0` refuses to publish, RGB under alpha 0 is scrubbed, partial alpha (1–254) is left as produced and reported as `partial_alpha_pct`. | The model drew a checkerboard / flat background (RGB image) — nothing can recover alpha from that, so the run fails instead of silently keying. |-| `chroma` | `grok` (only option), `codex` with `--alpha-mode chroma` | Generate on a `#FF00FF` (or `#00FF00`) background — pick the key by subject colour (magenta subjects → green key) — and matte it out through the frame extractor's canonical YCbCr matte (`remove_chroma_background_ycbcr`). Gradients and texture within that chroma family are supported. | `alpha_zero_pct: 0.0` after keying, or stale RGB under alpha 0. |+| `chroma` | `grok` (only option), `codex` with `--alpha-mode chroma` | Generate on a `#FF00FF` (or `#00FF00`) background — pick the key by subject colour (magenta subjects → green key) — and matte it out through the frame extractor's canonical YCbCr matte (`remove_chroma_background_ycbcr`), which keys from the background chroma it detects on the borders (`detect_background_key_ycc`) rather than from the pure key alone — the RGB matte behind `cutout`/`extract`/`slice-sheet` does the same since 2026-09-11 (`detect_background_key_rgb`), so every chroma path tolerates the slightly-off green/magenta generators actually paint. Gradients and texture within that chroma family are supported. | `alpha_zero_pct: 0.0` after keying, or stale RGB under alpha 0. |- **`auto` steps down to `chroma` when `--ref` is attached**, even on codex. Measured2026-09-08 (plan `sprite-gen/parts-rig`): codex `image_gen` with reference imagesdiff --git a/docs/sheet-slicing.md b/docs/sheet-slicing.mdindex 258c25c..1d9a5c4 100644--- a/docs/sheet-slicing.md+++ b/docs/sheet-slicing.md@@ -48,7 +48,10 @@ $SPRITE_GEN_ROOT/.venv/bin/python $SPRITE_GEN_ROOT/scripts/slice_sheet_cells.pyits height hierarchy.- Extraction tunables (`--key-threshold`, `--fringe-*`, `--spill-max-fraction`)pass through to `remove_chroma_background` with the same defaults as- `extract`.+ `extract`. The key distance is measured from the pure key *and* from the+ background colour detected on the sheet's borders, so a sheet the generator+ painted a darker green/magenta than asked still keys cleanly+ ([chroma-alpha.md](chroma-alpha.md)).## Geometry rules (and the field failures they encode)diff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 292f50b..d06f556 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -41,9 +41,21 @@ is a property of the motion state, owned by one table (`STATE_CANVAS`):| everything else | square | 1:1 | — | in-place motion fits the still |`--shape tall|wide|square` overrides the row; `--headroom` / `--lead` tune the room;-`--facing left` mirrors the wide layout. The padding is filled with the still's own-corner colour (its chroma key), and a still whose corners are not one flat colour is-refused — a non-flat background cannot be extended without guessing.+`--facing left` mirrors the wide layout. A still whose corners are not one flat colour+is refused — a non-flat background cannot be extended without guessing.++**The canvas owns key normalization.** Image models paint "`#00FF00`" a little+differently every run — (8, 166, 25) on 2026-09-11 — and the video model reproduces+the input colour almost exactly (a pure-key input came back as (16, 239, 11)). So when+the flat corners are a green/magenta key at *any* brightness (`--key auto`, the+default; `--key green|magenta` to insist, refused when the corners are not that+family), the still's background is repainted to the **exact declared key** and the+padding is that same pure key. The repaint mask is the `cutout` chroma matte's own+alpha-0 set — the pixels the canvas repaints are exactly the pixels `video-frames` will+erase, and the subject stays byte-identical. A corner that survives the matte fails loud+(the still is not on a key the engine can cut). `--key white` keeps the old behaviour: no+chroma key, the padding is the corner colour. The report records `key`, `key_painted`+(the colour the model actually used) and `normalized_px`.## 2. Clip — `sprite-gen video`@@ -59,10 +71,39 @@ quadruped and a legless blob into a contradiction (2026-09-09).`ffmpeg` extracts every frame; the clip's real fps is recorded (never assumed). Eachframe goes through the same `cutout` engine imported stills use (`--key auto` reads-the corners; green/magenta route to the extract matte). The report carries per-frame-alpha coverage and an **edge-contact check**: any opaque pixel in the top/left/right-4-pixel bands means the model framed too tight, and the run fails loud pointing at-`video-canvas`. `--allow-edge-contact` accepts the clipping on purpose.+the corners; green/magenta route to the extract matte). The matte keys from the+background colour **as the model painted it**, not only from the pure key: the flat+border colour is detected per frame (`extract.detect_background_key_rgb`, the mode of an+RGB histogram over the corner/border samples that are the key's hue family) and a pixel+is erased when it is within the key radius of *either* the pure key or that painted+colour. The 96 radius is unchanged — what moved is its centre. Before this (2026-09-11)+a (8, 162, 24) green sat at distance 96.38 from pure green while (7, 163, 24) sat at+95.34, so a clip was keyed half-and-half pixel by pixel and the edge check read the+leftover background as a clipped subject. The report records `chroma_key_painted` per+frame.++Two consequences worth knowing. Dark outline halo (the antialiased blend between subject+and a dark-painted key) is now erased with the background — on the three 2026-09-11+walk clips that was outline pixels only (0 newly opaque, interior holes ≤ 7 px per+frame, seam ratios moving in the third decimal, periods and start frames identical).+And **a key-family subject colour is at more risk the darker the painted background**:+the erase ball follows the detected key, so on a still painted (20, 120, 25) a dark+olive (40, 70, 35) inside the subject (distance 54.8) is erased, while the same olive+survives untouched on (8, 162, 24), (5, 200, 10) or pure-key backgrounds. Choose the key+away from the subject's hues ([chroma-alpha.md](chroma-alpha.md)) — that rule now covers+the painted key's darker variants too.++The report also carries per-frame alpha coverage and an **edge-contact check**: any+opaque pixel in the top/left/right 4-pixel bands fails the run. Each contact pixel is+classified by its *raw* colour — the declared key's hue family is **`residual`**+(background the matte did not erase), anything else is **`subject`** — and the two+defects fail with different messages: residual-only contact points at `video-canvas`+(normalize the base still and regenerate); subject contact means the model framed too+tight and points at a taller/wider canvas. When both occur the message names both.+`residual` can also be reported on the antialiased fringe where a subject genuinely+touches the edge (a key-tinted blend pixel with low alpha), so a "framed too tight"+message with a small residual count is still a framing problem, not a key problem.+`--allow-edge-contact` accepts the clipping on purpose.## 4. Loop — period first, seam second