Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

docs(frames): the drawing's own border, the body's rim and what a refused run leaves — video-pipeline sections 3 and 4, video-frames --out-dir help, CHANGELOG Unreleased (2.46.0 default changes and their tests)

Soohong Kimf01a7c14ca05
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 02cf61e..888d0e7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,9 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
## Unreleased
+- **Default change: A drawing cut by its own frame is no longer refused as framed too tight.** A bust or half-length drawing runs off the frame itself (its sleeve cut off at the left, its hair at the top), so in every frame of its clip the subject touches those edges and no canvas gives it room there. In 2.45.2 the default `video-frames` edge check refused every frame of such a clip with "the subject touches the frame edge ... framed too tight". Only a flag let it through: `--allow-subject-edge-contact`, which also lets through a subject that really leaves the frame, or `--allow-edge-contact`, which turns the check off. The default (`edge_policy` is now `border`, was `refuse`) reads the drawing's own border on the clip's first frame, keyed with its specks erased: a side (top, left, right) whose outermost row or column the subject covers for at least 2 % of its length (`frames.BORDER_MIN_FRACTION`) is a border side. Subject contact there is accepted in every frame, wherever along the side. Subject contact on any other side is still refused as framed too tight (a bust cut at the top and left whose hand leaves the right). The top / left / right numbers in a refusal are now the pixels the border does not take, so a caller that reads them to choose a taller or wider canvas reads the side that needs room. Leftover key is still refused, except the body's rim the matte cut loose: a key-only piece is accepted only on a border side the same frame's subject reaches, holding fewer than 8 pixels in all (`SPECK_MIN_PX`) and lying wholly within 10 px of the frame's body (`frames.RIM_REACH`, counted each way and diagonally; the body is the largest piece holding a pixel off the key). A thin outline blended with the key leaves such pieces at the edge between strands. Larger pieces, farther pieces, pieces on other sides or on a side the frame's subject left, and a detached stain beside a sleeve on the same side are refused as in 2.45.2. A clip whose first frame crosses no side is judged and worded exactly as in 2.45.2, byte for byte. The two flags judge as in 2.45.2. Taking a side as the border prints one stderr line. The report gains `edge_border` (`frame`, `sides`, `outer_px`, `min_fraction`, `rim_reach`, `rim_under_px`); each contact row gains `rim`, and under the default `border` (pixels taken) and `off` (pixels not taken, by side). Known limit: a hand that newly leaves the frame on a side that is already a border side is accepted; by position it cannot be told from the sleeve moving along the edge. [video pipeline](docs/video-pipeline.md), section 3.
+- **Default change: A refused run hands nothing over.** A caller that does not read the exit code used a refused run's output as if it had passed. In 2.45.2 a refused `video-frames` run left its keyed frames in `keyed/` and wrote no report. `keyed/` was not emptied before keying, so a pass after a longer earlier run in the same folder also held the earlier run's extra frames. A refused `video-loop` run could leave a mixed set: for example after the pin gate refused, `cycle/` and the strip were the refused run's while the GIF and WebP were an earlier run's; a refusal before the cut left the earlier run's whole set. Now `video-frames` keys into a hidden working folder and hands over `keyed/` only when the run passes. A refused run (the edge check, or a frame keyed to 0 % transparency) leaves its frames as `keyed.refused/` and no `keyed/`, not even an earlier run's. A pass removes `keyed.refused/`. The report is written either way, with `status` (`passed` or `refused`); a refused report adds `error`, `refused_dir` and `keyed_dir: null`. Keyed frames that pass are the same bytes as before. Every `video-loop` refusal that writes `status: failed` (selection, motion anchor, jump repair, jolt, pin, seam, a strip with no body, a GIF or WebP that could not be written or verified) removes `cycle/`, `<name>.strip.png`, `<name>.strip.json`, `<name>.gif`, `<name>.webp`, `cycle.source/` and `follow.source.png` from the output folder. A cut it had already written is kept as `cycle.refused/`, which the next pass removes. A strip with no body and a GIF or WebP that could not be written now also write the failed report. Argument errors touch nothing. [video pipeline](docs/video-pipeline.md), sections 3 and 4.
+- **Tests** - `tests/video/test_edge_border.py` (a bust cut at the top and left passes and the report names its border; a padded jump whose head leaves the top and a first frame with a 2 px wisp on the top are refused with the 2.45.2 message, byte for byte; a key-only patch with no border is worded as in 2.45.2; a hand leaving the right of a bust is framing, its message counting only the right; a hand leaving a border side elsewhere is accepted (the known limit); leftover key that is not the body's rim is refused (a patch on the right, a 48 px fringe beside the sleeve, a patch where the sleeve left, a detached 20 x 90 stain, 2 px pieces 11 and 14 px from the sleeve, pieces of 8 px and more, stripes hugging the head, stains beside the head, a fringe on a side that is not the border, a fringe above a head clear of the top band, a stain round a fleck of the body's colour); 1 px dots are judged piece by piece; 2 px pieces 2 to 10 px from the sleeve and a 7 px piece are the rim; an 8 px piece with 4 pixels in the band and a 7 px piece 11 px away are refused; the two flags judge as in 2.45.2; `classify_edge_contact` counts the rim; a refused `video-frames` run leaves `keyed.refused/`, no `keyed/` and a refused report; a pass after a refusal leaves `keyed/` alone; a pass over a longer run holds its own frames only; a frame keyed to 0 % transparency is refused the same way; an argument error touches nothing; a `video-loop` cut refused after it was written hands nothing over and keeps `cycle.refused/`, removed by the next pass; a cut refused before it was written hands nothing over; a loop argument error touches nothing), `tests/video/test_canvas_key_normalize.py` and `tests/video/test_fit_tight.py` (the 2.45.2 default's framing and mixed-contact tests now move the contact after the first frame and read the refused frames from `keyed.refused/`), `tests/video/test_edge_speck_lead_in.py` (the default policy is named `border`, and the refused frames are read from `keyed.refused/`).
- **Fix: Green screens that lean toward cyan or yellow are now recognised as border keys.** A green screen that leans toward cyan or yellow was not recognised as the key on the border. Examples: the studio chroma-key green `#00B140`, (0, 177, 64); (0, 200, 90), which leans toward cyan; (90, 210, 0), which leans toward yellow. Each sits more than 96 from pure green. With `--key green` the whole background stayed opaque, and `video-canvas` refused `--key green` on such a still; but with `--key auto` the old rule did not take the green route at all: `cutout` and `video-frames` routed the frame to the white (position-based) matte, and `video-canvas` treated the still as a non-key base and did not normalize it. `video-frames --spill auto` did not always choose `full`: when the reference still had the same leaning green, `auto` read the surviving background as key material and chose `small`; when it chose `full` (a reference whose own background the old rule did recognise), the clip's surviving background was despilled to a blue-grey instead of being erased. The cause was that the border rule `extract.is_border_key_candidate` held every unkeyed channel under 64. That bar was set between magentas, whose one unkeyed channel (green) tells a painted magenta from hot pink. Green has two unkeyed channels, and a green screen leans on one of them. For the fix, the darkest unkeyed channel keeps the bar (under 64, and at most 35 % of the brightest keyed channel). The brightest one only has to stay under half of the brightest keyed channel (`extract._BORDER_KEY_MAX_LEAN_RATIO`, 0.5), that is a hue nearer green than yellow or cyan (between 90 and 150 degrees). The border detector, `extract.detect_background_key_rgb`, reads the lean second. It first counts, on their own, the candidates the 2.45.2 rule admitted. When those make up at least 12% of the samples, it returns the mean of their most populated bin, exactly the colour 2.45.2 returned. Below 12%, the samples only the lean admits join them, but their most populated bin gives the colour only where the declared key's own cut would leave that background on the border. Two conditions, both needed: the samples only the lean admits that lie farther from the declared key than the cut (the radius the caller keys with: the new `threshold` argument of `detect_background_key_rgb`, which the engine sets from `--key-threshold`, 96 by default), in that most populated bin or in a bin beside it on every channel, make up at least 2% of the samples; and the candidates outside that bin and the bins beside it make up less than 12% of the samples (the border holds one key-coloured background). Otherwise the declared key is kept, as in 2.45.2. A leaning border the declared key clears by itself gains nothing from the leaning colour; its ball would only reach further into a subject near that colour: (0, 210, 80) sits 91.8 from pure green, inside the cut of 96, so the declared key erases that background on its own, and a teal subject (0, 154, 149), 180 from the key, is kept; read as the painted colour, (0, 210, 80), 88.9 from the teal, let the ball reach the teal subject through the background and erase all of it, with no background gained. The 2% keeps the soft rim where a subject meets an edge of the frame (its blend with the background crossing the cut there) from being read as background the cut left; and a background whose tail past the cut is under 2% of the border keys as before the lean, the few pixels past the cut included. A second key-coloured colour on the border may be the subject's own (a leaf along an edge, another green); read as part of the background it would let the ball take a subject colour near the dominant one. `cutout`, the reference window `video-frames --spill auto` keys, and the YCbCr path's decontamination pass hand the detector the radius they key with, as the engine does. The painted key's erase ball, the spill judgment's subject window and the `video-frames` edge-contact split take the lean only from a painted colour that leans itself (`extract._painted_key_leans`). A colour found the first way never leans itself (magenta has no lean; for green the floor of the mean of one 8-wide bin of 2.45.2 candidates is always one of them), so they read it as 2.45.2 did. `cutout --key auto` and `video-canvas --key` classify the flat corner colour with the border rule, lean included (there the question is what the painted colour can be): a corner colour 2.45.2 took routes as before, and a leaning corner colour now routes to green. The magenta key is exactly as before (its one unkeyed channel is both the darkest and the brightest), so hot pink (250, 77, 150) and purple (213, 112, 246) are still refused. For green, nothing that was a candidate is dropped. Ivory, cyan, teal, sea green, a grey-green with both unkeyed channels lit and a green as near cyan as green are still refused. The interior rule `is_key_family` is unchanged. As a consequence, every image whose border the 2.45.2 rule recognised (its candidates at least 12% of the corner and border samples) keys byte for byte as in 2.45.2 under the same key, whatever leaning green is mixed into that border and in whatever share. Under the same key, every image is keyed either exactly as in 2.45.2 or exactly as with the leaning colour read. As in 2.45.2: wherever the 2.45.2 rule recognised the border (its candidates at least 12% of the corner and border samples), wherever the declared key's cut clears a leaning border by itself, and wherever the border holds a second key-coloured colour. With the leaning colour: where the key's cut leaves the border's one leaning background. Read together, a sample admitted only through the lean can share the 8-wide bin of a background 2.45.2 already found and move its mean, while leaning samples in a bin of their own that outnumber the background would take the mode. A leaning-green part of the subject on the corners and the border can be taken as the painted colour only where the 2.45.2 rule's candidates make up less than 12% of the samples, the key's cut leaves it and no other key-coloured colour holds 12% of the border, that is, only where the subject crowds the border. A leaning-green patch buried in the subject on such a background is kept. [chroma-alpha](docs/chroma-alpha.md), the 2026-10-21 paragraph.
- **Tests** - `tests/frames/test_chroma_key_relative.py` (the border table with the three leaning greens accepted and a grey-green and a cyan-midpoint green refused; the detector still refuses hot pink, purple, ivory, cyan, (30, 60, 30), teal, sea green and those two; over the whole RGB cube the lean adds nothing for magenta, drops nothing for either key, and adds for green only hues between 90 and 150 degrees; without the lean the candidate field is the old rule colour for colour over the whole RGB cube; the engine and `cutout --key green` key the leaning greens out with the subject byte-identical; `cutout --key auto` routes a leaning green to `extract:green`; a leaning-green patch ((0, 160, 75) or (10, 120, 50)) buried in the subject is kept byte for byte on (8, 162, 24) and keyed with the background on (0, 200, 90); the edge-contact split reads a leaning piece as residual when the frame was keyed from a leaning background and as the subject's when it was keyed from (8, 162, 24); on (0, 200, 64) with a leaning green, (0, 200, 71) in its bin or (0, 200, 90) in a bin of its own, over the top-left corner (20.5% of the samples) or everywhere but a 32 x 32 block of the background (77.3%), the detector returns (0, 200, 64) and the frame keys exactly as from that colour pinned, a teal subject (0, 154, 149) just outside the ball keeping every pixel; a leaning-green border on (0, 200, 90), past the cut, with no 2.45.2 candidate, or with a block of one under 12% of the samples, is found through the lean and keys out; and over every 8-wide bin of green, the floor of the mean of 2.45.2 candidates is one of them; on (0, 210, 80), inside the key's cut, with no 2.45.2 candidate on the border or with 11% of the samples an interior-family green, the detector keeps the declared key and the frame keys exactly as with the key pinned, a teal subject (0, 154, 149) losing no pixel; a leaning-green leaf of the subject along the bottom edge over a leaning background the key clears, a second leaning green in a corner, and a soft rim crossing the cut where the subject meets the side edges each keep the declared key; with 1.5% of the border a step past the cut the key is kept, and with 2.5% the leaning colour is read; (0, 200, 90) is read with a cut of 96 and not with a cut of 110; over a sweep of borders around the cut every answer is the 2.45.2 answer or the answer that reads the lean, and both occur, as do borders where the two differ and the key is kept), `tests/frames/test_key_detect_banded.py` (the whole-image reference now states this rule, and leaning-green borders (past, across and inside the cut, and two leaning greens on one border) join the images the banded detector is checked against), `tests/video/test_canvas_key_normalize.py` (`video-canvas` repaints a leaning-green still to the exact key and accepts `--key green` on it), and `tests/video/test_video_spill.py` (on a leaning green, `--spill auto` reads the reference as `full` with no key material, and the frame keys with no opaque background pixel and no edge contact, the subject byte-identical, with `--decontam off` and `palette`).
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 872c824..226b1b3 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -20,7 +20,7 @@ still ──video-canvas──▶ canvas.png ──video──▶ clip.mp4 ─
| `sprite-gen video-canvas` | `sprite_gen/video/canvas.py` | still → padded still (state canvas) + report |
| `sprite-gen video` | `sprite_gen/gen/video.py` ([gen](video.md)) | still + prompt → mp4 + report |
| `sprite-gen video-prompt` | `sprite_gen/video/clip_prompt.py` | (direction, state) → the prompt `video-set` sends, for a clip made by a video MCP on your agent ([below](#a-clip-from-a-video-mcp-on-your-agent--zcre)) |
-| `sprite-gen video-frames` | `sprite_gen/video/frames.py` | mp4 → `raw/`, `keyed/` RGBA frames + report |
+| `sprite-gen video-frames` | `sprite_gen/video/frames.py` | mp4 → `raw/`, `keyed/` RGBA frames (`keyed.refused/` when refused) + report |
| `sprite-gen video-loop` | `sprite_gen/video/loop.py` | keyed frames → `cycle/`, `<name>.strip.png` + `.strip.json`, `<name>.gif`, `<name>.webp` + report |
| `sprite-gen video-set` | `sprite_gen/video/batch.py` | bases × states → one folder per item, `set.report.json`, `table.md` |
| `sprite-gen video-cycle-align` | `sprite_gen/video/align.py` | the loop directories of one walk or run, one per direction → one cycle length, each loop turned to start on a foot strike, + report ([loop repair](loop-repair.md) section 4) |
@@ -618,7 +618,7 @@ subject framed too tight, and `--anchor motion-auto`, which never cleans a frame
the loop's cells and widened them to reach it.
The report also carries per-frame alpha coverage and an **edge-contact check**, read after the
-specks are gone: any opaque pixel in the top/left/right 4-pixel bands fails the run. Each contact pixel is
+specks are gone: any opaque pixel in the top/left/right 4-pixel bands is contact. Each contact pixel is
classified by its *raw* colour and the piece of the keyed frame it is in (its opaque pixels, corners
joining them, as the specks are read): the declared key's hue family in a piece with nothing else in
it is **`residual`** (background the matte did not erase); anything else is **`subject`** — a
@@ -637,7 +637,65 @@ and still fails a frame where the key alone reaches the edge, which is a keying
whatever the framing. A frame where the subject touches the edge passes with the
key-tinted pixels beside it (fringe, or the key reflected on metal): the same reading the
refusal message gives a mixed contact. Contacts stay in the report either way, with `edge_policy` saying which rule
-applied (`refuse`, `subject-allowed`, `off`).
+applied (`border`, `subject-allowed`, `off`).
+
+**A drawing cut by its own frame** (2.46.0). A bust or half-length drawing runs off the frame
+itself (its sleeve is cut off at the left, its hair at the top). In every frame of its clip the
+subject touches those edges, and no taller or wider canvas gives it room there. The old default
+refused every frame of such a clip as "framed too tight". The new default, `edge_policy: "border"`,
+reads the drawing's own border on the clip's first frame (keyed, its specks erased). A side (top,
+left or right) is a border side when the subject's pixels on that side's outermost row or column
+cover at least 2 % of the side's length (`BORDER_MIN_FRACTION`, 0.02). A strand's tip grazing the
+edge covers less than that; a drawing cut there covers far more. A pixel that is only near the
+edge, inside the 4-pixel band but not on the outermost line, does not make a border.
+
+Subject contact on a border side is accepted in every frame, wherever it is along that side.
+Subject contact on any other side is still the clip framed too tight and refused, with the same
+message as before. Example: a bust cut at the top and left whose hand leaves the right is
+refused. The numbers in a refusal message (top / left / right px of the worst frame) are now the
+pixels the border does not take, side by side. So a caller that reads those numbers to choose a
+taller or wider canvas reads the side that needs room, not the border sides.
+
+Leftover key background is still refused, with one exception: the body's rim the matte cut
+loose. A thin outline drawn with the key's colour blended into it carries the key's hue, so the
+matte erases most of it and can leave a few pixels at the edge between strands of hair, on their
+own; read by colour they are residual. A key-only (residual) piece is accepted only when all of
+these hold: it is on a border side; the same frame's subject reaches that side's band; the whole
+piece holds fewer than 8 pixels (`SPECK_MIN_PX`, the speck size); and every pixel of the piece
+lies within 10 px of the frame's body (`RIM_REACH`). Distance is counted as a square: each way
+and diagonally one pixel is 1. The body is the frame's largest piece that holds a pixel off the
+key's colour, as the speck pass reads the largest piece as the body. The whole piece counts, in
+the band and out of it: an 8-pixel piece with 4 of its pixels in the band is refused. Refused as
+before, for example: a piece of 8 pixels or more; a piece with a pixel farther than 10 px from
+the body; a piece on a side that is not a border side; a piece on a border side the frame's
+subject has left; a stain near a small fleck of the subject's colour but far from the body (the
+reach is to the body, not to any piece of the subject); and a detached stain 30 rows above a
+sleeve on the same side. A corner pixel lies in two bands; it is accepted only when both sides
+accept it. `--allow-subject-edge-contact` and `--allow-edge-contact` judge exactly as before.
+
+A clip whose first frame crosses no side has no border: it is judged and its messages are
+worded exactly as before, byte for byte. When a side is read as the border, `video-frames`
+prints one line on stderr naming the sides and pointing at `edge_border` in the report. The
+report gains `edge_border` (under the default only): `frame` (the first frame's file name),
+`sides`, `outer_px` (the subject's pixels on each side's outermost line), `min_fraction`
+(0.02), `rim_reach` (10), `rim_under_px` (8). Each contact row gains `rim` (residual pixels
+that are the body's rim, whatever side), and under the default `border` (the pixels the border
+took) and `off` (those it did not take, by side: the numbers the refusal message gives). The
+existing row keys stay. Known limit: on a side that is already a border side, a hand that
+newly leaves the frame there is accepted, because by position it cannot be told from the sleeve
+moving along the edge.
+
+**What a refused run leaves** (2.46.0). A caller that does not read the exit code used to find
+a refused run's keyed frames in `keyed/` and pass them on. Now `keyed/` exists only after a run
+that passed. The frames are keyed in a hidden working folder beside it, renamed `keyed/` when
+the run passes, and left as `keyed.refused/` when it is refused (by the edge check, or by a
+frame keyed to 0 % transparency). A refused run leaves no `keyed/`, not even an earlier run's:
+the earlier one is removed when the run starts reading the clip. A pass removes
+`keyed.refused/`. A pass after a longer earlier run no longer finds the earlier run's extra
+frames in `keyed/`. The report (`frames.report.json`, or `--report`) is written either way. It
+gains `status`: `passed`, or `refused` with `error` (the message), `keyed_dir: null` and
+`refused_dir` (the `keyed.refused/` path). A run stopped by an argument error (an unknown flag
+value, a missing clip) touches nothing.
## 3a. Spill — key colour the model painted into the subject
@@ -1022,6 +1080,18 @@ cycle is known (the 2026-09-09 reel jump: 2.3 hops in 145 frames). It is an expl
instruction, not a failover: the report says `kind = "fixed"`, and the seam gate still
applies.
+Every `video-loop` refusal that writes `status = "failed"` (selection, the motion anchor, the
+jump repair, the jolt gate, the pin gate, the seam gate, a strip with no body to crop, and a
+GIF or WebP that could not be written or failed its verification) now removes what an earlier
+cut handed over in the output folder: `cycle/`, `<name>.strip.png`, `<name>.strip.json`,
+`<name>.gif`, `<name>.webp`, and `cycle.source/` and `follow.source.png`, which a later
+alignment and follow-through keep beside them. Before this, a refused cut could leave a mixed
+set: the refused run's `cycle/` and strip beside an earlier run's GIF and WebP. A cut the
+refused run had already written (a strip with no body, the pin and seam gates, the GIF and
+WebP) is kept as `cycle.refused/`, to look at. The next cut that passes removes it; a refusal that comes before
+the cut removes an earlier one. An argument error touches nothing. The report is written as
+before, with `status = "failed"`.
+
`--cycle pinned` is for a clip pinned to end on its first frame (`video --last-frame` set to
the start image). The cycle is every frame but the last, which re-renders the first, so the
wrap plays like the step into that last frame. The gate is the pin, not the seam ratio: the
diff --git a/sprite_gen/video/frames.py b/sprite_gen/video/frames.py
index 16a1d31..4e7ad21 100644
--- a/sprite_gen/video/frames.py
+++ b/sprite_gen/video/frames.py
@@ -693,7 +693,7 @@ def run_frames(clip: Path, out_dir: Path, *, key: str, allow_edge_contact: bool,
def add_arguments(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--alpha-mode", choices=("chroma", "brightness"), default="chroma", help="chroma key (default), or emitted light on black: brightness alpha, unpremultiplied RGB, no speck/edge cleanup; dark smoke becomes faint")
parser.add_argument("--clip", required=True, type=Path, help="mp4 from `sprite-gen video`, or one made elsewhere from a `video-canvas` still (a video MCP such as ZCRE); an audio track and a cover image are left out")
- parser.add_argument("--out-dir", required=True, type=Path, help="writes raw/ and keyed/ here")
+ parser.add_argument("--out-dir", required=True, type=Path, help="writes raw/ and keyed/ here; a refused run leaves keyed.refused/ and no keyed/")
parser.add_argument("--key", choices=("auto", "green", "magenta", "white"), default="auto", help="background key (auto reads the corners)")
parser.add_argument("--allow-edge-contact", action="store_true", help="accept any opaque pixel at the top/left/right edge (subject and leftover key background alike)")
parser.add_argument("--allow-subject-edge-contact", action="store_true", help="accept the subject touching the top/left/right edge (a `video-canvas --fit tight` clip) but still refuse leftover key background there")