Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
fix(video): a key-tinted rim joined to the subject at the edge is the subject
Soohong Kim9c23deeec847
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e7a7245..8bd610f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,9 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the## Unreleased+- **Fix: a key-tinted rim joined to the subject at the edge is the subject.** The edge check split its contact pixels by their raw colour alone: the key's hue was leftover background. A body that reaches an edge band brings its antialiased rim there first — hair tips, the crown's outline, the key blended into them — and on a frame where only that rim was in the band, `--allow-subject-edge-contact` refused the clip as leftover background. Each key-tinted contact pixel is now read with the piece of the keyed frame it is in (its opaque pixels, corners joining them, the pieces `drop_specks` reads): it is `residual` only when nothing in its piece is off the key's hue — background on its own — and `subject` when it is joined to the subject. A frame with such a rim in the band passes with `--allow-subject-edge-contact` and is recorded as the subject's contact; refused by default, it says the subject was framed too tight, and a mixed refusal no longer counts the rim's frames as carrying leftover background. Key-coloured background apart from the body is refused as before. A clip with no key-tinted pixel in a band, or none joined to the subject, keys and reports as before, byte for byte. [docs/video-pipeline.md](docs/video-pipeline.md#3-frames--extract-key-check-the-edges).+- **`tests/video/test_edge_speck_lead_in.py`**: a body that rises until only the key-tinted rim over its crown is in the top band was refused as leftover background with the subject allowed; now it passes as the subject's contact, and by default is refused as framed too tight. A strip of the key's colour against the edge, apart from the body, is still refused as leftover background. `tests/frames/test_chroma_key_relative.py`: Grok's magenta in the band is residual cut off from the rest, and the subject's when joined to it.+- **Output change: `video-canvas --shape wide` gives a walk or run no head-room.** A forced-wide canvas gave every state a jump's 35 % above the still, and above a square still that head-room set the width as well (2800 × 1575 for 1024 × 1024, 1216 columns in front of the still); a walker that small in its frame is one a video model may reframe. A walk or run forced wide now has a row of its own (`canvas.GAIT_WIDE`, read through `STATE_SHAPE_CANVAS`): 28 % in front, 20 % behind, nothing above — 1969 × 1108 for a 1024 × 1024 still. A jump, and every other state forced wide, keeps the forced-wide row (`WIDE_OVERRIDE`) and its canvas byte for byte; `--headroom` still sets the room above. [docs/video-pipeline.md](docs/video-pipeline.md#1-canvas--the-input-frame-decides-the-output-frame).- **`tests/video/test_edge_speck_lead_in.py`**: a walk or run forced wide is 1969 × 1108 with nothing above, a jump still 2800 × 1575.diff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 8810850..c599ba7 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -586,8 +586,12 @@ the loop's cells and widened them to reach it.The report also carries per-frame alpha coverage and an **edge-contact check**, read after thespecks are gone: any opaque pixel in the top/left/right 4-pixel bands fails the run. Each contact pixel is-classified by its *raw* colour — the declared key's hue family is **`residual`**-(background the matte did not erase), anything else is **`subject`** — and the two+classified by its *raw* colour and the piece of the keyed frame it is in (its opaque pixels, corners+joining them, as the specks are read): the declared key's hue family in a piece with nothing else in+it is **`residual`** (background the matte did not erase); anything else is **`subject`** — a+key-tinted pixel joined to the subject included, since a body that reaches a band brings its+antialiased rim, the key blended into it, there first, and on a frame where only that rim is in the+band, read by its colour alone, the clip was refused as leftover background. The twodefects fail with different messages: residual-only contact points at `video-canvas`(normalize the base still and regenerate); subject contact means the model framed tootight and points at a taller/wider canvas. When both occur the message names both.diff --git a/sprite_gen/video/frames.py b/sprite_gen/video/frames.pyindex beee2eb..f7c0777 100644--- a/sprite_gen/video/frames.py+++ b/sprite_gen/video/frames.py@@ -14,8 +14,10 @@ a little off and part of that background survives the matte, the edge band is"touched" by leftover *background* (2026-09-11: a (8, 162, 24) green read as"framed too tight"). The check therefore classifies every contact pixel by itsraw colour — the declared key's hue signature (`extract.is_border_key_candidate`,-the border rule: an edge pixel is border evidence) is **residual background**,-anything else is the **subject** — and the two fail+the border rule: an edge pixel is border evidence) in a piece of the keyed frame that+holds nothing else is **residual background**; anything else is the **subject**, a+key-tinted pixel joined to the subject included (its antialiased rim, the key blended+into it, reaches the band first) — and the two failwith different messages: residual points at `video-canvas` normalization of thebase still, subject contact at a taller/wider canvas.@@ -51,7 +53,7 @@ from PIL import Imagefrom sprite_gen._deps import npfrom sprite_gen.frames.cutout import cutoutfrom sprite_gen.frames.decontam import palette_from_stats-from sprite_gen.frames.extract import is_border_key_candidate+from sprite_gen.frames.extract import _border_key_candidate_field, _key_channel_splitfrom sprite_gen.frames.extract import _SPILL_FULL_MIN_TINT, DEFAULT_UNMIX_REACHfrom sprite_gen.spec.runio import atomic_write_text@@ -236,26 +238,37 @@ def classify_edge_contact(raw: Image.Image, keyed: Image.Image, chroma_key: tupl"""Split the opaque edge-band pixels into `subject` and `residual` (leftover key background).A contact pixel whose *raw* colour carries the declared key's hue signature- (`is_border_key_candidate`) is background the matte failed to erase, not the- subject. Without a chroma key+ (`is_border_key_candidate`) is either background the matte failed to erase or the subject's+ own rim: a body reaching the band brings its antialiased edge, the key blended into it, there+ first. It is residual only when the opaque piece of the keyed frame it belongs to — its opaque+ pixels, corners joining them, the pieces `drop_specks` reads — holds no pixel off the key's+ signature: background on its own. Joined to one, it is the subject's. Without a chroma key(matte route) every contact is the subject."""- alpha = keyed.convert("RGBA").getchannel("A").load()- rgb = raw.convert("RGB").load()- w, h = keyed.size- band = set()- band.update((x, y) for y in range(min(EDGE_ROWS, h)) for x in range(w))- band.update((x, y) for x in range(min(EDGE_ROWS, w)) for y in range(h))- band.update((x, y) for x in range(max(0, w - EDGE_ROWS), w) for y in range(h))- subject = residual = 0- for x, y in band:- if alpha[x, y] <= 0:- continue- if chroma_key is not None and is_border_key_candidate(rgb[x, y], chroma_key):- residual += 1- else:- subject += 1- return {"subject": subject, "residual": residual}+ alpha = np.asarray(keyed.convert("RGBA"))[..., 3]+ h, w = alpha.shape+ band = np.zeros((h, w), dtype=bool)+ band[:EDGE_ROWS, :] = True+ band[:, :EDGE_ROWS] = True+ band[:, max(0, w - EDGE_ROWS):] = True+ opaque = alpha > 0+ contact = band & opaque+ total = int(contact.sum())+ keyed_channels, unkeyed_channels = _key_channel_split(chroma_key) if chroma_key is not None else ([], [])+ if not total or not keyed_channels:+ return {"subject": total, "residual": 0}+ keyish = _border_key_candidate_field(np.asarray(raw.convert("RGB"), dtype=np.int32), keyed_channels, unkeyed_channels)+ suspect = contact & keyish+ if not suspect.any():+ return {"subject": total, "residual": 0}+ ys, starts, stops, labels, _ = _components(opaque, diagonal=True)+ piece = np.zeros((h, w), dtype=np.int64)+ for y, x0, x1, label in zip(ys, starts, stops, labels):+ piece[y, x0:x1] = label+ owned = np.zeros(len(ys), dtype=bool) # pieces holding a pixel off the key's signature+ owned[piece[opaque & ~keyish]] = True+ residual = int((suspect & ~owned[piece]).sum())+ return {"subject": total - residual, "residual": residual}def _reference_window(reference: Path, key: str) -> tuple[str, Image.Image | None, tuple[int, int, int] | None, dict[str, Any]]:diff --git a/tests/frames/test_chroma_key_relative.py b/tests/frames/test_chroma_key_relative.pyindex d47452e..c7b627e 100644--- a/tests/frames/test_chroma_key_relative.py+++ b/tests/frames/test_chroma_key_relative.py@@ -408,17 +408,24 @@ def color_dist(a: tuple[int, int, int], b: tuple[int, int, int]) -> float:def test_edge_contact_split_reads_grok_magenta_residue_as_residual() -> None:- """`video-frames` edge classification uses the same border rule: leftover Grok magenta is residue, hot pink is subject."""+ """`video-frames` edge classification uses the same border rule: leftover Grok magenta is residue, hot pink is subject.+ A residue is a piece of the keyed frame with nothing but the key's colour in it; joined to the subject, the same+ pixels are its rim."""from sprite_gen.video.frames import classify_edge_contactraw = Image.new("RGB", (40, 40), (110, 70, 50))keyed = Image.new("RGBA", (40, 40), (110, 70, 50, 255))for x in range(0, 10):raw.putpixel((x, 0), (216, 46, 147))- raw.putpixel((x, 1), HOT_PINK)+ raw.putpixel((x, 2), HOT_PINK)+ joined = classify_edge_contact(raw, keyed, MAGENTA)+ assert joined == {"subject": 40 * 40 - (40 - 4 * 2) * (40 - 4), "residual": 0} # every band pixel is opaque subject+ for x in range(0, 11): # a transparent row and column cut the residue off+ keyed.putpixel((x, 1), (0, 0, 0, 0))+ keyed.putpixel((10, 0), (0, 0, 0, 0))split = classify_edge_contact(raw, keyed, MAGENTA)assert split["residual"] == 10- assert split["subject"] == 40 * 40 - 10 - (40 - 4 * 2) * (40 - 4) # every other band pixel is opaque subject+ assert split["subject"] == 40 * 40 - 10 - 12 - (40 - 4 * 2) * (40 - 4) # every other opaque band pixel is subject# --- real stills (opt-in) ---------------------------------------------------------------diff --git a/tests/video/test_edge_speck_lead_in.py b/tests/video/test_edge_speck_lead_in.pyindex b82175f..ead93ab 100644--- a/tests/video/test_edge_speck_lead_in.py+++ b/tests/video/test_edge_speck_lead_in.py@@ -2,7 +2,8 @@"""A fleck far from the body is not the subject touching the edge, a clip's first frames that reframe asmall subject are not part of its walk, and a walk forced wide gets no room above.-* `video-frames` erases a speck — a small piece far from the body — before the edge check reads the frame.+* `video-frames` erases a speck — a small piece far from the body — before the edge check reads the frame, and+ reads a key-tinted pixel in the edge band as the subject's rim when its piece holds the subject too.* `video-loop` reads a walk's lead-in (the first frames off the walk's size) and searches the clip after it;what it finds, and what it refuses, is said in the clip's own frame numbers. The gait fallback scales backby the size read one cycle on, as the hold does. A cut after a lead-in reads the standing height on its@@ -115,6 +116,60 @@ def test_a_motion_auto_loop_of_a_clip_with_a_fleck_is_cut_without_it(tmp_path):assert all(row["specks"] == 1 for row in report["rows"])+# --- the edge check's rim --------------------------------------------------------------------------------------+++GREEN = (0, 255, 0)+RIM = (30, 160, 20) # the key's hue (a dark crown blended into green), which the matte keeps+++def _green(tmp_path: Path, n: int, paint) -> list[Path]:+ """`n` frames of a 30 x 84 body on green, `paint(k, image)` adding to frame k."""+ d = tmp_path / "raw"+ d.mkdir()+ files = []+ for k in range(n):+ image = Image.new("RGB", (80, 100), GREEN)+ image.paste(BODY, (25, 10, 55, 94))+ paint(k, image)+ files.append(d / f"frame-{k:04d}.png")+ image.save(files[-1])+ return files+++def test_a_body_whose_rim_alone_reaches_the_edge_touches_it(tmp_path):+ # Frame 1 rises 6 px: only the key-tinted rim over its crown reaches the top band. Read alone by its colour,+ # that rim was leftover background and refused even where the subject is allowed at the edge.+ def rise(k, image):+ top = 4 if k == 1 else 10+ image.paste(GREEN, (25, 0, 55, 10))+ image.paste(BODY, (25, top, 55, 94))+ image.paste(RIM, (25, top - 4, 55, top))+ files = _green(tmp_path, 3, rise)+ report = frames_mod.key_frames(files, tmp_path / "keyed", key="green", allow_subject=True)+ assert [c["frame"] for c in report["edge_contacts"]] == ["frame-0001.png"]+ contact = report["edge_contacts"][0]+ assert contact["top"] > 0 and contact["subject"] == contact["top"] and contact["residual"] == 0+ # Refused by default as what it is: the subject framed too tight.+ with pytest.raises(SystemExit, match="framed too tight") as refused:+ frames_mod.key_frames(files, tmp_path / "keyed2", key="green")+ assert "leftover" not in str(refused.value)+++def test_key_coloured_background_apart_from_the_body_is_still_residual(tmp_path):+ # A strip of the key's colour against the left edge, 21 px from the body and too large to be a speck: background.+ def strip(k, image):+ if k == 1:+ image.paste(RIM, (0, 40, 4, 60))+ files = _green(tmp_path, 3, strip)+ with pytest.raises(SystemExit, match="leftover chroma background") as refused:+ frames_mod.key_frames(files, tmp_path / "keyed", key="green", allow_subject=True)+ assert "framed too tight" not in str(refused.value)+ keyed = Image.open(tmp_path / "keyed" / "frame-0001.png")+ split = frames_mod.classify_edge_contact(Image.open(files[1]), keyed, GREEN)+ assert split["subject"] == 0 and split["residual"] > 0++# --- a walk's lead-in -----------------------------------------------------------------------------------------