Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

fix(video): a key-tinted rim joined to the subject at the edge is the subject

Soohong Kim9c23deeec847
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e7a7245..8bd610f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,9 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
## Unreleased
+- **Fix: a key-tinted rim joined to the subject at the edge is the subject.** The edge check split its contact pixels by their raw colour alone: the key's hue was leftover background. A body that reaches an edge band brings its antialiased rim there first — hair tips, the crown's outline, the key blended into them — and on a frame where only that rim was in the band, `--allow-subject-edge-contact` refused the clip as leftover background. Each key-tinted contact pixel is now read with the piece of the keyed frame it is in (its opaque pixels, corners joining them, the pieces `drop_specks` reads): it is `residual` only when nothing in its piece is off the key's hue — background on its own — and `subject` when it is joined to the subject. A frame with such a rim in the band passes with `--allow-subject-edge-contact` and is recorded as the subject's contact; refused by default, it says the subject was framed too tight, and a mixed refusal no longer counts the rim's frames as carrying leftover background. Key-coloured background apart from the body is refused as before. A clip with no key-tinted pixel in a band, or none joined to the subject, keys and reports as before, byte for byte. [docs/video-pipeline.md](docs/video-pipeline.md#3-frames--extract-key-check-the-edges).
+- **`tests/video/test_edge_speck_lead_in.py`**: a body that rises until only the key-tinted rim over its crown is in the top band was refused as leftover background with the subject allowed; now it passes as the subject's contact, and by default is refused as framed too tight. A strip of the key's colour against the edge, apart from the body, is still refused as leftover background. `tests/frames/test_chroma_key_relative.py`: Grok's magenta in the band is residual cut off from the rest, and the subject's when joined to it.
+
- **Output change: `video-canvas --shape wide` gives a walk or run no head-room.** A forced-wide canvas gave every state a jump's 35 % above the still, and above a square still that head-room set the width as well (2800 × 1575 for 1024 × 1024, 1216 columns in front of the still); a walker that small in its frame is one a video model may reframe. A walk or run forced wide now has a row of its own (`canvas.GAIT_WIDE`, read through `STATE_SHAPE_CANVAS`): 28 % in front, 20 % behind, nothing above — 1969 × 1108 for a 1024 × 1024 still. A jump, and every other state forced wide, keeps the forced-wide row (`WIDE_OVERRIDE`) and its canvas byte for byte; `--headroom` still sets the room above. [docs/video-pipeline.md](docs/video-pipeline.md#1-canvas--the-input-frame-decides-the-output-frame).
- **`tests/video/test_edge_speck_lead_in.py`**: a walk or run forced wide is 1969 × 1108 with nothing above, a jump still 2800 × 1575.
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 8810850..c599ba7 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -586,8 +586,12 @@ the loop's cells and widened them to reach it.
The report also carries per-frame alpha coverage and an **edge-contact check**, read after the
specks are gone: any opaque pixel in the top/left/right 4-pixel bands fails the run. Each contact pixel is
-classified by its *raw* colour — the declared key's hue family is **`residual`**
-(background the matte did not erase), anything else is **`subject`** — and the two
+classified by its *raw* colour and the piece of the keyed frame it is in (its opaque pixels, corners
+joining them, as the specks are read): the declared key's hue family in a piece with nothing else in
+it is **`residual`** (background the matte did not erase); anything else is **`subject`** — a
+key-tinted pixel joined to the subject included, since a body that reaches a band brings its
+antialiased rim, the key blended into it, there first, and on a frame where only that rim is in the
+band, read by its colour alone, the clip was refused as leftover background. The two
defects fail with different messages: residual-only contact points at `video-canvas`
(normalize the base still and regenerate); subject contact means the model framed too
tight and points at a taller/wider canvas. When both occur the message names both.
diff --git a/sprite_gen/video/frames.py b/sprite_gen/video/frames.py
index beee2eb..f7c0777 100644
--- a/sprite_gen/video/frames.py
+++ b/sprite_gen/video/frames.py
@@ -14,8 +14,10 @@ a little off and part of that background survives the matte, the edge band is
"touched" by leftover *background* (2026-09-11: a (8, 162, 24) green read as
"framed too tight"). The check therefore classifies every contact pixel by its
raw colour — the declared key's hue signature (`extract.is_border_key_candidate`,
-the border rule: an edge pixel is border evidence) is **residual background**,
-anything else is the **subject** — and the two fail
+the border rule: an edge pixel is border evidence) in a piece of the keyed frame that
+holds nothing else is **residual background**; anything else is the **subject**, a
+key-tinted pixel joined to the subject included (its antialiased rim, the key blended
+into it, reaches the band first) — and the two fail
with different messages: residual points at `video-canvas` normalization of the
base still, subject contact at a taller/wider canvas.
@@ -51,7 +53,7 @@ from PIL import Image
from sprite_gen._deps import np
from sprite_gen.frames.cutout import cutout
from sprite_gen.frames.decontam import palette_from_stats
-from sprite_gen.frames.extract import is_border_key_candidate
+from sprite_gen.frames.extract import _border_key_candidate_field, _key_channel_split
from sprite_gen.frames.extract import _SPILL_FULL_MIN_TINT, DEFAULT_UNMIX_REACH
from sprite_gen.spec.runio import atomic_write_text
@@ -236,26 +238,37 @@ def classify_edge_contact(raw: Image.Image, keyed: Image.Image, chroma_key: tupl
"""Split the opaque edge-band pixels into `subject` and `residual` (leftover key background).
A contact pixel whose *raw* colour carries the declared key's hue signature
- (`is_border_key_candidate`) is background the matte failed to erase, not the
- subject. Without a chroma key
+ (`is_border_key_candidate`) is either background the matte failed to erase or the subject's
+ own rim: a body reaching the band brings its antialiased edge, the key blended into it, there
+ first. It is residual only when the opaque piece of the keyed frame it belongs to — its opaque
+ pixels, corners joining them, the pieces `drop_specks` reads — holds no pixel off the key's
+ signature: background on its own. Joined to one, it is the subject's. Without a chroma key
(matte route) every contact is the subject.
"""
- alpha = keyed.convert("RGBA").getchannel("A").load()
- rgb = raw.convert("RGB").load()
- w, h = keyed.size
- band = set()
- band.update((x, y) for y in range(min(EDGE_ROWS, h)) for x in range(w))
- band.update((x, y) for x in range(min(EDGE_ROWS, w)) for y in range(h))
- band.update((x, y) for x in range(max(0, w - EDGE_ROWS), w) for y in range(h))
- subject = residual = 0
- for x, y in band:
- if alpha[x, y] <= 0:
- continue
- if chroma_key is not None and is_border_key_candidate(rgb[x, y], chroma_key):
- residual += 1
- else:
- subject += 1
- return {"subject": subject, "residual": residual}
+ alpha = np.asarray(keyed.convert("RGBA"))[..., 3]
+ h, w = alpha.shape
+ band = np.zeros((h, w), dtype=bool)
+ band[:EDGE_ROWS, :] = True
+ band[:, :EDGE_ROWS] = True
+ band[:, max(0, w - EDGE_ROWS):] = True
+ opaque = alpha > 0
+ contact = band & opaque
+ total = int(contact.sum())
+ keyed_channels, unkeyed_channels = _key_channel_split(chroma_key) if chroma_key is not None else ([], [])
+ if not total or not keyed_channels:
+ return {"subject": total, "residual": 0}
+ keyish = _border_key_candidate_field(np.asarray(raw.convert("RGB"), dtype=np.int32), keyed_channels, unkeyed_channels)
+ suspect = contact & keyish
+ if not suspect.any():
+ return {"subject": total, "residual": 0}
+ ys, starts, stops, labels, _ = _components(opaque, diagonal=True)
+ piece = np.zeros((h, w), dtype=np.int64)
+ for y, x0, x1, label in zip(ys, starts, stops, labels):
+ piece[y, x0:x1] = label
+ owned = np.zeros(len(ys), dtype=bool) # pieces holding a pixel off the key's signature
+ owned[piece[opaque & ~keyish]] = True
+ residual = int((suspect & ~owned[piece]).sum())
+ return {"subject": total - residual, "residual": residual}
def _reference_window(reference: Path, key: str) -> tuple[str, Image.Image | None, tuple[int, int, int] | None, dict[str, Any]]:
diff --git a/tests/frames/test_chroma_key_relative.py b/tests/frames/test_chroma_key_relative.py
index d47452e..c7b627e 100644
--- a/tests/frames/test_chroma_key_relative.py
+++ b/tests/frames/test_chroma_key_relative.py
@@ -408,17 +408,24 @@ def color_dist(a: tuple[int, int, int], b: tuple[int, int, int]) -> float:
def test_edge_contact_split_reads_grok_magenta_residue_as_residual() -> None:
- """`video-frames` edge classification uses the same border rule: leftover Grok magenta is residue, hot pink is subject."""
+ """`video-frames` edge classification uses the same border rule: leftover Grok magenta is residue, hot pink is subject.
+ A residue is a piece of the keyed frame with nothing but the key's colour in it; joined to the subject, the same
+ pixels are its rim."""
from sprite_gen.video.frames import classify_edge_contact
raw = Image.new("RGB", (40, 40), (110, 70, 50))
keyed = Image.new("RGBA", (40, 40), (110, 70, 50, 255))
for x in range(0, 10):
raw.putpixel((x, 0), (216, 46, 147))
- raw.putpixel((x, 1), HOT_PINK)
+ raw.putpixel((x, 2), HOT_PINK)
+ joined = classify_edge_contact(raw, keyed, MAGENTA)
+ assert joined == {"subject": 40 * 40 - (40 - 4 * 2) * (40 - 4), "residual": 0} # every band pixel is opaque subject
+ for x in range(0, 11): # a transparent row and column cut the residue off
+ keyed.putpixel((x, 1), (0, 0, 0, 0))
+ keyed.putpixel((10, 0), (0, 0, 0, 0))
split = classify_edge_contact(raw, keyed, MAGENTA)
assert split["residual"] == 10
- assert split["subject"] == 40 * 40 - 10 - (40 - 4 * 2) * (40 - 4) # every other band pixel is opaque subject
+ assert split["subject"] == 40 * 40 - 10 - 12 - (40 - 4 * 2) * (40 - 4) # every other opaque band pixel is subject
# --- real stills (opt-in) ---------------------------------------------------------------
diff --git a/tests/video/test_edge_speck_lead_in.py b/tests/video/test_edge_speck_lead_in.py
index b82175f..ead93ab 100644
--- a/tests/video/test_edge_speck_lead_in.py
+++ b/tests/video/test_edge_speck_lead_in.py
@@ -2,7 +2,8 @@
"""A fleck far from the body is not the subject touching the edge, a clip's first frames that reframe a
small subject are not part of its walk, and a walk forced wide gets no room above.
-* `video-frames` erases a speck — a small piece far from the body — before the edge check reads the frame.
+* `video-frames` erases a speck — a small piece far from the body — before the edge check reads the frame, and
+ reads a key-tinted pixel in the edge band as the subject's rim when its piece holds the subject too.
* `video-loop` reads a walk's lead-in (the first frames off the walk's size) and searches the clip after it;
what it finds, and what it refuses, is said in the clip's own frame numbers. The gait fallback scales back
by the size read one cycle on, as the hold does. A cut after a lead-in reads the standing height on its
@@ -115,6 +116,60 @@ def test_a_motion_auto_loop_of_a_clip_with_a_fleck_is_cut_without_it(tmp_path):
assert all(row["specks"] == 1 for row in report["rows"])
+# --- the edge check's rim --------------------------------------------------------------------------------------
+
+
+GREEN = (0, 255, 0)
+RIM = (30, 160, 20) # the key's hue (a dark crown blended into green), which the matte keeps
+
+
+def _green(tmp_path: Path, n: int, paint) -> list[Path]:
+ """`n` frames of a 30 x 84 body on green, `paint(k, image)` adding to frame k."""
+ d = tmp_path / "raw"
+ d.mkdir()
+ files = []
+ for k in range(n):
+ image = Image.new("RGB", (80, 100), GREEN)
+ image.paste(BODY, (25, 10, 55, 94))
+ paint(k, image)
+ files.append(d / f"frame-{k:04d}.png")
+ image.save(files[-1])
+ return files
+
+
+def test_a_body_whose_rim_alone_reaches_the_edge_touches_it(tmp_path):
+ # Frame 1 rises 6 px: only the key-tinted rim over its crown reaches the top band. Read alone by its colour,
+ # that rim was leftover background and refused even where the subject is allowed at the edge.
+ def rise(k, image):
+ top = 4 if k == 1 else 10
+ image.paste(GREEN, (25, 0, 55, 10))
+ image.paste(BODY, (25, top, 55, 94))
+ image.paste(RIM, (25, top - 4, 55, top))
+ files = _green(tmp_path, 3, rise)
+ report = frames_mod.key_frames(files, tmp_path / "keyed", key="green", allow_subject=True)
+ assert [c["frame"] for c in report["edge_contacts"]] == ["frame-0001.png"]
+ contact = report["edge_contacts"][0]
+ assert contact["top"] > 0 and contact["subject"] == contact["top"] and contact["residual"] == 0
+ # Refused by default as what it is: the subject framed too tight.
+ with pytest.raises(SystemExit, match="framed too tight") as refused:
+ frames_mod.key_frames(files, tmp_path / "keyed2", key="green")
+ assert "leftover" not in str(refused.value)
+
+
+def test_key_coloured_background_apart_from_the_body_is_still_residual(tmp_path):
+ # A strip of the key's colour against the left edge, 21 px from the body and too large to be a speck: background.
+ def strip(k, image):
+ if k == 1:
+ image.paste(RIM, (0, 40, 4, 60))
+ files = _green(tmp_path, 3, strip)
+ with pytest.raises(SystemExit, match="leftover chroma background") as refused:
+ frames_mod.key_frames(files, tmp_path / "keyed", key="green", allow_subject=True)
+ assert "framed too tight" not in str(refused.value)
+ keyed = Image.open(tmp_path / "keyed" / "frame-0001.png")
+ split = frames_mod.classify_edge_contact(Image.open(files[1]), keyed, GREEN)
+ assert split["subject"] == 0 and split["residual"] > 0
+
+
# --- a walk's lead-in -----------------------------------------------------------------------------------------