Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

fix(gen): a ref run whose raw came back transparent keeps its own alpha instead of being keyed

Soohong Kimf04b41e6692b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f044fbe..f1c1ea6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,11 @@
All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.
+## Unreleased
+
+- **A `gen --transparent` run with `--ref` keeps the alpha it comes back with**: `auto` still plans a chroma key before the model runs (2026-09-08: codex drew a checkerboard in 5 of 6 ref runs asked for alpha), but the raw is read before the key is applied (`chroma.classify_raw_alpha`). A raw with an alpha band, at least 5 % of it at alpha 0 and at least half of its border at alpha 0 is published on its own alpha through the native check, with `alpha.strategy: "native"` and `alpha.strategy_source: "refs-attached-raw-alpha"`. Keying it read the RGB under alpha 0 as the background and took the outline and light fills off: a transparent reference is answered with alpha, and a fox redrawn from one lost its outline and cream belly. A checkerboard, a key background or an opaque RGBA (no alpha band, or no alpha 0) is keyed as before (`refs-attached`); alpha 0 that misses either bar is keyed as before with a `warning`. `alpha.raw_alpha` records the verdict and its numbers on every ref run that planned the key. `--alpha-mode chroma` keys whatever comes back, and `--decontam palette` refuses a raw that came back transparent. [docs/gen.md](docs/gen.md#transparent-output--strategy-per-provider).
+- Measured 2026-10-04 on codex `image_gen` (subscription), one transparent front still as the reference, three takes per arm: with the planned key's prompt, a transparent reference came back with real alpha 3/3 and the same still on white came back on opaque white 3/3; asked for alpha (`--alpha-mode native`), both came back with real alpha 3/3, with no checkerboard in 6 runs. The step down is kept until a larger measurement replaces the 2026-09-08 one.
+
## v2.21.0 - A clip made by a video MCP on your agent (ZCRE) becomes a loop: the prompt comes out, the mp4 goes in
- **New `sprite-gen video-prompt`** prints the clip prompt `video-set` sends for one (direction, state) — `build_prompt` with `--character`, `--motion` and `--model` — so an agent that makes the clip itself, with a video MCP connected to it, sends the same sentence. `--json` adds the duration (the state's own), whether the clip ends on the canvas (`last_frame`), the loop cut (`cycle`), the canvas, clip, frames and loop steps with placeholders, and for a front or back walk the mid-step redraw (`start_still`). sprite-gen never calls the MCP: the agent uploads the `video-canvas` still, quotes, asks the user, generates and saves the mp4, and `video-frames` and `video-loop` take it from there. [docs/video-pipeline.md](docs/video-pipeline.md#a-clip-from-a-video-mcp-on-your-agent--zcre).
diff --git a/SKILL.md b/SKILL.md
index 365a535..055a2fc 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -108,6 +108,8 @@ front-facing observations also remain unchanged. Mirroring does not preserve lef
| Optional scene placement, lighting, camera and render | `scene-render`, `scene-inspect` | [scene](docs/scene.md) |
| Defaults | `defaults show`, `defaults save`, `defaults clear` | [user-workflow](docs/user-workflow.md#one-settings-owner) |
+`gen --transparent` with `--ref` plans a chroma key, and publishes a result that already came back transparent on its own alpha instead of keying its outline away; `alpha.strategy_source` in the report says which (`refs-attached` keyed, `refs-attached-raw-alpha` not) — [gen](docs/gen.md#transparent-output--strategy-per-provider).
+
Use existing automatic pipeline stages for background removal, extraction, alignment and export. Do not ask users to select each script. For a direct utility request, run that utility; no unrelated generation questions are needed. Preserve the row pipeline and component extraction for image sprites. One-shot grid generation and fixed cell cutting are not an alternative sprite-generation route.
For attack repeat coverage, observed one-shot returns and structured loop failure reports, follow [video-pipeline](docs/video-pipeline.md#one-shot-actions--cycle-autoperiodicone-shot).
diff --git a/docs/gen.md b/docs/gen.md
index 070030f..ca9dd44 100644
--- a/docs/gen.md
+++ b/docs/gen.md
@@ -264,6 +264,27 @@ strategy it can execute, and `--alpha-mode auto` (the default) follows it:
conservative default. `--alpha-mode native`
still forces native alpha with refs — and fails loud on an RGB result. So a ref run's
prompt must carry the chroma key, exactly as the sprite-row pipeline already does.
+- **A ref run whose raw already has a transparent background is not keyed.** The key
+ is planned before the model runs, but the raw is read before it is applied
+ (`chroma.classify_raw_alpha`): a raw with an alpha band, at least 5 % of its pixels
+ at alpha 0 and at least half of its one-pixel border at alpha 0 is `real-alpha`, and
+ it is published on its own alpha through the `native` check
+ (`alpha.strategy: "native"`, `alpha.strategy_source: "refs-attached-raw-alpha"`).
+ Keying it reads the RGB left under alpha 0 as the background colour and mattes the
+ outline and light fills away. A raw with no alpha band or no alpha-0 pixel — a drawn
+ checkerboard, a key background, an opaque RGBA — is `no-alpha` and keyed as before
+ (`strategy_source: "refs-attached"`). Alpha 0 that misses either bar is `ambiguous`:
+ keyed as before, with a `warning` in the report and on stderr. The verdict and its
+ numbers are in `alpha.raw_alpha` on every ref run that planned the key. The check
+ belongs to `auto`'s step down only — `--alpha-mode chroma` keys whatever comes back —
+ and `--decontam palette` refuses a `real-alpha` raw, which has no key to remove.
+ Measured 2026-10-04 (codex `image_gen`, subscription, one transparent front still as
+ the reference, three takes per arm): with the step down's prompt (no transparency
+ request, no key colour), a transparent (RGBA) reference came back with real alpha
+ 3/3 and an RGB one (the same still on white) on an opaque white background 3/3; with
+ the native request (`--alpha-mode native`), both came back with real alpha 3/3 —
+ no checkerboard in 6 runs, against 5 of 6 on 2026-09-08. The step down is kept
+ until a larger measurement replaces that one.
- `--alpha-mode chroma` on codex is for prompts that already carry a key background
(the sprite-row pipeline today): the native request is **not** added to the prompt
and the raw is keyed like a grok run.
diff --git a/sprite_gen/gen/__init__.py b/sprite_gen/gen/__init__.py
index 4a77c36..3805b33 100644
--- a/sprite_gen/gen/__init__.py
+++ b/sprite_gen/gen/__init__.py
@@ -184,8 +184,14 @@ def _make_provider(name: str, *, keep_session: bool):
# alpha source, so `auto` does not gamble on it — the decision is made before the
# model runs, printed, and recorded in the report (`alpha.strategy_source`).
# An explicit `--alpha-mode native` still forces it (and fails loud on RGB).
+# The key planned this way is applied only to a raw that needs it: a raw that came
+# back with a real transparent background anyway (`chroma.classify_raw_alpha`,
+# 2026-10-04: a transparent `--ref` is answered with alpha) is published on its own
+# alpha, measured as a native one (STRATEGY_SOURCE_REFS_RAW_ALPHA), because keying
+# it again mattes the outline away. A checkerboard or key background is still keyed.
STRATEGY_SOURCE_PROVIDER = "provider-default"
STRATEGY_SOURCE_REFS = "refs-attached"
+STRATEGY_SOURCE_REFS_RAW_ALPHA = "refs-attached-raw-alpha"
STRATEGY_SOURCE_EXPLICIT = "explicit"
@@ -373,7 +379,8 @@ def generate_image(
if strategy_source == STRATEGY_SOURCE_REFS:
print(
f"[gen] {len(refs)} reference image(s) attached — using chroma keying instead of "
- f"{backend.name}'s native alpha (native output with refs is unstable, 2026-09-08). "
+ f"{backend.name}'s native alpha (native output with refs is unstable, 2026-09-08); "
+ "a result that already has a transparent background keeps its own alpha. "
"Pass --alpha-mode native to force it.",
file=sys.stderr,
)
@@ -416,6 +423,36 @@ def generate_image(
raw = request.raw
raw_bytes = verify_png(raw)
+ # Only the key `auto` planned because of the refs is checked against the raw; an
+ # explicit --alpha-mode chroma is the caller's own key and runs as asked.
+ raw_alpha: dict[str, Any] | None = None
+ if strategy_source == STRATEGY_SOURCE_REFS:
+ raw_alpha = chroma_mod.classify_raw_alpha(raw)
+ if raw_alpha["verdict"] == chroma_mod.RAW_ALPHA_REAL:
+ if decontam == "palette":
+ raise SystemExit(
+ f"gen: --decontam palette removes a chroma key, but the raw came back with a "
+ f"transparent background ({raw_alpha['alpha_zero_pct']}% alpha 0) and has no key "
+ "to remove; run again without --decontam palette, or with --alpha-mode chroma "
+ "and a key background in the prompt"
+ )
+ strategy, strategy_source = TRANSPARENCY_NATIVE, STRATEGY_SOURCE_REFS_RAW_ALPHA
+ print(
+ f"[gen] the raw came back with a transparent background ({raw_alpha['alpha_zero_pct']}% "
+ f"alpha 0, {raw_alpha['border_alpha_zero_pct']}% of the border) — publishing its own "
+ "alpha instead of keying it.",
+ file=sys.stderr,
+ )
+ elif raw_alpha["verdict"] == chroma_mod.RAW_ALPHA_AMBIGUOUS:
+ raw_alpha["warning"] = (
+ f"the raw has some transparent pixels ({raw_alpha['alpha_zero_pct']}% alpha 0, "
+ f"{raw_alpha['border_alpha_zero_pct']}% of the border) but not a transparent background "
+ f"(needs {chroma_mod.RAW_ALPHA_MIN_ZERO_PCT}% and {chroma_mod.RAW_ALPHA_MIN_BORDER_ZERO_PCT}% "
+ "of the border); keyed as planned — check the outline, or pass --alpha-mode native"
+ )
+ print(f"[gen] warning: {raw_alpha['warning']}", file=sys.stderr)
+ raw_alpha_stats = {"raw_alpha": raw_alpha} if raw_alpha is not None else {}
+
chroma_stats: dict[str, Any] | None = None
alpha_stats: dict[str, Any] | None = None
out.parent.mkdir(parents=True, exist_ok=True)
@@ -423,12 +460,14 @@ def generate_image(
alpha_stats = {
"strategy": TRANSPARENCY_NATIVE,
"strategy_source": strategy_source,
+ **raw_alpha_stats,
**chroma_mod.verify_native_alpha(raw, out, white_check=white_check),
}
elif strategy == TRANSPARENCY_CHROMA:
chroma_stats = chroma_mod.key_transparent(raw, out, key=chroma_key, white_check=white_check,
decontam=decontam)
- alpha_stats = {"strategy": TRANSPARENCY_CHROMA, "strategy_source": strategy_source, **chroma_stats}
+ alpha_stats = {"strategy": TRANSPARENCY_CHROMA, "strategy_source": strategy_source,
+ **raw_alpha_stats, **chroma_stats}
else:
shutil.copyfile(raw, out)
verify_png(out)
diff --git a/sprite_gen/gen/chroma.py b/sprite_gen/gen/chroma.py
index 4aa10d9..ead2d93 100644
--- a/sprite_gen/gen/chroma.py
+++ b/sprite_gen/gen/chroma.py
@@ -30,6 +30,56 @@ KEYS: dict[str, dict[str, tuple[int, int, int]]] = {
}
+# What a generated raw is, before a planned chroma key runs on it (`classify_raw_alpha`).
+# A drawn checkerboard or a painted key background is an RGB picture: it has no alpha
+# band, or an alpha band with nothing transparent in it, and only keying can make it
+# transparent. A real cut-out has an alpha band whose zeros are the background, so they
+# cover a real part of the picture and reach its edge — keying that picture again
+# reads the RGB left under alpha 0 as the background colour and mattes the subject's
+# own outline and light fills away (2026-10-04, a fox's outline and cream belly).
+RAW_ALPHA_REAL = "real-alpha"
+RAW_ALPHA_NONE = "no-alpha"
+RAW_ALPHA_AMBIGUOUS = "ambiguous"
+RAW_ALPHA_MIN_ZERO_PCT = 5.0
+RAW_ALPHA_MIN_BORDER_ZERO_PCT = 50.0
+
+
+def classify_raw_alpha(path: Path) -> dict[str, Any]:
+ """Say whether a generated raw already carries a real transparent background.
+
+ `verdict` is `real-alpha` when the PNG has an alpha band, at least
+ RAW_ALPHA_MIN_ZERO_PCT % of its pixels are alpha 0 and at least
+ RAW_ALPHA_MIN_BORDER_ZERO_PCT % of its one-pixel border is alpha 0; `no-alpha`
+ when it has no alpha band or no alpha-0 pixel (a checkerboard, a key background,
+ an opaque RGBA); `ambiguous` for alpha-0 pixels that miss either bar. The alpha
+ band is the one `verify_native_alpha` reads, so a `real-alpha` raw is one it accepts.
+ """
+ with Image.open(path) as source:
+ mode = source.mode
+ has_alpha = "A" in source.getbands()
+ alpha = source.getchannel("A") if has_alpha else None
+ width, height = source.size
+ stats: dict[str, Any] = {"mode": mode, "has_alpha_band": has_alpha}
+ if alpha is None:
+ return {**stats, "verdict": RAW_ALPHA_NONE, "alpha_zero_pct": 0.0, "border_alpha_zero_pct": 0.0}
+ total = width * height
+ alpha_zero_pct = round(alpha.histogram()[0] / total * 100, 2) if total else 0.0
+ edges = [(0, 0, width, 1), (0, height - 1, width, height),
+ (0, 1, 1, height - 1), (width - 1, 1, width, height - 1)]
+ edges = [box for box in edges if box[2] > box[0] and box[3] > box[1]]
+ border_total = sum((r - l) * (b - t) for l, t, r, b in edges)
+ border_zero = sum(alpha.crop(box).histogram()[0] for box in edges)
+ border_alpha_zero_pct = round(border_zero / border_total * 100, 2) if border_total else 0.0
+ if alpha_zero_pct == 0.0:
+ verdict = RAW_ALPHA_NONE
+ elif alpha_zero_pct >= RAW_ALPHA_MIN_ZERO_PCT and border_alpha_zero_pct >= RAW_ALPHA_MIN_BORDER_ZERO_PCT:
+ verdict = RAW_ALPHA_REAL
+ else:
+ verdict = RAW_ALPHA_AMBIGUOUS
+ return {**stats, "verdict": verdict, "alpha_zero_pct": alpha_zero_pct,
+ "border_alpha_zero_pct": border_alpha_zero_pct}
+
+
def write_white_check(image: Image.Image, path: Path) -> None:
bg = Image.new("RGBA", image.size, (255, 255, 255, 255))
bg.alpha_composite(image)
diff --git a/tests/gen/test_gen.py b/tests/gen/test_gen.py
index 666435d..30b548d 100644
--- a/tests/gen/test_gen.py
+++ b/tests/gen/test_gen.py
@@ -636,6 +636,165 @@ def test_explicit_native_still_runs_with_refs(tmp_path: Path, monkeypatch) -> No
assert payload["alpha"]["strategy_source"] == "explicit"
+# Three raws a ref run can come back with (2026-10-04): a real cut-out (a transparent
+# --ref is answered with alpha), a drawn checkerboard and a painted key background.
+# The figure is a dark outline around an orange body with a cream belly — the two
+# colours a second key took off the real cut-out.
+_OUTLINE, _BODY, _CREAM = (20, 20, 20), (240, 140, 40), (245, 230, 200)
+_OUTLINE_PX, _CREAM_PX = (17, 32), (32, 37)
+
+
+def _outlined_figure(background: Image.Image) -> Image.Image:
+ from PIL import ImageDraw
+
+ alpha = (255,) if background.mode == "RGBA" else ()
+ draw = ImageDraw.Draw(background)
+ draw.ellipse((16, 12, 48, 52), fill=_OUTLINE + alpha)
+ draw.ellipse((19, 15, 45, 49), fill=_BODY + alpha)
+ draw.ellipse((26, 30, 38, 44), fill=_CREAM + alpha)
+ return background
+
+
+def _real_alpha_raw() -> Image.Image:
+ return _outlined_figure(Image.new("RGBA", (64, 64), (0, 0, 0, 0)))
+
+
+def _checkerboard_raw() -> Image.Image:
+ board = Image.new("RGB", (64, 64))
+ for y in range(64):
+ for x in range(64):
+ board.putpixel((x, y), (204, 204, 204) if (x // 8 + y // 8) % 2 else (255, 255, 255))
+ return _outlined_figure(board)
+
+
+def _key_background_raw() -> Image.Image:
+ return _outlined_figure(Image.new("RGB", (64, 64), (255, 0, 255)))
+
+
+def _ref_run(tmp_path: Path, monkeypatch, raw: Image.Image, **overrides):
+ fake = _FakeNativeProvider(raw)
+ monkeypatch.setattr(gen, "_make_provider", lambda name, *, keep_session: fake)
+ ref = tmp_path / "ref.png"
+ ref.write_bytes(_png_bytes(color=(0, 0, 0, 0)))
+ out = tmp_path / "asset.png"
+ report = tmp_path / "report.json"
+ rc = gen.run(**_gen_kwargs(out, report, ref=[ref], **overrides))
+ return fake, rc, out, report
+
+
+def test_ref_run_publishes_a_real_alpha_raw_without_keying_its_outline(
+ tmp_path: Path, monkeypatch, capsys
+) -> None:
+ fake, rc, out, report = _ref_run(tmp_path, monkeypatch, _real_alpha_raw())
+
+ assert rc == 0
+ assert fake.requests[0].native_alpha is False # still planned as a key: the prompt is unchanged
+ published = Image.open(out)
+ assert published.getpixel(_OUTLINE_PX) == _OUTLINE + (255,)
+ assert published.getpixel(_CREAM_PX) == _CREAM + (255,)
+ assert published.getchannel("A").tobytes() == _real_alpha_raw().getchannel("A").tobytes()
+ payload = json.loads(report.read_text(encoding="utf-8"))
+ assert payload["alpha"]["strategy"] == "native"
+ assert payload["alpha"]["strategy_source"] == "refs-attached-raw-alpha"
+ assert payload["alpha"]["method"] == "native"
+ assert payload["alpha"]["raw_alpha"]["verdict"] == "real-alpha"
+ assert payload["alpha"]["raw_alpha"]["border_alpha_zero_pct"] == 100.0
+ assert payload["chroma"] is None
+ assert "publishing its own alpha" in capsys.readouterr().err
+
+
[email protected]("raw", [_checkerboard_raw, _key_background_raw], ids=["checkerboard", "key-background"])
+def test_ref_run_still_keys_a_raw_with_no_alpha(tmp_path: Path, monkeypatch, raw) -> None:
+ fake, rc, out, report = _ref_run(tmp_path, monkeypatch, raw())
+
+ assert rc == 0
+ payload = json.loads(report.read_text(encoding="utf-8"))
+ assert payload["alpha"]["strategy"] == "chroma"
+ assert payload["alpha"]["strategy_source"] == "refs-attached"
+ assert payload["alpha"]["method"] == "ycbcr"
+ assert payload["alpha"]["raw_alpha"] == {
+ "mode": "RGB", "has_alpha_band": False, "verdict": "no-alpha",
+ "alpha_zero_pct": 0.0, "border_alpha_zero_pct": 0.0,
+ }
+ assert payload["chroma"]["key"] == "magenta"
+ assert Image.open(out).getpixel((0, 0)) == (0, 0, 0, 0) # the background was keyed out
+
+
+def test_key_background_ref_run_keeps_its_outline(tmp_path: Path, monkeypatch) -> None:
+ _, _, out, _ = _ref_run(tmp_path, monkeypatch, _key_background_raw())
+ published = Image.open(out)
+ assert published.getpixel(_OUTLINE_PX) == _OUTLINE + (255,)
+ assert published.getpixel(_CREAM_PX) == _CREAM + (255,)
+
+
+def test_ref_run_keys_an_ambiguous_raw_and_warns(tmp_path: Path, monkeypatch, capsys) -> None:
+ # A key background with a few transparent pixels in one corner: alpha 0 that is
+ # not a transparent background. Keyed as planned, and the report says why.
+ raw = _outlined_figure(Image.new("RGBA", (64, 64), (255, 0, 255, 255)))
+ for y in range(3):
+ for x in range(3):
+ raw.putpixel((x, y), (0, 0, 0, 0))
+ _, rc, _, report = _ref_run(tmp_path, monkeypatch, raw)
+
+ assert rc == 0
+ payload = json.loads(report.read_text(encoding="utf-8"))
+ assert payload["alpha"]["strategy"] == "chroma"
+ assert payload["alpha"]["strategy_source"] == "refs-attached"
+ assert payload["alpha"]["raw_alpha"]["verdict"] == "ambiguous"
+ assert "keyed as planned" in payload["alpha"]["raw_alpha"]["warning"]
+ assert "warning: the raw has some transparent pixels" in capsys.readouterr().err
+
+
+def test_ref_run_with_decontam_palette_refuses_a_real_alpha_raw(tmp_path: Path, monkeypatch) -> None:
+ fake = _FakeNativeProvider(_real_alpha_raw())
+ monkeypatch.setattr(gen, "_make_provider", lambda name, *, keep_session: fake)
+ ref = tmp_path / "ref.png"
+ ref.write_bytes(_png_bytes())
+ out = tmp_path / "asset.png"
+ with pytest.raises(SystemExit, match="has no key to remove"):
+ gen.generate_image("fake", "a mushroom", out, refs=[ref], transparent=True, decontam="palette")
+ assert not out.exists()
+
+
+def test_explicit_chroma_keys_even_a_real_alpha_raw(tmp_path: Path, monkeypatch) -> None:
+ # --alpha-mode chroma is the caller's own key: the raw check belongs to auto's step down only.
+ _, rc, _, report = _ref_run(tmp_path, monkeypatch, _real_alpha_raw(), alpha_mode="chroma")
+ assert rc == 0
+ payload = json.loads(report.read_text(encoding="utf-8"))
+ assert payload["alpha"]["strategy"] == "chroma"
+ assert payload["alpha"]["strategy_source"] == "explicit"
+ assert "raw_alpha" not in payload["alpha"]
+
+
+ ("image", "verdict"),
+ [
+ (Image.new("RGB", (16, 16), (255, 0, 255)), "no-alpha"),
+ (Image.new("RGBA", (16, 16), (255, 0, 255, 255)), "no-alpha"),
+ (Image.new("LA", (16, 16), (0, 0)), "real-alpha"),
+ ],
+ ids=["rgb", "opaque-rgba", "transparent-la"],
+)
+def test_classify_raw_alpha_reads_the_alpha_band(tmp_path: Path, image: Image.Image, verdict: str) -> None:
+ path = tmp_path / "raw.png"
+ image.save(path)
+ assert chroma_mod.classify_raw_alpha(path)["verdict"] == verdict
+
+
+def test_classify_raw_alpha_needs_a_transparent_border(tmp_path: Path) -> None:
+ # Plenty of alpha 0, all of it inside an opaque frame: not a transparent background.
+ image = Image.new("RGBA", (20, 20), (255, 0, 255, 255))
+ for y in range(4, 16):
+ for x in range(4, 16):
+ image.putpixel((x, y), (0, 0, 0, 0))
+ path = tmp_path / "raw.png"
+ image.save(path)
+ stats = chroma_mod.classify_raw_alpha(path)
+ assert stats["alpha_zero_pct"] == 36.0
+ assert stats["border_alpha_zero_pct"] == 0.0
+ assert stats["verdict"] == "ambiguous"
+
+
def test_provider_without_a_declared_strategy_fails_loud(tmp_path: Path, monkeypatch) -> None:
class _Undeclared(_FakeProvider):
name = "undeclared"