Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

video: video-to-sprite pipeline domain — canvas, frames, loop, set

Soohong Kim4664fac4356c
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
new file mode 100644
index 0000000..8c00a63
--- /dev/null
+++ b/docs/video-pipeline.md
@@ -0,0 +1,123 @@
+# Video → sprite pipeline (engine SSoT)
+
+One still becomes a whole motion set: the still is padded into the canvas a state
+needs, Grok Imagine animates it in place, the clip is keyed frame by frame, and one
+seamless cycle is cut out as a strip, a transparent GIF and a WebP — every stage
+measured and reported, nothing recovered silently. Everything here was first run by
+hand on 2026-09-08 (15 loops: 3 directions × 5 states) and the rules below are the
+ones that survived that day.
+
+```
+still ──video-canvas──▶ canvas.png ──video──▶ clip.mp4 ──video-frames──▶ keyed/*.png ──video-loop──▶ strip · gif · webp
+ └── video-set runs all four per (direction, state)
+```
+
+| Verb | Module | In → out |
+|---|---|---|
+| `sprite-gen video-canvas` | `sprite_gen/video/canvas.py` | still → padded still (state canvas) + report |
+| `sprite-gen video` | `sprite_gen/gen/video.py` ([gen](video.md)) | still + prompt → mp4 + report |
+| `sprite-gen video-frames` | `sprite_gen/video/frames.py` | mp4 → `raw/`, `keyed/` RGBA frames + report |
+| `sprite-gen video-loop` | `sprite_gen/video/loop.py` | keyed frames → `cycle/`, `<name>.strip.png` + `.strip.json`, `<name>.gif`, `<name>.webp` + report |
+| `sprite-gen video-set` | `sprite_gen/video/batch.py` | bases × states → one folder per item, `set.report.json`, `table.md` |
+
+Wrappers: `scripts/video_canvas.py`, `scripts/video_frames.py`, `scripts/video_loop.py`,
+`scripts/video_set.py`. Binaries: `ffmpeg`/`ffprobe` (frames), `img2webp` from libwebp
+(WebP with exact alpha). Both are declared in `SKILL.md` `required_bins`.
+
+## 1. Canvas — the input frame decides the output frame
+
+Grok Imagine keeps the input image's framing and **ignores `aspect_ratio` on
+image-to-video** (a `3:4` request still came back 960×960). A jump whose hair leaves
+the frame cannot be fixed by prompt — it was fixed by padding the still. So the canvas
+is a property of the motion state, owned by one table (`STATE_CANVAS`):
+
+| State | Shape | Ratio | Room | Why |
+|---|---|---|---|---|
+| `jump` | tall | 3:4 | 34 % head-room above the still | airborne frames need height |
+| `attack` | wide | 16:9 | 28 % in front (facing side) | swings and weapons extend forward |
+| `projectile` | wide | 16:9 | 34 % in front | the projectile travels away |
+| everything else | square | 1:1 | — | in-place motion fits the still |
+
+`--shape tall|wide|square` overrides the row; `--headroom` / `--lead` tune the room;
+`--facing left` mirrors the wide layout. The padding is filled with the still's own
+corner colour (its chroma key), and a still whose corners are not one flat colour is
+refused — a non-flat background cannot be extended without guessing.
+
+## 2. Clip — `sprite-gen video`
+
+Unchanged from [video.md](video.md): the user's own credential, fail-loud, `ftyp`-verified
+mp4. For loops, prompt for **in-place, evenly paced, returns-to-start** motion on a flat
+chroma fill ("walks in place on a treadmill", "hop … return to the exact starting
+stance … same height every time"). `video-set` carries those templates
+(`MOTION_TEXT` / `VIEW_TEXT`).
+
+## 3. Frames — extract, key, check the edges
+
+`ffmpeg` extracts every frame; the clip's real fps is recorded (never assumed). Each
+frame goes through the same `cutout` engine imported stills use (`--key auto` reads
+the corners; green/magenta route to the extract matte). The report carries per-frame
+alpha coverage and an **edge-contact check**: any opaque pixel in the top/left/right
+4-pixel bands means the model framed too tight, and the run fails loud pointing at
+`video-canvas`. `--allow-edge-contact` accepts the clipping on purpose.
+
+## 4. Loop — period first, seam second
+
+The 2026-09-08 lesson: a single-start "most similar later frame" search lands on the
+**1.5-cycle look-alike** of a gait (legs swapped) and produces a loop that hitches at
+the wrap (side walk picked 39 frames where the period was 28; run picked 25 where it
+was 17). `video-loop` therefore:
+
+1. builds the distance matrix `D` on 96-px premultiplied thumbnails;
+2. reads the **global period profile** `P[L] = mean_j |f[j] − f[j+L]|` and takes the
+ *smallest* local minimum that is within 15 % of the deepest one — exact repeats dip
+ again at 2× and 3× the period, the half-period look-alike dips noticeably less;
+3. only then picks the **start** with the best seam for that period (± 1 frame):
+ `seam = D[i][i+L]` over the mean adjacent distance inside the cycle.
+
+Windows come from the state profile (`STATE_PROFILES`, fractions of the clip length):
+idle 60–95 % (breathing is slow and not periodic — the lowest seam is a long window,
+and idle is exempt from the periodicity gate), walk 10–31 %, run 7–23 %, jump/attack
+11–45 %. `--min-len/--max-len` override.
+
+Gates, all fail-loud: no period (profile flat, `periodicity < 0.15`), loop seam ratio
+above `--seam-max` (2.0), GIF/WebP re-opened and checked (frame count, `loop=0`,
+transparent corners, no RGB under alpha 0 in the WebP).
+
+Outputs:
+
+- `cycle/frame-NNN.png` — the cycle frames, RGB under alpha 0 scrubbed, detached specks
+ below 1 % of the body erased.
+- `<name>.strip.png` + `<name>.strip.json` — a horizontal strip (union-cropped, **no bottom
+ pad** so feet meet the floor, bottom-aligned, ≤ 64 cells because Chrome caps image
+ dimensions near 32 767 px, ≤ 520 px tall) with `frames · w · h · body_h · delay_ms ·
+ cycle_frames · cycle_seconds`. `body_h` is the standing body height (median per-frame
+ bbox height): scale a jump strip — whose cells include air room — by `body_h`, not `h`,
+ and it reads the same size as a walk strip. `delay_ms = cycle_seconds / frames`, so a
+ 24 fps clip yields 41.67 ms cells; render at 24 fps to keep one cell per frame
+ (a 30 fps render of 24 fps cells is a 5:4 pulldown and judders).
+- `<name>.gif` — `n_out` frames evenly across the cycle, 1-bit alpha, disposal 2, `loop=0`.
+- `<name>.webp` — same frames, lossless, `img2webp -exact` (Pillow's animated WebP writer
+ does not pass `exact` and rewrites RGB under transparent pixels).
+
+## 5. Set — the batch
+
+`sprite-gen video-set --base side=side.png --base front=front.png --states idle,walk,run,jump,attack --out-dir set/`
+runs canvas → video → frames → loop for every (direction, state). The xAI team quota
+is **2 requests per second** (five parallel starts produced two HTTP 429s): starts are
+staggered (`--start-gap 2`) and a 429 gets a bounded, logged retry (15 s, 30 s). Items
+are idempotent (an existing clip is reused unless `--force`); one failure stops only
+its item and is listed in `table.md` with its stage and error. Exit code is non-zero
+when any item failed.
+
+## Measured on 2026-09-08 (claudecy SD paladin, 15 loops)
+
+| Direction | State | Period (frames) | Loop seam | Notes |
+|---|---|---|---|---|
+| side | walk | 28 | 0.81 | first pass picked 39 (1.4 cycles) |
+| side | run | 17 | 0.84 | first pass picked 25 (1.5 cycles) |
+| side | jump | 61 | 0.88 | tall canvas; square clip clipped the hair |
+| side | attack | 25 | 1.72 | return pose varies — the weakest state |
+| front | idle | 134 | 0.25 | long window |
+| back | jump | 43 | 0.67 | first tall clip jumped out of frame; "modest hop" prompt |
+
+All 15 passed the 2.0 seam gate; every GIF/WebP verified transparent with `loop=0`.
diff --git a/scripts/video_canvas.py b/scripts/video_canvas.py
new file mode 100644
index 0000000..54eaddb
--- /dev/null
+++ b/scripts/video_canvas.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Backward-compatible wrapper for sprite_gen.video.canvas (`sprite-gen video-canvas`)."""
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+import sprite_gen.video.canvas as _impl
+
+globals().update({name: value for name, value in vars(_impl).items() if name not in {"__name__", "__package__", "__loader__", "__spec__"}})
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/video_frames.py b/scripts/video_frames.py
new file mode 100644
index 0000000..9909bf4
--- /dev/null
+++ b/scripts/video_frames.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Backward-compatible wrapper for sprite_gen.video.frames (`sprite-gen video-frames`)."""
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+import sprite_gen.video.frames as _impl
+
+globals().update({name: value for name, value in vars(_impl).items() if name not in {"__name__", "__package__", "__loader__", "__spec__"}})
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/video_loop.py b/scripts/video_loop.py
new file mode 100644
index 0000000..6e41851
--- /dev/null
+++ b/scripts/video_loop.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Backward-compatible wrapper for sprite_gen.video.loop (`sprite-gen video-loop`)."""
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+import sprite_gen.video.loop as _impl
+
+globals().update({name: value for name, value in vars(_impl).items() if name not in {"__name__", "__package__", "__loader__", "__spec__"}})
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/video_set.py b/scripts/video_set.py
new file mode 100644
index 0000000..18cb22f
--- /dev/null
+++ b/scripts/video_set.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Backward-compatible wrapper for sprite_gen.video.batch (`sprite-gen video-set`)."""
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+import sprite_gen.video.batch as _impl
+
+globals().update({name: value for name, value in vars(_impl).items() if name not in {"__name__", "__package__", "__loader__", "__spec__"}})
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/sprite_gen/_modules.py b/sprite_gen/_modules.py
index 312436d..e2ab5f9 100644
--- a/sprite_gen/_modules.py
+++ b/sprite_gen/_modules.py
@@ -12,6 +12,10 @@ MODULE_DOMAIN = {
'migrate_breathe': 'spec',
'generate_image': 'gen',
'video': 'gen',
+ 'canvas': 'video',
+ 'frames': 'video',
+ 'loop': 'video',
+ 'batch': 'video',
'prepare': 'gen',
'extract': 'frames',
'cutout': 'frames',
diff --git a/sprite_gen/cli.py b/sprite_gen/cli.py
index 3817f61..2ffc354 100644
--- a/sprite_gen/cli.py
+++ b/sprite_gen/cli.py
@@ -13,6 +13,10 @@ from sprite_gen.compose import compose_atlas, compose_cycle, compose_gif, compos
from sprite_gen.qa import correction_loop, inspect, preview, score
from sprite_gen.frames import cutout, extract, slice_sheet, unpack_atlas
from sprite_gen.gen import prepare, video
+from sprite_gen.video import batch as video_batch
+from sprite_gen.video import canvas as video_canvas
+from sprite_gen.video import frames as video_frames
+from sprite_gen.video import loop as video_loop
from sprite_gen.effects import recolor
from sprite_gen.serve import serve_compose, serve_curation
from sprite_gen.spec import migrate_breathe, migrate_request
@@ -294,6 +298,27 @@ COMMANDS: dict[str, tuple[str, Callable[[argparse.ArgumentParser], None], Callab
_add_video,
video.run,
),
+ # Video -> sprite pipeline (docs/video-pipeline.md): canvas -> video -> frames -> loop, or video-set for a batch.
+ "video-canvas": (
+ "Pad a base still into the canvas a motion state needs (tall for jumps, wide for attacks, square otherwise).",
+ video_canvas.add_arguments,
+ video_canvas.run,
+ ),
+ "video-frames": (
+ "Extract a clip's frames and key the chroma out into RGBA frames (edge-contact checked).",
+ video_frames.add_arguments,
+ video_frames.run,
+ ),
+ "video-loop": (
+ "Find the true period in keyed frames and emit one seamless cycle: strip + meta, transparent GIF, WebP.",
+ video_loop.add_arguments,
+ video_loop.run,
+ ),
+ "video-set": (
+ "Directions x states end to end (canvas -> video -> frames -> loop), rate-limit aware, one report per item.",
+ video_batch.add_arguments,
+ video_batch.run,
+ ),
# The argument surface is `serve_curation.add_arguments` itself, not a copy of it: the
# webview's own `--help` and this subcommand are the same declaration, so `sprite-gen
# curation` cannot drift from the launch form the READMEs still document.
diff --git a/sprite_gen/video/__init__.py b/sprite_gen/video/__init__.py
new file mode 100644
index 0000000..21c3b09
--- /dev/null
+++ b/sprite_gen/video/__init__.py
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Video-to-sprite pipeline: one still -> a Grok Imagine clip -> keyed frames -> a
+transparent, seamless loop (cycle frames, strip, GIF, WebP) with a report at every stage.
+
+Stages, each its own module and CLI verb (contract: docs/video-pipeline.md):
+
+- `canvas` `sprite-gen video-canvas` pad the base still into the state's canvas
+ (tall for jumps, wide for attacks, square otherwise)
+- `frames` `sprite-gen video-frames` extract the clip's frames and key the chroma out
+- `loop` `sprite-gen video-loop` find the true period, cut one cycle, emit strip/GIF/WebP
+- `batch` `sprite-gen video-set` directions x states, rate-limit aware, one report each
+
+`sprite-gen video` (the clip generation itself) lives in `sprite_gen.gen.video`.
+"""
diff --git a/sprite_gen/video/batch.py b/sprite_gen/video/batch.py
new file mode 100644
index 0000000..8726c60
--- /dev/null
+++ b/sprite_gen/video/batch.py
@@ -0,0 +1,224 @@
+# SPDX-License-Identifier: Apache-2.0
+"""`sprite-gen video-set` — directions x states, end to end, one report per item.
+
+For every (direction, state) pair: canvas -> `sprite-gen video` -> frames -> loop.
+Clip generation is rate-limited by the xAI team quota (2 requests/second measured
+2026-09-08: five parallel POSTs produced two HTTP 429s), so starts are staggered
+and a 429 gets a bounded, logged retry. Stages are idempotent — an item whose
+clip already exists reuses it unless `--force` — and a failure stops only that
+item, never the batch. The batch ends with `set.report.json` and `table.md`; an
+item that failed is listed with its stage and error, never silently dropped.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import subprocess
+import sys
+import threading
+import time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any, Callable
+
+from sprite_gen.spec.runio import atomic_write_text
+from sprite_gen.video import canvas as canvas_mod
+from sprite_gen.video import frames as frames_mod
+from sprite_gen.video import loop as loop_mod
+
+START_GAP_SECONDS = 2.0
+RETRY_BACKOFF_SECONDS = (15, 30)
+VIEW_TEXT = {
+ "side": "seen from the exact side, facing right",
+ "front": "seen from the front, facing the viewer directly",
+ "back": "seen from directly behind, facing away from the viewer",
+}
+MOTION_TEXT = {
+ "idle": "stands in place in a relaxed idle: slow gentle breathing, a subtle weight sway, one natural eye blink. Feet never move.",
+ "walk": "walks in place on a treadmill: a steady, natural bipedal walk cycle, legs alternating with clear foot contacts, arms swinging gently.",
+ "run": "runs in place on a treadmill: a fast, energetic bipedal run cycle, knees lifting high, arms pumping, clear alternating strides.",
+ "jump": "performs a modest vertical hop in place over and over: crouch, spring up about half the body height, land softly, return to the exact starting stance, repeat at an even rhythm. Same height every time.",
+ "attack": "performs the same melee attack over and over: one clean strike in front, then returns to the exact ready stance, repeating at an even rhythm.",
+}
+COMMON_TEXT = (
+ "2D game sprite animation. The character {motion} The character is {view}. Stays centered in the frame and does "
+ "not move across the screen; the body and hair always stay fully inside the frame with margin. Camera completely "
+ "locked, no zoom, no pan, no reframing. The background stays a perfectly flat, pure chroma-key fill for the whole "
+ "clip — no shadows, no ground line, no particles, no lighting changes, no effects. Keep the design, colors and "
+ "proportions exactly as in the image. Consistent, evenly paced motion so the animation loops."
+)
+
+_start_lock = threading.Lock()
+_last_start = [0.0]
+
+
+def _staggered_start(gap: float) -> None:
+ with _start_lock:
+ wait = _last_start[0] + gap - time.monotonic()
+ if wait > 0:
+ time.sleep(wait)
+ _last_start[0] = time.monotonic()
+
+
+def build_prompt(direction: str, state: str, character: str | None) -> str:
+ motion = MOTION_TEXT.get(state, f"performs the '{state}' action in place, repeating at an even rhythm.")
+ view = VIEW_TEXT.get(direction, f"seen from the {direction}")
+ text = COMMON_TEXT.format(motion=motion, view=view)
+ return text.replace("The character", character, 1) if character else text
+
+
+def run_video_cli(image: Path, prompt: str, out: Path, report: Path, *, duration: int, resolution: str, log: Path) -> int:
+ cmd = [sys.executable, "-m", "sprite_gen.gen.video", "--image", str(image), "--prompt", prompt, "--out", str(out), "--duration", str(duration), "--resolution", resolution, "--no-audio", "--report", str(report)]
+ with log.open("w", encoding="utf-8") as fh:
+ return subprocess.run(cmd, stdout=fh, stderr=subprocess.STDOUT, text=True).returncode
+
+
+def run_item(
+ *,
+ item: str,
+ direction: str,
+ state: str,
+ base: Path,
+ root: Path,
+ character: str | None,
+ duration: int,
+ resolution: str,
+ key: str,
+ force: bool,
+ gap: float,
+ video_runner: Callable[..., int] = run_video_cli,
+) -> dict[str, Any]:
+ item_dir = root / item
+ item_dir.mkdir(parents=True, exist_ok=True)
+ result: dict[str, Any] = {"item": item, "direction": direction, "state": state, "dir": str(item_dir)}
+ try:
+ canvas_png = item_dir / "canvas.png"
+ canvas_report = canvas_mod.run_canvas(base, canvas_png, state=state, shape=None, facing="right" if direction != "left" else "left", headroom=None, lead=None, report_path=item_dir / "canvas.report.json")
+ result["canvas"] = {k: canvas_report[k] for k in ("shape", "canvas", "offset")}
+
+ clip = item_dir / "clip.mp4"
+ clip_report = item_dir / "clip.report.json"
+ if clip.exists() and clip_report.exists() and not force:
+ result["clip"] = {"reused": True}
+ else:
+ prompt = build_prompt(direction, state, character)
+ attempts: list[int] = []
+ for attempt in range(1 + len(RETRY_BACKOFF_SECONDS)):
+ _staggered_start(gap)
+ rc = video_runner(canvas_png, prompt, clip, clip_report, duration=duration, resolution=resolution, log=item_dir / "clip.log")
+ attempts.append(rc)
+ if rc == 0 and clip.exists():
+ break
+ text = (item_dir / "clip.log").read_text(encoding="utf-8", errors="replace") if (item_dir / "clip.log").exists() else ""
+ if "HTTP 429" in text and attempt < len(RETRY_BACKOFF_SECONDS):
+ time.sleep(RETRY_BACKOFF_SECONDS[attempt])
+ continue
+ break
+ result["clip"] = {"attempts": attempts}
+ if attempts[-1] != 0 or not clip.exists():
+ raise SystemExit(f"clip generation failed after {len(attempts)} attempt(s); see {item_dir / 'clip.log'}")
+
+ fr = frames_mod.run_frames(clip, item_dir / "frames", key=key, allow_edge_contact=False, report_path=item_dir / "frames.report.json")
+ result["frames"] = {k: fr[k] for k in ("fps", "frames", "alpha_zero_pct_min", "alpha_zero_pct_max")}
+ lp = loop_mod.run_loop(Path(fr["keyed_dir"]), item_dir / "loop", fps=float(fr["fps"]), state=state, min_len=None, max_len=None, n_out=None, seam_max=loop_mod.SEAM_RATIO_MAX, name=item, report_path=item_dir / "loop.report.json")
+ result["loop"] = {"cycle": lp["cycle"]["length"], "period": lp["cycle"]["period_global"], "cycle_ratio": round(lp["cycle"]["ratio"], 3), "seam_ratio": lp["resampled_seam_ratio"], "n_out": lp["n_out"], "gif": lp["gif"]["file"], "webp": lp["webp"]["file"], "strip": lp["strip"]["path"]}
+ result["ok"] = True
+ except SystemExit as exc:
+ result["ok"] = False
+ result["error"] = str(exc)
+ return result
+
+
+def write_table(results: list[dict[str, Any]], path: Path) -> str:
+ lines = ["| direction | state | cycle | period | seam | frames | status |", "|---|---|---|---|---|---|---|"]
+ for r in results:
+ if r.get("ok"):
+ lp = r["loop"]
+ lines.append(f"| {r['direction']} | {r['state']} | {lp['cycle']} | {lp['period']} | {lp['seam_ratio']:.2f} | {lp['n_out']} | OK |")
+ else:
+ lines.append(f"| {r['direction']} | {r['state']} | - | - | - | - | FAIL: {r.get('error', '')[:80]} |")
+ text = "\n".join(lines) + "\n"
+ atomic_write_text(path, text)
+ return text
+
+
+def run_set(
+ *,
+ bases: dict[str, Path],
+ states: list[str],
+ root: Path,
+ character: str | None,
+ duration: int,
+ resolution: str,
+ key: str,
+ concurrency: int,
+ force: bool,
+ gap: float,
+ video_runner: Callable[..., int] = run_video_cli,
+) -> dict[str, Any]:
+ root = root.expanduser().resolve()
+ root.mkdir(parents=True, exist_ok=True)
+ for direction, base in bases.items():
+ if not base.is_file():
+ raise SystemExit(f"video-set: base still for '{direction}' not found: {base}")
+ items = [(f"{d}-{s}", d, s) for d in bases for s in states]
+ results: list[dict[str, Any]] = []
+ with ThreadPoolExecutor(max_workers=max(1, concurrency)) as ex:
+ futures = {ex.submit(run_item, item=i, direction=d, state=s, base=bases[d], root=root, character=character, duration=duration, resolution=resolution, key=key, force=force, gap=gap, video_runner=video_runner): i for i, d, s in items}
+ for fut in as_completed(futures):
+ r = fut.result()
+ results.append(r)
+ print(json.dumps({k: r[k] for k in ("item", "ok") if k in r} | ({"error": r["error"]} if not r.get("ok") else {"seam": r["loop"]["seam_ratio"]}), ensure_ascii=False), flush=True)
+ results.sort(key=lambda r: [i for i, _, _ in items].index(r["item"]))
+ table = write_table(results, root / "table.md")
+ payload = {"kind": "sprite-gen-video-set-report", "root": str(root), "states": states, "directions": list(bases), "ok": sum(1 for r in results if r.get("ok")), "failed": [r["item"] for r in results if not r.get("ok")], "items": results}
+ atomic_write_text(root / "set.report.json", json.dumps(payload, ensure_ascii=False, indent=2) + "\n")
+ print(table)
+ return payload
+
+
+def _parse_bases(values: list[str]) -> dict[str, Path]:
+ bases: dict[str, Path] = {}
+ for v in values:
+ if "=" not in v:
+ raise SystemExit(f"video-set: --base expects direction=path, got {v!r}")
+ d, p = v.split("=", 1)
+ bases[d.strip()] = Path(p).expanduser().resolve()
+ if not bases:
+ raise SystemExit("video-set: at least one --base direction=path is required")
+ return bases
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+ parser.add_argument("--base", action="append", default=[], help="direction=still.png (repeatable: side=..., front=..., back=...)")
+ parser.add_argument("--states", default="idle,walk,run,jump,attack", help="comma list of motion states")
+ parser.add_argument("--out-dir", required=True, type=Path, help="batch root; one folder per direction-state")
+ parser.add_argument("--character", help="short subject phrase used in the prompts (e.g. 'The chibi paladin girl')")
+ parser.add_argument("--duration", type=int, default=6)
+ parser.add_argument("--resolution", default="720p")
+ parser.add_argument("--key", choices=("auto", "green", "magenta", "white"), default="auto")
+ parser.add_argument("--concurrency", type=int, default=3, help="parallel clip generations (starts are staggered regardless)")
+ parser.add_argument("--start-gap", type=float, default=START_GAP_SECONDS, help="seconds between clip request starts")
+ parser.add_argument("--force", action="store_true", help="regenerate clips that already exist")
+
+
+def run(**kwargs: object) -> int:
+ payload = run_set(
+ bases=_parse_bases(list(kwargs.get("base") or [])), # type: ignore[arg-type]
+ states=[s.strip() for s in str(kwargs.get("states") or "").split(",") if s.strip()],
+ root=Path(str(kwargs["out_dir"])), character=kwargs.get("character"), # type: ignore[arg-type]
+ duration=int(kwargs.get("duration") or 6), resolution=str(kwargs.get("resolution") or "720p"), key=str(kwargs.get("key") or "auto"),
+ concurrency=int(kwargs.get("concurrency") or 3), force=bool(kwargs.get("force")), gap=float(kwargs.get("start_gap") or START_GAP_SECONDS),
+ )
+ return 0 if not payload["failed"] else 1
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(prog="sprite-gen video-set", description=__doc__)
+ add_arguments(parser)
+ return run(**vars(parser.parse_args(argv)))
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/sprite_gen/video/canvas.py b/sprite_gen/video/canvas.py
new file mode 100644
index 0000000..2e5f1b2
--- /dev/null
+++ b/sprite_gen/video/canvas.py
@@ -0,0 +1,187 @@
+# SPDX-License-Identifier: Apache-2.0
+"""`sprite-gen video-canvas` — pad a base still into the canvas a motion state needs.
+
+Grok Imagine keeps the input image's framing and ignores `aspect_ratio` on
+image-to-video (2026-09-08 실측: a `3:4` request still returned 960x960). So the
+canvas is decided HERE, on the still: a jump needs head-room above (tall), an
+attack or projectile needs room in front (wide), everything else stays square.
+The state -> canvas table below is the single owner of that rule; `--shape`
+overrides it per call.
+
+The padding is filled with the still's own corner colour (its chroma key) so the
+clip stays keyable end to end. A still whose corners disagree is refused — a
+non-flat background cannot be extended without guessing.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from PIL import Image
+
+from sprite_gen.spec.runio import atomic_write_text
+
+SHAPE_SQUARE = "square"
+SHAPE_TALL = "tall"
+SHAPE_WIDE = "wide"
+SHAPES = (SHAPE_SQUARE, SHAPE_TALL, SHAPE_WIDE)
+
+
+@dataclass(frozen=True)
+class CanvasProfile:
+ shape: str
+ ratio: float # width / height
+ headroom: float # fraction of the canvas height kept empty ABOVE the subject (tall)
+ lead: float # fraction of the canvas width kept empty IN FRONT of the subject (wide)
+ why: str
+
+
+# The one table. Keys are state names as the sprite-request uses them; unknown
+# states fall through to `default`.
+STATE_CANVAS: dict[str, CanvasProfile] = {
+ "jump": CanvasProfile(SHAPE_TALL, 3 / 4, 0.34, 0.0, "airborne frames need head-room; hair clipped at 1:1"),
+ "attack": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.28, "weapon swings and projectiles extend in front"),
+ "projectile": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.34, "projectile travels away from the body"),
+ "default": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, "in-place motion fits the still's own frame"),
+}
+SHAPE_DEFAULTS: dict[str, CanvasProfile] = {
+ SHAPE_SQUARE: STATE_CANVAS["default"],
+ SHAPE_TALL: STATE_CANVAS["jump"],
+ SHAPE_WIDE: STATE_CANVAS["attack"],
+}
+CORNER_TOLERANCE = 24 # max per-channel spread across the four corners for a "flat" background
+
+
+def profile_for(state: str | None, shape: str | None = None) -> CanvasProfile:
+ """Resolve the canvas profile: an explicit `shape` wins, else the state's row, else default."""
+ if shape is not None:
+ if shape not in SHAPES:
+ raise SystemExit(f"video-canvas: unknown --shape {shape!r}; expected one of {', '.join(SHAPES)}")
+ return SHAPE_DEFAULTS[shape]
+ key = (state or "").strip().lower()
+ return STATE_CANVAS.get(key, STATE_CANVAS["default"])
+
+
+def corner_key(image: Image.Image) -> tuple[int, int, int]:
+ """The still's flat background colour, read from its four corners; refuses a non-flat one."""
+ rgb = image.convert("RGB")
+ w, h = rgb.size
+ corners = [rgb.getpixel(p) for p in ((0, 0), (w - 1, 0), (0, h - 1), (w - 1, h - 1))]
+ for channel in range(3):
+ values = [c[channel] for c in corners]
+ if max(values) - min(values) > CORNER_TOLERANCE:
+ raise SystemExit(
+ f"video-canvas: the still's corners are not one flat colour ({corners}); "
+ "generate the base on a flat chroma key before padding it"
+ )
+ return tuple(round(sum(c[i] for c in corners) / 4) for i in range(3)) # type: ignore[return-value]
+
+
+def pad_canvas(
+ image: Image.Image,
+ profile: CanvasProfile,
+ *,
+ facing: str = "right",
+ headroom: float | None = None,
+ lead: float | None = None,
+) -> tuple[Image.Image, dict[str, Any]]:
+ """Return (padded RGB image, placement report). Never downsizes the still."""
+ key = corner_key(image)
+ src = image.convert("RGB")
+ w, h = src.size
+ head = profile.headroom if headroom is None else headroom
+ front = profile.lead if lead is None else lead
+ if not 0 <= head < 0.9 or not 0 <= front < 0.9:
+ raise SystemExit("video-canvas: --headroom/--lead must be in [0, 0.9)")
+ if profile.shape == SHAPE_SQUARE:
+ side = max(w, h)
+ canvas_w, canvas_h = side, side
+ x, y = (side - w) // 2, side - h
+ elif profile.shape == SHAPE_TALL:
+ # the still becomes the bottom (1 - headroom) of a canvas at least as tall as the
+ # profile ratio demands for the still's width — never narrower than the still
+ canvas_h = max(h, round(h / (1 - head)), round(w / profile.ratio))
+ canvas_w = max(w, round(canvas_h * profile.ratio))
+ x, y = (canvas_w - w) // 2, canvas_h - h
+ else: # wide: extra width goes in front of the facing direction; at least the profile ratio
+ canvas_w = max(w, round(w / (1 - front)), round(h * profile.ratio))
+ canvas_h = max(h, round(canvas_w / profile.ratio))
+ y = canvas_h - h
+ x = 0 if facing == "right" else canvas_w - w
+ canvas = Image.new("RGB", (canvas_w, canvas_h), key)
+ canvas.paste(src, (x, y))
+ report = {
+ "shape": profile.shape,
+ "ratio": round(canvas_w / canvas_h, 4),
+ "canvas": [canvas_w, canvas_h],
+ "still": [w, h],
+ "offset": [x, y],
+ "headroom": head,
+ "lead": front,
+ "facing": facing,
+ "key_rgb": list(key),
+ "why": profile.why,
+ }
+ return canvas, report
+
+
+def run_canvas(
+ still: Path,
+ out: Path,
+ *,
+ state: str | None,
+ shape: str | None,
+ facing: str,
+ headroom: float | None,
+ lead: float | None,
+ report_path: Path | None,
+) -> dict[str, Any]:
+ still = still.expanduser().resolve()
+ if not still.is_file():
+ raise SystemExit(f"video-canvas: still not found: {still}")
+ profile = profile_for(state, shape)
+ canvas, report = pad_canvas(Image.open(still), profile, facing=facing, headroom=headroom, lead=lead)
+ out = out.expanduser().resolve()
+ out.parent.mkdir(parents=True, exist_ok=True)
+ tmp = out.with_name(out.name + ".part")
+ canvas.save(tmp, format="PNG")
+ tmp.replace(out)
+ payload = {"kind": "sprite-gen-video-canvas-report", "still": str(still), "out": str(out), "state": state, **report}
+ if report_path is not None:
+ atomic_write_text(report_path.expanduser().resolve(), json.dumps(payload, ensure_ascii=False, indent=2) + "\n")
+ return payload
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+ parser.add_argument("--still", required=True, type=Path, help="base still on a flat chroma background")
+ parser.add_argument("--out", required=True, type=Path, help="padded PNG to feed `sprite-gen video`")
+ parser.add_argument("--state", help="motion state name (jump/attack/projectile/... — selects the canvas row)")
+ parser.add_argument("--shape", choices=SHAPES, help="override the state's canvas shape")
+ parser.add_argument("--facing", choices=("right", "left"), default="right", help="which way the subject faces (wide canvases add room in front)")
+ parser.add_argument("--headroom", type=float, help="tall: empty fraction above the subject (default from the profile)")
+ parser.add_argument("--lead", type=float, help="wide: empty fraction in front of the subject (default from the profile)")
+ parser.add_argument("--report", type=Path, help="write the canvas report JSON here")
+
+
+def run(**kwargs: object) -> int:
+ payload = run_canvas(
+ Path(str(kwargs["still"])), Path(str(kwargs["out"])),
+ state=kwargs.get("state"), shape=kwargs.get("shape"), facing=str(kwargs.get("facing") or "right"), # type: ignore[arg-type]
+ headroom=kwargs.get("headroom"), lead=kwargs.get("lead"), report_path=kwargs.get("report"), # type: ignore[arg-type]
+ )
+ print(json.dumps(payload, ensure_ascii=False, indent=2))
+ return 0
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(prog="sprite-gen video-canvas", description=__doc__)
+ add_arguments(parser)
+ return run(**vars(parser.parse_args(argv)))
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/sprite_gen/video/frames.py b/sprite_gen/video/frames.py
new file mode 100644
index 0000000..e754fc5
--- /dev/null
+++ b/sprite_gen/video/frames.py
@@ -0,0 +1,165 @@
+# SPDX-License-Identifier: Apache-2.0
+"""`sprite-gen video-frames` — turn a generated clip into keyed RGBA frames.
+
+ffmpeg extracts every frame (the clip's own fps is recorded, not assumed), then
+each frame goes through the same `cutout` engine imported stills use (chroma
+green/magenta -> the extract matte; `auto` reads the corners). The report carries
+per-frame alpha coverage and an **edge-contact check**: a frame whose subject
+touches the top/left/right edge means the model framed too tight (a jump whose
+hair left the frame, 2026-09-08) and the run fails loud — the fix is a taller or
+wider canvas (`video-canvas`), not a quiet crop.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import shutil
+import subprocess
+from pathlib import Path
+from typing import Any
+
+from PIL import Image
+
+from sprite_gen.frames.cutout import cutout
+from sprite_gen.spec.runio import atomic_write_text
+
+EDGE_ROWS = 4 # rows/cols inspected at each edge
+EDGE_MAX_PIXELS = 0 # any opaque pixel on the top/left/right edge band = contact
+
+
+def _require(binary: str) -> str:
+ path = shutil.which(binary)
+ if not path:
+ raise SystemExit(f"video-frames: `{binary}` not found on PATH — install ffmpeg (brew install ffmpeg)")
+ return path
+
+
+def probe(clip: Path) -> dict[str, Any]:
+ ffprobe = _require("ffprobe")
+ proc = subprocess.run(
+ [ffprobe, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,r_frame_rate,nb_frames",
+ "-show_entries", "format=duration", "-of", "json", str(clip)],
+ capture_output=True, text=True,
+ )
+ if proc.returncode != 0:
+ raise SystemExit(f"video-frames: ffprobe failed on {clip}: {proc.stderr.strip()[:300]}")
+ data = json.loads(proc.stdout)
+ stream = (data.get("streams") or [{}])[0]
+ num, den = (stream.get("r_frame_rate") or "24/1").split("/")
+ fps = float(num) / float(den or 1)
+ return {"width": stream.get("width"), "height": stream.get("height"), "fps": fps,
+ "nb_frames": int(stream["nb_frames"]) if str(stream.get("nb_frames", "")).isdigit() else None,
+ "duration": float(data.get("format", {}).get("duration") or 0)}
+
+
+def extract(clip: Path, raw_dir: Path) -> list[Path]:
+ ffmpeg = _require("ffmpeg")
+ raw_dir.mkdir(parents=True, exist_ok=True)
+ for old in raw_dir.glob("frame-*.png"):
+ old.unlink()
+ proc = subprocess.run([ffmpeg, "-v", "error", "-y", "-i", str(clip), str(raw_dir / "frame-%04d.png")], capture_output=True, text=True)
+ if proc.returncode != 0:
+ raise SystemExit(f"video-frames: ffmpeg failed on {clip}: {proc.stderr.strip()[:300]}")
+ files = sorted(raw_dir.glob("frame-*.png"))
+ if not files:
+ raise SystemExit(f"video-frames: ffmpeg wrote no frames for {clip}")
+ return files
+
+
+def edge_contact(image: Image.Image) -> dict[str, int]:
+ """Opaque pixels in the top/left/right edge bands (the floor edge is expected contact)."""
+ alpha = image.convert("RGBA").getchannel("A")
+ w, h = alpha.size
+ px = alpha.load()
+ top = sum(1 for y in range(min(EDGE_ROWS, h)) for x in range(w) if px[x, y] > 0)
+ left = sum(1 for x in range(min(EDGE_ROWS, w)) for y in range(h) if px[x, y] > 0)
+ right = sum(1 for x in range(max(0, w - EDGE_ROWS), w) for y in range(h) if px[x, y] > 0)
+ return {"top": top, "left": left, "right": right}
+
+
+def key_frames(
+ raw_files: list[Path],
+ keyed_dir: Path,
+ *,
+ key: str = "auto",
+ check_edges: bool = True,
+) -> dict[str, Any]:
+ keyed_dir.mkdir(parents=True, exist_ok=True)
+ rows: list[dict[str, Any]] = []
+ contacts: list[dict[str, Any]] = []
+ for src in raw_files:
+ dst = keyed_dir / src.name
+ stats = cutout(src, dst, key=key)
+ image = Image.open(dst).convert("RGBA")
+ hist = image.getchannel("A").histogram()
+ w, h = image.size
+ alpha_zero_pct = round(100 * hist[0] / (w * h), 2)
+ row = {"frame": src.name, "alpha_zero_pct": alpha_zero_pct, "route": stats.get("route")}
+ if check_edges:
+ contact = edge_contact(image)
+ row["edge"] = contact
+ if any(v > EDGE_MAX_PIXELS for v in contact.values()):
+ contacts.append({"frame": src.name, **contact})
+ if hist[0] == 0:
+ raise SystemExit(f"video-frames: {src.name} keyed to 0% transparency — the background is not the chroma key")
+ rows.append(row)
+ report = {
+ "frames": len(rows),
+ "alpha_zero_pct_min": min(r["alpha_zero_pct"] for r in rows),
+ "alpha_zero_pct_max": max(r["alpha_zero_pct"] for r in rows),
+ "edge_contacts": contacts,
+ "rows": rows,
+ }
+ if contacts and check_edges:
+ worst = max(contacts, key=lambda c: c["top"] + c["left"] + c["right"])
+ raise SystemExit(
+ f"video-frames: the subject touches the frame edge in {len(contacts)} frame(s) "
+ f"(worst {worst['frame']}: top {worst['top']} / left {worst['left']} / right {worst['right']} px) — "
+ "the clip was framed too tight; regenerate from a taller/wider canvas (`sprite-gen video-canvas`) "
+ "or pass --allow-edge-contact to accept the clipping"
+ )
+ return report
+
+
+def run_frames(clip: Path, out_dir: Path, *, key: str, allow_edge_contact: bool, report_path: Path | None) -> dict[str, Any]:
+ clip = clip.expanduser().resolve()
+ if not clip.is_file():
+ raise SystemExit(f"video-frames: clip not found: {clip}")
+ out_dir = out_dir.expanduser().resolve()
+ meta = probe(clip)
+ raw_dir = out_dir / "raw"
+ keyed_dir = out_dir / "keyed"
+ files = extract(clip, raw_dir)
+ report = key_frames(files, keyed_dir, key=key, check_edges=not allow_edge_contact)
+ payload = {"kind": "sprite-gen-video-frames-report", "clip": str(clip), "out_dir": str(out_dir), "raw_dir": str(raw_dir), "keyed_dir": str(keyed_dir), "key": key, **meta, **report}
+ target = (report_path or (out_dir / "frames.report.json")).expanduser().resolve()
+ atomic_write_text(target, json.dumps(payload, ensure_ascii=False, indent=2) + "\n")
+ payload["report"] = str(target)
+ return payload
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+ parser.add_argument("--clip", required=True, type=Path, help="mp4 from `sprite-gen video`")
+ parser.add_argument("--out-dir", required=True, type=Path, help="writes raw/ and keyed/ here")
+ parser.add_argument("--key", choices=("auto", "green", "magenta", "white"), default="auto", help="background key (auto reads the corners)")
+ parser.add_argument("--allow-edge-contact", action="store_true", help="accept frames whose subject touches the top/left/right edge")
+ parser.add_argument("--report", type=Path, help="report JSON (default <out-dir>/frames.report.json)")
+
+
+def run(**kwargs: object) -> int:
+ payload = run_frames(Path(str(kwargs["clip"])), Path(str(kwargs["out_dir"])), key=str(kwargs.get("key") or "auto"),
+ allow_edge_contact=bool(kwargs.get("allow_edge_contact")), report_path=kwargs.get("report")) # type: ignore[arg-type]
+ summary = {k: payload[k] for k in ("clip", "keyed_dir", "fps", "frames", "alpha_zero_pct_min", "alpha_zero_pct_max", "edge_contacts", "report")}
+ print(json.dumps(summary, ensure_ascii=False, indent=2))
+ return 0
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(prog="sprite-gen video-frames", description=__doc__)
+ add_arguments(parser)
+ return run(**vars(parser.parse_args(argv)))
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/sprite_gen/video/loop.py b/sprite_gen/video/loop.py
new file mode 100644
index 0000000..ce8b17c
--- /dev/null
+++ b/sprite_gen/video/loop.py
@@ -0,0 +1,392 @@
+# SPDX-License-Identifier: Apache-2.0
+"""`sprite-gen video-loop` — cut one seamless cycle out of keyed frames and emit the
+loop as cycle frames, a horizontal strip (+ metadata), a transparent GIF and a WebP.
+
+Period first, seam second. A single-start "most similar later frame" search lands
+on the 1.5-cycle look-alike of a gait (legs swapped) and produces a loop that
+hitches at the wrap (2026-09-08 실측: side walk picked 39 frames where the period
+was 28, run picked 25 where it was 17). So the true period is read from the
+GLOBAL profile `P[L] = mean_j |f[j] - f[j+L]|` — its deepest local minimum inside
+the state's window — and only then is the start chosen as the best seam for that
+period. Idle motion is tiny and not strictly periodic: its window is opened to
+most of the clip, where the seam is lowest.
+
+Everything downstream is measured, never assumed: the seam ratio (wrap distance
+over the mean adjacent-frame distance inside the cycle) gates the run, the GIF and
+WebP are re-opened and verified (frame count, loop flag, transparent corners,
+no stale RGB under alpha 0). The strip metadata carries `body_h` — the standing
+body height (median per-frame bbox height) — so a consumer can scale a jump strip,
+whose cells include air room, to the same on-screen body size as a walk strip.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import os
+import shutil
+import subprocess
+from collections import deque
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from PIL import Image
+
+from sprite_gen._deps import np
+from sprite_gen.spec.runio import atomic_write_text
+from sprite_gen.util.gif_utils import save_clean_gif
+
+ANALYSIS_SIZE = 96 # thumbnail edge for the distance matrix
+STRIP_MAX_CELLS = 64 # Chrome caps image dimensions near 32767 px; 64 cells keeps strips renderable
+STRIP_MAX_HEIGHT = 520
+SEAM_RATIO_MAX = 2.0 # loop seam / mean adjacent distance inside the cycle
+SPECK_MIN_FRACTION = 0.01 # detached components smaller than this fraction of the body are keying specks
+PERIODICITY_MIN = 0.15 # the period must dip at least 15% below the profile mean (flat profile = no repeat)
+
+
+@dataclass(frozen=True)
+class LoopProfile:
+ min_frac: float # window as a fraction of the clip's frame count
+ max_frac: float
+ n_out: int # frames in the resampled GIF/WebP
+ why: str
+ periodic: bool = True # gate: the profile must show a real period (idle is exempt)
+
+
+# State -> detection window. Fractions of the clip length so 6 s and 10 s clips both work.
+STATE_PROFILES: dict[str, LoopProfile] = {
+ "idle": LoopProfile(0.60, 0.95, 16, "breathing is slow and not strictly periodic; the lowest seam is a long window", periodic=False),
+ "walk": LoopProfile(0.10, 0.31, 12, "full gait = two steps; window excludes the one-step half period"),
+ "run": LoopProfile(0.07, 0.23, 10, "faster gait"),
+ "jump": LoopProfile(0.11, 0.45, 12, "crouch-spring-land-return"),
+ "attack": LoopProfile(0.11, 0.45, 12, "swing and return to ready"),
+ "default": LoopProfile(0.10, 0.45, 12, "generic in-place action"),
+}
+
+
+def profile_for(state: str | None) -> LoopProfile:
+ return STATE_PROFILES.get((state or "").strip().lower(), STATE_PROFILES["default"])
+
+
+def _load_small(path: Path) -> np.ndarray:
+ im = Image.open(path).convert("RGBA")
+ im.thumbnail((ANALYSIS_SIZE, ANALYSIS_SIZE))
+ a = np.asarray(im, dtype=np.float32) / 255.0
+ rgb = a[..., :3] * a[..., 3:4] # premultiplied: transparent pixels contribute 0
+ return np.concatenate([rgb, a[..., 3:4]], axis=-1).reshape(-1)
+
+
+def distance_matrix(files: list[Path]) -> np.ndarray:
+ flat = np.stack([_load_small(f) for f in files])
+ n = len(files)
+ D = np.zeros((n, n), dtype=np.float32)
+ for i in range(n):
+ D[i] = np.abs(flat - flat[i]).mean(axis=1)
+ return D
+
+
+def detect_cycle(D: np.ndarray, *, min_len: int, max_len: int) -> dict[str, Any]:
+ """Global period (deepest local minimum of the averaged profile) then the best-seam start."""
+ n = D.shape[0]
+ max_len = min(max_len, n - 2)
+ if min_len < 2 or max_len < min_len:
+ raise SystemExit(f"video-loop: window [{min_len}, {max_len}] is empty for {n} frames")
+ adjacent = np.array([D[i, i + 1] for i in range(n - 1)])
+ prof: dict[int, float] = {}
+ for L in range(max(2, min_len // 2), max_len + 1):
+ prof[L] = float(np.mean([D[j, j + L] for j in range(0, n - L, 2)]))
+ cands = [L for L in range(min_len, max_len + 1) if L - 1 in prof and L + 1 in prof and prof[L] <= prof[L - 1] and prof[L] <= prof[L + 1]]
+ if not cands:
+ cands = list(range(min_len, max_len + 1))
+ # The true period is the SMALLEST minimum that is about as deep as the deepest one:
+ # exact repeats also dip at 2x and 3x the period, and a half-period look-alike dips
+ # noticeably less (near/far limb difference) — so a 15% depth tolerance separates both.
+ deepest = min(prof[L] for L in cands)
+ period = min(L for L in cands if prof[L] <= deepest * 1.15 + 1e-4) # abs floor: exact repeats sit at ~0
+ profile_mean = float(np.mean([prof[L] for L in prof]))
+ periodicity = (profile_mean - prof[period]) / profile_mean if profile_mean > 0 else 0.0
+ best: dict[str, Any] | None = None
+ for L in (period - 1, period, period + 1):
+ if L < min_len or L > max_len:
+ continue
+ for i in range(0, n - L):
+ seam = float(D[i, i + L])
+ inner = float(adjacent[i : i + L - 1].mean())
+ ratio = seam / inner if inner > 0 else math.inf
+ if best is None or ratio < best["ratio"]:
+ best = {"start": i, "length": L, "seam": seam, "inner_mean_adjacent": inner, "ratio": ratio}
+ assert best is not None
+ best["period_global"] = period
+ best["periodicity"] = round(periodicity, 4) # how far below the profile mean the period dips (0 = flat = no period)
+ best["profile_minima"] = [[L, round(prof[L], 5)] for L in sorted(cands, key=lambda L: prof[L])[:6]]
+ return best
+
+
+def _drop_specks(image: Image.Image, min_fraction: float) -> tuple[Image.Image, int]:
+ """Erase detached alpha components smaller than `min_fraction` of the largest one."""
+ a = np.asarray(image)[..., 3] > 16
+ H, W = a.shape
+ seen = np.zeros_like(a, dtype=bool)
+ comps: list[list[tuple[int, int]]] = []
+ for y in range(H):
+ for x in range(W):
+ if a[y, x] and not seen[y, x]:
+ q = deque([(y, x)])
+ seen[y, x] = True
+ pts: list[tuple[int, int]] = []
+ while q:
+ cy, cx = q.popleft()
+ pts.append((cy, cx))
+ for dy, dx in ((1, 0), (-1, 0), (0, 1), (0, -1)):
+ ny, nx = cy + dy, cx + dx
+ if 0 <= ny < H and 0 <= nx < W and a[ny, nx] and not seen[ny, nx]:
+ seen[ny, nx] = True
+ q.append((ny, nx))
+ comps.append(pts)
+ if not comps:
+ return image, 0
+ big = max(len(c) for c in comps)
+ px = image.load()
+ dropped = 0
+ for c in comps:
+ if len(c) < max(8, big * min_fraction):
+ for y, x in c:
+ px[x, y] = (0, 0, 0, 0)
+ dropped += 1
+ return image, dropped
+
+
+def _scrub(image: Image.Image) -> int:
+ px = image.load()
+ n = 0
+ for y in range(image.height):
+ for x in range(image.width):
+ r, g, b, a = px[x, y]
+ if a == 0 and (r or g or b):
+ px[x, y] = (0, 0, 0, 0)
+ n += 1
+ return n
+
+
+def build_strip(frames: list[Image.Image], *, max_cells: int = STRIP_MAX_CELLS, max_height: int = STRIP_MAX_HEIGHT, cycle_seconds: float) -> tuple[Image.Image, dict[str, Any]]:
+ """Union-crop (no bottom pad so feet meet the floor), scale, bottom-align, tile horizontally."""
+ L = len(frames)
+ idx = list(range(L))
+ if L > max_cells:
+ idx = [round(k * L / max_cells) for k in range(max_cells)]
+ chosen = [frames[i] for i in idx]
+ boxes = [im.getchannel("A").point(lambda v: 255 if v >= 8 else 0).getbbox() for im in chosen]
+ boxes = [b for b in boxes if b]
+ if not boxes:
+ raise SystemExit("video-loop: every cycle frame is fully transparent")
+ left = max(0, min(b[0] for b in boxes) - 8)
+ top = max(0, min(b[1] for b in boxes) - 8)
+ right = min(chosen[0].width, max(b[2] for b in boxes) + 8)
+ bottom = max(b[3] for b in boxes)
+ heights = sorted(b[3] - b[1] for b in boxes)
+ body_src = heights[len(heights) // 2]
+ scale = min(1.0, max_height / (bottom - top))
+ w = round((right - left) * scale)
+ h = round((bottom - top) * scale)
+ cells = [im.crop((left, top, right, bottom)).resize((w, h), Image.LANCZOS) for im in chosen]
+ strip = Image.new("RGBA", (w * len(cells), h), (0, 0, 0, 0))
+ for k, im in enumerate(cells):
+ strip.alpha_composite(im, (k * w, 0))
+ meta = {
+ "frames": len(cells),
+ "w": w,
+ "h": h,
+ "body_h": round(body_src * scale),
+ "delay_ms": round(1000 * cycle_seconds / len(cells), 2),
+ "cycle_frames": L,
+ "cycle_seconds": round(cycle_seconds, 4),
+ "subsampled": L > max_cells,
+ "top_margin_px": top,
+ }
+ return strip, meta
+
+
+def write_webp(frames: list[Image.Image], out: Path, *, delay_ms: int, workdir: Path) -> None:
+ """Animated WebP through libwebp's img2webp with -exact (Pillow's animated writer drops `exact`
+ and rewrites RGB under alpha 0 to white; measured 2026-09-08)."""
+ img2webp = shutil.which("img2webp")
+ if not img2webp:
+ raise SystemExit("video-loop: `img2webp` not found on PATH — install libwebp (brew install webp) for exact-alpha WebP")
+ workdir.mkdir(parents=True, exist_ok=True)
+ paths = []
+ for k, im in enumerate(frames):
+ p = workdir / f"webp-{k:03d}.png"
+ im.save(p)
+ paths.append(str(p))
+ proc = subprocess.run([img2webp, "-loop", "0", "-lossless", "-exact", "-d", str(delay_ms), *paths, "-o", str(out)], capture_output=True, text=True)
+ if proc.returncode != 0 or not out.is_file():
+ raise SystemExit(f"video-loop: img2webp failed: {proc.stderr.strip()[:300]}")
+
+
+def verify_animation(path: Path, *, expect_frames: int, check_stale: bool) -> dict[str, Any]:
+ im = Image.open(path)
+ n = getattr(im, "n_frames", 1)
+ loop = im.info.get("loop")
+ corners_ok = True
+ stale = 0
+ for k in range(n):
+ im.seek(k)
+ f = im.convert("RGBA")
+ w, h = f.size
+ corners_ok &= all(f.getpixel(c)[3] == 0 for c in ((0, 0), (w - 1, 0), (0, h - 1), (w - 1, h - 1)))
+ if check_stale:
+ stale += sum(1 for p in f.get_flattened_data() if p[3] == 0 and (p[0] or p[1] or p[2]))
+ report = {"file": path.name, "format": im.format, "n_frames": n, "loop": loop, "size": list(im.size), "corners_transparent": corners_ok, "stale_rgb_under_alpha0": stale, "bytes": path.stat().st_size}
+ problems = []
+ if n != expect_frames:
+ problems.append(f"{n} frames, expected {expect_frames}")
+ if loop != 0:
+ problems.append(f"loop={loop!r}, expected 0 (infinite)")
+ if not corners_ok:
+ problems.append("a corner is not transparent")
+ if stale:
+ problems.append(f"{stale} transparent pixels carry RGB")
+ if problems:
+ raise SystemExit(f"video-loop: {path.name} failed verification: {'; '.join(problems)}")
+ return report
+
+
+def run_loop(
+ frames_dir: Path,
+ out_dir: Path,
+ *,
+ fps: float,
+ state: str | None,
+ min_len: int | None,
+ max_len: int | None,
+ n_out: int | None,
+ seam_max: float,
+ name: str,
+ report_path: Path | None,
+) -> dict[str, Any]:
+ frames_dir = frames_dir.expanduser().resolve()
+ files = sorted(frames_dir.glob("*.png"))
+ if len(files) < 6:
+ raise SystemExit(f"video-loop: need at least 6 keyed frames in {frames_dir}, found {len(files)}")
+ prof = profile_for(state)
+ n = len(files)
+ lo = min_len if min_len is not None else max(4, round(n * prof.min_frac))
+ hi = max_len if max_len is not None else max(lo + 2, round(n * prof.max_frac))
+ n_out = n_out or prof.n_out
+ D = distance_matrix(files)
+ cycle = detect_cycle(D, min_len=lo, max_len=hi)
+ i, L = cycle["start"], cycle["length"]
+ if prof.periodic and cycle["periodicity"] < PERIODICITY_MIN:
+ raise SystemExit(
+ f"video-loop: no periodic cycle found — the period profile is flat (periodicity {cycle['periodicity']:.2f} < {PERIODICITY_MIN}) "
+ f"in window [{lo},{hi}]; the motion does not repeat, widen the window or regenerate the clip"
+ )
+ n_out = min(n_out, L) # never ask for more distinct frames than the cycle holds
+
+ out_dir = out_dir.expanduser().resolve()
+ cycle_dir = out_dir / "cycle"
+ cycle_dir.mkdir(parents=True, exist_ok=True)
+ for old in cycle_dir.glob("frame-*.png"):
+ old.unlink()
+ frames: list[Image.Image] = []
+ scrubbed = specks = 0
+ for k, f in enumerate(files[i : i + L]):
+ im = Image.open(f).convert("RGBA")
+ scrubbed += _scrub(im)
+ im, d = _drop_specks(im, SPECK_MIN_FRACTION)
+ specks += d
+ im.save(cycle_dir / f"frame-{k:03d}.png")
+ frames.append(im)
+
+ cycle_seconds = L / fps
+ strip, strip_meta = build_strip(frames, cycle_seconds=cycle_seconds)
+ strip_path = out_dir / f"{name}.strip.png"
+ strip.save(strip_path)
+ atomic_write_text(out_dir / f"{name}.strip.json", json.dumps(strip_meta, indent=2) + "\n")
+
+ # resampled GIF/WebP frames: same crop as the strip, n_out evenly across the cycle
+ idx = [min(L - 1, i2) for i2 in (round(k * L / n_out) for k in range(n_out))]
+ cells = [strip.crop((k * strip_meta["w"], 0, (k + 1) * strip_meta["w"], strip_meta["h"])) for k in range(strip_meta["frames"])]
+ pick = [cells[min(len(cells) - 1, round(j * len(cells) / L))] for j in idx]
+ seam_idx = [i + j for j in idx]
+ resampled_adjacent = float(np.mean([D[seam_idx[k], seam_idx[k + 1]] for k in range(len(seam_idx) - 1)]))
+ resampled_seam = float(D[seam_idx[-1], seam_idx[0]])
+ seam_ratio = resampled_seam / resampled_adjacent if resampled_adjacent > 0 else math.inf
+ delay_ms = max(20, round(1000 * cycle_seconds / n_out))
+ gif_path = out_dir / f"{name}.gif"
+ webp_path = out_dir / f"{name}.webp"
+ if seam_ratio > seam_max:
+ raise SystemExit(
+ f"video-loop: loop seam ratio {seam_ratio:.2f} exceeds {seam_max} (period {L} frames from {i}); "
+ "the cycle does not close — widen the window, regenerate with an 'evenly paced, returns to start' prompt, or pass --seam-max"
+ )
+ save_clean_gif(pick, gif_path, duration_ms=delay_ms, loop=0, alpha_threshold=128)
+ write_webp(pick, webp_path, delay_ms=delay_ms, workdir=out_dir / ".webp-frames")
+ shutil.rmtree(out_dir / ".webp-frames", ignore_errors=True)
+ gif_report = verify_animation(gif_path, expect_frames=n_out, check_stale=False)
+ webp_report = verify_animation(webp_path, expect_frames=n_out, check_stale=True)
+
+ payload = {
+ "kind": "sprite-gen-video-loop-report",
+ "frames_dir": str(frames_dir),
+ "out_dir": str(out_dir),
+ "state": state,
+ "fps": fps,
+ "frames_total": n,
+ "window": [lo, hi],
+ "profile": prof.why,
+ "cycle": cycle,
+ "cycle_seconds": round(cycle_seconds, 4),
+ "n_out": n_out,
+ "delay_ms": delay_ms,
+ "resampled_seam_ratio": round(seam_ratio, 4),
+ "seam_max": seam_max,
+ "scrubbed_rgb_pixels": scrubbed,
+ "specks_dropped": specks,
+ "strip": {"path": str(strip_path), **strip_meta},
+ "gif": gif_report,
+ "webp": webp_report,
+ }
+ target = (report_path or (out_dir / f"{name}.loop.report.json")).expanduser().resolve()
+ atomic_write_text(target, json.dumps(payload, ensure_ascii=False, indent=2) + "\n")
+ payload["report"] = str(target)
+ return payload
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+ parser.add_argument("--frames-dir", required=True, type=Path, help="keyed RGBA frames from `sprite-gen video-frames` (…/keyed)")
+ parser.add_argument("--out-dir", required=True, type=Path)
+ parser.add_argument("--fps", type=float, default=24.0, help="frame rate of the keyed frames (from the frames report)")
+ parser.add_argument("--state", help="motion state (idle/walk/run/jump/attack) — selects the detection window")
+ parser.add_argument("--min-len", type=int, help="override: minimum cycle length in frames")
+ parser.add_argument("--max-len", type=int, help="override: maximum cycle length in frames")
+ parser.add_argument("--n-out", type=int, help="frames in the GIF/WebP (default from the state profile)")
+ parser.add_argument("--seam-max", type=float, default=SEAM_RATIO_MAX, help=f"loop seam gate (default {SEAM_RATIO_MAX})")
+ parser.add_argument("--name", default="loop", help="basename for strip/gif/webp outputs")
+ parser.add_argument("--report", type=Path)
+
+
+def run(**kwargs: object) -> int:
+ payload = run_loop(
+ Path(str(kwargs["frames_dir"])), Path(str(kwargs["out_dir"])),
+ fps=float(kwargs.get("fps") or 24.0), state=kwargs.get("state"), # type: ignore[arg-type]
+ min_len=kwargs.get("min_len"), max_len=kwargs.get("max_len"), n_out=kwargs.get("n_out"), # type: ignore[arg-type]
+ seam_max=float(kwargs.get("seam_max") or SEAM_RATIO_MAX), name=str(kwargs.get("name") or "loop"), report_path=kwargs.get("report"), # type: ignore[arg-type]
+ )
+ summary = {k: payload[k] for k in ("state", "frames_total", "window", "cycle_seconds", "n_out", "delay_ms", "resampled_seam_ratio", "specks_dropped", "report")}
+ summary["cycle"] = {k: payload["cycle"][k] for k in ("start", "length", "period_global", "ratio")}
+ summary["strip"] = {k: payload["strip"][k] for k in ("path", "frames", "w", "h", "body_h", "delay_ms")}
+ print(json.dumps(summary, ensure_ascii=False, indent=2))
+ return 0
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(prog="sprite-gen video-loop", description=__doc__)
+ add_arguments(parser)
+ return run(**vars(parser.parse_args(argv)))
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/tests/packaging/test_package_surface.py b/tests/packaging/test_package_surface.py
index 075cdb0..984bff2 100644
--- a/tests/packaging/test_package_surface.py
+++ b/tests/packaging/test_package_surface.py
@@ -38,6 +38,10 @@ PACKAGE_RUN_MODULES = [
"slice_sheet",
"unpack_atlas",
"video",
+ "canvas",
+ "frames",
+ "loop",
+ "batch",
]
CLI_HELPERS = [
diff --git a/tests/video/test_video_pipeline.py b/tests/video/test_video_pipeline.py
new file mode 100644
index 0000000..efab7a2
--- /dev/null
+++ b/tests/video/test_video_pipeline.py
@@ -0,0 +1,270 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Video -> sprite pipeline contracts on synthetic frames: canvas profiles, edge-contact
+keying, global-period cycle detection (incl. the 1.5-cycle trap), seam gate, GIF/WebP
+verification, and the batch runner's stagger/429 handling."""
+
+from __future__ import annotations
+
+import json
+import math
+import shutil
+from pathlib import Path
+
+import pytest
+from PIL import Image
+
+from sprite_gen._deps import np
+from sprite_gen.video import batch as batch_mod
+from sprite_gen.video import canvas as canvas_mod
+from sprite_gen.video import frames as frames_mod
+from sprite_gen.video import loop as loop_mod
+
+GREEN = (0, 255, 0)
+HAS_FFMPEG = shutil.which("ffmpeg") is not None and shutil.which("ffprobe") is not None
+HAS_IMG2WEBP = shutil.which("img2webp") is not None
+
+
+def _still(tmp_path: Path, size=(120, 160), key=GREEN, name="still.png") -> Path:
+ im = Image.new("RGB", size, key)
+ for y in range(40, 150):
+ for x in range(40, 80):
+ im.putpixel((x, y), (200, 40, 40))
+ p = tmp_path / name
+ im.save(p)
+ return p
+
+
+# --- canvas -------------------------------------------------------------------
+
+
+def test_profile_table_routes_states_and_shape_overrides() -> None:
+ assert canvas_mod.profile_for("jump").shape == "tall"
+ assert canvas_mod.profile_for("attack").shape == "wide"
+ assert canvas_mod.profile_for("walk").shape == "square"
+ assert canvas_mod.profile_for(None).shape == "square"
+ assert canvas_mod.profile_for("jump", shape="wide").shape == "wide"
+ with pytest.raises(SystemExit, match="unknown --shape"):
+ canvas_mod.profile_for("jump", shape="round")
+
+
+def test_tall_canvas_keeps_headroom_and_key(tmp_path: Path) -> None:
+ still = Image.open(_still(tmp_path))
+ out, rep = canvas_mod.pad_canvas(still, canvas_mod.profile_for("jump"))
+ w, h = out.size
+ assert rep["shape"] == "tall" and abs(w / h - 3 / 4) < 0.02
+ assert rep["offset"][1] == h - 160 # still sits at the bottom
+ assert h - 160 >= round(h * 0.34) - 1 # headroom above the still
+ assert out.getpixel((0, 0)) == GREEN and rep["key_rgb"] == list(GREEN)
+ assert out.getpixel((rep["offset"][0] + 50, h - 100)) == (200, 40, 40) # still content intact
+
+
+def test_wide_canvas_puts_room_in_front_of_the_facing(tmp_path: Path) -> None:
+ still = Image.open(_still(tmp_path))
+ right, rep_r = canvas_mod.pad_canvas(still, canvas_mod.profile_for("attack"), facing="right")
+ left, rep_l = canvas_mod.pad_canvas(still, canvas_mod.profile_for("attack"), facing="left")
+ assert rep_r["offset"][0] == 0 and rep_l["offset"][0] == left.width - 120
+ assert abs(right.width / right.height - 16 / 9) < 0.02
+
+
+def test_square_canvas_never_shrinks_and_refuses_non_flat_corners(tmp_path: Path) -> None:
+ still = Image.open(_still(tmp_path))
+ out, rep = canvas_mod.pad_canvas(still, canvas_mod.profile_for("walk"))
+ assert out.size == (160, 160) and rep["offset"] == [20, 0]
+ bad = Image.new("RGB", (60, 60), GREEN)
+ bad.putpixel((0, 0), (255, 255, 255))
+ with pytest.raises(SystemExit, match="not one flat colour"):
+ canvas_mod.pad_canvas(bad, canvas_mod.profile_for("walk"))
+
+
+def test_run_canvas_writes_png_and_report(tmp_path: Path) -> None:
+ still = _still(tmp_path)
+ rep = canvas_mod.run_canvas(still, tmp_path / "o" / "canvas.png", state="jump", shape=None, facing="right", headroom=None, lead=None, report_path=tmp_path / "o" / "r.json")
+ assert (tmp_path / "o" / "canvas.png").is_file() and rep["kind"] == "sprite-gen-video-canvas-report"
+ assert json.loads((tmp_path / "o" / "r.json").read_text())["shape"] == "tall"
+
+
+# --- frames ----------------------------------------------------------------------
+
+
+def _raw_frames(tmp_path: Path, n: int, *, touch_top: bool = False) -> list[Path]:
+ d = tmp_path / "raw"
+ d.mkdir()
+ files = []
+ for k in range(n):
+ im = Image.new("RGB", (80, 100), GREEN)
+ top = 0 if touch_top else 20
+ for y in range(top, 90):
+ for x in range(25 + (k % 3), 55 + (k % 3)):
+ im.putpixel((x, y), (180, 60, 30))
+ p = d / f"frame-{k:04d}.png"
+ im.save(p)
+ files.append(p)
+ return files
+
+
+def test_key_frames_keys_green_and_reports_alpha(tmp_path: Path) -> None:
+ files = _raw_frames(tmp_path, 4)
+ rep = frames_mod.key_frames(files, tmp_path / "keyed", key="green")
+ assert rep["frames"] == 4 and rep["edge_contacts"] == []
+ assert rep["alpha_zero_pct_min"] > 50
+ keyed = Image.open(tmp_path / "keyed" / "frame-0000.png").convert("RGBA")
+ assert keyed.getpixel((0, 0))[3] == 0 and keyed.getpixel((40, 50))[3] > 0
+
+
+def test_key_frames_fails_loud_on_edge_contact_unless_allowed(tmp_path: Path) -> None:
+ files = _raw_frames(tmp_path, 3, touch_top=True)
+ with pytest.raises(SystemExit, match="touches the frame edge"):
+ frames_mod.key_frames(files, tmp_path / "keyed", key="green")
+ rep = frames_mod.key_frames(files, tmp_path / "keyed2", key="green", check_edges=False)
+ assert rep["frames"] == 3
+
+
[email protected](not HAS_FFMPEG, reason="ffmpeg/ffprobe not installed")
+def test_run_frames_extracts_a_real_clip(tmp_path: Path) -> None:
+ import subprocess
+
+ files = _raw_frames(tmp_path, 12)
+ clip = tmp_path / "clip.mp4"
+ subprocess.run(["ffmpeg", "-v", "error", "-y", "-framerate", "24", "-i", str(tmp_path / "raw" / "frame-%04d.png"), "-pix_fmt", "yuv420p", "-c:v", "libx264", "-crf", "12", str(clip)], check=True)
+ rep = frames_mod.run_frames(clip, tmp_path / "fr", key="green", allow_edge_contact=False, report_path=None)
+ assert rep["frames"] == 12 and abs(rep["fps"] - 24) < 0.01
+ assert Path(rep["report"]).is_file() and len(list(Path(rep["keyed_dir"]).glob("*.png"))) == 12
+
+
+# --- loop ----------------------------------------------------------------------------
+
+
+def _gait_frames(tmp_path: Path, *, period: int, n: int, size=(64, 64)) -> list[Path]:
+ """A 'walker': a body block plus one leg that swings with `period`; the swing is
+ mirror-symmetric every half period (the classic 1.5-cycle trap) but a side marker
+ breaks the symmetry slightly, exactly like a near/far leg in a side view."""
+ d = tmp_path / "keyed"
+ d.mkdir()
+ files = []
+ for t in range(n):
+ im = Image.new("RGBA", size, (0, 0, 0, 0))
+ for y in range(10, 40):
+ for x in range(26, 38):
+ im.putpixel((x, y), (200, 60, 60, 255))
+ phase = 2 * math.pi * t / period
+ leg_x = 32 + round(10 * math.sin(phase))
+ for y in range(40, 58):
+ for x in range(leg_x - 3, leg_x + 3):
+ if 0 <= x < size[0]:
+ im.putpixel((x, y), (60, 60, 200, 255))
+ # near-leg marker only on the first half of the period
+ if math.sin(phase) >= 0:
+ im.putpixel((leg_x, 50), (255, 255, 0, 255))
+ p = d / f"frame-{t:04d}.png"
+ im.save(p)
+ files.append(p)
+ return files
+
+
+def test_detect_cycle_finds_the_full_period_not_the_half_or_1_5x(tmp_path: Path) -> None:
+ files = _gait_frames(tmp_path, period=12, n=100)
+ D = loop_mod.distance_matrix(files)
+ cycle = loop_mod.detect_cycle(D, min_len=4, max_len=40)
+ assert cycle["period_global"] == 12
+ assert cycle["length"] in (11, 12, 13)
+ assert cycle["ratio"] < 1.0
+
+
+def test_detect_cycle_refuses_empty_window() -> None:
+ D = np.zeros((10, 10), dtype=np.float32)
+ with pytest.raises(SystemExit, match="window"):
+ loop_mod.detect_cycle(D, min_len=9, max_len=8)
+
+
+def test_profiles_scale_windows_with_clip_length() -> None:
+ p = loop_mod.profile_for("idle")
+ assert p.min_frac > loop_mod.profile_for("walk").min_frac
+ assert loop_mod.profile_for("unknown-state") is loop_mod.STATE_PROFILES["default"]
+
+
[email protected](not HAS_IMG2WEBP, reason="img2webp not installed")
+def test_run_loop_emits_strip_gif_webp_and_verifies(tmp_path: Path) -> None:
+ files = _gait_frames(tmp_path, period=12, n=96)
+ rep = loop_mod.run_loop(tmp_path / "keyed", tmp_path / "out", fps=24.0, state="walk", min_len=None, max_len=None, n_out=8, seam_max=2.0, name="walker", report_path=None)
+ assert rep["cycle"]["period_global"] == 12
+ assert rep["resampled_seam_ratio"] <= 2.0
+ strip = Image.open(rep["strip"]["path"])
+ assert strip.size == (rep["strip"]["w"] * rep["strip"]["frames"], rep["strip"]["h"])
+ assert rep["strip"]["body_h"] <= rep["strip"]["h"]
+ assert rep["gif"]["n_frames"] == 8 and rep["gif"]["loop"] == 0 and rep["gif"]["corners_transparent"]
+ assert rep["webp"]["n_frames"] == 8 and rep["webp"]["stale_rgb_under_alpha0"] == 0
+ assert json.loads((tmp_path / "out" / "walker.strip.json").read_text())["frames"] == rep["strip"]["frames"]
+ assert len(list((tmp_path / "out" / "cycle").glob("frame-*.png"))) == rep["cycle"]["length"]
+
+
+def test_run_loop_seam_gate_fails_loud_on_noise(tmp_path: Path, monkeypatch) -> None:
+ d = tmp_path / "keyed"
+ d.mkdir()
+ rng = np.random.default_rng(7)
+ for t in range(40):
+ arr = np.zeros((48, 48, 4), dtype=np.uint8)
+ mask = rng.random((48, 48)) > 0.6
+ arr[..., 0][mask] = 200
+ arr[..., 3][mask] = 255
+ Image.fromarray(arr, "RGBA").save(d / f"frame-{t:04d}.png")
+ with pytest.raises(SystemExit, match="seam ratio|no periodic cycle"):
+ loop_mod.run_loop(d, tmp_path / "out", fps=24.0, state="walk", min_len=None, max_len=None, n_out=6, seam_max=2.0, name="noise", report_path=None)
+ assert not (tmp_path / "out" / "noise.gif").exists()
+
+
+def test_drop_specks_erases_detached_slivers_only() -> None:
+ im = Image.new("RGBA", (40, 40), (0, 0, 0, 0))
+ for y in range(5, 35):
+ for x in range(10, 30):
+ im.putpixel((x, y), (200, 0, 0, 255))
+ im.putpixel((0, 0), (200, 0, 0, 255)) # a 1-px speck
+ im.putpixel((1, 0), (200, 0, 0, 255))
+ out, dropped = loop_mod._drop_specks(im, 0.01)
+ assert dropped == 1 and out.getpixel((0, 0))[3] == 0 and out.getpixel((20, 20))[3] == 255
+
+
+# --- batch -------------------------------------------------------------------------------
+
+
+def test_prompt_uses_state_and_view_and_optional_character() -> None:
+ p = batch_mod.build_prompt("side", "walk", "The chibi paladin girl")
+ assert p.startswith("2D game sprite animation. The chibi paladin girl walks")
+ assert "seen from the exact side" in p
+ assert "The character" in batch_mod.build_prompt("back", "jump", None)
+
+
+def test_run_set_staggers_retries_429_and_tables_failures(tmp_path: Path, monkeypatch) -> None:
+ base = _still(tmp_path)
+ calls: list[tuple[str, float]] = []
+ import time as _time
+
+ def fake_video(image, prompt, out, report, *, duration, resolution, log):
+ calls.append((out.parent.name, _time.monotonic()))
+ item = out.parent.name
+ if item == "side-run" and sum(1 for c in calls if c[0] == item) == 1:
+ log.write_text("video: generation request refused (HTTP 429): resource-exhausted")
+ return 1
+ if item == "side-jump":
+ log.write_text("video: generation failed for good")
+ return 1
+ out.write_bytes(b"\x00\x00\x00\x18ftypisom")
+ report.write_text("{}")
+ log.write_text("ok")
+ return 0
+
+ monkeypatch.setattr(batch_mod.time, "sleep", lambda s: None) # no real backoff waits
+ monkeypatch.setattr(frames_mod, "run_frames", lambda clip, out_dir, **kw: {"fps": 24.0, "frames": 10, "alpha_zero_pct_min": 60.0, "alpha_zero_pct_max": 70.0, "keyed_dir": str(out_dir / "keyed")})
+ monkeypatch.setattr(batch_mod.frames_mod, "run_frames", frames_mod.run_frames)
+ monkeypatch.setattr(batch_mod.loop_mod, "run_loop", lambda frames_dir, out_dir, **kw: {"cycle": {"length": 12, "period_global": 12, "ratio": 0.3}, "resampled_seam_ratio": 0.5, "n_out": 12, "gif": {"file": "x.gif"}, "webp": {"file": "x.webp"}, "strip": {"path": "x.png"}})
+
+ payload = batch_mod.run_set(bases={"side": base}, states=["walk", "run", "jump"], root=tmp_path / "set", character=None, duration=6, resolution="720p", key="green", concurrency=3, force=False, gap=0.0, video_runner=fake_video)
+
+ by = {r["item"]: r for r in payload["items"]}
+ assert by["side-walk"]["ok"] and by["side-walk"]["clip"]["attempts"] == [0]
+ assert by["side-run"]["ok"] and by["side-run"]["clip"]["attempts"] == [1, 0] # one 429 retry
+ assert not by["side-jump"]["ok"] and "clip generation failed" in by["side-jump"]["error"]
+ assert payload["failed"] == ["side-jump"]
+ table = (tmp_path / "set" / "table.md").read_text()
+ assert "| side | jump | - | - | - | - | FAIL" in table and "| side | walk | 12 | 12 | 0.50 | 12 | OK |" in table
+ assert (tmp_path / "set" / "set.report.json").is_file()
+ assert (tmp_path / "set" / "side-walk" / "canvas.png").is_file() # canvas ran for real