Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
feat(video-loop): the jolt index and the head's moves, reported on every walk; a gate only when asked
Soohong Kime5ea42b8c8bc
diff --git a/docs/loop-repair.md b/docs/loop-repair.mdindex 5ea7686..ecce0b9 100644--- a/docs/loop-repair.md+++ b/docs/loop-repair.md@@ -131,6 +131,66 @@ engine replaces the same frames as the experiment on every take tried — Lite s[3, 15, 19], Pro back-diagonal NE2 [9], Lite front S2 [0, 11, 14], and nothing in the smoothLite side E2. About 0.7 s a replaced frame on the Mac's GPU.+## 3. The jolt index — reported on every walk, a gate only when asked++A loop whose steps change unevenly jolts even when no single step is a jump. Every walk or run+loop's report carries `jolt`, measured on the loop as it will play (after section 2's repair):++| Field | What |+|---|---|+| `index` | the **jolt index**: for every step, how far its coverage change strays from the mean of its two neighbours' changes; the mean of that over the median step. 0 for even steps; alternating big and small steps read high |+| `hair_index` | the same inside the hair box |+| `step_max_over_median` | the largest step over the median (what section 2 repairs) |+| `head.x`, `head.y` | the head's place frame by frame in % of the body's height over the loop: x (sideways) the coverage centroid of the top fifth, y (up and down) the body's top line. Each: `step_max_pct` (largest move in one frame), `step_median_pct`, `max_over_median`, `range_pct`, `worst_into_frame` |+| `reference`, `warnings` | the reference bounds (jolt index **0.42**, head sideways step **0.75** % of the body height) and what exceeds them, in words |+| `gate`, `gated`, `over`, `passed` | the bounds the caller passed, whether they were enforced, what exceeded them |+| `measured` | `after the jump repair` or `as filmed` |++**By default nothing fails on the jolt.** A loop beyond a reference bound is kept, its report+lists the reason under `warnings`, and `video-loop` prints a+`video-loop: warning: … (reference bound; the loop is kept …)` line on stderr. The aim is fewer+refilms — section 2 repairs what it can without a new clip — and the reference bounds rest on+too few judged takes to refilm on (below).++**A gate only when asked.** `--jolt-max` and/or `--head-step-max` turn the bound passed into a+gate on the repaired loop: beyond it the loop fails with+`video-loop: loop jolts — … regenerate the clip`, which a caller that buys a new clip can match.+Only the bounds passed are checked. With `--repair off` the loop is cut and reported as filmed,+never gated. A frame the head cannot be tracked in is named rather than read as smooth.++### Where the reference bounds come from — 46 takes, 2026-10-03++Eight-direction SD set, five generated views, Pro and Lite, four filming rounds (46 loops cut by+`video-loop --anchor motion-auto`), measured on the strip cells before and after section 2's+repair. Eleven of them were judged by the maintainer: nine kept for the final sets, two+refused for how they move — a Lite side walk whose every step alternated (its sibling take was+"completely natural"), and a Lite back-diagonal walk whose head "jumps".++| After the repair | min | p25 | median | p75 | p90 | max | the 9 kept reach | the 2 refused |+|---|---|---|---|---|---|---|---|---|+| jolt index | 0.055 | 0.178 | 0.212 | 0.243 | 0.260 | 0.414 | 0.414 (Pro back diagonal) | 0.237 (side), 0.196 (back diagonal) |+| head sideways step, % | 0.18 | 0.43 | 0.68 | 0.83 | 1.07 | 1.71 | 0.68 | 0.27 (side), **0.82** (back diagonal) |+| head top-line step, % | 0.49 | 0.98 | 1.24 | 1.71 | 2.47 | 3.17 | 2.42 (Lite back view) | 0.75, 1.71 |++Before the repair the jolt index's median was 0.258 (max 0.456) and the head sideways step's+0.70 % (max 2.88 %): the repair lowers both.++Read:++- **The head's sideways step is the only measure that separates the judged takes**: every kept+ take stays at or under 0.68 %, the back-diagonal take refused for its head moves 0.82 %. The+ reference 0.75 sits between them.+- **The jolt index does not separate them.** The refused side walk (0.237) sits inside the kept+ range, and a kept Pro back-diagonal walk reads 0.414. Any bound under 0.414 would refuse a kept+ take, so the reference (0.42) sits at the kept takes' edge and flags nothing in this set. An+ alternating jolt like that side walk's is a known miss of this index.+- **The top line is reported, not bounded**: a walk seen from behind bobs more (the kept Lite+ back view reaches 2.42 %), so one bound would flag kept back views or pass the refused one.+- **Why no gate by default**: at the reference bounds, 18 of the 46 takes exceed one — 16 of 30+ Lite, 2 of 16 Pro, none of the nine kept. Lite's head sways about three times as far as Pro's,+ so as a default gate it would refilm about every other Lite walk: the opposite of the aim, on+ the strength of two refusals. The bounds stay a reference until more takes are judged.+## Related- [video-pipeline.md](video-pipeline.md) — the pipeline these repairs run insidediff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex cc6794f..14003b0 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -401,7 +401,9 @@ A walk or run loop then has its jump frames repaired before the strip is built:breaks a step 1.4× the loop's median (whole body, or the hair behind it) is replaced by RIFE'sframe between its two neighbours, at most three and never two side by side (`--repair auto`,the default; `--repair off` cuts as filmed). The report's `jump_repair` names the frames. See-[loop repair](loop-repair.md).+[loop repair](loop-repair.md). The repaired loop's jolt index and the head's frame-to-frame+moves are reported (`jolt`); beyond the reference bounds that is a warning line, and a gate+(`video-loop: loop jolts — …`) only when `--jolt-max` / `--head-step-max` are passed.### One-shot actions — `--cycle auto|periodic|one-shot`diff --git a/sprite_gen/video/loop.py b/sprite_gen/video/loop.pyindex ec5dd70..88aebcb 100644--- a/sprite_gen/video/loop.py+++ b/sprite_gen/video/loop.py@@ -29,6 +29,7 @@ import mathimport osimport shutilimport subprocess+import sysfrom collections import dequefrom dataclasses import dataclassfrom pathlib import Path@@ -821,6 +822,8 @@ def run_loop(repair: str = "auto",facing: str = "right",interpolate: rife_mod.Interpolate | None = None,+ jolt_max: float | None = None,+ head_step_max: float | None = None,) -> dict[str, Any]:if repair not in REPAIR_MODES:raise SystemExit(f"video-loop: unknown --repair {repair!r}; expected one of {', '.join(REPAIR_MODES)}")@@ -985,6 +988,29 @@ def run_loop(report_base["seam_measurement"] = "rendered-cells"else:report_base["jump_repair"] = {"applied": False, "why": "--repair off" if repair == "off" else f"not a gait state ({state})"}+ if prof.gait:+ # The jolt is measured on the loop as it will play — after the repair — and reported on+ # every walk and run (docs/loop-repair.md section 3). Beyond the reference bounds it is a+ # warning line; it fails the loop only against a bound the caller passed (--jolt-max /+ # --head-step-max), and only when the repair ran: --repair off cuts and reports as filmed.+ jolt = repair_mod.measure_jolt(frames, facing=facing)+ reference = {"jolt_max": repair_mod.JOLT_REFERENCE, "head_step_max": repair_mod.HEAD_STEP_REFERENCE}+ warnings = repair_mod.jolt_verdict(jolt, **reference)+ gated = repair == "auto" and (jolt_max is not None or head_step_max is not None)+ over = repair_mod.jolt_verdict(jolt, jolt_max=jolt_max, head_step_max=head_step_max) if gated else []+ report_base["jolt"] = {**jolt, "reference": reference, "warnings": warnings,+ "gate": {"jolt_max": jolt_max, "head_step_max": head_step_max} if gated else None,+ "gated": gated, "over": over, "passed": not over,+ "measured": "after the jump repair" if report_base["jump_repair"].get("replaced") else "as filmed"}+ if over:+ error = (f"video-loop: loop jolts — {'; '.join(over)} ({cycle.get('kind', 'periodic')} length {L} frames from {i}"+ f"{', after repairing frames ' + str(report_base['jump_repair']['replaced']) if report_base['jump_repair'].get('replaced') else ''}); "+ "the motion does not play smoothly — regenerate the clip")+ shutil.rmtree(out_dir / FALLBACK_FRAMES_DIR, ignore_errors=True)+ write_loop_report(target, {**report_base, "status": "failed", "error": error, "cycle": cycle})+ raise SystemExit(error)+ for line in warnings:+ print(f"video-loop: warning: {line} (reference bound; the loop is kept — docs/loop-repair.md)", file=sys.stderr)cycle_dir.mkdir(parents=True, exist_ok=True)for old in cycle_dir.glob("frame-*.png"):old.unlink()@@ -1111,6 +1137,8 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:parser.add_argument("--anchor", choices=ANCHOR_MODES, default=None, help="motion-auto: automatic stable regions, local repeat selection and XY ramp (walk/run); motion: fixed cut with explicit regions; body (default for walk/run): measure the last-to-first head-and-torso offset once and spread it as a ramp over the cycle; feet: remove in-canvas drift (a straight-line trend) so every cell stands on the mean foot line; none: leave placement as filmed (default for other states)")parser.add_argument("--anchor-region", type=motion_anchor.parse_region, action="append", help="motion anchor only: repeat twice with head then torso x0,y0,x1,y1 in the first selected frame; requires --cycle fixed and at least 6 frames")parser.add_argument("--repair", choices=REPAIR_MODES, default="auto", help="auto (default): in a walk or run loop, a frame that follows a jump (a step 1.4x the median, whole body or the hair behind it) is replaced by RIFE's frame between its neighbours, at most 3 and never two side by side, recorded in the report (docs/loop-repair.md); off: cut as filmed")+ parser.add_argument("--jolt-max", type=float, default=None, help=f"walk/run: fail the repaired loop when its jolt index (how far each step strays from its neighbours' mean, over the median step) exceeds this. Default: no gate — the index is reported and a value over {repair_mod.JOLT_REFERENCE} is a warning line")+ parser.add_argument("--head-step-max", type=float, default=None, help=f"walk/run: fail the repaired loop when the head's largest sideways move in one frame exceeds this %% of the body height. Default: no gate — reported, and over {repair_mod.HEAD_STEP_REFERENCE} is a warning line")parser.add_argument("--facing", choices=FACINGS, default="right", help="which way the body faces in the frames (default right): the hair the jump search watches is behind it")parser.add_argument("--name", default="loop", help="basename for strip/gif/webp outputs")parser.add_argument("--report", type=Path)@@ -1128,6 +1156,7 @@ def run(**kwargs: object) -> int:anchor=kwargs.get("anchor"), # None resolves by state inside run_loopanchor_regions=kwargs.get("anchor_region"),repair=str(kwargs.get("repair") or "auto"), facing=str(kwargs.get("facing") or "right"),+ jolt_max=kwargs.get("jolt_max"), head_step_max=kwargs.get("head_step_max"), # type: ignore[arg-type])summary = {k: payload[k] for k in ("state", "frames_total", "window", "cycle_seconds", "n_out", "delay_ms", "resampled_seam_ratio", "specks_dropped", "report")}summary["cycle"] = {k: payload["cycle"].get(k) for k in ("kind", "start", "length", "period_global", "ratio", "review_recommended", "half_period_guard")}@@ -1135,6 +1164,9 @@ def run(**kwargs: object) -> int:summary["periodic_attempt"] = payload["periodic_attempt"]["why_rejected"]if payload.get("motion_anchor", {}).get("applied") is False:summary["motion_anchor"] = payload["motion_anchor"]+ if payload.get("jolt"):+ summary["jolt"] = {"index": payload["jolt"]["index"], "warnings": payload["jolt"]["warnings"],+ "head_x_step_max_pct": payload["jolt"]["head"].get("x", {}).get("step_max_pct")}if payload.get("jump_repair", {}).get("replaced"):summary["jump_repair"] = {k: payload["jump_repair"][k] for k in ("replaced", "score_max_before", "score_max_after")}summary["strip"] = {k: payload["strip"][k] for k in ("path", "frames", "w", "h", "body_h", "delay_ms")}diff --git a/sprite_gen/video/repair.py b/sprite_gen/video/repair.pyindex 3bf7148..91b8f4e 100644--- a/sprite_gen/video/repair.py+++ b/sprite_gen/video/repair.py@@ -123,3 +123,93 @@ def repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *,record["stopped"] = stoppedrecord["score_max_after"] = round(float(scores["score"].max()), 4)return out, record+++# --- the jolt index (docs/loop-repair.md section 3) ------------------------------------------++# Reference bounds, at the edge of what was kept on the 2026-10-03 takes (section 3): nothing kept+# exceeds them, and the back-diagonal take refused because "it jumps" does. A loop beyond them is+# warned about; it fails only when a caller passes a bound (--jolt-max / --head-step-max), because+# eleven judged takes are too few to refilm on, and at these values 16 of 30 Lite takes would be.+JOLT_REFERENCE = 0.42 # alpha jolt index; the kept takes reached 0.414 after repair+HEAD_STEP_REFERENCE = 0.75 # head's sideways move in one frame, % of body height; kept <= 0.68, refused 0.82+HEAD_BAND = 0.20 # the head is the top fifth of the body's height over the loop+HEAD_MEDIAN_FLOOR = 0.05 # % of body height: a median step below this is the tracker's rounding+++def jolt_index(step_values: np.ndarray) -> float:+ """How far each step strays from the mean of its two neighbours, on average, over the median+ step. 0 for a loop whose steps change evenly; alternating big and small steps read high."""+ n = len(step_values)+ med = float(np.median(step_values))+ if n < 3 or med <= 0:+ return 0.0+ return float(np.mean([abs(step_values[k] - (step_values[k - 1] + step_values[(k + 1) % n]) / 2) for k in range(n)]) / med)+++def _track_stats(values: np.ndarray) -> dict[str, float]:+ n = len(values)+ d = np.abs(np.array([values[(k + 1) % n] - values[k] for k in range(n)]))+ med = float(np.median(d))+ return {"step_max_pct": round(float(d.max()), 3), "step_median_pct": round(med, 3),+ "max_over_median": round(float(d.max()) / max(med, HEAD_MEDIAN_FLOOR), 3),+ "range_pct": round(float(np.ptp(values)), 3), "worst_into_frame": int((np.argmax(d) + 1) % n)}+++def head_track(alphas: list[np.ndarray]) -> dict[str, Any]:+ """The head's place frame by frame, in % of the body's height over the loop: x is the coverage+ centroid of the top fifth (HEAD_BAND), y the body's top line. A take whose head pops sideways+ reads here while its coverage change stays ordinary (2026-10-03, Lite back diagonal)."""+ rows = np.where(np.max(np.stack(alphas), axis=0).max(axis=1) > 0.1)[0]+ if rows.size == 0:+ raise ValueError("every loop frame is fully transparent")+ top, bottom = int(rows.min()), int(rows.max())+ height = max(1, bottom - top)+ band = slice(top, top + max(1, int(height * HEAD_BAND)))+ xs, ys = [], []+ for a in alphas:+ b = a[band]+ mass = float(b.sum())+ xs.append(float((b.sum(axis=0) * np.arange(b.shape[1])).sum() / mass) if mass > 0 else float("nan"))+ filled = np.where(a.sum(axis=1) > 0.5)[0]+ ys.append(float(filled.min()) if filled.size else float("nan"))+ x = np.array(xs) / height * 100+ y = np.array(ys) / height * 100+ if not (np.isfinite(x).all() and np.isfinite(y).all()):+ raise ValueError("a loop frame has no head to track")+ return {"body_height_px": height, "x": _track_stats(x), "y": _track_stats(y)}+++def measure_jolt(frames: list[Image.Image], *, facing: str = "right") -> dict[str, Any]:+ """The jolt index of a loop as it plays, and the head's frame-to-frame moves."""+ al = coverage(frames, union_box(frames))+ whole, hair = steps(al), steps(al, hair_box(facing))+ med = float(np.median(whole))+ try:+ head: dict[str, Any] = head_track(al)+ except ValueError as exc:+ head = {"skipped": str(exc)} # recorded, and the head gate says it could not read it+ return {+ "index": round(jolt_index(whole), 4),+ "hair_index": round(jolt_index(hair), 4),+ "step_max_over_median": round(float(whole.max()) / med, 4) if med > 0 else None,+ "head": head,+ }+++def jolt_verdict(measured: dict[str, Any], *, jolt_max: float | None = JOLT_REFERENCE,+ head_step_max: float | None = HEAD_STEP_REFERENCE) -> list[str]:+ """What exceeds the bounds, in words; empty when the loop stays inside. A bound of None is not checked."""+ over = []+ if jolt_max is not None and measured["index"] > jolt_max:+ over.append(f"jolt index {measured['index']:.3f} exceeds {jolt_max}")+ if head_step_max is None:+ return over+ if "skipped" in measured["head"]:+ over.append(f"the head could not be tracked ({measured['head']['skipped']})")+ return over+ head = measured["head"]["x"]+ if head["step_max_pct"] > head_step_max:+ over.append(f"the head moves sideways {head['step_max_pct']:.2f} % of the body height in one frame "+ f"(into frame {head['worst_into_frame']}), over {head_step_max} %")+ return overdiff --git a/tests/video/test_loop_repair.py b/tests/video/test_loop_repair.pyindex 45d64c1..daaecdc 100644--- a/tests/video/test_loop_repair.py+++ b/tests/video/test_loop_repair.py@@ -158,3 +158,86 @@ def test_repair_off_cuts_as_filmed_and_says_so(tmp_path):rep = _run(tmp_path, _keyed(tmp_path, filmed), repair="off")assert rep["jump_repair"] == {"applied": False, "why": "--repair off"}assert rep["seam_measurement"] == "source-frames"+++# --- the jolt index and its gate (docs/loop-repair.md section 3) ---------------------------------+++def test_jolt_index_is_zero_for_even_steps_and_high_for_alternating_ones():+ assert repair.jolt_index(np.full(12, 0.3)) == 0.0+ alternating = np.array([0.2, 0.4] * 6)+ assert repair.jolt_index(alternating) == pytest.approx(0.2 / 0.3)+++def _head_pop(frames: list[Image.Image], k: int, dx: int) -> list[Image.Image]:+ """Frame k with the head (rows 10..30) moved `dx` px sideways."""+ out = list(frames)+ a = np.asarray(frames[k]).copy()+ head = a[10:30].copy()+ a[10:30] = 0+ a[10:30] = np.roll(head, dx, axis=1)+ out[k] = Image.fromarray(a, "RGBA")+ return out+++def test_a_head_that_pops_sideways_is_named_by_the_gate():+ filmed, _ = _loop()+ smooth = repair.measure_jolt(filmed)+ assert repair.jolt_verdict(smooth) == []+ assert smooth["index"] < repair.JOLT_REFERENCE and smooth["head"]["x"]["step_max_pct"] < repair.HEAD_STEP_REFERENCE+ popped = repair.measure_jolt(_head_pop(filmed, 7, 4))+ over = repair.jolt_verdict(popped)+ assert len(over) == 1 and "head moves sideways" in over[0] and "(into frame 7)" in over[0]+ assert popped["head"]["x"]["worst_into_frame"] == 7+++def test_a_loop_with_no_head_to_track_says_so_instead_of_passing():+ filmed, _ = _loop()+ blank = list(filmed)+ blank[3] = Image.new("RGBA", filmed[3].size, (0, 0, 0, 0))+ measured = repair.measure_jolt(blank)+ assert "skipped" in measured["head"]+ assert any("could not be tracked" in o for o in repair.jolt_verdict(measured))+++def test_video_loop_reports_the_jolt_of_the_loop_as_it_plays_after_the_repair(tmp_path):+ filmed, truth = _loop({10: 12})+ rep = _run(tmp_path, _keyed(tmp_path, filmed), interpolate=lambda a, b, t: truth[10])+ jolt = rep["jolt"]+ assert jolt["measured"] == "after the jump repair" and jolt["warnings"] == [] and jolt["passed"]+ assert jolt["gated"] is False and jolt["gate"] is None+ assert jolt["reference"] == {"jolt_max": repair.JOLT_REFERENCE, "head_step_max": repair.HEAD_STEP_REFERENCE}+ assert set(jolt["head"]["x"]) >= {"step_max_pct", "step_median_pct", "max_over_median", "range_pct"}+ assert set(jolt["head"]["y"]) >= {"step_max_pct", "step_median_pct", "max_over_median", "range_pct"}+++def test_by_default_a_jolting_loop_is_kept_with_a_warning_line(tmp_path, capsys):+ filmed, _ = _loop()+ popped = _head_pop(filmed, 7, 4)+ rep = _run(tmp_path, _keyed(tmp_path, popped), interpolate=lambda a, b, t: popped[7])+ assert rep["status"] == "passed" and rep["jolt"]["passed"] and rep["jolt"]["gated"] is False+ assert len(rep["jolt"]["warnings"]) == 1 and "head moves sideways" in rep["jolt"]["warnings"][0]+ assert "video-loop: warning: the head moves sideways" in capsys.readouterr().err+++def test_a_bound_the_caller_passes_refuses_a_jolting_loop(tmp_path):+ filmed, _ = _loop()+ popped = _head_pop(filmed, 7, 4)+ stays = lambda a, b, t: popped[7] # an in-between that cannot take the pop out+ with pytest.raises(SystemExit, match=r"^video-loop: loop jolts — the head moves sideways .* regenerate the clip$"):+ _run(tmp_path, _keyed(tmp_path, popped), interpolate=stays, head_step_max=repair.HEAD_STEP_REFERENCE)+ report = json.loads((tmp_path / "w.json").read_text())+ assert report["status"] == "failed" and report["jolt"]["passed"] is False and report["jolt"]["over"]+ assert report["jolt"]["gate"] == {"jolt_max": None, "head_step_max": repair.HEAD_STEP_REFERENCE}+ assert report["jump_repair"]["replaced"] == [7]+ # only the bound passed is checked: a jolt bound alone lets the head through+ again = tmp_path / "again"+ again.mkdir()+ rep = _run(again, _keyed(again, popped), interpolate=stays, jolt_max=10.0)+ assert rep["status"] == "passed" and rep["jolt"]["gated"] and rep["jolt"]["warnings"]+++def test_repair_off_measures_the_jolt_and_does_not_gate_it(tmp_path):+ filmed, _ = _loop()+ rep = _run(tmp_path, _keyed(tmp_path, _head_pop(filmed, 7, 4)), repair="off", head_step_max=0.1)+ assert rep["jolt"]["gated"] is False and rep["jolt"]["warnings"] and rep["status"] == "passed"