Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

feat(video-loop): the jolt index and the head's moves, reported on every walk; a gate only when asked

Soohong Kime5ea42b8c8bc
diff --git a/docs/loop-repair.md b/docs/loop-repair.md
index 5ea7686..ecce0b9 100644
--- a/docs/loop-repair.md
+++ b/docs/loop-repair.md
@@ -131,6 +131,66 @@ engine replaces the same frames as the experiment on every take tried — Lite s
[3, 15, 19], Pro back-diagonal NE2 [9], Lite front S2 [0, 11, 14], and nothing in the smooth
Lite side E2. About 0.7 s a replaced frame on the Mac's GPU.
+## 3. The jolt index — reported on every walk, a gate only when asked
+
+A loop whose steps change unevenly jolts even when no single step is a jump. Every walk or run
+loop's report carries `jolt`, measured on the loop as it will play (after section 2's repair):
+
+| Field | What |
+|---|---|
+| `index` | the **jolt index**: for every step, how far its coverage change strays from the mean of its two neighbours' changes; the mean of that over the median step. 0 for even steps; alternating big and small steps read high |
+| `hair_index` | the same inside the hair box |
+| `step_max_over_median` | the largest step over the median (what section 2 repairs) |
+| `head.x`, `head.y` | the head's place frame by frame in % of the body's height over the loop: x (sideways) the coverage centroid of the top fifth, y (up and down) the body's top line. Each: `step_max_pct` (largest move in one frame), `step_median_pct`, `max_over_median`, `range_pct`, `worst_into_frame` |
+| `reference`, `warnings` | the reference bounds (jolt index **0.42**, head sideways step **0.75** % of the body height) and what exceeds them, in words |
+| `gate`, `gated`, `over`, `passed` | the bounds the caller passed, whether they were enforced, what exceeded them |
+| `measured` | `after the jump repair` or `as filmed` |
+
+**By default nothing fails on the jolt.** A loop beyond a reference bound is kept, its report
+lists the reason under `warnings`, and `video-loop` prints a
+`video-loop: warning: … (reference bound; the loop is kept …)` line on stderr. The aim is fewer
+refilms — section 2 repairs what it can without a new clip — and the reference bounds rest on
+too few judged takes to refilm on (below).
+
+**A gate only when asked.** `--jolt-max` and/or `--head-step-max` turn the bound passed into a
+gate on the repaired loop: beyond it the loop fails with
+`video-loop: loop jolts — … regenerate the clip`, which a caller that buys a new clip can match.
+Only the bounds passed are checked. With `--repair off` the loop is cut and reported as filmed,
+never gated. A frame the head cannot be tracked in is named rather than read as smooth.
+
+### Where the reference bounds come from — 46 takes, 2026-10-03
+
+Eight-direction SD set, five generated views, Pro and Lite, four filming rounds (46 loops cut by
+`video-loop --anchor motion-auto`), measured on the strip cells before and after section 2's
+repair. Eleven of them were judged by the maintainer: nine kept for the final sets, two
+refused for how they move — a Lite side walk whose every step alternated (its sibling take was
+"completely natural"), and a Lite back-diagonal walk whose head "jumps".
+
+| After the repair | min | p25 | median | p75 | p90 | max | the 9 kept reach | the 2 refused |
+|---|---|---|---|---|---|---|---|---|
+| jolt index | 0.055 | 0.178 | 0.212 | 0.243 | 0.260 | 0.414 | 0.414 (Pro back diagonal) | 0.237 (side), 0.196 (back diagonal) |
+| head sideways step, % | 0.18 | 0.43 | 0.68 | 0.83 | 1.07 | 1.71 | 0.68 | 0.27 (side), **0.82** (back diagonal) |
+| head top-line step, % | 0.49 | 0.98 | 1.24 | 1.71 | 2.47 | 3.17 | 2.42 (Lite back view) | 0.75, 1.71 |
+
+Before the repair the jolt index's median was 0.258 (max 0.456) and the head sideways step's
+0.70 % (max 2.88 %): the repair lowers both.
+
+Read:
+
+- **The head's sideways step is the only measure that separates the judged takes**: every kept
+ take stays at or under 0.68 %, the back-diagonal take refused for its head moves 0.82 %. The
+ reference 0.75 sits between them.
+- **The jolt index does not separate them.** The refused side walk (0.237) sits inside the kept
+ range, and a kept Pro back-diagonal walk reads 0.414. Any bound under 0.414 would refuse a kept
+ take, so the reference (0.42) sits at the kept takes' edge and flags nothing in this set. An
+ alternating jolt like that side walk's is a known miss of this index.
+- **The top line is reported, not bounded**: a walk seen from behind bobs more (the kept Lite
+ back view reaches 2.42 %), so one bound would flag kept back views or pass the refused one.
+- **Why no gate by default**: at the reference bounds, 18 of the 46 takes exceed one — 16 of 30
+ Lite, 2 of 16 Pro, none of the nine kept. Lite's head sways about three times as far as Pro's,
+ so as a default gate it would refilm about every other Lite walk: the opposite of the aim, on
+ the strength of two refusals. The bounds stay a reference until more takes are judged.
+
## Related
- [video-pipeline.md](video-pipeline.md) — the pipeline these repairs run inside
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index cc6794f..14003b0 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -401,7 +401,9 @@ A walk or run loop then has its jump frames repaired before the strip is built:
breaks a step 1.4× the loop's median (whole body, or the hair behind it) is replaced by RIFE's
frame between its two neighbours, at most three and never two side by side (`--repair auto`,
the default; `--repair off` cuts as filmed). The report's `jump_repair` names the frames. See
-[loop repair](loop-repair.md).
+[loop repair](loop-repair.md). The repaired loop's jolt index and the head's frame-to-frame
+moves are reported (`jolt`); beyond the reference bounds that is a warning line, and a gate
+(`video-loop: loop jolts — …`) only when `--jolt-max` / `--head-step-max` are passed.
### One-shot actions — `--cycle auto|periodic|one-shot`
diff --git a/sprite_gen/video/loop.py b/sprite_gen/video/loop.py
index ec5dd70..88aebcb 100644
--- a/sprite_gen/video/loop.py
+++ b/sprite_gen/video/loop.py
@@ -29,6 +29,7 @@ import math
import os
import shutil
import subprocess
+import sys
from collections import deque
from dataclasses import dataclass
from pathlib import Path
@@ -821,6 +822,8 @@ def run_loop(
repair: str = "auto",
facing: str = "right",
interpolate: rife_mod.Interpolate | None = None,
+ jolt_max: float | None = None,
+ head_step_max: float | None = None,
) -> dict[str, Any]:
if repair not in REPAIR_MODES:
raise SystemExit(f"video-loop: unknown --repair {repair!r}; expected one of {', '.join(REPAIR_MODES)}")
@@ -985,6 +988,29 @@ def run_loop(
report_base["seam_measurement"] = "rendered-cells"
else:
report_base["jump_repair"] = {"applied": False, "why": "--repair off" if repair == "off" else f"not a gait state ({state})"}
+ if prof.gait:
+ # The jolt is measured on the loop as it will play — after the repair — and reported on
+ # every walk and run (docs/loop-repair.md section 3). Beyond the reference bounds it is a
+ # warning line; it fails the loop only against a bound the caller passed (--jolt-max /
+ # --head-step-max), and only when the repair ran: --repair off cuts and reports as filmed.
+ jolt = repair_mod.measure_jolt(frames, facing=facing)
+ reference = {"jolt_max": repair_mod.JOLT_REFERENCE, "head_step_max": repair_mod.HEAD_STEP_REFERENCE}
+ warnings = repair_mod.jolt_verdict(jolt, **reference)
+ gated = repair == "auto" and (jolt_max is not None or head_step_max is not None)
+ over = repair_mod.jolt_verdict(jolt, jolt_max=jolt_max, head_step_max=head_step_max) if gated else []
+ report_base["jolt"] = {**jolt, "reference": reference, "warnings": warnings,
+ "gate": {"jolt_max": jolt_max, "head_step_max": head_step_max} if gated else None,
+ "gated": gated, "over": over, "passed": not over,
+ "measured": "after the jump repair" if report_base["jump_repair"].get("replaced") else "as filmed"}
+ if over:
+ error = (f"video-loop: loop jolts — {'; '.join(over)} ({cycle.get('kind', 'periodic')} length {L} frames from {i}"
+ f"{', after repairing frames ' + str(report_base['jump_repair']['replaced']) if report_base['jump_repair'].get('replaced') else ''}); "
+ "the motion does not play smoothly — regenerate the clip")
+ shutil.rmtree(out_dir / FALLBACK_FRAMES_DIR, ignore_errors=True)
+ write_loop_report(target, {**report_base, "status": "failed", "error": error, "cycle": cycle})
+ raise SystemExit(error)
+ for line in warnings:
+ print(f"video-loop: warning: {line} (reference bound; the loop is kept — docs/loop-repair.md)", file=sys.stderr)
cycle_dir.mkdir(parents=True, exist_ok=True)
for old in cycle_dir.glob("frame-*.png"):
old.unlink()
@@ -1111,6 +1137,8 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--anchor", choices=ANCHOR_MODES, default=None, help="motion-auto: automatic stable regions, local repeat selection and XY ramp (walk/run); motion: fixed cut with explicit regions; body (default for walk/run): measure the last-to-first head-and-torso offset once and spread it as a ramp over the cycle; feet: remove in-canvas drift (a straight-line trend) so every cell stands on the mean foot line; none: leave placement as filmed (default for other states)")
parser.add_argument("--anchor-region", type=motion_anchor.parse_region, action="append", help="motion anchor only: repeat twice with head then torso x0,y0,x1,y1 in the first selected frame; requires --cycle fixed and at least 6 frames")
parser.add_argument("--repair", choices=REPAIR_MODES, default="auto", help="auto (default): in a walk or run loop, a frame that follows a jump (a step 1.4x the median, whole body or the hair behind it) is replaced by RIFE's frame between its neighbours, at most 3 and never two side by side, recorded in the report (docs/loop-repair.md); off: cut as filmed")
+ parser.add_argument("--jolt-max", type=float, default=None, help=f"walk/run: fail the repaired loop when its jolt index (how far each step strays from its neighbours' mean, over the median step) exceeds this. Default: no gate — the index is reported and a value over {repair_mod.JOLT_REFERENCE} is a warning line")
+ parser.add_argument("--head-step-max", type=float, default=None, help=f"walk/run: fail the repaired loop when the head's largest sideways move in one frame exceeds this %% of the body height. Default: no gate — reported, and over {repair_mod.HEAD_STEP_REFERENCE} is a warning line")
parser.add_argument("--facing", choices=FACINGS, default="right", help="which way the body faces in the frames (default right): the hair the jump search watches is behind it")
parser.add_argument("--name", default="loop", help="basename for strip/gif/webp outputs")
parser.add_argument("--report", type=Path)
@@ -1128,6 +1156,7 @@ def run(**kwargs: object) -> int:
anchor=kwargs.get("anchor"), # None resolves by state inside run_loop
anchor_regions=kwargs.get("anchor_region"),
repair=str(kwargs.get("repair") or "auto"), facing=str(kwargs.get("facing") or "right"),
+ jolt_max=kwargs.get("jolt_max"), head_step_max=kwargs.get("head_step_max"), # type: ignore[arg-type]
)
summary = {k: payload[k] for k in ("state", "frames_total", "window", "cycle_seconds", "n_out", "delay_ms", "resampled_seam_ratio", "specks_dropped", "report")}
summary["cycle"] = {k: payload["cycle"].get(k) for k in ("kind", "start", "length", "period_global", "ratio", "review_recommended", "half_period_guard")}
@@ -1135,6 +1164,9 @@ def run(**kwargs: object) -> int:
summary["periodic_attempt"] = payload["periodic_attempt"]["why_rejected"]
if payload.get("motion_anchor", {}).get("applied") is False:
summary["motion_anchor"] = payload["motion_anchor"]
+ if payload.get("jolt"):
+ summary["jolt"] = {"index": payload["jolt"]["index"], "warnings": payload["jolt"]["warnings"],
+ "head_x_step_max_pct": payload["jolt"]["head"].get("x", {}).get("step_max_pct")}
if payload.get("jump_repair", {}).get("replaced"):
summary["jump_repair"] = {k: payload["jump_repair"][k] for k in ("replaced", "score_max_before", "score_max_after")}
summary["strip"] = {k: payload["strip"][k] for k in ("path", "frames", "w", "h", "body_h", "delay_ms")}
diff --git a/sprite_gen/video/repair.py b/sprite_gen/video/repair.py
index 3bf7148..91b8f4e 100644
--- a/sprite_gen/video/repair.py
+++ b/sprite_gen/video/repair.py
@@ -123,3 +123,93 @@ def repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *,
record["stopped"] = stopped
record["score_max_after"] = round(float(scores["score"].max()), 4)
return out, record
+
+
+# --- the jolt index (docs/loop-repair.md section 3) ------------------------------------------
+
+# Reference bounds, at the edge of what was kept on the 2026-10-03 takes (section 3): nothing kept
+# exceeds them, and the back-diagonal take refused because "it jumps" does. A loop beyond them is
+# warned about; it fails only when a caller passes a bound (--jolt-max / --head-step-max), because
+# eleven judged takes are too few to refilm on, and at these values 16 of 30 Lite takes would be.
+JOLT_REFERENCE = 0.42 # alpha jolt index; the kept takes reached 0.414 after repair
+HEAD_STEP_REFERENCE = 0.75 # head's sideways move in one frame, % of body height; kept <= 0.68, refused 0.82
+HEAD_BAND = 0.20 # the head is the top fifth of the body's height over the loop
+HEAD_MEDIAN_FLOOR = 0.05 # % of body height: a median step below this is the tracker's rounding
+
+
+def jolt_index(step_values: np.ndarray) -> float:
+ """How far each step strays from the mean of its two neighbours, on average, over the median
+ step. 0 for a loop whose steps change evenly; alternating big and small steps read high."""
+ n = len(step_values)
+ med = float(np.median(step_values))
+ if n < 3 or med <= 0:
+ return 0.0
+ return float(np.mean([abs(step_values[k] - (step_values[k - 1] + step_values[(k + 1) % n]) / 2) for k in range(n)]) / med)
+
+
+def _track_stats(values: np.ndarray) -> dict[str, float]:
+ n = len(values)
+ d = np.abs(np.array([values[(k + 1) % n] - values[k] for k in range(n)]))
+ med = float(np.median(d))
+ return {"step_max_pct": round(float(d.max()), 3), "step_median_pct": round(med, 3),
+ "max_over_median": round(float(d.max()) / max(med, HEAD_MEDIAN_FLOOR), 3),
+ "range_pct": round(float(np.ptp(values)), 3), "worst_into_frame": int((np.argmax(d) + 1) % n)}
+
+
+def head_track(alphas: list[np.ndarray]) -> dict[str, Any]:
+ """The head's place frame by frame, in % of the body's height over the loop: x is the coverage
+ centroid of the top fifth (HEAD_BAND), y the body's top line. A take whose head pops sideways
+ reads here while its coverage change stays ordinary (2026-10-03, Lite back diagonal)."""
+ rows = np.where(np.max(np.stack(alphas), axis=0).max(axis=1) > 0.1)[0]
+ if rows.size == 0:
+ raise ValueError("every loop frame is fully transparent")
+ top, bottom = int(rows.min()), int(rows.max())
+ height = max(1, bottom - top)
+ band = slice(top, top + max(1, int(height * HEAD_BAND)))
+ xs, ys = [], []
+ for a in alphas:
+ b = a[band]
+ mass = float(b.sum())
+ xs.append(float((b.sum(axis=0) * np.arange(b.shape[1])).sum() / mass) if mass > 0 else float("nan"))
+ filled = np.where(a.sum(axis=1) > 0.5)[0]
+ ys.append(float(filled.min()) if filled.size else float("nan"))
+ x = np.array(xs) / height * 100
+ y = np.array(ys) / height * 100
+ if not (np.isfinite(x).all() and np.isfinite(y).all()):
+ raise ValueError("a loop frame has no head to track")
+ return {"body_height_px": height, "x": _track_stats(x), "y": _track_stats(y)}
+
+
+def measure_jolt(frames: list[Image.Image], *, facing: str = "right") -> dict[str, Any]:
+ """The jolt index of a loop as it plays, and the head's frame-to-frame moves."""
+ al = coverage(frames, union_box(frames))
+ whole, hair = steps(al), steps(al, hair_box(facing))
+ med = float(np.median(whole))
+ try:
+ head: dict[str, Any] = head_track(al)
+ except ValueError as exc:
+ head = {"skipped": str(exc)} # recorded, and the head gate says it could not read it
+ return {
+ "index": round(jolt_index(whole), 4),
+ "hair_index": round(jolt_index(hair), 4),
+ "step_max_over_median": round(float(whole.max()) / med, 4) if med > 0 else None,
+ "head": head,
+ }
+
+
+def jolt_verdict(measured: dict[str, Any], *, jolt_max: float | None = JOLT_REFERENCE,
+ head_step_max: float | None = HEAD_STEP_REFERENCE) -> list[str]:
+ """What exceeds the bounds, in words; empty when the loop stays inside. A bound of None is not checked."""
+ over = []
+ if jolt_max is not None and measured["index"] > jolt_max:
+ over.append(f"jolt index {measured['index']:.3f} exceeds {jolt_max}")
+ if head_step_max is None:
+ return over
+ if "skipped" in measured["head"]:
+ over.append(f"the head could not be tracked ({measured['head']['skipped']})")
+ return over
+ head = measured["head"]["x"]
+ if head["step_max_pct"] > head_step_max:
+ over.append(f"the head moves sideways {head['step_max_pct']:.2f} % of the body height in one frame "
+ f"(into frame {head['worst_into_frame']}), over {head_step_max} %")
+ return over
diff --git a/tests/video/test_loop_repair.py b/tests/video/test_loop_repair.py
index 45d64c1..daaecdc 100644
--- a/tests/video/test_loop_repair.py
+++ b/tests/video/test_loop_repair.py
@@ -158,3 +158,86 @@ def test_repair_off_cuts_as_filmed_and_says_so(tmp_path):
rep = _run(tmp_path, _keyed(tmp_path, filmed), repair="off")
assert rep["jump_repair"] == {"applied": False, "why": "--repair off"}
assert rep["seam_measurement"] == "source-frames"
+
+
+# --- the jolt index and its gate (docs/loop-repair.md section 3) ---------------------------------
+
+
+def test_jolt_index_is_zero_for_even_steps_and_high_for_alternating_ones():
+ assert repair.jolt_index(np.full(12, 0.3)) == 0.0
+ alternating = np.array([0.2, 0.4] * 6)
+ assert repair.jolt_index(alternating) == pytest.approx(0.2 / 0.3)
+
+
+def _head_pop(frames: list[Image.Image], k: int, dx: int) -> list[Image.Image]:
+ """Frame k with the head (rows 10..30) moved `dx` px sideways."""
+ out = list(frames)
+ a = np.asarray(frames[k]).copy()
+ head = a[10:30].copy()
+ a[10:30] = 0
+ a[10:30] = np.roll(head, dx, axis=1)
+ out[k] = Image.fromarray(a, "RGBA")
+ return out
+
+
+def test_a_head_that_pops_sideways_is_named_by_the_gate():
+ filmed, _ = _loop()
+ smooth = repair.measure_jolt(filmed)
+ assert repair.jolt_verdict(smooth) == []
+ assert smooth["index"] < repair.JOLT_REFERENCE and smooth["head"]["x"]["step_max_pct"] < repair.HEAD_STEP_REFERENCE
+ popped = repair.measure_jolt(_head_pop(filmed, 7, 4))
+ over = repair.jolt_verdict(popped)
+ assert len(over) == 1 and "head moves sideways" in over[0] and "(into frame 7)" in over[0]
+ assert popped["head"]["x"]["worst_into_frame"] == 7
+
+
+def test_a_loop_with_no_head_to_track_says_so_instead_of_passing():
+ filmed, _ = _loop()
+ blank = list(filmed)
+ blank[3] = Image.new("RGBA", filmed[3].size, (0, 0, 0, 0))
+ measured = repair.measure_jolt(blank)
+ assert "skipped" in measured["head"]
+ assert any("could not be tracked" in o for o in repair.jolt_verdict(measured))
+
+
+def test_video_loop_reports_the_jolt_of_the_loop_as_it_plays_after_the_repair(tmp_path):
+ filmed, truth = _loop({10: 12})
+ rep = _run(tmp_path, _keyed(tmp_path, filmed), interpolate=lambda a, b, t: truth[10])
+ jolt = rep["jolt"]
+ assert jolt["measured"] == "after the jump repair" and jolt["warnings"] == [] and jolt["passed"]
+ assert jolt["gated"] is False and jolt["gate"] is None
+ assert jolt["reference"] == {"jolt_max": repair.JOLT_REFERENCE, "head_step_max": repair.HEAD_STEP_REFERENCE}
+ assert set(jolt["head"]["x"]) >= {"step_max_pct", "step_median_pct", "max_over_median", "range_pct"}
+ assert set(jolt["head"]["y"]) >= {"step_max_pct", "step_median_pct", "max_over_median", "range_pct"}
+
+
+def test_by_default_a_jolting_loop_is_kept_with_a_warning_line(tmp_path, capsys):
+ filmed, _ = _loop()
+ popped = _head_pop(filmed, 7, 4)
+ rep = _run(tmp_path, _keyed(tmp_path, popped), interpolate=lambda a, b, t: popped[7])
+ assert rep["status"] == "passed" and rep["jolt"]["passed"] and rep["jolt"]["gated"] is False
+ assert len(rep["jolt"]["warnings"]) == 1 and "head moves sideways" in rep["jolt"]["warnings"][0]
+ assert "video-loop: warning: the head moves sideways" in capsys.readouterr().err
+
+
+def test_a_bound_the_caller_passes_refuses_a_jolting_loop(tmp_path):
+ filmed, _ = _loop()
+ popped = _head_pop(filmed, 7, 4)
+ stays = lambda a, b, t: popped[7] # an in-between that cannot take the pop out
+ with pytest.raises(SystemExit, match=r"^video-loop: loop jolts — the head moves sideways .* regenerate the clip$"):
+ _run(tmp_path, _keyed(tmp_path, popped), interpolate=stays, head_step_max=repair.HEAD_STEP_REFERENCE)
+ report = json.loads((tmp_path / "w.json").read_text())
+ assert report["status"] == "failed" and report["jolt"]["passed"] is False and report["jolt"]["over"]
+ assert report["jolt"]["gate"] == {"jolt_max": None, "head_step_max": repair.HEAD_STEP_REFERENCE}
+ assert report["jump_repair"]["replaced"] == [7]
+ # only the bound passed is checked: a jolt bound alone lets the head through
+ again = tmp_path / "again"
+ again.mkdir()
+ rep = _run(again, _keyed(again, popped), interpolate=stays, jolt_max=10.0)
+ assert rep["status"] == "passed" and rep["jolt"]["gated"] and rep["jolt"]["warnings"]
+
+
+def test_repair_off_measures_the_jolt_and_does_not_gate_it(tmp_path):
+ filmed, _ = _loop()
+ rep = _run(tmp_path, _keyed(tmp_path, _head_pop(filmed, 7, 4)), repair="off", head_step_max=0.1)
+ assert rep["jolt"]["gated"] is False and rep["jolt"]["warnings"] and rep["status"] == "passed"