Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
docs: size hold before the search, widened scale-back, video-follow; changelog
Soohong Kim9854f5fbd48f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a531da5..1de70f8 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,13 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## Unreleased (v2.24.0) - A walk filmed from its first frame loops at one size, a frame scaled up keeps its crown, and a soft part can follow the body++- **Output change: a walk or run loop whose clip changes size by 1 % or more is cut differently.** `video-loop --anchor motion-auto` now holds the clip at its first frame's size before the cycle search (`--size-hold auto`, the default; `size_hold` in the report): the subject's fitted height over the clip is measured (`gait_fallback.scale_drift`) and, at 1 % or more (`SIZE_HOLD_MIN`), every frame is scaled back to the first frame's height about its fitted foot point (`undo_scale`). Until now that ran only in the gait fallback, after the search had failed and at 3 % or more, so a clip that grew 1–3 % and still looped was cut on frames of changing size: the frame one cycle on was larger than the first, and the loop popped at the wrap. A clip under 1 % is cut exactly as before; `--size-hold off` gives the old cut. A clip that grows 2.5 % now loops at its size where it found no cycle before (`tests/video/test_gait_fallback.py`); a front catwalk filmed from its first frame (+2.0 %) went from a seam ratio of 1.44 to 1.19. [docs/video-pipeline.md](docs/video-pipeline.md#4-loop--period-first-seam-second-then-the-gait-floor).+- **Fix: scaling a frame back up cut off what it carried past the frame's edge.** A clip that shrinks is scaled up about its feet; when the feet also rose in the frame (a back walk going away), the crown landed above the frame and was cut in every frame it reached. Frames are now widened by the room the furthest one needs (`gait_fallback.undo_padding`, the same for every frame, `padding_ltrb` in `size_hold` and `gait_fallback`), so nothing is lost; a clip that needs no room keeps its frame size and bytes.+- **New `sprite-gen video-follow`**: a soft part of a looping body (a chest, a belly, a pouch on a strap) follows the body's motion. An ellipse in the strip's first cell (`--region cx,cy,rx,ry`, repeatable) is moved as a damped mass on the body's own bob — the crown's row and the head's middle read off the cells, the periodic steady state of x'' + 2ζωx' + ω²x = −body'' per harmonic of the cycle (`--freq` 2.4 Hz, `--zeta` 0.6, `--gain` 2.5 times the physical answer) — so it lags and settles with no kick, and the last frame leads into the first. Only the region's pixels change; a move that would fold it over is refused. The strip, GIF and WebP are rewritten and checked, `follow` is recorded in the strip meta, and the strip as it was is kept as `follow.source.png`, so a second run never moves a moved strip. The region is the caller's to give: the engine does not find it. [docs/video-pipeline.md](docs/video-pipeline.md#6-follow-through--video-follow).+- **`video-cycle-align` clears a follow-through**: it rebuilds the strip from the cut, so it removes `follow.source.png` and the `follow` record and says `follow_cleared` in the loop's row; a new `video-loop` cut removes them too. Run `video-follow` after the alignment.+## v2.23.0 - A walking side view shows the far wrist's item each time that arm swings forward; without `--handed` every prompt is 2.22.0's- **Without `--handed`, every prompt is 2.22.0's, byte for byte.** Clips (built-in or `--motion`, every model and pin), `video-prompt`, stills (view, facing, reference, key line, layout guide, the `--facing-fix regen` regeneration), `video --direction side` and the mid-step redraw send what 2.22.0 sent. `tests/gen/test_prompt_freeze.py` holds every prompt string in that table to `tests/fixtures/prompts-v2.22.0.json.gz`, drawn from the 2.22.0 tag's source. The engine's pieces go on through one place (`sprite_gen/gen/prompt_parts.py`, `Prompt.add`), which puts them on as 2.22.0 did; with `--handed` the prompt is that same prompt with the handed sentences put in, and nothing else changes. [docs/prompt-assembly.md](docs/prompt-assembly.md) lists the faults of the 2.22.0 prompts kept until a before-and-after comparison says to change them.diff --git a/SKILL.md b/SKILL.mdindex 486dccb..15041a3 100644--- a/SKILL.md+++ b/SKILL.md@@ -100,6 +100,7 @@ hold the arms still, and check the loop with `handed-check --state walk`.| Standalone video / animate a still, pin a last frame, reference images | `video` (`--image`, `--last-frame`, `--reference`) | [video](docs/video.md) || Continue or edit an existing clip | `video-extend`, `video-edit` | [video](docs/video.md) || Grok video sprites | `video-set` | [video-pipeline](docs/video-pipeline.md) |+| A soft part of a walk loop follows the body (a chest, a belly): an ellipse in the first cell, after `video-cycle-align` | `video-follow` | [video-pipeline](docs/video-pipeline.md#6-follow-through--video-follow) || Video sprites from a clip your agent makes with a connected video MCP (ZCRE, 지크) | `video-canvas`, `video-prompt --no-last-frame`, the agent's own MCP tools (quote, user approval, generate, save the mp4), then `video-frames`, `video-loop`. sprite-gen never calls the MCP. Walk, run and jump only: ZCRE's `grok-imagine-video-1.5` has no end frame, so idle, attack and diagonal walks are refused unless `--unpinned` | [video-pipeline](docs/video-pipeline.md#a-clip-from-a-video-mcp-on-your-agent--zcre) || Ordinary image or edit | `gen --provider codex` or `gen --provider grok` (subscription routes) | [gen](docs/gen.md) || Image generation with no login available (server, container, SaaS) | `gen --provider openai` — server/SaaS route on `OPENAI_API_KEY`, **billed per call**, never a default or a fallback | [gen](docs/gen.md#subscription-first--openai-is-named-or-it-does-not-run) |diff --git a/docs/README.md b/docs/README.mdindex 5ceef7a..26bc397 100644--- a/docs/README.md+++ b/docs/README.md@@ -16,7 +16,7 @@ flowchart LRdirection LRb1[video-canvas] --> b2[video] --> b3[video-frames] --> b4[video-loop]b5[video-set] -.runs all four.-> b1- b4 --> b6[video-cycle-align]+ b4 --> b6[video-cycle-align] --> b7[video-follow]endsubgraph C["C · utilities"]direction LR@@ -38,7 +38,7 @@ flowchart LR| Pipeline / tool group / workflow | Entry doc | Verbs ||---|---|---|| **A · atlas rows** — one still becomes a runtime sprite sheet | [run-contract.md](run-contract.md) | `prepare` → `gen` / `gen-set` → `extract` → `compose-atlas`; optional `curation` and recompose |-| **B · video → loop** — one still becomes transparent motion loops | [video-pipeline.md](video-pipeline.md) | `video-canvas` → `video` (or `video-prompt` and a video MCP on your agent) → `video-frames` → `video-loop`, `video-set`, `video-cycle-align`; `handed-check` for an item on one side |+| **B · video → loop** — one still becomes transparent motion loops | [video-pipeline.md](video-pipeline.md) | `video-canvas` → `video` (or `video-prompt` and a video MCP on your agent) → `video-frames` → `video-loop`, `video-set`, `video-cycle-align`, `video-follow`; `handed-check` for an item on one side || **C · utilities** — imported images in, clean cuts out | [sheet-slicing.md](sheet-slicing.md) | `cutout`, `slice-sheet`, `unpack-atlas` || **D · post-processing** — finished sheets, refined | [recolor.md](recolor.md) | `recolor`, `recolor-palette`, `compose-layers`, breathing (compose), `export-pngs`, `export-aseprite` || **E · asset tools** — independent background, shadow and motion tools | [asset-tools.md](asset-tools.md) | `background-tile`, `shadow`, `inspect-motion` |diff --git a/docs/loop-repair.md b/docs/loop-repair.mdindex 2441f37..15beb50 100644--- a/docs/loop-repair.md+++ b/docs/loop-repair.md@@ -277,7 +277,9 @@ carries `cycle_align` per state and `<state>.cycle-align.json`; a failed alignme`cycle-align:<state>` and the loops stay as cut. Without RIFE the alignment is skipped, notfailed (section 1, "Without RIFE"); install it and run `video-cycle-align` on the set's loops.Cutting a loop again with `video-loop` removes its `cycle.source/`, so the next alignment reads-the new cut.+the new cut. An alignment also clears a follow-through (`video-follow`, which moved the old+cells): `follow.source.png` and the strip's `follow` record are removed and the loop's row says+`follow_cleared`; run `video-follow` again after it.How much RIFE that is, on the 2026-10-03 sets (the experiment's `finalize.py`): resampling tothe median made 18 of 21 frames for a Lite side loop of 27, 20 of 21 for a back-diagonal loop ofdiff --git a/docs/loop-review.md b/docs/loop-review.mdindex d2039a4..eb81518 100644--- a/docs/loop-review.md+++ b/docs/loop-review.md@@ -94,7 +94,10 @@ fallback below looks past the state's window, and only after this search found n### The gait fallback: a slow walk, or a walk toward the cameraWhen the local search finds no cycle, `motion-auto` looks once more, for two things a-front or back gait does that the first search cannot see past:+front or back gait does that the first search cannot see past. (Since 2.24.0 a clip that changes+size by 1 % or more is held at its first frame's size before the first search —+[video-pipeline.md](video-pipeline.md) section 4, `size_hold` — so the fallback below finds+nothing left to scale back unless `--size-hold off` was passed.)- **It walked toward the camera, or away from it.** Asked to walk in place, a front walksometimes comes closer, and the body grows through the clip, so the same pose never matchesdiff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex af99bab..1fd25df 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -23,6 +23,7 @@ still ──video-canvas──▶ canvas.png ──video──▶ clip.mp4 ─| `sprite-gen video-frames` | `sprite_gen/video/frames.py` | mp4 → `raw/`, `keyed/` RGBA frames + report || `sprite-gen video-loop` | `sprite_gen/video/loop.py` | keyed frames → `cycle/`, `<name>.strip.png` + `.strip.json`, `<name>.gif`, `<name>.webp` + report || `sprite-gen video-set` | `sprite_gen/video/batch.py` | bases × states → one folder per item, `set.report.json`, `table.md` |+| `sprite-gen video-follow` | `sprite_gen/video/follow.py` | a loop directory + an ellipse → the strip, GIF and WebP with that region following the body ([section 6](#6-follow-through--video-follow)) |Wrappers: `scripts/video_canvas.py`, `scripts/video_frames.py`, `scripts/video_loop.py`,`scripts/video_set.py`. Binaries: `ffmpeg`/`ffprobe` (frames), `img2webp` from libwebp@@ -634,6 +635,26 @@ anatomical left/right contacts. The report records `half_period_guard.reason ="ambiguous-harmonic"` and `cycle.review_recommended = true`. See [loop review](loop-review.md)for the visual review contract and manual overrides.+**The size is held before the search** (`--anchor motion-auto`, `--size-hold auto`, the default).+A walk or run filmed from its first frame only (no end frame) often grows or shrinks as it plays:+a front walk comes a little closer with every step, a back walk goes away. The loop is then cut on+frames that change size, so the frame one cycle after the first is not the first's size and the+loop pops at the wrap. A straight-line fit of the subject's opaque height over the clip measures+it (`gait_fallback.scale_drift`); at 1 % or more (`SIZE_HOLD_MIN`) every frame is scaled back to+the first frame's fitted height about its fitted foot point before the cycle search, so the+search, the seam and the cells all read frames of one size. Below 1 % the fit is a head bob and+hair and the frames are searched as filmed. The report says what was measured and done+(`size_hold`: `drift`, `applied`, `padding_ltrb`). `--size-hold off` searches the frames as+filmed, as before 2.24.0. Measured on one front catwalk filmed from its first frame (+2.0 % over+3 s): the engine's seam ratio of the cut went from 1.44 to 1.19.++**Nothing is cut when a frame is scaled up.** A clip that shrinks is scaled up about its feet,+and when the feet also rose in the frame (a back walk going away toward the horizon) the crown+lands above the frame. Every frame is first widened by the room the furthest one reaches past+each edge (`gait_fallback.undo_padding`, the same for every frame; `padding_ltrb` in the+report), so no part is lost. A frame that needs no room keeps its size and bytes. The gait+fallback's own scale-back (below) widens the same way.+Gates, all fail-loud: no period (profile flat, below the recorded `periodicity_min`), loop seam ratioabove `--seam-max` (2.0), GIF/WebP re-opened and checked (frame count, `loop=0`,transparent corners, no RGB under alpha 0 in the WebP).@@ -777,6 +798,44 @@ per state, and a failed alignment is listed as `cycle-align:<state>`. Without RIis skipped with a warning (`applied: false`, and a line under the report's `warnings`), not failed. See[loop repair](loop-repair.md) section 4.+## 6. Follow-through — `video-follow`++A clip model draws a walk's body moving, but a soft part hanging off it (a chest, a belly, a+pouch on a strap) mostly moves with the body as one piece, even when the prompt asks it to+bounce. `video-follow` puts that follow-through back on a cut loop:++```bash+sprite-gen video-follow --loop-dir set/front-walk/loop --region 136,164,60,50 [--region …] \+ [--gain 2.5] [--freq 2.4] [--zeta 0.6] [--board follow.png]+```++- **The region** is an ellipse over the part in the strip's first cell, in cell pixels+ (`cx,cy,rx,ry`; repeatable). It is carried with the body's bob from cell to cell. Somebody has+ to say where it is — the engine does not find it: look at the first cell (or ask a vision model+ for the four numbers once per direction; a mirrored direction takes the mirror's region).+- **The motion** is the body's own: the crown's row (up and down) and the middle of the head+ (side to side), read off the cells. The part is a damped mass on the body — its offset x from+ where the body carries it answers x'' + 2ζωx' + ω²x = −body'' — solved in the loop's periodic+ steady state per harmonic of the cycle (the first six), so it lags the bob and settles, with no+ kick at a foot strike, and the last frame leads into the first. `--freq` (2.4 Hz) and `--zeta`+ (0.6) set the part; `--gain` (2.5) scales the physical answer and nothing else. On a front+ walk with a 12 px bob in a 600 px body that is a move of about ±5 px; `--gain 1` is the mass as+ measured, `0` gives back the strip as it was.+- **Only the region moves.** Inside it every pixel is moved by the offset times a weight that is 1+ at the centre and 0 at the rim (cos²), sampled as premultiplied bilinear colour; outside it no+ pixel changes. A move so large that the weight's slope folds the picture over+ (offset × π / (2 · radius) ≥ 1) is refused: lower `--gain` or give the region larger radii.+- **What it writes**: the strip, GIF and WebP over the loop's own (both animations re-opened and+ checked as `video-loop` checks them), and `follow` in `<name>.strip.json` (the regions, the+ settings, the body's bob, `dx_px`/`dy_px` per cell, `reach_px`). `cycle/` is left as cut. The+ strip as it was is kept as `follow.source.png`; running `video-follow` again reads from it, so a+ second run never moves a moved strip. `--board` writes the cells before and after where the part+ sits lowest and highest, on white.+- **Order**: after `video-cycle-align`. An alignment rebuilds the strip from the cut, removes+ `follow.source.png` and the `follow` record, and says so (`follow_cleared` in its loop row); a+ new cut with `video-loop` removes them too. Run `video-follow` again after either.+- A one-shot (`kind: one-shot`) is refused: the follow-through is a loop's steady state.+## What the rules were measured onEvery threshold above (the 15 % period tolerance, the 2.0 seam gate, the 0.15