Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

docs(video-loop): root motion and the air retimed, in video-pipeline section 4, SKILL.md and the CHANGELOG's Unreleased

Soohong Kim2dceca84d067
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d12be7d..5d91464 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,17 @@
All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.
+## Unreleased (v3.10.0)
+
+**Upgrading from 3.9.0: a cut played once with a body (`video-loop --cycle once`, `once-return`, `one-shot`, and so `video-set`'s single-play states) gains `root_motion` in its report and strip metadata: each cell's root, what it stands on, and its airs. Its cells and every other key keep their bytes and values. The air is drawn again only when asked (`--air-retime`).**
+
+- **New: root motion on every cut played once.** `root_motion` gives each cell's root (the subject's mass centre, as `inspect-motion`'s `centroid_px`, in cell px), the floor's row (`ground_y`), the runs of cells on the ground or in a contact the caller names (`--contact hand=12:20`, frames of the clip), and each air: its take-off and landing cells, the seconds between them, its apex and the cell it peaks in. With `--meters-per-px M` it adds each root in metres from the first cell's (`root_m`) and each apex in metres. A game reads `root` to place each cell and moves the body on its own physics. `video-loop` prints each air in one line (`air 0.42 s, apex 0.21 m`) and `video-set` puts `loop.air` on each item.
+- **New: `video-loop --air-retime gravity=G,air=T,speed=V`** (and `video-set --air-retime`). Each air with its take-off and landing in the cut is given `T` seconds, `G` m/s² and `V` m/s along, any of them, each the clip's own where not given; `G` and `V` need `--meters-per-px`. The air's cells are spread evenly over its new seconds, each the clip's pose at the same share of its air, made with RIFE between two frames where it falls between them (the held-cell fill's frames, judged the same way), and moved whole onto the parabola from the take-off's root to the landing's; the landing and the cells after it move along with it, so a planted foot stays planted. The take-off, the landing and the cells on the ground before it are the frames as filmed. The strip metadata's `air_retime` records each air's seconds and apex before and after, the gravity and speed used, how far along it moved, the cells made and the poses over cells. With no RIFE and a frame to make, the air is cut as filmed and a warning names the motion and the install line. A loop, or a cut with no such air, is cut as filmed and says why. A retimed cut is not restored from its source frames (`video-loop-repair`).
+- **New: `video-loop --cycle once --start S --length L`** plays exactly those frames once (`terminal.kind: given`): a move cut out of a longer motion that settles on no pose, such as a vault in a run. Before, a single-play cut refused `--start` and `--length`.
+- Python callers: `sprite_gen.video.root_motion` (`parse_retime`, `parse_contacts`, `clip_floor`, `root`, `touching`, `reading`, `read`, `sidecar`, `said`, `apex`, `retime`, `public`); `held_fill.at_times` (a cell at each given time, taken out of `upsample`); `qa.motion.centroid` and `qa.motion.contact_spans` (were `_contact_spans`); `nonloop.on_floor`, `nonloop.GROUND_PX`, `nonloop.given_ending`; `loop.run_loop(..., air_retime=, meters_per_px=, contacts=)`; `batch.run_set(..., air_retime=, meters_per_px=)`.
+- **Docs**: [video-pipeline.md](docs/video-pipeline.md) section 4, "Root motion" (the keys, how a game reads `root`, the mass centre and not a body part, `--cycle once --start --length`, the air retimed, what it takes from sprite-gen); `SKILL.md` (tell the user each air's seconds and apex).
+- **Tests**: `tests/video/test_root_motion.py` with `tests/video/jump_inputs.py` (a synthetic jump whose mass centre flies a parabola under a known gravity: the root read within half a pixel, the air's cells, seconds and apex, the ground and a named hand contact; the air doubled peaks at g T²/8 with twice its cells, every cell on the parabola, none shown again, the take-off and the ground before it byte for byte, the landing moved along whole; the air alone keeps the clip's gravity; no RIFE cut as filmed with the motion named; the six cuts `tests/video/test_held_fill.py` pins keep their bytes and their metadata but the new key; the refusals; `video-set` hands the handle on).
+
## v3.9.0 - Provider address environment variables, subscription logins off, and the capabilities verb
**Upgrading from 3.8.0: nothing changes unless you set one of the new variables. Each provider's calls can go to another address (`SPRITE_GEN_OPENAI_BASE_URL`, `SPRITE_GEN_XAI_BASE_URL`, `SPRITE_GEN_ANTHROPIC_BASE_URL`, `SPRITE_GEN_ELEVENLABS_BASE_URL`), `SPRITE_GEN_LOGINS=off` runs every call on its API key, and the new verb `sprite-gen capabilities` prints every verb and argument as one JSON line. Unset, every request goes where it went with the same bytes, and every output keeps its bytes except `sprite-gen --help`, which lists the new verb.**
diff --git a/SKILL.md b/SKILL.md
index 1c4296f..476c1db 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -133,6 +133,8 @@ For attack repeat coverage, observed one-shot returns and structured loop failur
Every cut is read for held cells (neighbour cells showing the same picture), and by default they are filled with RIFE's frames between and the cut read again (`video-loop --fill-held auto`; `off` cuts as filmed). Tell the user each motion's poses over cells in one line from `held_fill` (`poses/cells`, before and after, e.g. `24/27` to `27/27`). A cut wanted above its clip's rate is given with `--out-fps`, frames between, never repeats. Details in [loop repair](docs/loop-repair.md#7-held-cells-a-picture-shown-again-and-a-cut-at-a-higher-rate).
+A move played once (a jump, a vault: `once`, `once-return`, `one-shot`) carries its root motion in the strip sidecar by default, the cells unchanged: `root_motion` gives each cell's root (the body's mass centre, cell px), what it stands on, and each air's take-off, landing, seconds and apex; a game reads `root` to place each cell and moves the body on its own physics. A move cut out of a longer motion is named with `video-loop --cycle once --start S --length L`, a hand on something the clip does not draw with `--contact hand=A:B`, metres with `--meters-per-px`. To give the air its own length, gravity and speed, `--air-retime gravity=G,air=T,speed=V` redraws it on that parabola with RIFE's frames between; contacts and the take-off and landing stay as filmed. Tell the user each air in one line, its seconds and its apex (the summary's `air`, e.g. `air 0.42 s, apex 0.21 m`). Details in [video-pipeline](docs/video-pipeline.md#root-motion--a-cut-played-once-where-the-body-is-what-it-stands-on-its-air).
+
A pinned idle or attack whose last frame the video model drew again has its wrap blended by default (`video-loop --wrap-blend auto`, a redraw only; `off` cuts as filmed), with details in [loop repair](docs/loop-repair.md#6-the-redrawn-wrap-a-pinned-clip-whose-last-frame-the-model-drew-again).
Scene creation consumes finished assets and remains optional. Asset metadata owns frames, native durations and anchors; scene specs own placement, scale, playback rate, planes, camera and light. Measure stride only with declared same-foot contact and an isolated foot ROI; unknown contact stays unverified. Apply only a verified report for the exact selected asset with an explicit scene direction. Never infer walking direction from the bottommost silhouette, reverse frames or change source assets to make a scene work.
@@ -143,7 +145,7 @@ Scene creation consumes finished assets and remains optional. Asset metadata own
## RIFE for walk and run loops
-Without RIFE, tell the user which loops went unrepaired and which motions' held cells were not filled (`held_fill.unmade`, or `--out-fps` not met), and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).
+Without RIFE, tell the user which loops went unrepaired, which motions' held cells were not filled (`held_fill.unmade`, or `--out-fps` not met) and which airs were not retimed (`air_retime.why`), and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).
## Contracts and advanced tools
diff --git a/docs/README.md b/docs/README.md
index 5efa006..9b5f007 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -84,7 +84,7 @@ grouping is derived from `sprite_gen/_modules.py`, the one taxonomy table.
| [logo.md](logo.md) | `sprite-gen logo`: a title logo or wordmark lettered by the model into a title screen or poster, only its lettering redrawn on a chroma key, cut out with its glow kept — the steps, files, report, text check and refusals, the key choice, the logo matte and its known limit |
| [prompt-assembly.md](prompt-assembly.md) | What the engine adds to a still or clip prompt and when: the pieces, the 2.22.0 freeze without `--handed`, what counts as already said, and the notes on the caller's own words |
| [video.md](video.md) | `sprite-gen video` / `video-extend` / `video-edit` / `video-resume`: stills and clips to mp4 through Grok Imagine (image-to-video, last-frame pin, references, extension, editing, asking again for an unfinished generation) with the user's own credential |
-| [video-pipeline.md](video-pipeline.md) | Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, the batch, a set's directions on one sheet, a walk or run's stop |
+| [video-pipeline.md](video-pipeline.md) | Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, a single play's root motion and its air retimed, the batch, a set's directions on one sheet, a walk or run's stop |
| [loop-review.md](loop-review.md) | Automatic loop decisions, ambiguous gait review, visual evidence and explicit cut/alignment overrides |
| [loop-repair.md](loop-repair.md) | RIFE (where it runs, cost, licences, `sprite-gen rife install`, what runs without it), jump-frame repair, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the overlapped wrap (a part on its own beat at a two-step cut's wrap), the part pop (a hand or a foot that jumps at the wrap), the jolt index and its gate, one cycle length per direction set |
| [frame-interpolation.md](frame-interpolation.md) | Generative in-betweens for sprite frames, recorded as a take |
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index ded718c..d87a9ed 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -1,6 +1,6 @@
# Video → sprite pipeline (engine SSoT)
-> Owns: Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, the batch, a set's sheet, a walk or run's stop · Index: [docs/README.md](README.md)
+> Owns: Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, a single play's root motion and its air retimed, the batch, a set's sheet, a walk or run's stop · Index: [docs/README.md](README.md)
One still becomes a whole motion set: the still is padded into the canvas a state
needs, Grok Imagine animates it in place, the clip is keyed frame by frame, and one
@@ -551,6 +551,9 @@ the cut removes an earlier one. An argument error touches nothing.
the start image). The cycle is every frame but the last, which re-renders the first, so the
wrap plays like the step into that last frame. Where the pinned frame redraws the first, `--wrap-blend auto` (the default) takes the first frame's drawing back over the last half second, a redraw only, recorded as `wrap_blend`. Passing `--wrap-blend off` cuts as filmed, but [the seam bridge](loop-repair.md#the-seam-bridge-a-loop-that-is-not-a-walk-whose-wrap-jumps) can still make frames round the wrap of a loop that is not a walk or run; for such a loop, pass `--repair off --wrap-blend off` for the earlier bytes. See [loop repair](loop-repair.md#6-the-redrawn-wrap-a-pinned-clip-whose-last-frame-the-model-drew-again) for the gating and blending rules.
+`--cycle once --start S --length L` plays exactly those frames once, a move cut out of a longer motion that settles
+on no pose ([root motion](#root-motion--a-cut-played-once-where-the-body-is-what-it-stands-on-its-air)).
+
`--cycle once-return` is for a clip pinned to end on its first frame that plays once (`video-set
--play once-return`, [play modes](#play-modes)). It keeps every frame, the last too, so the strip
starts and ends on the first pose and a game plays it once and goes on from the pose it started in.
@@ -599,6 +602,78 @@ Outputs:
requires an explicit `--body-height` target and still respects that cap.
- `<name>.webp` — same frames, lossless, `img2webp -exact`.
+### Root motion — a cut played once: where the body is, what it stands on, its air
+
+A cut played once with a body (`once`, `once-return`, `one-shot`: a jump, a vault, a strike; not an effect's
+lifetime) carries its root motion in the strip sidecar, `root_motion`, by default. The cells are the cut's as
+before (`sprite_gen/video/root_motion.py`). Everything is read on the strip's cells, at their times as they play:
+
+- `root` — each cell's root, in cell px from the cell's top-left: the subject's mass centre (the mean of its pixels,
+ alpha over 0, as `inspect-motion` reads a frame's `centroid_px`). **A game draws cell k with that point on the
+ body's place.** Moving the body by the roots' steps plays the cut as filmed; moving it by the game's own physics
+ plays the same cells on the game's path. The cells keep the clip's pixels (a jump rises inside its cell), so a game
+ that draws every cell at one place gets the motion as filmed without reading the key.
+- `ground_y` — the floor's row, cell px: the lowest row any frame of the clip reaches. A cell stands on it where its
+ lowest body row is within 4 source px of it (`nonloop.on_floor`, the frames the strip's standing height reads).
+- `contacts` — each run of cells on the ground (`by: ground`) or in a contact the caller names over frames of the
+ clip, in `inspect-motion`'s spans (`--contact hand=12:20`): a hand on a wall the clip does not draw, which no floor
+ shows.
+- `air` — each run of cells touching nothing: `takeoff` and `landing` (the cells before and after), `seconds` between
+ them, `apex_px` (how far the root rises over the take-off's, cell px) and `apex_cell`. An air whose take-off or
+ landing is not in the cut has `seconds: null`.
+- With `--meters-per-px M` (the metres a pixel of the clip as filmed holds: the standing height in metres over the
+ strip's `body_src_h`): `meters_per_px` (a cell px's), `root_m` (each root from the first cell's, metres, up
+ positive) and each air's `apex_m`.
+
+Tell the person each air in one line, its seconds and its apex (`air 0.42 s, apex 0.21 m`: the summary's `air`, a
+`video-set` item's `loop.air`).
+
+The root is the mass centre (the centre of mass), not a body part: in the air it is the point that falls on a parabola
+under gravity whatever the arms and legs do. On a synthetic jump whose mass centre flies under a known gravity it is
+read within 0.46 px (0.47 % of the body's standing height, `tests/video/test_root_motion.py`). As the legs tuck and
+stretch it moves inside the body, by several percent of the body's height on a hard tuck, so a game that pins a body
+part to its path reads that part itself.
+
+**A move cut out of a longer motion — `--cycle once --start S --length L`.** A vault in a run settles on no pose,
+so `once` finds no end and `one-shot` no return. Given both, `once` plays exactly those frames of the clip once
+(`terminal.kind: given`), its end pose not read.
+
+**The air given its own length, gravity and speed — `--air-retime gravity=G,air=T,speed=V`.** Each air with its
+take-off and its landing in the cut is given `T` seconds, a gravity of `G` m/s² and a speed along of `V` m/s, any of
+them, each the clip's own where not given (its gravity read off the root's rise over the line from take-off to
+landing). `G` and `V` need `--meters-per-px`. The cut is made again at its rate (`--out-fps`, or the clip's): the
+cells up to the take-off and from the landing at the clip's pace, the air's spread evenly over its new seconds, each
+showing the clip's pose at the same share of its air, made with RIFE between the two frames it falls between where
+it falls between them (`held_fill.at_times`, the frames between [the held cells](loop-repair.md) get), never a
+frame shown again. Each air cell is then moved whole, by whole pixels, until its root stands on the parabola from the
+take-off's root to the landing's; the landing and every cell after it are moved along by as much further as the air
+now carries the body, so a planted foot stays planted. The take-off, the landing and the cells before the take-off are
+the frames as filmed. The strip sidecar's `air_retime` says what was asked and, for each air, `seconds_before` and
+`seconds`, `apex_before_px` and `apex_px` (`apex_m`), the gravity and speed used, `moved_along_px`; and the cells made,
+the cells shown again, the poses over the cells; `root_motion` is read on the retimed cells. A cut with no such air,
+or a loop, is cut as filmed and says why. With no RIFE installed and a frame to make, the air is cut as filmed and a
+warning names the motion, its air cells and the install line; an air given its own seconds at the clip's rate (every
+cell a frame) needs none. `video-set --air-retime --meters-per-px` hands both to every cut.
+
+A vault cut out of a run, its hand on a wall the clip does not draw: `--cycle once --start 10 --length 40
+--contact hand=12:20 --meters-per-px 0.005 --air-retime gravity=9.81,air=0.6,speed=4` gives the flight from the
+hand leaving the wall to the landing 0.6 s at 9.81 m/s² and 4 m/s, its seconds held to whole cells (14 cells at 24
+fps, 0.583 s), and says its seconds and apex.
+
+What it takes from sprite-gen, and what it writes new:
+
+| Question | Answered by |
+|---|---|
+| Where is the body in a frame | `qa.motion.centroid`, `inspect-motion`'s `centroid_px`, taken out of `analyze_motion` |
+| Does a frame stand on the floor | `nonloop.on_floor`, the strip's standing-height rule, taken out of `loop.build_strip` |
+| Which frames does a named contact span | `qa.motion.contact_spans`, `inspect-motion --contacts` |
+| A cell at a time between two frames, judged, never a frame shown again | `held_fill.at_times`, taken out of `held_fill.upsample`, with RIFE located once a frame is to be made (`loop._located`) |
+| The poses over the cells | `held_fill.same_steps`, `held_fill.reading` |
+| The air, its seconds and apex; the clip's gravity; the cells' times and places on the parabola | new (`root_motion.read`, `retime`): no engine function read an air or a gravity |
+
+A jump in place given `speed=` goes the `--facing` way. A landing on something higher than the clip's floor (a box,
+a ledge) reads as air until the body comes down to the floor: name it with `--contact`.
+
## 5. Set — the batch
`sprite-gen video-set --base side=side.png --base front=front.png --states idle,walk,run,jump,attack --out-dir set/`