Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

docs(video-loop): held cells found, filled and read again as the default step of every cut, and --out-fps, in loop-repair section 7, video-pipeline, curation, SKILL.md and the CHANGELOG's Unreleased

Soohong Kim136db9e0206e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 20b1254..cddf8e7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,18 @@
All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.
+## Unreleased (v3.8.0)
+
+**Upgrading from 3.7.0: `video-loop` (and so `video-set`) now finds neighbour cells of a cut that show the same picture and, by default, makes them again with RIFE's frames between. A cut with no such cell comes out byte for byte as before. A cut with them comes out with its held cells made, its length, beat, first and last cells and wrap unchanged. `--fill-held off` cuts as before. Every cut's report and strip metadata gain `held_fill`.**
+
+- **New: held cells found on every cut.** A frame shown twice or three times in a row (a model that stalls, a rate carried to another by repeating frames, a cut mapped to a higher rate by taking the nearer frame) is read on the analysis thumbnail the cut is chosen by. A step where no pixel changes by more than 0.1, but for at most 0.5 % of the subject's pixels, is the same picture. The report and the strip metadata record `held_fill`: the cells, the poses (the cells less those showing the picture before them again), and the held runs (first cell, how many), before and after. `video-loop` prints `poses/cells` in its summary and `video-set` puts `loop.poses` on each item.
+- **New: held cells filled by default (`--fill-held auto`).** Each run's cells are made between the run's first cell and the next pose at their own times (a run of three a third and two thirds of the way), judged as `video-cycle-align`'s frames are (smear, outline, ghost, hole, doubled) and for a part standing still on both sides (a foot planted, a hand on a ledge: its outline must stand where it stood, the fault `contact`). A frame refused is made again at a nearby fraction, else the run stays as filmed with a warning. Kept as filmed: a run longer than three cells (a hold the motion means), the opening and closing pose of a cut played once, a loop's last cells (their next pose is across the wrap), and every run of a cut drawn on twos or threes. In that case, its held-drawings reading (`cycle_drawings`, as `video-cycle-align` reads it) holds 2 or 3 and every run is that long (`held_fill.drawn`): limited animation, as filmed. A clip carried to a higher rate by repeating frames reads a hold too, but its runs are two and three cells by turns, and it is filled. With no RIFE the cells are delivered as filmed, never a failure, and a warning names the motion and `sprite-gen rife install`.
+- **New: `video-loop --out-fps F` and `video-set --out-fps F`.** A cut given above its clip's rate is made of frames between at each cell's own time, never frames shown again: a loop keeps its cycle's seconds, a cut played once runs from its first frame to its last. The strip metadata says `out_fps` and `source_fps`, and `source_cut.times` in place of `samples`. With no RIFE the cut is delivered at its own rate with a warning. `video-cycle-align` and `video-stop` refuse such a loop; `video-set` refuses it with the cycle alignment on (pass `--align-cycles off`).
+- A cut whose cells were made is read by the seam gate on its rendered cells, is not restored from its source frames (`video-loop-repair`: `source-processing-unsupported`), and `video-cycle-align` reads its drawings on its own cells.
+- Python callers: `sprite_gen.video.held_fill` (`same_steps`, `runs`, `reading`, `fill`, `upsample`, `times`, `still_kept`, `judge`, `public`, `said`); `loop.run_loop(..., fill_held=, out_fps=)`; `batch.run_set(..., fill_held=, out_fps=)`.
+- **Docs**: [loop-repair.md](docs/loop-repair.md) section 7 (the reading, the fill, the higher rate, the paths that give frames at a higher rate, what it takes from sprite-gen); [video-pipeline.md](docs/video-pipeline.md) section 4 (the default step: find, fill, read again; poses over cells in one line); `SKILL.md`.
+- **Tests**: `tests/video/test_held_fill.py` (synthetic clips and a stand-in interpolator that draws the clip again at the time asked: six cuts with no held cell pinned to their bytes on the engine before the fill; a stalled walk filled at a half, a third and two thirds, every other cell as filmed; `--fill-held off` byte for byte as before; no RIFE delivered as filmed with the motion named; a hold of five kept; a walk drawn on twos kept as filmed, byte for byte as `--fill-held off`; a walk carried from 24 to 60 fps by repeating frames filled to 60 poses in 60 cells; a once and a one-shot cut filled; a planted foot kept still and a dragged one refused; the ends of a cut played once and a loop's last cells kept; codec-like copies the same picture and a pixel's move not; a walk at 60 fps made of 60 poses in 60 cells; a one-shot at 60 fps keeping its first and last frames; no RIFE at its own rate; a rate under the clip's refused; the switches through `video-loop` and `video-set`; restoration, the alignment and a stop refusing what they cannot read).
+
## v3.7.0 - Breathe row width and height minimum and maximum sizing, and curator breathe editor range inputs
**Upgrading from 3.6.0: a Breathe row's breath can now be sized by the two ends it travels between, across and up, instead of a strength (`width_min`, `width_max`, `height_min`, `height_max`; flags `--width-min`, `--width-max`, `--height-min`, `--height-max`). Every existing sidecar, flag and output keeps its bytes, and so does every normalized sidecar; the curator's breathe editor shows four range inputs in place of the two depth selects.**
diff --git a/SKILL.md b/SKILL.md
index 890f4d0..3fda0be 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -130,6 +130,8 @@ Use existing automatic pipeline stages for background removal, extraction, align
For attack repeat coverage, observed one-shot returns and structured loop failure reports, follow [video-pipeline](docs/video-pipeline.md#one-shot-actions----cycle-autoperiodicone-shot).
+Every cut is read for held cells (neighbour cells showing the same picture), and by default they are filled with RIFE's frames between and the cut read again (`video-loop --fill-held auto`; `off` cuts as filmed). Tell the user each motion's poses over cells in one line from `held_fill` (`poses/cells`, before and after, e.g. `24/27` to `27/27`). A cut wanted above its clip's rate is given with `--out-fps`, frames between, never repeats. Details in [loop repair](docs/loop-repair.md#7-held-cells-a-picture-shown-again-and-a-cut-at-a-higher-rate).
+
A pinned idle or attack whose last frame the video model drew again has its wrap blended by default (`video-loop --wrap-blend auto`, a redraw only; `off` cuts as filmed), with details in [loop repair](docs/loop-repair.md#6-the-redrawn-wrap-a-pinned-clip-whose-last-frame-the-model-drew-again).
Scene creation consumes finished assets and remains optional. Asset metadata owns frames, native durations and anchors; scene specs own placement, scale, playback rate, planes, camera and light. Measure stride only with declared same-foot contact and an isolated foot ROI; unknown contact stays unverified. Apply only a verified report for the exact selected asset with an explicit scene direction. Never infer walking direction from the bottommost silhouette, reverse frames or change source assets to make a scene work.
@@ -140,7 +142,7 @@ Scene creation consumes finished assets and remains optional. Asset metadata own
## RIFE for walk and run loops
-Without RIFE, tell the user which loops went unrepaired and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).
+Without RIFE, tell the user which loops went unrepaired and which motions' held cells were not filled (`held_fill.unmade`, or `--out-fps` not met), and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).
## Contracts and advanced tools
diff --git a/docs/curation.md b/docs/curation.md
index 8dc27ec..6bd54b9 100644
--- a/docs/curation.md
+++ b/docs/curation.md
@@ -60,6 +60,8 @@ schema: [`recolor.md`](recolor.md).
The webview UI is bilingual (English / Korean). Pass `--lang en|ko` to match the user's language (it is also toggleable in the app); default is `en`. For isometric sets imported with `--pngs-dir`, a sibling `meta.json` tile/anchor adds a ground-grid overlay for aligning furniture with the shear handle.
+A video-loop `cycle/` imported with `--pngs-dir` labels each card of a cell the held fill made `(made)` and of a cell still showing the picture before it `(held)` ([loop-repair](loop-repair.md#7-held-cells-a-picture-shown-again-and-a-cut-at-a-higher-rate)).
+
## Editing a finished sprite sheet (no `frames/` source)
When only the combined sheet survives (a deployed asset whose run dir is gone), rebuild a curator-ready run dir with the inverse step before curating:
diff --git a/docs/loop-repair.md b/docs/loop-repair.md
index 7d7b491..8fd5510 100644
--- a/docs/loop-repair.md
+++ b/docs/loop-repair.md
@@ -1,6 +1,6 @@
# Loop repair — RIFE in-betweens for jump frames, the jolt index, one cycle per set
-> Owns: Where RIFE runs and what it costs, which model and augmentation it runs with, its two backends (the binary and PyTorch, the latter in this process or on a GPU elsewhere), how it is installed and what runs without it, jump-frame repair in `video-loop`, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the part pop (a hand or a foot that jumps at the wrap), the seam bridge (the wrap of a loop that is not a walk), the jolt index and its gate, cycle alignment across a direction set, the redrawn wrap of a pinned clip (`--wrap-blend`) · Index: [docs/README.md](README.md)
+> Owns: Where RIFE runs and what it costs, which model and augmentation it runs with, its two backends (the binary and PyTorch, the latter in this process or on a GPU elsewhere), how it is installed and what runs without it, jump-frame repair in `video-loop`, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the part pop (a hand or a foot that jumps at the wrap), the seam bridge (the wrap of a loop that is not a walk), the jolt index and its gate, cycle alignment across a direction set, the redrawn wrap of a pinned clip (`--wrap-blend`), held cells (a picture shown again) and a cut given above its clip's rate (`--fill-held`, `--out-fps`) · Index: [docs/README.md](README.md)
A walk loop cut from a generated clip can look right in every still and still hitch when it
plays: the video model redraws thin hair a little differently every frame, and now and then
@@ -2746,6 +2746,99 @@ The loop report records `wrap_blend` on every pinned cut: `applied`, `why`, `K`,
A `video-loop: warning:` line is emitted under two conditions. When the after check refuses the blend because a step into or through the ramp exceeds the bound or the wrap does not shrink, the cut is written as filmed, `wrap_blend.after_check` is set to "refused", and the warning states that the wrap blend was not used, why, and the measured numbers. When the wrap remains over `--seam-max` after blending because the two ends hold a part apart, the blend is applied anyway and a warning line reports that condition along with the share of the figure held as filmed.
+## 7. Held cells: a picture shown again, and a cut at a higher rate
+
+A clip can show one frame two or three times in a row: a video model that stalls, a rate carried to another by
+repeating frames (20 fps carried at 24 repeats one frame in six), or a cut mapped to a higher rate by taking the
+nearer frame. Played, the motion stops on that picture and jumps on. `video-loop` reads every cut it makes for held
+cells (loops that are periodic, fixed, or pinned; one-shot, once, and once-return cuts; and so every cut `video-set`
+makes and every clip of `video` cut through `video-frames` and `video-loop`). By default, it makes those cells again
+as the frames between the pose before them and the pose after (`sprite_gen/video/held_fill.py`). The default step
+of the pipeline is: find the held cells, fill them, read the cut again; the cut's line to the person is its poses
+over its cells.
+
+### The reading
+
+| Rule | Value | Why |
+|---|---|---|
+| The same picture | A step no pixel of which, on the analysis thumbnail the cut is chosen by (96 px on the longer side, premultiplied RGBA), changes by more than `PIXEL_SAME` 0.1, but for at most `CHANGED_MAX` 0.5 % of the subject's pixels | A textured synthetic figure, each pose shown three times, encoded by libx264 at crf 18, 23 and 28 and keyed by `video-frames`: a frame shown again changes no pixel by more than 0.008, 0.020 and 0.051 (no pixel over 0.1). An arm 8 px thick on a 120 x 140 figure grown by one pixel changes 1.2 % of the subject's pixels over 0.1. |
+| Not a distance over the mass | `nonloop.STABLE_MAX` (0.025), the once cut's settled end, is not the rule | The arm's pixel reads 0.003 of the subject's mass: under it, so the rule would call it held. |
+| Where read | On the cut's frames as loaded, before an anchor moves each frame apart, and again on the cells delivered | An anchor shifts each frame of a pair by its own amount: a picture shown again would no longer read as one. |
+| Not read across the wrap | A loop's last cell and its first are not a run | The wrap is the seam gate's, not a hold. |
+| A sub-thumbnail move | A small part moved a fraction of a thumbnail pixel reads as the same picture | Two or three pixels of a hand on a large frame play as a hold; made again between its neighbours it plays the same. |
+
+The record, `held_fill` in the loop report and in the strip metadata (the strip's without each try): `cells`,
+`poses` (the cells less the ones showing the picture before them) and `held_runs` (each run's first cell and how
+many) of the cells as delivered; `before`, the same of the cut as filmed with each run `filled` or `why` not (and
+the `faults` its frames between were refused for); `made`, the cells made; `unmade`, the cells left for want of
+RIFE with the install line; `out_fps`, the rate asked (below). `video-loop` prints `held_fill.poses_cells` (`27/27`)
+and `before` in its summary; `video-set` puts `loop.poses` on each item. In the curation view (a loop's `cycle/`
+imported with `unpack-atlas --pngs-dir`), the card of each cell made says `(made)` and of each cell still showing the
+picture before it `(held)`. A stop (`video-stop`) is read the same way and records `held_fill` with `mode` off: it
+plays the clip's filmed frames, none made between two others, and names a run it shows in a warning.
+
+### The fill: `--fill-held auto` (default), `off`
+
+| Rule | Value | Why |
+|---|---|---|
+| A run's cells made | Cell c of a run from cell a to the next pose b, at t = (c - a)/(b - a) between a and b: a run of two half way, of three a third and two thirds | The cut keeps its length, its beat, its first and last cells and its wrap: a made cell stands where a held cell stood, on its own time. |
+| Kept: a hold the motion means | A run longer than `HOLD_MAX` 3 cells | A drawing held on threes is three frames ([held.py](../sprite_gen/video/held.py)); longer is a pause the motion makes. A shorter pause meant as one is filled: `--fill-held off` keeps it. |
+| Kept: a cut drawn on twos or threes (`drawn`) | Every run, where the cut's held-drawings reading (`cycle_drawings`, else the clip's `drawings`: [held.py](../sprite_gen/video/held.py), which `video-cycle-align` reads) gives a hold of 2 or 3 and every run is that many cells, but one cut short at the cut's first or last cell | Limited animation: each drawing shown as long as the next, played at its own rate as filmed (section 4). A clip carried to a higher rate by repeating frames reads a hold too (a synthetic walk at 24 fps carried to 60 by the frame under each time reads a hold of 2), but its runs are two and three cells by turns: it is filled (60 poses over 60 cells). A clip that stalls now and then reads no hold (hold 1). |
+| Kept: the ends of a cut played once | A run that opens or closes a one-shot, once or once-return cut | The rest before the action and the pose it ends on are the cut's. |
+| Kept: a loop's last cells | A run that ends on a loop's last cell | Its next pose is across the wrap: made, the last cell would change and with it the wrap. A run that opens a loop is made (its first cell stays). A periodic cut chooses its wrap where the steps either side read alike, so a held run seldom closes it; a fixed cut whose last cells are held keeps them, and its wrap then steps from a held picture (cut it elsewhere, or `--fill-held off` to keep the whole cut as filmed). |
+| A frame made, judged | `align.measure_made` and `interpolation_quality.faults` (smear, outline, ghost, hole, doubled), the alignment's own | One acceptance policy for every frame RIFE makes (section 4). |
+| A part standing still, kept still (`still_kept`) | The outline pixels both frames draw in one place in one colour (solid at alpha 128, within 24 levels) must be drawn there in the frame made, within a pixel, for at least `STILL_KEPT_MIN` 0.8 of them in every window a tenth of the body high that holds 12 or more; else the fault `contact` | A foot planted or a hand on a ledge stays where it stood. A synthetic in-between that drags a planted foot 3 px along the floor keeps 0.50 of its window; one off by a pixel, 1.00. |
+| A frame refused | Made again at the fraction of `RETIME_T` nearest its own within half a cell of it, its ghost band under `RETIME_WINDOW_MAX` (section 4's bar for a frame made off its time); else the run is kept as filmed, with a warning naming the cells and the faults | A frame half way that RIFE cannot draw is often clean nearer one side. |
+| Whole or not at all | A run is made whole or kept whole | Half made, it would step back to the held picture and on again. |
+| Order | After the anchor and `source_projection`, before the ghost screen, the wrap blend, the part pop and the jump repair | Everything that reads the flow and the wrap reads the cut as it will play. |
+| No RIFE | The held cells are delivered as filmed, the cut passes, and a `warning:` line names the motion, the cells and `sprite-gen rife install`; `held_fill.unmade` records them | Never a failure, never silent. |
+| `--fill-held off` | Cut as filmed, byte for byte as before; the reading is recorded all the same | |
+| Byte for byte | A cut with no held cell is delivered as before: six synthetic cuts (a fixed and a periodic walk, a one-shot hop, a pinned swing, a once fall, a once-return swing) are pinned to their bytes on the engine before the fill, and the strip, the GIF, the WebP and `cycle/` match (`tests/video/test_held_fill.py`) | |
+
+A cut whose cells were made is read by the seam gate on its rendered cells (`seam_measurement`), restoration from
+the source frames (`video-loop-repair`) reads it as `source-processing-unsupported`, and `video-cycle-align` reads
+its drawings on its own cells, not on the clip's hold.
+
+### A rate above the clip's: `--out-fps F`
+
+`video-loop --out-fps F` (and `video-set --out-fps F`) gives the cut at F, above the clip's `--fps`: every cell at a
+time of its own, made by RIFE between the frames beside it, judged as the fill's frames are, never showing a frame
+again. A loop has round(L x F / fps) cells over its cycle's seconds, at times k x L / cells; a cut played once runs
+from its first frame to its last, round((L - 1) x F / fps) + 1 cells, each 1/F. A cell on a frame's time is that
+frame; one between two frames showing one picture (a hold kept) is that picture; one no frame made is taken for is
+the nearer frame, named (`out_fps.shown_again`). `cycle/` holds every cell; the strip holds them up to its cell and
+width caps, subsampled past them as any strip is (`subsampled`). The strip metadata says `out_fps` and `source_fps`,
+and `source_cut.times` in place of `samples`. With no RIFE the cut is delivered at its own rate with a `warning:` line
+and `out_fps.applied` false. `video-cycle-align` and `video-stop` refuse such a loop (each makes its own frames
+between from the cut at the clip's rate), and restoration reads it as `source-processing-unsupported`.
+
+Where sprite-gen gives frames at a higher rate than it was given them:
+
+| Path | Before | Now |
+|---|---|---|
+| `video-loop --gif-fps`, `--n-out` | The GIF and WebP never hold more frames than the cut (a higher request is capped at the cut's frames, never repeated) | The same; `--out-fps` makes the cells themselves at the higher rate |
+| `video-cycle-align --length` | RIFE's frames between; a frame shown again only with `--between nearest`, or where a frame made is refused and the nearer source frame is taken (named) | Unchanged: it already makes frames between |
+| `video-loop --out-fps` (new) | None | RIFE's frames between, never a frame shown again |
+| `scene-render` at a scene rate above an asset's | A cell of the asset shown over several scene frames | Unchanged: give the asset at the scene's rate (`--out-fps`) |
+| `video-stop --fps` | The rate of the frames given, not an output rate | Unchanged |
+
+### What it takes from sprite-gen
+
+| Part | Taken from | Why a part is its own |
+|---|---|---|
+| The thumbnail the steps are read on | `loop._small_features`, the analysis thumbnail the cut is chosen by | |
+| What a drawing held on threes is | `held.HOLDS` | |
+| Whether the cut is drawn on twos or threes | `held.measure_cycle` (`cycle_drawings`), `held.measure` (`drawings`) | |
+| The judgement of a frame made | `align.measure_made`, `interpolation_quality.faults`, `looks` | |
+| A frame made again at another fraction | `align.RETIME_T`, `align.RETIME_WINDOW_MAX` | |
+| RIFE located only once a frame is made; no RIFE as a warning | The pattern of `_repair_jumps` and `_bridge_seam` | |
+| The ghost screen's reading | `interpolation_quality.ghost_screen` | |
+| Solid pixels | `legs.LOW_ALPHA` | |
+| The window a tenth of the body high | `rife.GHOST_BAND` | |
+| The same picture (`same_steps`) | New | A distance over the mass (`nonloop.STABLE_MAX`, `held.measure`'s steps) reads a one-pixel move of a small part as held; `held.measure` reads a rhythm of redrawn drawings, not a frame shown again. |
+| A part standing still (`still_kept`) | New | `rife.hole` reads a hole through a limb, `inspect-motion` a foot's place in spans a person marks; neither reads whether an in-between keeps what both its frames hold still. |
+| The times of a higher rate (`times`, `upsample`) | New | `align.resample` resamples a loop as a ring to a length; a cut played once runs from its first frame to its last. |
+
## Related
- [video-pipeline.md](video-pipeline.md) — the pipeline these repairs run inside
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 90ca0b7..ded718c 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -401,6 +401,19 @@ them fails rather than being written.
See [loop review](loop-review.md)
for the visual review contract and manual overrides.
+**Held cells: found, filled, read again: the default step of every cut.** A cut whose neighbour cells show the same
+picture (a frame shown twice or three times: a model that stalls, a rate carried to another by repeating frames)
+stops on it as it plays. `video-loop` reads each cut it makes for them (loop, one-shot, once, once-return, so every
+cut of `video-set` and of a clip from `video`) and by default makes each run of them again with RIFE's frames between
+the pose before and the pose after (`--fill-held auto`), the cut's length, beat, first and last cells and wrap kept,
+a part standing still in both kept still; a run longer than three cells, the ends of a cut played once, a loop's
+last cells and a cut drawn on twos or threes (limited animation, `held_fill.drawn`) stay as filmed. Then it reads the
+cut again. The report and the strip metadata record the held runs (first cell, how many) and the poses over the
+cells (`held_fill`); tell the person that one line: `poses/cells`, e.g. `24/27` before and `27/27` after. With no
+RIFE the cells are delivered as filmed and a warning names the motion and the install line; `--fill-held off` cuts
+as before. A cut asked for above its clip's rate (`--out-fps 60` on a 24 fps clip) is made of frames between, never
+frames shown again ([loop repair](loop-repair.md) section 7).
+
**The fundamental period: recorded, never cut on the pixels' word.** A motion that repeats every
P frames repeats every 2P as well, and 2P can be the better repeat: a clip drawn on twos (a new
drawing every other frame) whose cycle is an odd number of frames shows each cycle half a drawing