Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
docs(video-loop): held cells found, filled and read again as the default step of every cut, and --out-fps, in loop-repair section 7, video-pipeline, curation, SKILL.md and the CHANGELOG's Unreleased
Soohong Kim136db9e0206e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 20b1254..cddf8e7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,18 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## Unreleased (v3.8.0)++**Upgrading from 3.7.0: `video-loop` (and so `video-set`) now finds neighbour cells of a cut that show the same picture and, by default, makes them again with RIFE's frames between. A cut with no such cell comes out byte for byte as before. A cut with them comes out with its held cells made, its length, beat, first and last cells and wrap unchanged. `--fill-held off` cuts as before. Every cut's report and strip metadata gain `held_fill`.**++- **New: held cells found on every cut.** A frame shown twice or three times in a row (a model that stalls, a rate carried to another by repeating frames, a cut mapped to a higher rate by taking the nearer frame) is read on the analysis thumbnail the cut is chosen by. A step where no pixel changes by more than 0.1, but for at most 0.5 % of the subject's pixels, is the same picture. The report and the strip metadata record `held_fill`: the cells, the poses (the cells less those showing the picture before them again), and the held runs (first cell, how many), before and after. `video-loop` prints `poses/cells` in its summary and `video-set` puts `loop.poses` on each item.+- **New: held cells filled by default (`--fill-held auto`).** Each run's cells are made between the run's first cell and the next pose at their own times (a run of three a third and two thirds of the way), judged as `video-cycle-align`'s frames are (smear, outline, ghost, hole, doubled) and for a part standing still on both sides (a foot planted, a hand on a ledge: its outline must stand where it stood, the fault `contact`). A frame refused is made again at a nearby fraction, else the run stays as filmed with a warning. Kept as filmed: a run longer than three cells (a hold the motion means), the opening and closing pose of a cut played once, a loop's last cells (their next pose is across the wrap), and every run of a cut drawn on twos or threes. In that case, its held-drawings reading (`cycle_drawings`, as `video-cycle-align` reads it) holds 2 or 3 and every run is that long (`held_fill.drawn`): limited animation, as filmed. A clip carried to a higher rate by repeating frames reads a hold too, but its runs are two and three cells by turns, and it is filled. With no RIFE the cells are delivered as filmed, never a failure, and a warning names the motion and `sprite-gen rife install`.+- **New: `video-loop --out-fps F` and `video-set --out-fps F`.** A cut given above its clip's rate is made of frames between at each cell's own time, never frames shown again: a loop keeps its cycle's seconds, a cut played once runs from its first frame to its last. The strip metadata says `out_fps` and `source_fps`, and `source_cut.times` in place of `samples`. With no RIFE the cut is delivered at its own rate with a warning. `video-cycle-align` and `video-stop` refuse such a loop; `video-set` refuses it with the cycle alignment on (pass `--align-cycles off`).+- A cut whose cells were made is read by the seam gate on its rendered cells, is not restored from its source frames (`video-loop-repair`: `source-processing-unsupported`), and `video-cycle-align` reads its drawings on its own cells.+- Python callers: `sprite_gen.video.held_fill` (`same_steps`, `runs`, `reading`, `fill`, `upsample`, `times`, `still_kept`, `judge`, `public`, `said`); `loop.run_loop(..., fill_held=, out_fps=)`; `batch.run_set(..., fill_held=, out_fps=)`.+- **Docs**: [loop-repair.md](docs/loop-repair.md) section 7 (the reading, the fill, the higher rate, the paths that give frames at a higher rate, what it takes from sprite-gen); [video-pipeline.md](docs/video-pipeline.md) section 4 (the default step: find, fill, read again; poses over cells in one line); `SKILL.md`.+- **Tests**: `tests/video/test_held_fill.py` (synthetic clips and a stand-in interpolator that draws the clip again at the time asked: six cuts with no held cell pinned to their bytes on the engine before the fill; a stalled walk filled at a half, a third and two thirds, every other cell as filmed; `--fill-held off` byte for byte as before; no RIFE delivered as filmed with the motion named; a hold of five kept; a walk drawn on twos kept as filmed, byte for byte as `--fill-held off`; a walk carried from 24 to 60 fps by repeating frames filled to 60 poses in 60 cells; a once and a one-shot cut filled; a planted foot kept still and a dragged one refused; the ends of a cut played once and a loop's last cells kept; codec-like copies the same picture and a pixel's move not; a walk at 60 fps made of 60 poses in 60 cells; a one-shot at 60 fps keeping its first and last frames; no RIFE at its own rate; a rate under the clip's refused; the switches through `video-loop` and `video-set`; restoration, the alignment and a stop refusing what they cannot read).+## v3.7.0 - Breathe row width and height minimum and maximum sizing, and curator breathe editor range inputs**Upgrading from 3.6.0: a Breathe row's breath can now be sized by the two ends it travels between, across and up, instead of a strength (`width_min`, `width_max`, `height_min`, `height_max`; flags `--width-min`, `--width-max`, `--height-min`, `--height-max`). Every existing sidecar, flag and output keeps its bytes, and so does every normalized sidecar; the curator's breathe editor shows four range inputs in place of the two depth selects.**diff --git a/SKILL.md b/SKILL.mdindex 890f4d0..3fda0be 100644--- a/SKILL.md+++ b/SKILL.md@@ -130,6 +130,8 @@ Use existing automatic pipeline stages for background removal, extraction, alignFor attack repeat coverage, observed one-shot returns and structured loop failure reports, follow [video-pipeline](docs/video-pipeline.md#one-shot-actions----cycle-autoperiodicone-shot).+Every cut is read for held cells (neighbour cells showing the same picture), and by default they are filled with RIFE's frames between and the cut read again (`video-loop --fill-held auto`; `off` cuts as filmed). Tell the user each motion's poses over cells in one line from `held_fill` (`poses/cells`, before and after, e.g. `24/27` to `27/27`). A cut wanted above its clip's rate is given with `--out-fps`, frames between, never repeats. Details in [loop repair](docs/loop-repair.md#7-held-cells-a-picture-shown-again-and-a-cut-at-a-higher-rate).+A pinned idle or attack whose last frame the video model drew again has its wrap blended by default (`video-loop --wrap-blend auto`, a redraw only; `off` cuts as filmed), with details in [loop repair](docs/loop-repair.md#6-the-redrawn-wrap-a-pinned-clip-whose-last-frame-the-model-drew-again).Scene creation consumes finished assets and remains optional. Asset metadata owns frames, native durations and anchors; scene specs own placement, scale, playback rate, planes, camera and light. Measure stride only with declared same-foot contact and an isolated foot ROI; unknown contact stays unverified. Apply only a verified report for the exact selected asset with an explicit scene direction. Never infer walking direction from the bottommost silhouette, reverse frames or change source assets to make a scene work.@@ -140,7 +142,7 @@ Scene creation consumes finished assets and remains optional. Asset metadata own## RIFE for walk and run loops-Without RIFE, tell the user which loops went unrepaired and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).+Without RIFE, tell the user which loops went unrepaired and which motions' held cells were not filled (`held_fill.unmade`, or `--out-fps` not met), and offer the install; after it, cut those loops again and run `video-cycle-align`. [loop-repair](docs/loop-repair.md).## Contracts and advanced toolsdiff --git a/docs/curation.md b/docs/curation.mdindex 8dc27ec..6bd54b9 100644--- a/docs/curation.md+++ b/docs/curation.md@@ -60,6 +60,8 @@ schema: [`recolor.md`](recolor.md).The webview UI is bilingual (English / Korean). Pass `--lang en|ko` to match the user's language (it is also toggleable in the app); default is `en`. For isometric sets imported with `--pngs-dir`, a sibling `meta.json` tile/anchor adds a ground-grid overlay for aligning furniture with the shear handle.+A video-loop `cycle/` imported with `--pngs-dir` labels each card of a cell the held fill made `(made)` and of a cell still showing the picture before it `(held)` ([loop-repair](loop-repair.md#7-held-cells-a-picture-shown-again-and-a-cut-at-a-higher-rate)).+## Editing a finished sprite sheet (no `frames/` source)When only the combined sheet survives (a deployed asset whose run dir is gone), rebuild a curator-ready run dir with the inverse step before curating:diff --git a/docs/loop-repair.md b/docs/loop-repair.mdindex 7d7b491..8fd5510 100644--- a/docs/loop-repair.md+++ b/docs/loop-repair.md@@ -1,6 +1,6 @@# Loop repair — RIFE in-betweens for jump frames, the jolt index, one cycle per set-> Owns: Where RIFE runs and what it costs, which model and augmentation it runs with, its two backends (the binary and PyTorch, the latter in this process or on a GPU elsewhere), how it is installed and what runs without it, jump-frame repair in `video-loop`, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the part pop (a hand or a foot that jumps at the wrap), the seam bridge (the wrap of a loop that is not a walk), the jolt index and its gate, cycle alignment across a direction set, the redrawn wrap of a pinned clip (`--wrap-blend`) · Index: [docs/README.md](README.md)+> Owns: Where RIFE runs and what it costs, which model and augmentation it runs with, its two backends (the binary and PyTorch, the latter in this process or on a GPU elsewhere), how it is installed and what runs without it, jump-frame repair in `video-loop`, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the part pop (a hand or a foot that jumps at the wrap), the seam bridge (the wrap of a loop that is not a walk), the jolt index and its gate, cycle alignment across a direction set, the redrawn wrap of a pinned clip (`--wrap-blend`), held cells (a picture shown again) and a cut given above its clip's rate (`--fill-held`, `--out-fps`) · Index: [docs/README.md](README.md)A walk loop cut from a generated clip can look right in every still and still hitch when itplays: the video model redraws thin hair a little differently every frame, and now and then@@ -2746,6 +2746,99 @@ The loop report records `wrap_blend` on every pinned cut: `applied`, `why`, `K`,A `video-loop: warning:` line is emitted under two conditions. When the after check refuses the blend because a step into or through the ramp exceeds the bound or the wrap does not shrink, the cut is written as filmed, `wrap_blend.after_check` is set to "refused", and the warning states that the wrap blend was not used, why, and the measured numbers. When the wrap remains over `--seam-max` after blending because the two ends hold a part apart, the blend is applied anyway and a warning line reports that condition along with the share of the figure held as filmed.+## 7. Held cells: a picture shown again, and a cut at a higher rate++A clip can show one frame two or three times in a row: a video model that stalls, a rate carried to another by+repeating frames (20 fps carried at 24 repeats one frame in six), or a cut mapped to a higher rate by taking the+nearer frame. Played, the motion stops on that picture and jumps on. `video-loop` reads every cut it makes for held+cells (loops that are periodic, fixed, or pinned; one-shot, once, and once-return cuts; and so every cut `video-set`+makes and every clip of `video` cut through `video-frames` and `video-loop`). By default, it makes those cells again+as the frames between the pose before them and the pose after (`sprite_gen/video/held_fill.py`). The default step+of the pipeline is: find the held cells, fill them, read the cut again; the cut's line to the person is its poses+over its cells.++### The reading++| Rule | Value | Why |+|---|---|---|+| The same picture | A step no pixel of which, on the analysis thumbnail the cut is chosen by (96 px on the longer side, premultiplied RGBA), changes by more than `PIXEL_SAME` 0.1, but for at most `CHANGED_MAX` 0.5 % of the subject's pixels | A textured synthetic figure, each pose shown three times, encoded by libx264 at crf 18, 23 and 28 and keyed by `video-frames`: a frame shown again changes no pixel by more than 0.008, 0.020 and 0.051 (no pixel over 0.1). An arm 8 px thick on a 120 x 140 figure grown by one pixel changes 1.2 % of the subject's pixels over 0.1. |+| Not a distance over the mass | `nonloop.STABLE_MAX` (0.025), the once cut's settled end, is not the rule | The arm's pixel reads 0.003 of the subject's mass: under it, so the rule would call it held. |+| Where read | On the cut's frames as loaded, before an anchor moves each frame apart, and again on the cells delivered | An anchor shifts each frame of a pair by its own amount: a picture shown again would no longer read as one. |+| Not read across the wrap | A loop's last cell and its first are not a run | The wrap is the seam gate's, not a hold. |+| A sub-thumbnail move | A small part moved a fraction of a thumbnail pixel reads as the same picture | Two or three pixels of a hand on a large frame play as a hold; made again between its neighbours it plays the same. |++The record, `held_fill` in the loop report and in the strip metadata (the strip's without each try): `cells`,+`poses` (the cells less the ones showing the picture before them) and `held_runs` (each run's first cell and how+many) of the cells as delivered; `before`, the same of the cut as filmed with each run `filled` or `why` not (and+the `faults` its frames between were refused for); `made`, the cells made; `unmade`, the cells left for want of+RIFE with the install line; `out_fps`, the rate asked (below). `video-loop` prints `held_fill.poses_cells` (`27/27`)+and `before` in its summary; `video-set` puts `loop.poses` on each item. In the curation view (a loop's `cycle/`+imported with `unpack-atlas --pngs-dir`), the card of each cell made says `(made)` and of each cell still showing the+picture before it `(held)`. A stop (`video-stop`) is read the same way and records `held_fill` with `mode` off: it+plays the clip's filmed frames, none made between two others, and names a run it shows in a warning.++### The fill: `--fill-held auto` (default), `off`++| Rule | Value | Why |+|---|---|---|+| A run's cells made | Cell c of a run from cell a to the next pose b, at t = (c - a)/(b - a) between a and b: a run of two half way, of three a third and two thirds | The cut keeps its length, its beat, its first and last cells and its wrap: a made cell stands where a held cell stood, on its own time. |+| Kept: a hold the motion means | A run longer than `HOLD_MAX` 3 cells | A drawing held on threes is three frames ([held.py](../sprite_gen/video/held.py)); longer is a pause the motion makes. A shorter pause meant as one is filled: `--fill-held off` keeps it. |+| Kept: a cut drawn on twos or threes (`drawn`) | Every run, where the cut's held-drawings reading (`cycle_drawings`, else the clip's `drawings`: [held.py](../sprite_gen/video/held.py), which `video-cycle-align` reads) gives a hold of 2 or 3 and every run is that many cells, but one cut short at the cut's first or last cell | Limited animation: each drawing shown as long as the next, played at its own rate as filmed (section 4). A clip carried to a higher rate by repeating frames reads a hold too (a synthetic walk at 24 fps carried to 60 by the frame under each time reads a hold of 2), but its runs are two and three cells by turns: it is filled (60 poses over 60 cells). A clip that stalls now and then reads no hold (hold 1). |+| Kept: the ends of a cut played once | A run that opens or closes a one-shot, once or once-return cut | The rest before the action and the pose it ends on are the cut's. |+| Kept: a loop's last cells | A run that ends on a loop's last cell | Its next pose is across the wrap: made, the last cell would change and with it the wrap. A run that opens a loop is made (its first cell stays). A periodic cut chooses its wrap where the steps either side read alike, so a held run seldom closes it; a fixed cut whose last cells are held keeps them, and its wrap then steps from a held picture (cut it elsewhere, or `--fill-held off` to keep the whole cut as filmed). |+| A frame made, judged | `align.measure_made` and `interpolation_quality.faults` (smear, outline, ghost, hole, doubled), the alignment's own | One acceptance policy for every frame RIFE makes (section 4). |+| A part standing still, kept still (`still_kept`) | The outline pixels both frames draw in one place in one colour (solid at alpha 128, within 24 levels) must be drawn there in the frame made, within a pixel, for at least `STILL_KEPT_MIN` 0.8 of them in every window a tenth of the body high that holds 12 or more; else the fault `contact` | A foot planted or a hand on a ledge stays where it stood. A synthetic in-between that drags a planted foot 3 px along the floor keeps 0.50 of its window; one off by a pixel, 1.00. |+| A frame refused | Made again at the fraction of `RETIME_T` nearest its own within half a cell of it, its ghost band under `RETIME_WINDOW_MAX` (section 4's bar for a frame made off its time); else the run is kept as filmed, with a warning naming the cells and the faults | A frame half way that RIFE cannot draw is often clean nearer one side. |+| Whole or not at all | A run is made whole or kept whole | Half made, it would step back to the held picture and on again. |+| Order | After the anchor and `source_projection`, before the ghost screen, the wrap blend, the part pop and the jump repair | Everything that reads the flow and the wrap reads the cut as it will play. |+| No RIFE | The held cells are delivered as filmed, the cut passes, and a `warning:` line names the motion, the cells and `sprite-gen rife install`; `held_fill.unmade` records them | Never a failure, never silent. |+| `--fill-held off` | Cut as filmed, byte for byte as before; the reading is recorded all the same | |+| Byte for byte | A cut with no held cell is delivered as before: six synthetic cuts (a fixed and a periodic walk, a one-shot hop, a pinned swing, a once fall, a once-return swing) are pinned to their bytes on the engine before the fill, and the strip, the GIF, the WebP and `cycle/` match (`tests/video/test_held_fill.py`) | |++A cut whose cells were made is read by the seam gate on its rendered cells (`seam_measurement`), restoration from+the source frames (`video-loop-repair`) reads it as `source-processing-unsupported`, and `video-cycle-align` reads+its drawings on its own cells, not on the clip's hold.++### A rate above the clip's: `--out-fps F`++`video-loop --out-fps F` (and `video-set --out-fps F`) gives the cut at F, above the clip's `--fps`: every cell at a+time of its own, made by RIFE between the frames beside it, judged as the fill's frames are, never showing a frame+again. A loop has round(L x F / fps) cells over its cycle's seconds, at times k x L / cells; a cut played once runs+from its first frame to its last, round((L - 1) x F / fps) + 1 cells, each 1/F. A cell on a frame's time is that+frame; one between two frames showing one picture (a hold kept) is that picture; one no frame made is taken for is+the nearer frame, named (`out_fps.shown_again`). `cycle/` holds every cell; the strip holds them up to its cell and+width caps, subsampled past them as any strip is (`subsampled`). The strip metadata says `out_fps` and `source_fps`,+and `source_cut.times` in place of `samples`. With no RIFE the cut is delivered at its own rate with a `warning:` line+and `out_fps.applied` false. `video-cycle-align` and `video-stop` refuse such a loop (each makes its own frames+between from the cut at the clip's rate), and restoration reads it as `source-processing-unsupported`.++Where sprite-gen gives frames at a higher rate than it was given them:++| Path | Before | Now |+|---|---|---|+| `video-loop --gif-fps`, `--n-out` | The GIF and WebP never hold more frames than the cut (a higher request is capped at the cut's frames, never repeated) | The same; `--out-fps` makes the cells themselves at the higher rate |+| `video-cycle-align --length` | RIFE's frames between; a frame shown again only with `--between nearest`, or where a frame made is refused and the nearer source frame is taken (named) | Unchanged: it already makes frames between |+| `video-loop --out-fps` (new) | None | RIFE's frames between, never a frame shown again |+| `scene-render` at a scene rate above an asset's | A cell of the asset shown over several scene frames | Unchanged: give the asset at the scene's rate (`--out-fps`) |+| `video-stop --fps` | The rate of the frames given, not an output rate | Unchanged |++### What it takes from sprite-gen++| Part | Taken from | Why a part is its own |+|---|---|---|+| The thumbnail the steps are read on | `loop._small_features`, the analysis thumbnail the cut is chosen by | |+| What a drawing held on threes is | `held.HOLDS` | |+| Whether the cut is drawn on twos or threes | `held.measure_cycle` (`cycle_drawings`), `held.measure` (`drawings`) | |+| The judgement of a frame made | `align.measure_made`, `interpolation_quality.faults`, `looks` | |+| A frame made again at another fraction | `align.RETIME_T`, `align.RETIME_WINDOW_MAX` | |+| RIFE located only once a frame is made; no RIFE as a warning | The pattern of `_repair_jumps` and `_bridge_seam` | |+| The ghost screen's reading | `interpolation_quality.ghost_screen` | |+| Solid pixels | `legs.LOW_ALPHA` | |+| The window a tenth of the body high | `rife.GHOST_BAND` | |+| The same picture (`same_steps`) | New | A distance over the mass (`nonloop.STABLE_MAX`, `held.measure`'s steps) reads a one-pixel move of a small part as held; `held.measure` reads a rhythm of redrawn drawings, not a frame shown again. |+| A part standing still (`still_kept`) | New | `rife.hole` reads a hole through a limb, `inspect-motion` a foot's place in spans a person marks; neither reads whether an in-between keeps what both its frames hold still. |+| The times of a higher rate (`times`, `upsample`) | New | `align.resample` resamples a loop as a ring to a length; a cut played once runs from its first frame to its last. |+## Related- [video-pipeline.md](video-pipeline.md) — the pipeline these repairs run insidediff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 90ca0b7..ded718c 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -401,6 +401,19 @@ them fails rather than being written.See [loop review](loop-review.md)for the visual review contract and manual overrides.+**Held cells: found, filled, read again: the default step of every cut.** A cut whose neighbour cells show the same+picture (a frame shown twice or three times: a model that stalls, a rate carried to another by repeating frames)+stops on it as it plays. `video-loop` reads each cut it makes for them (loop, one-shot, once, once-return, so every+cut of `video-set` and of a clip from `video`) and by default makes each run of them again with RIFE's frames between+the pose before and the pose after (`--fill-held auto`), the cut's length, beat, first and last cells and wrap kept,+a part standing still in both kept still; a run longer than three cells, the ends of a cut played once, a loop's+last cells and a cut drawn on twos or threes (limited animation, `held_fill.drawn`) stay as filmed. Then it reads the+cut again. The report and the strip metadata record the held runs (first cell, how many) and the poses over the+cells (`held_fill`); tell the person that one line: `poses/cells`, e.g. `24/27` before and `27/27` after. With no+RIFE the cells are delivered as filmed and a warning names the motion and the install line; `--fill-held off` cuts+as before. A cut asked for above its clip's rate (`--out-fps 60` on a 24 fps clip) is made of frames between, never+frames shown again ([loop repair](loop-repair.md) section 7).+**The fundamental period: recorded, never cut on the pixels' word.** A motion that repeats everyP frames repeats every 2P as well, and 2P can be the better repeat: a clip drawn on twos (a newdrawing every other frame) whose cycle is an odd number of frames shows each cycle half a drawing