Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
docs(face): 3.1.0 — docs/face-parts.md owns the face request, the pack, the face layer's rules, the player, video-face and face-bake; CHANGELOG v3.1.0, the docs index, two SKILL.md routes and triggers, node for face-bake
Soohong Kime4e16cacdfe2
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5faa809..c085882 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,25 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## v3.1.0 - Talking face layer++**Upgrading from 3.0.0:** Existing runs and commands do not change. This release adds two verbs and a JavaScript player. The wheel now also carries `sprite_gen/video/*.mjs`.++### Added++- `sprite-gen video-face` builds a face pack from a reference, a keyed talking clip, and keyed body motions.+- `sprite-gen face-bake` runs the pack's player through node to bake a voice onto body frames, with optional MP4 and beside video output.+- Face request kind `sprite-gen-face-request`, version 1, records face geometry, clip placement, a rest loop, and optional own-mouth spans or timelines.+- Face pack kind `sprite-gen-face-pack`, version 1, carries sprites, body cells, anchors, keeps, rules, a report, a copied player, and SHA-256 digests.+- The lower-face layer covers the mouth, jaw line, cheeks, and chin. The body's own dark grey and black that reach in from the layer's boundary, such as a headset boom or a choker, the body's own lines in the layer's feather, and a hand that reaches over the face from outside the layer stay above it.+- Each body frame receives a place, size, and tilt anchor. The pack records anchor residuals and corrects outlying readings against neighbouring frames.+- Talking frames are assigned to smile or neutral sets by closed-mouth lift. Entry chooses a matching set and rest.+- The player `sprite-gen-face-v1.mjs` contains the voice flap, face pick, scene plan, and live canvas playback rules.+- A clip's own mouth and sound finish before a requested line starts. The voice and layer then wait together for the first frame whose switch to the chosen rest is within the 1.4 jump line. Layer exit also uses that jump line.+- Bakes check pack digests, changed pixels outside the weight, and composite jumps. Refused bakes retain their frames for inspection.+- See [docs/face-parts.md](docs/face-parts.md) for the request, pack, player, commands, and limits.+- **Tests**: `tests/video/test_face_layer.py` uses `tests/talking_face.py` to check smile and neutral sets; clip entry set and rest; own-mouth timelines, including a close during voiced sound and refusal of a missing or never-closing timeline; a head 60 px from its place and tilted 3 degrees; hand, headset boom, and choker keeps; unchanged pixels outside the weight; synthetic cells keeping the key's colour under no coverage; digests and new-folder pack writes; the player flap against its original numpy rule and the pick against the draft Viterbi over every frame; a line waiting for a clip's own mouth and for a frame its layer can come on at; loop and post-greeting bakes with greeting frames unchanged byte for byte; and refusal of a changed pack.+## v3.0.0 - Remove rig layer composition**Upgrading from 2.46.0: `sprite-gen compose-layers` and its other launch forms are removed, and rig layer declarations are no longer read.** `compose-atlas` no longer writes a manifest `rig` block or `animation.rows.<state>.track` for a run that declared a rig. Runs that never declared `rig`, `layers` or a state `track` get the same manifest as before. Existing run files and `layers/` folders stay on disk.diff --git a/SKILL.md b/SKILL.mdindex 582698d..61b75ff 100644--- a/SKILL.md+++ b/SKILL.md@@ -1,7 +1,7 @@---name: sprite-gen-version: 3.0.0-description: "Generates images and game sprites through GPT or Grok with guided provider choices, separate saved defaults, automatic cleanup and optional curation. Handles sprite requests, ordinary image generation/editing, typography logos (로고, 타이포그래피, 타이틀 글자, 타이틀 로고, wordmark, logo, title lettering: a title screen drawn with the title in it, its lettering lifted out transparent), standalone image-to-video clips (i2v, animate this still, 그록 영상, 이매진 비디오, 스틸 움직여줘, first/last frame, reference-to-video, 영상 이어붙이기, 영상 편집, extend/edit a clip), clips from a video MCP on the agent (ZCRE, 지크 MCP), chroma removal, animation atlases, video loops, 큐레이션뷰, image candidates, 팔레트 스왑, palette swap, recolor, engine exports, repeating backgrounds, projected shadows, motion/contact inspection and optional scene composition from existing assets."+version: 3.1.0+description: "Generates images and game sprites through GPT or Grok with guided provider choices, separate saved defaults, automatic cleanup and optional curation. Handles sprite requests, ordinary image generation/editing, typography logos (로고, 타이포그래피, 타이틀 글자, 타이틀 로고, wordmark, logo, title lettering: a title screen drawn with the title in it, its lettering lifted out transparent), standalone image-to-video clips (i2v, animate this still, 그록 영상, 이매진 비디오, 스틸 움직여줘, first/last frame, reference-to-video, 영상 이어붙이기, 영상 편집, extend/edit a clip), clips from a video MCP on the agent (ZCRE, 지크 MCP), chroma removal, animation atlases, video loops, 큐레이션뷰, image candidates, 팔레트 스왑, palette swap, recolor, engine exports, repeating backgrounds, projected shadows, talking face layers over body motions (말하는 얼굴, 입 맞추기, 얼굴 층, lip sync, face pack), motion/contact inspection and optional scene composition from existing assets."license: Apache-2.0depends_on:required_bins:@@ -11,6 +11,8 @@ depends_on:why: "video-frames (clip -> frames) and video-set"- name: img2webpwhy: "video-loop WebP with exact alpha (libwebp); Pillow's animated writer drops -exact"+ - name: node+ why: "face-bake: the face pack's player (sprite-gen-face-v1.mjs) picks the frames"- name: rife-ncnn-vulkanwhy: "video-loop jump repair and video-cycle-align (RIFE v4.6 in-betweens with the -x -z test-time augmentation, release 20221029). Install once with `sprite-gen rife install` (sha256-checked, into the user data directory); SPRITE_GEN_RIFE or PATH override it; Linux without a GPU also needs libvulkan1 mesa-vulkan-drivers. Without it, walk and run loops are cut as filmed with a warning"required_scripts:@@ -119,6 +121,8 @@ hold the arms still, and check the loop with `handed-check --state walk`.| Where that ellipse is carried in every other cell, and what the move changes there (an arm crossing a chest moves with it; the stretch, the bend and the seam): a record and boards of every cell, nothing moved | `video-sway-inspect` (old name `video-follow-inspect`) | [video-pipeline](docs/video-pipeline.md#inspecting-a-sway--video-sway-inspect) || A set's directions on one sheet, standing on one ground line (a 3×3 board, an engine import): rows in compass order S SW W NW N NE E SE from each loop's `--view`, one cell size, every row as cut, an optional centre picture (`--center`) at the rows' standing height; loops of different lengths are laid and said to differ (`same_length` false) — align the set first | `video-set-sheet` | [video-pipeline](docs/video-pipeline.md#7-one-sheet-for-a-set--video-set-sheet) || A walk or run that stops, from the clip's own start played backwards: a one-shot strip (`loop: false`, `hold_last: true`), GIF, WebP and report beside the loop, ending on the still as filmed; leaves the loop from the cell nearest the entry (`from_cell`), drift taken out across the frames by a straight share; a seam past 1.4 steps (`--seam-max`), a run-up that comes back to a pose it left (another motion, however short) or one over 1.5 cycles (`--run-up-max`) refused; ghost screen checked, cells placed and anchored to match the loop's pivot | `video-stop` | [video-pipeline](docs/video-pipeline.md#8-a-walk-or-run-that-stops--video-stop) |+| Build a talking lower-face pack anchored to every frame of body motions from a reference and keyed clips. | `video-face` | [face-parts](docs/face-parts.md) |+| Bake a voice and the pack's face layer onto body frames, optionally after a clip's own mouth and sound. | `face-bake` | [face-parts](docs/face-parts.md#face-bake) || Compare the currently displayed loop with a final candidate, including after follow-through | `video-loop-compare` | [loop-comparison](docs/loop-comparison.md) || Restore post-processing damage in the current cut from its exact source times, preserving normal cells and playback | `video-source-manifest` → `video-loop-repair` → `video-loop-compare` | [source restoration](docs/loop-comparison.md#restoring-the-active-cut-from-source) || Video sprites from a clip your agent makes with a connected video MCP (ZCRE, 지크) | `video-canvas`, `video-prompt --no-last-frame`, the agent's own MCP tools (quote, user approval, generate, save the mp4), then `video-frames` and `video-loop` (a walk or run with `--anchor motion-auto`, which holds its size), and for a walk or run cut in two or more directions `video-cycle-align --view …` over their loop directories (a set it stops on a loop that may hold two cycles: look at that loop, count its strides, and pass `--cycles <loop>=<k>`; a loop the report lists under `unnamed_feet` — its view could not tell the feet apart, or only by too small a margin (`low-margin`): look at its first candidate frame, say which own foot lands there, and align again with `--foot <loop>=left|right`). `video-set` is not run on this route, so the set stage it runs by default (`--align-cycles auto`) is yours to run. sprite-gen never calls the MCP. Walk, run and jump only: ZCRE's `grok-imagine-video-1.5` has no end frame, so idle, attack and diagonal walks are refused unless `--unpinned` | [video-pipeline](docs/video-pipeline.md#a-clip-from-a-video-mcp-on-your-agent--zcre) |diff --git a/docs/README.md b/docs/README.mdindex 6fdb6f0..2892854 100644--- a/docs/README.md+++ b/docs/README.md@@ -89,6 +89,7 @@ grouping is derived from `sprite_gen/_modules.py`, the one taxonomy table.| [loop-repair.md](loop-repair.md) | RIFE (where it runs, cost, licences, `sprite-gen rife install`, what runs without it), jump-frame repair, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the overlapped wrap (a part on its own beat at a two-step cut's wrap), the jolt index and its gate, one cycle length per direction set || [frame-interpolation.md](frame-interpolation.md) | Generative in-betweens for sprite frames, recorded as a take || [seamless-video-loop.md](seamless-video-loop.md) | Making a non-looping ambient clip loop forever (RIFE seam bridge) — a different job from pipeline B |+| [face-parts.md](face-parts.md) | Face requests, face packs, talking face rules, live player, `video-face`, and `face-bake`. |## Curationdiff --git a/docs/face-parts.md b/docs/face-parts.mdnew file mode 100644index 0000000..38bd905--- /dev/null+++ b/docs/face-parts.md@@ -0,0 +1,250 @@+# `sprite-gen video-face` and `face-bake`: a talking face layer over body motions++This document owns the face request, the face pack, the rules of the face layer, the player `sprite-gen-face-v1.mjs`, `video-face`, and `face-bake`.++A body motion and its talking face are two layers. `video-face` cuts the lower face from a talking clip into a face pack and anchors it on every frame of every body motion. The layer covers the mouth, jaw line, cheeks, and chin down to the neck line. A mouth alone would leave the body's moving jaw and cheeks visible under a closed mouth. The pack's JavaScript player picks face frames for a voice. It draws them live in a browser page, or `face-bake` runs the same player through node and writes the result ahead of time.++## Quick start++Put a reference still, a talking clip, and body clips in a face request. Clip frames are keyed `frame-*.png` files.++```bash+sprite-gen video-face --request face-request.json --out pack/+sprite-gen face-bake --pack pack/ --voice line.wav --out baked/+```++To play a body clip from its start before the line, name it with `--play`. Its own mouth and sound play first.++```bash+sprite-gen face-bake --pack pack/ --voice line.wav --out baked/ --play greet --tags "[surprised]" --video+```++| `video-face` flag | Meaning |+|---|---|+| `--request` | Face request JSON containing the reference, face, talking clip, and body clips. |+| `--out` | Pack folder to write. It must not exist. The pack is written in a temporary folder beside it and renamed when whole. |+| `--jobs` | Processes reading anchors at once. The default is one per CPU. Anchor reading takes most of a pack's time. |++| `face-bake` flag | Meaning |+|---|---|+| `--pack` | Face pack folder. |+| `--voice` | Line voice in any sound format ffmpeg reads. |+| `--out` | Output folder, new or empty. |+| `--play` | Body clip to play from its start before the line, with its own mouth and sound first. |+| `--start-cell` | Loop cell where the line is requested without `--play`. Default: `0`. |+| `--tags` | Line tags, such as `[surprised]`. |+| `--lead` | Seconds of silence before the voice. Default: `0.5`. |+| `--tail` | Seconds of silence after the voice. Default: `0.6`. |+| `--video` | Also write `baked.mp4` with the frames over a flat matte and the sound. |+| `--beside` | Also write `beside.mp4`. It puts the supplied `frame-*.png` files on the left and the bake on the right, with the bake's sound. Supply exactly as many frames as the bake writes. |+| `--video-height` | MP4 height. Default: the frames' height. |+| `--matte` | MP4 background as six hex digits. Default: `2a2430`. |++`face-bake` needs `ffmpeg` on PATH to read the voice and write MP4 files. It needs `node` on PATH to run the pack's player. Without node, it stops and says so.++## The request++The request has kind `sprite-gen-face-request` and version `1`. Paths are relative to the request file. This example uses a 480 px reference.++```json+{+ "kind": "sprite-gen-face-request", "version": 1, "fps": 24,+ "reference": "reference.png",+ "face": {+ "mouth": [240, 262],+ "eyes": [[175, 174, 215, 206], [265, 174, 305, 206]],+ "lower_face": {"top": 228, "box": [120, 228, 360, 400], "seeds": [[195, 255], [285, 255], [240, 305], [215, 280]]},+ "band_bottom": 300+ },+ "talk": {"frames": "talk", "place": [110, 90, 260]},+ "body": [+ {"name": "idle", "loop": true, "frames": "idle"},+ {"name": "greet", "frames": "greet", "mouth_timeline": "greet-mouth.tsv", "sound": "greet.wav"},+ {"name": "cheek", "frames": "cheek"},+ {"name": "reaction", "frames": "reaction", "place": [110, 90, 260], "own_mouth": [0, 3.0]}+ ]+}+```++| Field | Meaning |+|---|---|+| `fps` | Pack frame rate. |+| `reference` | Rest pose and still from which the clips were made. Its pixels define the `face` coordinates. |+| `face.mouth` | Centre of the closed mouth. The mouth reading, open-area zone, and step box sit around it. |+| `face.eyes` | Two boxes in `x0, y0, x1, y1` order. They are excluded from the layer's weight and anchor band. |+| `face.lower_face.top` | Top edge across the cheeks, in reference y coordinates. Weight is zero above it. |+| `face.lower_face.box` | Bounds within which a lower face can be. |+| `face.lower_face.seeds` | Skin points on the cheeks and chin from which the lower face grows. They must sit on reference skin. |+| `face.band_bottom` | Y coordinate below which the anchor band reads nothing. The neck and choker below it belong to the body. A hand counts only above it. |+| `talk.frames` | Folder of keyed, square `frame-*.png` talking frames. It needs six or more frames and can use `video-frames` output. |+| `talk.place` | `[x, y, side]` showing where the talking clip's square frame lies on the reference. Default: `[0, 0, reference width]`. |+| `body[]` | Body motions. Exactly one has `loop: true`; it is the rest motion to which the others return and talks only through the face layer. |+| `body[].name` | Letters, digits, `-`, and `_`. |+| `body[].place` | Placement in the same form as `talk.place`. |+| `body[].sound` | The clip's own sound, copied into the pack. |+| `body[].own_mouth` | `[0, end seconds]`, the initial span during which the clip uses its own mouth. |+| `body[].mouth_timeline` | Tab-separated file with `frame`, `state`, and `sound` columns, with one row per frame from `0`. Use this or `own_mouth`, never both. |++For a mouth timeline, the own-mouth span ends at the first `closed` frame after the last `voiced` frame. A mouth that closes while sound is still voiced is inside a word. `video-face` refuses a timeline if the mouth never closes after its last sound or if its rows do not match the clip's frames.++## What `video-face` does++### 1. Anchors++Each frame of each clip is laid on the reference with the engine's cell resampler and its `place`. An anchor reads the similarity that maps the reference face to that frame: place, size, and tilt.++The anchor band runs from the feather of the reference's own weight to 30 px beyond it. It stays above `band_bottom` and excludes the eyes. It excludes the jaw because the jaw moves during speech.++The first place match uses masked normalised cross-correlation within 48 px. If the best match sits on the search rim, the search extends to 96 px. Size is tried from 0.96 to 1.04 in steps of 0.02, and tilt from -4 to 4 degrees in steps of 1. Each pair is laid through that place and matched again within 3 px. The least cost wins. A parabola through its neighbours refines size and tilt.++A best match on the grid rim is read again on a grid twice as wide: size 0.92 to 1.08 and tilt -8 to 8 degrees. The place required by a size and tilt can be more than 3 px from the first reading. A 3 by 3 grid is therefore read again around the place, size, and tilt it found until the place stays, for at most 4 passes.++Each anchor number is checked against the median of 5 surrounding frames, wrapping around a loop. A reading farther than three scaled MADs and farther than 0.5 px in place, 0.01 in size, or 0.3 degrees in tilt is replaced by the window medians. Anchors depend only on the reference and the frame. Naming other clips in the request does not change them. `--jobs` runs anchor reading in processes.++### 2. Weight++Lower faces are read from the reference and every clip frame, then brought into reference coordinates through their anchors and joined. A lower face is opaque skin lighter than luma 175, grown from the seeds inside the box below the top edge. Its outline and mouth lines stop that growth.++A frame adds no more than 12 px beyond the reference's own lower face. This includes a dropped jaw without growing into a hand or sleeve. Holes from open mouths and lines are filled.++The weight is 1 on the union grown by 6 px. It falls to 0 over the next 10 px with a cosine-squared curve. It is 0 above the top edge and rises to 1 over 14 px. It is 0 on the eye boxes, grown by 6 px and feathered over 5 px.++### 3. Talking frames++Each talking frame is read in reference coordinates through its anchor. The mouth is the dark line or opening nearest `face.mouth`, at least 18 luma below the box's skin and away from headset grey. Its corners are its leftmost and rightmost pixels.++Lift is how far the mouth's top edge in its middle third sits below the chord between the corners, in px. Positive lift means a smile with raised corners. The reading also records height and open area. Open area counts pixels around the mouth more than 40 luma from the reference's closed mouth.++A mouth 5 px tall or less is closed. A closed mouth with lift of at least 0.35 px smiles; below that it is neutral. A closed mouth is one line, not a gap. Omitting one between two frames of the same expression would split that run into two turns.++An open run takes the set of the closed frames on both sides. A run between a smiling and a neutral closed frame turns between expressions and belongs to neither set. The step between any two talking frames is read on the face around the mouth with the engine's loop analysis features, in units of the talking clip's median step.++### 4. Sprites++A talking frame in the smile or neutral set becomes a sprite at the talking clip's own resolution. Its alpha starts with the weight and removes what must remain the body's where the face meets it. This includes dark grey and black reaching inward from the boundary, such as a headset boom or choker, and lines drawn in the feather. The removal prevents a second, faint line. Median colour over the feather moves the sprite's colour onto the reference's colour.++A sprite leaves out the weight's outermost 2 px, where the weight is under 0.1. Mapping a sprite to the talking clip's pixels and back spreads its edge by a pixel, and nothing may change outside the weight. Where a sprite does not cover a pixel, the lay leaves that pixel exactly as it was, including the colour a keyed cell keeps under no coverage.++### 5. Body frames++Each body cell is copied with its anchor and a keep image for anything that must stay above the face layer. The keep contains the body's own dark grey and black reaching inward from the boundary, its lines in the feather, and a hand.++A hand differs from the reference face by more than 48 in any channel and reaches inward from outside the layer. The frame's light, warm skin is divided into pieces using drawn lines as walls. Teeth and white streaks do not count as skin. The piece that overlaps the reference's lower face most is the face. Other skin pieces touched by the outside piece belong to the hand, with their surrounding lines. A hand counts where at least 400 px lie on the weight's flat part above `band_bottom`.++The body's own open mouth is kept only where it meets a hand. The face's own skin is never counted as hand skin.++### 6. Entry++The clip's own mouth is read at the end of `own_mouth`, or at the first frame if there is no such span. The same 0.35 px lift line selects smile or neutral.++The rest is a closed talking frame in that set from which the line opens and to which it closes. The nearest lift wins. Among closed frames within 0.5 px of that nearest lift, the one with the smallest switch wins.++### 7. Switch++For every body frame and every rest used by the pack, the switch measures how far laying that rest moves the face. It divides by the larger of the body's median step and the talking clip's median step. A body that only breathes can step much less than a talking face. The layer comes on or goes off only when the switch is within the jump line, 1.4.++### 8. Rules++The pack records the flap's numbers, the pick's costs, the open area wanted for each mouth level, the jump line of 1.4, and the enter line. Level `L` of 6 wants `L/6` of the talking clip's widest open area. The enter line is the talking clip's largest step divided by its median step.++## The pack++The pack has kind `sprite-gen-face-pack` and version `1`. `face-bake` checks every listed file against its SHA-256 digest. It refuses a pack changed after creation.++| Path | What |+|---|---|+| `face-pack.json` | FPS, size, rules, requested face geometry, and every talking frame's set, open area, mouth reading, sprite file, and map to the reference. It also contains the two sets and their closed frames, the step matrix, body cells, anchors as maps from the reference, keeps, `own_mouth`, entry and switch tables, own sounds, player, and every file's SHA-256 digest. |+| `sprite-gen-face-v1.mjs` | Player copied from the package. |+| `face/frame-NNNNN.png` | Sprites indexed by talking frame. |+| `body/<name>/frame-NNNNN.png` | Body cells. |+| `body/<name>/keep-NNNNN.png` | Parts of a body cell that stay on top, where present. |+| `body/<name>/sound.wav` | A clip's own sound. |+| `weight.npy` | Layer weight in reference coordinates. |+| `report.json` | Per clip anchor ranges; band residual median, p90, and maximum with anchor, place only, and no alignment; misread frames; frames using the wide grid or wide search; and hand frames. It also records expression sets, each body's entry, and how many frames can switch on for each rest. |++## The player++`sprite-gen-face-v1.mjs` is the one place where the line, pick, and scene rules run. It exports these functions for browser and node.++| Export | Result |+|---|---|+| `mouthTrack(samples, sampleRate, rules, { fps, tags })` | Voice mouth levels from 0 to 6, one per frame. |+| `pickFaces(pack, levels, set, rest)` | A talking frame in one set or its rest for each output frame. |+| `planScene(pack, { play, start, levels, layer })` | Body, cell, face, and layer state for each output frame. It also returns `own_mouth`, the own-mouth end frame; `line_start`, the frame the voice and layer start; and `wait`, the frames waited after the own mouth. |+| `loadFacePack(url)` | Loads a pack. |+| `createFacePlayer(canvas, pack, { audioContext })` | Live player with `start()`, `stop()`, `play(name)`, and `say(voice, { tags })`. |++### The line++The voice is folded to mono. Loudness is RMS over 1/48 s windows sampled every 1/96 s, smoothed over three windows, and divided by the line's loudest value.++A syllable is a peak above 0.12 that stands 0.05 above the lower of its neighbouring dips, searched 10 steps to either side. If two syllables are within 1/12 s, the louder remains. A syllable opens to level 2 through 5 according to its loudness within the line's range. With `[surprised]` or `[gasps]`, a syllable louder than 0.85 opens to level 6.++The peak level lands on the frame one ahead of the syllable. The preceding frame gets half that level, rounded up. The following frame gets two fifths, rounded down. Other frames are closed. No frame changes by more than 3 levels; a neighbour is raised instead of cutting an opening. Rounding follows Python's rule: halves go to the even neighbour.++### The pick++The pick uses Viterbi over each frame in one expression set and the rest, for every output frame. It starts and ends on the rest. Frames outside the set are never shown, so the expression entered for a line remains its expression.++A frame normally advances to the talking clip's next frame. Holding costs 0.6. Jumping to a frame within 1.4 median steps of that next frame costs 1.5 plus the step. Entering from rest or exiting to it costs 1.0 and is allowed only within the enter line.++A frame's cost is the distance between its open area and the area wanted by the level, measured in one level's area. Opening less than wanted costs three times as much as opening more. The rest costs the wanted area.++### The scene++A line requested during a body clip's `own_mouth` span waits until that span ends. The clip's own mouth and sound play through it.++The body at the end of that span names the line's set and rest. The voice and layer start together at the first frame whose switch to that rest is within the jump line, 1.4. Until then the clip shows as it is, with the layer off and no voice playing. A voice never plays on a frame whose mouth cannot move. If the layer is already on, the voice and layer start at once in its current set and rest. A clip that plays once returns to the loop from the loop's first cell. If no suitable frame occurs within the body clip and one round of the loop, `planScene` refuses the line and the player throws.++### Live++The audio context supplies one frame clock. `play(name)` starts a body clip and its own sound. `say(voice)` decodes the voice, computes levels, and plays it at the plan's `line_start` with the layer. Requests run one after another. A line requested during a clip's own mouth is planned after that span.++After a line, the layer stays at its rest until a loop frame can switch off within the jump line. Body cells load when first drawn. Each frame draws the body cell, then the sprite using the cell anchor times the sprite map as a canvas transform, then the keep image. These are plain source-over draws. Sprite colour is already matched in the pack, so live drawing does no colour step.++## `face-bake`++`face-bake` loads the pack and checks its digests. It decodes the voice to mono 48 kHz with ffmpeg. It adds `--lead` and `--tail` silence after the mono fold, so a stereo voice cannot delay one channel alone. Through the package's `face_plan.mjs`, node runs the pack's own `mouthTrack` and `planScene`. The bake lays each frame as the live page does.++With `--play`, the sound track contains the clip's own sound for its `own_mouth` span, then silence while the line waits, then the padded voice.++| File | What |+|---|---|+| `frames/frame-NNNNN.png` | RGBA frames at the pack's size. |+| `voice-baked.wav` | Whole sound track, mono 48 kHz, as long as the frames. |+| `voice-padded.wav` | Voice with lead and tail silence. |+| `bake.json` | Per frame body, cell, face frame, layer state, level, pixels changed outside the weight, and composite, body, and layer steps. It also records set, rest, `own_mouth_frames`, `line_start`, `wait_frames`, levels, `over_line`, `over_sum`, and verdict. |+| `baked.mp4` | Frames over the matte with sound, when `--video` is set. |+| `beside.mp4` | Supplied frames on the left and bake on the right, with bake sound, when `--beside` is set. |++Steps are read on the face around the mouth in reference coordinates. A composite step over 1.4 times the largest of the body's own step, the layer's own step, and the body's median step is `over_line`. Body and layer motion in one frame can add up even when each is within the line. A frame that is `over_line` and also exceeds the sum of the body and layer steps is `over_sum`.++If the planner finds no suitable frame within the body clip and one round of the loop, `face-bake` stops and names the reason. The bake is refused with exit code 1 and verdict `refused` if any frame is `over_sum` or if any pixel outside the layer's weight changes. The frames are still written for inspection.++## What it takes from sprite-gen++| Part | Taken from | Why / owner |+|---|---|---|+| Keyed clip frames | `sprite-gen video-frames` | The request uses its `keyed/` folders. |+| Clip frame laid on the reference at another size | `util.resample.resize_cell` | Cell resampling owned by `docs/video-pipeline.md`, "Cells". |+| Sprite mapped onto a body frame | `util.resample.transform_cell`, after one `resize_cell` when laid smaller than 0.9 | Cell transform owned by `docs/video-pipeline.md`, "Cells". |+| Anchor place match | `video.auto_motion._match_costs` and `_best` on `video.motion_anchor._features` | Same masked normalised cross-correlation and nearest-offset tie rule as the `video-loop` motion anchor. |+| Frame steps | `video.loop._small_features` | Loop analysis features. |+| Jump line | `video.stop.SEAM_MAX` | Stop and loop jump line, 1.4. |+| Mask growth and pieces | `video.parts._dilate`, `_erode`, and `_pieces` | Tail mask morphology. |+| Packaged JS player and digests | Sound pack's `sprite-gen-audio-v2.mjs` and `audio/manifest.py` | One rule in JS, copied into the pack. |+| JS run from Python | node, as in the breathe JS mirror test | Runs the pack's player for a bake. |+| Similarity anchor with size and tilt | New code here | The engine has no size and tilt reading. |+| Mouth reading and expression sets | New code here | The engine has no mouth reading. |+| Hand keep | New code here | The engine has no hand over face reading. |+| Flap and pick | New player rules here | Moves the rules into the player. |++## Limits++A three-quarter head turn is not a similarity transform. The anchor reads it as well as a similarity can. `report.json` gives the band residual; a large residual means visible seams.++A hand is kept only where the frame draws its outline. A finger without a line separating it from the cheek becomes part of the face skin and is covered by the layer.++There is no transition yet from a smile back to neutral. After a line in the smile set, the layer rests smiling until a loop frame is within the jump line of that rest. A neutral loop may never reach that condition.++Body cells are carried at the reference size.diff --git a/pyproject.toml b/pyproject.tomlindex 7bf77f8..7433fa0 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -12,7 +12,7 @@ build-backend = "setuptools.build_meta"name = "sprite-gen"# Release discipline: keep this package metadata version synchronized with# SKILL.md's `version:` field in the same release commit.-version = "3.0.0"+version = "3.1.0"description = "Component-row pipeline for clean 2D game sprites and animation atlases"readme = "README.md"license = "Apache-2.0"