Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
AI Frame Interpolation (in-betweens)
Owns: Generative in-betweens for sprite frames, recorded as a take · Index: docs/README.md
sprite_gen/effects/interpolate.py (scripts/interpolate_frames.py) makes an in-between
frame between two frames of one state — a half-closed eyelid between open/closed
idle frames, a mid-swing arm between windup and strike.
Doctrine position
"AI touches raw generation only" applies here too: interpolation is a raw-stage AI
step. The output is never a final frame — it is recorded as a take
(raw/<state>.takes/<label>.png + states.<state>.takes in sprite-request.json),
and logical frames are always baked by the deterministic extraction path (chroma
removal → components → grid snap → shared palette). The curator orders the new frame
in the play sequence like any other take frame.
Backend: generative (codex / grok) — flow-based VFI retired
The backend is the engine's own generation layer (sprite_gen.gen): the two aligned
frames go in as reference images and the model draws the in-between pose.
Default provider codex (GPT image_gen); --provider grok selects xAI Imagine.
Flow-based VFI (RIFE) was retired (maintainer, 2026-07-17). Rationale, measured on a
real 3-way comparison (hero down_action arm swing, sheet tween-3way-compare.png
in the tray): VFI interpolates motion photometrically, so appearance
changes come out as a blurry cross-fade; generative in-betweens drew the mid-pose in
clean discrete pixels. codex preserved identity best; grok drifted; RIFE smeared.
Auth prerequisites (applies to the curator Tween button too)
Generation always runs through the server machine's image provider — the browser/webview never sees or carries any credential. The Tween button POSTs to the local curation server, which uses the shared image providers (Codex subprocess or direct Grok API) with their own machine-local credentials.
- Required on the machine running the curation server / CLI:
codexCLI installed and logged in (ChatGPT OAuth) for--provider codex- a valid Grok login or
XAI_API_KEYfor--provider grok(see gen)
- Setup and provider topology:
docs/gen.md. - Not authenticated / required Codex CLI missing → the provider fails loudly and the message surfaces as-is (CLI stderr / curator status bar). No silent fallback between providers.
Usage
$SPRITE_GEN_ROOT/.venv/bin/python $SPRITE_GEN_ROOT/scripts/interpolate_frames.py \
--run-dir <run> --state down_idle --between 1 2 \
[--provider codex|grok] [--t 0.5] [--label blink_mid] [--extract]
--between A B— frame indices on the state's primary strip (component order).--provider— generation backend; defaultcodex(3-way comparison winner).--t— interpolation time inside (0, 1); default 0.5. Non-midpoint values are expressed in the prompt ("closer to A/B").--label— take label; defaulttween_<A>_<B>_t<t>. Re-running the same label overwrites the same take file and does not duplicate thetakesentry (idempotent).--extract— run a full-batch re-extraction afterwards. Partial extraction is deliberately not offered: the run-wide shared palette is built per extraction batch (CHANGELOG v1.56.17 known limitation), so a single-state extract would shade that row from a different palette.
In the curator, the row-header Tween button does the same thing: open the
popover, click two cards to pick the pair (blue border), choose GPT/Grok, Generate.
While the follow-up full-batch re-extraction runs, /api/run reports busy (503)
instead of a manifest-consistency error.
Pipeline internals
- Chroma-remove the primary strip, extract components, tighten to solid bbox.
- Registration (
register_row_frames) puts both frames on one canvas aligned by upper-body alpha overlap — static pixels coincide, so the model reads the pair as "same drawing, only the moving part differs". - Canvas pads to /32 on the request chroma color; the prompt requires the same flat chroma background so the extractor keys it out later.
tween_promptcomposes the prompt deterministically from request truth (character description, chroma color, t) andsprite_gen.gen.generate_image()runs the provider with both frames attached as refs.- The generated mid frame is saved as the take raw; extraction re-derives the state
row (the frame appears labeled
<label>#0) and pixel-unfake quantization snaps it onto the logical grid.
Quality note: judge the result after pixel-unfake re-quantization, and touch up stray pixels with the curator's pixel editor if needed. Identity drift is a reroll (regenerate the take), not a local repair.
Testing
tests/test_frame_interpolation.py covers the plumbing with an injected stub
interpolator (alignment canvas contract, prompt composition, take write, request
idempotency, loud rejections including unknown providers). Real provider generation
is exercised manually, not in CI.
Related
- docs/README.md — documentation index