Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
docs: sprite-gen video setup contract, v1.61.0
Soohong Kim9f32d30d2b2e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 85bb765..3337fbd 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,13 @@All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.+## v1.61.0 - Image to Video++- Added `sprite-gen video`: one still + prompt → a verified mp4 through Grok Imagine (`POST /v1/videos/generations`), with duration 1–15 s, 480p/720p/1080p, optional aspect ratio and audio flag, and a `sprite-gen-video-report` JSON.+- Credentials are the user's own and never part of the repo: `XAI_API_KEY` when set, otherwise the `grok` CLI login file (`~/.grok/auth.json`, `GROK_HOME` honoured). The report records `auth_source`; tokens and download URLs are never printed or written.+- An expired grok login fails before any upload with the exact refresh command; a set-but-empty `XAI_API_KEY`, a missing credential, a refused request, a failed/expired generation, a poll timeout, or a non-mp4 download each fail by name and write nothing.+- New wrapper `scripts/generate_sprite_video.py` and docs at `docs/video.md`.+## v1.60.0 - Native Alpha- `sprite-gen gen --transparent` now follows a per-provider transparency strategy declared once on each adapter (`Provider.transparency`). `codex` asks `image_gen` for a genuinely transparent background and publishes the measured alpha (`native`, first choice); `grok` keeps deterministic chroma keying because Grok Imagine returns JPEG only.diff --git a/README.md b/README.mdindex 1bb7a3e..789573d 100644--- a/README.md+++ b/README.md@@ -252,6 +252,15 @@ implementation. Transparent stills follow a per-provider strategy: `codex` asksis published; `grok` is keyed out of a chroma background. See[`docs/gen.md`](docs/gen.md) for the CLI and verification contract.+### Image to video++`sprite-gen video --image still.png --prompt "…" --out clip.mp4` animates one+still through Grok Imagine using **your own** credential — the `grok` CLI login+(`grok login`, SuperGrok Imagine quota) or an `XAI_API_KEY`. Nothing is shipped+with the repo; the report says which one ran. An expired login stops before+uploading and tells you the exact refresh command. Setup, parameters, and the+verification contract: [`docs/video.md`](docs/video.md).+## AttributionThe component-row workflow is inspired by the Apache-2.0 licensed `hatch-pet` skill, but targets generic game sprite atlases and includes no pet packages or pet visual assets.diff --git a/SKILL.md b/SKILL.mdindex 21947f8..9768904 100644--- a/SKILL.md+++ b/SKILL.md@@ -1,6 +1,6 @@---name: sprite-gen-version: 1.60.0+version: 1.61.0description: "Generate clean 2D game sprites and animation atlases with a component-row pipeline: base identity, numeric sprite-request SSoT, per-state layout guides, image-gen row strips, chroma-key alpha cleanup, connected-component frame extraction, cell-based atlas composition, QA reports, and runtime manifest frame_layout. Its curation webview also serves ANY image-candidate set (icons, logos, generated drafts) — agent chat can't render images, this can: unpack_atlas_run --pngs-dir import, then serve_curation side-by-side compare/pick. Palette-swap bake (`sprite-gen recolor`) turns a base sheet + palette map into N colourway sheets; the curation view blink-compares and adopts a pick into curation.json.recolor.picked. Curation triggers (KR/EN): 큐레이션, 큐레이션뷰, 큐레이션 해줘, 이미지 후보 보여줘/안 보임, 나란히 비교, 골라볼게 띄워줘, curation view, show image candidates side by side, let me pick. Recolor triggers (KR/EN): 팔레트 스왑, 팔레트 베이크, 리컬러, 색깔 바꾸기, 컬러웨이, 색 변형, 팔레트 맵, 색갈이, palette swap, recolor, colourway, colorway, bake variants, palette map."license: Apache-2.0depends_on:@@ -12,6 +12,7 @@ depends_on:required_scripts:- scripts/prepare_sprite_run.py- scripts/generate_sprite_image.py+ - scripts/generate_sprite_video.py- scripts/extract_sprite_row_frames.py- scripts/interpolate_frames.py- scripts/compose_sprite_atlas.py@@ -195,6 +196,7 @@ Scripts are explicit pipeline commands, not hidden imports. One job each (stage- `sprite_gen/compose/compose_layers.py` (`sprite-gen compose-layers`) — deterministic composite bake for a run that declares a **rig**. Stacks curated rows onto each other by integer pivot translation + arbitrary alpha masks (no resampling, rotation or scale of its own) into `<run-dir>/layers/<name>.png` + `<name>.manifest.json` + `layers.report.json`. Optional and opt-in: a request with no `rig` / `track` / `layers` is not a layer run and is refused by name rather than composed. All-or-nothing — declaration, composition **and** publish: one violation reports every violation and writes nothing. Declaration schema, CLI usage, and what `prepare` carries: [`docs/layer-tracks.md`](docs/layer-tracks.md).- `unpack_atlas_run.py` — inverse of compose: rebuild a curator-ready run dir from a finished sheet (`--grid` > `--manifest` > auto-detect) or import a PNG folder (`--pngs-dir`, with sibling `meta.json` labels/iso grid).- `export_curated_pngs.py` — export curated frames back to named PNGs with the transform baked in, into `<run-dir>/curated/`; the deliverable for imported still sets.+- `sprite_gen/gen/video.py` (`sprite-gen video`) — one still → verified mp4 through Grok Imagine (`POST /v1/videos/generations`) with the user's own credential: `XAI_API_KEY` if set, else the `grok` CLI login (`~/.grok/auth.json`, SuperGrok Imagine quota). Expired login fails before upload with the refresh command; tokens/URLs never reach the report. The `grok-imagine-video` skill is a thin shuttle over this. Contract: [`docs/video.md`](docs/video.md).- `cutout.py` (`sprite-gen cutout`) — background remover for **imported** images (not pipeline output, which is already keyed). Routes on the corner background colour (`--key auto|white|magenta|green`): **white/ivory** → position matte (corner flood-fill keeps interior highlights unholed → decontaminated soft-alpha border + soft erode); **magenta/green key** → reuse the verified `extract.remove_chroma_background` engine as-is (no drift — key colours are absent from objects so its colour-only cut is safe there). `--white-check` writes cyan/magenta/yellow verification composites. No Silent Fallback (leftover non-zero RGB under transparency raises).- `slice_sheet_cells.py` — slice a multi-figure grid sheet (same character, N expressions/variants in one image) into per-cell standing cuts: v1.13 chroma alpha + centroid cell assignment + merged-figure split/in-cell re-label + neighbour-debris drop + per-cell height normalization + shared feet baseline. For dialogue cut-in portraits (立ち絵), not animation rows. Detail: [`docs/sheet-slicing.md`](docs/sheet-slicing.md).- `check_visible_magenta.py` — optional screenshot QA guard for visible chroma-key leakage.@@ -505,6 +507,7 @@ sprite-gen (this SKILL.md = behavior contract + hub)│├─ GENERATION ── "raw/<state>.png from prompts (the one AI step)"│ ├─ docs/gen.md # sprite-gen gen provider CLI · verified PNG/report · image-gen shuttle+│ ├─ docs/video.md # sprite-gen video · Grok Imagine i2v with the user's own grok login / XAI_API_KEY│ ├─ docs/frame-interpolation.md # generative in-between (codex/grok) → take raw · auth prereqs · RIFE retire rationale│ └─ docs/seamless-video-loop.md # non-looping AI video clip → seamless loop: flow-matched cut + RIFE seam bridge│diff --git a/docs/video.md b/docs/video.mdnew file mode 100644index 0000000..5d922fa--- /dev/null+++ b/docs/video.md@@ -0,0 +1,96 @@+# `sprite-gen video` — image to video with your own Grok login (engine SSoT)++`sprite-gen video` animates one still into a short mp4 through **Grok Imagine**+(xAI `POST /v1/videos/generations`). It is the video counterpart of+[`sprite-gen gen`](gen.md): one call = one still (+ prompt) → one **verified** mp4+on disk plus a machine-readable report. The `grok-imagine-video` skill is a thin+shuttle over this command.++No credential is shipped with this repository. You bring your own, in one of two+forms, and every run reports which one it used.++## Setup — pick one credential++| `auth_source` | What you need | Billing | How to set it up |+|---|---|---|---|+| `grok-login` (default) | the `grok` CLI signed in once | your SuperGrok **Imagine quota** (no console spend) | install the grok CLI, run `grok login` (`--oauth` for a browser, `--device-auth` for a headless box). It writes `~/.grok/auth.json`; this tool only reads it. |+| `XAI_API_KEY` | an xAI console API key | console credit | `export XAI_API_KEY=xai-…` |++Resolution order is fixed: **`XAI_API_KEY` wins when set**, otherwise the grok+login file (`GROK_HOME` relocates `~/.grok`). Neither is a fallback for the other —+a set-but-empty `XAI_API_KEY` is an error, and with no key and no login the run+stops with both setup paths spelled out.++### The login token expires — and this tool does not refresh it++The grok CLI stores an OIDC access token that lasts about six hours+(2026-09-08 measurement: `expires_at` 10:56Z, refreshed to 18:26Z by the next grok+command). `sprite-gen video` reads `expires_at` **before uploading anything**; if+it has passed, the run fails with the refresh prescription instead of gambling on+a 403 mid-upload:++```+video: the grok login token expired at 2026-09-08T10:56:34Z (now …); nothing was uploaded.+ refresh it with any grok CLI command that reaches the API, e.g. `grok -p ok --output-format plain`,+ or sign in again with `grok login`. This tool never rewrites ~/.grok/auth.json itself.+```++Why not refresh it here: `auth.json` is the grok CLI's file, the refresh token in+it may rotate, and a second writer would break the login the user relies on+everywhere else. The same shape as `sprite-gen gen`'s `codex login status` gate.++### Why the direct API call and not Grok Build's `image_to_video` tool++Grok Build's built-in `image_to_video` tool posts without `output.upload_url`, and+on Zero-Data-Retention teams the API answers `HTTP 400 — Zero Data Retention teams+must provide output.upload_url for video generation`. The same login calling+`/v1/videos/generations` directly (no `output.upload_url`) succeeds and bills the+Imagine quota (verified 2026-08-22, re-verified 2026-09-08). So this engine calls+the API itself and never routes through the agent-side tool.++## CLI++```bash+sprite-gen video \+ --image still.png \ # PNG / JPEG / WebP; sent inline as a data URL+ --prompt "Camera locked. Gentle idle sway, tail flick." \ # or --prompt-file+ --out clip.mp4 \+ [--duration 6] # 1..15 seconds (default 6)+ [--resolution 720p] # 480p | 720p | 1080p (default 720p)+ [--aspect-ratio 1:1] # 1:1 16:9 9:16 4:3 3:4 3:2 2:3 (default: the still's ratio)+ [--audio | --no-audio] # default: the API's default (audio on)+ [--model grok-imagine-video-1.5]+ [--report clip.report.json]+```++Backward-compatible wrapper: `$SPRITE_GEN_ROOT/.venv/bin/python $SPRITE_GEN_ROOT/scripts/generate_sprite_video.py …` (same args).++What happens, in order:++1. Validate the request (prompt, still, duration, resolution, aspect ratio) and+ resolve the credential — all before any network call.+2. `POST /v1/videos/generations` with `model`, `prompt`, `duration`, `resolution`,+ optional `aspect_ratio` / `generate_audio`, and `image.url` as a base64 data URL.+ A `401/403` names the credential source and its fix; any reply without a+ `request_id` fails with the API's own error text.+3. `GET /v1/videos/{request_id}` every 4 s until `status` is `done`. `failed`,+ `expired`, a non-2xx poll, or the timeout (`SPRITE_GEN_VIDEO_TIMEOUT_SECONDS`,+ default 600) fail loudly. Nothing is written on any failure path.+4. Download `video.url`, verify the bytes start with an mp4 `ftyp` box, then move+ the file into `--out` atomically (`.part` staging).++The report (`sprite-gen-video-report`) carries `auth_source`, `model`,+`request_id`, `bytes`, `host`, requested/reported duration, resolution, aspect+ratio, audio flag, `elapsed_seconds`, and `polls`. **Tokens and download URLs are+never printed or written** — only the download host (`vidgen.x.ai`).++## Quotas and limits++- Imagine quota is a weekly SuperGrok allowance; when it is exhausted the API+ refuses the POST and the run fails with that message (no retry loop here).+- Duration 1–15 s, resolutions 480p/720p/1080p, the seven aspect ratios above —+ from the xAI video docs as of 2026-09-08. A value outside those is rejected+ locally before the call.+- Output is whatever the model returns (typically H.264 mp4 with audio unless+ `--no-audio`). Downstream frame extraction is a separate step and not part of+ this command.diff --git a/pyproject.toml b/pyproject.tomlindex 83b5fd0..98d5f96 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -12,7 +12,7 @@ build-backend = "setuptools.build_meta"name = "sprite-gen"# Release discipline: keep this package metadata version synchronized with# SKILL.md's `version:` field in the same release commit.-version = "1.60.0"+version = "1.61.0"description = "Component-row pipeline for clean 2D game sprites and animation atlases"readme = "README.md"license = "Apache-2.0"