Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
Merge remote-tracking branch 'origin/main'
Soohong Kim8df5fd5775f5
diff --cc CHANGELOG.mdindex 4fd1125,bf9c35f..bd4bd3d--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -4,9 -4,10 +4,11 @@@ All notable public changes to `sprite-g## Unreleased+ - **Behavior change: a refused provider request is sent again only within the time limit of its call, and the waits count in that limit.** Previously a request its provider documents as not taken (a 429, an overload) was sent again after 8 s and 16 s, or after the answer's `retry-after` (at most 120 s), twice, whatever time the call had left, and the time limits did not count those waits: `SPRITE_GEN_GEN_TIMEOUT_SECONDS` bounded each send of an OpenAI or Grok image and of the OpenAI facing check on its own, the Grok facing check's 120 s each read, the clip's poll timeout counted from the send xAI accepted, and a Message Batch's status and results reads waited outside `--batch-wait`. A caller that stops the run at its own ceiling, set above those limits, could kill a request taken after two refusals and hear nothing from the engine. Now each limit counts from the first send and holds the waits: the image calls and the OpenAI facing check count `SPRITE_GEN_GEN_TIMEOUT_SECONDS`, the Grok facing check its 120 s, each of their sends reading only what is left of it; the clip its poll timeout (`SPRITE_GEN_VIDEO_TIMEOUT_SECONDS`; the POST itself is still not cut, its answer being the one way to the request id); a batch's reads `--batch-wait`. A wait that would reach the limit is not waited: the refusal ends the call as any refusal does, after one stderr line, `[gen] openai refused the request (429 rate_limit_exceeded); nothing was made — not asking again: the 16s wait would reach the time limit (12s left)`. The limit is read again after the wait, right before the send, as a sleep may wake later than it was asked to: one that woke at the limit or past it sends nothing either, and the refusal ends the call the same way (`… — not asking again: the time limit came during the 8s wait`). A clip's polls and downloads read their poll timeout the same way: a poll or download whose wait woke at it or past it is not sent, and the run ends unfinished on the last one as it came, where the one the transport found no time left to send was counted as a poll or download that got no answer (`timed out`). A realtime `pixel-draw` call has no time limit of its own and is unchanged.+ - **Tests** — `tests/gen/test_retry.py` (a request sent again only while its wait ends before the time limit; the answer's `retry-after` counted the same; a wait that ends at the limit not waited; a wait that woke at the limit or past it sends nothing, the limit read by one judgment, `too_late`, before the wait and right before the send; every direct provider call hands `retry` its limit; the Grok facing check and a Message Batch read whose wait woke past their limit send nothing), `tests/gen/test_openai_provider.py` and `tests/gen/test_grok_provider.py` (with 20 s, the second send waits only the 12 s left and the second 429 ends the call `sent 2 times`; with 8.01 s, an 8 s wait that woke 20 ms late sends nothing and the 429 ends the image call, or the OpenAI facing check, as it came), `tests/gen/test_video.py` (the poll timeout counts from the first send; a 429 whose next wait would reach it ends the run with its status, and so does one whose 8 s wait woke 20 ms past an 8.01 s timeout, with no second POST; a poll or a download whose wait woke past the timeout is not sent, and the run ends on the last one as it came), `tests/gen/test_pixel_draw.py` (each batch read is handed the end of `--batch-wait`). `tests/http_answer.py` is one offline HTTP answer the provider tests share, read whole or a chunk at a time under a deadline.- **Behavior change: requests can pack sound without images.** Previously, a version 1 request required a nonempty `assets` list (refused with "assets must be a nonempty list"), and version 2 required at least one asset, image, tile, UI kit or icon (refused with "game pack request declares nothing to pack"), preventing games drawn in code from packing audio alone. In both versions, `assets` may now be empty or absent when sound is declared (see [docs/game-pack.md#sound-only-packs](docs/game-pack.md#sound-only-packs)). A sound-only request writes exactly `game-pack.json`, `audio-<id>.ogg`, `audio-<id>.m4a` and `sprite-gen-audio-v2.mjs` without atlas pages; its manifest sets `textures` to `[]` and `animations` to `{}` (version 2 also keeps empty `images`, `tiles` and `ui`), while audio entries, sound files and helper module bytes remain identical to packs with images. Requests declaring no assets, scene entries or sounds are refused in both versions with "game pack request declares nothing to pack: give it an asset, a scene entry or a sound" and write nothing. Packs with images (with or without sound) and packs without sound remain byte-for-byte unchanged, and repacking a sound-only request returns the same output. Tests: `tests/compose/test_game_pack_audio.py` (sound-only requests in versions 1 and 2 with `assets` empty or absent write only the manifest, sound files and helper; manifest matches image packs except for empty `textures` and `animations`; sound files and helper match bytes; repacking returns identical results; requests with neither images nor sound are refused without creating an output directory for version 1 and version 2). Synthetic fixtures only.- **New verb: `video-stop` cuts a walk or run that stops from the clip's own start played backwards.** Given the clip's keyed frames and a walk or run loop (`--frames-dir`, `--loop`), it writes a one-shot strip (`<name>.strip.png`), sidecar (`loop: false`, `hold_last: true`), GIF, WebP, and report (`<name>.stop.report.json`). The stop leaves the loop from the cell nearest the entry (`from_cell`), the first frame after the opening rest within `SAME_POSE` (0.35 steps) of a loop cell, and ends on the clip's first frame, the still as filmed. A seam past the jump line (1.4 steps, `--seam-max`) is refused by `video-stop` with exit code 1 and `status: "failed"` in the report, and so is another motion in the run-up between the rest and the entry, however short: a run-up frame back within `SAME_POSE` (0.35 steps) of a pose the clip had left by `TURN_BACK_AWAY` (one step) or more (the still, or the run-up's own), refused with "the run-up turns back — another motion, that the stop would play backwards", naming the three frames (`turn_back`, the nearest such comeback, is reported either way). A run-up longer than `RUN_UP_MAX_CYCLES` (1.5 cycles) — `source_cut.length` frames, overridden by `--run-up-max` — which plays strides unlike the loop's, is refused with "strides unlike the loop's, that the stop would play backwards"; `--run-up-max` bounds only that length and never lets another motion through. The seam refusal, the another-motion refusal and the run-up refusal can all be named in one message. Drift is taken out by laying the entry on its cell and the first frame on the loop's ground and mean coverage, moving frames between by a straight share of the difference per step; a run leaning away moves the still a few pixels, and drift not straight leaves a little of itself. Strips wider than 32,000 px are refused, `--speed 2` plays every other frame (still kept last), `--max-frames N` spaces frames evenly, and frames shown pass the ghost screen (`interpolation_quality`) as filmed. The pose thumbnails (`gait_fallback.poses`, 48 px cut about the feet) are now public, as the verb reads poses with them ([video-pipeline](docs/video-pipeline.md#8-a-walk-or-run-that-stops--video-stop)).+- **Fix: the poses of one image row keep one size and stay on their anchor.** Each pose of a row was shrunk into its cell on its own, so a raised or extended weapon shrank that pose's body (PR #107 gave the row one shrink ratio). That ratio fitted the widest pose by its own width, and `fit_to_cell` then placed it with its `align_x` anchor (the legs, by default) on the cell's centre line: a weapon held far out crossed the cell and the pose was pushed back in, body and all, so the body jumped back in that frame alone (a greatsword thrust: 39 px in a 256 cell). The ratio is now one function, `fit_scale`, which keeps every pose inside the safe area where its anchor puts it; `fit_to_cell` takes it for a frame alone, the row takes it over every pose, and `qa/inspect` fits a row the same way instead of pose by pose. With `bbox-center` nothing changes. A weapon held far to one side now costs size in a square cell: give such a row a wider cell ([pixel-unfake](docs/pixel-unfake.md)). Test: `tests/frames/test_row_scale.py`.- **New: a motion plays as its request says — `--play loop|once-return|once-hold|lifetime` — and `video-set --motion` gives each state or item its own motion text.** A state's name used to fix its sentence, its end-frame pin, its cut and whether its strip loops, so every new motion (a punch, typing at a desk, a block, a tool swing) was a change to the engine. `video-set --play STATE=MODE` or `ITEM=MODE` and `video-prompt --play MODE` take the mode as a value of the request, and `video-set --motion STATE=TEXT` or `ITEM=TEXT` the caller's words in place of the state's sentence (an item's own value wins over its state's; a key that names neither, a key given twice or a mode the state cannot play is refused before any clip). A word the engine does not name is filmed under that name in that mode: a once-return is pinned to end on its first frame, asked for as one action that returns to the first pose, and cut whole by the new `video-loop --cycle once-return` (every frame, the last too, gated by the pin), its `strip.json` saying `loop: false`, `hold_last: false`, `returns: true` with the first pose in its first and last cells; a once-hold is cut at its held pose (`once`), the first settled pose whose drawing is not its first — a shield raised inside a body outline that keeps its size and floor is one, the first drawing come back, in place or slid sideways, is not — while the six terminal states keep their body-geometry screen (any other state cut `--cycle once` is screened by its drawing too); a lifetime is framed and filmed as an effect whatever its name; a loop asked for is searched as a loop (`periodic`) and never failed over to a single play. Named states are presets: without `--play` every state is filmed, prompted and cut as before, and an attack played once-return is the same prompt and the same pinned clip, cut whole and played once. `effect`, the six terminal states and the cyclic states refuse any other mode. `set.report.json` records each item's `play` (and `motion`); `video-prompt --json` records `play`. Not measured on a clip model: the sentences for a motion of your own ([video-pipeline](docs/video-pipeline.md#play-modes)).- **Tests** — `tests/video/test_play.py` (every named state given no mode is pinned, prompted and cut as before, and the same when its preset is named, save that a loop asked for is never failed over; a mode a state cannot play is refused; a once-return asked for pinned with its return sentence and cut whole, or searched where the source cannot pin; the attack played once-return is the attack's prompt; a named state played another way takes the mode's sentence; a once-hold with its end frame; a once-hold of your own ending on a new drawing inside the same body box cut at its held pose, the same clip as `cast` still refused by its body geometry, and one that comes back to its first drawing, in place or slid sideways, refused; a lifetime filmed as an effect whatever its name; a loop of your own; `video-set` modes and texts per state and per item, refused before any clip when a key or mode is wrong; a once-return that does not come back refused; a once-return cut subsampled with both ends kept and played once; the CLIs) and the once-return item filmed end to end in `tests/video/test_clip_prompt.py`, its strip's first and last cells the first pose. Synthetic fixtures only.- **Behavior change: the UI kit cut reads the drawing with one exact judgment instead of a centring heuristic.** Previously, each part went to the box that holds its centroid, with every box widened by half the gap. When that left a box empty, the layout was fitted to the drawing (least squares of box centres to the centres of what the boxes held, then assign again, at most 8 rounds) and kept when it left fewer required boxes empty, or as many and fewer boxes in all, reported as `layout_fit`. The only refusal of the cut was an empty frame. With several band elements missing on a drifted sheet, a drawn element could be cut under a neighbour's name, a drawn part named missing, or an undrawn element present; a drawing that fit the layout in two ways was cut silently under one of them; and a fit dragged by a small element placed off its box's centre gave wrong readings. The cut is the one reading within 1 px of the least cost, found by one exact search over the drifts the cut considers (a scale from 0.875 to 1.125 and an offset of at most 120 px at 1024), and every refusal comes from that one result ([ui-kit](docs/ui-kit.md#reading-the-drawing)). Every drawn element is cut under its own name, or the sheet is refused. A drawing that fits the layout in two ways is refused, naming both readings. New refusals: drifted further, ambiguous, off-centre, and off the layout. A sheet drawn on its guide is cut exactly as before, report bytes included. A sheet whose every part lies at least 1 px inside its box under some drift within the accepted range (along each axis a scale from 0.9 to 1.1 and an offset of at most 96 px at 1024), with all seven frames drawn, is cut exactly as drawn or refused as fitting the layout in two ways, never refused for another reason. A sheet with stray marks (parts in no box under the drift it was drawn with) is outside it.diff --cc tests/gen/test_grok_provider.pyindex 46ee98a,0b51a09..cb93538--- a/tests/gen/test_grok_provider.py+++ b/tests/gen/test_grok_provider.py@@@ -138,7 -127,56 +129,56 @@@ def test_a_rate_limit_still_there_afterassert len(api["calls"]) == 3 and api["slept"] == [8.0, 16.0]+ def test_a_rate_limit_is_sent_again_only_within_the_time_limit_of_the_image(tmp_path, api, monkeypatch, capsys):+ """SPRITE_GEN_GEN_TIMEOUT_SECONDS is the image's time limit from its first send (gen.retry): the 8 s wait+ leaves 12 s of 20 for the second send to wait in, the 16 s one would reach it, so the second 429 ends+ the call with its status and its stderr line says why it was not sent again."""+ now = [0.0]++ def sleep(seconds):+ api["slept"].append(seconds)+ now[0] += seconds++ for module in (grok, retry, xai):+ monkeypatch.setattr(module, "time", SimpleNamespace(monotonic=lambda: now[0]))+ monkeypatch.setattr(retry, "_sleep", sleep)+ monkeypatch.setattr(grok, "GEN_TIMEOUT_SECONDS", 20)+ limited = (429, {"code": "resource-exhausted", "error": "slow down"})+ api["script"] = [limited, limited, (200, api["body"])]+ with pytest.raises(SystemExit) as error:+ gen.generate_image("grok", "x", tmp_path / "out.png")+ assert [timeout for _request, timeout in api["calls"]] == [20, 12] and api["slept"] == [8.0]+ assert "(HTTP 429) code=resource-exhausted; sent 2 times, no provider fallback" in str(error.value)+ assert ("[gen] grok refused the request (429 resource-exhausted); nothing was made — not asking again: "+ "the 16s wait would reach the time limit (12s left)") in capsys.readouterr().err+++ def test_a_rate_limit_whose_wait_woke_past_the_time_limit_is_not_sent_again(tmp_path, api, monkeypatch, capsys):+ """A sleep may wake later than asked: with 8.01 s, the 8 s wait after a 429 woke 20 ms late, past the+ image's time limit. Nothing is sent then: the 429 ends the call as it came and its stderr line says why —+ not a send that finds no time left and ends as a request the server may have taken."""+ now = [0.0]++ def sleep(seconds):+ api["slept"].append(seconds)+ now[0] += seconds + 0.020++ for module in (grok, retry, xai):+ monkeypatch.setattr(module, "time", SimpleNamespace(monotonic=lambda: now[0]))+ monkeypatch.setattr(retry, "_sleep", sleep)+ monkeypatch.setattr(grok, "GEN_TIMEOUT_SECONDS", 8.010)+ api["script"] = [(429, {"code": "resource-exhausted", "error": "slow down"}), (200, api["body"])]+ with pytest.raises(SystemExit) as error:+ gen.generate_image("grok", "x", tmp_path / "out.png")+ assert len(api["calls"]) == 1 and api["slept"] == [8.0]+ assert "(HTTP 429) code=resource-exhausted; no retry or provider fallback" in str(error.value)+ assert capsys.readouterr().err.splitlines()[-1] == (+ "[gen] grok refused the request (429 resource-exhausted); nothing was made — not asking again: "+ "the time limit came during the 8s wait")+ assert not (tmp_path / "out.png").exists()++[email protected]("body", [{}, {"data": []}, {"data": [None]}, {"data": [{}, {}]},[email protected]("body", [{}, {"data": [None]}, {"data": [{}, {}]},{"data": [{"b64_json": "???"}]},{"data": [{"b64_json": base64.b64encode(b"invalid image").decode()}]},{"data": [{"url": "https://signed.invalid/private"}]}])diff --cc tests/gen/test_openai_provider.pyindex e8404a8,06018f3..964a7fc--- a/tests/gen/test_openai_provider.py+++ b/tests/gen/test_openai_provider.py@@@ -222,7 -219,64 +206,64 @@@ def test_a_refusal_still_there_after_twassert len(api["calls"]) == 3 and api["slept"] == [8.0, 16.0]+ def test_a_refusal_is_sent_again_only_within_the_time_limit_of_the_image(tmp_path, api, monkeypatch, capsys):+ """SPRITE_GEN_GEN_TIMEOUT_SECONDS is the image's time limit from its first send (gen.retry): the waits+ before a refused request goes again count in it and each send waits only what is left of it, so a+ refusal whose next wait would reach it ends the call, its line saying why, before a caller's ceiling."""+ now = [0.0]++ def sleep(seconds):+ api["slept"].append(seconds)+ now[0] += seconds++ for module in (openai, retry, xai):+ monkeypatch.setattr(module, "time", SimpleNamespace(monotonic=lambda: now[0]))+ monkeypatch.setattr(retry, "_sleep", sleep)+ monkeypatch.setattr(openai, "GEN_TIMEOUT_SECONDS", 20)+ limited = (429, {"error": {"type": "rate_limit_error", "code": "rate_limit_exceeded"}}, {})+ api["script"] = [limited, limited, (200, api["body"], {})]+ with pytest.raises(SystemExit) as refused:+ gen.generate_image("openai", "x", tmp_path / "out.png")+ assert [timeout for _request, timeout in api["calls"]] == [20, 12] and api["slept"] == [8.0]+ assert "(HTTP 429) code=rate_limit_exceeded; sent 2 times, no provider fallback" in str(refused.value)+ assert ("[gen] openai refused the request (429 rate_limit_exceeded); nothing was made — not asking again: "+ "the 16s wait would reach the time limit (12s left)") in capsys.readouterr().err+ assert not (tmp_path / "out.png").exists()+++ @pytest.mark.parametrize("call", ["image", "facing check"])+ def test_a_refusal_whose_wait_woke_past_the_time_limit_is_not_sent_again(tmp_path, api, monkeypatch, capsys, call):+ """A sleep may wake later than asked: with 8.01 s, the 8 s wait after a 429 woke 20 ms late, past the time+ limit. The limit is read again right before the send, so nothing is sent and the 429 ends the call as it+ came, its stderr line saying why — not a send that finds no time left and ends as a request the server may+ have taken."""+ now = [0.0]++ def sleep(seconds):+ api["slept"].append(seconds)+ now[0] += seconds + 0.020++ for module in (openai, retry, xai):+ monkeypatch.setattr(module, "time", SimpleNamespace(monotonic=lambda: now[0]))+ monkeypatch.setattr(retry, "_sleep", sleep)+ monkeypatch.setattr(openai, "GEN_TIMEOUT_SECONDS", 8.010)+ limited = (429, {"error": {"type": "rate_limit_error", "code": "rate_limit_exceeded"}}, {})+ api["script"] = [limited, (200, api["body"], {})]+ Image.new("RGB", (4, 4), (20, 90, 200)).save(tmp_path / "raw.png")+ with pytest.raises(SystemExit) as refused:+ if call == "image":+ gen.generate_image("openai", "x", tmp_path / "out.png")+ else:+ openai.OpenAIProvider().inspect_facing(tmp_path / "raw.png", tmp_path)+ assert len(api["calls"]) == 1 and api["slept"] == [8.0]+ assert "HTTP 429" in str(refused.value) and "request failed or timed out" not in str(refused.value)+ assert capsys.readouterr().err.splitlines()[-1] == (+ "[gen] openai refused the request (429 rate_limit_exceeded); nothing was made — not asking again: "+ "the time limit came during the 8s wait")+ assert not (tmp_path / "out.png").exists()++[email protected]("body", [{}, {"data": []}, {"data": [None]}, {"data": [{}, {}]},[email protected]("body", [{}, {"data": [None]}, {"data": [{}, {}]},{"data": [{"b64_json": "???"}]},{"data": [{"b64_json": base64.b64encode(b"invalid image").decode()}]},{"data": [{"url": "https://signed.invalid/private"}]}])