Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
feat(video): every walk and run moves naturally in place, and a caller's own gait is held there
Soohong Kimbc76fe2905de
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 4a244d1..d03801d 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -113,6 +113,15 @@ steps and a loose arm swing kept the feet low but shuffled. Two clips each is athe default is generic on purpose, because how a character walks (high steps, a stroll, amarch) is the caller's to say. The front run keeps its sentence.+Since 2.16.0 every walk and run takes that form, from every view: the side walk "walks+naturally in place, as if on a treadmill, without moving across the screen" (it said "moves in+place on a treadmill" as a place and asked for "an even left-right or front-back rhythm"), the+side run "runs naturally in place …", and the front and back runs "runs naturally in place,+facing (away from) the viewer, as if on a treadmill …" (the front run spelled its strides out+and said "toward the viewer" and "without coming any closer" in one clause). On one character,+480p 3 s on the API: the front run kept facing in 2 of 2, the side run ran in 2 of 2, the back+run kept facing away in the one clip whose frames passed; the side walk walked in 2 of 2.+An attack is one timed strike, not a repeat. `MOTION_TEXT["attack"]` asks for one attack: a windup(about 0.5 s), one strike in front (about 0.25 s), a held impact pose (about 0.3 s) and a recoveryto the exact starting stance (about 0.5 s), then `HOLD_TEXT["attack"]`:@@ -148,7 +157,19 @@ it for a repeat.`build_prompt(direction, state, character, facing, motion=...)` takes a caller's own motionparagraph — whole sentences about the subject, such as a request interpreter writes perrequest — in place of the built-in state sentence. The frame, camera, background and design-rules stay the engine's, and an attack keeps its `HOLD_TEXT` sentence after the paragraph.+rules stay the engine's, and an attack keeps its `HOLD_TEXT` sentence after the paragraph. A+walk or run gets `GAIT_HOLD_TEXT` for its view after the paragraph instead: it stays in place as+if on a treadmill and keeps facing the viewer, away from the viewer or `{facing}` the whole time,+so a caller describes the gait (a sneak, a march, a lazy stroll) and the engine keeps it on the+spot. On one character, one clip each: a march lifted the knees high as asked and stayed in+place, a lazy stroll seen from behind kept facing away, and a tiptoe sneak kept facing the viewer+but read only as a slightly hunched walk.++`build_prompt(..., pinned=True)` says the clip is pinned to end on its first frame, so any state+gets `PINNED_LOOP_TEXT` (the return to the first pose) instead of the evenly paced repeat; left+out, only `PINNED_LOOP_STATES` do. A caller that retries a walk pinned after no cycle was found+says True; two such front-walk clips closed on their first frame (seam 0.12 and 0.24 of an+ordinary step).## 3. Frames — extract, key, check the edgesdiff --git a/sprite_gen/video/batch.py b/sprite_gen/video/batch.pyindex d500dcb..e1016f7 100644--- a/sprite_gen/video/batch.py+++ b/sprite_gen/video/batch.py@@ -71,6 +71,18 @@ HOLD_TEXT = {"drawn, with anything it holds, and the body keeps facing the same direction without turning."),}+# What stays put while a caller's own walk or run moves, by view: in place, and facing the way the image+# faces. A request interpreter may ask for any gait (a sneak, a march, a stroll); the engine keeps it on the+# spot and turned the right way, as the built-in gait sentences do. `{facing}` is the side view's.+GAIT_STATES = frozenset({"walk", "run"})+GAIT_HOLD_TEXT = {+ "front": "It stays in place as if on a treadmill, without coming any closer, and keeps facing the viewer the whole time.",+ "back": (+ "It stays in place as if on a treadmill, without moving any farther away, and keeps facing away from the viewer "+ "the whole time."+ ),+ "side": "It stays in place as if on a treadmill, without moving across the screen, and keeps facing {facing} the whole time.",+}MOTION_TEXT = {# A side-view full body asked for "a subtle weight sway" and an evenly paced loop steps in place# more often than not, so the feet are held and walking is named as what not to do.@@ -80,8 +92,8 @@ MOTION_TEXT = {"hair and loose cloth, and one natural blink if the face has eyes. The feet never lift, step, shuffle or slide ""— no walking, no marching in place, no turning."),- "walk": "moves in place on a treadmill: a steady locomotion cycle for this body type with clear repeating ground contacts and an even left-right or front-back rhythm the body already has.",- "run": "moves in place on a treadmill: a fast locomotion cycle for this body type with a bounding rhythm and clear repeating ground contacts.",+ "walk": "walks naturally in place, as if on a treadmill, without moving across the screen.",+ "run": "runs naturally in place, as if on a treadmill, without moving across the screen.","jump": "performs a modest vertical hop in place over and over: compress, spring up about half the body height, land softly, return to the exact starting stance, repeat at an even rhythm. Same height every time.","attack": ("performs one melee attack with what it is already holding (bare hands only if it holds nothing), keeping "@@ -92,25 +104,23 @@ MOTION_TEXT = {"cheer": "celebrates in place: rises into a raised, spread-out cheer pose, holds it for a beat, then settles back to the exact starting stance, repeating at an even rhythm.","wave": "waves in place: lifts one side into a friendly wave, sways it a few times, then settles back to the exact starting stance, repeating at an even rhythm.",}-# A walk seen from the front or from behind, and a run seen from the front, in place of the body-neutral-# sentences above. Asked for "an even left-right or front-back rhythm", a walk that faces the viewer-# stepped sideways, crossed its feet or turned the body to the side in most of the takes. The walk says-# only that it walks naturally, which way it faces and that it stays in place: the 2.13 sentence spelled out-# the steps ("each one lifting and landing straight forward and back") and some takes marched in place,-# knees up to the waist and arms stiff, while "walks naturally" swung the arms and kept the knees low. How a-# character walks is the caller's to say; the engine's default stays generic. The run keeps saying which way-# its strides go and that the body does not turn. A side view keeps MOTION_TEXT, and so does a run seen-# from behind, which was not measured. "as if on a treadmill" keeps the treadmill a comparison: said as a-# place, it was sometimes drawn under the feet.+# A walk or run seen from the front or from behind, in place of the side view's sentences above. Every+# gait says only that it moves naturally, which way it faces and that it stays in place. Asked for "an+# even left-right or front-back rhythm", a walk that faces the viewer stepped sideways or turned the body+# (2.13 fixed the facing by naming it); spelling the steps out ("each one lifting and landing straight+# forward and back") then made some takes march in place, knees up to the waist and arms stiff, while+# "walks naturally" swung the arms and kept the knees low (2.15). 2.16 gives the run and the side view the+# same form. How a character walks is the caller's to say (`build_prompt(motion=...)`, held in place by+# `GAIT_HOLD_TEXT`); the engine's default stays generic. "as if on a treadmill" keeps the treadmill a+# comparison: said as a place, it was sometimes drawn under the feet.VIEW_MOTION_TEXT = {("walk", "front"): "walks naturally in place, facing the viewer, as if on a treadmill, without coming any closer.",("walk", "back"): ("walks naturally in place, facing away from the viewer, as if on a treadmill, without moving any farther away."),- ("run", "front"): (- "runs in place toward the viewer, as if on a treadmill, without coming any closer: a fast run cycle for this "- "body type in which every stride lifts and lands straight forward and back under the body, with clear "- "repeating ground contacts. It never turns to the side, never steps sideways and never crosses its feet."+ ("run", "front"): "runs naturally in place, facing the viewer, as if on a treadmill, without coming any closer.",+ ("run", "back"): (+ "runs naturally in place, facing away from the viewer, as if on a treadmill, without moving any farther away."),}COMMON_TEXT = (@@ -154,19 +164,25 @@ def _staggered_start(gap: float) -> None:_last_start[0] = time.monotonic()-def build_prompt(direction: str, state: str, character: str | None, facing: str = "right", motion: str | None = None) -> str:+def build_prompt(direction: str, state: str, character: str | None, facing: str = "right", motion: str | None = None,+ pinned: bool | None = None) -> str:"""The clip prompt for one (direction, state).`motion` replaces the built-in state sentence with the caller's own description of themotion, written as whole sentences about the subject (a request interpreter's output, for- instance). What stays put (`HOLD_TEXT`), the repeat count (`REPEAT_TEXT`) and the frame,- camera, background and design rules stay the engine's.+ instance). What stays put (`HOLD_TEXT`; for a walk or run, in place and facing the image's way,+ `GAIT_HOLD_TEXT`), the repeat count (`REPEAT_TEXT`) and the frame, camera, background and+ design rules stay the engine's.++ `pinned` says the clip is pinned to end on its first frame (`video --last-frame`): it then asks+ for the return to the first pose instead of an evenly paced repeat (`PINNED_LOOP_TEXT`). None+ leaves it to the state (`PINNED_LOOP_STATES`); a caller that pins a walk or run says True."""validate_facing(facing)view = VIEW_TEXT.get(direction, f"seen from the {direction}").format(facing=facing)if state in ACTION_TEXT_STATES:template = ACTION_COMMON_TEXT- elif state in PINNED_LOOP_STATES:+ elif (state in PINNED_LOOP_STATES) if pinned is None else pinned:template = PINNED_LOOP_TEXTelse:template = COMMON_TEXT@@ -176,6 +192,8 @@ def build_prompt(direction: str, state: str, character: str | None, facing: strraise SystemExit("video: motion description is empty")head = "2D game sprite animation. The character {motion} The character is {view}."hold = f" {HOLD_TEXT[state]}" if state in HOLD_TEXT else ""+ if state in GAIT_STATES and direction in GAIT_HOLD_TEXT:+ hold += f" {GAIT_HOLD_TEXT[direction].format(facing=facing)}"repeat = f" {REPEAT_TEXT[state]}" if state in REPEAT_TEXT else ""subject = character.strip().rstrip(".") if character else "The character"return f"2D game sprite animation. {motion}{hold}{repeat} {subject} is {view}." + template[len(head):]diff --git a/tests/video/test_video_pipeline.py b/tests/video/test_video_pipeline.pyindex 0c54ca1..fd54b59 100644--- a/tests/video/test_video_pipeline.py+++ b/tests/video/test_video_pipeline.py@@ -481,36 +481,62 @@ def test_drop_specks_erases_detached_slivers_only() -> None:def test_prompt_uses_state_and_view_and_optional_character() -> None:p = batch_mod.build_prompt("side", "walk", "The armored knight")- assert p.startswith("2D game sprite animation. The armored knight moves in place on a treadmill")+ assert p.startswith("2D game sprite animation. The armored knight walks naturally in place, as if on a treadmill")assert "seen from the exact side" in passert "The character" in batch_mod.build_prompt("back", "jump", None)-def test_front_and_back_gaits_say_which_way_the_steps_go() -> None:+def test_every_gait_walks_or_runs_naturally_in_place() -> None:+ """Since 2.16.0 every walk and run, from any view, says only that it moves naturally, which way it+ faces and that it stays in place: spelling the steps out made takes march, and "on a treadmill"+ said as a place, or "an even left-right rhythm", are gone from every view."""for state, direction, opening in (("walk", "front", "walks naturally in place, facing the viewer"),("walk", "back", "walks naturally in place, facing away from the viewer"),- ("run", "front", "runs in place toward the viewer"),+ ("walk", "side", "walks naturally in place"),+ ("run", "front", "runs naturally in place, facing the viewer"),+ ("run", "back", "runs naturally in place, facing away from the viewer"),+ ("run", "side", "runs naturally in place"),):- sentence = batch_mod.VIEW_MOTION_TEXT[(state, direction)]+ sentence = batch_mod.VIEW_MOTION_TEXT.get((state, direction)) or batch_mod.MOTION_TEXT[state]p = batch_mod.build_prompt(direction, state, None)- assert p == batch_mod.COMMON_TEXT.format(motion=sentence, view=batch_mod.VIEW_TEXT[direction])- assert f"{opening}, as if on a treadmill" in p- assert "left-right" not in p and "on a treadmill:" not in p- # A walk says only that it walks naturally: how the steps go is the caller's to say. The run- # keeps naming its strides and that the body does not turn.- if state == "walk":- assert sentence.startswith(opening) and sentence.count(".") == 1- assert ("It never turns to the side, never steps sideways" in p) == (state == "run")- # A side view, a run from behind and every other front or back state keep the state's own sentence.- assert batch_mod.MOTION_TEXT["walk"] in batch_mod.build_prompt("side", "walk", None)- assert batch_mod.MOTION_TEXT["run"] in batch_mod.build_prompt("side", "run", None)- assert batch_mod.MOTION_TEXT["run"] in batch_mod.build_prompt("back", "run", None)+ assert p == batch_mod.COMMON_TEXT.format(motion=sentence, view=batch_mod.VIEW_TEXT[direction].format(facing="right"))+ assert sentence.startswith(opening) and sentence.count(".") == 1+ assert ", as if on a treadmill" in p+ for gone in ("left-right", "on a treadmill:", "lifts and lands", "never turns to the side"):+ assert gone not in passert batch_mod.MOTION_TEXT["idle"] in batch_mod.build_prompt("back", "idle", None)named = batch_mod.build_prompt("back", "walk", "The armored knight")assert named.startswith("2D game sprite animation. The armored knight walks naturally in place, facing away from the viewer")+def test_a_callers_gait_is_held_in_place_and_facing_the_images_way() -> None:+ """A request interpreter's own walk or run (a sneak, a march) is followed by the engine's in-place+ and facing sentence for the view; any other state's paragraph is not."""+ sneak = "The knight sneaks along on tiptoe, setting each foot down slowly."+ for direction in ("front", "back", "side"):+ for state in ("walk", "run"):+ p = batch_mod.build_prompt(direction, state, "The knight", "left", motion=sneak)+ hold = batch_mod.GAIT_HOLD_TEXT[direction].format(facing="left")+ assert p.startswith(f"2D game sprite animation. {sneak} {hold} The knight is {batch_mod.VIEW_TEXT[direction].format(facing='left')}.")+ assert batch_mod.MOTION_TEXT[state] not in p+ assert "keeps facing left the whole time" in batch_mod.GAIT_HOLD_TEXT["side"].format(facing="left")+ jump = batch_mod.build_prompt("front", "jump", None, motion="The knight hops twice.")+ assert all(text.split(",")[0] not in jump for text in batch_mod.GAIT_HOLD_TEXT.values())+++def test_pinned_picks_the_return_sentence_for_any_state() -> None:+ """A walk or run pinned to end on its first frame asks for the return, not an even repeat; left+ unsaid, only the states pinned by default do."""+ walk = batch_mod.build_prompt("front", "walk", None, pinned=True)+ assert "The last frame returns to the exact pose of the first frame" in walk and "evenly paced" not in walk+ assert "evenly paced" in batch_mod.build_prompt("front", "walk", None)+ assert batch_mod.build_prompt("side", "idle", None) == batch_mod.build_prompt("side", "idle", None, pinned=True)+ assert "evenly paced" in batch_mod.build_prompt("side", "idle", None, pinned=False)+ # An attack keeps its action template whichever way it is pinned.+ assert batch_mod.build_prompt("side", "attack", None, pinned=False) == batch_mod.build_prompt("side", "attack", None)++def test_motion_templates_do_not_assume_a_body_plan() -> None:# the templates were first written for a biped; a quadruped or a legless blob must not# be prompted into a contradiction (2026-09-09 generalization run)