Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

feat(video): every walk and run moves naturally in place, and a caller's own gait is held there

Soohong Kimbc76fe2905de
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 4a244d1..d03801d 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -113,6 +113,15 @@ steps and a loose arm swing kept the feet low but shuffled. Two clips each is a
the default is generic on purpose, because how a character walks (high steps, a stroll, a
march) is the caller's to say. The front run keeps its sentence.
+Since 2.16.0 every walk and run takes that form, from every view: the side walk "walks
+naturally in place, as if on a treadmill, without moving across the screen" (it said "moves in
+place on a treadmill" as a place and asked for "an even left-right or front-back rhythm"), the
+side run "runs naturally in place …", and the front and back runs "runs naturally in place,
+facing (away from) the viewer, as if on a treadmill …" (the front run spelled its strides out
+and said "toward the viewer" and "without coming any closer" in one clause). On one character,
+480p 3 s on the API: the front run kept facing in 2 of 2, the side run ran in 2 of 2, the back
+run kept facing away in the one clip whose frames passed; the side walk walked in 2 of 2.
+
An attack is one timed strike, not a repeat. `MOTION_TEXT["attack"]` asks for one attack: a windup
(about 0.5 s), one strike in front (about 0.25 s), a held impact pose (about 0.3 s) and a recovery
to the exact starting stance (about 0.5 s), then `HOLD_TEXT["attack"]`:
@@ -148,7 +157,19 @@ it for a repeat.
`build_prompt(direction, state, character, facing, motion=...)` takes a caller's own motion
paragraph — whole sentences about the subject, such as a request interpreter writes per
request — in place of the built-in state sentence. The frame, camera, background and design
-rules stay the engine's, and an attack keeps its `HOLD_TEXT` sentence after the paragraph.
+rules stay the engine's, and an attack keeps its `HOLD_TEXT` sentence after the paragraph. A
+walk or run gets `GAIT_HOLD_TEXT` for its view after the paragraph instead: it stays in place as
+if on a treadmill and keeps facing the viewer, away from the viewer or `{facing}` the whole time,
+so a caller describes the gait (a sneak, a march, a lazy stroll) and the engine keeps it on the
+spot. On one character, one clip each: a march lifted the knees high as asked and stayed in
+place, a lazy stroll seen from behind kept facing away, and a tiptoe sneak kept facing the viewer
+but read only as a slightly hunched walk.
+
+`build_prompt(..., pinned=True)` says the clip is pinned to end on its first frame, so any state
+gets `PINNED_LOOP_TEXT` (the return to the first pose) instead of the evenly paced repeat; left
+out, only `PINNED_LOOP_STATES` do. A caller that retries a walk pinned after no cycle was found
+says True; two such front-walk clips closed on their first frame (seam 0.12 and 0.24 of an
+ordinary step).
## 3. Frames — extract, key, check the edges
diff --git a/sprite_gen/video/batch.py b/sprite_gen/video/batch.py
index d500dcb..e1016f7 100644
--- a/sprite_gen/video/batch.py
+++ b/sprite_gen/video/batch.py
@@ -71,6 +71,18 @@ HOLD_TEXT = {
"drawn, with anything it holds, and the body keeps facing the same direction without turning."
),
}
+# What stays put while a caller's own walk or run moves, by view: in place, and facing the way the image
+# faces. A request interpreter may ask for any gait (a sneak, a march, a stroll); the engine keeps it on the
+# spot and turned the right way, as the built-in gait sentences do. `{facing}` is the side view's.
+GAIT_STATES = frozenset({"walk", "run"})
+GAIT_HOLD_TEXT = {
+ "front": "It stays in place as if on a treadmill, without coming any closer, and keeps facing the viewer the whole time.",
+ "back": (
+ "It stays in place as if on a treadmill, without moving any farther away, and keeps facing away from the viewer "
+ "the whole time."
+ ),
+ "side": "It stays in place as if on a treadmill, without moving across the screen, and keeps facing {facing} the whole time.",
+}
MOTION_TEXT = {
# A side-view full body asked for "a subtle weight sway" and an evenly paced loop steps in place
# more often than not, so the feet are held and walking is named as what not to do.
@@ -80,8 +92,8 @@ MOTION_TEXT = {
"hair and loose cloth, and one natural blink if the face has eyes. The feet never lift, step, shuffle or slide "
"— no walking, no marching in place, no turning."
),
- "walk": "moves in place on a treadmill: a steady locomotion cycle for this body type with clear repeating ground contacts and an even left-right or front-back rhythm the body already has.",
- "run": "moves in place on a treadmill: a fast locomotion cycle for this body type with a bounding rhythm and clear repeating ground contacts.",
+ "walk": "walks naturally in place, as if on a treadmill, without moving across the screen.",
+ "run": "runs naturally in place, as if on a treadmill, without moving across the screen.",
"jump": "performs a modest vertical hop in place over and over: compress, spring up about half the body height, land softly, return to the exact starting stance, repeat at an even rhythm. Same height every time.",
"attack": (
"performs one melee attack with what it is already holding (bare hands only if it holds nothing), keeping "
@@ -92,25 +104,23 @@ MOTION_TEXT = {
"cheer": "celebrates in place: rises into a raised, spread-out cheer pose, holds it for a beat, then settles back to the exact starting stance, repeating at an even rhythm.",
"wave": "waves in place: lifts one side into a friendly wave, sways it a few times, then settles back to the exact starting stance, repeating at an even rhythm.",
}
-# A walk seen from the front or from behind, and a run seen from the front, in place of the body-neutral
-# sentences above. Asked for "an even left-right or front-back rhythm", a walk that faces the viewer
-# stepped sideways, crossed its feet or turned the body to the side in most of the takes. The walk says
-# only that it walks naturally, which way it faces and that it stays in place: the 2.13 sentence spelled out
-# the steps ("each one lifting and landing straight forward and back") and some takes marched in place,
-# knees up to the waist and arms stiff, while "walks naturally" swung the arms and kept the knees low. How a
-# character walks is the caller's to say; the engine's default stays generic. The run keeps saying which way
-# its strides go and that the body does not turn. A side view keeps MOTION_TEXT, and so does a run seen
-# from behind, which was not measured. "as if on a treadmill" keeps the treadmill a comparison: said as a
-# place, it was sometimes drawn under the feet.
+# A walk or run seen from the front or from behind, in place of the side view's sentences above. Every
+# gait says only that it moves naturally, which way it faces and that it stays in place. Asked for "an
+# even left-right or front-back rhythm", a walk that faces the viewer stepped sideways or turned the body
+# (2.13 fixed the facing by naming it); spelling the steps out ("each one lifting and landing straight
+# forward and back") then made some takes march in place, knees up to the waist and arms stiff, while
+# "walks naturally" swung the arms and kept the knees low (2.15). 2.16 gives the run and the side view the
+# same form. How a character walks is the caller's to say (`build_prompt(motion=...)`, held in place by
+# `GAIT_HOLD_TEXT`); the engine's default stays generic. "as if on a treadmill" keeps the treadmill a
+# comparison: said as a place, it was sometimes drawn under the feet.
VIEW_MOTION_TEXT = {
("walk", "front"): "walks naturally in place, facing the viewer, as if on a treadmill, without coming any closer.",
("walk", "back"): (
"walks naturally in place, facing away from the viewer, as if on a treadmill, without moving any farther away."
),
- ("run", "front"): (
- "runs in place toward the viewer, as if on a treadmill, without coming any closer: a fast run cycle for this "
- "body type in which every stride lifts and lands straight forward and back under the body, with clear "
- "repeating ground contacts. It never turns to the side, never steps sideways and never crosses its feet."
+ ("run", "front"): "runs naturally in place, facing the viewer, as if on a treadmill, without coming any closer.",
+ ("run", "back"): (
+ "runs naturally in place, facing away from the viewer, as if on a treadmill, without moving any farther away."
),
}
COMMON_TEXT = (
@@ -154,19 +164,25 @@ def _staggered_start(gap: float) -> None:
_last_start[0] = time.monotonic()
-def build_prompt(direction: str, state: str, character: str | None, facing: str = "right", motion: str | None = None) -> str:
+def build_prompt(direction: str, state: str, character: str | None, facing: str = "right", motion: str | None = None,
+ pinned: bool | None = None) -> str:
"""The clip prompt for one (direction, state).
`motion` replaces the built-in state sentence with the caller's own description of the
motion, written as whole sentences about the subject (a request interpreter's output, for
- instance). What stays put (`HOLD_TEXT`), the repeat count (`REPEAT_TEXT`) and the frame,
- camera, background and design rules stay the engine's.
+ instance). What stays put (`HOLD_TEXT`; for a walk or run, in place and facing the image's way,
+ `GAIT_HOLD_TEXT`), the repeat count (`REPEAT_TEXT`) and the frame, camera, background and
+ design rules stay the engine's.
+
+ `pinned` says the clip is pinned to end on its first frame (`video --last-frame`): it then asks
+ for the return to the first pose instead of an evenly paced repeat (`PINNED_LOOP_TEXT`). None
+ leaves it to the state (`PINNED_LOOP_STATES`); a caller that pins a walk or run says True.
"""
validate_facing(facing)
view = VIEW_TEXT.get(direction, f"seen from the {direction}").format(facing=facing)
if state in ACTION_TEXT_STATES:
template = ACTION_COMMON_TEXT
- elif state in PINNED_LOOP_STATES:
+ elif (state in PINNED_LOOP_STATES) if pinned is None else pinned:
template = PINNED_LOOP_TEXT
else:
template = COMMON_TEXT
@@ -176,6 +192,8 @@ def build_prompt(direction: str, state: str, character: str | None, facing: str
raise SystemExit("video: motion description is empty")
head = "2D game sprite animation. The character {motion} The character is {view}."
hold = f" {HOLD_TEXT[state]}" if state in HOLD_TEXT else ""
+ if state in GAIT_STATES and direction in GAIT_HOLD_TEXT:
+ hold += f" {GAIT_HOLD_TEXT[direction].format(facing=facing)}"
repeat = f" {REPEAT_TEXT[state]}" if state in REPEAT_TEXT else ""
subject = character.strip().rstrip(".") if character else "The character"
return f"2D game sprite animation. {motion}{hold}{repeat} {subject} is {view}." + template[len(head):]
diff --git a/tests/video/test_video_pipeline.py b/tests/video/test_video_pipeline.py
index 0c54ca1..fd54b59 100644
--- a/tests/video/test_video_pipeline.py
+++ b/tests/video/test_video_pipeline.py
@@ -481,36 +481,62 @@ def test_drop_specks_erases_detached_slivers_only() -> None:
def test_prompt_uses_state_and_view_and_optional_character() -> None:
p = batch_mod.build_prompt("side", "walk", "The armored knight")
- assert p.startswith("2D game sprite animation. The armored knight moves in place on a treadmill")
+ assert p.startswith("2D game sprite animation. The armored knight walks naturally in place, as if on a treadmill")
assert "seen from the exact side" in p
assert "The character" in batch_mod.build_prompt("back", "jump", None)
-def test_front_and_back_gaits_say_which_way_the_steps_go() -> None:
+def test_every_gait_walks_or_runs_naturally_in_place() -> None:
+ """Since 2.16.0 every walk and run, from any view, says only that it moves naturally, which way it
+ faces and that it stays in place: spelling the steps out made takes march, and "on a treadmill"
+ said as a place, or "an even left-right rhythm", are gone from every view."""
for state, direction, opening in (
("walk", "front", "walks naturally in place, facing the viewer"),
("walk", "back", "walks naturally in place, facing away from the viewer"),
- ("run", "front", "runs in place toward the viewer"),
+ ("walk", "side", "walks naturally in place"),
+ ("run", "front", "runs naturally in place, facing the viewer"),
+ ("run", "back", "runs naturally in place, facing away from the viewer"),
+ ("run", "side", "runs naturally in place"),
):
- sentence = batch_mod.VIEW_MOTION_TEXT[(state, direction)]
+ sentence = batch_mod.VIEW_MOTION_TEXT.get((state, direction)) or batch_mod.MOTION_TEXT[state]
p = batch_mod.build_prompt(direction, state, None)
- assert p == batch_mod.COMMON_TEXT.format(motion=sentence, view=batch_mod.VIEW_TEXT[direction])
- assert f"{opening}, as if on a treadmill" in p
- assert "left-right" not in p and "on a treadmill:" not in p
- # A walk says only that it walks naturally: how the steps go is the caller's to say. The run
- # keeps naming its strides and that the body does not turn.
- if state == "walk":
- assert sentence.startswith(opening) and sentence.count(".") == 1
- assert ("It never turns to the side, never steps sideways" in p) == (state == "run")
- # A side view, a run from behind and every other front or back state keep the state's own sentence.
- assert batch_mod.MOTION_TEXT["walk"] in batch_mod.build_prompt("side", "walk", None)
- assert batch_mod.MOTION_TEXT["run"] in batch_mod.build_prompt("side", "run", None)
- assert batch_mod.MOTION_TEXT["run"] in batch_mod.build_prompt("back", "run", None)
+ assert p == batch_mod.COMMON_TEXT.format(motion=sentence, view=batch_mod.VIEW_TEXT[direction].format(facing="right"))
+ assert sentence.startswith(opening) and sentence.count(".") == 1
+ assert ", as if on a treadmill" in p
+ for gone in ("left-right", "on a treadmill:", "lifts and lands", "never turns to the side"):
+ assert gone not in p
assert batch_mod.MOTION_TEXT["idle"] in batch_mod.build_prompt("back", "idle", None)
named = batch_mod.build_prompt("back", "walk", "The armored knight")
assert named.startswith("2D game sprite animation. The armored knight walks naturally in place, facing away from the viewer")
+def test_a_callers_gait_is_held_in_place_and_facing_the_images_way() -> None:
+ """A request interpreter's own walk or run (a sneak, a march) is followed by the engine's in-place
+ and facing sentence for the view; any other state's paragraph is not."""
+ sneak = "The knight sneaks along on tiptoe, setting each foot down slowly."
+ for direction in ("front", "back", "side"):
+ for state in ("walk", "run"):
+ p = batch_mod.build_prompt(direction, state, "The knight", "left", motion=sneak)
+ hold = batch_mod.GAIT_HOLD_TEXT[direction].format(facing="left")
+ assert p.startswith(f"2D game sprite animation. {sneak} {hold} The knight is {batch_mod.VIEW_TEXT[direction].format(facing='left')}.")
+ assert batch_mod.MOTION_TEXT[state] not in p
+ assert "keeps facing left the whole time" in batch_mod.GAIT_HOLD_TEXT["side"].format(facing="left")
+ jump = batch_mod.build_prompt("front", "jump", None, motion="The knight hops twice.")
+ assert all(text.split(",")[0] not in jump for text in batch_mod.GAIT_HOLD_TEXT.values())
+
+
+def test_pinned_picks_the_return_sentence_for_any_state() -> None:
+ """A walk or run pinned to end on its first frame asks for the return, not an even repeat; left
+ unsaid, only the states pinned by default do."""
+ walk = batch_mod.build_prompt("front", "walk", None, pinned=True)
+ assert "The last frame returns to the exact pose of the first frame" in walk and "evenly paced" not in walk
+ assert "evenly paced" in batch_mod.build_prompt("front", "walk", None)
+ assert batch_mod.build_prompt("side", "idle", None) == batch_mod.build_prompt("side", "idle", None, pinned=True)
+ assert "evenly paced" in batch_mod.build_prompt("side", "idle", None, pinned=False)
+ # An attack keeps its action template whichever way it is pinned.
+ assert batch_mod.build_prompt("side", "attack", None, pinned=False) == batch_mod.build_prompt("side", "attack", None)
+
+
def test_motion_templates_do_not_assume_a_body_plan() -> None:
# the templates were first written for a biped; a quadruped or a legless blob must not
# be prompted into a contradiction (2026-09-09 generalization run)