Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

fix(cycle-align): the view picks the foot-strike signal; the swings only say whether there are legs

Soohong Kim616689aaed97
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5c42f4f..12380ee 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,7 +8,7 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
- **Fix: a black smear between crossing legs in a frame RIFE made.** RIFE runs its flow twice, once for the colour and once for the coverage, and where two legs pass each other the runs disagree: the coverage says body where the colour still carries what lay around the body. 2.24 premultiplied the colour over black, so the disagreement came out black. The colour image now carries the body's own colour around the body (`rife.bleed`: the colour from 0.8 % of the body's height inside its solid edge, past a drawn outline, pushed out over the transparent area), so a disagreement reads as the body's fill; inside the body the colour is the frame's own, outline included. On two outlined legs walking through each other the made frame at t = 0.5 had 6 % of its body dark beyond either neighbour; now none (`tests/video/test_rife.py`). A limb RIFE cannot follow is still soft and pale, never black. [docs/loop-repair.md](docs/loop-repair.md#1-rife--what-runs-where-and-what-it-costs).
- **A smear is measured on every made frame, never passed over quietly.** `cycle_align.smear` lists per made frame `dark_excess` (dark pixels inside the body beyond the darker neighbour's count — a moved hat or watch keeps its count) and `partial_excess` (part-covered pixels beyond the more ragged neighbour's); over 0.1 % dark, the frame is named in the alignment report's `warnings`, on stderr and in `video-set`'s `warnings` (`rife.smear`, `align.SMEAR_WARN`).
- **New `video-cycle-align --between nearest`** (`video-set --align-between nearest`): a time between two source frames takes the nearer source frame, so nothing is made and no RIFE is needed; the motion keeps the filmed frames up to half a frame off their time. `rife` stays the default. A loop stretched this way shows a frame twice; its GIF and WebP are checked for one frame per run of identical cells, since both hold a repeated frame as one of twice the delay.
-- **Fix: the loop started where an ear tip was lowest.** The turn took the frame whose alpha box top was lowest, and a long ear, a hat's point or an antenna owns that edge and flops on its own rhythm, so each view started at another point of its step. The start is now read off the legs (`align.foot_strike`, `turned_on` in the loop row): the widest stride where the foot band (the lowest 8 %) swings by 15 % of the body's height or more (side and diagonal views, a heel landing in front); otherwise the lowest solid row, where it swings by 1.5 % or more (front and back views, the foot nearer the viewer drawn lowest); and only for a body with no legs to read, the body's top line, the first row at least half as wide as the frame's widest, so a narrow ear is passed over. A body that hardly bobs turns on its legs as one that bobs does (`tests/video/test_cycle_align.py`: a drawn walker whose ears are lowest half a step off its heel strike, with and without a bob).
+- **Fix: the loop started where an ear tip was lowest.** The turn took the frame whose alpha box top was lowest, and a long ear, a hat's point or an antenna owns that edge and flops on its own rhythm, so each view started at another point of its step. The start is now read off the legs (`align.foot_strike`, `turned_on` in the loop row): side and diagonal views on the widest stride of the foot band (the lowest 8 %), a heel landing in front; front and back views on the lowest solid row, the foot nearer the viewer drawn lowest; and only for a body with no legs to read, the body's top line, the first row at least half as wide as the frame's widest, so a narrow ear is passed over. With `--view` (and in `video-set`, always) the view picks the signal and the swings only say whether there are legs: a side or diagonal view where the foot band swings by 15 % of the body's height or the lowest row by 1.5 %, a front or back view where the lowest row swings by 1.5 %. A short-legged walk misleads a threshold either way — from the front the foot behind lifts out of the foot band, which then swings like a side stride mid-step; from the side a short step opens the feet by less than 15 % — so the thresholds choose the signal only without `--view`: the stride where it swings by 15 % or more, otherwise the lowest row where it swings by 1.5 % or more. A body that hardly bobs turns on its legs as one that bobs does (`tests/video/test_cycle_align.py`: a drawn walker whose ears are lowest half a step off its heel strike, with and without a bob; a big-headed, short-legged walker from the front and from the side).
- **New: every view starts as the same own foot lands** (`video-cycle-align --view front --view side@right …`, one per `--loop-dir`, `--start-foot right|left`, default right; `video-set` passes each item's view and facing). A walk lands twice a cycle, half a cycle apart, and until now the turn took either. The foot is told by how the view draws it (`align.strike_foot`, with `handedness.placement` for which own side is where): front and back by which foot is drawn lower (nearer the viewer), side and diagonal by which leg is shaded (the far one). Where the cue does not follow the step (its once-a-cycle swing under a quarter of its spread) or the two strikes do not differ enough, the foot is not named rather than guessed: the loop starts on its larger strike with `start_foot: null` and `foot_why`, and the report's `warnings` say so; without `--view` the report says once that no foot was chosen. Each loop row carries `view`, `start_foot` and `foot` (the cue, its values and margin). [docs/loop-repair.md](docs/loop-repair.md#4-one-cycle-for-a-direction-set--video-cycle-align-video-set---align-cycles-auto).
## v2.24.0 - A walk filmed from its first frame loops at one size, a frame scaled up keeps its crown, and a soft part can follow the body
diff --git a/docs/loop-repair.md b/docs/loop-repair.md
index 059a06a..6f98dc8 100644
--- a/docs/loop-repair.md
+++ b/docs/loop-repair.md
@@ -283,15 +283,23 @@ sprite-gen video-cycle-align --loop-dir set/front-walk/loop --loop-dir set/side-
- **Foot strike**: each loop is then turned to start as a heel lands (`align.foot_strike`), read
off one signal smoothed 1-2-1 — never off the frame's top edge, which a long ear, a hat's
point or an antenna owns and which flops on its own rhythm:
- - `stride`, where the width of the foot band (the lowest 8 % of the frame) swings by 15 % of
- the body's height or more — a side or diagonal walk. The feet are widest apart as the front
- heel lands.
- - `reach`, where instead the lowest solid row swings by 1.5 % or more — a front or back walk,
- whose feet pass one behind the other: the foot nearer the viewer is drawn lowest, and lowest
- when the feet are furthest apart.
- - `body_low`, where neither moves — a body with no legs to read: its top line lowest, the top
- line being the first row whose longest solid run is at least half the frame's widest, so a
- narrow ear or antenna is passed over.
+ - `stride` — a side or diagonal walk: the width of the foot band (the lowest 8 % of the frame)
+ is widest as the front heel lands.
+ - `reach` — a front or back walk, whose feet pass one behind the other: the foot nearer the
+ viewer is drawn lowest, and lowest when the feet are furthest apart, so the lowest solid row is.
+ - `body_low` — a body with no legs to read: its top line lowest, the top line being the first
+ row whose longest solid run is at least half the frame's widest, so a narrow ear or antenna is
+ passed over.
+
+ With a view (`--view`, below) the view picks the signal, and the swings only say whether there
+ are legs to read: a side or diagonal view turns on `stride` where the foot band swings by 15 %
+ of the body's height or the lowest row by 1.5 %; a front or back view turns on `reach` where the
+ lowest row swings by 1.5 %. A short-legged figure's swings mislead a threshold either way: from
+ the front its foot band widens past 15 % as the foot behind lifts out of the band, mid-step, and
+ from the side a short step opens the feet by less than 15 %. Without a view the picture alone
+ says: `stride` where the foot band swings by 15 % or more, else `reach` where the lowest row
+ swings by 1.5 %, else `body_low` (and the report says no view was given). A loop with a view and
+ no legs to read has `foot_why` with both swings.
A body that hardly bobs still turns on its legs; the body's bob is read only where there are
no legs. The loop row says which (`turned_on`) and the two swings (`stride_swing`,
diff --git a/sprite_gen/video/align.py b/sprite_gen/video/align.py
index c5e41f4..5300f08 100644
--- a/sprite_gen/video/align.py
+++ b/sprite_gen/video/align.py
@@ -47,7 +47,8 @@ RATE_TOLERANCE = 0.002 # frame rates read back from the strip metadata agree wi
LOW_ALPHA = 128 # the foot-strike turn reads solid pixels only
# The foot-strike turn (docs/loop-repair.md section 4). A walk seen from the side or a diagonal opens
# its feet by a good part of its height each step; from the front or back its foot band hardly
-# changes width, but the foot nearer the viewer is drawn lower by a few hundredths of it.
+# changes width, but the foot nearer the viewer is drawn lower by a few hundredths of it. With a
+# view the view picks the signal and these only say whether there are legs to read (`foot_strike`).
STRIDE_MIN = 0.15 # the foot band's swing (of the body's height) that says the feet open along the picture
REACH_MIN = 0.015 # the lowest row's swing (of the body's height) that says a foot steps toward the viewer
BODY_RUN = 0.5 # the body's top line: the first row at least half as wide as the frame's widest
@@ -225,12 +226,18 @@ def foot_strike(frames: list[Image.Image], *, view: str | None = None, foot: str
"""Where a loop starts: the frame a heel has just landed, read off one signal smoothed over its
neighbours (1-2-1), never off the frame's top edge, which a long ear or a hat's point owns.
- - `stride`: seen from the side or a diagonal the feet are widest apart as the front heel
- lands; used where the stride swings by STRIDE_MIN of the body's height or more.
+ - `stride`: seen from the side or a diagonal the feet are widest apart as the front heel lands.
- `reach`: seen from the front or back the feet pass one behind the other and the stride
hardly moves, but the foot nearest the viewer is drawn lowest, and lowest when the feet are
- furthest apart — as the heel lands; used where the lowest row swings by REACH_MIN or more.
- - `body_low`: neither — a body with no legs to read — starts where its top line is lowest.
+ furthest apart — as the heel lands.
+ - `body_low`: a body with no legs to read starts where its top line is lowest.
+
+ With the loop's `view` the view picks the signal and the swings only say whether there are legs
+ to read: a side or diagonal view turns on the stride where either swing says legs (the stride by
+ STRIDE_MIN of the body's height, or the lowest row by REACH_MIN); a front or back view turns on
+ the reach where the lowest row swings by REACH_MIN, since its foot band widens as a foot is
+ lifted out of it, mid-step, not as a heel lands. Without a view the picture alone says: the
+ stride where it swings by STRIDE_MIN, else the reach where it swings by REACH_MIN.
A walk has two such moments a cycle, half a cycle apart: the larger, and the frame half a cycle
on (not the other peak: one step can stride much less than the other, or not peak at all).
@@ -242,7 +249,14 @@ def foot_strike(frames: list[Image.Image], *, view: str | None = None, foot: str
sig = strike_signals(frames)
height = float(np.median(sig["height"]))
swing = {k: float(max(sig[k]) - min(sig[k])) / (1.0 if k == "stride" else height) for k in ("stride", "reach")}
- by = "stride" if swing["stride"] >= STRIDE_MIN else "reach" if swing["reach"] >= REACH_MIN else "body_low"
+ least = {"stride": STRIDE_MIN, "reach": REACH_MIN}
+ legs = [k for k in least if swing[k] >= least[k]]
+ name, _, facing = (view or "").partition("@")
+ if view is None:
+ by = legs[0] if legs else "body_low"
+ else:
+ handed_mod.validate_view(name, facing or None)
+ by = ("stride" if legs else "body_low") if name in handed_mod.LATERAL_VIEWS else ("reach" if "reach" in legs else "body_low")
signal = sig["top" if by == "body_low" else by]
n = len(signal)
smooth = [(signal[(k - 1) % n] + 2 * signal[k] + signal[(k + 1) % n]) / 4 for k in range(n)]
@@ -252,10 +266,9 @@ def foot_strike(frames: list[Image.Image], *, view: str | None = None, foot: str
"strikes": [first, second], "start_foot": None}
if view is None:
return {**out, "foot_why": "no view given for this loop"}
- name, _, facing = view.partition("@")
- handed_mod.validate_view(name, facing or None)
if by == "body_low":
- return {**out, "foot_why": "no legs to read"}
+ return {**out, "foot_why": f"no legs to read: the foot band swings {swing['stride']:.3f} of the body's height "
+ f"(legs from {STRIDE_MIN}), the lowest row {swing['reach']:.3f} (from {REACH_MIN})"}
found = strike_foot(frames, (first, second), name, facing or None)
if found["feet"] is None:
return {**out, "foot": found, "foot_why": found["why"]}
diff --git a/tests/video/test_cycle_align.py b/tests/video/test_cycle_align.py
index a62ee17..c882e33 100644
--- a/tests/video/test_cycle_align.py
+++ b/tests/video/test_cycle_align.py
@@ -15,7 +15,7 @@ from pathlib import Path
import numpy as np
import pytest
-from PIL import Image
+from PIL import Image, ImageDraw
from sprite_gen.video import align
from sprite_gen.video import loop as loop_mod
@@ -156,6 +156,58 @@ def test_a_foot_the_view_cannot_tell_is_not_named():
assert unseen["start_foot"] is None and unseen["foot_why"] == "no view given for this loop"
+def _chibi(phase: float, *, view: str, step: float = 22, lift: int = 8, persp: int = 6, apart: int = 14) -> Image.Image:
+ """An earless walker with short legs under a big head, at `phase` (radians): its own right heel
+ lands at 0, its left at pi. Each leg swings `step` degrees either way about the hip; the stance
+ leg stays planted, so the hip rides lowest as a heel lands, and the foot behind lifts by `lift`
+ px mid-step. `front`: the feet `apart` px either side of the middle, its right foot on the
+ picture's left, the foot stepping toward the viewer drawn up to `persp` px lower. `side`: faces
+ right, its near right leg drawn light and `persp` px lower over the shaded far left leg."""
+ img = Image.new("RGBA", (300, 360))
+ draw = ImageDraw.Draw(img)
+ ground, leg, head, a = 340, 42, 50, math.radians(step)
+ swing = {"right": a * math.cos(phase), "left": -a * math.cos(phase)}
+ stance = "right" if phase % (2 * math.pi) < math.pi else "left"
+ hip, mid = ground - leg * math.cos(swing[stance]), 150
+ for own in ("left", "right") if view == "side" else ("right", "left"):
+ lifted = lift * abs(math.sin(phase)) if own != stance else 0
+ if view == "side":
+ fx, fy, tone = mid + leg * math.sin(swing[own]), ground - lifted - (persp if own == "left" else 0), 210 if own == "right" else 140
+ else:
+ fx, fy, tone = mid - apart if own == "right" else mid + apart, ground - lifted + persp * (math.sin(swing[own]) / math.sin(a) - 1), 200
+ draw.line([(mid if view == "side" else fx, hip), (fx, fy - 6)], fill=(tone, tone, tone, 255), width=7)
+ draw.rectangle([fx - 9, fy - 8, fx + 9, fy], fill=(tone - 60, tone - 60, tone - 60, 255))
+ torso = int(leg * 0.9)
+ draw.rectangle([mid - 22, hip - torso, mid + 22, hip], fill=(230, 200, 180, 255))
+ draw.ellipse([mid - head, hip - torso - 2 * head, mid + head, hip - torso], fill=(240, 210, 190, 255))
+ return img
+
+
[email protected]("view, figure, kw, by", [
+ ("front", "front", {"lift": 8, "persp": 6}, "reach"), # its feet lift out of the foot band: the band swings like a side stride
+ ("front", "front", {"lift": 12, "persp": 6}, "reach"),
+ ("front", "front", {"lift": 4, "persp": 10}, "reach"),
+ ("front", "front", {"lift": 20, "persp": 0, "apart": 16}, "body_low"), # no foot drawn nearer: the band's swing is a lift, not a strike
+ ("side@right", "side", {"step": 10, "persp": 6}, "stride"), # a short step under STRIDE_MIN still opens the feet as a heel lands
+ ("side@right", "side", {"step": 14, "persp": 3, "lift": 12}, "stride"),
+])
+def test_the_view_picks_the_signal_so_a_short_legged_walk_starts_on_a_strike(view, figure, kw, by):
+ """A short-legged walk's swings mislead the thresholds: from the front its stride swings past
+ STRIDE_MIN as the foot behind lifts out of the band, and from the side a short step swings under
+ it. The view says which signal the step is on; the start is on a strike (or one frame after it,
+ the heel just landed), and a foot it names is the one landing there."""
+ frames = [_chibi(2 * math.pi * k / 12, view=figure, **kw) for k in range(12)]
+ turned = frames[5:] + frames[:5] # its right heel lands at turned[7], its left at turned[1]
+ strike = align.foot_strike(turned, view=view)
+ assert strike["by"] == by, strike
+ landed = {"right": 7, "left": 1}
+ near = min(landed, key=lambda f: min((strike["start"] - landed[f]) % 12, (landed[f] - strike["start"]) % 12))
+ assert (strike["start"] - landed[near]) % 12 in (0, 1), strike
+ assert strike["start_foot"] in (None, near), strike
+ if by == "body_low":
+ assert strike["foot_why"].startswith("no legs to read"), strike
+
+
def _cut(tmp_path: Path, name: str, length: int) -> Path:
keyed = tmp_path / f"{name}-keyed"
keyed.mkdir()