Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

feat(video-canvas): a side walk or run keeps room around its still — its square is widened by what the stride and a trailing tail lack

Soohong Kim935fb63db959
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8bce3ce..085c225 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,23 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
## Unreleased
+**Upgrading from 3.3.0: `video-set` now frames a walk or run filmed from the side on a square widened by the room its still lacks.** A still that leaves less than `GAIT_ROOM` clear in front of its silhouette (0.15 of the silhouette's height) or behind it (0.25) gains the columns it lacks on that side; the square gains the same number of rows above the still. A still that leaves the room keeps its canvas to the byte. Every other state, a front, back or diagonal view, `--shape` and `--fit tight` are framed as before. Each side walk or run's `canvas.report.json`, and its item in `set.report.json`, gains `room`. `video-canvas` gains `--view` and reads the room only with `--view side`.
+
+- **New: the walk's room (`canvas.gait_room`, `GAIT_ROOM`, `ROOM_VIEWS`).** Filmed from the side, a walk's stride reaches in front of the standing silhouette, and behind it the stride and a part that trails behind the body (a big tail, a long coat) swing past it. Where either met the left or right edge, `video-frames` refused the clip, and the walk had to be filmed again with `--shape wide`. The `walk` and `run` rows now read the still's room: the empty columns between the silhouette and the left and right edges of the square the row makes of the still, in front and behind, as a fraction of the silhouette's height. A side under its `GAIT_ROOM` (more behind than in front) gains what it lacks, and the frame stays square: the clip keeps the 1:1 a walk is filmed at, and the still is never scaled. The report's `room` records the view, the silhouette, the room on each side, the bound, `short`, `added_px` (columns per side) and `added_rows`, and a widened canvas's `why` says why. `--shape square` keeps the still's own square. `--shape wide` is still the walk's 16:9 row (`GAIT_WIDE`). See [video-pipeline](docs/video-pipeline.md) section 1.
+- **Changed: `canvas.subject_rows` is now `canvas.subject_box`**, which reads the columns as well as the rows. `--fit tight` reads its rows from it, byte for byte.
+- **Tests**: `tests/video/test_walk_canvas_room.py` covers these cases:
+ - a side walk whose tail reaches 6 px of the edge behind, facing either way, gets a square widened behind and above by what it lacks, the still unscaled, through `video-set`'s item and report;
+ - a person with room keeps the default row's canvas byte for byte;
+ - front, back, diagonal and unnamed views are not read;
+ - each bound flips one pixel either side of its `GAIT_ROOM`, the one behind larger than the one in front, and both short sides each gain their own lack;
+ - a still taller than it is wide is read in its square;
+ - a white base is read on its corner colour;
+ - a run is read at the walk's bounds;
+ - `--shape square`, `--shape wide`, `--fit tight` and another state are not read;
+ - `--view` works from the command line.
+
+ Four `video-set` tests' canvas stubs now take `view`, and `test_edge_speck_lead_in.py` reads the walk's own row.
+
**Upgrading from 3.3.0: A walk or run loop is cut to the same bytes, whether a part pops in it or not; every walk and run report gains `part_pop` and `retake`.** `--unfixed refuse` also refuses a loop recorded `retake` reason `part-pop`, whatever `--repair`. No flags are added; no frames are made and no other cut is taken.
- **New: The part reading and the `retake` record.** Squares 8 % of the body height are followed drawing to drawing and a part whose jerk leaves the curve its steps either side draw pops at the wrap. The cut stays and the report records a take to film again (`retake`, reason `part-pop`): `remedy` is `retake` where the engine's search chose the cut, and `recut-or-retake` where the cut was given (`--cycle fixed`, or `pinned`); `searched` says which. See [loop-repair](docs/loop-repair.md) section 2, "The part pop".
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 6d66b65..d87ef0b 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -41,6 +41,45 @@ at the crown gets only the quarter — pass `--headroom 0.26` or more for an ove
back before the strike); `--facing left` mirrors the wide layout. A still whose corners are not one flat colour
is refused — a non-flat background cannot be extended without guessing.
+**A walk or run from the side keeps room around its still (`GAIT_ROOM`).** Filmed from the side, the stride
+reaches in front of the standing silhouette; behind it, the stride and a part that trails behind the body (a big
+tail, a long coat) swing past it, in the picture's plane. When either meets the left or right edge, `video-frames` refuses the
+clip ([section 3](#3-frames--extract-key-check-the-edges)). So the `walk` and `run` rows read the still's room
+(`canvas.gait_room`) when the view is `side`. The room is the empty columns between the silhouette and the left and right edges of the square the row
+makes of the still (`max(w, h)` wide, the still centred, so a still taller than it is wide gains room on either
+side). It is counted on two sides: in front of the subject (the way it faces) and behind it, each as a fraction of
+the silhouette's height. The silhouette is every pixel of the normalized still that is not the exact key
+(`canvas.subject_box`; on a `--key white` base, every pixel more than the corner tolerance off the corner colour).
+Each side keeps its `GAIT_ROOM`: 0.15 of the silhouette's height in front, for the stride, and 0.25 behind,
+for the stride and whatever trails behind the body. A side that keeps less gains the whole pixels it lacks, and the square grows by the same number of rows, all of them above the still. So the frame stays
+square and the still is never scaled. A still that leaves the room on both sides gets the canvas it got before, byte for byte.
+
+The square is widened rather than moved to the 16:9 wide row: a walk is filmed at 1:1, and the wide row
+(`GAIT_WIDE`, 0.28 in front and 0.20 behind) films a square still's subject at about two thirds of the height a
+1:1 clip gives it. Only the side view is read (`ROOM_VIEWS`): a diagonal walk strides half into the picture, and
+a front or back walk wholly into it. A run is read at the walk's bounds.
+
+The report's `room` records the view, the silhouette box on the still, `front_px` / `behind_px` and their
+fractions `front` / `behind`, `need` / `need_px` (each side's bound),
+`short` (the sides under the bound, front first), `added_px`
+(the columns each side gained) and `added_rows` (the rows the square gained above the still). A widened canvas's
+`why` says why. `video-set` passes each item's view and copies
+`room` into the item's `canvas` (with the `why` when the square was widened). `video-canvas` reads the room only
+with `--view side`; without `--view`, the canvas is the state's row as before. `--shape square` keeps the
+still's own square whatever its room, `--shape wide` frames a walk or run on `GAIT_WIDE` as before, and
+`--fit tight` adds no room.
+
+What the room takes from sprite-gen, and what is new:
+
+| Part | Taken from | Why |
+|---|---|---|
+| The silhouette on the still | `canvas.subject_box`: the reading `--fit tight` makes of the subject's rows, widened to its columns (`subject_rows` became `subject_box`; tight reads its rows from it, byte for byte) | One reading of where the subject is on a still the canvas normalized |
+| The background it is read on | `canvas.normalize_key`: the `cutout` matte's alpha-0 set repainted to the exact key, the pixels `video-frames` erases | The silhouette is what the clip's frames will keep |
+| The square it is read in, and the placement | The square row of `pad_canvas` (`max(w, h)`, the still centred and standing on the bottom edge) | The room is counted in the frame the clip is made from |
+| The view names | `handedness.VIEWS`, which `video-prompt --direction` and `video-set --base` take | One vocabulary for the view |
+| The gate the room keeps the clip off | `video-frames`' edge-contact check (`frames.edge_contact`, `frames.classify_edge_contact`), unchanged | The canvas cannot see the clip; the gate still judges it |
+| Written new: `gait_room`, `GAIT_ROOM`, `ROOM_VIEWS` | — | No engine function reads, before a clip exists, the room a motion needs around a still. `frames.extract.safe_area_fit` judges a finished frame's size against a sheet cell's safe area, a centred fit rather than the room on one side of a still. The edge check reads frames once the clip exists. |
+
**`--fit tight` adds no room.** It is the framing for a fixed body height at a low
clip resolution: the room above makes the subject a small part of the clip, and a
`--body-height` target then has to upscale it. A motion that leaves the frame is clipped: pair it
@@ -123,11 +162,12 @@ rest, stage by stage, with the verbs `video-set` runs: `video-prompt` prints the
would send, `video-frames` and `video-loop` cut each mp4 the agent saved, and for a walk or run cut in
two or more directions `video-cycle-align` gives them one cycle length, each turned to start on a
foot strike ([loop repair](loop-repair.md) section 4). `video-set` itself is not run on this route, so
-what it does for a set happens only when you run it. What crosses the boundary is a PNG, a prompt and
+what it does for a set happens only when you run it: `--view` names the still's view to `video-canvas`, as `video-set`
+does, so a side walk keeps its room ([section 1](#1-canvas--the-input-frame-decides-the-output-frame)). What crosses the boundary is a PNG, a prompt and
an mp4.
```bash
-sprite-gen video-canvas --still still.png --state walk --facing right --out item/canvas.png --report item/canvas.report.json
+sprite-gen video-canvas --still still.png --state walk --facing right --view side --out item/canvas.png --report item/canvas.report.json
sprite-gen video-prompt --direction side --state walk --facing right --no-last-frame --json > item/prompt.json
# the agent, with its ZCRE tools and grok-imagine-video-1.5: prepare_upload for item/canvas.png and the
# transfer it describes, query_zcre upload to confirm it, query_zcre quote with prompt.json's prompt and
@@ -326,13 +366,15 @@ Pass `--decontam off` to keep the matte as it is. Method, guards and measurement
## 3b. Canvas shape for raised limbs and wide costumes
-`video-set` picks the canvas from the state row alone. Two things that are not a jump or
+`video-set` picks the canvas from the state row, and for a walk or run from the side from its
+still's room too ([section 1](#1-canvas--the-input-frame-decides-the-output-frame), `GAIT_ROOM`). Two things that are not a jump or
an attack still leave a 1:1 frame: limbs raised in a celebration, and a costume that is
wider than the body (a skirt, a veil, a held object). Both fail `video-frames`'s
edge-contact check — the clip was made, the frames were cut, and the run stops at the
gate. The state table now routes `cheer`, `wave` and `celebrate` to the wide canvas, and
`video-set --shape wide` forces it for every state of a batch when the costume is the
-reason.
+reason. A side walk or run whose still leaves a big tail or a wide costume too little room no longer needs
+`--shape wide`: its square is widened by the room the still lacks.
## Video Processing Pipeline and Terminal Gate
diff --git a/sprite_gen/video/batch.py b/sprite_gen/video/batch.py
index 0a8a991..54ba644 100644
--- a/sprite_gen/video/batch.py
+++ b/sprite_gen/video/batch.py
@@ -804,12 +804,19 @@ def run_item(
else:
still = item_dir / "facing.png"
facing_report = facing_mod.prepare_still(base, still, facing=facing, fix=facing_fix)
- # a lifetime is framed as an effect, whatever its name: on black, its centre kept
- canvas_report = canvas_mod.run_canvas(still, canvas_png, state="effect" if lifetime else state, shape=shape, facing=facing if direction not in ("front", "back") else "right", headroom=None, lead=None, report_path=canvas_report_path, fit=fit)
+ # a lifetime is framed as an effect, whatever its name: on black, its centre kept. The view tells the
+ # canvas whether a walk or run crosses the picture, whose still's room it reads (`canvas.gait_room`)
+ canvas_report = canvas_mod.run_canvas(still, canvas_png, state="effect" if lifetime else state, shape=shape, facing=facing if direction not in ("front", "back") else "right", headroom=None, lead=None, report_path=canvas_report_path, fit=fit, view=direction)
if facing_report is not None:
canvas_report["facing_check"] = facing_report
atomic_write_text(canvas_report_path, json.dumps(canvas_report, ensure_ascii=False, indent=2) + "\n")
result["canvas"] = {k: canvas_report[k] for k in ("fit", "shape", "canvas", "offset") if k in canvas_report}
+ if "room" in canvas_report:
+ # the still's room around its silhouette, and why the square is widened when it is short
+ result["canvas"]["room"] = {k: canvas_report["room"][k]
+ for k in ("view", "front_px", "behind_px", "front", "behind", "need", "short", "added_px", "added_rows")}
+ if canvas_report["room"]["short"]:
+ result["canvas"]["why"] = canvas_report["why"]
if "facing_check" in canvas_report:
result["facing"] = canvas_report["facing_check"]
@@ -1258,7 +1265,7 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--key", choices=("auto", "green", "magenta", "white"), default="auto")
parser.add_argument("--concurrency", type=int, default=3, help="parallel clip generations (starts are staggered regardless)")
parser.add_argument("--start-gap", type=float, default=START_GAP_SECONDS, help="seconds between clip request starts")
- parser.add_argument("--shape", choices=canvas_mod.SHAPES, help="force one canvas shape for every state (e.g. wide for a costume or arms that leave a 1:1 frame)")
+ parser.add_argument("--shape", choices=canvas_mod.SHAPES, help="force one canvas shape for every state (e.g. wide for a costume or arms that leave a 1:1 frame; square keeps a side walk or run square whatever room its still leaves)")
parser.add_argument("--anchor", choices=tuple(a for a in loop_mod.ANCHOR_MODES if a != "motion"), default="none", help="motion-auto: automatic regions, local period and XY correction (walk/run only); feet: remove in-canvas drift")
parser.add_argument("--spill", choices=frames_mod.SPILL_MODES, default="auto", help="auto: judge key reflections from each item's canvas still (default); small / full: force")
parser.add_argument("--fit", choices=canvas_mod.FITS, default="state", help="state: each state's room for the motion (default); tight: no room, the subject fills the frame and a motion that leaves it is clipped (use with --body-height at a low --resolution)")
diff --git a/sprite_gen/video/canvas.py b/sprite_gen/video/canvas.py
index 9ad65f0..0dd2094 100644
--- a/sprite_gen/video/canvas.py
+++ b/sprite_gen/video/canvas.py
@@ -5,6 +5,8 @@ Grok Imagine keeps the input image's framing and ignores `aspect_ratio` on
image-to-video (2026-09-08 실측: a `3:4` request still returned 960x960). So the
canvas is decided HERE, on the still: a jump needs head-room above (tall), an
attack needs room above and in front, a projectile needs room in front (wide), everything else stays square.
+A walk or run filmed from the side stays square too, the square widened by whatever room the still does
+not leave its silhouette for the stride and what trails behind it (`gait_room`, `--view`).
The state -> canvas table below is the single owner of that rule; `--shape`
overrides it per call, with the state's own row for that shape where it has one (a walk or run
forced wide gets no head-room).
@@ -43,6 +45,7 @@ from PIL import Image
from sprite_gen._deps import np
from sprite_gen.frames.cutout import KEY_TARGETS, extract_route
from sprite_gen.frames.extract import is_border_key_candidate
+from sprite_gen.gen.handedness import VIEWS
from sprite_gen.spec.runio import atomic_write_text
from sprite_gen.util.console import utf8_streams
@@ -60,8 +63,23 @@ class CanvasProfile:
lead: float # fraction of the canvas width kept empty IN FRONT of the subject (wide)
trail: float # fraction of the canvas width kept empty BEHIND the subject (wide)
why: str
-
-
+ # A square row that, filmed in a view the motion crosses (ROOM_VIEWS), reads the still's room around its
+ # silhouette and widens the square by the room it lacks (`gait_room`).
+ room: bool = False
+
+
+# Filmed from the side, a walk's stride reaches in front of the standing silhouette, and behind it the stride
+# and whatever trails behind the body — a big tail, a long coat — swing past it, in the picture's plane. The
+# square a walk is filmed on keeps GAIT_ROOM of the silhouette's height clear between the silhouette and the
+# frame's edge on each side, more behind than in front: where the still leaves less, the square is widened by what
+# it lacks on that side, the extra height going above (`video-frames` refuses a clip whose subject meets a side of
+# the frame). The frame stays square — the clip keeps the 1:1 a walk is filmed at, and the subject loses only the
+# share of the clip the room takes. A diagonal walk strides half into the picture and a front or back walk wholly
+# into it, so their room is not read (ROOM_VIEWS). A run is read at the walk's bounds. The rule:
+# docs/video-pipeline.md section 1.
+GAIT_ROOM = {"front": 0.15, "behind": 0.25}
+ROOM_VIEWS = frozenset({"side"})
+GAIT_ROOM_WHY = "the still leaves its silhouette too little room for a walk or run from the side: the square is widened by the room it lacks"
# The one table. Keys are state names as the sprite-request uses them; unknown
# states fall through to `default`.
STATE_CANVAS: dict[str, CanvasProfile] = {
@@ -85,6 +103,9 @@ STATE_CANVAS: dict[str, CanvasProfile] = {
"cast": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.25, 0.25, 0.25, "room for the cast terminal pose"),
"hurt_hold": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.1, 0.32, 0.32, "room for the hurt_hold terminal pose"),
"transform": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.25, 0.25, 0.25, "room for the transform terminal pose"),
+ # In place on the still's own frame, as any state, while the still leaves the stride its room (GAIT_ROOM).
+ "walk": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, 0.0, "in-place motion fits the still's own frame", room=True),
+ "run": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, 0.0, "in-place motion fits the still's own frame", room=True),
"default": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, 0.0, "in-place motion fits the still's own frame"),
}
# `--shape wide` is not the attack row. It is forced on a state whose own row is
@@ -202,14 +223,49 @@ def normalize_key(image: Image.Image, kind: str) -> tuple[Image.Image, dict[str,
return Image.fromarray(data, "RGB"), report
-def subject_rows(image: Image.Image, fill: tuple[int, int, int], tolerance: int) -> tuple[int, int]:
- """First and one-past-last row holding a pixel that is not the background `fill`."""
+def subject_box(image: Image.Image, fill: tuple[int, int, int], tolerance: int) -> tuple[int, int, int, int] | None:
+ """The box (left, top, right, bottom; right and bottom one past) of the pixels more than `tolerance` off the
+ background `fill` on some channel, or None when there are none."""
data = np.asarray(image.convert("RGB"), dtype=np.int16)
- diff = np.abs(data - np.array(fill, dtype=np.int16)).max(axis=2)
- rows = np.flatnonzero((diff > tolerance).any(axis=1))
+ subject = np.abs(data - np.array(fill, dtype=np.int16)).max(axis=2) > tolerance
+ rows = np.flatnonzero(subject.any(axis=1))
if rows.size == 0:
- raise SystemExit("video-canvas: --fit tight found no subject on the still's background")
- return int(rows[0]), int(rows[-1]) + 1
+ return None
+ cols = np.flatnonzero(subject.any(axis=0))
+ return int(cols[0]), int(rows[0]), int(cols[-1]) + 1, int(rows[-1]) + 1
+
+
+def gait_room(image: Image.Image, fill: tuple[int, int, int], tolerance: int, *, facing: str, view: str) -> dict[str, Any]:
+ """The empty margin between the still's silhouette (`subject_box`) and the left and right edges of the square frame
+ a square row makes of it (`pad_canvas`: max(w, h) wide, the still centred, so a still taller than it is wide gains
+ room on either side), read as the room in front of the subject (the side it faces) and behind it, in px and of the
+ silhouette's height. `short` names the sides under their GAIT_ROOM of that height, front first; `added_px` is what
+ each side lacks, in whole px, the columns the square gains there, and `added_rows` the rows it gains above."""
+ box = subject_box(image, fill, tolerance)
+ if box is None:
+ raise SystemExit("video-canvas: found no subject on the still's background to read a walk's room around")
+ left, top, right, bottom = box
+ height = bottom - top
+ w, h = image.size
+ x = (max(w, h) - w) // 2
+ margin = {"left": x + left, "right": max(w, h) - x - right}
+ ahead, behind = ("right", "left") if facing == "right" else ("left", "right")
+ room = {"front": margin[ahead], "behind": margin[behind]}
+ need = {side: GAIT_ROOM[side] * height for side in ("front", "behind")}
+ added = {side: max(0, math.ceil(need[side] - room[side])) for side in ("front", "behind")}
+ return {
+ "view": view,
+ "silhouette": [left, top, right, bottom],
+ "front_px": room["front"],
+ "behind_px": room["behind"],
+ "front": round(room["front"] / height, 4),
+ "behind": round(room["behind"] / height, 4),
+ "need": dict(GAIT_ROOM),
+ "need_px": {side: round(need[side], 1) for side in ("front", "behind")},
+ "short": [side for side in ("front", "behind") if room[side] < need[side]],
+ "added_px": added,
+ "added_rows": added["front"] + added["behind"],
+ }
def tight_canvas(src: Image.Image, fill: tuple[int, int, int], *, tolerance: int) -> tuple[Image.Image, dict[str, Any]]:
@@ -220,7 +276,10 @@ def tight_canvas(src: Image.Image, fill: tuple[int, int, int], *, tolerance: int
split evenly, because a tight frame adds no room in front on purpose.
"""
w, h = src.size
- top, bottom = subject_rows(src, fill, tolerance)
+ box = subject_box(src, fill, tolerance)
+ if box is None:
+ raise SystemExit("video-canvas: --fit tight found no subject on the still's background")
+ _, top, _, bottom = box
head = round((bottom - top) * TIGHT_HEADROOM)
kept_top = max(0, top - head)
band = src.crop((0, kept_top, w, bottom))
@@ -255,11 +314,16 @@ def pad_canvas(
trail: float | None = None,
key: str = "auto",
fit: str = "state",
+ view: str | None = None,
) -> tuple[Image.Image, dict[str, Any]]:
"""Return (padded RGB image, placement report). Never downsizes the still.
On a green/magenta base the background is normalized to the exact key and
the padding is that key; otherwise the padding is the corner colour.
+ A `room` row filmed in one of ROOM_VIEWS (`view`) has the still's room read
+ (`gait_room`, the report's `room`); a side that is short gains the columns it
+ lacks and the square the rows to match, above the still. With no `view` the
+ room is not read.
"""
corner = corner_key(image)
kind = resolve_key(corner, key)
@@ -287,6 +351,9 @@ def pad_canvas(
"why": "tight: no room for the motion; the subject fills the frame and a motion that leaves it is clipped",
}
return canvas, report
+ room = None
+ if profile.room and view in ROOM_VIEWS:
+ room = gait_room(src, tuple(fill), 0 if kind is not None else CORNER_TOLERANCE, facing=facing, view=view) # type: ignore[arg-type]
w, h = src.size
head = profile.headroom if headroom is None else headroom
front = profile.lead if lead is None else lead
@@ -295,8 +362,15 @@ def pad_canvas(
raise SystemExit("video-canvas: --headroom/--lead/--trail must be in [0, 0.9) and lead + trail below 0.9")
if profile.shape == SHAPE_SQUARE:
side = max(w, h)
+ x = (side - w) // 2
+ if room is not None and room["short"]:
+ # the columns each side lacks; the square's rows grow to match, above the still
+ added = {"left": room["added_px"]["behind" if facing == "right" else "front"],
+ "right": room["added_px"]["front" if facing == "right" else "behind"]}
+ side += added["left"] + added["right"]
+ x += added["left"]
canvas_w, canvas_h = side, side
- x, y = (side - w) // 2, side - h
+ y = side - h
elif profile.shape == SHAPE_TALL:
# the still becomes the bottom (1 - headroom) of a canvas at least as tall as the
# profile ratio demands for the still's width — never narrower than the still
@@ -326,7 +400,8 @@ def pad_canvas(
"key_rgb": list(fill),
"corner_rgb": list(corner),
**key_report,
- "why": profile.why,
+ "why": GAIT_ROOM_WHY if room is not None and room["short"] else profile.why,
+ **({"room": room} if room is not None else {}),
}
return canvas, report
@@ -344,6 +419,7 @@ def run_canvas(
key: str = "auto",
trail: float | None = None,
fit: str = "state",
+ view: str | None = None,
) -> dict[str, Any]:
still = still.expanduser().resolve()
if not still.is_file():
@@ -358,7 +434,7 @@ def run_canvas(
canvas, report = effect.canvas(image)
else:
profile = profile_for(state, shape)
- canvas, report = pad_canvas(Image.open(still), profile, facing=facing, headroom=headroom, lead=lead, trail=trail, key=key, fit=fit)
+ canvas, report = pad_canvas(Image.open(still), profile, facing=facing, headroom=headroom, lead=lead, trail=trail, key=key, fit=fit, view=view)
out = out.expanduser().resolve()
out.parent.mkdir(parents=True, exist_ok=True)
tmp = out.with_name(out.name + ".part")
@@ -383,6 +459,7 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--trail", type=float, help="wide: empty fraction behind the subject, for a weapon drawn back (default from the profile)")
parser.add_argument("--key", choices=KEYS, default="auto", help="chroma key of the still (auto reads the corners; green/magenta are normalized to the exact key; white pads with the corner colour)")
parser.add_argument("--fit", choices=FITS, default="state", help="state: the state's room for the motion (default); tight: drop the empty rows around the subject and add no room, so it fills the frame and a motion that leaves it is clipped")
+ parser.add_argument("--view", choices=VIEWS, help="the still's view; a walk or run from the side has the still's room around its silhouette read, and its square widened by the room it lacks (`room` in the report). Left out, the room is not read")
parser.add_argument("--report", type=Path, help="write the canvas report JSON here")
@@ -391,7 +468,7 @@ def run(**kwargs: object) -> int:
Path(str(kwargs["still"])), Path(str(kwargs["out"])),
state=kwargs.get("state"), shape=kwargs.get("shape"), facing=str(kwargs.get("facing") or "right"), # type: ignore[arg-type]
headroom=kwargs.get("headroom"), lead=kwargs.get("lead"), trail=kwargs.get("trail"), report_path=kwargs.get("report"), # type: ignore[arg-type]
- key=str(kwargs.get("key") or "auto"), fit=str(kwargs.get("fit") or "state"),
+ key=str(kwargs.get("key") or "auto"), fit=str(kwargs.get("fit") or "state"), view=kwargs.get("view"), # type: ignore[arg-type]
)
print(json.dumps(payload, ensure_ascii=False, indent=2))
return 0
diff --git a/tests/video/test_attack_motion.py b/tests/video/test_attack_motion.py
index 0a11c29..6e0ceb4 100644
--- a/tests/video/test_attack_motion.py
+++ b/tests/video/test_attack_motion.py
@@ -97,7 +97,7 @@ def test_video_set_gives_every_state_the_same_body_height(tmp_path: Path, monkey
seen: list = []
cycles: dict = {}
- def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state"):
+ def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state", view=None):
Image.new("RGB", (32, 32), (0, 255, 0)).save(out)
return {"shape": "square", "canvas": [32, 32], "offset": [0, 0]}
diff --git a/tests/video/test_edge_speck_lead_in.py b/tests/video/test_edge_speck_lead_in.py
index c6ec2c1..c7c6ee6 100644
--- a/tests/video/test_edge_speck_lead_in.py
+++ b/tests/video/test_edge_speck_lead_in.py
@@ -371,8 +371,12 @@ def test_a_walk_or_run_forced_wide_gets_no_room_above():
assert canvas.profile_for(state, shape="wide") is canvas.WIDE_OVERRIDE
_, jump = canvas.pad_canvas(still, canvas.profile_for("jump", shape="wide"))
assert jump["canvas"] == [2800, 1575]
- # Unforced, a walk is square as before.
- assert canvas.profile_for("walk") is canvas.STATE_CANVAS["default"]
+ # Unforced, a walk is square as before: its own row frames a still as the default does, and reads the still's
+ # room only from the side (tests/video/test_walk_canvas_room.py).
+ walk, default = canvas.profile_for("walk"), canvas.STATE_CANVAS["default"]
+ assert walk is canvas.STATE_CANVAS["walk"] and walk.room and not default.room
+ assert (walk.shape, walk.ratio, walk.headroom, walk.lead, walk.trail, walk.why) == (
+ default.shape, default.ratio, default.headroom, default.lead, default.trail, default.why)
assert math.isclose(canvas.GAIT_WIDE.ratio, 16 / 9)
diff --git a/tests/video/test_fit_tight.py b/tests/video/test_fit_tight.py
index fac1131..c0441d5 100644
--- a/tests/video/test_fit_tight.py
+++ b/tests/video/test_fit_tight.py
@@ -189,7 +189,7 @@ def test_the_cli_takes_the_new_flags() -> None:
def test_video_set_fit_tight_reaches_canvas_and_frames(tmp_path: Path, monkeypatch) -> None:
seen: dict[str, object] = {}
- def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state"):
+ def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state", view=None):
seen["fit"] = fit
Image.new("RGB", (32, 18), GREEN).save(out)
return {"fit": fit, "shape": "wide", "canvas": [32, 18], "offset": [0, 0]}
diff --git a/tests/video/test_gait_guard_anchor_wide.py b/tests/video/test_gait_guard_anchor_wide.py
index 30e6ab9..ec2e7ad 100644
--- a/tests/video/test_gait_guard_anchor_wide.py
+++ b/tests/video/test_gait_guard_anchor_wide.py
@@ -202,7 +202,7 @@ def test_raised_limb_states_get_a_wide_canvas() -> None:
def test_video_set_passes_shape_and_anchor_through(tmp_path: Path, monkeypatch, state, anchor) -> None:
seen: dict[str, object] = {}
- def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state"):
+ def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state", view=None):
seen["shape"] = shape
Image.new("RGB", (32, 32), (0, 255, 0)).save(out)
return {"shape": shape or "square", "canvas": [32, 32], "offset": [0, 0]}
diff --git a/tests/video/test_idle_still.py b/tests/video/test_idle_still.py
index f9d024d..4933844 100644
--- a/tests/video/test_idle_still.py
+++ b/tests/video/test_idle_still.py
@@ -67,7 +67,7 @@ def test_the_cli_takes_pinned() -> None:
def test_video_set_pins_idle_and_cuts_it_whole(tmp_path: Path, monkeypatch) -> None:
seen: dict[str, object] = {}
- def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state"):
+ def fake_canvas(base, out, *, state, shape, facing, headroom, lead, report_path, fit="state", view=None):
Image.new("RGB", (32, 32), (0, 255, 0)).save(out)
return {"fit": fit, "shape": "square", "canvas": [32, 32], "offset": [0, 0]}
diff --git a/tests/video/test_walk_canvas_room.py b/tests/video/test_walk_canvas_room.py
new file mode 100644
index 0000000..79abf87
--- /dev/null
+++ b/tests/video/test_walk_canvas_room.py
@@ -0,0 +1,190 @@
+# SPDX-License-Identifier: Apache-2.0
+"""A walk or run filmed from the side is framed on a square that keeps GAIT_ROOM of the silhouette's height clear in
+front of it and, more, behind it: a still that leaves less — a big tail reaching for the frame's edge behind the body — gets
+the columns it lacks on that side, the square's rows to match above it, and the report says why
+(`canvas.gait_room`, docs/video-pipeline.md section 1). A still that leaves the room keeps its square to the byte.
+Front, back and diagonal walks are not read."""
+from __future__ import annotations
+
+import json
+import math
+from pathlib import Path
+
+import pytest
+from PIL import Image, ImageOps
+
+from sprite_gen.video import batch, canvas
+
+GREEN = (0, 255, 0)
+FUR = (150, 90, 40)
+TAILED_H = 320 # the tailed still's silhouette, rows 60..380
+
+
+def _tailed(facing: str = "right") -> Image.Image:
+ """A side still 400 square, silhouette rows 60..380 (320 tall): a body, a head in front, and a tail behind it
+ reaching to 6 px of the frame's edge (0.019 of the silhouette's height); 60 px clear in front (0.188, as much as
+ the stride needs)."""
+ still = Image.new("RGB", (400, 400), GREEN)
+ still.paste(FUR, (120, 160, 300, 380)) # body and legs
+ still.paste(FUR, (260, 60, 340, 170)) # head, in front
+ still.paste(FUR, (6, 150, 120, 200)) # tail, behind
+ return still if facing == "right" else ImageOps.mirror(still)
+
+
+def _lacks(room_px: int, side: str = "behind", height: int = TAILED_H) -> int:
+ return max(0, math.ceil(canvas.GAIT_ROOM[side] * height - room_px))
+
+
+def _narrow() -> Image.Image:
+ """A person's side still: 60 px wide, 350 tall, 170 px clear on either side."""
+ still = Image.new("RGB", (400, 400), GREEN)
+ still.paste(FUR, (170, 40, 230, 390))
+ return still
+
+
+def _clip_runner(sizes: list[tuple[int, int]]):
+ """A clip runner that records the canvas it is given and makes no clip (the item fails after its canvas)."""
+ def video(image, prompt, out, report, **kw):
+ with Image.open(image) as im:
+ sizes.append(im.size)
+ return 1
+ return video
+
+
+def _side_item(tmp_path: Path, still: Image.Image, facing: str, **options):
+ base = tmp_path / f"base-{facing}.png"
+ still.save(base)
+ sizes: list[tuple[int, int]] = []
+ result = batch.run_item(item=f"side-{facing}-walk", direction="side", state=options.pop("state", "walk"), base=base,
+ root=tmp_path / "set", character=None, duration=3, resolution="720p", key="green", force=False,
+ gap=0, video_runner=_clip_runner(sizes), facing=facing,
+ prepare_side=lambda b: (b, {"requested": facing, "fix": "none", "action": "none"}), **options)
+ item = tmp_path / "set" / f"side-{facing}-walk"
+ return result, json.loads((item / "canvas.report.json").read_text()), sizes, Image.open(item / "canvas.png")
+
+
[email protected]("facing", ["right", "left"])
+def test_video_set_widens_a_side_walks_square_by_the_room_its_tail_lacks(tmp_path, facing):
+ result, report, sizes, made = _side_item(tmp_path, _tailed(facing), facing)
+ # Not filmed on the still's own 400 square, whose edge the tail is 6 px from: on a larger square.
+ assert len(sizes) == 1 and sizes[0][0] == sizes[0][1] > 400, f"filmed on {sizes}"
+ add = _lacks(6)
+ side = 400 + add
+ assert _lacks(60, "front") == 0
+ # Square still, the columns the tail lacks behind it, the rows to match above the still; nothing scaled.
+ assert add > 0 and sizes == [(side, side)] and report["shape"] == "square" and report["canvas"] == [side, side]
+ x = add if facing == "right" else 0
+ assert report["offset"] == [x, add]
+ assert made.crop((x, add, x + 400, add + 400)).tobytes() == _tailed(facing).tobytes()
+ behind = (0, add) if facing == "right" else (400, side)
+ assert made.crop((behind[0], 0, behind[1], side)).getextrema() == ((0, 0), (255, 255), (0, 0))
+ room = report["room"]
+ assert room["view"] == "side" and room["short"] == ["behind"] and room["added_px"] == {"front": 0, "behind": add}
+ assert room["added_rows"] == add
+ assert (room["behind_px"], room["front_px"]) == (6, 60) and room["silhouette"][1:4:2] == [60, 380]
+ assert room["behind"] == round(6 / TAILED_H, 4) and room["need"] == canvas.GAIT_ROOM
+ assert room["need_px"] == {side: round(canvas.GAIT_ROOM[side] * TAILED_H, 1) for side in ("front", "behind")}
+ assert report["why"] == canvas.GAIT_ROOM_WHY
+ # The set's report says it too: the view read, the room behind, the columns and rows the square gained, and why.
+ said = result["canvas"]["room"]
+ assert result["canvas"]["canvas"] == [side, side] and said["view"] == "side" and said["short"] == ["behind"]
+ assert (said["behind_px"], said["added_px"], said["added_rows"]) == (6, {"front": 0, "behind": add}, add)
+ assert result["canvas"]["why"] == canvas.GAIT_ROOM_WHY
+
+
+def test_a_side_walk_with_room_keeps_the_square_byte_for_byte(tmp_path):
+ result, report, sizes, made = _side_item(tmp_path, _narrow(), "right")
+ assert sizes == [(400, 400)] and report["shape"] == "square" and report["offset"] == [0, 0]
+ assert report["room"]["short"] == [] and report["room"]["added_px"] == {"front": 0, "behind": 0}
+ assert report["why"] == canvas.STATE_CANVAS["default"].why
+ assert "why" not in result["canvas"] and result["canvas"]["room"]["front"] == round(170 / 350, 4)
+ # The canvas is the one the default row makes, and the report that row's plus the room it read.
+ padded, before = canvas.pad_canvas(_narrow(), canvas.STATE_CANVAS["default"], facing="right")
+ assert made.tobytes() == padded.tobytes()
+ assert {k: v for k, v in report.items() if k in before} == json.loads(json.dumps(before))
+
+
[email protected]("view", ["front", "back", "front_diagonal", "back_diagonal", None])
+def test_a_walk_not_filmed_from_the_side_is_not_read(tmp_path, view):
+ still = tmp_path / "still.png"
+ _tailed().save(still)
+ report = canvas.run_canvas(still, tmp_path / "c.png", state="walk", shape=None, facing="right", headroom=None,
+ lead=None, report_path=None, view=view)
+ assert report["shape"] == "square" and report["canvas"] == [400, 400] and "room" not in report
+
+
+def test_the_bounds_are_gait_room_of_the_silhouettes_height_more_behind_than_in_front():
+ assert canvas.GAIT_ROOM["behind"] > canvas.GAIT_ROOM["front"] > 0
+ need = {side: canvas.GAIT_ROOM[side] * 300 for side in ("front", "behind")}
+ for behind, short in ((math.ceil(need["behind"]), []), (math.ceil(need["behind"]) - 1, ["behind"])):
+ still = Image.new("RGB", (400, 400), GREEN)
+ still.paste(FUR, (behind, 50, 300, 350)) # 300 tall, 100 px clear in front
+ _, report = canvas.pad_canvas(still, canvas.profile_for("walk"), facing="right", view="side")
+ assert report["room"]["behind_px"] == behind and report["room"]["short"] == short
+ assert report["canvas"] == ([400, 400] if not short else [401, 401]) and report["offset"][0] == len(short)
+ # In front too, at its own bound: a stride reaches ahead of the silhouette. Both short: each side gains what it lacks.
+ still = Image.new("RGB", (400, 400), GREEN)
+ still.paste(FUR, (2, 50, 400 - math.ceil(need["front"]) + 1, 350))
+ _, report = canvas.pad_canvas(still, canvas.profile_for("walk"), facing="right", view="side")
+ lacks = {"front": 1, "behind": math.ceil(need["behind"] - 2)}
+ assert report["room"]["short"] == ["front", "behind"] and report["room"]["added_px"] == lacks
+ assert report["room"]["added_rows"] == lacks["front"] + lacks["behind"]
+ side = 400 + lacks["front"] + lacks["behind"]
+ assert report["canvas"] == [side, side] and report["offset"] == [lacks["behind"], side - 400]
+
+
+def test_the_room_is_read_in_the_square_the_row_makes_of_the_still():
+ # 300 wide and 400 tall: the square is 400 wide with the still centred, 50 px of it on either side of the still.
+ still = Image.new("RGB", (300, 400), GREEN)
+ still.paste(FUR, (6, 60, 250, 380)) # 6 px from the still's own edge behind, 50 in front
+ padded, report = canvas.pad_canvas(still, canvas.profile_for("walk"), facing="right", view="side")
+ assert (report["room"]["behind_px"], report["room"]["front_px"]) == (56, 100)
+ add = _lacks(56)
+ assert padded.size == (400 + add, 400 + add) and report["offset"] == [50 + add, add]
+
+
+def test_a_white_base_is_read_on_its_corner_colour():
+ ivory = (246, 244, 236)
+ still = Image.new("RGB", (400, 400), ivory)
+ still.paste(FUR, (120, 160, 300, 380))
+ still.paste(FUR, (260, 60, 340, 170))
+ still.paste(FUR, (6, 150, 120, 200))
+ still.putpixel((2, 2), (250, 250, 250)) # within the corner tolerance: background, not subject
+ padded, report = canvas.pad_canvas(still, canvas.profile_for("walk"), facing="right", key="white", view="side")
+ assert report["room"]["behind_px"] == 6 and padded.size == (400 + _lacks(6),) * 2
+ assert padded.getpixel((0, 0)) == ivory
+
+
+def test_a_run_is_read_at_the_walks_bound_and_a_named_shape_or_tight_fit_is_not_read(tmp_path):
+ still = tmp_path / "still.png"
+ _tailed().save(still)
+ def frame(**kw):
+ options = {"state": "walk", "shape": None, "facing": "right", "headroom": None, "lead": None, "report_path": None,
+ "view": "side", **kw}
+ return canvas.run_canvas(still, tmp_path / "c.png", **options)
+ widened = [400 + _lacks(6)] * 2
+ assert frame()["canvas"] == widened and frame(state="run")["canvas"] == widened
+ assert frame(state="run")["room"]["short"] == ["behind"]
+ # `--shape square` keeps the still's square whatever its room; `--shape wide` is the walk's wide row, as before.
+ assert frame(shape="square")["canvas"] == [400, 400] and "room" not in frame(shape="square")
+ forced = frame(shape="wide")
+ assert forced["shape"] == "wide" and forced["why"] == canvas.GAIT_WIDE.why and "room" not in forced
+ # A tight canvas adds no room, by its own rule.
+ assert "room" not in frame(fit="tight") and frame(fit="tight")["fit"] == "tight"
+ # Any other state keeps its row.
+ assert frame(state="idle")["canvas"] == [400, 400] and "room" not in frame(state="idle")
+
+
+def test_video_canvas_view_from_the_command_line(tmp_path, capsys):
+ still = tmp_path / "still.png"
+ _tailed("left").save(still)
+ report = tmp_path / "r.json"
+ assert canvas.main(["--still", str(still), "--out", str(tmp_path / "c.png"), "--state", "walk", "--facing", "left",
+ "--view", "side", "--report", str(report)]) == 0
+ written = json.loads(report.read_text())
+ add = _lacks(6)
+ assert written["canvas"] == [400 + add] * 2 and written["offset"] == [0, add]
+ assert written["room"]["short"] == ["behind"] and written["room"]["behind_px"] == 6
+ assert json.loads(capsys.readouterr().out) == written
+ with pytest.raises(SystemExit):
+ canvas.main(["--still", str(still), "--out", str(tmp_path / "c.png"), "--state", "walk", "--view", "top"])