Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
fix(video-loop): an eased stretch never leaves more than half the loop made
Soohong Kim9dd5050e4665
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6cd37b4..813003c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,8 +4,8 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the## Unreleased-- **New: `video-loop --repair auto` (and `on`) eases a part that rushes on its way — the frames either side of the rush are made again at times that spread it, the body's own steps kept.** A part can keep its way and still jump: a tail that runs short of the cycle rushes on at the wrap, covering in one step what it covers in eight elsewhere, and each frame there lies near enough the line between its neighbours that the flow reading lets it pass, or takes the frame at the wrap out and leaves the rush over two steps. `repair.surges` reads the speed on the flow reading's own path and windows: a run of one or two steps whose mean step is at least 3.0 usual steps and 3.0 times the mean of the three steps either side, the part going on the way it came through it (a step that turns back is a frame flung, the flow repair's). After the flow groups, the worst rush is eased over three or four drawings either side, made from the frames as filmed (a frame the flow made inside is made again, `supersedes`) and kept between two frames as filmed: `repair.ease_times` gives every frame a time, on a grid of a twentieth of a step, where the part's step changes from one step to the next as little as it can — from the speed it came in with to the speed it goes on with — while the foot line's centre and the body's mass (`loop.foot_centre`, `loop.body_centre`) and the lowest solid row move in a step no further than the stretch's largest step as it played and half a pixel, and every other window of the flow reading no further than its own largest step there; within 2 % of that, the timing nearest the frames' own. How far the part has gone is the path of its coverage's centre in the windows that read the rush, which grows as far as the part moves where a blurred distance stops growing. RIFE makes each drawing between the frames as filmed either side of its time, one call a drawing, at most nine, outside the three of the flow groups; the stretch is taken only where no frame made has an interpolation fault, no body measure moves a pixel past the stretch's largest step, the part's largest step falls, no frame is left further off the flow and over its limit, and no new rush appears in the rest of the body (`rejected_for`). On the synthetic walker the tail a tenth of a cycle short — under the flow reading's limit — goes from 5.8 usual steps into the wrap to at most 2.7; a quarter short, after the flow repair, from 7.2 to 5.0 and still read a rush, as far as the legs allow; ordinary walks and a flung frame read no rush. A loop with no rush costs one more reading and no call. `jump_repair` carries `surge`, `surge_after`, `ease_skipped`, `ease_calls` and the round `why: ease`, `replaced` its frames, and the summary `eased`; without RIFE the rush is cut as filmed with a warning line, and `--repair on` refuses ([loop-repair](docs/loop-repair.md) section 2, "The eased retime").-- **Tests** — `tests/video/test_loop_ease.py` (a tail that rushes on at the wrap after the flow repair is eased over three or four drawings either side, in play order, made only from frames as filmed — the flow's frame at the wrap made again — every frame outside as it was, the part's largest step down by a quarter or more and the body's feet, mass and ground within the stretch's own steps and a pixel; a rush under the flow reading's limit eased, none left; ordinary walks, drawn every frame and on twos, have no rush and nothing is made; a frame flung against the way is not a rush; on twos each drawing made once and shown on both its frames; a made frame pushed sideways refuses the stretch for drift, and one with its outline melted for a fault, the loop left as the flow left it; without an interpolator the rush named and nothing made; the same frames whichever way the body faces; the search keeps every frame at its own time where the part moves evenly, and spreads a rush as far as the body's bound lets it; `video-loop` playing the frames made in a loop as long, a rush without RIFE cut as filmed with a warning line, `--repair on` without RIFE refused), and `tests/video/test_loop_flow.py`'s walker draws its dark parts whole on asking (`dark_whole`), so that a frame drawn between two phases has no more dark pixels than they have.+- **New: `video-loop --repair auto` (and `on`) eases a part that rushes on its way — the frames either side of the rush are made again at times that spread it, the body's own steps kept.** A part can keep its way and still jump: a tail that runs short of the cycle rushes on at the wrap, covering in one step what it covers in eight elsewhere, and each frame there lies near enough the line between its neighbours that the flow reading lets it pass, or takes the frame at the wrap out and leaves the rush over two steps. `repair.surges` reads the speed on the flow reading's own path and windows: a run of one or two steps whose mean step is at least 3.0 usual steps and 3.0 times the mean of the three steps either side, the part going on the way it came through it (a step that turns back is a frame flung, the flow repair's). After the flow groups, the worst rush is eased over three or four drawings either side, made from the frames as filmed (a frame the flow made inside is made again, `supersedes`) and kept between two frames as filmed, never leaving more than half the loop made with the rounds before it (as much as `video-loop-compare` reads back as the repair's): `repair.ease_times` gives every frame a time, on a grid of a twentieth of a step, where the part's step changes from one step to the next as little as it can — from the speed it came in with to the speed it goes on with — while the foot line's centre and the body's mass (`loop.foot_centre`, `loop.body_centre`) and the lowest solid row move in a step no further than the stretch's largest step as it played and half a pixel, and every other window of the flow reading no further than its own largest step there; within 2 % of that, the timing nearest the frames' own. How far the part has gone is the path of its coverage's centre in the windows that read the rush, which grows as far as the part moves where a blurred distance stops growing. RIFE makes each drawing between the frames as filmed either side of its time, one call a drawing, at most nine, outside the three of the flow groups; the stretch is taken only where no frame made has an interpolation fault, no body measure moves a pixel past the stretch's largest step, the part's largest step falls, no frame is left further off the flow and over its limit, and no new rush appears in the rest of the body (`rejected_for`). On the synthetic walker the tail a tenth of a cycle short — under the flow reading's limit — goes from 5.8 usual steps into the wrap to at most 2.7; a quarter short, after the flow repair, from 7.2 to 5.0 and still read a rush, as far as the legs allow; ordinary walks and a flung frame read no rush. A loop with no rush costs one more reading and no call. `jump_repair` carries `surge`, `surge_after`, `ease_skipped`, `ease_calls` and the round `why: ease`, `replaced` its frames, and the summary `eased`; without RIFE the rush is cut as filmed with a warning line, and `--repair on` refuses ([loop-repair](docs/loop-repair.md) section 2, "The eased retime").+- **Tests** — `tests/video/test_loop_ease.py` (a tail that rushes on at the wrap after the flow repair is eased over three or four drawings either side, in play order, made only from frames as filmed — the flow's frame at the wrap made again — every frame outside as it was, the part's largest step down by a quarter or more and the body's feet, mass and ground within the stretch's own steps and a pixel; a rush under the flow reading's limit eased, none left; ordinary walks, drawn every frame and on twos, have no rush and nothing is made; a frame flung against the way is not a rush; on twos each drawing made once and shown on both its frames; a made frame pushed sideways refuses the stretch for drift, and one with its outline melted for a fault, the loop left as the flow left it; without an interpolator the rush named and nothing made; the same frames whichever way the body faces; a fourteen-frame loop eased three a side where four would leave more than half of it made; the search keeps every frame at its own time where the part moves evenly, and spreads a rush as far as the body's bound lets it; `video-loop` playing the frames made in a loop as long, a rush without RIFE cut as filmed with a warning line, `--repair on` without RIFE refused), and `tests/video/test_loop_flow.py`'s walker draws its dark parts whole on asking (`dark_whole`), so that a frame drawn between two phases has no more dark pixels than they have.- **Behavior change: a frame the jump repair or the cycle alignment makes between outlined drawings loses its outline only where it is lighter than its neighbours' edge rank by rank, too.** The fault `outline` read the made frame's edge against one dark line (a solid pixel under 70/255 within 2 px, `outline_loss` over 5 %), so art outlined in a mid tone — a brown or a painted line whose tone lies about 70/255 — had its line on the near side of the line in the drawings and on the far side wherever the flow softened it by a few tones: every frame made between two such drawings read as melted, and the repair and `--between auto` refused it. `rife.outline_shift(made, a, b)` reads the same edge, each pixel by how dark the darkest solid pixel within 2 px of it is, in order, against the lighter neighbour's edge at the same rank, and counts it where it is lighter by more than 50/255 (`rife.OUTLINE_SHIFT`). The fault `outline` now needs both over 5 %: a softened line is lighter by less at every rank and is kept, whatever tone the art outlines in; a line gone to the fill beside it is lighter by its whole contrast and is lost. Art outlined near-black reads the same on both, and art outlined lighter than 70/255 is still not judged by the outline. On the two-tone walker of the tests, the drawing between with its line 20 to 45 tones lighter reads `outline_loss` 57 % and `outline_shift` 0 (refused before, kept now); with its legs' line turned to fill, 24 % and 48 % (refused). A line less than 50/255 darker than the fill it melts into is not read as lost. The rank reading is read only between outlined drawings: `rife.outlined(frame)` is the share of a frame's edge whose line, the darkest solid pixel within 2 px, is darker than its fill, its colour from twice the bleed's depth inside (`rife.LINE_DEPTH`), by more than 50/255, and both neighbours must read at least 75 % (`rife.OUTLINED`); elsewhere `outline_shift` is `null` and the dark line judges alone, as before. Art painted without a line has no line to lose, and a part the flow melts there — a boot washed pale — can come out lighter by less than 50/255 at every rank: read rank by rank it would be kept and delivered, where refused it is replaced by the nearer filmed frame. On the walker of the tests painted without a line (outlined on 4 % of its edge), the drawing between with its near boot washed 40 tones lighter reads `outline_loss` 7.8 % and, rank by rank, 0: refused. The 75 % is set on frames made between drawn walks and judged by eye so that no frame seen as broken that the dark line alone refused is kept. The jump repair's proposals and the alignment's `smear` rows carry `outline_shift`, and the alignment's warning line names it where it was read; `video-loop-compare` and `video-loop-repair` keep their versioned measures, `rife.smear` alone, and judge the outline on the dark line as before ([loop-repair](docs/loop-repair.md) section 4).- **Tests** — `tests/video/test_outline_shift.py` (a mid-tone line drawn lighter is kept and one gone to fill is lost; the versioned measures judge on the dark line alone, and so does a measure whose rank reading was not read; art outlined near-black reads the same on both; a frame with no edge reads 0 and neighbours with none are not read; the jump repair takes a lighter line and refuses one gone to fill; the alignment keeps the one and takes the nearer frame for the other; on the walker painted without a line, a boot washed 40 tones lighter is refused by the jump repair and replaced by the alignment, and the two-tone and black walkers read outlined on all their edge; through RIFE, a short step of two-tone legs keeps its outline and legs crossing far lose it on both).- **Behavior change: every RIFE call runs the binary's test-time augmentation, `-x -z` (`rife.TTA`).** `-x` runs the pair as its eight flips and transposes and `-z` both ways in time, and the results are averaged, so a frame made from a to b is the one made from b to a. Neither rife-ncnn-vulkan nor Practical-RIFE has released a model since 2024; the model stays `rife-v4.6`. On the synthetic pairs of `tests/video/test_rife.py`, `-x -z` is the one setting of the four (none, `-x`, `-z`, both) that makes legs crossing far the same both ways with no ghost, and it keeps the most of a short step's outline. On the Mac's GPU it adds a tenth to half a second a frame; on Modal's CPU it costs 15 to 19 times a plain call's CPU — 12 to 30 s a 280 × 520 frame on two cores, 44 to 61 s a 960 × 960 frame. `SPRITE_GEN_RIFE_TTA` names another (`-x`, `-z`, both, or empty for none; anything else is refused before a frame is asked for), for a CPU-only machine that aligns sets. The interpolator a report names (`jump_repair.interpolator`, the alignment's `interpolator`) carries `tta`. A call past 600 s (120 s before) is stopped and reported as a RIFE that failed, where it used to end in Python's own timeout ([loop-repair](docs/loop-repair.md) section 1, "Which setting").diff --git a/docs/loop-repair.md b/docs/loop-repair.mdindex 75b0142..5c2ab42 100644--- a/docs/loop-repair.md+++ b/docs/loop-repair.md@@ -470,7 +470,9 @@ for a two-step one nine. It is made from the frames as filmed: a drawing the floinside it is made again (`supersedes`), never made from, and a stretch never ends on one. A stretchnever holds a frame given way, refused, or read off the flow turning back. Four a side is takenonly where it eases the part more than three by over 2 % (`EASE_SLACK`); one stretch a loop, the-worst rush that has one.+worst rush that has one. With the frames the ghost and flow rounds replaced, a stretch makes at+most half the loop (`MADE_SHARE`): `video-loop-compare` reads no more than that back as the+repair's (its source restoration, [loop comparison](loop-comparison.md)).**The times.** How far the part has gone is the path of its coverage's centre inside each windowthat reads the rush, in px, through the frames as filmed: a centre moves as far as the part does,@@ -526,8 +528,8 @@ loop that does not, nothing.`jump_repair` carries `surge` (the worst rush on the loop the flow groups left: `from`, `to`,`steps`, `frames`, `windows`, `ratio`, `speed`, `around`), `surge_after` (on the loop as it nowplays, or `null`), and `ease_skipped` where a rush was read and no stretch was made (`no-interpolator`, `the loop is too short to ease it`, `the frames around it were given way, refused,-or flung against the way`, `no timing keeps the body in bound`). The round `why: ease` carries the+interpolator`, `the stretch would leave more than half the loop made`, `the frames around it were+given way, refused, or flung against the way`, `no timing keeps the body in bound`). The round `why: ease` carries the`surge` it eased, `width`, `anchors` (the frames kept at its ends), `frames` and `drawings` (thestretch in play order), `sources` (the frames as filmed it was made from), `supersedes`, `span`(frames from anchor to anchor), `made` (per drawing: `frames`, `time` and `own` in frames from thediff --git a/sprite_gen/video/repair.py b/sprite_gen/video/repair.pyindex 9f8786a..0c4bed5 100644--- a/sprite_gen/video/repair.py+++ b/sprite_gen/video/repair.py@@ -364,6 +364,9 @@ EASE_SLACK = 0.02 # of the timings within this share of the least change of theDRIFT_PIXEL = 1.0EASE_MARGIN = 0.5DRIFT_MEASURES = ("foot_centre", "body_centre", "ground")+# With the frames the ghost and flow rounds replaced, a stretch leaves at most this share of the loop made:+# `video-loop-compare` reads no more back as the repair's (`source.py`, "repair-locations-unverified").+MADE_SHARE = 0.5def drift_points(frame: Image.Image) -> np.ndarray:@@ -519,13 +522,15 @@ def _part_accel(progress: np.ndarray, at: list[float], times: np.ndarray, into:def _ease(frames: list[Image.Image], out: list[Image.Image], interpolate: Interpolate | None,- box: tuple[int, int, int, int], blocked: set[int], made: set[int], reads: bool,+ box: tuple[int, int, int, int], blocked: set[int], made: set[int], taken: set[int], reads: bool,record: dict[str, Any], ease: bool) -> list[Image.Image]:"""The eased retime stage of `repair_jumps`, on the loop `out` as the ghost and flow rounds left it.`blocked` holds the frames no stretch may hold (given way, refused; the frames read off the flow against the wayare added here), `made` the frames the flow rounds made: a stretch makes them again, never from them, and never- ends on one. Records `surge` and, where a stretch was made, an `ease` round (`ease_calls`); `ease_skipped` says why+ ends on one. `taken` holds every frame the rounds replaced: with the stretch they are at most MADE_SHARE of the+ loop. Records `surge` and, where a stretch+ was made, an `ease` round (`ease_calls`); `ease_skipped` says whynone was. With `ease` false the surge is read and nothing made."""flow = _flow(out, box)found = _surges(flow)@@ -549,10 +554,10 @@ def _ease(frames: list[Image.Image], out: list[Image.Image], interpolate: Interpfor s in found:for k in EASE_WIDTHS:T = s["r"] + 2 * k- if T >= n:- why = "the loop is too short to ease it"- continuestretch = [(s["d"] - k + i) % n for i in range(T + 1)] # drawings in play order, the two kept at the ends+ if T >= n or len(taken | {t for d in stretch[1:-1] for t in runs[d]}) > int(count * MADE_SHARE):+ why = "the stretch would leave more than half the loop made"+ continueends = (stretch[0], stretch[-1])if any(t in blocked for d in stretch for t in runs[d]) or any(t in made for d in ends for t in runs[d]):continue@@ -817,7 +822,7 @@ def repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *,off_max_after=plan["off_max_before"])given = {j for j in replaced if j in ghosts}blocked = given | set(refused)- out = _ease(frames, out, interpolate, box, blocked, set(replaced) - given, reads, record, ease)+ out = _ease(frames, out, interpolate, box, blocked, set(replaced) - given, set(replaced), reads, record, ease)eased = [r for r in record["rounds"] if r["why"] == "ease" and r["outcome"] == "accepted"]if eased:record["replaced"] = [*replaced, *(t for t in eased[0]["replaced"] if t not in replaced)]diff --git a/tests/video/test_loop_ease.py b/tests/video/test_loop_ease.pyindex e25d624..8583566 100644--- a/tests/video/test_loop_ease.py+++ b/tests/video/test_loop_ease.py@@ -290,3 +290,16 @@ def test_repair_on_without_rife_refuses_a_rush(tmp_path, monkeypatch):_no_rife(monkeypatch, tmp_path)with pytest.raises(SystemExit, match=r"for a part that rushes.*sprite-gen rife install"):_cut(tmp_path, _walk(lag=0.1), repair="on")+++def test_a_stretch_never_leaves_more_than_half_the_loop_made(monkeypatch):+ # Fourteen frames: four drawings a side would remake eight, more than the seven video-loop-compare reads back as the+ # repair's; three a side remakes six.+ frames = _walk(14, lag=0.15)+ monkeypatch.setattr(repair, "MADE_SHARE", 1.0)+ _, unbound = repair.repair_jumps(frames, Lerp(frames, lag=0.15, n=14))+ assert _ease(unbound)["width"] == 4 and len(unbound["replaced"]) == 8+ monkeypatch.setattr(repair, "MADE_SHARE", 0.5)+ _, record = repair.repair_jumps(frames, Lerp(frames, lag=0.15, n=14))+ round_ = _ease(record)+ assert round_["outcome"] == "accepted" and round_["width"] == 3 and len(record["replaced"]) <= 14 // 2