Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

fix(video-loop): a frame the overlapped wrap makes is read with RIFE's single passes, and one torn where they disagree ends its run

Soohong Kima6fc0de40fa2
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2442e4e..1befd44 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,8 +4,8 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the
## Unreleased
-- **New: `video-loop --anchor motion-auto` closes a two-step cut's wrap that a part on its own beat leaves open, with the clip's frames a cycle apart.** A walk cut two steps long by the clip's crossings closes for the legs; a tail swinging round on a beat of its own closes on no cut of two steps, so the seam gate refused every one — and the eased retime does not reach it, since the part turns back at the wrap, where its frames read off the flow against the way. The clip goes on past the cut, and its frame a cycle after one of the cut's shows the legs at the same point of the walk. Under `--repair auto` (and `on`), where such a cut's wrap reads over `--seam-max` on its frames as corrected (`loop.wrap_on_cells`), the frames beside the wrap are made again before the jump repair (`repair.overlap_wrap`): each by RIFE between its two frames of one time — the clip's frame a cycle apart, moved by the analysis translation between the two and corrected as the cut frame was (`loop.cycle_apart`), and the cut's own — so the wrap plays a step the clip filmed and the part goes over to the cut's way over those frames, each frame at its own time. The weights (`repair.overlap_weights`) are the ease in and out x²(3 − 2x), held off it only in the steps where the rest of the body or a body measure, read on the line between the two frames, would go past its bound; a weight of 0 or 1 shows a frame whole, with no call, and a frame at one weight is made once, whatever the run. Four frames, then six, eight, ten and twelve (`OVERLAP_WIDTHS`), on the side — after the wrap or before it — whose frames a cycle apart differ least, never past half the loop; a run is taken only where no made frame has an interpolation fault, the gate passes it, the loop played round still covers two steps (`loop.ring_steps`), no body measure — the foot line, the torso, the ground — moves a pixel past the largest step of the stretch as the loop plays it (the eased retime's bound on the same torso, never the clip's frames a cycle apart: the part going over pulls the body's mass, which is reported, while a torso the clip's frame a cycle on shows elsewhere would slide) — the part steps no further than over the stretch, no new rush is read, no window of the rest of the body steps further than its own largest step over the stretch, and no frame is left further off the flow over its limit (`rejected_for`). The jump repair after it makes nothing from, beside or again over those frames (`jump_repair.overlapped`). The report carries `wrap_overlap` (each try, and the run taken), the summary names it, and without RIFE the wrap is cut as filmed with a warning line (`--repair on` refuses). It costs at most forty RIFE calls, where every width is made and refused. On the test walker (front, 26 frames a cycle, the tail on a 41-frame beat) the cut 29/26 reads 6.23 at the gate as filmed and 3.17 after the jump repair alone; four frames after the wrap rush the tail and are refused, six are taken, and the loop passes at 1.04, two steps, in ten calls. On a second walker whose big tuft bobs on the loop and a cycle on is carried 8 px aside a frame, the walker in its place, four frames are taken at the ease (3.41 → 1.53 at the gate, four calls): the torso moves 0.05 px, the body's mass 3.95 px against the stretch's 0.71, reported; where the whole walker also stands 3 px aside a cycle on, its torso sliding, every run is held at 0.85 of the cut's own frame next to the wrap, the wrap stays open and the cut is kept as filmed for the gate to refuse ([loop-repair](docs/loop-repair.md#the-overlapped-wrap--a-part-on-its-own-beat-that-no-cut-of-two-steps-closes)).
-- **Tests** — `tests/video/test_loop_overlap.py` (a front walker whose tail swings round on its own beat, refused at the seam on every cut of two steps, is closed at the wrap: each frame beside it made between the clip's frame a cycle apart and the cut's own at its eased weight, once a frame and weight, four frames refused for a rush and six taken, two steps on the loop played round, the body within its drift, the rest of it no faster, every other frame the clip's own and nothing made by the jump repair; on a walker whose big tuft a cycle on is carried aside, every run's drift bound is the stretch's own as the loop plays it — the torso's, which stands still there, never the clip's frames a cycle apart — and the body's mass is reported beside it; with the tuft alone carried over the run is taken near the ease, the torso within its pixel while the mass moves 2 px and more past its own; with the whole walker also standing aside a cycle on, its torso sliding, no run moves it more than a pixel in a step on the frames shown, drawn again, the frame next to the wrap is held off the ease and the cut is kept as filmed; `--repair off` refused as before; without RIFE cut as filmed with a warning, and with `--repair on` refused asking for it; a made frame with a fault refuses every run and the cut is kept as filmed; a weighting search on lines drawn by hand: the ease where nothing binds, held off it in the one step a window of the rest of the body is bounded in, none where a body measure may not move).
+- **New: `video-loop --anchor motion-auto` closes a two-step cut's wrap that a part on its own beat leaves open, with the clip's frames a cycle apart.** A walk cut two steps long by the clip's crossings closes for the legs; a tail swinging round on a beat of its own closes on no cut of two steps, so the seam gate refused every one — and the eased retime does not reach it, since the part turns back at the wrap, where its frames read off the flow against the way. The clip goes on past the cut, and its frame a cycle after one of the cut's shows the legs at the same point of the walk. Under `--repair auto` (and `on`), where such a cut's wrap reads over `--seam-max` on its frames as corrected (`loop.wrap_on_cells`), the frames beside the wrap are made again before the jump repair (`repair.overlap_wrap`): each by RIFE between its two frames of one time — the clip's frame a cycle apart, moved by the analysis translation between the two and corrected as the cut frame was (`loop.cycle_apart`), and the cut's own — so the wrap plays a step the clip filmed and the part goes over to the cut's way over those frames, each frame at its own time. The weights (`repair.overlap_weights`) are the ease in and out x²(3 − 2x), held off it only in the steps where the rest of the body or a body measure, read on the line between the two frames, would go past its bound; a weight of 0 or 1 shows a frame whole, with no call, and a frame at one weight is made once, whatever the run. Four frames, then six, eight, ten and twelve (`OVERLAP_WIDTHS`), on the side — after the wrap or before it — whose frames a cycle apart differ least, never past half the loop; a run is taken only where no made frame has an interpolation fault — a part torn among them, below — the gate passes it, the loop played round still covers two steps (`loop.ring_steps`), no body measure — the foot line, the torso, the ground — moves a pixel past the largest step of the stretch as the loop plays it (the eased retime's bound on the same torso, never the clip's frames a cycle apart: the part going over pulls the body's mass, which is reported, while a torso the clip's frame a cycle on shows elsewhere would slide) — the part steps no further than over the stretch, no new rush is read, no window of the rest of the body steps further than its own largest step over the stretch, and no frame is left further off the flow over its limit (`rejected_for`). The jump repair after it makes nothing from, beside or again over those frames (`jump_repair.overlapped`). A frame made between two frames a cycle apart bridges the part's whole move at once, and where the part swings over a smaller one — a tail across a hand — RIFE drags the small part apart with it, which on art painted without a line no other reading sees: each frame is also read with RIFE's sixteen single passes (`rife.Rife.read`: the binary's eight flips and transposes, both ways in time, run one at a time with no augmentation and turned back, `rife.passes`) and refused as `torn` where the forward passes' mean and the backward passes' mean put the colour and coverage more than 0.05 apart over more than 0.21 of a window a tenth of the body high (`rife.torn`, `interpolation_quality.TORN_WARN`); the frame delivered is still the binary's own. The frames are made the weight nearest a half first, and the first frame with a fault ends its run, the rest of it not made (`unmade`), so a clip whose part crosses another on every run is refused the overlapped wrap and kept as filmed for the gate to refuse. The report carries `wrap_overlap` (each try with the weights chosen, the frames made in play order and in the order made, and the run taken; the binary's calls), the summary names it, a warning line names the frame that ended each refused run, and without RIFE the wrap is cut as filmed with a warning line (`--repair on` refuses). Each frame made costs the binary 34 calls (its frame, and its sixteen passes in colour and coverage); at most forty frames, where every width is made whole and refused. On the test walker (front, 26 frames a cycle, the tail on a 41-frame beat) the cut 29/26 reads 6.23 at the gate as filmed and 3.17 after the jump repair alone; four frames after the wrap rush the tail and are refused, six are taken, and the loop passes at 1.04, two steps, in ten calls. On a second walker whose big tuft bobs on the loop and a cycle on is carried 8 px aside a frame, the walker in its place, four frames are taken at the ease (3.41 → 1.53 at the gate, four calls): the torso moves 0.05 px, the body's mass 3.95 px against the stretch's 0.71, reported; where the whole walker also stands 3 px aside a cycle on, its torso sliding, every run is held at 0.85 of the cut's own frame next to the wrap, the wrap stays open and the cut is kept as filmed for the gate to refuse. On a third walker, painted without a line and seen from behind, whose big tail swings from 48 to −8 degrees across its hand between two frames a cycle apart, the real RIFE's frames at weights 0.5 and 0.7 read torn 0.44 and 0.26 with no other fault; swung 10 degrees past no hand, its frames read 0.04 at most ([loop-repair](docs/loop-repair.md#the-overlapped-wrap--a-part-on-its-own-beat-that-no-cut-of-two-steps-closes)).
+- **Tests** — `tests/video/test_loop_overlap.py` (a front walker whose tail swings round on its own beat, refused at the seam on every cut of two steps, is closed at the wrap: each frame beside it made between the clip's frame a cycle apart and the cut's own at its eased weight, once a frame and weight, four frames refused for a rush and six taken, two steps on the loop played round, the body within its drift, the rest of it no faster, every other frame the clip's own and nothing made by the jump repair; on a walker whose big tuft a cycle on is carried aside, every run's drift bound is the stretch's own as the loop plays it — the torso's, which stands still there, never the clip's frames a cycle apart — and the body's mass is reported beside it; with the tuft alone carried over the run is taken near the ease, the torso within its pixel while the mass moves 2 px and more past its own; with the whole walker also standing aside a cycle on, its torso sliding, no run moves it more than a pixel in a step on the frames shown, drawn again, the frame next to the wrap is held off the ease and the cut is kept as filmed; `--repair off` refused as before; without RIFE cut as filmed with a warning, and with `--repair on` refused asking for it; a made frame with a fault ends its run there — the frame nearest a half made first, the rest not made — every width refused and the cut kept as filmed; a stand-in whose passes disagree over a patch of the body ends every run at its first frame as torn, the warning naming it; the real RIFE (marked `real_rife`) drags the painted walker's hand apart between its frames a cycle apart and the frames are refused as torn, no other reading refusing them, while a swing past no hand reads nothing; a weighting search on lines drawn by hand: the ease where nothing binds, held off it in the one step a window of the rest of the body is bounded in, none where a body measure may not move); `tests/video/test_rife.py` (`Rife.read` makes the frame as every call does and then sixteen single passes with no augmentation, the pair flipped and transposed and run both ways in time: with a binary that hands back its first frame, turned back, they disagree by exactly how far the pair differs and only there; `rife.torn` reads a patch a window wide where the passes disagree and not lines a pixel or two wide, scattered pixels, a faint patch or a frame with no body).
- **Behavior change: `video-loop --anchor motion-auto` cuts a walk two steps long — the left foot forward once and the right once — wherever the clip shows its feet pass.** Left foot forward and right foot forward are nearly the same silhouette from the side or a diagonal, so the pixel search could take one step for a cycle (a half step loop: one foot goes forward), three steps where a tail on its own beat repeated there (the wrap puts the nearer foot back: a stomp in place), or refuse a slow walk whose cycle its window could not hold. The feet are now read (`crossings.read`): where they are seen apart, each run of frames with them together is a crossing; where they never are (front, back), the foot band's lean about the body's axis changing side is, and the lean must turn over a step on. Every crossing counted is one the clip shows whole: a run cut by the clip's first or last frame counts only where its twin confirms it, every step a cut plays — on both sides of its wrap — lies within √2 of the clip's own step, and a cut is two steps only when the walk's phase advances by two over it, its wrap joins two points of a step within [1/√2 − 1, √2 − 1] of each other under every bound of what the clip does not show, and the loop as played shows two crossings too (`two`; `refuted` one or three steps; anything else `uncertain`, never two). The search's cut is kept, every image as before, where it is a two within 1.5 frames of phase; otherwise the cut is the two-step cut in phase whose top band does not pop at the wrap, whose wrap is within the seam gate of its step and whose seam is the smoothest by the flow reading (both seam frames under `repair.FLOW_LIMIT`, within 0.5 of the smoothest, the wrap nearest the clip's own way on), read on the frames moved as the cut will be corrected, of any length the count allows inside an explicit `--min-len`/`--max-len` (`method: observed-two-steps-v1`); of the first four, the first whose seam the gate will pass, read as the gate reads it (`loop.seam_on_cells`), is taken. A window that holds no two steps of a walk whose steps are seen is refused, naming both (`no cut of two steps in --max-len 25: …`), never cut one step long. A clip whose feet are not read is cut exactly as before. The report records `cycle.step_count` (the crossings, the step range, the verdict of the cut taken and of the search's cut, `chosen_by`); a cut on two steps records `cycle.steps` `{count: 2, by: crossings}` and `strip.json` `steps: 2`, so `video-cycle-align` reads its half-way return as its step, and the step screen's pixel suspicion is settled (`pixel_suspect`). Where the clip shows its steps but no cut of two, the search runs again refusing the cuts the crossings refute, and a refusal lists each window's count, those refuted last. `--steps` is still the word on the search's cut. `repair.clip_flow_states` and `repair.ring_scores` read the flow of every cut of a clip on one path; `flow_scores` and the repair read as before ([loop-repair](docs/loop-repair.md) section 3, "The step count").
- **Tests** — `tests/video/test_crossings.py` (a walk that stalls proves no two steps across the stall, on either side of the wrap; a hold before or after the cut, a stall with the feet apart, a stand with them together, a stomp in place, the stance a clip opens on, a pass the first frame cuts with or without a twin that confirms it, a change the clip ends on: none two; half a cycle and three steps refuted, a wrap half a step off and a wrap that slips feet together in uncertain; noise at the threshold and a wobble at the axis no steps; a hold the step range absorbs stays two and a longer one does not; two crossings set no range; a lean that does not turn over is not read; every two lies in the lengths two steps cover; the lean and gap off drawn frames), `tests/video/test_walk_steps.py` (on synthetic walkers the pixel search alone cuts one step, one step past its window, three steps by the gait fallback, nothing at all, and across a stalled step: each is cut two steps long, in phase, the strip saying `steps: 2`; a walk the search already cuts two steps in phase is cut byte for byte as before; a legless body is cut as the search cuts it; a count given is the word; the set aligns a loop cut on two steps as one cycle; `detect` refuses what the crossings refute and orders a refusal by the count), the step screen's tests in `tests/video/test_one_step_cut.py` and the pixel search's in `tests/video/test_auto_motion.py`, `tests/video/test_loop_prop_seam.py` and `tests/video/test_loop_prop_seam_held.py` (a walker whose legs open and close once a drawing and never pass each other, which the count reads as one step a drawing) read their walker's feet as unseen (`tests/conftest.py`, `feet_unseen`), and `tests/video/test_explicit_window.py` refuses a `--max-len 25` that holds one step of the 42-frame walker.
- **New: `video-loop --repair auto` (and `on`) eases a part that rushes on its way — the frames either side of the rush are made again at times that spread it, the body's own steps kept.** A part can keep its way and still jump: a tail that runs short of the cycle rushes on at the wrap, covering in one step what it covers in eight elsewhere, and each frame there lies near enough the line between its neighbours that the flow reading lets it pass, or takes the frame at the wrap out and leaves the rush over two steps. `repair.surges` reads the speed on the flow reading's own path and windows: a run of one or two steps whose mean step is at least 3.0 usual steps and 3.0 times the mean of the three steps either side, the part going on the way it came through it (a step that turns back is a frame flung, the flow repair's). After the flow groups, the worst rush is eased over three or four drawings either side, made from the frames as filmed (a frame the flow made inside is made again, `supersedes`) and kept between two frames as filmed, never leaving more than half the loop made with the rounds before it (as much as `video-loop-compare` reads back as the repair's): `repair.ease_times` gives every frame a time, on a grid of a twentieth of a step, where the part's step changes from one step to the next as little as it can — from the speed it came in with to the speed it goes on with — while the foot line's centre (`loop.foot_centre`), the torso and the lowest solid row move in a step no further than the stretch's largest step as it played and half a pixel, and every other window of the flow reading no further than its own largest step there; the torso is the head and torso the motion anchor tracks (`repair.Torso`: `auto_motion.discover` on the loop as cut, each frame matched by the anchor's masked NCC within its own fine search, 14 px across and 28 down, x to a fraction of a pixel), so a part swaying on its own beat — a tail, the hair's ends — which pulls the body's mass along and is what the repair moves, is not drift; where the anchor finds no head and torso, or one reaches the edge of that search on a frame of the loop, the body's mass (`loop.body_centre`) stands in, and the mass is reported beside the drift either way (`drift.body_mass`, `torso`); within 2 % of that, the timing nearest the frames' own. How far the part has gone is the path of its coverage's centre in the windows that read the rush, which grows as far as the part moves where a blurred distance stops growing. RIFE makes each drawing between the frames as filmed either side of its time, one call a drawing, at most nine, outside the three of the flow groups; the stretch is taken only where no frame made has an interpolation fault, no body measure moves a pixel past the stretch's largest step, the part's largest step falls, no frame is left further off the flow and over its limit, no new rush appears in the rest of the body, and no window of the rest of the body steps further on the frames made than its own largest step over the stretch as it played (`rejected_for`, `the rest faster`: the search reads the rest of the body on the line between the frames as filmed, and the frames made, which go the body's own way between them, step further where a part moves far between frames). On the synthetic walker drawn over 24 frames a cycle, the tail a quarter short goes, after the flow repair, from 5.3 usual steps into the wrap to 3.4 with no rush left; over 16 frames, 15 % short and under the flow reading's limit, from 3.6 to 1.7; over 36 frames every stretch is refused, the rest of the body 1.5 % to 19 % past its own largest step on the frames made; ordinary walks and a flung frame read no rush. A loop with no rush costs one more reading and no call. `jump_repair` carries `surge`, `surge_after`, `ease_skipped`, `ease_calls` and the round `why: ease` (its `rest`: per window the rush is not read in, `step_max_before`, `step_max_after` and `faster`), `replaced` its frames, and the summary `eased`; without RIFE the rush is cut as filmed with a warning line, and `--repair on` refuses ([loop-repair](docs/loop-repair.md) section 2, "The eased retime").
diff --git a/docs/loop-repair.md b/docs/loop-repair.md
index 25030a7..dc32a53 100644
--- a/docs/loop-repair.md
+++ b/docs/loop-repair.md
@@ -598,7 +598,7 @@ frames.
| When | `--repair auto` or `on`; a cut whose two steps the clip's crossings show (`cycle.steps.by: crossings`) and whose wrap the gate reads over `--seam-max` on its frames as corrected (`loop.wrap_on_cells`) | the legs close; what does not is a part on its own beat. A cut whose steps are not counted keeps the gate's word |
| The frames a cycle apart | the clip's frame a cycle after each of the cut's first frames (`after`), or before each of its last (`before`), one past the widest run, as far as the clip after its lead-in holds them; each moved by the analysis translation between the two — the drift the clip walks over a cycle — and corrected as its cut frame was (`loop.cycle_apart`) | placed where the cut frame is, the in-between moves the part, not the body |
| How many | four frames, then six, eight, ten and twelve (`OVERLAP_WIDTHS`), never past half the loop (`MADE_SHARE`), on the side whose frames a cycle apart differ least from the cut's over the run | the fewer made, the more of the loop is the clip's own; a longer run where a shorter one rushes the part or leaves the body no room, its weights going over in smaller steps |
-| Each frame | made by RIFE between its two frames of one time — the late (the clip's frame a cycle after, or the loop's own before the wrap) and the early (the loop's own, or the clip's frame a cycle before) — at the weight of the early the search chooses (`repair.overlap_weights`, below); a weight of 0 or 1 shows that frame whole, no call, and a frame at one weight is made once, whatever the run | each frame at its own time, the legs where they were on both; next to the wrap near the frame that continues the clip across it |
+| Each frame | made by RIFE between its two frames of one time — the late (the clip's frame a cycle after, or the loop's own before the wrap) and the early (the loop's own, or the clip's frame a cycle before) — at the weight of the early the search chooses (`repair.overlap_weights`, below), and read with RIFE's sixteen single passes for a part torn (`rife.Rife.read`, `repair.overlap_frame`, below); a weight of 0 or 1 shows that frame whole, no call, and a frame at one weight is made once, whatever the run | each frame at its own time, the legs where they were on both; next to the wrap near the frame that continues the clip across it |
| The weights | in play order, from 0 at the frame before the run to 1 at the frame after it, never going back, on a grid of a twentieth: of those that keep every window of the rest of the body within its own largest step over the stretch as the loop plays it and every body measure within the stretch's largest step and half a pixel (below), read on the line between the two frames' states, the one nearest the ease in and out x²(3 − 2x), x = p/(width + 1) at the run's p-th frame (`repair.overlap_weight`) | the part goes over in an ease in and out, held off it only in the steps where the body has no room: a window that closes all but a step still moves by what it differs a cycle apart, and a weight that moves there pushes it past its own step |
The stretch runs from the frame before the run to the frame after it, the wrap inside, and the body is
@@ -612,7 +612,7 @@ as far as that lets it. The run is taken only where, on the frames made:
| Read on the frames made | Refused for (`rejected_for`) |
|---|---|
-| no made frame carries an interpolation fault (section 4) | `fault` |
+| no made frame carries an interpolation fault (section 4), a part torn among them (`torn`, below); the frames are made the weight nearest a half first, and the first with a fault ends the run, the rest of it not made (`unmade`) | `fault` |
| the seam gate's reading passes | `seam` |
| the loop, played round three times, still covers two steps: the crossings read on it, its middle turn judged `two` (`loop.ring_steps`) | `steps` |
| no body measure (`repair.drift_points`: the foot line, the torso, the ground) moves in a step more than a pixel past the largest step of the stretch as the loop plays it | `drift` |
@@ -626,6 +626,18 @@ as before. The jump repair after it makes nothing from, beside or again over a f
(`jump_repair.overlapped`); a filmed ghost beside one gives way to the filmed frame on its other side.
A loop drawn on twos or threes is not overlapped: a frame made beside the wrap would split a drawing.
+**A part torn.** A frame made between two frames a cycle apart bridges the part's whole move in one.
+Where the part swings over a smaller one — a tail across a hand — RIFE's flow drags the small part apart
+with it, and on art painted without a line no other reading sees it (section 4, "A part torn"). So each
+frame the run makes is read with RIFE's sixteen single passes as well, and a frame they disagree on over
+a part is refused (`torn`). The frame nearest a half is made first, where the part's move is bridged
+furthest; the first frame refused ends the run. A clip whose part crosses another on every run is
+refused the overlapped wrap and kept as filmed, for the gate to refuse: a torn hand is not delivered.
+On the file's third walker — painted without a line, seen from behind, 480 × 640, its big tail swung
+from 48 to −8 degrees across its hand between two frames a cycle apart — the real RIFE's frames at
+weights 0.5 and 0.7 read 0.44 and 0.26, over the 0.21 that names a part torn, with no other fault;
+swung 10 degrees past no hand, its frames at 0.3, 0.5 and 0.7 read 0.04 at most.
+
On the walker of `tests/video/test_loop_overlap.py` — seen from the front, 26 frames a cycle, its
tail swinging round on a beat of 41 frames, 73 frames, with the stand-in in-between (measured
2026-10-18):
@@ -646,21 +658,28 @@ clip carries the tuft on its bob and 8 px aside a frame.
| the whole walker also 3 px aside a cycle on (its torso slides) | every run of four to twelve held at 0.85 of the cut's own frame next to the wrap, all refused (`seam`), 15 calls | 3.41 → 3.04 to 3.15 | 0.03 → 0.05 at most | 0.71 → 1.96 to 3.91 |
The report's `wrap_overlap` carries `seam` (the reading that asked for it), `seam_max`, `widths`,
-`calls`, and each try in `tries` — `side`, `width`, `frames`, `stretch`, `eased` and `off_ease` (the
-ease and how far the weights left it), `made` (per frame: `frame`, its weight `t`, `proposal`, `faults`,
-`look`), `shown` (the frames shown otherwise than as cut), `seam`, `steps`, `part` (`windows`, `apart`,
-`step_max_before`, `step_max_after`), `rest` (`windows`, `step_max_bound`, `step_max_after`, `faster`),
+`calls` (the frames made), and each try in `tries` — `side`, `width`, `frames`, `stretch`, `part`
+(`windows`, `apart`), `eased` and `off_ease` (the ease and how far the weights left it), `weights` (those
+the search chose), `made` (per frame made, in play order: `frame`, its weight `t`, `proposal` with its
+`torn`, `faults`, `look`) and `made_order` (the frames in the order made); where a fault ended the run,
+`unmade` (the frames it did not make), `faults`, `look`, `outcome` and `rejected_for` (`fault`); where the
+run was made whole, `shown` (the frames shown otherwise than as cut), `seam`, `steps`, the part's
+`step_max_before` and `step_max_after`, `rest` (`windows`, `step_max_bound`, `step_max_after`, `faster`),
`drift`, `off_max`, `rushes`, `faults`, `look`, `outcome` and `rejected_for` (`no weighting keeps the
body in bound` where the search found none and nothing was made); where a run was taken, `side`,
-`width`, `frames` (those shown otherwise than as cut) and `seam_after`; and `skipped` where none was tried (drawn on twos, a loop too short
+`width`, `frames` (those shown otherwise than as cut) and `seam_after`; `interpolator`, with the binary's
+calls (`binary_calls`) where RIFE was located for it; and `skipped` where none was tried (drawn on twos, a loop too short
for a run that leaves half of it filmed, the clip holds no frames a cycle from the cut's beside the wrap,
-no interpolator). The summary names it. A made frame
+no interpolator). The summary names it; a `warning:` line names the frame that ended each run refused for
+a fault ("fault (torn at frame 3, the rest not made)"). A made frame
that cross-fades two drawings inside its outline is kept and named in a `warning:` line, as for the
jump repair. Without RIFE the wrap is cut as filmed with a warning line (`why`: "RIFE not installed —
-the wrap cut as filmed"), and `--repair on` refuses. Each run tried makes its frames with RIFE, a frame
-at one weight once: at most forty calls where every width is made and refused, outside the jump
-repair's — the first walker above takes ten, the second four where it is taken and fifteen where its
-torso slides.
+the wrap cut as filmed"), and `--repair on` refuses. Each frame made costs the binary 34 calls (the frame
+as every call makes it, and its sixteen passes in colour and coverage), a frame at one weight made once,
+whatever the run; a run ends at its first fault, the frames past it not made, so a wrap whose every run
+is refused for a part torn costs the frames each run makes up to its first torn one, and one whose every
+run is made whole and refused for something else at most forty frames, outside the jump repair's — the
+first walker above takes ten frames, the second four where it is taken and fifteen where its torso slides.
## 3. The jolt index — reported on every walk, a gate only when asked
@@ -1258,6 +1277,38 @@ sprite-gen video-cycle-align --loop-dir set/front-walk/loop --loop-dir set/side-
a step of 13 degrees or more between far-apart drawings, most outlined in grey. A fault would
give way for clean frames and still pass blends; a line to look at costs neither. 0.015 is set
by eye between the cross-faded frames and the clean ones, a reference, not a measured optimum.
+- **A part torn — the overlapped wrap's frames** (`rife.passes`, `rife.torn`): between two frames a
+ cycle apart a part on its own beat moves its whole way at once (section 2, "The overlapped wrap"),
+ and where it swings over a smaller part — a tail across a hand — RIFE's flow drags the small part
+ apart with it: the hand broken into blocks, much of its coverage gone, what is left part-covered. On
+ art painted without a line nothing above reads it: there is no line to lose (and the outline
+ readings judge against the less outlined neighbour, the one whose tail meets the air), the coverage
+ goes rather than staying part-covered in a band, and nothing goes darker. RIFE's augmentation runs
+ the pair eight ways (every flip and transpose) and both ways in time, combines the sixteen inside
+ the binary and gives back one frame; run one at a time with no augmentation, each turned back, the
+ passes guess differently exactly where the flow finds no one way through.
+ - `torn`: per pixel, how far the mean of the eight forward passes and the mean of the eight
+ backward ones put the premultiplied colour and coverage apart (`rife.passes`); the pixels where
+ they are more than **0.05** apart (`rife.TORN_LEVEL`), in regions at least **3 px** thick
+ (`rife.TORN_THICK`: an edge the passes place a pixel or two apart opens away, as does hair they
+ redraw strand by strand); and of those, the largest share of a window **a tenth** of the made
+ frame's body height on a side (`rife.TORN_WINDOW`, about a hand on a walk; 8 px at least, the
+ windows stepping by a quarter of it). Read in a window rather than over the body, so a small part
+ torn on a large body is not lost in it.
+ - Over **0.21** (`interpolation_quality.TORN_WARN`) the frame is torn: the fault `torn`. Set by eye
+ on drawn walks, between the frames seen torn — a hand dragged apart beside a tail swung far, legs
+ melted into one shape, a part doubled — which read over it, and the frames seen as clean, which
+ read under it; a reference, not a measured optimum. On the painted walker of
+ `tests/video/test_loop_overlap.py` (section 2) the frames made across the hand read 0.44 and
+ 0.26 and those of a swing past no hand 0.04 at most.
+ - Read only on the overlapped wrap's frames: each is made as every call makes it and read with its
+ passes, 34 calls of the binary against two for a frame anywhere else. The jump repair, the eased
+ retime and the cycle alignment make their frames between a walk's neighbours, a move RIFE carries,
+ and do not read it.
+ - The frame delivered is the binary's, not the passes' mean: where the passes disagree the mean
+ blends their guesses part-covered, which the augmentation, combining them inside the binary, does
+ not. Nor can the binary's own augmented calls be read for it: a call with `-x` from a to b and one
+ from b to a agree where the single passes do not.
- **What `auto` costs**: a frame replaced shows the nearer source frame, so the motion there
steps as filmed, up to half a frame off its time, and the loop shows fewer distinct drawings a
second than `rife` — but no melted one. A clip drawn on twos has a long step between drawings
diff --git a/sprite_gen/video/interpolation_quality.py b/sprite_gen/video/interpolation_quality.py
index ebff04e..a36a587 100644
--- a/sprite_gen/video/interpolation_quality.py
+++ b/sprite_gen/video/interpolation_quality.py
@@ -1,10 +1,11 @@
# SPDX-License-Identifier: Apache-2.0
"""The shared acceptance policy for the frames repair and cycle alignment deliver.
-Measurements live in ``rife`` (``smear``, ``outline_shift``, ``ghost``, ``crossfade``); these are the existing alignment
-bounds, also applied before replacing a filmed middle frame during jump repair, the ghost screen
+Measurements live in ``rife`` (``smear``, ``outline_shift``, ``ghost``, ``crossfade``, ``torn``); these are the existing
+alignment bounds, also applied before replacing a filmed middle frame during jump repair, the ghost screen
every frame a loop path delivers passes, filmed or made, the clean frame a filmed ghost gives way
-to, and the cross-fade a made frame is named for, kept (docs/loop-repair.md section 4).
+to, and the cross-fade a made frame is named for, kept; and the torn part a frame of the overlapped wrap
+is refused for (docs/loop-repair.md section 4).
"""
from collections.abc import Container
@@ -26,6 +27,11 @@ GHOST_WARN = 0.003
# far-apart drawings outlined in grey reads over it too, and a cross-fade drawn without outlines mostly
# under it — the reading does not part the two, so it is no fault (docs/loop-repair.md section 4).
CROSSFADE_LOOK = 0.015
+# A made frame whose single passes disagree (`rife.torn`) over more than this share of a window a tenth of the body
+# high is torn: set by eye on drawn walks, between the frames seen torn (a hand dragged apart beside a tail swung far,
+# legs melted into one shape, a part doubled) and the frames seen as clean. Read only where a frame is read with its
+# passes — the overlapped wrap's frames (docs/loop-repair.md sections 2 and 4).
+TORN_WARN = 0.21
def faults(measure: dict[str, float | None]) -> list[str]:
@@ -38,12 +44,14 @@ def faults(measure: dict[str, float | None]) -> list[str]:
lighter than the neighbours' by more than `rife.OUTLINE_SHIFT` (docs/loop-repair.md section 4).
Where it is None (art painted without a line) and in the versioned measures of
`video-loop-compare` and `video-loop-repair`, which carry `rife.smear` alone, the dark line
- judges alone."""
+ judges alone. A part torn where the measure carries how far the frame's passes disagree
+ (`torn`, the overlapped wrap's frames: `rife.passes`, `rife.torn`)."""
shift = measure.get("outline_shift")
outline = measure["outline_loss"] if shift is None else min(measure["outline_loss"], shift)
return ([*(["smear"] if measure["dark_excess"] > SMEAR_WARN else []),
*(["outline"] if outline > OUTLINE_WARN else []),
- *(["ghost"] if (measure.get("ghost") or 0.0) > GHOST_WARN else [])])
+ *(["ghost"] if (measure.get("ghost") or 0.0) > GHOST_WARN else []),
+ *(["torn"] if (measure.get("torn") or 0.0) > TORN_WARN else [])])
def looks(measure: dict[str, float | None]) -> list[str]:
diff --git a/sprite_gen/video/loop.py b/sprite_gen/video/loop.py
index 0a8e34a..c01518b 100644
--- a/sprite_gen/video/loop.py
+++ b/sprite_gen/video/loop.py
@@ -912,14 +912,29 @@ def _repair_jumps(frames: list[Image.Image], interpolate: rife_mod.Interpolate |
made_by: list[dict[str, str]] = []
calls = 0
- def lazy(a: Image.Image, b: Image.Image, t: float) -> Image.Image:
- nonlocal interpolate, calls
+ def located() -> rife_mod.Interpolate:
+ nonlocal interpolate
if interpolate is None:
- located = rife_mod.Rife()
- made_by.append(located.describe())
- interpolate = located
+ found = rife_mod.Rife()
+ made_by.append(found.describe())
+ interpolate = found
+ return interpolate
+
+ def lazy(a: Image.Image, b: Image.Image, t: float) -> Image.Image:
+ nonlocal calls
calls += 1
- return interpolate(a, b, t)
+ return located()(a, b, t)
+
+ def lazy_read(a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ # the overlapped wrap reads each frame it makes with RIFE's single passes, for a part torn: an interpolator given
+ # that cannot say how far its passes disagree is refused, never read as agreeing
+ nonlocal calls
+ read = getattr(located(), "read", None)
+ if read is None:
+ raise ValueError("the overlapped wrap reads how far RIFE's single passes disagree on each frame it makes, and "
+ "the interpolator given has no `read` (rife.Rife.read)")
+ calls += 1
+ return read(a, b, t)
overlap: dict[str, Any] | None = None
overlapped: list[int] = []
@@ -929,15 +944,16 @@ def _repair_jumps(frames: list[Image.Image], interpolate: rife_mod.Interpolate |
asked = partial(repair_mod.overlap_wrap, others=wrap["others"], seam=wrap_on_cells, steps=ring_steps,
seam_max=wrap["seam_max"], reads=bool(screen["reads"]), torso=torso)
try:
- frames, overlap = asked(frames, interpolate=lazy)
+ frames, overlap = asked(frames, read=lazy_read)
except rife_mod.RifeNotInstalled as exc:
if required:
raise
- frames, overlap = asked(frames, interpolate=None)
+ frames, overlap = asked(frames, read=None)
overlap.update(why="RIFE not installed — the wrap cut as filmed", rife=str(exc), install=rife_mod.INSTALL_COMMAND)
overlapped = overlap["frames"]
if overlap["calls"]:
- overlap["interpolator"] = {"kind": "rife-ncnn-vulkan", **made_by[0]} if made_by else {"kind": "injected"}
+ overlap["interpolator"] = ({"kind": "rife-ncnn-vulkan", **made_by[0], "binary_calls": interpolate.calls}
+ if made_by else {"kind": "injected"})
wrap_calls = calls
try:
repaired, record = repair_mod.repair_jumps(frames, lazy, facing=facing, screen=screen, overlapped=overlapped,
@@ -1010,7 +1026,15 @@ def _overlap_lines(overlap: dict[str, Any]) -> list[str]:
"and cut it again (docs/loop-repair.md section 2)"]
if "skipped" in overlap:
return [f"{said}; the overlapped wrap was not made: {overlap['skipped']} (docs/loop-repair.md section 2)"]
- refused = "; ".join(f"{t['width']} frames {t['side']} the wrap: {', '.join(t['rejected_for'])}" for t in overlap["tries"])
+
+ def why(t: dict[str, Any]) -> str:
+ if "unmade" not in t:
+ return ", ".join(t["rejected_for"])
+ stopped = next(r for r in t["made"] if r["faults"]) # the first frame with a fault ended the run
+ rest = ", the rest not made" if t["unmade"] else ""
+ return f"{', '.join(t['rejected_for'])} ({', '.join(stopped['faults'])} at frame {stopped['frame']}{rest})"
+
+ refused = "; ".join(f"{t['width']} frames {t['side']} the wrap: {why(t)}" for t in overlap["tries"])
return [f"{said}, and the overlapped wrap was refused ({refused}): cut as filmed (docs/loop-repair.md section 2)"]
diff --git a/sprite_gen/video/repair.py b/sprite_gen/video/repair.py
index 5befe66..b345714 100644
--- a/sprite_gen/video/repair.py
+++ b/sprite_gen/video/repair.py
@@ -25,7 +25,9 @@ repair is for, not drift.
A part on a beat of its own — a tail swinging round every 41 frames on a walk of 26 — closes on no cut of two steps,
and turns back at the wrap, where no retime reaches it. Before the rounds above, the frames beside such a cut's wrap
are made again between the clip's frames a cycle apart and the cut's own (`overlap_wrap`), so the wrap plays a step
-the clip filmed (docs/loop-repair.md section 2, "The overlapped wrap").
+the clip filmed (docs/loop-repair.md section 2, "The overlapped wrap"). Those frames bridge the part's whole move in
+one, and where it swings over a smaller part — a tail across a hand — RIFE drags the small part apart with it: each is
+read with RIFE's single passes and refused where they disagree there (`overlap_frame`, `rife.torn`).
Measured on the loop as it plays — cyclic, the last frame followed by the first — and inside
the union box of the body over the whole loop, the way the strip cells are cut, so a frame's
@@ -47,7 +49,7 @@ from sprite_gen._deps import np
from sprite_gen.video import auto_motion, motion_anchor
from sprite_gen.video import held as held_mod
from sprite_gen.video import legs as legs_mod
-from sprite_gen.video.rife import Interpolate, crossfade, ghost, outline_shift, smear
+from sprite_gen.video.rife import Interpolate, Read, crossfade, ghost, outline_shift, smear, torn
from sprite_gen.video.interpolation_quality import clean_beside, faults, ghost_screen, looks
# Interpolation calls at most per loop, a drawing made one call, including rejected proposals. A loop that needs more is
@@ -874,7 +876,23 @@ def overlap_weights(step: np.ndarray, body: np.ndarray, rest: list[int], rest_bo
"off_ease": round(float(np.abs(np.array(weights) - own / (G - 1)).max()), 4)}
-def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Image]], interpolate: Interpolate | None, *,
+def overlap_frame(read: Read, a: Image.Image, b: Image.Image, u: float, *, reads: bool) -> tuple[Image.Image, dict[str, Any]]:
+ """A frame of the overlapped wrap: RIFE's frame at weight `u` of the early frame `b` from the late frame `a`, with how
+ far its single passes disagree there (`read`, `rife.Rife.read`), and what it reads — the measures every made frame
+ carries, its ghost where the loop's ghost screen reads it (`reads`), and a part torn (`rife.torn`) — with its faults
+ and what to look at (docs/loop-repair.md section 2, "The overlapped wrap")."""
+ made, apart = read(a, b, u)
+ if made.mode != "RGBA" or made.size != b.size:
+ raise ValueError("repair interpolator must return an RGBA frame of the original size")
+ if apart.shape != (b.height, b.width):
+ raise ValueError("repair interpolator must read how far its passes disagree at every pixel of the frame")
+ measure = {**smear(made, a, b), "outline_shift": outline_shift(made, a, b), "ghost": ghost(made) if reads else None,
+ "crossfade": crossfade(made, a, b, u), "torn": torn(apart, made)}
+ return made, {"proposal": measure, "look": looks(measure),
+ "faults": faults(measure) + ([] if made.getchannel("A").getbbox() else ["empty"])}
+
+
+def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Image]], read: Read | None, *,
seam: Callable[[list[Image.Image]], float], steps: Callable[[list[Image.Image]], dict[str, Any]],
seam_max: float, reads: bool, torso: Torso | None = None) -> tuple[list[Image.Image], dict[str, Any]]:
"""The overlapped wrap (docs/loop-repair.md section 2, "The overlapped wrap") of a loop cut two steps long whose wrap
@@ -887,10 +905,12 @@ def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Im
inside; each of its frames has two of one time, the late (the clip's frame a cycle after, or the loop's own before the
wrap) and the early (the loop's own, or the clip's frame a cycle before). Each frame of the run is shown at the weight
of its early frame `overlap_weights` chooses on the line between the two frames' states — an ease in and out, moved off
- it only as far as the rest of the body and the body's measures need — as RIFE's frame between them (`interpolate`), or
- either one whole at 0 or 1.
+ it only as far as the rest of the body and the body's measures need — as RIFE's frame between them, read with its
+ single passes (`read`, `overlap_frame`), or either one whole at 0 or 1. The frames are made the weight nearest a half
+ first: the part's whole move is bridged there, and a frame torn is torn there first.
- The run is taken only where, on the frames made: no frame has an interpolation fault; the gate passes it; the loop
+ The first made frame with an interpolation fault ends the run — a part torn (`rife.torn`) among them — and the rest of
+ it is not made (`unmade`). The run is taken only where, on the frames made: no frame has a fault; the gate passes it; the loop
played round still covers two steps (`steps`, the clip's crossings read on it); no body measure moves in a step more
than a pixel past the largest step the stretch takes as the loop plays it (`drift_points`, the eased retime's bound,
the body's point the torso: `torso`, the `Torso` of `frames` where none is given, recorded as `torso`; the body's
@@ -933,7 +953,7 @@ def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Im
if not plans:
record["skipped"] = "the clip holds no frames a cycle from the loop's beside the wrap"
return frames, record
- if interpolate is None:
+ if read is None:
record["skipped"] = "no interpolator"
return frames, record
torso = torso if torso is not None else Torso(frames)
@@ -988,9 +1008,11 @@ def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Im
record["tries"].append(tried)
continue
new = list(frames)
- made_rows = []
- for p, u in enumerate(chosen["weights"], start=1):
- q = stretch[p]
+ weights = chosen["weights"]
+ rows: dict[int, dict[str, Any]] = {}
+ made_order = []
+ for p in sorted(range(1, P), key=lambda p: (abs(weights[p - 1] - 0.5), p)):
+ u, q = weights[p - 1], stretch[p]
a, b = images[late[p]], images[early[p]]
row: dict[str, Any] = {"frame": q, "t": round(u, 4), "proposal": None, "faults": [], "look": []}
if u <= 0.0:
@@ -1000,18 +1022,24 @@ def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Im
else:
key = (late[p], early[p], row["t"])
if key not in made_at:
- made = interpolate(a, b, u)
+ made_at[key] = overlap_frame(read, a, b, u, reads=reads)
record["calls"] += 1
- if made.mode != "RGBA" or made.size != b.size:
- raise ValueError("repair interpolator must return an RGBA frame of the original size")
- measure = {**smear(made, a, b), "outline_shift": outline_shift(made, a, b),
- "ghost": ghost(made) if reads else None, "crossfade": crossfade(made, a, b, u)}
- made_at[key] = (made, {"proposal": measure, "look": looks(measure),
- "faults": faults(measure) + ([] if made.getchannel("A").getbbox() else ["empty"])})
frame, said = made_at[key]
row.update(said)
- made_rows.append(row)
+ made_order.append(q)
+ rows[p] = row
new[q] = frame
+ if row["faults"]:
+ break
+ made_rows = [rows[p] for p in sorted(rows)]
+ wrong = sorted({f for r in made_rows for f in r["faults"]})
+ tried.update(eased=chosen["eased"], off_ease=chosen["off_ease"], weights=[round(w, 4) for w in weights], made=made_rows,
+ made_order=made_order)
+ if wrong:
+ tried.update(unmade=[stretch[p] for p in range(1, P) if p not in rows], faults=wrong,
+ look=sorted({w for r in made_rows for w in r["look"]}), outcome="rejected", rejected_for=["fault"])
+ record["tries"].append(tried)
+ continue
shown = [q for q in run if new[q] is not frames[q]]
after_flow = _flow(new, box, (flow["runs"], flow["hold"]))
rel_after = after_flow["step"] / scale
@@ -1026,15 +1054,12 @@ def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Im
rushes = [_said_surge(x, after_flow) for x in touching(after_flow, set(steps_at)) if (x["d"], x["r"]) not in had]
seam_after = float(seam(new))
counted = steps(new)
- wrong = sorted({f for r in made_rows for f in r["faults"]})
- rejected = ((["fault"] if wrong else []) + (["seam"] if seam_after > seam_max else [])
+ rejected = ((["seam"] if seam_after > seam_max else [])
+ (["steps"] if counted.get("verdict") != "two" else []) + (["drift"] if over else [])
+ (["no gain"] if part and part_after >= part_before else [])
+ (["off the flow"] if off_after[stretch].max() >= FLOW_LIMIT and off_after[stretch].max() > off_before[stretch].max() else [])
+ (["a new rush"] if rushes else []) + (["the rest faster"] if faster else []))
- tried.update({
- "eased": chosen["eased"], "off_ease": chosen["off_ease"], "made": made_rows, "shown": shown, "seam": round(seam_after, 4),
- "steps": {key: counted.get(key) for key in ("verdict", "k", "ring", "why")}})
+ tried.update({"shown": shown, "seam": round(seam_after, 4), "steps": {key: counted.get(key) for key in ("verdict", "k", "ring", "why")}})
tried["part"].update(step_max_before=round(part_before, 4), step_max_after=round(part_after, 4))
tried.update({
"rest": {"windows": [[round(v, 4) for v in flow["fractions"][w]] for w in rest],
diff --git a/sprite_gen/video/rife.py b/sprite_gen/video/rife.py
index 496069a..83bf91f 100644
--- a/sprite_gen/video/rife.py
+++ b/sprite_gen/video/rife.py
@@ -14,7 +14,9 @@ the flow failed and melted a limb into the fill; `outline_shift` reads that edge
neighbours' rank by rank, so a mid-tone line the flow only softened is not read as lost — where the
neighbours are outlined (`outlined`), and not on art painted without a line; `ghost` reads a band of
part coverage on a frame alone, filmed or made; `crossfade` reads two drawings cross-faded inside a
-whole silhouette, a reading to look at, not a fault (docs/loop-repair.md section 4).
+whole silhouette, a reading to look at, not a fault (docs/loop-repair.md section 4). `passes` runs
+the sixteen single passes the binary's augmentation combines inside one call, one at a time, and
+`torn` reads where they disagree: a small part RIFE drags apart beside a large one moving far.
The binary's own CPU path (`-g -1`) returns a wrong frame with rife-v4.6 on both macOS and
Linux (measured 2026-10-03: mean error 39 against 3.3 through Vulkan), so it is never passed;
@@ -28,7 +30,8 @@ puts it (`install_root()`, docs/loop-repair.md section 1). A RIFE that cannot be
RIFE that is found and then fails is `RifeUnavailable` and always an error.
The interpolator is a plain callable `(a, b, t) -> frame`, so a caller can be handed a stand-in
-(`Interpolate`), and a test needs no binary.
+(`Interpolate`), and a test needs no binary. A caller that reads the passes takes a callable
+`(a, b, t) -> (frame, disagreement)` (`Read`, `Rife.read`).
"""
from __future__ import annotations
@@ -92,12 +95,26 @@ GHOST_THICK = 5
CROSSFADE_RANGE = 7
CROSSFADE_WINDOW = 0.08
CROSSFADE_KEEP = 0.6
+# `passes`: the binary's augmentation runs the pair turned eight ways (`-x`: every flip and transpose) and both ways in
+# time (`-z`), combines the sixteen inside, and gives back one frame — where the passes disagree, the frame does not say
+# so. Run one at a time with no augmentation, each turned back, they can be read apart. Each pass is two calls, colour
+# and coverage, as `between` makes them.
+PASS_GEOMETRIES = tuple((transpose, flips) for transpose in (False, True) for flips in range(4)) # flips: 1 left-right, 2 up-down
+PASS_CALLS = 2 * 2 * len(PASS_GEOMETRIES)
+# `torn`: the forward passes' mean and the backward passes' mean put a different colour or coverage at a pixel where
+# they are more than this apart (premultiplied RGBA, 0..1); an edge they place a pixel or two apart is thinner than
+# TORN_THICK pixels and opened away; a part torn is read in windows TORN_WINDOW of the body's height on a side, the
+# size of a hand on a walk.
+TORN_LEVEL = 0.05
+TORN_THICK = 3
+TORN_WINDOW = 0.10
# One call past this is stopped and reported as a RIFE that failed. TTA's slowest call measured, on a
# 960 x 960 frame on Modal's CPU, took about 30 s, and Modal's hosts differ by up to 2.5 times on one frame
# (2026-10-15, docs/loop-repair.md section 1).
CALL_TIMEOUT_SECONDS = 600
Interpolate = Callable[[Image.Image, Image.Image, float], Image.Image]
+Read = Callable[[Image.Image, Image.Image, float], tuple[Image.Image, np.ndarray]]
# The one line that installs it, quoted wherever a missing RIFE is reported.
INSTALL_COMMAND = "sprite-gen rife install"
@@ -182,19 +199,22 @@ def tta() -> tuple[str, ...]:
return modes
-def _call(binary: Path, model: Path, a: Image.Image, b: Image.Image, t: float, tmp: Path) -> Image.Image:
+def _call(binary: Path, model: Path, a: Image.Image, b: Image.Image, t: float, tmp: Path,
+ modes: tuple[str, ...] | None = None) -> Image.Image:
+ """One run of the binary from `a` to `b` at `t`, with the augmentation `tta()` names, or `modes` (`passes`: none)."""
+ modes = tta() if modes is None else modes
a.save(tmp / "a.png")
b.save(tmp / "b.png")
out = tmp / "o.png"
out.unlink(missing_ok=True)
try:
proc = subprocess.run(
- [str(binary), "-0", str(tmp / "a.png"), "-1", str(tmp / "b.png"), "-o", str(out), "-s", f"{t:.4f}", "-m", str(model), *tta()],
+ [str(binary), "-0", str(tmp / "a.png"), "-1", str(tmp / "b.png"), "-o", str(out), "-s", f"{t:.4f}", "-m", str(model), *modes],
capture_output=True, text=True, timeout=CALL_TIMEOUT_SECONDS,
)
except subprocess.TimeoutExpired as exc:
raise RifeUnavailable(f"{binary.name} made no frame in {CALL_TIMEOUT_SECONDS} s and was stopped "
- f"(augmentation {' '.join(tta()) or 'none'}, SPRITE_GEN_RIFE_TTA)") from exc
+ f"(augmentation {' '.join(modes) or 'none'}, SPRITE_GEN_RIFE_TTA)") from exc
if proc.returncode != 0 or not out.is_file():
raise RifeUnavailable(f"{binary.name} failed (exit {proc.returncode}): {(proc.stderr or proc.stdout).strip()[-300:]}")
with Image.open(out) as im:
@@ -245,11 +265,68 @@ def between(a: Image.Image, b: Image.Image, t: float, *, binary: Path, model: Pa
cov = [Image.fromarray(np.uint8(x[..., 3] * 255 + 0.5), "L").convert("RGB") for x in (fa, fb)]
rgb = np.asarray(_call(binary, model, colour[0], colour[1], t, tmp), dtype=np.float32) / 255.0
alpha = np.asarray(_call(binary, model, cov[0], cov[1], t, tmp).convert("L"), dtype=np.float32) / 255.0
+ return _joined(rgb, alpha)
+
+
+def _joined(rgb: np.ndarray, alpha: np.ndarray) -> Image.Image:
+ """The RGBA frame of a colour run and a coverage run (0..1): coverage under ALPHA_FLOOR dropped, the colour kept
+ wherever there is coverage."""
alpha = np.where(alpha < ALPHA_FLOOR, 0.0, alpha)
out = np.dstack([np.where(alpha[..., None] > 0, rgb, 0.0), alpha])
return Image.fromarray(np.uint8(np.clip(out, 0, 1) * 255 + 0.5), "RGBA")
+def _turned(x: np.ndarray, geometry: tuple[bool, int]) -> np.ndarray:
+ """An image (H x W x C) turned one of PASS_GEOMETRIES' ways: transposed, then flipped."""
+ transpose, flips = geometry
+ if transpose:
+ x = x.transpose(1, 0, 2)
+ if flips & 1:
+ x = x[:, ::-1]
+ if flips & 2:
+ x = x[::-1]
+ return np.ascontiguousarray(x)
+
+
+def _turned_back(x: np.ndarray, geometry: tuple[bool, int]) -> np.ndarray:
+ """`_turned` undone."""
+ transpose, flips = geometry
+ if flips & 2:
+ x = x[::-1]
+ if flips & 1:
+ x = x[:, ::-1]
+ if transpose:
+ x = x.transpose(1, 0, 2)
+ return np.ascontiguousarray(x)
+
+
+def passes(a: Image.Image, b: Image.Image, t: float, *, binary: Path, model: Path, tmp: Path) -> np.ndarray:
+ """How far the binary's sixteen single passes from `a` to `b` at fraction `t` disagree, per pixel (docs/loop-repair.md
+ section 4).
+
+ Each pass is the pair turned one of PASS_GEOMETRIES' ways and run with no augmentation — from `a` to `b` at `t`, and
+ from `b` to `a` at 1 - t — colour and coverage as `between` runs them, and turned back. The disagreement is how far
+ apart the mean of the eight forward passes and the mean of the eight backward ones put the premultiplied colour and
+ coverage (0..1, `torn` reads it). Their mean is not the frame: where they disagree it blends their guesses part-covered,
+ which the augmentation, combining them inside the binary, does not. PASS_CALLS calls of the binary."""
+ if a.size != b.size:
+ raise ValueError(f"rife: frames differ in size ({a.size} vs {b.size})")
+ fa, fb = (np.asarray(f.convert("RGBA"), dtype=np.float32) / 255.0 for f in (a, b))
+ colour = [np.uint8(np.clip(bleed(x), 0, 1) * 255 + 0.5) for x in (fa, fb)]
+ cover = [np.repeat(np.uint8(x[..., 3:] * 255 + 0.5), 3, axis=-1) for x in (fa, fb)]
+ rgbs, alphas = [], []
+ for geometry in PASS_GEOMETRIES:
+ for first, second, at in ((0, 1, t), (1, 0, 1 - t)):
+ runs = [_call(binary, model, Image.fromarray(_turned(pair[first], geometry), "RGB"),
+ Image.fromarray(_turned(pair[second], geometry), "RGB"), at, tmp, ()) for pair in (colour, cover)]
+ rgbs.append(_turned_back(np.asarray(runs[0], dtype=np.float32) / 255.0, geometry))
+ alphas.append(_turned_back(np.asarray(runs[1].convert("L"), dtype=np.float32)[..., None] / 255.0, geometry)[..., 0])
+ rgb, alpha = np.stack(rgbs), np.stack(alphas)
+ pre = np.concatenate([rgb * alpha[..., None], alpha[..., None]], axis=-1)
+ apart = np.sqrt(((pre[0::2].mean(axis=0) - pre[1::2].mean(axis=0)) ** 2).sum(axis=-1) / 4.0)
+ return apart.astype(np.float32)
+
+
def _mask(mask: np.ndarray, size: int, op: type) -> np.ndarray:
return np.asarray(Image.fromarray(np.uint8(mask) * 255).filter(op(size))) > 0
@@ -418,8 +495,32 @@ def crossfade(made: Image.Image, a: Image.Image, b: Image.Image, t: float) -> fl
return round(lost / max(1, int(solid.sum())), 5)
+def torn(apart: np.ndarray, made: Image.Image) -> float:
+ """A part torn in a made frame, read on how far its single passes disagree (`passes`): of the pixels where they are
+ more than TORN_LEVEL apart, those in regions at least TORN_THICK pixels thick, and of them the largest share of a
+ window TORN_WINDOW of the made frame's body height on a side (8 px at least; the windows step by a quarter of it). A
+ hand the flow drags apart beside a tail swung far fills its window; an edge the passes place a pixel apart, or hair
+ they redraw strand by strand, leaves it nearly empty. Read in a window, not over the body, so a small part torn on a
+ large body is not lost in it. 0 where the frame has no solid body (docs/loop-repair.md section 4)."""
+ alpha = np.asarray(made.convert("RGBA").getchannel("A"), dtype=np.float32) / 255.0
+ rows = np.nonzero((alpha >= 0.5).any(axis=1))[0]
+ if rows.size == 0:
+ return 0.0
+ side = max(8, int(round((rows[-1] - rows[0] + 1) * TORN_WINDOW)))
+ thick = _mask(_mask(apart > TORN_LEVEL, TORN_THICK, ImageFilter.MinFilter), TORN_THICK, ImageFilter.MaxFilter)
+ h, w = thick.shape
+ if side >= min(h, w):
+ return round(float(thick.sum()) / (side * side), 5)
+ sums = np.pad(thick.astype(np.float64).cumsum(axis=0).cumsum(axis=1), ((1, 0), (1, 0)))
+ step = max(1, side // 4)
+ ys, xs = np.arange(0, h - side + 1, step)[:, None], np.arange(0, w - side + 1, step)[None, :]
+ covered = sums[ys + side, xs + side] - sums[ys, xs + side] - sums[ys + side, xs] + sums[ys, xs]
+ return round(float(covered.max()) / (side * side), 5)
+
+
class Rife:
- """A located RIFE as an `Interpolate` callable. Counts the frames it made (`made`)."""
+ """A located RIFE as an `Interpolate` callable, and as a `Read` (`read`). Counts the frames it made (`made`) and the
+ times it ran the binary (`calls`)."""
def __init__(self, binary: Path | None = None, model: Path | None = None):
if binary is None or model is None:
@@ -427,12 +528,24 @@ class Rife:
self.binary, self.model = binary, model
self.tta = tta() # refused here, before a frame is asked for
self.made = 0
+ self.calls = 0
def __call__(self, a: Image.Image, b: Image.Image, t: float) -> Image.Image:
with tempfile.TemporaryDirectory(prefix="sprite-gen-rife-") as td:
frame = between(a, b, t, binary=self.binary, model=self.model, tmp=Path(td))
self.made += 1
+ self.calls += 2
return frame
+ def read(self, a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ """The frame at `t`, made as every call makes it (`between`), and how far the sixteen single passes disagree
+ there (`passes`): 2 + PASS_CALLS calls of the binary."""
+ with tempfile.TemporaryDirectory(prefix="sprite-gen-rife-") as td:
+ frame = between(a, b, t, binary=self.binary, model=self.model, tmp=Path(td))
+ apart = passes(a, b, t, binary=self.binary, model=self.model, tmp=Path(td))
+ self.made += 1
+ self.calls += 2 + PASS_CALLS
+ return frame, apart
+
def describe(self) -> dict[str, str]:
return {"binary": str(self.binary), "model": self.model.name, "tta": " ".join(self.tta)}
diff --git a/tests/video/test_loop_overlap.py b/tests/video/test_loop_overlap.py
index 258564e..1acac83 100644
--- a/tests/video/test_loop_overlap.py
+++ b/tests/video/test_loop_overlap.py
@@ -6,8 +6,10 @@ the loop passes the gate, still two steps, its body standing where it stood.
The walker is seen from the front, 26 frames a cycle, its tail swinging round on a beat of 41 frames, in a 73-frame clip.
The interpolator is a stand-in that draws the walker between two frames the way an optical-flow in-between does: the body
-at the time between them, the tail's tip on the line between its two places. A second walker, a big tuft bobbing on a
-short tail, is carried over at the wrap with its torso where it stood, or with its torso sliding. Synthetic frames only."""
+at the time between them, the tail's tip on the line between its two places — and says its passes agree. A second walker,
+a big tuft bobbing on a short tail, is carried over at the wrap with its torso where it stood, or with its torso sliding.
+A third, painted without a line and seen from behind, swings a big tail across its hand between two frames a cycle apart,
+and the real RIFE tears the hand there. Synthetic frames only."""
from __future__ import annotations
@@ -20,7 +22,8 @@ import numpy as np
import pytest
from PIL import Image, ImageDraw
-from sprite_gen.video import loop, repair
+from sprite_gen.video import loop, repair, rife
+from sprite_gen.video.interpolation_quality import TORN_WARN, faults
from tests.video.test_loop_repair import _no_rife
FRAMES, CYCLE, TAIL = 73, 26, 41
@@ -94,6 +97,10 @@ class Between:
out.paste(drawn, (dx, dy))
return out
+ def read(self, a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ """The frame between, and its passes agreeing everywhere: a drawing has one way through."""
+ return self(a, b, t), np.zeros((b.height, b.width), dtype=np.float32)
+
def _bobbing(body: float, tip: tuple[float, float], aside: int = 0) -> Image.Image:
"""The second walker: the walker with a big tuft on a short tail, whose bob moves the picture and not the body's mass."""
@@ -124,6 +131,9 @@ class Drawn:
self.calls.append((ba, bb, round(t, 4)))
return self.draw(bb, (xa + t * (xb - xa), ya + t * (yb - ya)), round(sa + t * (sb - sa)))
+ def read(self, a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ return self(a, b, t), np.zeros((b.height, b.width), dtype=np.float32)
+
def _second(slid: int = 0) -> tuple[Drawn, list[Image.Image], dict[str, dict[int, Image.Image]]]:
"""The second walker's loop as cut, its tuft bobbing and the walker in its place; and the clip's frames a cycle after
@@ -136,7 +146,7 @@ def _second(slid: int = 0) -> tuple[Drawn, list[Image.Image], dict[str, dict[int
def _overlapped(drawn: Drawn, frames: list[Image.Image], others: dict) -> tuple[list[Image.Image], dict]:
"""The overlapped wrap of the second walker's loop, its steps taken as two (the walker shows no crossings)."""
- return repair.overlap_wrap(frames, others, drawn, seam=loop.wrap_on_cells, steps=lambda f: {"verdict": "two"},
+ return repair.overlap_wrap(frames, others, drawn.read, seam=loop.wrap_on_cells, steps=lambda f: {"verdict": "two"},
seam_max=loop.SEAM_RATIO_MAX, reads=True)
@@ -186,18 +196,22 @@ def test_a_tail_on_its_own_beat_is_closed_at_the_wrap_with_the_clips_frames_a_cy
assert taken["rest"]["faster"] == [] and taken["faults"] == [] and taken["rushes"] == []
assert taken["part"]["step_max_after"] < taken["part"]["step_max_before"]
# each frame of a run shown between the clip's frame a cycle apart and the cut's own, at the weight the search chose,
- # the weights never going back toward the frame a cycle apart; RIFE asked only for a frame between the two, and once
- # for a frame at one weight, whatever the run
+ # the weights never going back toward the frame a cycle apart; RIFE asked only for a frame between the two, the weight
+ # nearest a half first, and once for a frame at one weight, whatever the run
calls, asked = list(between.calls), set()
for tried in overlap["tries"]:
sign = 1 if tried["side"] == "after" else -1
assert tried["frames"] == repair.overlap_run(tried["side"], tried["width"], L)
weights = [r["t"] for r in tried.get("made", [])]
assert weights == sorted(weights)
- for r in tried.get("made", []):
- q = r["frame"]
+ rows = {r["frame"]: r for r in tried.get("made", [])}
+ order = tried.get("made_order", [])
+ assert sorted(order) == sorted(q for q, r in rows.items() if 0 < r["t"] < 1)
+ assert [abs(rows[q]["t"] - 0.5) for q in order] == sorted(abs(rows[q]["t"] - 0.5) for q in order)
+ for q in order:
+ r = rows[q]
late, early = (s + q + L, s + q) if sign > 0 else (s + q, s + q - L)
- if 0 < r["t"] < 1 and (tried["side"], q, r["t"]) not in asked:
+ if (tried["side"], q, r["t"]) not in asked:
asked.add((tried["side"], q, r["t"]))
ka, kb, t = calls.pop(0)
assert (ka, kb) == (late, early) and abs(t - r["t"]) < 1e-3
@@ -287,20 +301,34 @@ def test_without_rife_and_the_repair_on_the_loop_fails_asking_for_it(tmp_path, m
assert "sprite-gen rife install" in report["error"]
-def test_a_made_frame_with_a_fault_refuses_the_run_and_the_wrap_is_cut_as_filmed(tmp_path, capsys):
- """A stand-in that returns nothing: every run is refused (`fault`), each width tried once, and the gate refuses the
- cut as filmed."""
+def _stopped(tried: dict) -> dict:
+ """The one frame a run made before it ended: the frame nearest a half, made first, and its fault ended the run."""
+ (stopped,) = tried["made"]
+ assert tried["made_order"] == [stopped["frame"]] and stopped["faults"]
+ assert abs(stopped["t"] - 0.5) == min(abs(w - 0.5) for w in tried["weights"])
+ assert sorted([stopped["frame"], *tried["unmade"]]) == sorted(tried["frames"])
+ return stopped
+
+
+def test_a_made_frame_with_a_fault_ends_its_run_and_the_wrap_is_cut_as_filmed(tmp_path, capsys):
+ """A stand-in that returns nothing: each run ends at the first frame it makes (`fault`), the rest of it not made, each
+ width tried once, and the gate refuses the cut as filmed."""
calls = []
- def empty(a: Image.Image, b: Image.Image, t: float) -> Image.Image:
- calls.append(t)
- return Image.new("RGBA", b.size)
+ class Empty:
+ def __call__(self, a: Image.Image, b: Image.Image, t: float) -> Image.Image:
+ calls.append(t)
+ return Image.new("RGBA", b.size)
- report = _cut(tmp_path, _keyed(tmp_path), interpolate=empty)
+ def read(self, a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ return self(a, b, t), np.zeros((b.height, b.width), dtype=np.float32)
+
+ report = _cut(tmp_path, _keyed(tmp_path), interpolate=Empty())
assert report["status"] == "failed" and report["error"].startswith("video-loop: loop seam ratio")
overlap = report["wrap_overlap"]
assert overlap["frames"] == [] and [t["width"] for t in overlap["tries"]] == list(repair.OVERLAP_WIDTHS)
- assert all(t["outcome"] == "rejected" and "fault" in t["rejected_for"] for t in overlap["tries"])
+ assert all(t["outcome"] == "rejected" and t["rejected_for"] == ["fault"] and _stopped(t)["faults"] == ["empty"]
+ for t in overlap["tries"])
# the overlapped wrap's calls first, once a frame and weight; the jump repair then reads the cut as filmed, its wrap off
# the flow
made = list(dict.fromkeys((t["side"], r["frame"], r["t"]) for t in overlap["tries"] for r in t["made"] if 0 < r["t"] < 1))
@@ -308,6 +336,111 @@ def test_a_made_frame_with_a_fault_refuses_the_run_and_the_wrap_is_cut_as_filmed
assert "the overlapped wrap was refused" in capsys.readouterr().err
+class Torn(Between):
+ """The stand-in, saying its passes disagree over a patch of the body on a frame made far from both of its two (a
+ weight between 0.2 and 0.8): a part torn there."""
+
+ def read(self, a: Image.Image, b: Image.Image, t: float) -> tuple[Image.Image, np.ndarray]:
+ frame = self(a, b, t)
+ apart = np.zeros((b.height, b.width), dtype=np.float32)
+ if 0.2 < t < 0.8:
+ x0, y0, x1, y1 = frame.getbbox()
+ cx, cy = (x0 + x1) // 2, (y0 + y1) // 2
+ apart[cy - 15:cy + 15, cx - 15:cx + 15] = 0.5
+ return frame, apart
+
+
+def test_a_frame_its_passes_disagree_on_ends_its_run_and_the_wrap_is_cut_as_filmed(tmp_path, capsys):
+ """The first walker with a stand-in whose passes disagree over a patch of its body on a frame made far from both of its
+ two: each run makes the frame nearest a half first, reads it torn, and ends there, the rest of it not made; every
+ width is refused (`fault`, torn), and the gate refuses the cut as filmed, the warning naming the torn frame."""
+ torn = Torn()
+ report = _cut(tmp_path, _keyed(tmp_path), interpolate=torn)
+ assert report["status"] == "failed" and report["error"].startswith("video-loop: loop seam ratio")
+ overlap = report["wrap_overlap"]
+ assert overlap["frames"] == [] and [t["width"] for t in overlap["tries"]] == list(repair.OVERLAP_WIDTHS)
+ for t in overlap["tries"]:
+ assert t["outcome"] == "rejected" and t["rejected_for"] == ["fault"] and t["faults"] == ["torn"]
+ stopped = _stopped(t)
+ assert stopped["faults"] == ["torn"] and stopped["proposal"]["torn"] > TORN_WARN
+ # one call a run, the overlapped wrap's first; the jump repair then reads the cut as filmed
+ made = list(dict.fromkeys((t["side"], t["made"][0]["frame"], t["made"][0]["t"]) for t in overlap["tries"]))
+ assert overlap["calls"] == len(made)
+ assert [t for _, _, t in torn.calls[:len(made)]] == pytest.approx([t for _, _, t in made], abs=1e-3)
+ err = capsys.readouterr().err
+ assert "the overlapped wrap was refused" in err and "fault (torn at frame" in err
+
+
+PAINTED = (480, 640) # the third walker's canvas: the tail's tip swings over 200 px between its two frames
+
+
+def _shape(draw) -> np.ndarray:
+ im = Image.new("L", PAINTED, 0)
+ draw(ImageDraw.Draw(im))
+ return np.asarray(im) > 127
+
+
+def _paint(canvas: np.ndarray, mask: np.ndarray, dark: tuple, light: tuple, across: bool = True) -> None:
+ """`mask` painted over `canvas` in a soft gradient from `dark` to `light`, left to right (or top to bottom): a fill with
+ no outline."""
+ ys, xs = np.nonzero(mask)
+ v = xs if across else ys
+ v = (v - v.min()) / max(1, v.max() - v.min())
+ canvas[ys, xs, :3] = (np.array(dark)[None] * (1 - v[:, None]) + np.array(light)[None] * v[:, None]) / 255
+ canvas[ys, xs, 3] = 1
+
+
+def _painted(tail: float) -> Image.Image:
+ """The third walker, seen from behind and painted without a line: legs, boots, shorts, a shirt, a head, the left arm
+ hanging with a four-fingered hand, and a big furred tail from the hips in front of it all, swung `tail` degrees toward
+ the hand (0 hangs straight down). Two of its frames a cycle apart differ by the tail alone."""
+ f = np.zeros((PAINTED[1], PAINTED[0], 4))
+ cx = 250
+ _paint(f, _shape(lambda d: (d.rounded_rectangle((cx - 52, 380, cx - 6, 600), 18, fill=255),
+ d.rounded_rectangle((cx + 6, 380, cx + 52, 600), 18, fill=255))), (232, 180, 150), (250, 214, 190))
+ _paint(f, _shape(lambda d: (d.rectangle((cx - 56, 560, cx - 2, 620), fill=255), d.rectangle((cx + 2, 560, cx + 56, 620), fill=255))),
+ (110, 60, 30), (150, 90, 50), across=False)
+ _paint(f, _shape(lambda d: d.rounded_rectangle((cx - 62, 300, cx + 62, 400), 20, fill=255)), (50, 90, 160), (90, 130, 200), across=False)
+ _paint(f, _shape(lambda d: d.rounded_rectangle((cx - 66, 130, cx + 66, 310), 30, fill=255)), (225, 225, 235), (250, 250, 255))
+ _paint(f, _shape(lambda d: d.ellipse((cx - 55, 20, cx + 55, 140), fill=255)), (200, 190, 220), (240, 235, 250), across=False)
+ _paint(f, _shape(lambda d: d.rounded_rectangle((cx - 96, 150, cx - 68, 330), 14, fill=255)), (235, 190, 160), (250, 215, 190))
+ _paint(f, _shape(lambda d: (d.ellipse((cx - 104, 324, cx - 62, 364), fill=255),
+ *[d.rounded_rectangle((cx - 104 + 10 * i, 354, cx - 96 + 10 * i, 392 - 4 * abs(i - 1.5)), 4, fill=255)
+ for i in range(4)])), (230, 170, 140), (250, 205, 180))
+ a = math.radians(tail)
+ root, length, half = (cx - 20, 330), 230, 85
+ tip = (root[0] - length * math.sin(a), root[1] + length * math.cos(a))
+ nx, ny = math.cos(a), math.sin(a)
+ tuft = _shape(lambda d: (d.polygon([(root[0] - 30 * nx, root[1] - 30 * ny), (root[0] + 30 * nx, root[1] + 30 * ny),
+ (tip[0] + half * nx, tip[1] + half * ny), (tip[0] - half * nx, tip[1] - half * ny)], fill=255),
+ d.ellipse((tip[0] - half, tip[1] - half, tip[0] + half, tip[1] + half), fill=255)))
+ ys, xs = np.nonzero(tuft)
+ along = (xs - root[0]) * -math.sin(a) + (ys - root[1]) * math.cos(a)
+ across = (xs - root[0]) * nx + (ys - root[1]) * ny
+ fur = 0.85 + 0.15 * (0.5 + 0.5 * np.sin(across / 7.0 + along / 40.0)) # bands along the tail
+ f[ys, xs, :3] = np.array([236, 232, 244])[None] / 255 * fur[:, None]
+ f[ys, xs, 3] = 1
+ return Image.fromarray(np.uint8(np.clip(f, 0, 1) * 255 + 0.5), "RGBA")
+
+
[email protected](900) # 102 calls of the binary at 480 x 640: over a minute on a GPU, minutes on a CPU's Vulkan driver
+def test_real_rife_a_frame_made_across_a_hand_is_torn_and_refused_and_a_short_swing_is_kept():
+ """The third walker: between its frame with the tail swung over the hand (48 degrees) and its frame a cycle on with
+ the tail hanging (-8 degrees), the real RIFE drags the hand apart at weights 0.5 and 0.7, and only the passes say so —
+ the frame is refused as torn, and without that reading nothing refuses it (painted without a line, the outline readings
+ pass it, and its coverage keeps no ghost band). Swung 10 degrees, past no hand, its frame half way reads nothing."""
+ interpolate = rife.Rife()
+ early = _painted(-8)
+ for u in (0.5, 0.7):
+ _, said = repair.overlap_frame(interpolate.read, _painted(48), early, u, reads=True)
+ assert said["faults"] == ["torn"] and said["proposal"]["torn"] > TORN_WARN
+ assert faults({**said["proposal"], "torn": None}) == []
+ _, said = repair.overlap_frame(interpolate.read, _painted(2), early, 0.5, reads=True)
+ assert said["faults"] == [] and said["proposal"]["torn"] < TORN_WARN / 2
+ assert interpolate.calls == 3 * (2 + rife.PASS_CALLS)
+
+
def test_the_weights_ease_from_the_late_frame_to_the_early_off_the_ease_only_where_the_body_needs_it():
"""`overlap_weights` on lines drawn by hand: three frames between a late frame and an early one. Unbounded, they are
the ease in and out (`overlap_weight`) on the grid; a window of the rest of the body that moves with the share taken
diff --git a/tests/video/test_rife.py b/tests/video/test_rife.py
index c8260fa..6cbcfd7 100644
--- a/tests/video/test_rife.py
+++ b/tests/video/test_rife.py
@@ -153,6 +153,67 @@ def _disc(cx: int) -> Image.Image:
return Image.fromarray(im, "RGBA")
+def _marked(cx: int) -> Image.Image:
+ """The disc with a block and a bar off its centre lines: no flip or transpose of it is itself."""
+ im = np.asarray(_disc(cx)).copy()
+ im[10:22, cx - 30:cx - 14] = (40, 160, 90, 255)
+ im[70:74, cx + 4:cx + 30] = (230, 220, 40, 128)
+ return Image.fromarray(im, "RGBA")
+
+
+def _premultiplied(frame: Image.Image) -> np.ndarray:
+ """A frame's colour and coverage as `rife.passes` gives them to the binary, premultiplied (0..1)."""
+ x = np.asarray(frame, dtype=np.float32) / 255.0
+ colour = np.uint8(np.clip(rife.bleed(x), 0, 1) * 255 + 0.5) / 255.0
+ cover = np.uint8(x[..., 3:] * 255 + 0.5) / 255.0
+ return np.concatenate([colour * cover, cover], axis=-1)
+
+
+def test_read_makes_the_frame_as_every_call_and_reads_sixteen_single_passes_turned_back(monkeypatch, tmp_path):
+ """`Rife.read`: the frame as `between` makes it, with the augmentation every call asks for, then the sixteen single
+ passes with none — the pair flipped and transposed eight ways, each from a to b at t and from b to a at 1 - t. A binary
+ that hands back its first frame makes every forward pass a and every backward pass b, once each pass is turned back:
+ the passes disagree by exactly how far a and b differ, and only where they differ."""
+ monkeypatch.delenv("SPRITE_GEN_RIFE_TTA", raising=False)
+ binary, log = _recording_binary(tmp_path)
+ interpolate = rife.Rife(binary, tmp_path / rife.MODEL)
+ a, b = _marked(40), _marked(72)
+ frame, apart = interpolate.read(a, b, 0.25)
+ calls = [line.split() for line in log.read_text().splitlines()]
+ assert len(calls) == 2 + rife.PASS_CALLS == interpolate.calls and interpolate.made == 1
+ flags = [[x for x in argv if x in ("-x", "-z", "-u", "-g")] for argv in calls]
+ assert flags[:2] == [["-x", "-z"], ["-x", "-z"]] and all(f == [] for f in flags[2:])
+ times = [argv[argv.index("-s") + 1] for argv in calls[2:]]
+ assert times.count("0.2500") == times.count("0.7500") == rife.PASS_CALLS // 2
+ assert np.array_equal(np.asarray(frame), np.asarray(a)) # the frame between, from the copying binary: a itself
+ pa, pb = _premultiplied(a), _premultiplied(b)
+ expected = np.sqrt(((pa - pb) ** 2).sum(axis=-1) / 4.0)
+ assert apart.shape == (96, 128) and np.allclose(apart, expected, atol=1e-5)
+ assert apart[pa[..., 3] + pb[..., 3] == 0].max() == 0 and apart.max() > 0.5
+
+
+def test_torn_reads_a_thick_patch_in_a_window_and_not_an_edge_or_scattered_pixels():
+ """`rife.torn` on disagreement drawn by hand over a body 100 px high, windows of 10 px: a patch a window wide where the
+ passes disagree fills its window; lines a pixel or two wide, pixels scattered one in five, or a patch they disagree on
+ by less than TORN_LEVEL leave it empty; a frame with no body reads nothing."""
+ body = np.zeros((160, 128, 4), dtype=np.uint8)
+ body[30:130, 34:94] = (200, 180, 160, 255)
+ made = Image.fromarray(body, "RGBA")
+ apart = np.zeros((160, 128), dtype=np.float32)
+ patch = apart.copy()
+ patch[60:72, 50:62] = 0.3
+ assert rife.torn(patch, made) == 1.0
+ lines = apart.copy()
+ lines[:, 40] = lines[:, 70:72] = lines[90, :] = 0.5
+ assert rife.torn(lines, made) == 0.0
+ scattered = np.where(np.random.default_rng(7).random(apart.shape) < 0.2, 0.5, 0.0).astype(np.float32)
+ assert rife.torn(scattered, made) < 0.05
+ faint = apart.copy()
+ faint[60:72, 50:62] = rife.TORN_LEVEL * 0.8
+ assert rife.torn(faint, made) == 0.0
+ assert rife.torn(patch, Image.new("RGBA", (128, 160))) == 0.0
+
+
@pytest.mark.real_rife
def test_real_rife_makes_the_frame_between_in_colour_and_alpha():
interpolate = rife.Rife()