Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
feat(video-loop): a two-step cut's wrap that a part on its own beat leaves open is closed with the clip's frames a cycle apart
Soohong Kim363a6275dbc8
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 1ef5839..0769dad 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,8 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the## Unreleased+- **New: `video-loop --anchor motion-auto` closes a two-step cut's wrap that a part on its own beat leaves open, with the clip's frames a cycle apart.** A walk cut two steps long by the clip's crossings closes for the legs; a tail swinging round on a beat of its own closes on no cut of two steps, so the seam gate refused every one — and the eased retime does not reach it, since the part turns back at the wrap, where its frames read off the flow against the way. The clip goes on past the cut, and its frame a cycle after one of the cut's shows the legs at the same point of the walk. Under `--repair auto` (and `on`), where such a cut's wrap reads over `--seam-max` on its frames as corrected (`loop.wrap_on_cells`), the frames beside the wrap are made again before the jump repair (`repair.overlap_wrap`): each by RIFE between its two frames of one time — the clip's frame a cycle apart, moved by the analysis translation between the two and corrected as the cut frame was (`loop.cycle_apart`), and the cut's own — so the wrap plays a step the clip filmed and the part goes over to the cut's way over those frames, each frame at its own time. The weights (`repair.overlap_weights`) are the ease in and out x²(3 − 2x), held off it only in the steps where the rest of the body or a body measure, read on the line between the two frames, would go past its bound; a weight of 0 or 1 shows a frame whole, with no call. Four frames, then six (`OVERLAP_WIDTHS`), on the side — after the wrap or before it — whose frames a cycle apart differ least, never past half the loop; a run is taken only where no made frame has an interpolation fault, the gate passes it, the loop played round still covers two steps (`loop.ring_steps`), no body measure moves a pixel past the largest step there of either run its frames are made from — the stretch as the loop plays it and the frames a cycle apart as the clip filmed them, whose step into the run the wrap now plays — the part steps no further than over the stretch, no new rush is read, no window of the rest of the body steps further than its own largest step over either run, and no frame is left further off the flow over its limit (`rejected_for`). The jump repair after it makes nothing from, beside or again over those frames (`jump_repair.overlapped`). The report carries `wrap_overlap` (each try, and the run taken), the summary names it, and without RIFE the wrap is cut as filmed with a warning line (`--repair on` refuses). On the test walker (front, 26 frames a cycle, the tail on a 41-frame beat) the cut 29/26 reads 6.23 at the gate as filmed and 3.17 after the jump repair alone; four frames after the wrap rush the tail and are refused, six are taken, and the loop passes at 1.04, two steps ([loop-repair](docs/loop-repair.md#the-overlapped-wrap--a-part-on-its-own-beat-that-no-cut-of-two-steps-closes)).+- **Tests** — `tests/video/test_loop_overlap.py` (a front walker whose tail swings round on its own beat, refused at the seam on every cut of two steps, is closed at the wrap: each frame beside it made between the clip's frame a cycle apart and the cut's own at its eased weight, four frames refused for a rush and six taken, two steps on the loop played round, the body within its drift, the rest of it no faster, every other frame the clip's own and nothing made by the jump repair; `--repair off` refused as before; without RIFE cut as filmed with a warning, and with `--repair on` refused asking for it; a made frame with a fault refuses every run and the cut is kept as filmed; a weighting search on lines drawn by hand: the ease where nothing binds, held off it in the one step a window of the rest of the body is bounded in, none where a body measure may not move).- **Behavior change: `video-loop --anchor motion-auto` cuts a walk two steps long — the left foot forward once and the right once — wherever the clip shows its feet pass.** Left foot forward and right foot forward are nearly the same silhouette from the side or a diagonal, so the pixel search could take one step for a cycle (a half step loop: one foot goes forward), three steps where a tail on its own beat repeated there (the wrap puts the nearer foot back: a stomp in place), or refuse a slow walk whose cycle its window could not hold. The feet are now read (`crossings.read`): where they are seen apart, each run of frames with them together is a crossing; where they never are (front, back), the foot band's lean about the body's axis changing side is, and the lean must turn over a step on. Every crossing counted is one the clip shows whole: a run cut by the clip's first or last frame counts only where its twin confirms it, every step a cut plays — on both sides of its wrap — lies within √2 of the clip's own step, and a cut is two steps only when the walk's phase advances by two over it, its wrap joins two points of a step within [1/√2 − 1, √2 − 1] of each other under every bound of what the clip does not show, and the loop as played shows two crossings too (`two`; `refuted` one or three steps; anything else `uncertain`, never two). The search's cut is kept, every image as before, where it is a two within 1.5 frames of phase; otherwise the cut is the two-step cut in phase whose top band does not pop at the wrap, whose wrap is within the seam gate of its step and whose seam is the smoothest by the flow reading (both seam frames under `repair.FLOW_LIMIT`, within 0.5 of the smoothest, the wrap nearest the clip's own way on), read on the frames moved as the cut will be corrected, of any length the count allows inside an explicit `--min-len`/`--max-len` (`method: observed-two-steps-v1`); of the first four, the first whose seam the gate will pass, read as the gate reads it (`loop.seam_on_cells`), is taken. A window that holds no two steps of a walk whose steps are seen is refused, naming both (`no cut of two steps in --max-len 25: …`), never cut one step long. A clip whose feet are not read is cut exactly as before. The report records `cycle.step_count` (the crossings, the step range, the verdict of the cut taken and of the search's cut, `chosen_by`); a cut on two steps records `cycle.steps` `{count: 2, by: crossings}` and `strip.json` `steps: 2`, so `video-cycle-align` reads its half-way return as its step, and the step screen's pixel suspicion is settled (`pixel_suspect`). Where the clip shows its steps but no cut of two, the search runs again refusing the cuts the crossings refute, and a refusal lists each window's count, those refuted last. `--steps` is still the word on the search's cut. `repair.clip_flow_states` and `repair.ring_scores` read the flow of every cut of a clip on one path; `flow_scores` and the repair read as before ([loop-repair](docs/loop-repair.md) section 3, "The step count").- **Tests** — `tests/video/test_crossings.py` (a walk that stalls proves no two steps across the stall, on either side of the wrap; a hold before or after the cut, a stall with the feet apart, a stand with them together, a stomp in place, the stance a clip opens on, a pass the first frame cuts with or without a twin that confirms it, a change the clip ends on: none two; half a cycle and three steps refuted, a wrap half a step off and a wrap that slips feet together in uncertain; noise at the threshold and a wobble at the axis no steps; a hold the step range absorbs stays two and a longer one does not; two crossings set no range; a lean that does not turn over is not read; every two lies in the lengths two steps cover; the lean and gap off drawn frames), `tests/video/test_walk_steps.py` (on synthetic walkers the pixel search alone cuts one step, one step past its window, three steps by the gait fallback, nothing at all, and across a stalled step: each is cut two steps long, in phase, the strip saying `steps: 2`; a walk the search already cuts two steps in phase is cut byte for byte as before; a legless body is cut as the search cuts it; a count given is the word; the set aligns a loop cut on two steps as one cycle; `detect` refuses what the crossings refute and orders a refusal by the count), the step screen's tests in `tests/video/test_one_step_cut.py` and the pixel search's in `tests/video/test_auto_motion.py`, `tests/video/test_loop_prop_seam.py` and `tests/video/test_loop_prop_seam_held.py` (a walker whose legs open and close once a drawing and never pass each other, which the count reads as one step a drawing) read their walker's feet as unseen (`tests/conftest.py`, `feet_unseen`), and `tests/video/test_explicit_window.py` refuses a `--max-len 25` that holds one step of the 42-frame walker.- **New: `video-loop --repair auto` (and `on`) eases a part that rushes on its way — the frames either side of the rush are made again at times that spread it, the body's own steps kept.** A part can keep its way and still jump: a tail that runs short of the cycle rushes on at the wrap, covering in one step what it covers in eight elsewhere, and each frame there lies near enough the line between its neighbours that the flow reading lets it pass, or takes the frame at the wrap out and leaves the rush over two steps. `repair.surges` reads the speed on the flow reading's own path and windows: a run of one or two steps whose mean step is at least 3.0 usual steps and 3.0 times the mean of the three steps either side, the part going on the way it came through it (a step that turns back is a frame flung, the flow repair's). After the flow groups, the worst rush is eased over three or four drawings either side, made from the frames as filmed (a frame the flow made inside is made again, `supersedes`) and kept between two frames as filmed, never leaving more than half the loop made with the rounds before it (as much as `video-loop-compare` reads back as the repair's): `repair.ease_times` gives every frame a time, on a grid of a twentieth of a step, where the part's step changes from one step to the next as little as it can — from the speed it came in with to the speed it goes on with — while the foot line's centre and the body's mass (`loop.foot_centre`, `loop.body_centre`) and the lowest solid row move in a step no further than the stretch's largest step as it played and half a pixel, and every other window of the flow reading no further than its own largest step there; within 2 % of that, the timing nearest the frames' own. How far the part has gone is the path of its coverage's centre in the windows that read the rush, which grows as far as the part moves where a blurred distance stops growing. RIFE makes each drawing between the frames as filmed either side of its time, one call a drawing, at most nine, outside the three of the flow groups; the stretch is taken only where no frame made has an interpolation fault, no body measure moves a pixel past the stretch's largest step, the part's largest step falls, no frame is left further off the flow and over its limit, no new rush appears in the rest of the body, and no window of the rest of the body steps further on the frames made than its own largest step over the stretch as it played (`rejected_for`, `the rest faster`: the search reads the rest of the body on the line between the frames as filmed, and the frames made, which go the body's own way between them, step further where a part moves far between frames). On the synthetic walker drawn over 24 frames a cycle, the tail a quarter short goes, after the flow repair, from 5.3 usual steps into the wrap to 3.4 with no rush left; over 16 frames, 15 % short and under the flow reading's limit, from 3.6 to 1.7; over 36 frames every stretch is refused, the rest of the body 1.5 % to 19 % past its own largest step on the frames made; ordinary walks and a flung frame read no rush. A loop with no rush costs one more reading and no call. `jump_repair` carries `surge`, `surge_after`, `ease_skipped`, `ease_calls` and the round `why: ease` (its `rest`: per window the rush is not read in, `step_max_before`, `step_max_after` and `faster`), `replaced` its frames, and the summary `eased`; without RIFE the rush is cut as filmed with a warning line, and `--repair on` refuses ([loop-repair](docs/loop-repair.md) section 2, "The eased retime").diff --git a/SKILL.md b/SKILL.mdindex a66ab77..b8ad97a 100644--- a/SKILL.md+++ b/SKILL.md@@ -144,7 +144,7 @@ Scene creation consumes finished assets and remains optional. Asset metadata own## RIFE for walk and run loops-`video-loop` repairs a walk's or run's jump frames, eases a part that rushes on its way at the wrap and gives its filmed ghosts way (a half-drawn frame the video model left, `ghost_given_way`), and `video-set` gives a direction set one cycle length, both with RIFE in-betweens. Install it once per machine: `$SPRITE_GEN_ROOT/.venv/bin/sprite-gen rife install` (pinned rife-ncnn-vulkan 20221029 and model rife-v4.6, SHA-256 checked, into the user data directory, ending with a check frame; Linux without a GPU first needs `libvulkan1 mesa-vulkan-drivers`). Every call runs the binary's test-time augmentation `-x -z`; on a CPU it costs 15 to 19 times a plain call, so a CPU-only machine aligning a set may set `SPRITE_GEN_RIFE_TTA=` (none) or `-z`. Without it every loop is still cut — a jump or a rush as filmed, each filmed ghost given way to the clean frame beside it — and what was left unrepaired says so: a `warning:` line on stderr and `jump_repair.rife` with `jump_repair.unmade` (the jump left as filmed) or `jump_repair.ease_skipped` (the rush) in the loop report (`applied` alone is not the sign: a ghost given way reads true), `cycle_align.<state>.applied: false` and `warnings` in `set.report.json`. When that happens, tell the user which loops went unrepaired and offer the install; after it, cut those loops again and run `video-cycle-align`. `video-loop --repair on` and `video-cycle-align` fail without RIFE instead. [loop-repair](docs/loop-repair.md).+`video-loop` repairs a walk's or run's jump frames, eases a part that rushes on its way at the wrap, closes a two-step cut's wrap that a part on its own beat leaves open with the clip's frames a cycle apart, and gives its filmed ghosts way (a half-drawn frame the video model left, `ghost_given_way`), and `video-set` gives a direction set one cycle length, both with RIFE in-betweens. Install it once per machine: `$SPRITE_GEN_ROOT/.venv/bin/sprite-gen rife install` (pinned rife-ncnn-vulkan 20221029 and model rife-v4.6, SHA-256 checked, into the user data directory, ending with a check frame; Linux without a GPU first needs `libvulkan1 mesa-vulkan-drivers`). Every call runs the binary's test-time augmentation `-x -z`; on a CPU it costs 15 to 19 times a plain call, so a CPU-only machine aligning a set may set `SPRITE_GEN_RIFE_TTA=` (none) or `-z`. Without it every loop is still cut — a jump or a rush as filmed, each filmed ghost given way to the clean frame beside it — and what was left unrepaired says so: a `warning:` line on stderr and `jump_repair.rife` with `jump_repair.unmade` (the jump left as filmed) or `jump_repair.ease_skipped` (the rush) in the loop report (`applied` alone is not the sign: a ghost given way reads true), `cycle_align.<state>.applied: false` and `warnings` in `set.report.json`. When that happens, tell the user which loops went unrepaired and offer the install; after it, cut those loops again and run `video-cycle-align`. `video-loop --repair on` and `video-cycle-align` fail without RIFE instead. [loop-repair](docs/loop-repair.md).## Contracts and advanced toolsdiff --git a/docs/README.md b/docs/README.mdindex 18d442e..2e832a3 100644--- a/docs/README.md+++ b/docs/README.md@@ -85,7 +85,7 @@ grouping is derived from `sprite_gen/_modules.py`, the one taxonomy table.| [video.md](video.md) | `sprite-gen video` / `video-extend` / `video-edit` / `video-resume`: stills and clips to mp4 through Grok Imagine (image-to-video, last-frame pin, references, extension, editing, asking again for an unfinished generation) with the user's own credential || [video-pipeline.md](video-pipeline.md) | Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, the batch, a set's directions on one sheet || [loop-review.md](loop-review.md) | Automatic loop decisions, ambiguous gait review, visual evidence and explicit cut/alignment overrides |-| [loop-repair.md](loop-repair.md) | RIFE (where it runs, cost, licences, `sprite-gen rife install`, what runs without it), jump-frame repair, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the jolt index and its gate, one cycle length per direction set |+| [loop-repair.md](loop-repair.md) | RIFE (where it runs, cost, licences, `sprite-gen rife install`, what runs without it), jump-frame repair, the flow reading (frames off a walk's way), the eased retime (a part that rushes on its way), the overlapped wrap (a part on its own beat at a two-step cut's wrap), the jolt index and its gate, one cycle length per direction set || [frame-interpolation.md](frame-interpolation.md) | Generative in-betweens for sprite frames, recorded as a take || [seamless-video-loop.md](seamless-video-loop.md) | Making a non-looping ambient clip loop forever (RIFE seam bridge) — a different job from pipeline B |diff --git a/docs/loop-repair.md b/docs/loop-repair.mdindex 9577c20..4bb83f4 100644--- a/docs/loop-repair.md+++ b/docs/loop-repair.md@@ -223,7 +223,10 @@ how much a step changes ([the flow reading](#the-flow-reading--a-frame-off-the-wbelow) — are taken out and made again by RIFE between the frames either side. Where a part keeps its wayand rushes — a tail that runs short of the cycle, rushing on at the wrap — the frames either side ofthe rush are then made again at times that spread it, the body's own steps kept ([the eased-retime](#the-eased-retime--a-part-that-rushes-on-its-way) below).+retime](#the-eased-retime--a-part-that-rushes-on-its-way) below). Before all of it, a cut two steps+long whose wrap a part on its own beat leaves open has the frames beside its wrap made again between+the clip's frames a cycle apart and its own ([the overlapped+wrap](#the-overlapped-wrap--a-part-on-its-own-beat-that-no-cut-of-two-steps-closes) below).| Rule | Value | Why ||---|---|---|@@ -559,6 +562,79 @@ seam gate reads the rendered cells, and the summary names them (`eased`). Withoucut as filmed with a warning line (`why`: "RIFE not installed — the rush cut as filmed"), and`--repair on` refuses.+### The overlapped wrap — a part on its own beat that no cut of two steps closes++A walk cut two steps long by the clip's crossings ([the step+count](#the-step-count--two-steps-read-off-the-clips-crossings), section 3) closes for the legs. A+part on a beat of its own — a tail swinging round every 41 frames on a walk of 26 — closes on no cut+of two steps: the cut's last frame and its first show it in two places the clip never filmed side by+side, and the seam gate refuses every one. Cutting again does not help, and the eased retime does not+reach it: where the part turns back at the wrap, the frames there read off the flow against the way,+which the retime never eases over.++The clip goes on past the cut. Its frame a cycle after one of the cut's shows the legs at the same+point of the walk, and the part where the clip carried it. `video-loop --anchor motion-auto` makes the+frames beside the wrap again between the two (`repair.overlap_wrap`), before the jump repair: the wrap+then plays a step the clip filmed, and the part goes over from the clip's way to the cut's over those+frames.++| Rule | Value | Why |+|---|---|---|+| When | `--repair auto` or `on`; a cut whose two steps the clip's crossings show (`cycle.steps.by: crossings`) and whose wrap the gate reads over `--seam-max` on its frames as corrected (`loop.wrap_on_cells`) | the legs close; what does not is a part on its own beat. A cut whose steps are not counted keeps the gate's word |+| The frames a cycle apart | the clip's frame a cycle after each of the cut's first frames (`after`), or before each of its last (`before`), one past the widest run, as far as the clip after its lead-in holds them; each moved by the analysis translation between the two — the drift the clip walks over a cycle — and corrected as its cut frame was (`loop.cycle_apart`) | placed where the cut frame is, the in-between moves the part, not the body |+| How many | four frames, then six (`OVERLAP_WIDTHS`), never past half the loop (`MADE_SHARE`), on the side whose frames a cycle apart differ least from the cut's over the run | the fewer made, the more of the loop is the clip's own; six where four rush the part |+| Each frame | made by RIFE between its two frames of one time — the late (the clip's frame a cycle after, or the loop's own before the wrap) and the early (the loop's own, or the clip's frame a cycle before) — at the weight of the early the search chooses (`repair.overlap_weights`, below); a weight of 0 or 1 shows that frame whole, no call | each frame at its own time, the legs where they were on both; next to the wrap near the frame that continues the clip across it |+| The weights | in play order, from 0 at the frame before the run to 1 at the frame after it, never going back, on a grid of a twentieth: of those that keep every window of the rest of the body within its own largest step over either run (below) and every body measure within the largest step either run takes and half a pixel, read on the line between the two frames' states, the one nearest the ease in and out x²(3 − 2x), x = p/(width + 1) at the run's p-th frame (`repair.overlap_weight`) | the part goes over in an ease in and out, held off it only in the steps where the rest of the body has no room: a window that closes all but a step still moves by what it differs a cycle apart, and a weight that moves there pushes it past its own step |++The stretch runs from the frame before the run to the frame after it, the wrap inside, and its frames+are made between two runs the clip filmed: the stretch as the loop plays it, and the frames a cycle+apart as the clip filmed them, from the loop's own frame on the far side of the wrap (`after`: the+cut's last frame, then the clip's frames a cycle after the run and one more). A made frame next to the+wrap stands near the clip's frame a cycle on, and the wrap then plays the clip's own step into it: the+body moves there as the clip moved it, which the stretch as the loop played it need not show. The run+is taken only where, on the frames made:++| Read on the frames made | Refused for (`rejected_for`) |+|---|---|+| no made frame carries an interpolation fault (section 4) | `fault` |+| the seam gate's reading passes | `seam` |+| the loop, played round three times, still covers two steps: the crossings read on it, its middle turn judged `two` (`loop.ring_steps`) | `steps` |+| no body measure (`repair.drift_points`) moves in a step more than a pixel past the largest step either run takes there | `drift` |+| the part — the windows of the flow reading where the frames a cycle apart differ by a usual step or more somewhere in the run (`OVERLAP_APART`) — steps no further than it did over the stretch as the loop played it | `no gain` |+| no rush the loop did not have is read across the stretch (`repair.surges`) | `a new rush` |+| no window of the rest of the body steps further than its own largest step over either run | `the rest faster` |+| no frame of the stretch is read further off the flow than before and at or over its limit | `off the flow` |++The first run taken ends the search; where none is, the cut is kept as filmed and the gate refuses it+as before. The jump repair after it makes nothing from, beside or again over a frame it made+(`jump_repair.overlapped`); a filmed ghost beside one gives way to the filmed frame on its other side.+A loop drawn on twos or threes is not overlapped: a frame made beside the wrap would split a drawing.++On the walker of `tests/video/test_loop_overlap.py` — seen from the front, 26 frames a cycle, its+tail swinging round on a beat of 41 frames, 73 frames, with the stand-in in-between (measured+2026-10-18):++| The cut 29/26 | Seam gate (2.0) | Jolt index | The loop played round |+|---|---|---|---|+| as filmed; every cut of two steps the gate was asked reads 6.0 to 6.2 | 6.23 | 0.71 | two steps |+| the jump repair alone: the flow repair takes the frame at the wrap, and the tail turns back there (turn −0.91), so the retime eases nothing | 3.17 | 0.44 | two steps |+| overlapped: four frames after the wrap refused (`a new rush`), six taken at the ease (weights 0.05, 0.2, 0.4, 0.6, 0.8, 0.95) | 1.04 | 0.30 | two steps |++The report's `wrap_overlap` carries `seam` (the reading that asked for it), `seam_max`, `widths`,+`calls`, and each try in `tries` — `side`, `width`, `frames`, `stretch`, `eased` and `off_ease` (the+ease and how far the weights left it), `made` (per frame: `frame`, its weight `t`, `proposal`, `faults`,+`look`), `shown` (the frames shown otherwise than as cut), `seam`, `steps`, `part` (`windows`, `apart`,+`step_max_before`, `step_max_after`), `rest` (`windows`, `step_max_bound`, `step_max_after`, `faster`),+`drift`, `off_max`, `rushes`, `faults`, `look`, `outcome` and `rejected_for` (`no weighting keeps the+body in bound` where the search found none and nothing was made); where a run was taken, `side`,+`width`, `frames` (those shown otherwise than as cut) and `seam_after`; and `skipped` where none was tried (drawn on twos, a loop too short+for a run that leaves half of it filmed, the clip holds no frames a cycle from the cut's beside the wrap,+no interpolator). The summary names it. A made frame+that cross-fades two drawings inside its outline is kept and named in a `warning:` line, as for the+jump repair. Without RIFE the wrap is cut as filmed with a warning line (`why`: "RIFE not installed —+the wrap cut as filmed"), and `--repair on` refuses. It costs at most ten RIFE calls, outside the jump+repair's.+## 3. The jolt index — reported on every walk, a gate only when askedA loop whose steps change unevenly jolts even when no single step is a jump. Every walk or run@@ -807,8 +883,10 @@ far the wrap is from joining the same point of the walk, in frames, the worst ov(`loop.seam_on_cells`: the cut corrected as it will be, its wrap over its step on the analysisthumbnail; `step_count.seam_gate.tried`): the correction of one cut can leave a jump at its wrap thatno reading of the clip shows. A part on a beat of its own — a tail, a staff — that closes on no cut of- two steps leaves every one with a wrap that jumps: the first is cut, still two steps, and the seam gate- says whether it closes, as for any cut.+ two steps leaves every one with a wrap that jumps: the first is cut, still two steps; where the gate+ would refuse its wrap, the frames beside it are made again from the clip's frames a cycle apart ([the+ overlapped wrap](#the-overlapped-wrap--a-part-on-its-own-beat-that-no-cut-of-two-steps-closes),+ section 2), and the seam gate says whether it closes, as for any cut.The pixel search's repeat rule is not asked: a part off the legs' beat — a tail on its own beat —repeats on its own, and its seam is the flow repair's (section 2). `step_count.repeat` records thesearch's word on the cut, and a cut it would refuse, or one that passed no gate, is a warning line.diff --git a/docs/video-pipeline.md b/docs/video-pipeline.mdindex 716876c..7419194 100644--- a/docs/video-pipeline.md+++ b/docs/video-pipeline.md@@ -937,7 +937,11 @@ take a clean frame there. Then a part that keeps its way and rushes — a tail tthe cycle, rushing on at the wrap — has the three or four drawings either side of the rush madeagain at times that spread it, made from the frames as filmed, the body's feet, mass and groundline moving no further in a step, nor the rest of it faster, than they did there (the eased-retime, up to nine calls of its own). The report's `jump_repair` names the frames. RIFE+retime, up to nine calls of its own). Before all of it, a cut two steps long whose wrap a part+on its own beat leaves open — a tail swinging round on a beat of its own, closing on no cut of two+steps — has the four or six frames beside its wrap made again between the clip's frames a cycle+apart and its own, so the wrap plays a step the clip filmed (the overlapped wrap, `wrap_overlap`).+The report's `jump_repair` names the frames. RIFEis installed once with `sprite-gen rife install`; without it a loop that needs a frame is cut asfilmed with a warning, each filmed ghost given way to the clean frame beside it (`--repair on`fails instead). A frame shown twice in a row is one frame of the GIF and WebP, checked so, and the seam gate reads a frame given way as filmed (`seam_as_filmed`). See [loop repair](loop-repair.md). The repaired loop's jolt index and the head's frame-to-framediff --git a/sprite_gen/video/loop.py b/sprite_gen/video/loop.pyindex da7342d..fc59076 100644--- a/sprite_gen/video/loop.py+++ b/sprite_gen/video/loop.py@@ -892,16 +892,22 @@ def shown_runs(cells: list[Image.Image]) -> int:def _repair_jumps(frames: list[Image.Image], interpolate: rife_mod.Interpolate | None, *, facing: str,- required: bool, screen: dict[str, Any]) -> tuple[list[Image.Image], dict[str, Any]]:+ required: bool, screen: dict[str, Any], wrap: dict[str, Any] | None = None+ ) -> tuple[list[Image.Image], dict[str, Any], dict[str, Any] | None]:"""`repair_mod.repair_jumps` on the cut as filmed and its ghost `screen`, with RIFE located onlyonce a frame is to be made: a loop with no frame off its flow and no ghost between two clean framesneeds no binary, nor one with no part that rushes (the eased retime, `repair.surges`). The report names theinterpolator that made the frames.+ `wrap` (the clip's frames a cycle from the cut's beside its wrap, `cycle_apart`, and the gate's `seam_max`) asks+ first for the overlapped wrap (`repair_mod.overlap_wrap`) of a cut two steps long whose wrap the gate refuses;+ the frames it makes are then neither made from, nor beside, nor again (`overlapped`). Its record is returned+ third, None where it was not asked.+A frame to make and no RIFE installed: `required` lets `RifeNotInstalled` through; otherwise- each filmed ghost gives way to the clean frame beside it, a jump (a frame off the flow) and a rush stay as- filmed, and the record says why where a jump or a rush was left (`rife`, `install`; `applied` false where- nothing was replaced)."""+ each filmed ghost gives way to the clean frame beside it, a jump (a frame off the flow), a rush and a wrap that does+ not close stay as filmed, and the record says why where a jump or a rush was left (`rife`, `install`; `applied`+ false where nothing was replaced)."""made_by: list[dict[str, str]] = []calls = 0@@ -914,14 +920,30 @@ def _repair_jumps(frames: list[Image.Image], interpolate: rife_mod.Interpolate |calls += 1return interpolate(a, b, t)+ overlap: dict[str, Any] | None = None+ overlapped: list[int] = []+ if wrap is not None:+ asked = partial(repair_mod.overlap_wrap, others=wrap["others"], seam=wrap_on_cells, steps=ring_steps,+ seam_max=wrap["seam_max"], reads=bool(screen["reads"]))+ try:+ frames, overlap = asked(frames, interpolate=lazy)+ except rife_mod.RifeNotInstalled as exc:+ if required:+ raise+ frames, overlap = asked(frames, interpolate=None)+ overlap.update(why="RIFE not installed — the wrap cut as filmed", rife=str(exc), install=rife_mod.INSTALL_COMMAND)+ overlapped = overlap["frames"]+ if overlap["calls"]:+ overlap["interpolator"] = {"kind": "rife-ncnn-vulkan", **made_by[0]} if made_by else {"kind": "injected"}+ wrap_calls = callstry:- repaired, record = repair_mod.repair_jumps(frames, lazy, facing=facing, screen=screen)+ repaired, record = repair_mod.repair_jumps(frames, lazy, facing=facing, screen=screen, overlapped=overlapped)except rife_mod.RifeNotInstalled as exc:if required:raise# RIFE is located before the first frame is made, so nothing was made yet: the stage runs again# with no interpolator, each ghost giving way to the clean frame beside it and a jump left as filmed.- repaired, record = repair_mod.repair_jumps(frames, None, facing=facing, screen=screen)+ repaired, record = repair_mod.repair_jumps(frames, None, facing=facing, screen=screen, overlapped=overlapped)record["applied"] = bool(record["replaced"])if "unmade" in record:record.update(why="RIFE not installed — cut as filmed" if not record["replaced"] else@@ -932,14 +954,12 @@ def _repair_jumps(frames: list[Image.Image], interpolate: rife_mod.Interpolate |record.update(why="RIFE not installed — the rush cut as filmed" if not record["replaced"] else"RIFE not installed — the rush cut as filmed, each filmed ghost given way to the clean frame beside it",rife=str(exc), install=rife_mod.INSTALL_COMMAND)- return repaired, record+ return repaired, record, overlapframes = repairedrecord["applied"] = bool(record["replaced"])- if made_by:- record["interpolator"] = {"kind": "rife-ncnn-vulkan", **made_by[0]}- elif calls:- record["interpolator"] = {"kind": "injected"}- return frames, record+ if calls > wrap_calls:+ record["interpolator"] = {"kind": "rife-ncnn-vulkan", **made_by[0]} if made_by else {"kind": "injected"}+ return frames, record, overlapdef _ghost_lines(jump_repair: dict[str, Any]) -> list[str]:@@ -970,6 +990,25 @@ def _ghost_lines(jump_repair: dict[str, Any]) -> list[str]:return lines+def _overlap_lines(overlap: dict[str, Any]) -> list[str]:+ """What became of the overlapped wrap (`repair.overlap_wrap`), in words (`video-loop: warning:` lines): the frames a+ person should look at where it was made, and why it was not where it was not."""+ said = (f"the cut's wrap does not close (seam {overlap['seam']:.2f} over {overlap['seam_max']:g}): a part on a beat of its "+ "own closes on no cut of two steps")+ if overlap["frames"]:+ taken = overlap["tries"][-1]+ return [f"frame {r['frame']} was made at the wrap between the clip's frame a cycle apart and its own, and may cross-fade "+ f"two drawings inside its outline (crossfade {r['proposal']['crossfade']:.4f}); kept, a reading to look at "+ "(docs/loop-repair.md section 2)" for r in taken["made"] if r["look"]]+ if "rife" in overlap:+ return [f"{said}, and RIFE is not installed, so the overlapped wrap was not made — run `{rife_mod.INSTALL_COMMAND}` "+ "and cut it again (docs/loop-repair.md section 2)"]+ if "skipped" in overlap:+ return [f"{said}; the overlapped wrap was not made: {overlap['skipped']} (docs/loop-repair.md section 2)"]+ refused = "; ".join(f"{t['width']} frames {t['side']} the wrap: {', '.join(t['rejected_for'])}" for t in overlap["tries"])+ return [f"{said}, and the overlapped wrap was refused ({refused}): cut as filmed (docs/loop-repair.md section 2)"]++def _shifts(analysis: dict[str, Any]) -> tuple[np.ndarray, int]:"""The analysis translation `auto_motion.analyse` measured, whole pixels a frame, and the room it needs."""shifts = np.rint(np.asarray(analysis["analysis_only_translation_xy"], dtype=np.float64)).astype(int)@@ -1075,14 +1114,19 @@ class SeamFlow:def seam_on_cells(frames: list[Image.Image]) -> float:"""The seam gate's reading of a motion-auto cut before it is cut: its frames corrected as the cut will be- (`auto_motion.discover`, `auto_motion.correct_cycle`), cropped to their union box, the wrap over the mean step on- the analysis thumbnail — the rendered cells the gate reads, short of the strip's scale and the jump repair.+ (`auto_motion.discover`, `auto_motion.correct_cycle`), then read as `wrap_on_cells` reads them.Infinite where the frames cannot be corrected."""try:regions, _ = auto_motion.discover(frames, reference_index=0)frames, _ = auto_motion.correct_cycle(frames, regions, coarse_dx=body_wrap_offset(frames, search=motion_anchor.COARSE_SEARCH))except ValueError:return math.inf+ return wrap_on_cells(frames)+++def wrap_on_cells(frames: list[Image.Image]) -> float:+ """The seam gate's reading of a loop's frames as corrected: cropped to their union box, the wrap over the mean step+ on the analysis thumbnail — the rendered cells the gate reads, short of the strip's scale."""boxes = [f.getbbox() for f in frames]boxes = [b for b in boxes if b]box = (min(b[0] for b in boxes), min(b[1] for b in boxes), max(b[2] for b in boxes), max(b[3] for b in boxes))@@ -1091,6 +1135,42 @@ def seam_on_cells(frames: list[Image.Image]) -> float:return float(np.abs(flat[-1] - flat[0]).mean()) / step if step > 0 else math.inf+def ring_steps(frames: list[Image.Image]) -> dict[str, Any]:+ """The steps a loop shows as it plays round: the crossings read on its frames played three times over+ (`crossings.read`), the turn in the middle judged (`Reading.judge`) — a two where the loop still walks two steps,+ joined where the walk goes on."""+ ring = [*frames, *frames, *frames]+ signals = leg_signals(ring)+ try:+ reading = crossings_mod.read(crossings_mod.foot_signals(ring, gap=signals["gap"] if signals else None))+ except ValueError as exc:+ return {"verdict": "uncertain", "code": "unread", "why": str(exc)}+ return reading.judge(len(frames), len(frames))+++def cycle_apart(files: list[Path], *, start: int, length: int, skip: int, analysis: dict[str, Any],+ motion: dict[str, Any], size: tuple[int, int]) -> dict[str, dict[int, Image.Image]]:+ """The clip's frames a cycle from a motion-auto cut's frames beside its wrap, for `repair.overlap_wrap`: a cycle after+ its first frames ("after") and before its last ("before"), one past the widest run, as far as the clip after its+ lead-in holds them. Each is placed where its cut frame is: moved by the analysis translation between the two+ (`_shifts`, the drift the clip walks over a cycle), then corrected as the cut frame was (`motion`'s shift and padding)."""+ shifts, _ = _shifts(analysis)+ left, top = motion["padding_ltrb"][:2]+ widest = max(repair_mod.OVERLAP_WIDTHS)+ out: dict[str, dict[int, Image.Image]] = {}+ for side, sign in zip(repair_mod.OVERLAP_SIDES, (1, -1)):+ out[side] = {}+ for q in repair_mod.overlap_run(side, widest + 1, length):+ cut, other = start + q, start + q + sign * length+ if not skip <= other < len(files):+ continue+ dx, dy = (int(v) for v in shifts[other - skip] - shifts[cut - skip] + np.asarray(motion["shifts_xy"][q]))+ canvas = Image.new("RGBA", size)+ canvas.paste(Image.open(files[other]).convert("RGBA"), (left + dx, top + dy))+ out[side][q] = canvas+ return out++def _by_steps(cycle: dict[str, Any] | None, refused: ValueError | None, *, frames: list[Image.Image], analysis: dict[str, Any],D: np.ndarray, trajectory: np.ndarray, window: CutWindow, searched: dict[str, Any], detect: dict[str, Any],seam_max: float) -> dict[str, Any]:@@ -1541,17 +1621,27 @@ def run_loop(if repair != "off" and prof.gait:report_base["jump_repair"] = Nonefilmed = frames+ wrap = None+ if anchor == "motion-auto" and (cycle.get("steps") or {}).get("by") == "crossings" and wrap_on_cells(frames) > seam_max:+ # Two steps by the clip's own crossings, and a wrap the gate refuses: a part on a beat of its own closes on no+ # cut of two steps. The clip's frames a cycle from the cut's beside the wrap make it again (the overlapped wrap).+ wrap = {"others": cycle_apart(files, start=i, length=L, skip=skip, analysis=analysis, motion=motion,+ size=frames[0].size), "seam_max": seam_max}try:- frames, report_base["jump_repair"] = _repair_jumps(frames, interpolate, facing=facing, required=repair == "on",- screen=report_base["ghost_screen"])+ frames, report_base["jump_repair"], overlap = _repair_jumps(frames, interpolate, facing=facing, required=repair == "on",+ screen=report_base["ghost_screen"], wrap=wrap)except (ValueError, rife_mod.RifeUnavailable) as exc:fix = (f"install RIFE (`{rife_mod.INSTALL_COMMAND}`), or pass --repair auto to cut it as filmed with a warning"if isinstance(exc, rife_mod.RifeNotInstalled) else "fix RIFE, or pass --repair off to cut it as filmed")- error = (f"video-loop: {exc}; the loop has a frame to make (after a jump, for a part that rushes, or for a filmed "- f"ghost between two clean frames) — {fix} (docs/loop-repair.md)")+ error = (f"video-loop: {exc}; the loop has a frame to make (after a jump, for a part that rushes, at a wrap a part "+ f"on its own beat does not close, or for a filmed ghost between two clean frames) — {fix} (docs/loop-repair.md)")shutil.rmtree(out_dir / FALLBACK_FRAMES_DIR, ignore_errors=True)write_loop_report(target, {**report_base, "status": "failed", "error": error, "cycle": cycle})raise SystemExit(error) from exc+ if overlap is not None:+ report_base["wrap_overlap"] = overlap+ for line in _overlap_lines(overlap):+ print(f"video-loop: warning: {line}", file=sys.stderr)# What became of each filmed ghost: given way (the cell and what it shows now) or kept, no clean frame beside it.ghost_rounds = [r for r in report_base["jump_repair"]["rounds"] if r.get("why") == "ghost"]# A cell given way to the frame beside it is that frame shown twice in a row — a step of nothing and a step of two,@@ -1869,6 +1959,9 @@ def run(**kwargs: object) -> int:for key in ("ghost_given_way", "ghost_kept"):if payload.get(key):summary[key] = payload[key]+ if payload.get("wrap_overlap"):+ overlap = payload["wrap_overlap"]+ summary["wrap_overlap"] = {k: overlap[k] for k in ("seam", "frames", "side", "width", "seam_after", "skipped") if k in overlap}if payload.get("jump_repair", {}).get("replaced"):summary["jump_repair"] = {k: payload["jump_repair"][k] for k in ("replaced", "off_max_before", "off_max_after")}eased = [r for r in payload["jump_repair"]["rounds"] if r.get("why") == "ease" and r["outcome"] == "accepted"]diff --git a/sprite_gen/video/repair.py b/sprite_gen/video/repair.pyindex 9aab6ad..49229fe 100644--- a/sprite_gen/video/repair.py+++ b/sprite_gen/video/repair.py@@ -20,6 +20,11 @@ or four drawings each way, are made again by RIFE at times chosen so the part'sfaster, than they did in that stretch — read again on the frames made, which go the body's own way between theframes as filmed (docs/loop-repair.md section 2, "The eased retime").+A part on a beat of its own — a tail swinging round every 41 frames on a walk of 26 — closes on no cut of two steps,+and turns back at the wrap, where no retime reaches it. Before the rounds above, the frames beside such a cut's wrap+are made again between the clip's frames a cycle apart and the cut's own (`overlap_wrap`), so the wrap plays a step+the clip filmed (docs/loop-repair.md section 2, "The overlapped wrap").+Measured on the loop as it plays — cyclic, the last frame followed by the first — and insidethe union box of the body over the whole loop, the way the strip cells are cut, so a frame'sscore does not depend on where the canvas put the body.@@ -31,7 +36,7 @@ It picked the frames until the flow reading did, and names a fast swing a jump;from __future__ import annotations-from collections.abc import Iterable+from collections.abc import Callable, Iterablefrom typing import Anyfrom PIL import Image@@ -724,19 +729,265 @@ def _ease(frames: list[Image.Image], out: list[Image.Image], interpolate: Interpreturn out+# --- the overlapped wrap (docs/loop-repair.md section 2, "The overlapped wrap") ---------------------------------------++# A cut two steps long closes for the legs; a part on a beat of its own — a tail swinging every 41 frames on a walk of 26 —+# closes on no cut of two steps, and the seam gate refuses every one. The clip goes on past the cut: its frame a cycle+# after one of the cut's shows the legs at the same point of the walk and the part where the clip carried it. The frames+# beside the wrap are made again between the two, each at its own time, the weight going over from the clip's frame a+# cycle on, next to the wrap, to the cut's own frame away from it: the wrap plays a step the clip filmed, and the part goes+# over from the clip's way to the cut's over those frames, the legs where they were on both.+OVERLAP_WIDTHS = (4, 6) # frames beside the wrap made again, the fewer first+OVERLAP_SIDES = ("after", "before") # the clip's frames a cycle after the loop's first frames, or a cycle before its last+OVERLAP_APART = 1.0 # usual steps: a window whose frames a cycle apart differ by this much somewhere in the run is the part+OVERLAP_GRID = 20 # weights the search tries between a frame's two, 0 and 1 included: OVERLAP_GRID + 1 points+++def overlap_run(side: str, width: int, length: int) -> list[int]:+ """The loop frames the overlapped wrap makes on `side`, from the one next to the wrap away from it."""+ return list(range(width)) if side == "after" else [length - 1 - j for j in range(width)]+++def overlap_weight(p: int, width: int) -> float:+ """The ease in and out the weights search keeps nearest to: of the run's `p`-th frame in play order (1..width), the+ share of the early frame of its two — 0 the late, 1 the early — x²(3 − 2x), x = p/(width + 1)."""+ x = p / (width + 1)+ return x * x * (3 - 2 * x)+++def overlap_weights(step: np.ndarray, body: np.ndarray, rest: list[int], rest_bound: np.ndarray,+ body_bound: np.ndarray) -> dict[str, Any] | None:+ """The weights to make the overlapped wrap's frames at (docs/loop-repair.md section 2, "The overlapped wrap").++ The stretch's positions in play order, 0 and P (the frames either side of the run, kept), hold each two frames of one+ time: the late and the early, shown at weight u of the early (0..1, OVERLAP_GRID + 1 points), u 0 at position 0 and+ 1 at P. `step[p, i, k]` (windows, usual steps) is the step from position p at grid point i to position p + 1 at grid+ point k, read on the line between the two frames' states, and `body[p, i, k]` (measures) how far each body measure+ moves over it.++ Of the weightings that never go back toward the late frame and keep every window of the `rest` within `rest_bound`+ and every body measure within `body_bound` in each step, the one nearest the ease `overlap_weight` (the least sum of+ squares): the part goes over from the late way to the early in an ease in and out, moved off it only as far as the+ rest of the body needs. None where no weighting keeps in bound."""+ P, G = step.shape[0], step.shape[1]+ ok = np.triu(np.ones((G, G), dtype=bool))[None] & np.ones((P, 1, 1), dtype=bool)+ if rest:+ ok &= np.all(step[..., rest] <= rest_bound + 1e-9, axis=-1)+ ok &= np.all(body <= body_bound + 1e-9, axis=-1)+ own = np.array([overlap_weight(p, P - 1) * (G - 1) for p in range(1, P)])+ cost = np.where(np.arange(G) == 0, 0.0, np.inf) # position 0: the late frame+ back = []+ for p in range(P):+ c = np.where(ok[p], cost[:, None], np.inf) # (i, k)+ arg = c.argmin(axis=0)+ cost = c[arg, np.arange(G)] + ((np.arange(G) - own[p]) ** 2.0 if p < P - 1 else 0.0)+ back.append(arg)+ if not np.isfinite(cost[-1]):+ return None+ path = [G - 1] # position P: the early frame+ for arg in reversed(back):+ path.append(int(arg[path[-1]]))+ weights = [k / (G - 1) for k in reversed(path[:-1])][:-1]+ return {"weights": weights, "eased": [round(float(v / (G - 1)), 4) for v in own],+ "off_ease": round(float(np.abs(np.array(weights) - own / (G - 1)).max()), 4)}+++def overlap_wrap(frames: list[Image.Image], others: dict[str, dict[int, Image.Image]], interpolate: Interpolate | None, *,+ seam: Callable[[list[Image.Image]], float], steps: Callable[[list[Image.Image]], dict[str, Any]],+ seam_max: float, reads: bool) -> tuple[list[Image.Image], dict[str, Any]]:+ """The overlapped wrap (docs/loop-repair.md section 2, "The overlapped wrap") of a loop cut two steps long whose wrap+ the seam gate refuses (`seam`: the gate's reading of a loop, over `seam_max`).++ `others[side][q]` is the clip's frame a cycle from loop frame q — a cycle after it for the first frames ("after"), a+ cycle before it for the last ("before"), one frame past the widest run — placed where frame q is. One run is tried a+ width (OVERLAP_WIDTHS, the fewer first), on the side whose frames a cycle apart differ least from the loop's over it,+ and never past MADE_SHARE of the loop. The stretch runs from the frame before the run to the frame after it, the wrap+ inside; each of its frames has two of one time, the late (the clip's frame a cycle after, or the loop's own before the+ wrap) and the early (the loop's own, or the clip's frame a cycle before), and two runs the clip filmed pass through+ them: the stretch as the loop plays it, and the frames a cycle apart as the clip filmed them, from the loop's own frame+ on the far side of the wrap. Each frame of the run is shown at the weight of its early frame `overlap_weights` chooses+ on the line between the two frames' states — an ease in and out, moved off it only as far as the rest of the body and+ the body's measures need — as RIFE's frame between them (`interpolate`), or either one whole at 0 or 1.++ The run is taken only where, on the frames made: no frame has an interpolation fault; the gate passes it; the loop+ played round still covers two steps (`steps`, the clip's crossings read on it); no body measure moves in a step more+ than a pixel past the largest step either run takes there (`drift_points`); the part — the windows where the frames a+ cycle apart differ by OVERLAP_APART usual steps or more — steps no further than it did over the stretch (`no gain`);+ no window of the rest of the body steps further than its own largest step over either run (`the rest faster`); no+ rush the loop did not have is read across the stretch (`a new rush`); and no frame of it is read further off the flow+ than before and at or over the limit. The first run taken ends the search. Returns the loop and the record (`frames`:+ the frames shown otherwise than as cut)."""+ n = len(frames)+ record: dict[str, Any] = {"seam": round(float(seam(frames)), 4), "seam_max": seam_max, "widths": list(OVERLAP_WIDTHS),+ "tries": [], "frames": [], "calls": 0}+ box = union_box(frames)+ flow = _flow(frames, box)+ if flow["hold"] > 1:+ record["skipped"] = f"drawn on {flow['hold']}s: a frame made beside the wrap would split a drawing"+ return frames, record+ images = {**{("loop", q): f for q, f in enumerate(frames)}, **{(side, q): f for side in OVERLAP_SIDES for q, f in others.get(side, {}).items()}}+ keys = list(images)+ at = {key: k for k, key in enumerate(keys)}+ states = flow_states([images[key] for key in keys], box)+ points = {key: drift_points(images[key]) for key in keys}+ windows, scale = flow["windows"], flow["scale"]++ def between(a: np.ndarray, b: np.ndarray) -> np.ndarray:+ return np.array([np.linalg.norm(b[y0:y1, x0:x1] - a[y0:y1, x0:x1]) for y0, y1, x0, x1 in windows])++ def apart(side: str, q: int) -> np.ndarray:+ return between(states[at[side, q]], states[at["loop", q]]) / scale++ widths = [width for width in OVERLAP_WIDTHS if width + 2 <= n and width <= int(n * MADE_SHARE)]+ if not widths:+ record["skipped"] = f"a loop of {n} frames has no run beside its wrap that leaves half of it filmed"+ return frames, record+ plans = []+ for width in widths:+ sides = [(float(sum(apart(side, q).max() for q in overlap_run(side, width, n))), side) for side in OVERLAP_SIDES+ if all(q in others.get(side, {}) for q in overlap_run(side, width + 1, n))]+ if sides:+ plans.append((width, min(sides)[1]))+ if not plans:+ record["skipped"] = "the clip holds no frames a cycle from the loop's beside the wrap"+ return frames, record+ if interpolate is None:+ record["skipped"] = "no interpolator"+ return frames, record+ rel = flow["step"] / scale+ off_before = (flow["off"] / scale).max(axis=1)+ grid = np.linspace(0.0, 1.0, OVERLAP_GRID + 1)++ def touching(fl: dict[str, Any], steps_at: set[int]) -> list[dict[str, Any]]:+ return [x for x in _surges(fl) if {(x["d"] + i) % n for i in range(x["r"])} & steps_at]++ for width, side in plans:+ run = overlap_run(side, width, n)+ # in play order, from the frame before the run to the frame after it, the wrap inside+ first = 0 if side == "after" else n - width+ stretch = [(first - 1 + i) % n for i in range(width + 2)]+ inner = set(stretch[1:-1])+ late = [(side, q) if side == "after" and q in inner else ("loop", q) for q in stretch]+ early = [("before", q) if side == "before" and q in inner else ("loop", q) for q in stretch]+ # the same stretch as the clip filmed it a cycle apart: the loop's own frame on the far side of the wrap+ apart_at = set(overlap_run(side, width + 1, n))+ filmed = [(side, q) if q in apart_at else ("loop", q) for q in stretch]+ played = [("loop", q) for q in stretch]+ diff = np.stack([apart(side, q) for q in run]).max(axis=0)+ part = [w for w in range(len(windows)) if diff[w] >= OVERLAP_APART]+ rest = [w for w in range(len(windows)) if w not in part]+ steps_at = stretch[:-1]++ def run_steps(run_keys: list[tuple[str, int]]) -> np.ndarray:+ return np.array([between(states[at[a]], states[at[b]]) for a, b in zip(run_keys, run_keys[1:])])++ filmed_step = run_steps(filmed)+ step_bound = np.maximum(flow["step"][steps_at], filmed_step).max(axis=0) / scale+ bound = np.max([np.abs(np.diff(np.array([points[key] for key in keys_]), axis=0)).max(axis=0) for keys_ in (played, filmed)], axis=0)+ # the line between each position's two frames, a step at a time, on the weight grid (`overlap_weights`)+ L_ = [states[at[key]] for key in late]+ E_ = [states[at[key]] for key in early]+ P = len(stretch) - 1+ line = np.zeros((P, len(grid), len(grid), len(windows)))+ moves = np.zeros((P, len(grid), len(grid), len(DRIFT_MEASURES)))+ for p in range(P):+ for w, (y0, y1, x0, x1) in enumerate(windows):+ a = (L_[p + 1] - L_[p])[y0:y1, x0:x1].ravel()+ b = (E_[p + 1] - L_[p + 1])[y0:y1, x0:x1].ravel()+ c = (E_[p] - L_[p])[y0:y1, x0:x1].ravel()+ u, v = grid[:, None], grid[None, :]+ sq = a @ a + v * v * (b @ b) + u * u * (c @ c) + 2 * v * (a @ b) - 2 * u * (a @ c) - 2 * u * v * (b @ c)+ line[p, :, :, w] = np.sqrt(np.maximum(sq, 0.0)) / scale[w]+ dl, de = points[late[p]], points[early[p]]+ nl, ne = points[late[p + 1]], points[early[p + 1]]+ moves[p] = np.abs((nl + grid[None, :, None] * (ne - nl)) - (dl + grid[:, None, None] * (de - dl)))+ chosen = overlap_weights(line, moves, rest, step_bound[rest], bound + DRIFT_PIXEL - EASE_MARGIN)+ tried: dict[str, Any] = {"side": side, "width": width, "frames": run, "stretch": stretch,+ "part": {"windows": [[round(v, 4) for v in flow["fractions"][w]] for w in part],+ "apart": [round(float(diff[w]), 4) for w in part]}}+ if chosen is None:+ tried.update(outcome="rejected", rejected_for=["no weighting keeps the body in bound"])+ record["tries"].append(tried)+ continue+ new = list(frames)+ made_rows = []+ for p, u in enumerate(chosen["weights"], start=1):+ q = stretch[p]+ a, b = images[late[p]], images[early[p]]+ row: dict[str, Any] = {"frame": q, "t": round(u, 4), "proposal": None, "faults": [], "look": []}+ if u <= 0.0:+ frame = a+ elif u >= 1.0:+ frame = b+ else:+ frame = interpolate(a, b, u)+ record["calls"] += 1+ if frame.mode != "RGBA" or frame.size != b.size:+ raise ValueError("repair interpolator must return an RGBA frame of the original size")+ measure = {**smear(frame, a, b), "outline_shift": outline_shift(frame, a, b),+ "ghost": ghost(frame) if reads else None, "crossfade": crossfade(frame, a, b, u)}+ row.update(proposal=measure, faults=faults(measure) + ([] if frame.getchannel("A").getbbox() else ["empty"]),+ look=looks(measure))+ made_rows.append(row)+ new[q] = frame+ shown = [q for q in run if new[q] is not frames[q]]+ after_flow = _flow(new, box, (flow["runs"], flow["hold"]))+ rel_after = after_flow["step"] / scale+ moved = np.abs(np.diff(np.array([drift_points(new[q]) for q in stretch]), axis=0)).max(axis=0)+ over = bool(np.any(~np.isfinite(moved)) or np.any(moved > bound + DRIFT_PIXEL + 1e-9))+ part_before = float(rel[steps_at][:, part].max()) if part else 0.0+ part_after = float(rel_after[steps_at][:, part].max()) if part else 0.0+ rest_after = rel_after[steps_at][:, rest].max(axis=0) if rest else np.zeros(0)+ faster = [w for w, a in zip(rest, rest_after) if a > step_bound[w] + 1e-9]+ off_after = (after_flow["off"] / scale).max(axis=1)+ had = {(x["d"], x["r"]) for x in touching(flow, set(steps_at))}+ rushes = [_said_surge(x, after_flow) for x in touching(after_flow, set(steps_at)) if (x["d"], x["r"]) not in had]+ seam_after = float(seam(new))+ counted = steps(new)+ wrong = sorted({f for r in made_rows for f in r["faults"]})+ rejected = ((["fault"] if wrong else []) + (["seam"] if seam_after > seam_max else [])+ + (["steps"] if counted.get("verdict") != "two" else []) + (["drift"] if over else [])+ + (["no gain"] if part and part_after >= part_before else [])+ + (["off the flow"] if off_after[stretch].max() >= FLOW_LIMIT and off_after[stretch].max() > off_before[stretch].max() else [])+ + (["a new rush"] if rushes else []) + (["the rest faster"] if faster else []))+ tried.update({+ "eased": chosen["eased"], "off_ease": chosen["off_ease"], "made": made_rows, "shown": shown, "seam": round(seam_after, 4),+ "steps": {key: counted.get(key) for key in ("verdict", "k", "ring", "why")}})+ tried["part"].update(step_max_before=round(part_before, 4), step_max_after=round(part_after, 4))+ tried.update({+ "rest": {"windows": [[round(v, 4) for v in flow["fractions"][w]] for w in rest],+ "step_max_bound": [round(float(step_bound[w]), 4) for w in rest],+ "step_max_after": [round(float(v), 4) for v in rest_after],+ "faster": [[round(v, 4) for v in flow["fractions"][w]] for w in faster]},+ "drift": {"measures": list(DRIFT_MEASURES), "bound": [round(float(v), 3) for v in bound],+ "moved": [round(float(v), 3) for v in moved], "over": over},+ "off_max": {"before": round(float(off_before[stretch].max()), 4), "after": round(float(off_after[stretch].max()), 4)},+ "rushes": rushes, "faults": wrong, "look": sorted({w for r in made_rows for w in r["look"]}),+ "outcome": "rejected" if rejected else "accepted"})+ if rejected:+ tried["rejected_for"] = rejected+ record["tries"].append(tried)+ if not rejected:+ record.update(side=side, width=width, frames=shown, seam_after=round(seam_after, 4))+ return new, record+ return frames, record++def _give_way(frames: list[Image.Image], j: int, ghosts: dict[int, float], interpolate: Interpolate | None,- out: list[Image.Image]) -> dict[str, Any]:+ out: list[Image.Image], overlapped: set[int] = frozenset()) -> dict[str, Any]:"""The round of a filmed ghost at `j` (`why` ghost), its cell replaced in `out` where it gives way.Where both frames beside it are clean, RIFE's frame half way between them is proposed and takenwithout a fault (`taken` rife); else, or where there is no interpolator, the clean frame beside itthe cycle alignment would take (`interpolation_quality.clean_beside`, `taken` {source}); where- both are filmed ghosts too it is kept (`taken` None)."""+ both are filmed ghosts too it is kept (`taken` None). A frame the overlapped wrap made (`overlapped`) is not+ filmed: nothing is made from it, and a ghost beside it gives way to the filmed frame on its other side."""n = len(frames)a, b = (j - 1) % n, (j + 1) % nmeasure: dict[str, Any] | None = Nonewrong: list[str] = []no_proposal = ("a frame beside it is a filmed ghost too" if a in ghosts or b in ghosts+ else "a frame beside it was made at the wrap" if a in overlapped or b in overlappedelse "no interpolator" if interpolate is None else None)if no_proposal is None:assert interpolate is not None@@ -748,7 +999,7 @@ def _give_way(frames: list[Image.Image], j: int, ghosts: dict[int, float], interwrong = faults(measure)if not made.getchannel("A").getbbox():wrong.append("empty")- beside = clean_beside(j, ghosts, n)+ beside = clean_beside(j, set(ghosts) | overlapped, n)taken: Any = "rife" if measure is not None and not wrong else {"source": beside} if beside is not None else Noneif taken == "rife":out[j] = made@@ -764,7 +1015,8 @@ def _give_way(frames: list[Image.Image], j: int, ghosts: dict[int, float], interdef repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *, facing: str = "right",limit: float = FLOW_LIMIT, span: int = FLOW_SPAN, max_frames: int = MAX_REPAIRS,- screen: dict[str, Any] | None = None, ease: bool = True) -> tuple[list[Image.Image], dict[str, Any]]:+ screen: dict[str, Any] | None = None, ease: bool = True,+ overlapped: Iterable[int] = ()) -> tuple[list[Image.Image], dict[str, Any]]:"""Replace each filmed ghost the screen names, then take out each group of frames off the loop's flow andmake it again with RIFE between the frames either side, worst first (docs/loop-repair.md section 2).@@ -809,16 +1061,19 @@ def repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *,record.update(stopped="loop shorter than 4 frames", off_max_after=record["off_max_before"])return out, recordreplaced: list[int] = []- ghosts = {g["frame"]: g["ghost"] for g in screen["ghosts"]}+ overlapped = set(overlapped)+ if overlapped:+ record["overlapped"] = sorted(overlapped)+ ghosts = {g["frame"]: g["ghost"] for g in screen["ghosts"] if g["frame"] not in overlapped}for j in sorted(ghosts):- round_ = _give_way(frames, j, ghosts, interpolate, out)+ round_ = _give_way(frames, j, ghosts, interpolate, out, overlapped)record["rounds"].append(round_)if "replaced" in round_:replaced.append(j)refused: list[int] = []calls = 0while True:- plan = flow_targets(out, limit=limit, span=span, box=box, fixed=[*replaced, *refused])+ plan = flow_targets(out, limit=limit, span=span, box=box, fixed=[*replaced, *refused, *overlapped])record.setdefault("hold", plan["hold"])record.setdefault("drawings", plan["drawings"])if not plan["groups"]:@@ -866,8 +1121,8 @@ def repair_jumps(frames: list[Image.Image], interpolate: Interpolate | None, *,record.update(replaced=replaced, refused=refused, attempts=calls, stopped=stopped, unfixed=plan["unfixed"],off_max_after=plan["off_max_before"])given = {j for j in replaced if j in ghosts}- blocked = given | set(refused)- out = _ease(frames, out, interpolate, box, blocked, set(replaced) - given, set(replaced), reads, record, ease)+ blocked = given | set(refused) | overlapped+ out = _ease(frames, out, interpolate, box, blocked, set(replaced) - given, set(replaced) | overlapped, reads, record, ease)eased = [r for r in record["rounds"] if r["why"] == "ease" and r["outcome"] == "accepted"]if eased:record["replaced"] = [*replaced, *(t for t in eased[0]["replaced"] if t not in replaced)]diff --git a/tests/video/test_loop_overlap.py b/tests/video/test_loop_overlap.pynew file mode 100644index 0000000..b81c707--- /dev/null+++ b/tests/video/test_loop_overlap.py@@ -0,0 +1,206 @@+# SPDX-License-Identifier: Apache-2.0+"""The overlapped wrap (docs/loop-repair.md section 2, "The overlapped wrap"): a walk cut two steps long closes for the+legs, and a tail on a beat of its own closes on no cut of two steps, so the seam gate refuses every one. The clip goes on+past the cut; the frames beside the wrap are made again between the clip's frame a cycle apart and the cut's own, and+the loop passes the gate, still two steps, its body standing where it stood.++The walker is seen from the front, 26 frames a cycle, its tail swinging round on a beat of 41 frames, in a 73-frame clip.+The interpolator is a stand-in that draws the walker between two frames the way an optical-flow in-between does: the body+at the time between them, the tail's tip on the line between its two places. Synthetic frames only."""++from __future__ import annotations++import hashlib+import json+import math+from pathlib import Path++import numpy as np+import pytest+from PIL import Image, ImageDraw++from sprite_gen.video import loop, repair+from tests.video.test_loop_repair import _no_rife++FRAMES, CYCLE, TAIL = 73, 26, 41+REACH, LIFT, TUFT = 40, 14, 18 # the tail's tip swings round this far sideways and up and down, px; its tuft's radius+++def _tip(k: float) -> tuple[float, float]:+ a = 2 * math.pi * k / TAIL+ return REACH * math.sin(a), LIFT * math.cos(a)+++def _walker(body: float, tip: tuple[float, float]) -> Image.Image:+ """The walker at `body` frames into its walk, its tail's tip at `tip`: legs side by side, the nearer foot drawn lower+ each step, arms swinging opposite, and a tail from the hips ending in a tuft."""+ ph = 2 * math.pi * (body + 5) / CYCLE+ im = Image.new("RGBA", (300, 240))+ d = ImageDraw.Draw(im)+ cx, y = 150, 40 + round(2 * math.cos(2 * ph))+ drop = round(6 * math.sin(ph))+ tx, ty = tip+ d.line((cx, y + 60, cx + tx, y + 72 + ty), fill=(180, 120, 40, 255), width=18)+ d.ellipse((cx + tx - TUFT, y + 72 + ty - TUFT, cx + tx + TUFT, y + 72 + ty + TUFT), fill=(200, 140, 60, 255))+ d.rectangle((cx - 12, y + 80, cx - 1, y + 120 + drop), fill=(60, 60, 70, 255))+ d.rectangle((cx, y + 80, cx + 11, y + 120 - drop), fill=(60, 60, 70, 255))+ d.rectangle((cx - 16, y + 30, cx + 16, y + 84), fill=(200, 80, 80, 255))+ d.rectangle((cx - 12, y, cx + 12, y + 28), fill=(230, 200, 170, 255))+ sw = round(8 * math.sin(ph))+ d.rectangle((cx - 22, y + 34 + sw, cx - 16, y + 70 + sw), fill=(225, 225, 225, 255))+ d.rectangle((cx + 16, y + 34 - sw, cx + 22, y + 70 - sw), fill=(225, 225, 225, 255))+ d.point((cx - 6, y + 40 + round(body) % 11), fill=(250, 250, 250, 255)) # no two frames the same picture+ return im+++def _filmed(k: int) -> Image.Image:+ return _walker(k, _tip(k))+++def _key(frame: Image.Image) -> str:+ """A frame's picture wherever the canvas puts it: its pixels inside its own box."""+ return hashlib.sha256(frame.crop(frame.getbbox()).tobytes()).hexdigest()+++class Between:+ """A stand-in RIFE for the walker: between two frames of the clip — wherever the loop placed them — at t, the body at+ the time t of the way from the first to the second (the shorter way round its cycle), the tail's tip t of the way along+ the line between its two places, placed where the second frame is. Records each call as (a, b, t) by clip frame, and+ refuses a frame the clip did not film."""++ def __init__(self, frames: int = FRAMES):+ self.of = {_key(_filmed(k)): k for k in range(frames)}+ self.calls: list[tuple[int, int, float]] = []++ def __call__(self, a: Image.Image, b: Image.Image, t: float) -> Image.Image:+ ka, kb = self.of[_key(a)], self.of[_key(b)] # KeyError: made from a frame the clip did not film+ self.calls.append((ka, kb, round(t, 4)))+ apart = (ka - kb + CYCLE / 2) % CYCLE - CYCLE / 2+ (xa, ya), (xb, yb) = _tip(ka), _tip(kb)+ drawn = _walker(kb + (1 - t) * apart, (xa + t * (xb - xa), ya + t * (yb - ya)))+ # where the loop put b: its box against the box of the frame the clip filmed+ dx, dy = (p - q for p, q in zip(b.getbbox()[:2], _filmed(kb).getbbox()[:2]))+ out = Image.new("RGBA", b.size)+ out.paste(drawn, (dx, dy))+ return out+++def _keyed(root: Path) -> Path:+ keyed = root / "keyed"+ keyed.mkdir(parents=True)+ for k in range(FRAMES):+ _filmed(k).save(keyed / f"{k:03}.png")+ return keyed+++def _cut(root: Path, keyed: Path, *, repair_mode: str = "auto", interpolate=None) -> dict:+ """The walker cut as `video-loop --state walk --anchor motion-auto` cuts it; the report, whether it passed or not."""+ try:+ loop.run_loop(keyed, root / "out", fps=24.0, state="walk", min_len=None, max_len=None, n_out=None,+ seam_max=loop.SEAM_RATIO_MAX, name="loop", report_path=root / "loop.json", anchor="motion-auto",+ repair=repair_mode, interpolate=interpolate)+ except SystemExit:+ pass+ return json.loads((root / "loop.json").read_text())+++def test_a_tail_on_its_own_beat_is_closed_at_the_wrap_with_the_clips_frames_a_cycle_apart(tmp_path):+ keyed = _keyed(tmp_path)+ between = Between()+ report = _cut(tmp_path, keyed, interpolate=between)+ c = report["cycle"]+ s, L = c["start"], c["length"]+ assert c["steps"] == {"count": 2, "by": "crossings"} and abs(L - CYCLE) <= 1+ overlap = report["wrap_overlap"]+ assert overlap["seam"] > loop.SEAM_RATIO_MAX # the cut as filmed: the gate refuses it+ assert report["status"] == "passed" and report["resampled_seam_ratio"] <= loop.SEAM_RATIO_MAX+ taken = overlap["tries"][-1]+ assert taken["outcome"] == "accepted" and (overlap["width"], overlap["seam_after"]) == (taken["width"], taken["seam"])+ assert taken["seam"] <= loop.SEAM_RATIO_MAX and taken["steps"]["verdict"] == "two" and not taken["drift"]["over"]+ assert taken["rest"]["faster"] == [] and taken["faults"] == [] and taken["rushes"] == []+ assert taken["part"]["step_max_after"] < taken["part"]["step_max_before"]+ # each frame of a run shown between the clip's frame a cycle apart and the cut's own, at the weight the search chose,+ # the weights never going back toward the frame a cycle apart; RIFE asked only for a frame between the two+ assert overlap["calls"] == len(between.calls) == sum(0 < r["t"] < 1 for t in overlap["tries"] for r in t.get("made", []))+ calls = list(between.calls)+ for tried in overlap["tries"]:+ sign = 1 if tried["side"] == "after" else -1+ assert tried["frames"] == repair.overlap_run(tried["side"], tried["width"], L)+ weights = [r["t"] for r in tried.get("made", [])]+ assert weights == sorted(weights)+ for r in tried.get("made", []):+ q = r["frame"]+ late, early = (s + q + L, s + q) if sign > 0 else (s + q, s + q - L)+ if 0 < r["t"] < 1:+ ka, kb, t = calls.pop(0)+ assert (ka, kb) == (late, early) and abs(t - r["t"]) < 1e-3+ run = overlap["frames"]+ # every other frame is the clip's own, and nothing else was made+ assert report["jump_repair"]["overlapped"] == run and report["jump_repair"]["replaced"] == []+ cells = sorted((tmp_path / "out" / "cycle").glob("frame-*.png"))+ assert len(cells) == L+ filmed = {_key(_filmed(k)): k for k in range(FRAMES)}+ assert [filmed.get(_key(Image.open(p).convert("RGBA"))) for p in cells] == [None if q in run else s + q for q in range(L)]+++def test_the_cut_as_filmed_is_refused_at_the_seam_with_the_repair_off(tmp_path):+ report = _cut(tmp_path, _keyed(tmp_path), repair_mode="off")+ assert report["status"] == "failed" and report["error"].startswith("video-loop: loop seam ratio")+ assert report["cycle"]["steps"] == {"count": 2, "by": "crossings"} and "wrap_overlap" not in report+++def test_without_rife_the_wrap_is_cut_as_filmed_and_says_so(tmp_path, monkeypatch, capsys):+ _no_rife(monkeypatch, tmp_path)+ report = _cut(tmp_path, _keyed(tmp_path))+ assert report["status"] == "failed" and report["error"].startswith("video-loop: loop seam ratio")+ overlap = report["wrap_overlap"]+ assert overlap["frames"] == [] and overlap["install"] == "sprite-gen rife install" and overlap["calls"] == 0+ assert "the overlapped wrap was not made" in capsys.readouterr().err+++def test_without_rife_and_the_repair_on_the_loop_fails_asking_for_it(tmp_path, monkeypatch):+ _no_rife(monkeypatch, tmp_path)+ report = _cut(tmp_path, _keyed(tmp_path), repair_mode="on")+ assert report["status"] == "failed" and "at a wrap a part on its own beat does not close" in report["error"]+ assert "sprite-gen rife install" in report["error"]+++def test_a_made_frame_with_a_fault_refuses_the_run_and_the_wrap_is_cut_as_filmed(tmp_path, capsys):+ """A stand-in that returns nothing: every run is refused (`fault`), each width tried once, and the gate refuses the+ cut as filmed."""+ calls = []++ def empty(a: Image.Image, b: Image.Image, t: float) -> Image.Image:+ calls.append(t)+ return Image.new("RGBA", b.size)++ report = _cut(tmp_path, _keyed(tmp_path), interpolate=empty)+ assert report["status"] == "failed" and report["error"].startswith("video-loop: loop seam ratio")+ overlap = report["wrap_overlap"]+ assert overlap["frames"] == [] and [t["width"] for t in overlap["tries"]] == list(repair.OVERLAP_WIDTHS)+ assert all(t["outcome"] == "rejected" and "fault" in t["rejected_for"] for t in overlap["tries"])+ # the overlapped wrap's calls first; the jump repair then reads the cut as filmed, its wrap off the flow+ made = [r["t"] for t in overlap["tries"] for r in t["made"] if 0 < r["t"] < 1]+ assert overlap["calls"] == len(made) and calls[:len(made)] == pytest.approx(made, abs=1e-3)+ assert "the overlapped wrap was refused" in capsys.readouterr().err+++def test_the_weights_ease_from_the_late_frame_to_the_early_off_the_ease_only_where_the_body_needs_it():+ """`overlap_weights` on lines drawn by hand: three frames between a late frame and an early one. Unbounded, they are+ the ease in and out (`overlap_weight`) on the grid; a window of the rest of the body that moves with the share taken+ in a step, bounded in the second step to a tenth, moves the weights off the ease there and no further; a body measure+ that may not move leaves no weighting."""+ P, G = 4, repair.OVERLAP_GRID + 1+ grid = np.linspace(0.0, 1.0, G)+ step = np.zeros((P, G, G, 2))+ step[..., 0] = 1.0 + 4.0 * np.abs(grid[None, :] - grid[:, None]) # the part, unbounded+ body = np.zeros((P, G, G, 3))+ eased = [round(repair.overlap_weight(p, P - 1) * (G - 1)) / (G - 1) for p in range(1, P)]+ free = repair.overlap_weights(step, body, [], np.zeros(0), np.ones(3))+ assert free is not None and free["weights"] == pytest.approx(eased) and free["off_ease"] < 0.5 / (G - 1) + 1e-9+ step[1, ..., 1] = np.abs(grid[None, :] - grid[:, None]) # the rest of the body, in the second step only+ held = repair.overlap_weights(step, body, [1], np.array([0.1]), np.ones(3))+ assert held is not None and held["weights"][1] - held["weights"][0] <= 0.1 + 1e-9+ assert held["weights"] == sorted(held["weights"]) and held["off_ease"] > free["off_ease"]+ body[...] = 1.0+ assert repair.overlap_weights(step, body, [], np.zeros(0), np.full(3, 0.5)) is None