Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
Seamless Video Loop (ambient clip → infinite loop)
Owns: Making a non-looping ambient clip loop forever (RIFE seam bridge) — a different job from pipeline B · Index: docs/README.md
Turn a non-looping AI-generated ambient video clip (an sprite-gen gen video, a
Seedance/Veo/Kling i2v clip, a title/hero backdrop) into a clip that loops forever with
no visible seam — for a title-screen background, a hero loop, a store-page ambient shot.
Not to be confused with frame-interpolation.md: that is a
generative in-between for sprite frames (identity-preserving, RIFE deliberately
retired there). This doc is RIFE optical-flow interpolation used to bridge a video
loop seam — a different job, where RIFE's fluid morph is exactly right.
Why AI clips don't just loop
An image-to-video model paints continuous, non-repeating motion. The last frame is never the first frame, and there is usually no pair of near-identical frames anywhere. So a raw loop cuts hard from the end state back to the start state and the eye catches a "pop".
Common approaches that fail this contract:
- Crossfade the seam — the dissolve is visible as a fade in/out. Rejected.
- Ping-pong / reverse (
forward + reverse) — mathematically seamless, but the motion plays backwards for half the loop (embers fall, honey un-drips). Rejected. - A model-native first/last-frame loop — availability and endpoint guarantees vary by provider, and equal endpoint images do not guarantee equal motion. Treat it as an input clip, not proof that the seam is continuous.
- Trim to the best single-frame match — halves the pop but still pops, because matching two frames' position ignores their motion (see next section).
The pipeline
Two stages: find the best forward loop segment numerically, then bridge its seam with RIFE so the wrap is continuous motion, not a cut or a fade.
1. Find the loop segment — match flow, not two frames
Extract frames (downscaled for speed), then over candidate cuts (i, j) minimise a cost
that matches both position and motion flow at the seam. Position-only is not enough:
these clips flicker, so velocity[t] = |frame[t+1] - frame[t]| alternates every frame;
a cut that matches position but lands on the opposite flicker phase jumps in speed.
cost(i, j) = Σ_k w_k · D(frame[i+k], frame[j+k]) # windowed position match (flow shape)
+ β · Σ_k w_k · |vel[i+k] - vel[j+k]| # windowed velocity match (phase/speed)
w_k= small Gaussian window (radius ~3 frames) so the trajectory around the seam matches, not just the boundary frame.β ≈ 5. Without it the cut lands on a velocity mismatch (measured on t06:|Δvel|0.28); with it the velocities align (|Δvel|0.04) with the same position quality.- Constrain
j - i ≥ minlen(~2.5 s) so the loop isn't too short.
The chosen segment is frames[i .. j], trimmed hard from the full clip (ffmpeg -ss i/fps -t (j-i)/fps). Play it forward; the only remaining discontinuity is the wrap j → i.
2. Bridge the seam with RIFE
Interpolate the wrap frame_j → frame_i with rife-ncnn-vulkan (neural optical-flow
frame interpolation) and append those in-between frames after the segment. The loop then
plays forward, morphs j → i over the bridge as continuous motion, and wraps to i.
No fade, no reverse.
rife-ncnn-vulkan interpolates one midpoint per call, so recurse for N frames (depth 3 → 7 frames ≈ 0.29 s at 24 fps):
R="rife-ncnn-vulkan"; J=frame_j.png; I=frame_i.png
"$R" -0 "$J" -1 "$I" -o m.png # 50%
"$R" -0 "$J" -1 m.png -o q1.png ; "$R" -0 m.png -1 "$I" -o q3.png # 25% / 75%
"$R" -0 "$J" -1 q1.png -o e1.png ; "$R" -0 q1.png -1 m.png -o e2.png
"$R" -0 m.png -1 q3.png -o e3.png ; "$R" -0 q3.png -1 "$I" -o e4.png
# append after the segment frames, in order: e1 q1 e2 m e3 q3 e4, then re-encode at the clip fps
frame_i itself is the loop's first frame, so the bridge stops at e4 (just before i) —
the wrap e4 → i is one more small RIFE step.
Verify with numbers, not one screenshot
A single still cannot tell a playing loop from a paused first frame. Judge numerically:
- Loop seam: output
first-vs-lastframe mean-abs-diff should drop (t06: hard cut 1.65 → RIFE bridge 0.87). Frame-diff alone is necessary, not sufficient — the flow-match in stage 1 is what removes the velocity jump the eye actually reads. - Motion present:
first-vs-middiff should stay high (a paused clip reads ~0). - To confirm a built player is actually playing (not showing a static fallback of the same scene), capture two frames ~1.5 s apart and diff them — motion ⇒ the video plays.
Tools
- rife-ncnn-vulkan (nihui, prebuilt macOS release) — runs on Apple Silicon via
Vulkan/MoltenVK (verified on M4 Max). Ships model sets (
rife-v4,rife-anime, …); the default is fine for fluid ambient motion. - ffmpeg — frame extract, trim, and re-encode.
minterpolate(its built-in motion-compensated interpolation) is a fallback but is block-based and fails on isolated 2-frame pairs; prefer RIFE.
Runtime note
For a game/app, ship the looped clips as the player's native codec and let the player loop
(Unity VideoPlayer.isLooping = true, HTML <video loop>). Because the seam is already
seamless in the file, the engine's plain loop needs no crossfade.
Related
- docs/README.md — documentation index