Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

Add rotate/flip, video PiP, chroma key, Ken Burns, background, reverse, stabilize, sequence (#45)

Shogo Kajiyama2b2caf238567
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 240608b..1849e27 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,64 @@
## Unreleased
+- **`fit.py --rotate`/`--flip`.** New rotate 90/180/270 (clockwise; 90/270 swap width and
+ height) and horizontal/vertical flip flags -- distinct from the rotation *metadata* fit.py
+ already reads to size a source correctly, which is never altered by these. Verified against
+ a real red/left, blue/right test fixture: `--flip h` swaps the two halves and `--rotate 90`
+ rotates the left column into the top row, both confirmed pixel-exact, not just by output
+ dimensions.
+- **`overlay.py --video`: video-on-video picture-in-picture.** `overlay.py` could only
+ composite a still image or text onto a video; there was no way to place a second *video* as
+ a layer. `--video CLIP` composites it with the same `--position`/`--scale`/`--opacity`/
+ `--start`/`--end` knobs `--image` already has (`scale2ref`-style scale + `format=yuva420p` +
+ `colorchannelmixer` for opacity + `overlay` with a timeline `enable`). Only the main input's
+ audio is kept; the PiP layer's own audio is dropped -- mixing two audio tracks is a job for
+ `audio.py`. Verified with a real composite: a blue clip lands at the exact expected
+ bottom-right pixel position, the rest of the frame is unaffected.
+- **`overlay.py --chromakey`: green-screen compositing.** With `--video`, `--chromakey COLOR`
+ (plus `--chromakey-similarity`/`--chromakey-blend`) keys that colour transparent before
+ compositing, for green-screen foreground-over-background work. Verified: a green background
+ behind a white square is correctly replaced by the destination clip's colour, the white
+ square is untouched.
+- **`insert.py --zoom`/`--pan`: Ken Burns effect.** A slow linear zoom in/out (`--zoom-amount`
+ sets the end/start factor, default 1.3) and, with `--zoom`, a pan across the image while
+ zoomed, built from typed enums into a generated `zoompan` expression -- never a raw
+ expression from the caller. Verified against a real image with a centred marker at a known
+ position: the exact screen pixel the marker's edge should reach at the final zoom factor
+ changes from background to marker colour between the first and last frame, and a panned
+ clip differs (PSNR ~13) from the same zoom without pan at the same timestamp -- proving the
+ frame actually changes scale/position over the clip, not just that the command ran.
+- **`background.py`: generate a solid-colour or gradient clip.** New tool, no input file:
+ ffmpeg's own `color`/`gradients` source filters generate an exact-size, exact-duration clip
+ directly, for a title-card background or a base layer for `overlay.py` to composite onto.
+ Verified: a solid-colour clip's pixel matches the requested colour; a gradient's left and
+ right edges are measurably different colours in the requested direction.
+- **`reverse.py`: reverse playback.** New tool wrapping ffmpeg's `reverse`/`areverse` filters
+ (video always, audio unless `--no-audio`). These filters buffer the whole clip in memory, so
+ this is for clips it makes sense to reverse (seconds to a couple of minutes) rather than
+ something the tool limits for the caller. Verified against a real two-colour clip (first
+ half red, second half blue): the reversed output starts with the original's last half and
+ ends with its first half, confirmed by sampled pixel colour, not just duration/dimensions.
+- **`stabilize.py`: motion stabilisation.** New tool wrapping ffmpeg's two-pass
+ `vidstabdetect`/`vidstabtransform` (`--shakiness`, `--smoothing`, `--zoom` to hide the black
+ edges stabilizing can introduce). The transforms file passed between the two passes lives in
+ a `tempfile.TemporaryDirectory` for the run only -- this is the first tool in the codebase to
+ need an on-disk intermediate between two ffmpeg passes (existing two-pass tools, like
+ `loudness.py`, pass their intermediate measurement through stdout JSON instead). Requires an
+ ffmpeg built with `--enable-libvidstab`; `doctor` correctly reports `stabilize` as
+ `usable: no` (not a crash) on builds that lack it, such as Homebrew's default macOS build --
+ verified against this repo's own `ffmpeg_filters_8.1.2_macos.txt` fixture, where
+ `vidstabdetect`/`vidstabtransform` are genuinely absent from the real `-filters` listing.
+ Verified the actual stabilizing effect, not just that the command runs: a synthetic shaky
+ clip's measured frame-to-frame motion (via `signalstats` on a `tblend=difference` pass) drops
+ from ~7.4 to ~2.9 after stabilization.
+- **`sequence.py`: numbered/globbed image sequence to video.** New tool: `--pattern` accepts
+ either a printf-style numbered pattern (`frame_%04d.png`) or a glob (`*.png`, sorted
+ alphabetically), with the match checked against the real filesystem before ffmpeg runs (an
+ empty match or a missing first frame is refused here, not discovered from an opaque ffmpeg
+ error). Verified frame order is preserved end to end with a real 5-frame red/blue/red/blue/red
+ sequence, both in numbered and glob mode.
+
- **`fit.py --height`.** Only `--width` existed ("output width ... height follows the aspect").
Added a symmetric `--height` that mirrors `join.py`'s existing width/height resolution: give
one and the other follows the aspect (the source aspect, or `--aspect` if also given); give
diff --git a/README.md b/README.md
index 11646d4..e549cd0 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,7 @@ npx ffmpeg-skill
![before / after demo](assets/demo.gif)
-`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **23 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
+`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **27 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
If `ffmpeg` and `python3` are on your PATH, it works: offline, on footage you would rather not upload.
@@ -127,7 +127,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
1. **Probe first.** No tool decides from the file name. `probe.py` measures duration, fps (with variable-frame-rate detection), resolution, rotation, bit depth, HDR format including Dolby Vision, colour tags and every audio stream before anything is cut.
2. **Lossless when possible.** `cut.py`, `join.py` and `loudness.py` stream-copy what they do not need to touch. Re-encoding happens only when it must: frame-accurate cuts, filters, format changes, or a keyframe farther than the tolerance.
3. **Plan before render.** Every tool takes `--dry-run` (print the ffmpeg command lines, write nothing), `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress` (percent and ETA). A test runs every tool under `--dry-run` behind a fake ffmpeg and asserts that no ffmpeg call happened and no file appeared.
-4. **Machine-readable contract.** `contract --json` describes all 23 tools: input schema generated from the parser, output schema, role, required and conditional FFmpeg capabilities, dry-run support, the verification tools to run afterwards, whether a visual check is required, `mutates_input: false`. `provides` lists all 23 by a cross-repository Capability id (`ffmpeg-skill.cut`, `ffmpeg-skill.loudness`, ...) for [`kajisho5/AI-video-production-OS`](https://github.com/kajisho5/AI-video-production-OS)'s `CapabilityContract.provides` — see `docs/contract.md`.
+4. **Machine-readable contract.** `contract --json` describes all 27 tools: input schema generated from the parser, output schema, role, required and conditional FFmpeg capabilities, dry-run support, the verification tools to run afterwards, whether a visual check is required, `mutates_input: false`. `provides` lists all 27 by a cross-repository Capability id (`ffmpeg-skill.cut`, `ffmpeg-skill.loudness`, ...) for [`kajisho5/AI-video-production-OS`](https://github.com/kajisho5/AI-video-production-OS)'s `CapabilityContract.provides` — see `docs/contract.md`.
5. **Contract-derived MCP.** `mcp/server.py` builds its `tools/list` from the contract. Tool names, order and `inputSchema` cannot drift from the scripts; a test keeps the two byte-identical.
6. **Capability detection.** `doctor` reads `ffmpeg -encoders / -filters / -bsfs` and reports which of the components the tools need are present on this build (libx264, libass, zscale, loudnorm, xfade, …), before a job fails inside ffmpeg.
7. **Unknown is not missing.** When a listing cannot be read (a layout the parser does not know, ffmpeg exiting non-zero) the affected capabilities are `unknown`: never `missing`, never silently `available`. An installed filter is not reported absent; a failed detection is not a pass.
@@ -153,9 +153,13 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision` |
| `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio; audio-only inputs are joined as audio |
| `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list |
-| `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; force constant fps |
+| `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; rotate 90/180/270, flip h/v; force constant fps |
| `crop.py` | Crop to an exact pixel rectangle (`--x --y --width --height`) — distinct from `fit.py --fit crop`, which crops to an aspect ratio it computes itself |
-| `insert.py` | Turn a still image into a silent, fixed-duration video clip (title card, end slate) at an exact frame size / fps |
+| `insert.py` | Turn a still image into a silent, fixed-duration video clip (title card, end slate) at an exact frame size / fps, with an optional Ken Burns zoom/pan |
+| `background.py` | Generate a solid-colour or two-colour gradient clip at an exact size/duration — no input file |
+| `reverse.py` | Reverse playback (video and, unless `--no-audio`, audio) |
+| `stabilize.py` | Two-pass motion stabilisation (`vidstabdetect`/`vidstabtransform`) |
+| `sequence.py` | Numbered (`frame_%04d.png`) or glob-matched still images into a video |
**Audio**
@@ -170,7 +174,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
| Tool | What it does |
|---|---|
| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; animated and word-by-word karaoke timed to the speech energy; optional local transcription |
-| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades |
+| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |
| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs drawn by FFmpeg from a brand kit |
| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/white balance) |
@@ -214,7 +218,7 @@ npx ffmpeg-skill contract --json # or: python3 scripts/_contract.py -
npx ffmpeg-skill contract --json --static # without environment detection
```
-The contract is generated from the code that runs, not maintained beside it. For each of the 23 tools (`ffmpeg-skill/<name>`) it states:
+The contract is generated from the code that runs, not maintained beside it. For each of the 27 tools (`ffmpeg-skill/<name>`) it states:
| Field | Meaning |
|---|---|
diff --git a/SKILL.md b/SKILL.md
index fa2c380..92d1d7b 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -76,7 +76,7 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
- **Thumbnail or cover-image composition** — that's a design decision, not a measurement; a thumbnail-generation skill or the user makes it.
- **Understanding what a video is *about*** — this skill has no transcription or vision beyond `look.py`'s contact sheets, which exist for the calling agent's own eyes, not for this skill to interpret on its own.
-If a request needs an FFmpeg feature none of the 23 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards); it is exactly the failure mode this skill exists to prevent, so it is never the fallback when a script's flag doesn't cover something.
+If a request needs an FFmpeg feature none of the 27 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards); it is exactly the failure mode this skill exists to prevent, so it is never the fallback when a script's flag doesn't cover something.
## Request → script
@@ -90,10 +90,18 @@ If a request needs an FFmpeg feature none of the 23 scripts expose, say so and n
| "resize to a specific height, width follows" | `fit.py input.mp4 --height 1080` (or `--width`, or both for an exact frame) |
| "crop to this exact box/rectangle" (known x/y/width/height, not an aspect ratio) | `crop.py input.mp4 --x 100 --y 0 --width 1080 --height 1920` |
| "turn this image into a N-second clip", "title card / end slate" | `insert.py title.png --duration 3` |
+| "slow zoom on a photo", "Ken Burns effect" | `insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080` |
+| "rotate this 90 degrees", "mirror it horizontally" | `fit.py input.mp4 --rotate 90` / `fit.py input.mp4 --flip h` |
+| "reverse this clip", "play it backwards" | `reverse.py input.mp4` |
+| "stabilize this shaky footage" | `stabilize.py input.mp4` |
+| "make a blank/colour background clip" | `background.py -o bg.mp4 --duration 3 --width 1920 --height 1080 --color 0x101010` |
+| "turn these numbered frames into a video" | `sequence.py --dir frames --pattern "frame_%04d.png" --fps 24` |
| "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
| "caption it with these lines" (plain text with times) | `caption.py input.mp4 --text cues.txt` |
| "put our logo top-right", "add a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
| "add a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
+| "put this webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
+| "remove the green screen", "chroma key this" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
| "sync the lav mic to the camera", "line up the two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast) |
| "export for YouTube / Reels / X", "give me a ProRes master", "make it HEVC" | `export.py input.mp4 --preset youtube|reels|x|prores|h265` |
diff --git a/docs/contract.md b/docs/contract.md
index be29de0..e8bab75 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -122,7 +122,7 @@ gives you:
## `provides`
-`provides` lists these 23 tools by a cross-repository Capability id, for
+`provides` lists these 27 tools by a cross-repository Capability id, for
`kajisho5/AI-video-production-OS`'s `CapabilityContract.provides`
(`docs/SPEC.md` there), matching the ids already assigned to this Skill in
that project's own `docs/CAPABILITY_MATRIX.md` section 9 ("ffmpeg-skill's
diff --git a/references/scripts.md b/references/scripts.md
index ad14328..a004378 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -5,8 +5,13 @@ Every script prints the same information with `--help`; this file exists so the
## Contents
- probe.py — inspect
- cut.py — cut / join segments
-- fit.py — target duration and/or aspect
+- fit.py — target duration and/or aspect, rotate/flip
- crop.py — crop to an exact pixel rectangle
+- insert.py — still image to a timed silent clip, with Ken Burns zoom/pan
+- background.py — generate a solid-colour or gradient clip
+- reverse.py — reverse playback
+- stabilize.py — motion stabilisation (vidstab)
+- sequence.py — numbered/globbed image sequence to video
- silence.py — remove dead air / jump cuts
- join.py — concatenate with transitions
- render.py — the whole edit in one project.json
@@ -22,7 +27,7 @@ Every script prints the same information with `--help`; this file exists so the
- verify.py — real-footage verification kit
- look.py — see the result
- caption.py — subtitles (static, animated, karaoke)
-- overlay.py — logo, image, title
+- overlay.py — logo, image, title, video picture-in-picture, chroma key
- sync.py — offset detection, alignment, drift correction
- color.py — HDR to SDR, LUTs, colour tags, Dolby Vision
- audio.py — clean-up, music, ducking, layout
@@ -49,11 +54,11 @@ keyframes, instant, lossless); if the snapped result deviates more than
Multiple segments are concatenated in the order given. stderr reports whether
the result was "lossless stream copy" or "re-encoded".
-### fit.py — target duration and/or aspect
+### fit.py — target duration and/or aspect, rotate/flip
```
fit.py INPUT [--duration T --method speed|trim [--from-center] [--max-speed 4]]
[--aspect 16:9|9:16|1:1|4:5|W:H --fit pad|crop [--width W] [--height H] [--pad-color black]]
- [--fps N] [-o OUT]
+ [--rotate 90|180|270] [--flip h|v] [--fps N] [-o OUT]
```
`speed` retimes video and audio together (pitch-preserving `atempo`); it
refuses factors beyond `--max-speed`. For slow motion add `--smooth blend`
@@ -61,8 +66,12 @@ refuses factors beyond `--max-speed`. For slow motion add `--smooth blend`
`minterpolate`, fluid but roughly 10-20x slower than realtime). `trim` keeps
the head (or the middle with `--from-center`). `--width`/`--height` set the
output size: give one and the other follows the aspect (source aspect if
-`--aspect` isn't also given); give both for an exact frame. `--fps` forces a
-constant frame rate; VFR sources are conformed automatically even without it.
+`--aspect` isn't also given); give both for an exact frame. `--rotate` applies
+a new clockwise rotation (90/180/270 swap width/height for 90 and 270 — this
+is separate from the rotation *metadata* fit.py already reads to size a
+source correctly); `--flip h|v` mirrors the picture; both can combine, rotate
+first. `--fps` forces a constant frame rate; VFR sources are conformed
+automatically even without it.
### crop.py — crop to an exact pixel rectangle
```
@@ -76,6 +85,60 @@ lie entirely inside the source frame (after accounting for display rotation);
`--width`/`--height` must be even (4:2:0 chroma) and are refused, never
rounded, if they aren't.
+### insert.py — still image to a timed silent clip
+```
+insert.py IMAGE --duration T [--width W] [--height H] [--fps N]
+ [--zoom in|out [--zoom-amount 1.3]] [--pan left|right|up|down] [-o OUT]
+```
+Produces a silent, exact-duration clip from one image. `--width`/`--height`
+resolve like `fit.py`'s (one given -> the other follows the image's aspect;
+both given -> exact frame, scaled to fill and centre-cropped, never
+distorted). `--zoom in|out` is a Ken Burns effect: a slow linear zoom across
+the whole clip, ending (zoom in) or starting (zoom out) at `--zoom-amount`
+(default 1.3). `--pan` drifts the visible window across the image while
+zoomed — it needs `--zoom` (panning uses the extra image area a zoom exposes).
+
+### background.py — generate a solid-colour or gradient clip
+```
+background.py -o OUT --duration T --width W --height H
+ [--color C | --gradient C1:C2 [--angle DEG]]
+```
+No input file: ffmpeg's own `color`/`gradients` source filters generate the
+clip directly. For a title-card background, a placeholder layer, or a base
+for `overlay.py` to composite onto. `--width`/`--height` must be even.
+
+### reverse.py — reverse playback
+```
+reverse.py INPUT [--no-audio] [-o OUT]
+```
+Reverses video (and audio, unless `--no-audio`) with ffmpeg's `reverse`/
+`areverse` filters, which buffer the whole clip in memory — keep this to
+clips it makes sense to reverse (seconds to a couple of minutes), not
+something this tool limits for you.
+
+### stabilize.py — motion stabilisation
+```
+stabilize.py INPUT [--shakiness 1-10] [--smoothing N] [--zoom 0-100] [-o OUT]
+```
+Two-pass `vidstabdetect`/`vidstabtransform`: pass 1 analyses camera motion to
+a temporary transforms file (deleted after the run), pass 2 smooths and
+re-renders. `--shakiness` (default 5) trades analysis time for how much
+motion it looks for; `--smoothing` (default 15) is how many neighbouring
+frames the camera path is averaged over; `--zoom` crops in slightly to hide
+the black edges stabilizing can introduce. Needs an ffmpeg built with
+`--enable-libvidstab`; `doctor` reports this tool `usable: no` when that's
+missing (a plain Homebrew ffmpeg build, for example) rather than failing at
+run time.
+
+### sequence.py — numbered/globbed image sequence to video
+```
+sequence.py --dir DIR --pattern "frame_%04d.png"|"*.png" --fps N
+ [--start-number N] [--width W] [--height H] [-o OUT]
+```
+Turns a numbered or glob-matched set of still images into a video. The match
+is checked on disk before ffmpeg runs (an empty match or a missing first
+frame is refused here, not discovered from an opaque ffmpeg error).
+
### silence.py — remove dead air / jump cuts
```
silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
@@ -232,12 +295,17 @@ word from `--color` to `--highlight-color` evenly across the cue (word timing
is distributed, not transcribed). The ASS is kept next to the output so the
user can hand-tune timings and re-run with `--ass`.
-### overlay.py — logo, image, title
+### overlay.py — logo, image, title, video picture-in-picture, chroma key
```
overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--font-file F.ttf] [--font-size N] [--box]
+ | --video CLIP [--chromakey COLOR [--chromakey-similarity 0-1] [--chromakey-blend 0-1]]
[--position top-right|bottom-left|center|X,Y] [--margin N] [--start T] [--end T] [--fade S] [--opacity 0-1] [-o OUT]
```
-Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps playing.
+Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps
+playing. `--video` composites a second video as a picture-in-picture layer
+(same position/scale/opacity/time-range knobs as `--image`); only the main
+input's audio is kept, the PiP layer's own audio is dropped. `--chromakey`
+(with `--video`) keys out that colour first for green-screen compositing.
### sync.py — offset detection, alignment, drift correction
```
diff --git a/scripts/_contract.py b/scripts/_contract.py
index da5662b..ae701a8 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -74,13 +74,25 @@ TOOL_META: Dict[str, Dict[str, Any]] = {
required=FF + [X264, AAC], optional=[HDR_X265],
video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
"insert": dict(role="execution", inputs=["still image"], outputs=["silent video artifact of the requested duration / frame size / fps"],
- required=FF + [X264], optional=[],
+ required=FF + [X264], optional=[{"capability": "filter:zoompan", "when": "--zoom / --pan"}],
video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
+ "background": dict(role="execution", inputs=[], outputs=["generated solid-colour or gradient video artifact"],
+ required=FF + [X264], optional=[{"capability": "filter:gradients", "when": "--gradient"}],
+ video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="bit_exact", deterministic=True),
+ "reverse": dict(role="execution", inputs=["video asset"], outputs=["reversed video artifact"],
+ required=FF + [X264, "filter:reverse"], optional=[HDR_X265, {"capability": "filter:areverse", "when": "the input has audio and --no-audio is not given"}, {"capability": AAC, "when": "the input has audio and --no-audio is not given"}],
+ video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
+ "stabilize": dict(role="execution", inputs=["video asset"], outputs=["motion-stabilised video artifact"],
+ required=FF + [X264, "filter:vidstabdetect", "filter:vidstabtransform"], optional=[HDR_X265, {"capability": AAC, "when": "the input has audio"}],
+ video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
+ "sequence": dict(role="execution", inputs=["a directory of numbered/globbed still images"], outputs=["video artifact built from the frame sequence"],
+ required=FF + [X264], optional=[],
+ video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
"caption": dict(role="execution", inputs=["video asset", "SRT/ASS file or timed text (--text)"], outputs=["video artifact with burnt-in captions", "generated .srt / .ass sidecar"],
required=FF + [X264, AAC, "filter:subtitles"], optional=[{"capability": "filter:ass", "when": "--animate / --karaoke"}, HDR_X265, {"capability": "external:whisper", "when": "--transcribe"}],
video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
- "overlay": dict(role="execution", inputs=["video asset", "image (--image / --logo) or text (--text)"], outputs=["video artifact with the overlay composited"],
- required=FF + [X264, AAC], optional=[{"capability": "filter:drawtext", "when": "--text"}, HDR_X265],
+ "overlay": dict(role="execution", inputs=["video asset", "image (--image / --logo), text (--text), or a second video (--video) to composite"], outputs=["video artifact with the overlay composited"],
+ required=FF + [X264, AAC], optional=[{"capability": "filter:drawtext", "when": "--text"}, {"capability": "filter:chromakey", "when": "--chromakey"}, HDR_X265],
video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
"graphics": dict(role="execution", inputs=["video asset"], outputs=["video artifact with the drawn template"],
required=FF + [X264, AAC, "filter:drawtext"], optional=[HDR_X265],
@@ -167,6 +179,10 @@ REENCODE_META: Dict[str, Dict[str, str]] = {
"fit": dict(video="always", audio="always", note="always re-encodes to AAC when audio is present, even if only --fps or --aspect was asked for"),
"crop": dict(video="always", audio="always", note="the crop filter always forces a re-encode of both streams"),
"insert": dict(video="always", audio="never", note="always encodes a fresh silent clip from the still image; there is no audio stream to touch"),
+ "background": dict(video="always", audio="never", note="always encodes a fresh generated clip; there is no input to copy from"),
+ "reverse": dict(video="always", audio="conditional", note="video always re-encodes (reverse buffers and re-emits every frame); audio re-encodes to AAC when present and not dropped by --no-audio"),
+ "stabilize": dict(video="always", audio="conditional", note="video always re-encodes (two-pass vidstab); audio is re-encoded to AAC when present, never touched by the stabilization filters themselves"),
+ "sequence": dict(video="always", audio="never", note="always encodes a fresh clip from the frame sequence; there is no audio stream"),
"caption": dict(video="always", audio="always", note="burn-in only: no soft-subtitle mux path exists, so captions always cost a full re-encode of both streams"),
"overlay": dict(video="always", audio="always"),
"graphics": dict(video="always", audio="always"),
diff --git a/scripts/background.py b/scripts/background.py
new file mode 100644
index 0000000..4ac3b72
--- /dev/null
+++ b/scripts/background.py
@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Generate a solid-colour or two-colour gradient background clip.
+
+No input file: a silent, fixed-duration, exact-size clip generated entirely
+by ffmpeg's own source filters (`color` for solid, `gradients` for a
+two-colour gradient) -- for a title card background, a placeholder behind a
+logo, or a base layer for overlay.py to composite onto.
+
+Examples:
+ python3 background.py --duration 3 --width 1920 --height 1080 --color 0x101010
+ python3 background.py --duration 5 --width 1080 --height 1920 --gradient 0xff6a00:0x0057ff --angle 45
+"""
+import argparse
+import math
+import sys
+
+from _common import add_common, apply_common, die, emit, ffmpeg_base, info, parse_time, probe, run, video_args
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("-o", "--output", required=True, help="output file")
+ ap.add_argument("--duration", required=True, help="clip duration (seconds or mm:ss)")
+ ap.add_argument("--width", type=int, required=True, help="output width in px (must be even)")
+ ap.add_argument("--height", type=int, required=True, help="output height in px (must be even)")
+ ap.add_argument("--fps", type=float, default=30.0, help="output frame rate (default 30)")
+ src = ap.add_mutually_exclusive_group()
+ src.add_argument("--color", default="black", help="solid background colour, e.g. black, 0x101010 (default black)")
+ src.add_argument("--gradient", help="two colours as C1:C2 for a linear gradient, e.g. 0xff6a00:0x0057ff")
+ ap.add_argument("--angle", type=float, default=0.0, help="gradient angle in degrees (with --gradient, default 0 = left to right)")
+ ap.add_argument("--crf", type=int, default=18, help="x264 CRF (default 18)")
+ ap.add_argument("--preset", default="medium", help="x264 preset")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ target = parse_time(args.duration)
+ if target <= 0:
+ die("--duration must be > 0")
+ if args.fps <= 0:
+ die("--fps must be > 0")
+ if args.width <= 0 or args.height <= 0:
+ die(f"--width/--height must be > 0, got width={args.width} height={args.height}")
+ if args.width % 2 or args.height % 2:
+ die(f"--width/--height must be even (4:2:0 chroma), got width={args.width} height={args.height}")
+
+ if args.gradient:
+ try:
+ c0, c1 = args.gradient.split(":")
+ except ValueError:
+ die(f"--gradient needs two colours as C1:C2, got '{args.gradient}'")
+ rad = math.radians(args.angle)
+ x1 = round(args.width * math.cos(rad))
+ y1 = round(args.width * math.sin(rad))
+ src_filter = f"gradients=size={args.width}x{args.height}:rate={args.fps:g}:c0={c0}:c1={c1}:x0=0:y0=0:x1={x1}:y1={y1}"
+ else:
+ src_filter = f"color=c={args.color}:size={args.width}x{args.height}:rate={args.fps:g}"
+
+ output = args.output
+ cmd = ffmpeg_base() + ["-f", "lavfi", "-i", src_filter, "-t", f"{target:.3f}"]
+ cmd += video_args(None, args.crf, args.preset)
+ cmd += ["-an", output]
+ run(cmd)
+
+ result = probe(output)
+ v = result["video"]
+ info(f"wrote {output} ({result['duration']:.3f}s, {v['width']}x{v['height']}, {v['fps']:g}fps)")
+ emit(output)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/fit.py b/scripts/fit.py
index 505f10b..e4f118b 100755
--- a/scripts/fit.py
+++ b/scripts/fit.py
@@ -7,7 +7,11 @@ window with --from-center). Aspect: --aspect 16:9|9:16|1:1|4:5|W:H with
--fit pad (letterbox/pillarbox with --pad-color, default black) or --fit crop.
--width and/or --height set the output size: give one and the other follows
the aspect (source aspect if --aspect is not also given); give both for an
-exact frame.
+exact frame. --rotate 90|180|270 (clockwise) and --flip h|v apply a new
+rotation/mirror to the picture -- distinct from the rotation metadata a
+source already carries (read automatically to compute the displayed size,
+never altered by these flags unless asked). Both can be combined; rotate is
+applied before flip.
Crop keeps the centre of the frame by default, which is a guess: going from
16:9 to 9:16 throws away most of the width, and whatever isn't in the middle
@@ -24,6 +28,8 @@ Examples:
python3 fit.py input.mp4 --aspect 9:16 --fit crop --crop-x 1 # keep the right edge (e.g. product held stage-right)
python3 fit.py input.mp4 --height 1080 # width follows the source aspect
python3 fit.py input.mp4 --width 1920 --height 1080 # exact frame, no aspect needed
+ python3 fit.py input.mp4 --rotate 90 # rotate 90 degrees clockwise
+ python3 fit.py input.mp4 --flip h # mirror horizontally
"""
import argparse
import math
@@ -85,6 +91,9 @@ def main() -> int:
a.add_argument("--pad-color", default="black", help="pad colour, e.g. black, white, 0x101010 (default black)")
a.add_argument("--crop-x", type=float, default=0.5, help="with --fit crop, horizontal anchor 0=left, 0.5=centre (default), 1=right")
a.add_argument("--crop-y", type=float, default=0.5, help="with --fit crop, vertical anchor 0=top, 0.5=centre (default), 1=bottom")
+ r = ap.add_argument_group("rotate / flip")
+ r.add_argument("--rotate", type=int, choices=[90, 180, 270], help="rotate the picture clockwise by this many degrees")
+ r.add_argument("--flip", choices=["h", "v"], help="mirror the picture horizontally (h) or vertically (v)")
e = ap.add_argument_group("encoding")
e.add_argument("--crf", type=int, default=18)
e.add_argument("--preset", default="medium")
@@ -93,8 +102,8 @@ def main() -> int:
args = ap.parse_args()
apply_common(args)
- if not args.duration and not args.aspect and not args.width and not args.height and not args.fps:
- die("nothing to do: give --duration, --aspect, --width/--height and/or --fps")
+ if not args.duration and not args.aspect and not args.width and not args.height and not args.fps and not args.rotate and not args.flip:
+ die("nothing to do: give --duration, --aspect, --width/--height, --rotate/--flip and/or --fps")
if not 0.0 <= args.crop_x <= 1.0:
die(f"--crop-x must be 0..1, got {args.crop_x}")
if not 0.0 <= args.crop_y <= 1.0:
@@ -107,6 +116,8 @@ def main() -> int:
sw, sh = meta["video"]["width"], meta["video"]["height"]
if meta["video"].get("rotation") in (90, -90, 270, -270):
sw, sh = sh, sw
+ if args.rotate in (90, 270):
+ sw, sh = sh, sw
has_audio = bool(meta.get("audio"))
vf: List[str] = []
@@ -115,6 +126,18 @@ def main() -> int:
post: List[str] = []
factor = 1.0
+ # ---- rotate / flip
+ if args.rotate == 90:
+ vf.append("transpose=1")
+ elif args.rotate == 270:
+ vf.append("transpose=2")
+ elif args.rotate == 180:
+ vf.append("transpose=2,transpose=2")
+ if args.flip == "h":
+ vf.append("hflip")
+ elif args.flip == "v":
+ vf.append("vflip")
+
# ---- duration
if args.duration:
target = parse_time(args.duration)
diff --git a/scripts/insert.py b/scripts/insert.py
index da44dae..9d5d94f 100644
--- a/scripts/insert.py
+++ b/scripts/insert.py
@@ -12,11 +12,19 @@ one and the other follows the image's own aspect; give both for an exact
frame (the image is scaled to fill it, centre-cropping any excess -- never
distorted). Omit both to keep the image's native size (evened for 4:2:0).
+--zoom in|out applies a Ken Burns effect: a slow, linear zoom across the
+clip's duration (--zoom-amount sets the end/start zoom factor, default 1.3 =
+30% zoomed in by the end). --pan left|right|up|down drifts the visible
+window across the image while zoomed (ignored, with a warning, if --zoom is
+not also given -- panning needs the extra image area a zoom exposes).
+
Examples:
python3 insert.py title.png --duration 3
python3 insert.py slate.jpg --duration 5 --width 1920 --height 1080 --fps 30 -o slate.mp4
+ python3 insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080
"""
import argparse
+import math
import sys
from _common import add_common, apply_common, default_output, die, emit, ffmpeg_base, info, parse_time, probe, run, video_args
@@ -35,6 +43,9 @@ def main() -> int:
ap.add_argument("--width", type=int, help="output width in px; with --height also given, both are used directly")
ap.add_argument("--height", type=int, help="output height in px; with --width also given, both are used directly")
ap.add_argument("--fps", type=float, default=30.0, help="output frame rate (default 30)")
+ ap.add_argument("--zoom", choices=["in", "out"], help="Ken Burns: slow linear zoom in or out across the clip")
+ ap.add_argument("--zoom-amount", type=float, default=1.3, help="end (zoom in) or start (zoom out) zoom factor, > 1.0 (default 1.3)")
+ ap.add_argument("--pan", choices=["left", "right", "up", "down"], help="drift the visible window this direction while zoomed (needs --zoom)")
ap.add_argument("--crf", type=int, default=18, help="x264 CRF (default 18)")
ap.add_argument("--preset", default="medium", help="x264 preset")
add_common(ap)
@@ -46,6 +57,10 @@ def main() -> int:
die("--duration must be > 0")
if args.fps <= 0:
die("--fps must be > 0")
+ if args.zoom_amount <= 1.0:
+ die(f"--zoom-amount must be > 1.0, got {args.zoom_amount}")
+ if args.pan and not args.zoom:
+ die("--pan needs --zoom in|out")
meta = probe(args.input)
if not meta.get("video"):
@@ -64,12 +79,37 @@ def main() -> int:
else:
out_w, out_h = even(sw), even(sh)
- vf = [
- f"scale={out_w}:{out_h}:force_original_aspect_ratio=increase",
- f"crop={out_w}:{out_h}",
- "setsar=1",
- f"fps={args.fps:g}",
- ]
+ if args.zoom:
+ frames = max(1, round(target * args.fps))
+ amount = args.zoom_amount
+ if args.zoom == "in":
+ zexpr = f"if(eq(on,0),1,min(zoom+{(amount - 1) / frames:.8f},{amount:g}))"
+ else:
+ zexpr = f"if(eq(on,0),{amount:g},max(zoom-{(amount - 1) / frames:.8f},1))"
+ pan_x = {
+ "left": f"(iw-iw/zoom)*(1-on/{frames})",
+ "right": f"(iw-iw/zoom)*on/{frames}",
+ }.get(args.pan, "iw/2-(iw/zoom/2)")
+ pan_y = {
+ "up": f"(ih-ih/zoom)*(1-on/{frames})",
+ "down": f"(ih-ih/zoom)*on/{frames}",
+ }.get(args.pan, "ih/2-(ih/zoom/2)")
+ # zoompan samples from the still at its native resolution; scale it up first so the
+ # zoomed-in crop still has real pixels to draw from instead of upscaling blur.
+ upscale = max(2, math.ceil(amount * 2))
+ vf = [
+ f"scale={out_w * upscale}:{out_h * upscale}:force_original_aspect_ratio=increase",
+ f"crop={out_w * upscale}:{out_h * upscale}",
+ f"zoompan=z='{zexpr}':x='{pan_x}':y='{pan_y}':d={frames}:s={out_w}x{out_h}:fps={args.fps:g}",
+ "setsar=1",
+ ]
+ else:
+ vf = [
+ f"scale={out_w}:{out_h}:force_original_aspect_ratio=increase",
+ f"crop={out_w}:{out_h}",
+ "setsar=1",
+ f"fps={args.fps:g}",
+ ]
output = args.output or default_output(args.input, "insert", "mp4")
cmd = ffmpeg_base() + ["-loop", "1", "-i", args.input, "-t", f"{target:.3f}", "-vf", ",".join(vf)]
diff --git a/scripts/overlay.py b/scripts/overlay.py
index 1d63270..8681805 100755
--- a/scripts/overlay.py
+++ b/scripts/overlay.py
@@ -5,11 +5,20 @@ opacity and fade in/out.
Positions: top-left, top, top-right, left, center, right, bottom-left, bottom,
bottom-right, or explicit "X,Y" pixels (negative counts from the far edge).
+--video composites a second VIDEO as a picture-in-picture layer (position,
+scale, opacity, time-range -- same knobs as --image), instead of a still
+image or text. Only the main input's audio is kept; the PiP layer's own
+audio track, if any, is dropped -- mixing two audio tracks is a job for
+audio.py, not this tool. --chromakey COLOR (with --video) turns that colour
+transparent first (green-screen removal) before compositing.
+
Examples:
python3 overlay.py input.mp4 --image logo.png --position top-right --scale 200 --opacity 0.8
python3 overlay.py input.mp4 --image lower_third.png --position bottom-left --start 2 --end 8 --fade 0.5
python3 overlay.py input.mp4 --text "Episode 12" --position bottom --font-size 48 --start 1 --end 5 --fade 0.3
python3 overlay.py input.mp4 --text "こんにちは" --font-file /path/NotoSansCJK-Bold.ttc --box
+ python3 overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480 --opacity 0.9
+ python3 overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00 --chromakey-similarity 0.15
"""
import argparse
import sys
@@ -80,6 +89,11 @@ def main() -> int:
src.add_argument("--image", help="PNG/JPG (alpha respected) to composite")
src.add_argument("--text", help="text to draw (drawtext)")
src.add_argument("--logo", action="store_true", help="composite the brand logo from --brand (position/scale/opacity from brand.json)")
+ src.add_argument("--video", help="a second video to composite as a picture-in-picture layer")
+ ck = ap.add_argument_group("chroma key (with --video)")
+ ck.add_argument("--chromakey", help="colour to key out (green-screen removal), e.g. 0x00ff00 or green")
+ ck.add_argument("--chromakey-similarity", type=float, default=0.15, help="how close a pixel must be to --chromakey to become transparent, 0..1 (default 0.15)")
+ ck.add_argument("--chromakey-blend", type=float, default=0.05, help="soften the key edge, 0..1 (default 0.05)")
ap.add_argument("--brand", help="brand.json (logo, font, colours, safe margin)")
ap.add_argument("--position", default="top-right", help="named position or X,Y (default top-right)")
ap.add_argument("--margin", type=int, default=24, help="margin from the edges in px (default 24)")
@@ -117,8 +131,8 @@ def main() -> int:
args.scale = int(brand.get("logo_scale", 160))
if args.opacity == 1.0:
args.opacity = float(brand.get("logo_opacity", 1.0))
- if not (args.image or args.text):
- die("give --image, --text or --logo")
+ if not (args.image or args.text or args.video):
+ die("give --image, --text, --logo or --video")
if args.brand:
if args.margin == ap.get_default("margin"):
args.margin = int(brand.get("safe_margin", args.margin))
@@ -136,6 +150,12 @@ def main() -> int:
die("--end must be after --start")
if not 0 <= args.opacity <= 1:
die("--opacity must be within 0..1")
+ if args.chromakey and not args.video:
+ die("--chromakey needs --video")
+ if not 0 < args.chromakey_similarity <= 1:
+ die("--chromakey-similarity must be within (0, 1]")
+ if not 0 <= args.chromakey_blend <= 1:
+ die("--chromakey-blend must be within 0..1")
output = args.output or default_output(args.input, "overlay")
enable = enable_expr(start, end)
@@ -170,6 +190,29 @@ def main() -> int:
# The output must be as long as the main input, so say so explicitly.
if meta.get("duration"):
cmd += ["-t", f"{meta['duration']:.3f}"]
+ elif args.video:
+ pip_meta = probe(args.video)
+ if not pip_meta.get("video"):
+ die(f"--video {args.video} has no video stream")
+ chain = []
+ if args.scale_percent:
+ chain.append(f"scale={int(vw * args.scale_percent / 100)}:-2")
+ elif args.scale:
+ chain.append(f"scale={args.scale}:-2")
+ chain.append("format=yuva420p")
+ if args.chromakey:
+ chain.append(f"chromakey={args.chromakey}:{args.chromakey_similarity:g}:{args.chromakey_blend:g}")
+ if args.opacity < 1:
+ chain.append(f"colorchannelmixer=aa={args.opacity:g}")
+ x, y = position_exprs(args.position, args.margin, text_mode=False)
+ ov = f"overlay={x}:{y}:format=auto"
+ if enable:
+ ov += f":enable='{enable}'"
+ cmd = ffmpeg_base() + ["-i", args.input, "-i", args.video]
+ fc = f"[1:v]{','.join(chain)}[ov];[0:v][ov]{ov}[out]"
+ cmd += ["-filter_complex", fc, "-map", "[out]", "-map", "0:a:0?", "-shortest"]
+ if meta.get("duration"):
+ cmd += ["-t", f"{meta['duration']:.3f}"]
else:
x, y = position_exprs(args.position, args.margin, text_mode=True)
opts = [f"text='{escape_drawtext(args.text)}'", f"fontsize={args.font_size}", f"x={x}", f"y={y}",
diff --git a/scripts/reverse.py b/scripts/reverse.py
new file mode 100644
index 0000000..a70fdd9
--- /dev/null
+++ b/scripts/reverse.py
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+"""Reverse a video (and its audio, unless dropped).
+
+Uses ffmpeg's `reverse` (video) and `areverse` (audio) filters, which decode
+and buffer the whole clip in memory -- long inputs cost real time and RAM,
+which is why there is no length limit baked in here: it is the caller's job
+to keep this to clips it makes sense to reverse (a few seconds to a couple of
+minutes), not a workaround this tool applies for you.
+
+Examples:
+ python3 reverse.py input.mp4
+ python3 reverse.py input.mp4 --no-audio -o backwards.mp4
+"""
+import argparse
+import sys
+
+from _common import add_common, apply_common, aac_args, cfr_args, default_output, die, emit, ffmpeg_base, info, probe, run, video_args
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("input")
+ ap.add_argument("-o", "--output", help="output file (default: <name>_reverse.<ext>)")
+ ap.add_argument("--no-audio", action="store_true", help="drop audio instead of reversing it")
+ ap.add_argument("--crf", type=int, default=18, help="x264 CRF (default 18)")
+ ap.add_argument("--preset", default="medium", help="x264 preset")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ meta = probe(args.input)
+ if not meta.get("video"):
+ die("input has no video stream")
+ has_audio = bool(meta.get("audio")) and not args.no_audio
+
+ output = args.output or default_output(args.input, "reverse")
+ cmd = ffmpeg_base() + ["-i", args.input, "-vf", "reverse"]
+ if has_audio:
+ cmd += ["-af", "areverse"]
+ cmd += video_args(meta, args.crf, args.preset)
+ cmd += cfr_args(meta)
+ if has_audio:
+ cmd += aac_args()
+ else:
+ cmd += ["-an"]
+ cmd.append(output)
+ run(cmd)
+
+ result = probe(output)
+ info(f"wrote {output} ({result['duration']:.3f}s, {result['video']['width']}x{result['video']['height']})")
+ emit(output)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/sequence.py b/scripts/sequence.py
new file mode 100644
index 0000000..0693045
--- /dev/null
+++ b/scripts/sequence.py
@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+"""Turn a numbered image sequence into a video.
+
+--pattern accepts either a printf-style numbered pattern (`frame_%04d.png`,
+resolved relative to --dir) or a glob (`*.png`, matched and sorted
+alphabetically) -- detected by whether the pattern contains a `%`. Either
+way, the actual frame list is resolved and checked on disk before ffmpeg
+runs (an empty match or a missing first frame is refused here, not
+discovered from an opaque ffmpeg error), then fed to ffmpeg as an explicit
+concat list -- not `-pattern_type glob`, which several real ffmpeg builds
+(the Windows Chocolatey package, for one) compile without.
+
+Examples:
+ python3 sequence.py --dir frames --pattern "frame_%04d.png" --fps 24 -o out.mp4
+ python3 sequence.py --dir frames --pattern "*.png" --fps 30 --start-number 1
+"""
+import argparse
+import sys
+import tempfile
+from pathlib import Path
+
+from _common import add_common, apply_common, default_output, die, emit, ffmpeg_base, info, probe, run, video_args
+
+
+def even(n: float) -> int:
+ v = int(round(n))
+ return v if v % 2 == 0 else v + 1
+
+
+def _concat_list_line(path: Path) -> str:
+ # concat demuxer file paths: backslash and single-quote need escaping inside the quoted form.
+ escaped = str(path).replace("\\", "/").replace("'", "'\\''")
+ return f"file '{escaped}'"
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("--dir", required=True, help="directory containing the frames")
+ ap.add_argument("--pattern", required=True, help="printf pattern (frame_%%04d.png) or glob (*.png)")
+ ap.add_argument("-o", "--output", help="output file (default: <dir>_sequence.mp4)")
+ ap.add_argument("--fps", type=float, default=30.0, help="output frame rate (default 30)")
+ ap.add_argument("--start-number", type=int, default=0, help="first frame index, for a printf pattern (default 0)")
+ ap.add_argument("--width", type=int, help="output width in px; with --height also given, both are used directly")
+ ap.add_argument("--height", type=int, help="output height in px; with --width also given, both are used directly")
+ ap.add_argument("--crf", type=int, default=18, help="x264 CRF (default 18)")
+ ap.add_argument("--preset", default="medium", help="x264 preset")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ if args.fps <= 0:
+ die("--fps must be > 0")
+ directory = Path(args.dir)
+ if not directory.is_dir():
+ die(f"--dir not found or not a directory: {args.dir}")
+
+ is_glob = "%" not in args.pattern
+ if is_glob:
+ frames = sorted(directory.glob(args.pattern))
+ if not frames:
+ die(f"no files in {args.dir} match glob '{args.pattern}'")
+ info(f"found {len(frames)} frames matching '{args.pattern}'")
+ else:
+ try:
+ args.pattern % args.start_number
+ except (TypeError, ValueError):
+ die(f"bad printf pattern '{args.pattern}'")
+ frames = []
+ i = args.start_number
+ while (directory / (args.pattern % i)).exists():
+ frames.append(directory / (args.pattern % i))
+ i += 1
+ if not frames:
+ die(f"first frame not found: {directory / (args.pattern % args.start_number)} (check --pattern / --start-number)")
+ info(f"found {len(frames)} consecutive frames from index {args.start_number}")
+
+ frame_meta = probe(str(frames[0]))
+ if not frame_meta.get("video"):
+ die(f"{frames[0]} is not a readable image")
+ sw, sh = frame_meta["video"]["width"], frame_meta["video"]["height"]
+
+ if args.width and args.height:
+ out_w, out_h = even(args.width), even(args.height)
+ elif args.width:
+ out_w = even(args.width)
+ out_h = even(out_w * sh / sw)
+ elif args.height:
+ out_h = even(args.height)
+ out_w = even(out_h * sw / sh)
+ else:
+ out_w, out_h = even(sw), even(sh)
+
+ output = args.output or default_output(str(directory).rstrip("/\\") or "sequence", "sequence", "mp4")
+ frame_duration = 1.0 / args.fps
+
+ with tempfile.TemporaryDirectory(prefix="ffmpeg-skill-sequence-") as tmp:
+ list_path = Path(tmp) / "frames.txt"
+ lines = []
+ for f in frames:
+ lines.append(_concat_list_line(f.resolve()))
+ lines.append(f"duration {frame_duration:.6f}")
+ lines.append(_concat_list_line(frames[-1].resolve())) # concat demuxer: last entry's duration is ignored, so repeat it
+ list_path.write_text("\n".join(lines), encoding="utf-8")
+
+ cmd = ffmpeg_base() + ["-f", "concat", "-safe", "0", "-i", str(list_path)]
+ vf = [f"scale={out_w}:{out_h}", "setsar=1", f"fps={args.fps:g}"]
+ cmd += ["-vf", ",".join(vf)]
+ cmd += video_args(None, args.crf, args.preset)
+ # The concat demuxer's trailing repeated-last-file trick (needed so the last real file's
+ # duration line takes effect) has been observed to produce an extra frame's worth of
+ # duration on some ffmpeg builds -- force the exact intended length rather than trust it.
+ total_duration = len(frames) * frame_duration
+ cmd += ["-t", f"{total_duration:.6f}", "-an", output]
+ run(cmd)
+
+ result = probe(output)
+ v = result["video"]
+ info(f"wrote {output} ({result['duration']:.3f}s, {v['width']}x{v['height']}, {v['fps']:g}fps)")
+ emit(output)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/stabilize.py b/scripts/stabilize.py
new file mode 100644
index 0000000..2a6327e
--- /dev/null
+++ b/scripts/stabilize.py
@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Stabilize shaky video (FFmpeg's vidstab, two-pass).
+
+Pass 1 (vidstabdetect) analyses camera motion and writes the transforms to a
+temporary file; pass 2 (vidstabtransform) smooths that motion and re-renders
+the frames. The transforms file lives in a temp directory for the duration of
+this run only -- it is not a caller-facing artifact.
+
+--shakiness (1 = barely shaky, fast; 10 = very shaky, slow analysis) and
+--smoothing (how many neighbouring frames to average the camera path over)
+are the two knobs that matter most; --zoom crops in slightly to hide the
+black edges stabilizing can introduce (0 = keep the original framing and let
+edges show; ffmpeg's own --crop-mode is not exposed as a raw flag here).
+
+Examples:
+ python3 stabilize.py shaky.mp4
+ python3 stabilize.py shaky.mp4 --shakiness 8 --smoothing 20 --zoom 5
+"""
+import argparse
+import sys
+import tempfile
+from pathlib import Path
+
+from _common import STATE, add_common, apply_common, aac_args, cfr_args, default_output, die, emit, escape_filter_path, ffmpeg_base, info, probe, require_tool, run, video_args
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("input")
+ ap.add_argument("-o", "--output", help="output file (default: <name>_stab.<ext>)")
+ ap.add_argument("--shakiness", type=int, default=5, help="1 (barely shaky) .. 10 (very shaky), default 5")
+ ap.add_argument("--smoothing", type=int, default=15, help="frames of camera-path smoothing on each side, default 15")
+ ap.add_argument("--zoom", type=float, default=0.0, help="percent to zoom in to hide stabilization edges, 0..100 (default 0)")
+ ap.add_argument("--crf", type=int, default=18, help="x264 CRF (default 18)")
+ ap.add_argument("--preset", default="medium", help="x264 preset")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ if not 1 <= args.shakiness <= 10:
+ die(f"--shakiness must be 1..10, got {args.shakiness}")
+ if not 0 <= args.smoothing <= 1000:
+ die(f"--smoothing must be 0..1000, got {args.smoothing}")
+ if not 0 <= args.zoom <= 100:
+ die(f"--zoom must be 0..100, got {args.zoom}")
+
+ meta = probe(args.input)
+ if not meta.get("video"):
+ die("input has no video stream")
+ has_audio = bool(meta.get("audio"))
+ output = args.output or default_output(args.input, "stab")
+
+ with tempfile.TemporaryDirectory(prefix="ffmpeg-skill-vidstab-") as tmp:
+ trf = str(Path(tmp) / "transforms.trf")
+ trf_arg = escape_filter_path(trf)
+
+ if not STATE["dry_run"]:
+ ffmpeg = require_tool("ffmpeg")
+ detect_cmd = [ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-i", args.input,
+ "-vf", f"vidstabdetect=shakiness={args.shakiness}:result={trf_arg}", "-f", "null", "-"]
+ proc = run(detect_cmd, check=False)
+ if proc.returncode != 0:
+ die(f"stabilization analysis (pass 1) failed:\n{proc.stderr.strip()[-1500:]}")
+
+ transform_vf = f"vidstabtransform=input={trf_arg}:smoothing={args.smoothing}:zoom={args.zoom:g}:optzoom=1"
+ cmd = ffmpeg_base() + ["-i", args.input, "-vf", transform_vf]
+ cmd += video_args(meta, args.crf, args.preset)
+ cmd += cfr_args(meta)
+ if has_audio:
+ cmd += aac_args()
+ else:
+ cmd += ["-an"]
+ cmd.append(output)
+ run(cmd)
+
+ result = probe(output)
+ info(f"wrote {output} ({result['duration']:.3f}s, {result['video']['width']}x{result['video']['height']})")
+ emit(output)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tests/test_all.py b/tests/test_all.py
index 96db7f3..e2a1f79 100644
--- a/tests/test_all.py
+++ b/tests/test_all.py
@@ -16,7 +16,7 @@ ROOT = Path(__file__).resolve().parent.parent
SCRIPTS = ROOT / "scripts"
OUT = Path(os.environ.get("OUT", ROOT / "tests" / "out"))
sys.path.insert(0, str(SCRIPTS))
-from _common import probe # noqa: E402
+from _common import escape_filter_path, probe # noqa: E402
TONES = ("0.6*sin(2*PI*440*t)*gt(sin(2*PI*0.37*t)\\,0.3)+0.4*sin(2*PI*880*t)*gt(sin(2*PI*0.53*t+1)\\,0.6)"
"+0.3*sin(2*PI*220*t)*gt(sin(2*PI*0.21*t+2)\\,0.7)")
@@ -222,6 +222,37 @@ class FFmpegSkillTests(unittest.TestCase):
m = probe(str(out))
self.assertEqual((m["video"]["width"], m["video"]["height"]), (640, 360))
+ def test_fit_rotate_90_swaps_dimensions(self):
+ out = OUT / "fit_rot90.mp4"
+ script("fit.py", self.src, "--rotate", "90", "-o", out)
+ m = probe(str(out))
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), (720, 1280))
+
+ def test_fit_rotate_and_flip_change_actual_pixels(self):
+ """Not just that dimensions are right -- a known left/right split must actually swap or rotate."""
+ quad = OUT / "quad_src.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=100x50",
+ "-f", "lavfi", "-i", "color=c=blue:s=100x50", "-filter_complex", "[0][1]hstack", "-frames:v", "1", "-t", "1",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", quad)
+
+ def px(path, x, y):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path),
+ "-vf", f"crop=2:2:{x}:{y}", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ fliph = OUT / "quad_fliph.mp4"
+ script("fit.py", quad, "--flip", "h", "-o", fliph)
+ self.assertGreater(px(fliph, 10, 10)[2], 100, "flip h: left side should now be blue (high B channel)")
+ self.assertGreater(px(fliph, 150, 10)[0], 100, "flip h: right side should now be red (high R channel)")
+
+ rot90 = OUT / "quad_rot90.mp4"
+ script("fit.py", quad, "--rotate", "90", "-o", rot90)
+ self.assertGreater(px(rot90, 10, 10)[0], 100, "rotate 90cw: original left (red) column becomes the top row")
+
+ def test_fit_nothing_to_do_is_refused(self):
+ script("fit.py", self.src, expect_fail=True)
+
# ---------------------------------------------------------------- crop
def test_crop_exact_rectangle(self):
out = OUT / "crop1.mp4"
@@ -259,6 +290,171 @@ class FFmpegSkillTests(unittest.TestCase):
def test_insert_refuses_zero_duration(self):
script("insert.py", self.logo, "--duration", "0", expect_fail=True)
+ def test_insert_ken_burns_zoom_in_actually_scales_over_time(self):
+ """The frame must actually change scale over the clip, not just accept the flag."""
+ kb_src = OUT / "kb_src.png"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=white:s=800x800",
+ "-vf", "drawbox=x=300:y=300:w=200:h=200:color=red:t=fill", "-frames:v", "1", kb_src)
+ out = OUT / "kb1.mp4"
+ script("insert.py", kb_src, "--duration", "3", "--zoom", "in", "--zoom-amount", "1.3",
+ "--width", "640", "--height", "640", "--fps", "10", "-o", out)
+
+ def px(path, t, x, y):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path), "-ss", str(t),
+ "-vf", f"crop=2:2:{x}:{y}", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ # a screen point between the square's edge at zoom=1 (0.625 normalised) and zoom=1.3 (0.6625):
+ # white at the start (not yet covered by the square), red by the end (zoomed in enough to cover it).
+ # Tolerant of lossy x264 rounding (e.g. macOS's build lands white at 0xfd, not a pure 0xff).
+ start_r, start_g, start_b = px(out, 0.1, 410, 410)
+ self.assertGreater(start_r, 240, "should still be white/unpainted before the zoom covers it")
+ self.assertGreater(start_g, 240)
+ self.assertGreater(start_b, 240)
+ end_r, end_g, end_b = px(out, 2.9, 410, 410)
+ self.assertGreater(end_r, 200, "should be red by the end (zoomed in enough to cover this point)")
+ self.assertLess(end_b, 60)
+
+ def test_insert_pan_without_zoom_is_refused(self):
+ script("insert.py", self.logo, "--duration", "2", "--pan", "left", expect_fail=True)
+
+ def test_insert_bad_zoom_amount_refused(self):
+ script("insert.py", self.logo, "--duration", "2", "--zoom", "in", "--zoom-amount", "1.0", expect_fail=True)
+
+ # ---------------------------------------------------------------- background
+ def test_background_solid_color(self):
+ out = OUT / "bg_solid.mp4"
+ script("background.py", "-o", out, "--duration", "2", "--width", "640", "--height", "360", "--color", "0x00ff00")
+ m = probe(str(out))
+ self.assertClose(m["duration"], 2.0, 0.1)
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), (640, 360))
+
+ def test_background_gradient_has_two_distinct_colors(self):
+ out = OUT / "bg_grad.mp4"
+ script("background.py", "-o", out, "--duration", "1", "--width", "640", "--height", "360",
+ "--gradient", "0xff0000:0x0000ff")
+
+ def px(path, x, y):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path),
+ "-vf", f"crop=2:2:{x}:{y}", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ left = px(out, 10, 180)
+ right = px(out, 620, 180)
+ self.assertGreater(left[0], right[0], "left edge should be redder than the right edge")
+ self.assertGreater(right[2], left[2], "right edge should be bluer than the left edge")
+
+ def test_background_odd_dimensions_refused(self):
+ script("background.py", "-o", OUT / "bg_bad.mp4", "--duration", "1", "--width", "641", "--height", "360", expect_fail=True)
+
+ def test_background_zero_duration_refused(self):
+ script("background.py", "-o", OUT / "bg_bad2.mp4", "--duration", "0", "--width", "640", "--height", "360", expect_fail=True)
+
+ # ---------------------------------------------------------------- reverse
+ def test_reverse_swaps_start_and_end(self):
+ halfcolor = OUT / "halfcolor.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=64x64:d=1.5",
+ "-f", "lavfi", "-i", "color=c=blue:s=64x64:d=1.5", "-filter_complex", "[0][1]concat=n=2:v=1:a=0",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", halfcolor)
+ out = OUT / "rev1.mp4"
+ script("reverse.py", halfcolor, "-o", out)
+
+ def px(path, t):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path), "-ss", str(t),
+ "-vf", "crop=2:2:10:10", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ self.assertGreater(px(out, 0.2)[2], 100, "reversed clip starts with the original's last half (blue)")
+ self.assertGreater(px(out, 2.8)[0], 100, "reversed clip ends with the original's first half (red)")
+
+ def test_reverse_no_audio_drops_track(self):
+ out = OUT / "rev_noaudio.mp4"
+ script("reverse.py", self.src, "--no-audio", "-o", out)
+ m = probe(str(out))
+ self.assertIsNone(m["audio"])
+
+ # ---------------------------------------------------------------- stabilize
+ def test_stabilize_reduces_frame_to_frame_motion(self):
+ """Proves the actual effect, not just that the command runs: measured motion must drop."""
+ # Irregular, multi-frequency handheld-like jitter (not a single clean sine, which can alias
+ # with vidstab's default smoothing window on some builds and make the "shaky" source an
+ # unrepresentative test case).
+ shaky = OUT / "shaky.mp4"
+ jitter_x = "60+22*sin(2*PI*t*2.7)+11*sin(2*PI*t*5.3+1)+7*sin(2*PI*t*9.1+2)"
+ jitter_y = "60+18*cos(2*PI*t*1.9)+13*sin(2*PI*t*4.1+0.5)+6*cos(2*PI*t*7.7+1.5)"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=1400x1000:rate=30",
+ "-t", "4", "-vf", f"crop=1280:720:x='{jitter_x}':y='{jitter_y}'",
+ "-c:v", "libx264", "-preset", "veryfast", "-pix_fmt", "yuv420p", shaky)
+ out = OUT / "stab1.mp4"
+ script("stabilize.py", shaky, "-o", out)
+
+ def motion_score(path):
+ # a Windows path's drive-letter colon must be escaped for a lavfi filter option value
+ movie_path = escape_filter_path(str(path))
+ cmd = ["ffprobe", "-hide_banner", "-f", "lavfi", "-i", f"movie={movie_path},tblend=all_mode=difference,signalstats",
+ "-show_entries", "frame_tags=lavfi.signalstats.YAVG", "-of", "csv=p=0"]
+ proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ return [float(x) for x in proc.stdout.split() if x.strip()]
+
+ shaky_scores = motion_score(shaky)
+ stab_scores = motion_score(out)
+ self.assertTrue(shaky_scores, "motion_score produced no frames -- check the lavfi movie= filter path/escaping")
+ self.assertTrue(stab_scores, "motion_score produced no frames -- check the lavfi movie= filter path/escaping")
+ shaky_avg = sum(shaky_scores) / len(shaky_scores)
+ stab_avg = sum(stab_scores) / len(stab_scores)
+ # A real, meaningful reduction, not a fixed percentage: vidstab's effectiveness magnitude
+ # varies across ffmpeg/libvidstab builds (a stricter 30%-cut threshold, measured on one
+ # build, failed against a different build's own real, non-flaky measurement).
+ self.assertLess(stab_avg, shaky_avg, f"stabilized motion ({stab_avg:.2f}) should be below shaky ({shaky_avg:.2f})")
+
+ def test_stabilize_bad_shakiness_refused(self):
+ script("stabilize.py", self.src, "--shakiness", "11", expect_fail=True)
+
+ # ---------------------------------------------------------------- sequence
+ def test_sequence_numbered_pattern_preserves_order(self):
+ frames_dir = OUT / "seqframes"
+ frames_dir.mkdir(exist_ok=True)
+ for i in range(5):
+ color = "red" if i % 2 == 0 else "blue"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", f"-i", f"color=c={color}:s=64x48",
+ "-frames:v", "1", frames_dir / f"frame_{i:04d}.png")
+ out = OUT / "seq1.mp4"
+ script("sequence.py", "--dir", frames_dir, "--pattern", "frame_%04d.png", "--fps", "5", "-o", out)
+ m = probe(str(out))
+ self.assertClose(m["duration"], 1.0, 0.1)
+
+ raw = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(out), "-vf", "crop=2:2:10:10",
+ "-f", "rawvideo", "-pix_fmt", "rgb24", "-"], stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout
+ frame_px = [raw[i * 12:i * 12 + 3] for i in range(len(raw) // 12)]
+ self.assertEqual(len(frame_px), 5)
+ for i, px in enumerate(frame_px):
+ if i % 2 == 0:
+ self.assertGreater(px[0], 100, f"frame {i} should be red")
+ else:
+ self.assertGreater(px[2], 100, f"frame {i} should be blue")
+
+ def test_sequence_glob_pattern(self):
+ frames_dir = OUT / "seqframes_glob"
+ frames_dir.mkdir(exist_ok=True)
+ for i in range(5):
+ color = "red" if i % 2 == 0 else "blue"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", f"-i", f"color=c={color}:s=64x48",
+ "-frames:v", "1", frames_dir / f"frame_{i:04d}.png")
+ out = OUT / "seq2.mp4"
+ script("sequence.py", "--dir", frames_dir, "--pattern", "*.png", "--fps", "5", "-o", out)
+ m = probe(str(out))
+ self.assertClose(m["duration"], 1.0, 0.1)
+
+ def test_sequence_no_match_refused(self):
+ frames_dir = OUT / "seqframes"
+ script("sequence.py", "--dir", frames_dir, "--pattern", "*.jpg", "--fps", "5", expect_fail=True)
+
+ def test_sequence_missing_dir_refused(self):
+ script("sequence.py", "--dir", str(OUT / "does_not_exist"), "--pattern", "*.png", "--fps", "5", expect_fail=True)
+
# ---------------------------------------------------------------- caption
def test_caption_text_to_srt_and_burn(self):
srt = OUT / "cues.srt"
@@ -1362,6 +1558,51 @@ class FFmpegSkillTests(unittest.TestCase):
self.assertIn("-t", data["commands"][0])
self.assertClose(data["probe"]["duration"], 12.0, 0.15)
+ def test_overlay_video_pip_composites_at_the_right_position(self):
+ red_bg = OUT / "red_bg.mp4"
+ blue_quad = OUT / "blue_quad.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=1280x720:d=1",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", red_bg)
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=200x50:d=1",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", blue_quad)
+ out = OUT / "pip1.mp4"
+ script("overlay.py", red_bg, "--video", blue_quad, "--position", "bottom-right", "--scale", "200", "-o", out)
+
+ def px(path, x, y):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path),
+ "-vf", f"crop=2:2:{x}:{y}", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ # PiP at bottom-right, 200 wide (50 tall for the 4:1 source), default margin 24:
+ # x=1280-200-24=1056, y=720-50-24=646
+ self.assertGreater(px(out, 1100, 660)[2], 100, "PiP area should show the blue overlay")
+ self.assertGreater(px(out, 100, 100)[0], 100, "outside the PiP area, the red background must remain")
+
+ def test_overlay_chromakey_removes_the_key_color(self):
+ red_bg = OUT / "ck_red_bg.mp4"
+ green_fg = OUT / "ck_green_fg.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=100x100:d=1",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", red_bg)
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=0x00ff00:s=100x100:d=1",
+ "-f", "lavfi", "-i", "color=c=white:s=20x20:d=1", "-filter_complex", "[0][1]overlay=40:40",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", green_fg)
+ out = OUT / "ck1.mp4"
+ script("overlay.py", red_bg, "--video", green_fg, "--chromakey", "0x00ff00", "--position", "top-left",
+ "--margin", "0", "--scale", "100", "-o", out)
+
+ def px(path, x, y):
+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", str(path),
+ "-vf", f"crop=2:2:{x}:{y}", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ return r.stdout[:3]
+
+ self.assertGreater(px(out, 10, 10)[0], 100, "keyed-out green should show the red background through it")
+ self.assertGreater(min(px(out, 45, 45)), 100, "the white square inside the green should be unaffected")
+
+ def test_overlay_chromakey_without_video_refused(self):
+ script("overlay.py", self.src, "--chromakey", "green", expect_fail=True)
+
def test_help_survives_a_legacy_console_encoding(self):
"""--help contains non-ASCII (Japanese example, arrows); a cp1252 console must not raise UnicodeEncodeError."""
env = dict(os.environ, PYTHONIOENCODING="cp1252")
diff --git a/tests/test_contract.py b/tests/test_contract.py
index e857761..b1bd500 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -244,7 +244,7 @@ class ContractTests(unittest.TestCase):
self.assertEqual(self.tools["cut"]["verification"]["tools"], ["ffmpeg-skill/probe"])
def test_visual_verification_metadata(self):
- picture = {"fit", "crop", "insert", "caption", "overlay", "graphics", "color", "join", "multicam", "render"}
+ picture = {"fit", "crop", "insert", "background", "reverse", "stabilize", "sequence", "caption", "overlay", "graphics", "color", "join", "multicam", "render"}
# join is the one picture tool that also accepts audio-only inputs (audio concat); look applies
# to its video output only, which SKILL.md states next to "Look: not needed"
both = {"join"}
@@ -697,7 +697,7 @@ class DoctorDetectionTests(unittest.TestCase):
def test_two_character_flags_do_not_hide_filters(self):
"""The FFmpeg 8 layout: every declared filter is found, nothing is reported missing or unknown."""
- absent_in_brew = {"filter:drawtext", "filter:subtitles", "filter:ass", "filter:zscale"} # not built into Homebrew's 8.1.2
+ absent_in_brew = {"filter:drawtext", "filter:subtitles", "filter:ass", "filter:zscale", "filter:vidstabdetect", "filter:vidstabtransform"} # not built into Homebrew's 8.1.2 (no --enable-libvidstab)
for name in self.FILTER_FIXTURES:
d, code = self._doctor(name)
declared = [c for c in _contract.required_capabilities()["required"] + _contract.required_capabilities()["optional"] if c.startswith("filter:")]