Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
overlay.py --place: lay ranges of a video layer at chosen times in one pass (#353)
Shogo Kajiyama91bcd466dc7e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 3aff3f2..c6aeebb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,9 @@## Unreleased-(nothing yet)+### Added++- **`overlay.py --place SRC_START-SRC_END@AT`** (with `--video`, repeatable): show that range of the layer starting at `AT` on the main timeline, hidden elsewhere. One pass lays several pieces of one layer at their own times -- graphics rendered back to back (only the seconds that have any) placed where each belongs -- instead of a full-length mostly-transparent layer. Overlapping pieces, ranges past the layer's end, and `--start`/`--end`/`--fade` alongside are refused.## 2.12.1diff --git a/README.md b/README.mdindex 2e050b7..5b552d6 100644--- a/README.md+++ b/README.md@@ -247,7 +247,7 @@ These are the rules the skill file gives the agent and the code enforces.| Tool | What it does ||---|---|| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; wraps to the safe area — with `--platform`, the destination's own left/right safe zone, which since 1.17.2 is also what the burnt ASS states as its side margins — with a phrase-aware breaker (`--wrap phrase|measured`) and `--max-lines`/`--min-duration`/`--offset`; `--mode mux` takes a repeated `--srt file:lang` for several language-tagged, toggleable tracks in one file; picks a font by script for non-Latin text (`--lang`); animated and word-by-word karaoke timed to the speech energy or real word timings; `--fit-size` shrinks the size until a cue fits `--max-lines` instead of splitting the sentence (on by default on the delivery-template path since 1.17.1, where the size comes from the platform table); says `caption text unchanged` when it burned the cues exactly as given; optional local transcription |-| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--platform NAME` keeps them clear of that destination's UI; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |+| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--platform NAME` keeps them clear of that destination's UI; `--video` for picture-in-picture, `--chromakey` for green-screen compositing, `--place` to lay ranges of a layer at chosen times || `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs, social stickers, opening hook cards and meme captions drawn by FFmpeg from a brand kit; `--platform NAME` keeps them inside that destination's safe zone; `--text-render` routes shaping scripts through libass and `--emoji-assets` composites colour emoji || `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |diff --git a/demos/build.py b/demos/build.pyindex 8daa6cb..ac0d755 100644--- a/demos/build.py+++ b/demos/build.py@@ -1015,6 +1015,17 @@ def demo_greenscreen(ctx):return before, after+def demo_overlay_place(ctx):+ """overlay.py --place: two ranges of one layer, each shown at its own time on the main+ timeline; the main picture is untouched between them."""+ before = FIX / "motion.mp4"+ after = ctx.path("after.mp4")+ ctx.script("overlay.py", before, "--video", FIX / "mandel.mp4", "--position", "bottom-right",+ "--scale", "480", "--place", "[email protected]", "--place", "[email protected]",+ "--preset", "veryfast", "-o", after)+ return before, after++def demo_color_lut(ctx):before = FIX / "motion.mp4"after = ctx.path("after.mp4")@@ -1576,6 +1587,9 @@ _ROWS = [("greenscreen", PICTURE, "Green screen onto a generated background","background.py draws the gradient plate from nothing (there is no input file), then overlay.py keys the chroma green out of the subject and composites it on top.",demo_greenscreen, "video", None),+ ("overlay_place", PICTURE, "Pieces of one layer at their own times",+ "overlay.py --place lays two ranges of the fractal clip on the test pattern: its first 1.5 s at 0.5 s, its 4-5.5 s at 3.5 s. Between them the layer is hidden and the main picture is untouched -- one pass, no full-length mostly-transparent layer to render first.",+ demo_overlay_place, "video", None),("color_lut", PICTURE, "Apply a .cube LUT","A 3D LUT applied at full strength: highlights warm, shadows cool. The same flag takes a camera vendor's Log-to-709 transform or a creative look -- the file decides, the tool does not.",demo_color_lut, "video", None),diff --git a/docs/demos.md b/docs/demos.mdindex dc4b116..59dbac1 100644--- a/docs/demos.md+++ b/docs/demos.md@@ -448,6 +448,16 @@ python3 scripts/overlay.py demos/out/greenscreen_plate.mp4 --video demos/out/fix**Look for:** background.py draws the gradient plate from nothing (there is no input file), then overlay.py keys the chroma green out of the subject and composites it on top.+### Pieces of one layer at their own times++++```bash+python3 scripts/overlay.py demos/out/fixtures/motion.mp4 --video demos/out/fixtures/mandel.mp4 --position bottom-right --scale 480 --place [email protected] --place [email protected] --preset veryfast -o demos/out/overlay_place_after.mp4+```++**Look for:** overlay.py --place lays two ranges of the fractal clip on the test pattern: its first 1.5 s at 0.5 s, its 4-5.5 s at 3.5 s. Between them the layer is hidden and the main picture is untouched -- one pass, no full-length mostly-transparent layer to render first.+### Apply a .cube LUTdiff --git a/docs/demos/overlay_place.gif b/docs/demos/overlay_place.gifnew file mode 100644index 0000000..ef63418Binary files /dev/null and b/docs/demos/overlay_place.gif differdiff --git a/references/scripts.md b/references/scripts.mdindex 6435c99..b9a3e35 100644--- a/references/scripts.md+++ b/references/scripts.md@@ -1534,6 +1534,7 @@ stream selection would have picked.overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--font-file F.ttf] [--font-size N] [--box][--emoji auto|mono|none] [--emoji-assets DIR]| --video CLIP [--chromakey COLOR [--chromakey-similarity 0-1] [--chromakey-blend 0-1]]+ [--place SRC_START-SRC_END@AT ...][--position top-right|bottom-left|center|X,Y] [--margin N] [--platform NAME] [--start T] [--end T] [--fade S] [--opacity 0-1] [-o OUT]````--platform NAME` (1.14) takes each edge's margin from that destination's safe zone@@ -1550,6 +1551,20 @@ playing. `--scale` / `--scale-percent` set the width of an image, logo orinput's audio is kept, the PiP layer's own audio is dropped. `--chromakey`(with `--video`) keys out that colour first for green-screen compositing.+`--place SRC_START-SRC_END@AT` (with `--video`, repeatable) shows only that range+of the layer, starting at `AT` on the main timeline; outside its placed ranges+the layer is hidden and the main picture is untouched. One pass lays several+pieces of one layer at their own times -- e.g. graphics rendered back to back+(only the seconds that have graphics) and placed where each belongs:+`--video cues.mov --position 0,0 --margin 0 --place 0-2.5@12 --place 2.5-6@40`.+An `hh:mm:ss:ff` timecode counts the layer's frames in `SRC_START-SRC_END` and+the main video's in `AT` (an `@fps` suffix names the rate explicitly).+Refused (`kind: input`): a range that ends before it starts, negative times,+two pieces overlapping on the main timeline, a range past the layer's last+picture (by more than half a frame), a+piece starting after the main video ends, `--place` without `--video`, and+`--place` with `--start`/`--end`/`--fade` (the placements already say when).+`--fade S` fades the overlay in at `--start` (or 0); the fade-out happensonly at `--end`, so a logo with no `--end` stays to the last frame.diff --git a/scripts/overlay.py b/scripts/overlay.pyindex 6c20fa5..b36fbe4 100755--- a/scripts/overlay.py+++ b/scripts/overlay.py@@ -7,7 +7,10 @@ bottom-right, or explicit "X,Y" pixels (negative counts from the far edge).--video composites a second VIDEO as a picture-in-picture layer (position,scale, opacity, time-range -- same knobs as --image), instead of a still-image or text. Only the main input's audio is kept; the PiP layer's own+image or text. --place SRC_START-SRC_END@AT (repeatable, with --video) shows only+that range of the layer, starting at AT on the main timeline: one pass lays+several pieces of a layer (say, graphics rendered back to back) at their own+times, and the main picture shows untouched everywhere else. Only the main input's audio is kept; the PiP layer's ownaudio track, if any, is dropped -- mixing two audio tracks is a job foraudio.py, not this tool. --chromakey COLOR (with --video) turns that colourtransparent first (green-screen removal) before compositing.@@ -19,6 +22,7 @@ Examples:python3 overlay.py input.mp4 --text "こんにちは" --font-file /path/NotoSansCJK-Bold.ttc --boxpython3 overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480 --opacity 0.9python3 overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00 --chromakey-similarity 0.15+ python3 overlay.py talk.mp4 --video cues.mov --position 0,0 --margin 0 --place 0-2.5@12 --place 2.5-6@40"""import argparseimport re@@ -140,6 +144,34 @@ def per_line_drawtext(lines: List[str], x: str, y: str, font_opt: str, args: argreturn ",".join(filters)+# one time inside --place: seconds, mm:ss, hh:mm:ss.ms or hh:mm:ss:ff with an optional @fps+# suffix (the time grammar's own); times are never negative, so '-' only separates the range+_PLACE_TIME = r"[^@\-]+(?:@[0-9.]+)?"+++def parse_placements(specs: List[str], main_fps: Optional[float], layer_fps: Optional[float]) -> List[tuple]:+ """--place SRC_START-SRC_END@AT -> [(src_start, src_end, at)], refusing empty, negative or+ overlapping placements (two pieces on screen at once would stack in an order nobody asked for).+ SRC_START/SRC_END are positions in the layer, so an hh:mm:ss:ff timecode there counts the+ layer's frames; AT is a position on the main timeline and counts the main video's."""+ out = []+ for spec in specs:+ m = re.fullmatch(rf"\s*({_PLACE_TIME})-({_PLACE_TIME})@({_PLACE_TIME})\s*", spec)+ if not m:+ die(f"--place {spec!r}: use SRC_START-SRC_END@AT, e.g. 0-2.5@12 (times are not negative)")+ a = time_arg(m.group(1), "--place", layer_fps)+ b = time_arg(m.group(2), "--place", layer_fps)+ at = time_arg(m.group(3), "--place", main_fps)+ if b <= a:+ die(f"--place {spec!r}: the layer range ends before it starts")+ out.append((a, b, at))+ spans = sorted((at, at + (b - a)) for a, b, at in out)+ for (s1, e1), (s2, _e2) in zip(spans, spans[1:]):+ if s2 < e1 - 1e-6:+ die(f"--place: pieces overlap on the main timeline ({s1:g}-{e1:g} s and from {s2:g} s)")+ return out++def main() -> int:ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)ap.add_argument("input")@@ -153,6 +185,10 @@ def main() -> int:src.add_argument("--text", help="text to draw (drawtext)")src.add_argument("--logo", action="store_true", help="composite the brand logo from --brand (position/scale/opacity from brand.json)")src.add_argument("--video", help="a second video to composite as a picture-in-picture layer")+ ap.add_argument("--place", action="append", default=[], metavar="SRC_START-SRC_END@AT",+ help="with --video, repeatable: show this range of the layer starting at AT on the main "+ "timeline (e.g. 0-2.5@12); the layer is hidden outside its placed ranges. Times in "+ "seconds or HH:MM:SS. Not with --start/--end/--fade")ck = ap.add_argument_group("chroma key (with --video)")ck.add_argument("--chromakey", help="colour to key out (green-screen removal), e.g. 0x00ff00 or green")ck.add_argument("--chromakey-similarity", type=float, default=0.15, help="how close a pixel must be to --chromakey to become transparent, 0..1 (default 0.15)")@@ -304,6 +340,10 @@ def main() -> int:die("--chromakey-similarity must be within (0, 1]")if not 0 <= args.chromakey_blend <= 1:die("--chromakey-blend must be within 0..1")+ if args.place and not args.video:+ die("--place needs --video")+ if args.place and (args.start or args.end or args.fade):+ die("--place sets when each piece shows: drop --start / --end / --fade")output = args.output or default_output(args.input, "overlay")enable = enable_expr(start, end)@@ -355,6 +395,24 @@ def main() -> int:pip_meta = probe(args.video)if not pip_meta.get("video"):die(f"--video {args.video} has no video stream")+ layer_fps = pip_meta["video"].get("fps")+ placements = parse_placements(args.place, fps, layer_fps) if args.place else []+ # bounds from the video streams (a longer audio track would let a range run past the last+ # picture); the container duration only when a stream has none+ layer_dur = pip_meta["video"].get("duration") or pip_meta.get("duration")+ main_dur = meta["video"].get("duration") or meta.get("duration")+ # half a layer frame of slack absorbs a hand-typed end that rounds the last frame's end+ # (2.033 for 61 frames at 30 fps); anything longer asks for pictures the layer does not have+ slack = 0.5 / layer_fps if layer_fps else 0.02+ for a, b, at in placements:+ if layer_dur and a >= layer_dur:+ # the slack below is for an end rounded past the last frame; a start there selects+ # no picture at all and the "placed" piece would silently never show+ die(f"--place {a:g}-{b:g}@{at:g}: starts at or after the layer's end ({layer_dur:.3f} s)")+ if layer_dur and b > layer_dur + slack:+ die(f"--place {a:g}-{b:g}@{at:g}: the layer is only {layer_dur:.3f} s long")+ if main_dur and at >= main_dur:+ die(f"--place {a:g}-{b:g}@{at:g}: starts after the main video ends ({main_dur:.3f} s)")chain = []if args.scale_percent:chain.append(f"scale={int(vw * args.scale_percent / 100)}:-2")@@ -376,8 +434,26 @@ def main() -> int:base, pip = matrix_chain(meta, meta), matrix_chain(pip_meta, meta)if base or pip:base, pip = base or label_chain(meta, meta), pip or label_chain(pip_meta, meta)- fc = (f"[1:v]{pip}{','.join(chain)}[ov];"- + (f"[0:v]{base}null[base];[base]" if base else "[0:v]") + f"[ov]{ov}[out]")+ if placements:+ # each placed piece: its range of the layer, re-timed to start at AT; eof_action=pass shows+ # the main picture untouched once a piece ends (the default would freeze its last frame)+ pieces = len(placements)+ # trim selects by timestamp: start the layer's at 0 so SRC_START/SRC_END count from its+ # first picture even when its video stream starts later than the file (audio first)+ fc = f"[1:v]setpts=PTS-STARTPTS,{pip}{','.join(chain)},split={pieces}" + "".join(f"[p{i}]" for i in range(pieces)) + ";"+ for i, (a, b, at) in enumerate(placements):+ fc += f"[p{i}]trim=start={a:.6f}:end={b:.6f},setpts=PTS-STARTPTS+{at:.6f}/TB[q{i}];"+ prev = f"[0:v]{base}null[m]" if base else None+ if prev:+ fc += prev + ";"+ cur = "[m]" if base else "[0:v]"+ for i in range(pieces):+ nxt = "[out]" if i == pieces - 1 else f"[m{i}]"+ fc += f"{cur}[q{i}]{ov}:eof_action=pass{nxt}" + ("" if i == pieces - 1 else ";")+ cur = nxt+ else:+ fc = (f"[1:v]{pip}{','.join(chain)}[ov];"+ + (f"[0:v]{base}null[base];[base]" if base else "[0:v]") + f"[ov]{ov}[out]")cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?"]if meta.get("duration"):# See the --image branch above: -t (exact, bounds only the main input) instead ofdiff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.jsonindex 548037c..ffcb425 100644--- a/tests/fixtures/mcp_tools.json+++ b/tests/fixtures/mcp_tools.json@@ -791,6 +791,7 @@"opacity","output","overwrite",+ "place","plan","platform","position",diff --git a/tests/test_picture.py b/tests/test_picture.pyindex 51f2aab..1f44f21 100644--- a/tests/test_picture.py+++ b/tests/test_picture.py@@ -2102,6 +2102,88 @@ class PictureTests(MediaFixtures):self.assertGreater(px(out, 10, 10)[0], 100, "keyed-out green should show the red background through it")self.assertGreater(min(px(out, 45, 45)), 100, "the white square inside the green should be unaffected")+ def test_overlay_place_lays_layer_ranges_at_their_times(self):+ """--place SRC_START-SRC_END@AT shows only that range of the layer, from AT on the main+ timeline, and the main picture untouched elsewhere: a 4 s layer whose first half is blue and+ second half green, placed 2-3@1 (green at 1-2 s) and 0-1@4 (blue at 4-5 s) on a 6 s red main."""+ main = OUT / "place_main.mp4"+ layer = OUT / "place_layer.mov"+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=160x90:r=30:d=6",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", main)+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=160x90:r=30:d=2",+ "-f", "lavfi", "-i", "color=c=0x00ff00:s=160x90:r=30:d=2", "-filter_complex", "[0][1]concat=n=2:v=1:a=0",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", layer)+ out = OUT / "placed.mp4"+ doc = json.loads(script("overlay.py", main, "--video", layer, "--position", "0,0", "--margin", "0",+ "--place", "2-3@1", "--place", "0-1@4", "--json", "-o", out).stdout)+ self.assertEqual(doc["status"], "completed")+ self.assertAlmostEqual(probe(str(out))["duration"], 6.0, delta=0.1)++ def rgb_at(t):+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-ss", str(t), "-i", str(out),+ "-vf", "crop=4:4:40:40", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)+ return tuple(r.stdout[:3])++ def dominant(t):+ return "rgb"[max(range(3), key=lambda i: rgb_at(t)[i])]++ self.assertEqual([dominant(t) for t in (0.5, 1.5, 2.5, 4.5, 5.5)], ["r", "g", "r", "b", "r"])++ def test_overlay_place_counts_from_the_layer_s_first_picture(self):+ """A layer whose video starts a second after its audio (and so after the file's 0): `0-1`+ is its first second of picture, not the second of silence before it."""+ main = OUT / "place_late_main.mp4"+ layer = OUT / "place_late_layer.mkv" # Matroska keeps the late start (an MP4 edit list would hide it)+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=red:s=160x90:r=30:d=4",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", main)+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=160x90:r=30:d=2",+ "-f", "lavfi", "-i", "sine=d=3", "-vf", "setpts=PTS+1/TB",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-c:a", "aac", layer)+ out = OUT / "place_late.mp4"+ script("overlay.py", main, "--video", layer, "--position", "0,0", "--margin", "0",+ "--place", "0-1@1", "-o", out)+ r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-ss", "1.5", "-i", str(out),+ "-vf", "crop=4:4:40:40", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)+ self.assertGreater(r.stdout[2], r.stdout[0], "the layer's first second (blue) should show at 1-2 s")++ def test_overlay_place_refusals(self):+ layer = OUT / "place_ref_layer.mp4"+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=64x64:r=30:d=2",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", layer)+ long_audio = OUT / "place_ref_long_audio.mp4" # 2 s of picture, 4 s of sound+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=64x64:r=30:d=2",+ "-f", "lavfi", "-i", "sine=d=4", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-c:a", "aac", long_audio)+ for args in (["--video", layer, "--place", "0-1@1", "--place", "[email protected]"],+ ["--video", long_audio, "--place", "0-3@0"], # past the layer's picture # overlap on the main timeline+ ["--video", layer, "--place", "0-5@0"], # past the layer's end+ ["--video", layer, "--place", "2-2.01@1"], # starts at the layer's end+ ["--video", layer, "--place", "0-2.04@0"], # 0.04 s past it: over half a frame+ ["--video", layer, "--place", "1-0.5@0"], # backwards range+ ["--video", layer, "--place", "0-1"], # no @AT+ ["--video", layer, "--place", "0-1@0", "--start", "1"], # with --start+ ["--image", self.logo, "--place", "0-1@0"]): # without --video+ with self.subTest(args=args):+ proc = script("overlay.py", self.src, *args, "--json", "-o", OUT / "place_ref.mp4", expect_fail=True)+ self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input")++ def test_overlay_place_timecodes_count_each_timeline_s_frames(self):+ """SRC_START/SRC_END are layer positions, so hh:mm:ss:ff there counts the layer's frames;+ AT counts the main video's. An @fps suffix in any of the three names its rate explicitly."""+ layer = OUT / "place_24.mp4"+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "color=c=blue:s=64x64:r=24:d=3",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", layer)+ main_fps = probe(str(self.src))["video"]["fps"]+ for spec, want in (("00:00:00:12-00:00:01:00@00:00:00:00", "trim=start=0.500000:end=1.000000"),+ ("00:00:00:12@25-2@1", "trim=start=0.480000:end=2.000000"),+ ("0-1@00:00:00:15@30", "setpts=PTS-STARTPTS+0.500000/TB")):+ with self.subTest(spec=spec):+ doc = json.loads(script("overlay.py", self.src, "--video", layer, "--place", spec, "--dry-run",+ "--json", "-o", OUT / "place_tc.mp4").stdout)+ self.assertIn(want, doc["commands"][0])+ self.assertNotEqual(round(main_fps), 24, "the main fixture must not share the layer's rate")+def test_overlay_chromakey_without_video_refused(self):script("overlay.py", self.src, "--chromakey", "green", expect_fail=True)