Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

Release 0.5.0: render.py projects, scenes.py highlights, check.py compliance, evals, manual CI (#6)

kajisho5a4803bf43a08
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 0000000..4e7be77
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,33 @@
+# Manual-only for now: the account's Actions minutes are exhausted for this cycle.
+# After the reset (Oct 1), add:
+# push: { branches: [main] }
+# pull_request:
+# under `on:` to run on every change.
+name: tests
+on:
+ workflow_dispatch:
+jobs:
+ test:
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [ubuntu-latest, macos-latest, windows-latest]
+ runs-on: ${{ matrix.os }}
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: "3.9"
+ - name: Install ffmpeg (Linux)
+ if: runner.os == 'Linux'
+ run: sudo apt-get update && sudo apt-get install -y ffmpeg fonts-dejavu-core
+ - name: Install ffmpeg (macOS)
+ if: runner.os == 'macOS'
+ run: brew install ffmpeg
+ - name: Install ffmpeg (Windows)
+ if: runner.os == 'Windows'
+ run: choco install ffmpeg -y --no-progress
+ - run: ffmpeg -version | head -1
+ shell: bash
+ - run: python tests/test_all.py
+ - run: node bin/install.js
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6bf5ae7..d0752f4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,14 @@
# Changelog
+## 0.5.0
+
+- `render.py` (new): declarative edits from one `project.json` (clips → join → silence → fit → captions → overlays → audio → loudness → export → check). `--init` writes a starter, `--dry-run` prints every command, `--stop-after` for iterating.
+- `scenes.py` (new): scene changes (scdet) and audio peaks, highlight proposals sized to a target duration, `--edl` for `cut.py --segments`, per-scene contact sheet.
+- `check.py` (new): pre-delivery compliance for youtube / shorts / reels / tiktok / x / linkedin / broadcast / podcast / custom: duration, aspect, resolution, fps, VFR, codec, pixel format, colour/HDR, file size, loudness, true peak, with the fix command per failure.
+- `evals/`: 24 natural-language routing tasks with expected scripts, plus a transcript scorer.
+- `.github/workflows/ci.yml`: 3-OS matrix, manual trigger only until the Actions quota resets.
+- `join.py`: `--width` alone keeps the first clip's aspect; `--fast` no longer breaks `export.py` presets.
+
## 0.4.1
Fixes found by running `verify.py` on a real iPhone clip (Dolby Vision 8.4 / HLG, 10-bit HEVC, 60 fps VFR, portrait rotation, extra metadata tracks):
diff --git a/README.md b/README.md
index e76f4d7..9490aea 100644
--- a/README.md
+++ b/README.md
@@ -17,6 +17,9 @@ npx ffmpeg-skill
- **Probe first, verify last** — the skill forces the agent to read real duration/fps/resolution before editing and to check the result after, so you get "final.mp4: 59.98 s, 1080×1920, 30 fps" instead of guesses.
- **Lossless when possible** — cuts and joins use stream copy by default; re-encoding only happens when it must (frame-accurate cuts, filters, format changes).
- **Cut & join** segments with `mm:ss` / `hh:mm:ss.ms` times.
+- **Declarative edits** — describe the whole edit in a `project.json` (clips, transitions, captions, overlays, music, loudness, export, check) and re-render after every tweak.
+- **Scene detection and highlight picks** — find cuts and loud moments, get a 60-second digest proposal as a cut list.
+- **Delivery checks** — PASS/FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure.
- **Multicam** — align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list.
- **Real-footage verification kit** — run the whole toolchain on your own device files and get a PASS/FAIL report.
- **Silence removal / jump cuts** — detect dead air, keep a margin around speech, render frame-accurate in one pass; export the cut list for hand editing.
@@ -92,6 +95,9 @@ More examples: [examples/README.md](examples/README.md). To see everything run e
|--------|--------------|
| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision, colour space, rotation, audio channels as JSON; `--analyze` flags Log footage |
| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact |
+| `render.py` | Render a whole edit from `project.json`; `--init`, `--dry-run`, `--stop-after` |
+| `scenes.py` | Scene changes, audio peaks, highlight proposals and per-scene sheet |
+| `check.py` | Pre-delivery compliance per platform (duration, aspect, codec, colour, loudness, size) |
| `multicam.py` | Align cameras/recorders by audio and switch between them from a time list |
| `verify.py` | Run the toolchain on real device files and report PASS/FAIL per step |
| `silence.py` | Detect and remove silences (jump cuts), list or export the cut list |
@@ -119,6 +125,7 @@ All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stder
```bash
bash examples/make_demo.sh # generates footage, runs every script, rebuilds assets/demo.gif
python3 tests/test_all.py # end-to-end tests incl. VFR, rotated, 5.1, 10-bit HDR10 and drifting sources (needs ffmpeg)
+python3 evals/run.py --list # routing eval prompts (see evals/)
node bin/install.js --dir /tmp/skills # try the installer without touching ~/.claude
```
diff --git a/SKILL.md b/SKILL.md
index f6a0de7..ae96749 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -1,6 +1,6 @@
---
name: ffmpeg-skill
-description: Professional video editing with local FFmpeg — cut, remove silences, join with transitions, multicam switching, caption (animated/karaoke timed to speech), fit to duration/aspect, sync audio with drift correction, HDR/HLG/Dolby Vision to SDR, LUTs and Log detection, denoise/duck/mix audio, loudness, overlays, platform exports, frame inspection and a real-footage verification kit; Python stdlib scripts, no cloud or API keys.
+description: Professional video editing with local FFmpeg — declarative project rendering, scene detection and highlight picks, delivery compliance checks, cut, silence removal, transitions, multicam, captions (animated/karaoke timed to speech), fit to duration/aspect, audio sync with drift correction, HDR/HLG/Dolby Vision to SDR, LUTs, audio clean-up and ducking, loudness, overlays, platform exports, frame inspection and a real-footage verification kit; Python stdlib scripts, no cloud or API keys.
---
# ffmpeg-skill
@@ -30,15 +30,23 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
`--fast` gives a quick preview-quality render (x264 veryfast), `--progress`
prints percent and ETA on stderr for long encodes.
4. **Chain operations in a sensible order.** Colour (HDR→SDR / LUT) → cut →
- fit → caption/overlay → sync → audio → loudness → export. Do the destructive/aspect changes before burning
- text so captions are sized for the final frame. Re-encode as few times as
- possible: if several re-encoding steps are needed, keep intermediates at
- CRF 18 (the default) and only use `export.py` for the last step.
-5. **Verify the output.** Run `probe.py` on each result and confirm duration,
+ join → silence → fit → caption/overlay → sync → audio → loudness → export.
+ Do frame changes (fit/crop) before captions and overlays so text is sized
+ for the final frame. Re-encode as few times as possible: keep intermediates
+ at CRF 18 (the default) and only use `export.py` for the last step; for
+ anything with more than two steps use `render.py` with a project.json.
+5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X`
+ for the destination the user named; fix FAILs, mention WARNs.
+6. **Verify the output.** Run `probe.py` on each result and confirm duration,
resolution, fps and audio match what was requested. Report those numbers to
the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
-6. **Keep the user's originals.** Never overwrite the source file. Write new
+7. **Keep the user's originals.** Never overwrite the source file. Write new
files next to the input or where the user asked.
+8. **Look at the picture.** After captioning, overlaying, cropping or colour
+ work run `look.py OUTPUT` (contact sheet) or `look.py OUTPUT --at T` and
+ view the PNG: text inside the frame and not over faces, logos where asked,
+ crops keeping the subject, colours not washed out. Fix and re-run before
+ reporting. Numbers from probe are not enough.
## Request → script
@@ -61,6 +69,9 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
| "stitch these clips together", "add a crossfade between them" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
| "show me what it looks like", "check the captions are readable" | `look.py output.mp4` then view the PNG |
| "what would you run?", "don't render yet" | any script with `--dry-run` |
+| "make a 60 s highlight from this hour", "find the good bits" | `scenes.py long.mp4 --highlights 6 --target 60 --edl picks.txt` → `cut.py --segments` |
+| "is this OK to upload?", "check it meets the Reels spec" | `check.py final.mp4 --platform reels` |
+| "set it up so I can tweak and re-render", "several changes to the same edit" | `render.py --init project.json`, edit, `render.py project.json` |
| "three cameras, cut between them" | `multicam.py camA.mp4 camB.mp4 camC.mp4 --switch "0-20:0,20-40:1,40-60:2"` |
| "it's an iPhone Dolby Vision clip and players show it wrong" | `color.py clip.mov --to-sdr` or `color.py clip.mov --strip-dovi` (keep HDR, drop the DV layer) |
| "does it look like Log / S-Log / flat footage?" | `probe.py clip.mp4 --analyze` (`looks_like_log`) then `color.py --lut` |
@@ -132,6 +143,36 @@ Normalises every clip to one frame size, fps, `yuv420p` and 48 kHz stereo
`acrossfade`. Output length = sum of clips − transition × (n−1). Clips must be
longer than 2 × the transition. Use `--transition none` for a plain cut.
+### render.py — the whole edit in one project.json
+```
+render.py --init project.json # starter file
+render.py project.json [--fast] [--dry-run] [--stop-after STAGE] [--work DIR --keep]
+```
+Stages: clips (cut, optional speed) → join (transition) → silence → fit →
+captions → overlays → audio → loudness → export → check. Keys mirror the
+CLI flags of each script (see the docstring). Use it whenever an edit has
+more than two steps or the user is likely to ask for changes: edit the JSON,
+re-render, and the result is reproducible. `--dry-run --json` prints the
+complete command plan for review.
+
+### scenes.py — scene changes and highlight candidates
+```
+scenes.py INPUT [--threshold 10] [--min-scene 1] [--highlights N [--target SECONDS] [--max-scene 15]] [--edl picks.txt] [--sheet scenes.png] [--json]
+```
+Lists scenes with audio energy, the loudest moments, and (with
+`--highlights`) proposes N ranges that add up to `--target` seconds, biased to
+the loudest window of each scene. Review the sheet + JSON, adjust the EDL, then
+`cut.py --segments`. It is a proposal engine, not a judgement of content:
+tell the user what it picked and why (energy, scene length).
+
+### check.py — pre-delivery compliance
+```
+check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|broadcast|podcast|custom [--no-loudness] [--json]
+ [--max-duration S] [--aspect 9:16] [--lufs -14] [--tp -1] [--max-mb N]
+```
+PASS/WARN/FAIL per check with the script that fixes it. Run it as the final
+step before reporting a deliverable; fix FAILs, mention WARNs.
+
### multicam.py — align several cameras and switch between them
```
multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."] | [--auto N] [--audio IDX] [--fix-drift]
diff --git a/evals/run.py b/evals/run.py
new file mode 100755
index 0000000..e71338b
--- /dev/null
+++ b/evals/run.py
@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+"""Score an agent transcript against evals/tasks.json.
+
+Usage:
+ python3 evals/run.py transcript.txt --task 5 # did the transcript for task 5 call the expected script(s)?
+ python3 evals/run.py --list # print the prompts (paste them to the agent one by one)
+ python3 evals/run.py results/ # folder of <id>.txt transcripts -> pass rate
+
+A transcript "passes" when every expected script name (or flag) appears in it.
+Pair with skill-creator's eval runner for automated, multi-run measurement.
+"""
+import argparse
+import json
+import sys
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+TASKS = json.loads((HERE / "tasks.json").read_text(encoding="utf-8"))["tasks"]
+
+
+def score(text: str, task: dict) -> tuple:
+ missing = [e for e in task["expect"] if e not in text]
+ return (not missing, missing)
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("target", nargs="?", help="transcript file or folder of <id>.txt files")
+ ap.add_argument("--task", type=int, help="task id for a single transcript")
+ ap.add_argument("--list", action="store_true")
+ args = ap.parse_args()
+ if args.list:
+ for t in TASKS:
+ print(f"{t['id']:2d}. {t['request']} -> {', '.join(t['expect'])}")
+ return 0
+ if not args.target:
+ ap.error("give a transcript file/folder or --list")
+ p = Path(args.target)
+ results = []
+ if p.is_dir():
+ for t in TASKS:
+ f = p / f"{t['id']}.txt"
+ if f.exists():
+ ok, missing = score(f.read_text(encoding="utf-8", errors="replace"), t)
+ results.append((t, ok, missing))
+ else:
+ if not args.task:
+ ap.error("--task ID is required for a single transcript")
+ t = next((x for x in TASKS if x["id"] == args.task), None)
+ if not t:
+ ap.error(f"no task {args.task}")
+ ok, missing = score(p.read_text(encoding="utf-8", errors="replace"), t)
+ results.append((t, ok, missing))
+ passed = sum(1 for _, ok, _ in results if ok)
+ for t, ok, missing in results:
+ print(f"{'PASS' if ok else 'FAIL'} {t['id']:2d} {t['request'][:60]:60s}" + ("" if ok else f" missing: {', '.join(missing)}"))
+ print(f"{passed}/{len(results)} passed")
+ return 0 if passed == len(results) else 1
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/evals/tasks.json b/evals/tasks.json
new file mode 100644
index 0000000..461a23d
--- /dev/null
+++ b/evals/tasks.json
@@ -0,0 +1,29 @@
+{
+ "description": "Routing eval for ffmpeg-skill: natural-language requests and the script(s) an agent should reach for. Run with evals/run.py against an agent transcript, or use skill-creator's eval runner with these as prompts.",
+ "tasks": [
+ {"id": 1, "request": "Make this 60 seconds and vertical for Reels.", "expect": ["probe.py", "fit.py", "export.py"], "args_hint": "--duration 60 --aspect 9:16; export --preset reels"},
+ {"id": 2, "request": "Cut from 1:20 to 2:05 without losing quality.", "expect": ["probe.py", "cut.py"], "args_hint": "--start 1:20 --end 2:05 (no --accurate unless VFR)"},
+ {"id": 3, "request": "Burn these subtitles in, Japanese font, at the top.", "expect": ["caption.py"], "args_hint": "--font 'Noto Sans CJK JP' --position top"},
+ {"id": 4, "request": "TikTok style captions where each word pops and highlights.", "expect": ["caption.py"], "args_hint": "--animate pop --karaoke"},
+ {"id": 5, "request": "The lav mic audio is out of sync with the camera, fix it.", "expect": ["sync.py"], "args_hint": "--replace-audio"},
+ {"id": 6, "request": "It's a 90 minute talk and the audio drifts by the end.", "expect": ["sync.py"], "args_hint": "--fix-drift"},
+ {"id": 7, "request": "Normalise the audio for YouTube.", "expect": ["loudness.py"], "args_hint": "-I -14 --tp -1"},
+ {"id": 8, "request": "Put my logo top right with a fade in.", "expect": ["overlay.py"], "args_hint": "--image logo.png --position top-right --fade"},
+ {"id": 9, "request": "Export a ProRes master and an H.265 copy.", "expect": ["export.py"], "args_hint": "--preset prores; --preset h265"},
+ {"id": 10, "request": "This iPhone video looks washed out on YouTube.", "expect": ["probe.py", "color.py"], "args_hint": "probe shows hdr; color --to-sdr"},
+ {"id": 11, "request": "Apply the S-Log3 LUT to this Sony footage.", "expect": ["color.py"], "args_hint": "--lut file.cube"},
+ {"id": 12, "request": "Remove the background hiss and add some quiet music under the voice.", "expect": ["audio.py"], "args_hint": "--voice --music bed.mp3 --duck"},
+ {"id": 13, "request": "Convert the 5.1 to stereo.", "expect": ["audio.py"], "args_hint": "--downmix"},
+ {"id": 14, "request": "Cut out all the pauses, it's a talking head.", "expect": ["silence.py"], "args_hint": "default or --threshold -40"},
+ {"id": 15, "request": "Stitch these three clips with a crossfade.", "expect": ["join.py"], "args_hint": "--transition fade"},
+ {"id": 16, "request": "Two cameras and a Zoom recorder, cut between them every 20 seconds.", "expect": ["multicam.py"], "args_hint": "--audio 2 --auto 20"},
+ {"id": 17, "request": "Show me what the result looks like before you export.", "expect": ["look.py"], "args_hint": "contact sheet or --at"},
+ {"id": 18, "request": "Is this ready to upload to Instagram?", "expect": ["check.py"], "args_hint": "--platform reels"},
+ {"id": 19, "request": "Make a 60 second highlight reel from this hour of footage.", "expect": ["scenes.py", "cut.py"], "args_hint": "scenes --highlights N --target 60 --edl; cut --segments"},
+ {"id": 20, "request": "Half speed slow motion but smooth, not stuttery.", "expect": ["fit.py"], "args_hint": "--duration 2x --smooth interpolate"},
+ {"id": 21, "request": "Test the toolkit on my GoPro and OBS files first.", "expect": ["verify.py"], "args_hint": "folder --report"},
+ {"id": 22, "request": "Set up the whole edit so I can tweak numbers and re-render.", "expect": ["render.py"], "args_hint": "--init project.json then render"},
+ {"id": 23, "request": "Tell me what you'd run before actually rendering.", "expect": ["--dry-run"], "args_hint": "any script with --dry-run"},
+ {"id": 24, "request": "Why is the video stuttering after the cut? It's a screen recording.", "expect": ["probe.py", "fit.py"], "args_hint": "VFR suspected -> --fps 30 / accurate"}
+ ]
+}
diff --git a/examples/README.md b/examples/README.md
index 112ac93..c8624c3 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -101,6 +101,26 @@ python3 $S/fit.py input.mp4 --duration 60 --aspect 9:16 --dry-run
python3 $S/export.py input.mp4 --preset reels --json # structured result incl. probe of the output
```
+### "Do the whole edit and let me tweak it"
+```bash
+python3 $S/render.py --init project.json # fill in clips, captions, music, export, check
+python3 $S/render.py project.json --dry-run # review the command plan
+python3 $S/render.py project.json --fast # preview
+python3 $S/render.py project.json # final
+```
+
+### "Make a 60 second highlight from an hour"
+```bash
+python3 $S/scenes.py event.mp4 --highlights 6 --target 60 --edl picks.txt --sheet scenes.png
+python3 $S/cut.py event.mp4 --segments "$(paste -sd, picks.txt)" --accurate -o digest.mp4
+```
+
+### "Is it OK to upload?"
+```bash
+python3 $S/check.py final.mp4 --platform reels
+python3 $S/check.py spot.mov --platform broadcast # EBU R128 -23 LUFS
+```
+
### "Three cameras and a recorder — cut it together"
```bash
python3 $S/multicam.py camA.mp4 camB.mp4 camC.mp4 zoom.wav --offsets-only
diff --git a/package.json b/package.json
index 716dedd..d021118 100644
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
{
"name": "ffmpeg-skill",
- "version": "0.4.1",
- "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
+ "version": "0.5.0",
+ "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: declarative project rendering, scene detection, delivery checks, cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
"keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],
"license": "MIT",
"author": "kajisho5",
diff --git a/scripts/_common.py b/scripts/_common.py
index bc93054..18d112e 100755
--- a/scripts/_common.py
+++ b/scripts/_common.py
@@ -55,6 +55,7 @@ def require_tool(name: str) -> str:
return "" # unreachable
+X264_PRESETS = ("ultrafast", "superfast", "veryfast", "faster", "fast", "medium", "slow", "slower", "veryslow", "placebo")
STATE: Dict[str, Any] = {"dry_run": False, "json": False, "commands": [], "progress": False, "fast": False, "duration_hint": None}
@@ -72,7 +73,7 @@ def apply_common(args: "argparse.Namespace") -> None:
STATE["json"] = bool(getattr(args, "json", False))
STATE["progress"] = bool(getattr(args, "progress", False))
STATE["fast"] = bool(getattr(args, "fast", False))
- if STATE["fast"] and hasattr(args, "preset"):
+ if STATE["fast"] and getattr(args, "preset", None) in X264_PRESETS:
args.preset = "veryfast"
@@ -163,7 +164,7 @@ def probe(path: str) -> Dict[str, Any]:
if not os.path.exists(path):
if STATE["dry_run"]:
return {"file": path, "dry_run": True, "format": None, "duration": 0.0, "size_bytes": 0, "bitrate": None,
- "video": {"codec": None, "width": 0, "height": 0, "fps": None, "pix_fmt": None, "hdr": False,
+ "video": {"codec": None, "width": 1920, "height": 1080, "fps": 30.0, "pix_fmt": None, "hdr": False,
"color_transfer": None, "color_primaries": None, "rotation": 0, "variable_frame_rate_suspected": False},
"audio": {"codec": None, "channels": 0, "sample_rate": 0}, "subtitle_streams": 0}
die(f"input not found: {path}")
diff --git a/scripts/check.py b/scripts/check.py
new file mode 100755
index 0000000..30c929b
--- /dev/null
+++ b/scripts/check.py
@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+"""Pre-delivery compliance check: does this file meet the platform's spec?
+
+Checks duration, frame size / aspect, fps, codec, pixel format, colour tags,
+file size, integrated loudness and true peak against the chosen platform
+and prints a PASS/WARN/FAIL table. Exit code 1 when anything FAILs.
+
+Platforms: youtube, shorts, reels, tiktok, x, linkedin, broadcast (EBU R128), podcast, custom
+
+Examples:
+ python3 check.py final.mp4 --platform youtube
+ python3 check.py reel.mp4 --platform reels --json
+ python3 check.py spot.mov --platform broadcast
+ python3 check.py clip.mp4 --platform custom --max-duration 30 --aspect 1:1 --lufs -16
+"""
+import argparse
+import json
+import re
+import sys
+from fractions import Fraction
+from typing import Any, Dict, List
+
+from _common import add_common, apply_common, die, emit, info, probe, require_tool, run
+
+SPECS: Dict[str, Dict[str, Any]] = {
+ "youtube": {"max_duration": 12 * 3600, "aspects": ["16:9", "9:16", "1:1", "4:3"], "min_height": 720, "fps_max": 60, "codecs": ["h264", "hevc", "prores", "av1", "vp9"], "max_bytes": 256 * 1024 ** 3, "lufs": -14, "lufs_tol": 2.0, "tp": -1.0, "sdr_only": False},
+ "shorts": {"max_duration": 180, "aspects": ["9:16", "1:1"], "min_height": 1080, "fps_max": 60, "codecs": ["h264", "hevc"], "max_bytes": 256 * 1024 ** 3, "lufs": -14, "lufs_tol": 2.0, "tp": -1.0, "sdr_only": False},
+ "reels": {"max_duration": 90, "aspects": ["9:16", "4:5", "1:1"], "min_height": 1080, "fps_max": 60, "codecs": ["h264", "hevc"], "max_bytes": 4 * 1024 ** 3, "lufs": -14, "lufs_tol": 2.0, "tp": -1.0, "sdr_only": True},
+ "tiktok": {"max_duration": 600, "aspects": ["9:16", "1:1"], "min_height": 1080, "fps_max": 60, "codecs": ["h264", "hevc"], "max_bytes": 4 * 1024 ** 3, "lufs": -14, "lufs_tol": 2.0, "tp": -1.0, "sdr_only": True},
+ "x": {"max_duration": 140, "aspects": ["16:9", "1:1", "9:16"], "min_height": 720, "fps_max": 60, "codecs": ["h264"], "max_bytes": 512 * 1024 ** 2, "lufs": -14, "lufs_tol": 3.0, "tp": -1.0, "sdr_only": True},
+ "linkedin": {"max_duration": 600, "aspects": ["16:9", "1:1", "9:16", "4:5"], "min_height": 720, "fps_max": 60, "codecs": ["h264"], "max_bytes": 5 * 1024 ** 3, "lufs": -14, "lufs_tol": 3.0, "tp": -1.0, "sdr_only": True},
+ "broadcast": {"max_duration": None, "aspects": ["16:9"], "min_height": 1080, "fps_max": 60, "codecs": ["prores", "dnxhd", "h264", "hevc", "mpeg2video"], "max_bytes": None, "lufs": -23, "lufs_tol": 1.0, "tp": -1.0, "sdr_only": False},
+ "podcast": {"max_duration": None, "aspects": None, "min_height": 0, "fps_max": None, "codecs": None, "max_bytes": None, "lufs": -16, "lufs_tol": 1.0, "tp": -1.0, "sdr_only": False},
+ "custom": {"max_duration": None, "aspects": None, "min_height": 0, "fps_max": None, "codecs": None, "max_bytes": None, "lufs": None, "lufs_tol": 2.0, "tp": -1.0, "sdr_only": False},
+}
+
+
+def measure_loudness(path: str) -> Dict[str, float]:
+ ffmpeg = require_tool("ffmpeg")
+ proc = run([ffmpeg, "-hide_banner", "-nostdin", "-i", path, "-vn", "-af", "loudnorm=I=-14:TP=-1:LRA=11:print_format=json", "-f", "null", "-"], quiet=True, check=False)
+ m = re.search(r"\{[^{}]*\"input_i\"[^{}]*\}", proc.stderr, re.S)
+ if not m:
+ return {}
+ d = json.loads(m.group(0))
+ try:
+ return {"lufs": float(d["input_i"]), "tp": float(d["input_tp"]), "lra": float(d["input_lra"])}
+ except (KeyError, ValueError):
+ return {}
+
+
+def aspect_name(w: int, h: int) -> str:
+ f = Fraction(w, h)
+ for name, target in (("16:9", Fraction(16, 9)), ("9:16", Fraction(9, 16)), ("1:1", Fraction(1)), ("4:5", Fraction(4, 5)), ("4:3", Fraction(4, 3)), ("21:9", Fraction(21, 9))):
+ if abs(float(f) - float(target)) < 0.02:
+ return name
+ return f"{f.numerator}:{f.denominator}"
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("input")
+ ap.add_argument("--platform", choices=sorted(SPECS), default="youtube")
+ ap.add_argument("--max-duration", type=float, help="override max duration in seconds")
+ ap.add_argument("--aspect", help="override allowed aspect (e.g. 9:16 or 16:9,1:1)")
+ ap.add_argument("--lufs", type=float, help="override loudness target")
+ ap.add_argument("--tp", type=float, help="override true-peak ceiling")
+ ap.add_argument("--max-mb", type=float, help="override max file size in MB")
+ ap.add_argument("--no-loudness", action="store_true", help="skip the loudness measurement (faster)")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ spec = dict(SPECS[args.platform])
+ if args.max_duration is not None:
+ spec["max_duration"] = args.max_duration
+ if args.aspect:
+ spec["aspects"] = [a.strip() for a in args.aspect.split(",")]
+ if args.lufs is not None:
+ spec["lufs"] = args.lufs
+ if args.tp is not None:
+ spec["tp"] = args.tp
+ if args.max_mb is not None:
+ spec["max_bytes"] = int(args.max_mb * 1024 * 1024)
+
+ meta = probe(args.input)
+ v, a = meta.get("video") or {}, meta.get("audio") or {}
+ rows: List[Dict[str, Any]] = []
+
+ def row(name: str, status: str, value: Any, expect: Any, fix: str = "") -> None:
+ rows.append({"check": name, "status": status, "value": value, "expected": expect, "fix": fix})
+
+ dur = meta.get("duration") or 0.0
+ if spec["max_duration"]:
+ row("duration", "PASS" if dur <= spec["max_duration"] else "FAIL", f"{dur:.2f}s", f"<= {spec['max_duration']:g}s", "fit.py --duration N or cut.py")
+ else:
+ row("duration", "PASS", f"{dur:.2f}s", "any")
+
+ if v:
+ w, h = v["width"], v["height"]
+ if v.get("rotation") in (90, -90, 270, -270):
+ w, h = h, w
+ asp = aspect_name(w, h)
+ if spec["aspects"]:
+ row("aspect", "PASS" if asp in spec["aspects"] else "FAIL", asp, "/".join(spec["aspects"]), f"fit.py --aspect {spec['aspects'][0]} --fit pad|crop")
+ else:
+ row("aspect", "PASS", asp, "any")
+ short = min(w, h)
+ if spec["min_height"]:
+ row("resolution", "PASS" if short >= spec["min_height"] else "WARN", f"{w}x{h}", f"short side >= {spec['min_height']}", "upscaling will not add detail; re-export from the master")
+ fps = v.get("fps") or 0
+ if spec["fps_max"]:
+ row("fps", "PASS" if fps <= spec["fps_max"] + 0.01 else "FAIL", f"{fps:g}", f"<= {spec['fps_max']}", "fit.py --fps 30")
+ row("vfr", "PASS" if not v.get("variable_frame_rate_suspected") else "WARN", "variable" if v.get("variable_frame_rate_suspected") else "constant", "constant", "fit.py --fps N (any re-encode conforms it)")
+ if spec["codecs"]:
+ row("video codec", "PASS" if v.get("codec") in spec["codecs"] else "FAIL", v.get("codec"), "/".join(spec["codecs"]), "export.py --preset " + args.platform.replace("shorts", "reels").replace("tiktok", "reels").replace("linkedin", "youtube"))
+ pf = v.get("pix_fmt") or ""
+ if args.platform in ("reels", "tiktok", "x", "linkedin"):
+ row("pixel format", "PASS" if pf == "yuv420p" else "FAIL", pf, "yuv420p (8-bit 4:2:0)", "export.py preset re-encodes to yuv420p")
+ if spec["sdr_only"] and v.get("hdr"):
+ row("colour", "FAIL", v.get("hdr_format"), "SDR BT.709", "color.py --to-sdr")
+ else:
+ tags = (v.get("color_primaries"), v.get("color_transfer"))
+ ok = v.get("hdr") or tags == ("bt709", "bt709") or (args.platform in ("podcast", "custom"))
+ row("colour", "PASS" if ok else "WARN", f"{tags[0]}/{tags[1]}" + (f" ({v.get('hdr_format')})" if v.get("hdr") else ""), "bt709/bt709 tagged (or HDR)", "color.py --retag bt709 when the picture really is 709")
+ elif args.platform not in ("podcast", "custom"):
+ row("video", "FAIL", "none", "video stream", "")
+
+ size = meta.get("size_bytes") or 0
+ if spec["max_bytes"]:
+ row("file size", "PASS" if size <= spec["max_bytes"] else "FAIL", f"{size / 1024 / 1024:.1f} MB", f"<= {spec['max_bytes'] / 1024 / 1024:.0f} MB", "export.py --crf 24 or lower resolution")
+
+ if a:
+ row("audio", "PASS", f"{a.get('codec')} {a.get('channels')}ch {a.get('sample_rate')}Hz", "present")
+ if a.get("sample_rate") and a["sample_rate"] not in (44100, 48000):
+ row("sample rate", "WARN", a["sample_rate"], "44100 or 48000", "loudness.py --sample-rate 48000")
+ if not args.no_loudness and spec["lufs"] is not None:
+ lm = measure_loudness(args.input)
+ if lm:
+ diff = abs(lm["lufs"] - spec["lufs"])
+ row("loudness", "PASS" if diff <= spec["lufs_tol"] else "FAIL", f"{lm['lufs']:.1f} LUFS", f"{spec['lufs']:g} ± {spec['lufs_tol']:g} LUFS", f"loudness.py -I {spec['lufs']:g}")
+ row("true peak", "PASS" if lm["tp"] <= spec["tp"] + 0.05 else "FAIL", f"{lm['tp']:.1f} dBTP", f"<= {spec['tp']:g} dBTP", f"loudness.py --tp {spec['tp']:g}")
+ elif args.platform in ("podcast",):
+ row("audio", "FAIL", "none", "audio stream", "audio.py --replace")
+ else:
+ row("audio", "WARN", "none", "audio stream", "audio.py --replace (silent uploads are often rejected)")
+
+ failed = [r for r in rows if r["status"] == "FAIL"]
+ warned = [r for r in rows if r["status"] == "WARN"]
+ if not args.json:
+ width = max(len(r["check"]) for r in rows)
+ print(f"{args.input} — {args.platform}")
+ for r in rows:
+ line = f" {r['status']:4s} {r['check']:{width}s} {r['value']} (expected {r['expected']})"
+ if r["status"] != "PASS" and r["fix"]:
+ line += f" -> {r['fix']}"
+ print(line)
+ print(f" {len(rows)} checks, {len(failed)} failed, {len(warned)} warnings")
+ emit(None, platform=args.platform, checks=rows, failed=len(failed), warnings=len(warned), ok=not failed)
+ return 1 if failed else 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/export.py b/scripts/export.py
index 9822bb0..2587e0a 100755
--- a/scripts/export.py
+++ b/scripts/export.py
@@ -22,7 +22,7 @@ import argparse
import sys
from typing import Dict, List
-from _common import add_common, apply_common, emit, cfr_args, default_output, die, ffmpeg_base, info, probe, run
+from _common import STATE, add_common, apply_common, emit, cfr_args, default_output, die, ffmpeg_base, info, probe, run
PRESETS: Dict[str, Dict] = {
"youtube": {"w": 1920, "h": 1080, "ext": "mp4", "video": ["-c:v", "libx264", "-preset", "slow", "-crf", "18", "-profile:v", "high", "-pix_fmt", "yuv420p"], "audio": ["-c:a", "aac", "-b:a", "192k", "-ar", "48000"], "max": None, "desc": "1080p H.264, AAC 192k"},
@@ -94,6 +94,8 @@ def main() -> int:
video = list(p["video"])
if args.crf is not None and "-crf" in video:
video[video.index("-crf") + 1] = str(args.crf)
+ if STATE["fast"] and "-preset" in video:
+ video[video.index("-preset") + 1] = "veryfast"
cmd += video
if "-r" not in video:
cmd += cfr_args(meta)
diff --git a/scripts/fit.py b/scripts/fit.py
index 5672348..66a8a79 100755
--- a/scripts/fit.py
+++ b/scripts/fit.py
@@ -102,6 +102,8 @@ def main() -> int:
if target <= 0:
die("target duration must be > 0")
if args.method == "speed":
+ if src_dur <= 0 and STATE["dry_run"]:
+ src_dur = target # planning against an intermediate that does not exist yet
factor = src_dur / target # >1 = speed up
if factor > args.max_speed or factor < 1 / args.max_speed:
die(f"required speed factor {factor:.2f}x exceeds --max-speed {args.max_speed}x; use --method trim or raise the limit")
diff --git a/scripts/join.py b/scripts/join.py
index 96595fd..83f3961 100755
--- a/scripts/join.py
+++ b/scripts/join.py
@@ -14,7 +14,7 @@ import argparse
import sys
from typing import List
-from _common import video_args, aac_args, add_common, apply_common, default_output, die, emit, ffmpeg_base, info, probe, run, x264_args
+from _common import STATE, video_args, aac_args, add_common, apply_common, default_output, die, emit, ffmpeg_base, info, probe, run, x264_args
TRANSITIONS = ["fade", "dissolve", "wipeleft", "wiperight", "wipeup", "wipedown", "slideleft", "slideright",
"circleopen", "circleclose", "fadeblack", "fadewhite", "smoothleft", "smoothright", "radial", "none"]
@@ -44,17 +44,24 @@ def main() -> int:
if not m.get("video"):
die(f"{p} has no video stream")
first = metas[0]["video"]
- w = args.width or first["width"]
- h = args.height or first["height"]
- if first.get("rotation") in (90, -90, 270, -270) and not (args.width or args.height):
- w, h = h, w
+ fw, fh = first["width"], first["height"]
+ if first.get("rotation") in (90, -90, 270, -270):
+ fw, fh = fh, fw
+ if args.width and args.height:
+ w, h = args.width, args.height
+ elif args.width:
+ w, h = args.width, int(round(args.width * fh / fw))
+ elif args.height:
+ w, h = int(round(args.height * fw / fh)), args.height
+ else:
+ w, h = fw, fh
fps = args.fps or first.get("fps") or 30.0
fps = round(fps) if abs(fps - round(fps)) < 0.02 else fps
w, h = w - (w % 2), h - (h % 2)
durs = [m.get("duration") or 0.0 for m in metas]
d = args.duration if args.transition != "none" else 0.0
for p, dur in zip(args.inputs, durs):
- if d and dur <= d * 2:
+ if d and dur <= d * 2 and not STATE["dry_run"]:
die(f"{p} is only {dur:.2f}s, too short for a {d:.2f}s transition; shorten --duration")
cmd = ffmpeg_base()
diff --git a/scripts/loudness.py b/scripts/loudness.py
index 326e487..c2ec386 100755
--- a/scripts/loudness.py
+++ b/scripts/loudness.py
@@ -18,11 +18,13 @@ import os
import re
import sys
-from _common import add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run
+from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run
def measure(path: str, I: float, tp: float, lra: float) -> dict:
+ if STATE["dry_run"]:
+ return {"input_i": "-20.0", "input_tp": "-3.0", "input_lra": "8.0", "input_thresh": "-30.0", "target_offset": "0.0"}
ffmpeg = require_tool("ffmpeg")
cmd = [ffmpeg, "-hide_banner", "-nostdin", "-i", path, "-vn", "-af", f"loudnorm=I={I}:TP={tp}:LRA={lra}:print_format=json", "-f", "null", "-"]
proc = run(cmd, check=False)
diff --git a/scripts/render.py b/scripts/render.py
new file mode 100755
index 0000000..ff9c62e
--- /dev/null
+++ b/scripts/render.py
@@ -0,0 +1,333 @@
+#!/usr/bin/env python3
+"""Declarative edits: describe the whole edit in one project.json and render it
+in one command. Change a number, re-render. Non-destructive: sources are never
+touched, intermediates live in a work directory.
+
+Project format (all keys optional except clips):
+{
+ "output": "final.mp4",
+ "frame": {"aspect": "9:16", "width": 1080, "fps": 30},
+ "clips": [
+ {"src": "a.mp4", "in": "0:05", "out": "0:20"},
+ {"src": "b.mp4", "in": 3, "out": 12, "speed": 1.25},
+ {"src": "c.mp4"}
+ ],
+ "transition": {"type": "fade", "duration": 0.5},
+ "silence": {"threshold": -38, "min_silence": 0.8},
+ "captions": {"text": "cues.txt", "srt": null, "animate": "pop", "karaoke": true, "font": "Noto Sans CJK JP", "size": 28, "position": "bottom"},
+ "overlays": [
+ {"image": "logo.png", "position": "top-right", "scale": 160, "opacity": 0.9},
+ {"text": "Episode 12", "position": "bottom", "start": 1, "end": 5, "fade": 0.3, "box": true}
+ ],
+ "audio": {"voice": true, "music": "bed.mp3", "music_volume": -16, "duck": true, "fade_out": 2},
+ "loudness": {"lufs": -14, "tp": -1},
+ "fit": {"duration": 60},
+ "export": {"preset": "reels"},
+ "check": {"platform": "reels"}
+}
+
+Stages run in this order: clips (cut) → join → silence → fit → captions →
+overlays → audio → loudness → export → check. Missing stages are skipped.
+
+Examples:
+ python3 render.py --init project.json # write a commented starter project
+ python3 render.py project.json # render
+ python3 render.py project.json --dry-run # show every command without rendering
+ python3 render.py project.json --fast # preview quality
+"""
+import argparse
+import json
+import os
+import subprocess
+import sys
+from pathlib import Path
+from typing import Any, Dict, List
+
+from _common import STATE, add_common, apply_common, die, emit, info, probe
+
+HERE = Path(__file__).resolve().parent
+
+TEMPLATE = {
+ "output": "final.mp4",
+ "frame": {"aspect": "16:9", "width": 1920, "fps": 30},
+ "clips": [{"src": "REPLACE_ME.mp4", "in": "0:00", "out": "0:30"}],
+ "transition": {"type": "fade", "duration": 0.5},
+ "silence": None,
+ "captions": None,
+ "overlays": [],
+ "audio": None,
+ "loudness": {"lufs": -14, "tp": -1},
+ "fit": None,
+ "export": {"preset": "youtube"},
+ "check": {"platform": "youtube"},
+}
+
+
+def sh(script: str, *argv: Any, extra: List[str] = None) -> str:
+ """Run a sibling script, forwarding --fast / --dry-run, returning its printed output path."""
+ cmd = [sys.executable, str(HERE / script)] + [str(a) for a in argv] + (extra or [])
+ if STATE["fast"]:
+ cmd.append("--fast")
+ if STATE["dry_run"]:
+ cmd.append("--dry-run")
+ info("→ " + " ".join(os.path.basename(c) if i < 2 else c for i, c in enumerate(cmd)))
+ proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ for line in proc.stderr.splitlines():
+ if line.startswith("$ ") or line.startswith("[dry-run]"):
+ STATE["commands"].append(line[2:] if line.startswith("$ ") else line)
+ elif line.strip():
+ info(" " + line)
+ if proc.returncode != 0:
+ die(f"{script} failed")
+ out = proc.stdout.strip().splitlines()
+ return out[-1] if out else ""
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("project", nargs="?", help="project.json")
+ ap.add_argument("--init", metavar="FILE", help="write a starter project file and exit")
+ ap.add_argument("--work", help="work directory for intermediates (default: <output>_work)")
+ ap.add_argument("--keep", action="store_true", help="keep intermediates (default: kept only when --work is given)")
+ ap.add_argument("--stop-after", choices=["clips", "join", "silence", "fit", "captions", "overlays", "audio", "loudness", "export"], help="stop after this stage (for iterating)")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ if args.init:
+ Path(args.init).write_text(json.dumps(TEMPLATE, indent=2) + "\n", encoding="utf-8")
+ info(f"wrote {args.init}; edit clips/src and run: render.py {args.init}")
+ print(args.init)
+ return 0
+ if not args.project:
+ die("give a project.json (or --init FILE)")
+ try:
+ proj: Dict[str, Any] = json.loads(Path(args.project).read_text(encoding="utf-8"))
+ except (OSError, ValueError) as exc:
+ die(f"cannot read project: {exc}")
+ base = Path(args.project).resolve().parent
+
+ def rel(p: Any) -> str:
+ p = str(p)
+ return p if os.path.isabs(p) else str(base / p)
+
+ clips = proj.get("clips") or []
+ if not clips:
+ die("project.clips is empty")
+ output = rel(proj.get("output") or "final.mp4")
+ work = Path(args.work) if args.work else Path(str(Path(output).with_suffix("")) + "_work")
+ work.mkdir(parents=True, exist_ok=True)
+ frame = proj.get("frame") or {}
+ trans = proj.get("transition") or {}
+ stages_done: List[str] = []
+
+ # ---- clips
+ parts: List[str] = []
+ for i, c in enumerate(clips):
+ src = rel(c["src"])
+ if not STATE["dry_run"]:
+ probe(src)
+ needs_cut = c.get("in") is not None or c.get("out") is not None
+ part = str(work / f"clip{i:02d}.mp4")
+ if needs_cut:
+ argv: List[Any] = [src, "-o", part, "--accurate"]
+ if c.get("in") is not None:
+ argv += ["--start", c["in"]]
+ if c.get("out") is not None:
+ argv += ["--end", c["out"]]
+ sh("cut.py", *argv)
+ else:
+ part = src
+ if c.get("speed"):
+ spd = float(c["speed"])
+ dur = (probe(part).get("duration") or 0.0) if not STATE["dry_run"] else 10.0
+ fitted = str(work / f"clip{i:02d}_speed.mp4")
+ sh("fit.py", part, "--duration", f"{dur / spd:.3f}", "-o", fitted)
+ part = fitted
+ parts.append(part)
+ stages_done.append("clips")
+ current = parts[0]
+ if args.stop_after == "clips":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- join
+ if len(parts) > 1:
+ current = str(work / "joined.mp4")
+ argv = list(parts) + ["-o", current, "--transition", trans.get("type", "fade"), "--duration", str(trans.get("duration", 0.5))]
+ if frame.get("width"):
+ argv += ["--width", str(frame["width"])]
+ if frame.get("height"):
+ argv += ["--height", str(frame["height"])]
+ if frame.get("fps"):
+ argv += ["--fps", str(frame["fps"])]
+ sh("join.py", *argv)
+ stages_done.append("join")
+ if args.stop_after == "join":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- silence
+ sil = proj.get("silence")
+ if sil:
+ nxt = str(work / "tight.mp4")
+ argv = [current, "-o", nxt]
+ for k, flag in (("threshold", "--threshold"), ("min_silence", "--min-silence"), ("margin", "--margin")):
+ if sil.get(k) is not None:
+ argv += [flag, str(sil[k])]
+ sh("silence.py", *argv)
+ current = nxt
+ stages_done.append("silence")
+ if args.stop_after == "silence":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- fit (duration and/or frame)
+ fit = dict(proj.get("fit") or {})
+ if frame.get("aspect"):
+ fit.setdefault("aspect", frame["aspect"])
+ if frame.get("width") and len(parts) == 1:
+ fit.setdefault("width", frame["width"])
+ if frame.get("fps") and len(parts) == 1:
+ fit.setdefault("fps", frame["fps"])
+ if fit:
+ nxt = str(work / "fit.mp4")
+ argv = [current, "-o", nxt]
+ for k, flag in (("duration", "--duration"), ("method", "--method"), ("aspect", "--aspect"), ("fit", "--fit"), ("width", "--width"), ("fps", "--fps"), ("smooth", "--smooth")):
+ if fit.get(k) is not None:
+ argv += [flag, str(fit[k])]
+ sh("fit.py", *argv)
+ current = nxt
+ stages_done.append("fit")
+ if args.stop_after == "fit":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- captions
+ cap = proj.get("captions")
+ if cap:
+ nxt = str(work / "captioned.mp4")
+ argv = [current, "-o", nxt]
+ if cap.get("text"):
+ argv += ["--text", rel(cap["text"])]
+ elif cap.get("srt"):
+ argv += ["--srt", rel(cap["srt"])]
+ elif cap.get("ass"):
+ argv += ["--ass", rel(cap["ass"])]
+ else:
+ die("captions needs text, srt or ass")
+ for k, flag in (("font", "--font"), ("size", "--size"), ("color", "--color"), ("position", "--position"), ("margin", "--margin"), ("animate", "--animate"), ("highlight_color", "--highlight-color"), ("outline", "--outline")):
+ if cap.get(k) is not None:
+ argv += [flag, str(cap[k])]
+ for k, flag in (("karaoke", "--karaoke"), ("bold", "--bold"), ("box", "--box")):
+ if cap.get(k):
+ argv.append(flag)
+ sh("caption.py", *argv)
+ current = nxt
+ stages_done.append("captions")
+ if args.stop_after == "captions":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- overlays
+ for i, ov in enumerate(proj.get("overlays") or []):
+ nxt = str(work / f"overlay{i:02d}.mp4")
+ argv = [current, "-o", nxt]
+ if ov.get("image"):
+ argv += ["--image", rel(ov["image"])]
+ elif ov.get("text"):
+ argv += ["--text", ov["text"]]
+ else:
+ die(f"overlays[{i}] needs image or text")
+ for k, flag in (("position", "--position"), ("start", "--start"), ("end", "--end"), ("fade", "--fade"), ("opacity", "--opacity"), ("scale", "--scale"), ("font_size", "--font-size"), ("font", "--font"), ("font_file", "--font-file"), ("margin", "--margin")):
+ if ov.get(k) is not None:
+ argv += [flag, str(ov[k])]
+ if ov.get("box"):
+ argv.append("--box")
+ sh("overlay.py", *argv)
+ current = nxt
+ if "overlays" not in stages_done:
+ stages_done.append("overlays")
+ if args.stop_after == "overlays":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- audio
+ au = proj.get("audio")
+ if au:
+ nxt = str(work / "audio.mp4")
+ argv = [current, "-o", nxt]
+ for k, flag in (("music", "--music"), ("replace", "--replace")):
+ if au.get(k):
+ argv += [flag, rel(au[k])]
+ for k, flag in (("music_volume", "--music-volume"), ("fade_in", "--fade-in"), ("fade_out", "--fade-out"), ("gain", "--gain"), ("duck_amount", "--duck-amount")):
+ if au.get(k) is not None:
+ argv += [flag, str(au[k])]
+ for k, flag in (("voice", "--voice"), ("denoise", "--denoise"), ("duck", "--duck"), ("music_loop", "--music-loop"), ("stereo", "--stereo"), ("mono", "--mono"), ("downmix", "--downmix")):
+ if au.get(k):
+ argv.append(flag)
+ sh("audio.py", *argv)
+ current = nxt
+ stages_done.append("audio")
+ if args.stop_after == "audio":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- loudness
+ ld = proj.get("loudness")
+ if ld:
+ nxt = str(work / "loudnorm.mp4")
+ argv = [current, "-o", nxt]
+ if ld.get("lufs") is not None:
+ argv += ["-I", str(ld["lufs"])]
+ if ld.get("tp") is not None:
+ argv += ["--tp", str(ld["tp"])]
+ sh("loudness.py", *argv)
+ current = nxt
+ stages_done.append("loudness")
+ if args.stop_after == "loudness":
+ emit(current, stages=stages_done)
+ return 0
+
+ # ---- export
+ ex = proj.get("export")
+ if ex and ex.get("preset"):
+ argv = [current, "--preset", ex["preset"], "-o", output]
+ if ex.get("fit"):
+ argv += ["--fit", ex["fit"]]
+ if ex.get("crf") is not None:
+ argv += ["--crf", str(ex["crf"])]
+ sh("export.py", *argv)
+ stages_done.append("export")
+ else:
+ if not STATE["dry_run"]:
+ import shutil
+ shutil.copyfile(current, output)
+ info(f"copied final stage to {output}")
+ current = output
+
+ # ---- check
+ ck = proj.get("check")
+ check_result = None
+ if ck and ck.get("platform") and not STATE["dry_run"]:
+ proc = subprocess.run([sys.executable, str(HERE / "check.py"), output, "--platform", ck["platform"], "--json"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ try:
+ check_result = json.loads(proc.stdout)
+ except ValueError:
+ check_result = {"error": proc.stderr.strip()[-300:]}
+ if check_result.get("failed"):
+ info(f"check: {check_result['failed']} FAIL — " + "; ".join(f"{r['check']}={r['value']} ({r['fix']})" for r in check_result["checks"] if r["status"] == "FAIL"))
+ else:
+ info(f"check: OK for {ck['platform']}")
+ stages_done.append("check")
+
+ if not args.keep and not args.work and not STATE["dry_run"]:
+ import shutil
+ shutil.rmtree(work, ignore_errors=True)
+ info(f"rendered {output} via {' → '.join(stages_done)}")
+ emit(output, stages=stages_done, check=check_result)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/scenes.py b/scripts/scenes.py
new file mode 100755
index 0000000..04161e0
--- /dev/null
+++ b/scripts/scenes.py
@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Find scene changes and loud moments, and propose highlight candidates so the
+agent can plan an edit or a digest without watching the whole file.
+
+Scene cuts come from ffmpeg's scdet; energy peaks from a 0.5 s RMS envelope
+of the audio. Highlight candidates are the scenes ranked by audio energy
+(and, optionally, by motion).
+
+Examples:
+ python3 scenes.py talk.mp4 # scenes + peaks, JSON
+ python3 scenes.py event.mp4 --highlights 5 --target 60 # 5 candidate ranges summing to ~60 s
+ python3 scenes.py event.mp4 --highlights 4 --edl picks.txt # cut.py --segments compatible list
+ python3 scenes.py event.mp4 --sheet scenes.png # one thumbnail per scene
+"""
+import argparse
+import math
+import os
+import re
+import struct
+import subprocess
+import sys
+from typing import Dict, List, Tuple
+
+from _common import add_common, apply_common, die, emit, ffmpeg_base, info, print_json, probe, require_tool, run
+
+SCENE_RE = re.compile(r"lavfi\.scd\.time=([0-9.]+)")
+
+
+def detect_scenes(path: str, threshold: float, min_len: float, duration: float) -> List[float]:
+ ffmpeg = require_tool("ffmpeg")
+ proc = subprocess.run([ffmpeg, "-hide_banner", "-nostdin", "-i", path, "-an", "-vf",
+ f"scale=320:-2,scdet=threshold={threshold}:sc_pass=1,metadata=print:file=-", "-f", "null", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ times = [float(t) for t in SCENE_RE.findall(proc.stdout)]
+ cuts = [0.0]
+ for t in times:
+ if t - cuts[-1] >= min_len:
+ cuts.append(t)
+ if duration - cuts[-1] < min_len and len(cuts) > 1:
+ cuts.pop()
+ return cuts
+
+
+def audio_envelope(path: str, step_s: float) -> List[float]:
+ ffmpeg = require_tool("ffmpeg")
+ proc = subprocess.run([ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin", "-i", path, "-vn", "-ac", "1", "-ar", "8000", "-f", "s16le", "-"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ n = len(proc.stdout) // 2
+ if n == 0:
+ return []
+ samples = struct.unpack(f"<{n}h", proc.stdout[: n * 2])
+ step = max(1, int(8000 * step_s))
+ env = []
+ for i in range(0, n, step):
+ block = samples[i:i + step]
+ env.append(math.sqrt(sum(x * x for x in block) / len(block)) / 32768.0)
+ return env
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("input")
+ ap.add_argument("--threshold", type=float, default=10.0, help="scdet threshold 0-100 (default 10; lower = more cuts)")
+ ap.add_argument("--min-scene", type=float, default=1.0, help="ignore cuts closer than this in seconds (default 1)")
+ ap.add_argument("--highlights", type=int, default=0, help="number of highlight ranges to propose")
+ ap.add_argument("--target", type=float, help="with --highlights: total seconds the picks should add up to (trims long scenes)")
+ ap.add_argument("--max-scene", type=float, default=15.0, help="cap a highlight range at this many seconds (default 15)")
+ ap.add_argument("--edl", help="write highlight ranges as START-END lines (cut.py --segments format)")
+ ap.add_argument("--sheet", help="write a contact sheet PNG with the first frame of every scene")
+ add_common(ap)
+ args = ap.parse_args()
+ apply_common(args)
+
+ meta = probe(args.input)
+ if not meta.get("video"):
+ die("input has no video stream")
+ dur = meta.get("duration") or 0.0
+ cuts = detect_scenes(args.input, args.threshold, args.min_scene, dur)
+ bounds = cuts + [dur]
+ step_s = 0.5
+ env = audio_envelope(args.input, step_s) if meta.get("audio") else []
+
+ scenes = []
+ for i in range(len(bounds) - 1):
+ s, e = bounds[i], bounds[i + 1]
+ if e - s <= 0.05:
+ continue
+ seg = env[int(s / step_s): max(int(s / step_s) + 1, int(e / step_s))] if env else []
+ energy = (sum(seg) / len(seg)) if seg else 0.0
+ peak = max(seg) if seg else 0.0
+ scenes.append({"index": len(scenes), "start": round(s, 3), "end": round(e, 3), "duration": round(e - s, 3),
+ "audio_rms": round(energy, 4), "audio_peak": round(peak, 4)})
+ peaks = []
+ if env:
+ thr = sorted(env)[int(len(env) * 0.9)] if len(env) > 10 else max(env)
+ for i, val in enumerate(env):
+ if val >= thr and val > 0.02 and (i == 0 or env[i - 1] < val) and (i == len(env) - 1 or env[i + 1] <= val):
+ peaks.append({"time": round(i * step_s, 2), "rms": round(val, 4)})
+ peaks = sorted(peaks, key=lambda p: -p["rms"])[:20]
+ peaks.sort(key=lambda p: p["time"])
+
+ result: Dict = {"file": args.input, "duration": round(dur, 3), "scene_count": len(scenes), "scenes": scenes, "audio_peaks": peaks}
+ info(f"{len(scenes)} scenes, {len(peaks)} audio peaks over {dur:.1f}s")
+
+ if args.highlights:
+ ranked = sorted(scenes, key=lambda sc: (-sc["audio_rms"], sc["start"]))[: args.highlights]
+ picks: List[Tuple[float, float]] = []
+ budget = args.target if args.target else None
+ per = (budget / max(1, len(ranked))) if budget else args.max_scene
+ for sc in ranked:
+ length = min(sc["duration"], per, args.max_scene)
+ # take the loudest window inside the scene
+ best_s = sc["start"]
+ if env and length < sc["duration"]:
+ best, best_s = -1.0, sc["start"]
+ win = max(1, int(length / step_s))
+ lo, hi = int(sc["start"] / step_s), max(int(sc["start"] / step_s) + 1, int(sc["end"] / step_s) - win)
+ for i in range(lo, hi + 1):
+ val = sum(env[i:i + win])
+ if val > best:
+ best, best_s = val, i * step_s
+ picks.append((round(best_s, 2), round(min(sc["end"], best_s + length), 2)))
+ picks.sort()
+ result["highlights"] = [{"start": s, "end": e, "duration": round(e - s, 2)} for s, e in picks]
+ result["highlights_total"] = round(sum(e - s for s, e in picks), 2)
+ info(f"proposed {len(picks)} highlight ranges totalling {result['highlights_total']:.1f}s")
+ if args.edl:
+ with open(args.edl, "w", encoding="utf-8") as fh:
+ for s, e in picks:
+ fh.write(f"{s:.2f}-{e:.2f}\n")
+ info(f"wrote {args.edl}")
+
+ if args.sheet:
+ n = len(scenes)
+ cols = min(4, max(1, n))
+ rows = max(1, math.ceil(n / cols))
+ tile_w = 1280 // cols // 2 * 2
+ # exactly one frame per scene: the frame index at the scene start
+ fps = meta["video"].get("fps") or 30.0
+ expr = "+".join(f"eq(n\\,{int(round(sc['start'] * fps))})" for sc in scenes)
+ vf = (f"select='{expr}',scale={tile_w}:-2,drawtext=text='%{{pts\\:hms}}':fontcolor=white:fontsize=h/14:box=1:[email protected]:boxborderw=4:x=6:y=6,"
+ f"tile={cols}x{rows}:padding=2:margin=2:color=0x202020")
+ run(ffmpeg_base() + ["-i", args.input, "-vf", vf, "-frames:v", "1", "-fps_mode", "vfr", args.sheet])
+ info(f"wrote {args.sheet}")
+ result["sheet"] = args.sheet
+
+ if args.json:
+ emit(None, **result)
+ else:
+ print_json(result)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tests/test_all.py b/tests/test_all.py
index 331ad10..7421698 100644
--- a/tests/test_all.py
+++ b/tests/test_all.py
@@ -549,6 +549,94 @@ class FFmpegSkillTests(unittest.TestCase):
self.assertLessEqual(data["levels"]["y_max"], 255)
self.assertFalse(data["levels"]["looks_like_log"])
+ # ---------------------------------------------------------------- v0.5: check / scenes / render
+ def test_check_compliance(self):
+ reels = OUT / "export_reels.mp4"
+ if not reels.exists():
+ script("export.py", self.src, "--preset", "reels", "--fit", "crop", "-o", reels)
+ data = json.loads(script("check.py", reels, "--platform", "reels", "--json", expect_fail=True).stdout)
+ names = {r["check"]: r["status"] for r in data["checks"]}
+ self.assertEqual(names["aspect"], "PASS")
+ self.assertEqual(names["pixel format"], "PASS")
+ self.assertEqual(names["loudness"], "FAIL", "unnormalised test tone is far from -14 LUFS")
+ self.assertFalse(data["ok"])
+ # after loudness.py the same file passes
+ norm = OUT / "reels_norm.mp4"
+ script("loudness.py", reels, "-o", norm)
+ data = json.loads(script("check.py", norm, "--platform", "reels", "--json").stdout)
+ self.assertTrue(data["ok"], [r for r in data["checks"] if r["status"] != "PASS"])
+ # HDR on an SDR-only platform fails the colour check
+ data = json.loads(script("check.py", self.hdr, "--platform", "x", "--no-loudness", "--json", expect_fail=True).stdout)
+ self.assertEqual({r["check"]: r["status"] for r in data["checks"]}["colour"], "FAIL")
+ # custom overrides
+ data = json.loads(script("check.py", self.src, "--platform", "custom", "--max-duration", "5", "--no-loudness", "--json", expect_fail=True).stdout)
+ self.assertEqual({r["check"]: r["status"] for r in data["checks"]}["duration"], "FAIL")
+
+ def test_scenes_and_highlights(self):
+ src = OUT / "scenes_src.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
+ "-f", "lavfi", "-i", "testsrc2=size=640x360:rate=30:d=4", "-f", "lavfi", "-i", "smptebars=size=640x360:rate=30:d=4",
+ "-f", "lavfi", "-i", "mandelbrot=size=640x360:rate=30",
+ "-f", "lavfi", "-i", "aevalsrc='0.6*sin(2*PI*440*t)*between(t\\,5\\,7)+0.3*sin(2*PI*330*t)*between(t\\,9\\,10)':s=48000",
+ "-filter_complex", "[2:v]trim=0:4,setpts=PTS-STARTPTS[m];[0:v][1:v][m]concat=n=3:v=1:a=0[v]",
+ "-map", "[v]", "-map", "3:a", "-t", "12", "-c:v", "libx264", "-preset", "veryfast", "-c:a", "aac", src)
+ edl = OUT / "picks.txt"
+ sheet = OUT / "scenes.png"
+ data = json.loads(script("scenes.py", src, "--highlights", "2", "--target", "6", "--edl", edl, "--sheet", sheet, "--json").stdout)
+ self.assertEqual(data["scene_count"], 3)
+ self.assertEqual([round(sc["start"]) for sc in data["scenes"]], [0, 4, 8])
+ self.assertEqual(len(data["highlights"]), 2)
+ self.assertClose(data["highlights_total"], 6.0, 0.6)
+ # the loudest pick must cover the 5-7 s tone
+ self.assertTrue(any(h["start"] <= 5.5 and h["end"] >= 6.5 for h in data["highlights"]), data["highlights"])
+ self.assertTrue(sheet.exists())
+ self.assertGreater(png_size(sheet)[0], 1200, "three tiles across")
+ # EDL feeds cut.py
+ segs = ",".join(edl.read_text().split())
+ out = OUT / "digest.mp4"
+ script("cut.py", src, "--segments", segs, "--accurate", "--fast", "-o", out)
+ self.assertClose(probe(str(out))["duration"], 6.0, 0.6)
+
+ def test_render_project(self):
+ proj = OUT / "project.json"
+ proj.write_text(json.dumps({
+ "output": "render_final.mp4",
+ "frame": {"aspect": "9:16", "width": 720, "fps": 30},
+ "clips": [{"src": "source.mp4", "in": "0:01", "out": "0:05"}, {"src": "source.mp4", "in": 6, "out": 10, "speed": 1.25}],
+ "transition": {"type": "fade", "duration": 0.5},
+ "captions": {"text": "cues.txt", "animate": "pop", "karaoke": True, "size": 26},
+ "overlays": [{"text": "render test", "position": "top-left", "start": 0.5, "end": 3, "fade": 0.3, "box": True}],
+ "audio": {"music": "long_ref.wav", "music_volume": -20, "duck": True, "fade_out": 1},
+ "loudness": {"lufs": -14, "tp": -1},
+ "export": {"preset": "reels"},
+ "check": {"platform": "reels"},
+ }), encoding="utf-8")
+ if not (OUT / "cues.txt").exists():
+ (OUT / "cues.txt").write_text("0:00-0:03 Hello world\n", encoding="utf-8")
+ plan = json.loads(script("render.py", proj, "--dry-run", "--json").stdout)
+ self.assertTrue(plan["dry_run"])
+ self.assertFalse((OUT / "render_final.mp4").exists())
+ data = json.loads(script("render.py", proj, "--fast", "--json").stdout)
+ self.assertEqual(data["stages"], ["clips", "join", "fit", "captions", "overlays", "audio", "loudness", "export", "check"])
+ self.assertTrue(data["check"]["ok"], data["check"])
+ m = probe(str(OUT / "render_final.mp4"))
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), (1080, 1920))
+ self.assertClose(m["duration"], 4 + 3.2 - 0.5, 0.4)
+ self.assertFalse((OUT / "render_final_work").exists(), "work dir removed when not kept")
+ init = OUT / "init.json"
+ script("render.py", "--init", init)
+ self.assertIn("clips", json.loads(init.read_text()))
+ # --stop-after keeps the intermediate
+ out = json.loads(script("render.py", proj, "--fast", "--stop-after", "join", "--work", OUT / "rw", "--json").stdout)
+ self.assertEqual(out["stages"], ["clips", "join"])
+ self.assertTrue(Path(out["output"]).exists())
+
+ def test_join_width_keeps_aspect(self):
+ out = OUT / "join_w.mp4"
+ script("join.py", self.src, self.src, "--transition", "none", "--width", "640", "--fast", "-o", out)
+ m = probe(str(out))["video"]
+ self.assertEqual((m["width"], m["height"]), (640, 360))
+
# ---------------------------------------------------------------- help
def test_every_script_has_help(self):
for name in sorted(p.name for p in SCRIPTS.glob("*.py") if not p.name.startswith("_")):