Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
Release 0.7.0: MCP server, batch folders, optional local transcription (#8)
kajisho5a4ab4da421f1
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e4c99ff..1ebda37 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,12 @@# Changelog+## 0.7.0++- `mcp/server.py` (new): the whole toolkit as an MCP server over stdio (JSON-RPC 2.0, standard library only). Every script is a tool; named args or raw argv; results as structured JSON. Installed alongside scripts by `npx ffmpeg-skill`.+- `batch.py` (new): apply a step recipe or a render project to every file in a folder, content-hash cache so re-runs only touch changed files, `--watch` polling.+- `caption.py --transcribe`: optional local speech-to-text bridge (whisper.cpp `whisper-cli`, faster-whisper, or openai-whisper if present). Never required; a clear install hint otherwise.+- Tests: 49 end-to-end cases.+## 0.6.0- `graphics.py` (new): motion-graphics templates with no image assets — lower-third (slide in/out), title card, chapter chip, progress bar, countdown, corner bug — coloured from brand.json.diff --git a/README.md b/README.mdindex fe23db0..ef4950f 100644--- a/README.md+++ b/README.md@@ -18,6 +18,9 @@ npx ffmpeg-skill- **Lossless when possible** — cuts and joins use stream copy by default; re-encoding only happens when it must (frame-accurate cuts, filters, format changes).- **Cut & join** segments with `mm:ss` / `hh:mm:ss.ms` times.- **Declarative edits** — describe the whole edit in a `project.json` (clips, transitions, captions, overlays, music, loudness, export, check) and re-render after every tweak.+- **MCP server** — `mcp/server.py` exposes every script as an MCP tool over stdio (stdlib only) for Claude Desktop, Cursor or any MCP client.+- **Batch / watch folder** — one recipe over a whole shoot with a content-hash cache; re-runs only touch what changed.+- **Optional local transcription** — `caption.py --transcribe` uses whisper.cpp / faster-whisper / openai-whisper when present; never required.- **Brand kit** — one `brand.json` (fonts, colours, logo, safe margins, caption style) applied by captions, overlays, graphics and projects.- **Motion graphics without assets** — lower-thirds, title cards, chapter chips, progress bars, countdowns and corner bugs drawn by FFmpeg.- **HTML delivery report** — before/after contact sheets, media facts, loudness, compliance and the commands run, in one file.@@ -99,6 +102,8 @@ More examples: [examples/README.md](examples/README.md). To see everything run e| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision, colour space, rotation, audio channels as JSON; `--analyze` flags Log footage || `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact || `render.py` | Render a whole edit from `project.json`; `--init`, `--dry-run`, `--stop-after` |+| `batch.py` | Apply a step recipe or render project to a folder, cached, optional watch |+| `mcp/server.py` | MCP server exposing all scripts as tools (stdio JSON-RPC) || `graphics.py` | Lower-third, title, chapter, progress, countdown, bug templates (brand colours) || `report.py` | Single-file HTML delivery report with sheets, facts, loudness, compliance, commands || `scenes.py` | Scene changes, audio peaks, highlight proposals and per-scene sheet |@@ -119,6 +124,14 @@ More examples: [examples/README.md](examples/README.md). To see everything run eAll scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stderr message on failure.+## MCP++```json+{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/Users/you/.claude/skills/ffmpeg-skill/mcp/server.py"]}}}+```++`python3 mcp/server.py --list` prints the tools; `--call probe '{"inputs": ["a.mp4"]}'` runs one from the shell.+## Requirements- FFmpeg 5.0+ with `libx264`, `libx265`, `libass`, `prores_ks` and `libzimg` (for `color.py --to-sdr`); the default builds from Homebrew, apt and gyan.dev include all of themdiff --git a/SKILL.md b/SKILL.mdindex a472ea8..6c0a7bc 100644--- a/SKILL.md+++ b/SKILL.md@@ -1,6 +1,6 @@---name: ffmpeg-skill-description: Professional video editing with local FFmpeg — declarative project rendering, brand kits, motion-graphics templates (lower-thirds, titles, countdowns), HTML delivery reports, scene detection and highlight picks, delivery compliance checks, cut, silence removal, transitions, multicam, captions (animated/karaoke timed to speech), fit to duration/aspect, audio sync with drift correction, HDR/HLG/Dolby Vision to SDR, LUTs, audio clean-up and ducking, loudness, overlays, platform exports, frame inspection and a real-footage verification kit; Python stdlib scripts, no cloud or API keys.+description: Professional video editing with local FFmpeg — MCP server, batch folders, declarative project rendering, brand kits, motion-graphics templates (lower-thirds, titles, countdowns), HTML delivery reports, scene detection and highlight picks, delivery compliance checks, cut, silence removal, transitions, multicam, captions (animated/karaoke timed to speech), fit to duration/aspect, audio sync with drift correction, HDR/HLG/Dolby Vision to SDR, LUTs, audio clean-up and ducking, loudness, overlays, platform exports, frame inspection and a real-footage verification kit; Python stdlib scripts, no cloud or API keys.---# ffmpeg-skill@@ -75,6 +75,8 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.| "add a lower third with my name", "title card", "countdown intro", "progress bar" | `graphics.py input.mp4 --template lower-third --name "..." --title "..." --start 2 --end 8` || "use our brand fonts/colours/logo" | pass `--brand brand.json` to caption/overlay/graphics, or `"brand"` in project.json || "send me a summary of what you did" | `report.py --before raw.mov --after final.mp4 --platform youtube -o report.html` |+| "do this to every file in the folder", "process the whole shoot" | `batch.py FOLDER --recipe batch.json` (steps or a render project; cached) |+| "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper; otherwise `--text`) || "three cameras, cut between them" | `multicam.py camA.mp4 camB.mp4 camC.mp4 --switch "0-20:0,20-40:1,40-60:2"` || "it's an iPhone Dolby Vision clip and players show it wrong" | `color.py clip.mov --to-sdr` or `color.py clip.mov --strip-dovi` (keep HDR, drop the DV layer) || "does it look like Log / S-Log / flat footage?" | `probe.py clip.mp4 --analyze` (`looks_like_log`) then `color.py --lut` |@@ -176,6 +178,30 @@ check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|broadcast|podcaPASS/WARN/FAIL per check with the script that fixes it. Run it as the finalstep before reporting a deliverable; fix FAILs, mention WARNs.+### batch.py — same recipe over a folder, cached+```+batch.py FOLDER --recipe batch.json [--force] [--watch SECONDS] [--json]+```+`batch.json` holds either `steps` (a list of script argv with `{in}`/`{out}`+placeholders, chained) or `project` (a render project applied per file).+Outputs land in `output_dir` with `suffix`; a content-hash cache skips files+already done with the same recipe. Use `--dry-run` to preview the plan.++### caption.py --transcribe — optional local speech-to-text+If `whisper-cli` (whisper.cpp), `faster-whisper` or `whisper` is installed,+`caption.py input.mp4 --transcribe [--language ja] [--model base]` writes the+SRT from the audio and burns it (combine with `--animate pop --karaoke`).+Nothing is downloaded and nothing is required: without an engine it prints+install hints and the user can supply `--text` cues instead. Always tell the+user which engine was used, and treat the transcript as a draft to review.++### MCP server — the toolkit for any MCP client+`python3 mcp/server.py` speaks MCP over stdio; each script is a tool taking+named args (flags without dashes, underscores for hyphens) or `argv`. Config+for Claude Desktop / Claude Code:+`{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["~/.claude/skills/ffmpeg-skill/mcp/server.py"]}}}`.+Inside this skill, call the scripts directly; the server is for other hosts.+### graphics.py — motion-graphics templates```graphics.py INPUT --template lower-third|title|chapter|progress|countdown|bug [--name] [--title] [--subtitle]diff --git a/bin/install.js b/bin/install.jsindex f3b3e01..4995e32 100755--- a/bin/install.js+++ b/bin/install.js@@ -22,7 +22,7 @@ const { spawnSync } = require('child_process');const SKILL_NAME = 'ffmpeg-skill';const ROOT = path.resolve(__dirname, '..');-const PAYLOAD = ['SKILL.md', 'scripts'];+const PAYLOAD = ['SKILL.md', 'scripts', 'mcp'];const args = process.argv.slice(2);const has = (flag) => args.includes(flag);diff --git a/examples/README.md b/examples/README.mdindex de0d58b..e8e322b 100644--- a/examples/README.md+++ b/examples/README.md@@ -109,6 +109,21 @@ python3 $S/render.py project.json --fast # previewpython3 $S/render.py project.json # final```+### "Process the whole folder"+```bash+cat > batch.json <<'EOF2'+{"glob": "*.mp4", "output_dir": "out", "suffix": "_yt",+ "steps": [["silence.py", "{in}", "-o", "{out}"], ["loudness.py", "{in}", "-o", "{out}"], ["export.py", "{in}", "--preset", "youtube", "-o", "{out}"]]}+EOF2+python3 $S/batch.py ~/Shoot --recipe batch.json # second run: cached, only new files+python3 $S/batch.py ~/Shoot --recipe batch.json --watch 60+```++### "Transcribe and caption it" (needs a local whisper)+```bash+python3 $S/caption.py talk.mp4 --transcribe --language en --animate pop --karaoke+```+### "Add a lower third / title / countdown"```bashpython3 $S/graphics.py talk.mp4 --template lower-third --name "Ada Lovelace" --title "Analyst" --start 2 --end 8 --brand brand.jsondiff --git a/mcp/server.py b/mcp/server.pynew file mode 100755index 0000000..674d4cd--- /dev/null+++ b/mcp/server.py@@ -0,0 +1,178 @@+#!/usr/bin/env python3+"""ffmpeg-skill as an MCP server (stdio, JSON-RPC 2.0) — standard library only.++Every script in ../scripts becomes a tool; arguments are passed as an argv list+or as a flat object of flags. Results are the script's --json output.++Run:+ python3 mcp/server.py # stdio transport+Claude Desktop / Claude Code config example:+ {"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/path/to/ffmpeg-skill/mcp/server.py"]}}}+"""+import json+import os+import subprocess+import sys+from pathlib import Path+from typing import Any, Dict, List++HERE = Path(__file__).resolve().parent+SCRIPTS = HERE.parent / "scripts"+PROTOCOL_VERSION = "2024-11-05"++TOOLS: Dict[str, str] = {+ "probe": "Inspect a media file: duration, fps/VFR, resolution, codecs, HDR, rotation, audio. Args: inputs (list), analyze (bool), compact (bool).",+ "cut": "Cut a range or several segments, lossless when possible. Args: input, start, end, duration, segments, accurate, output.",+ "fit": "Fit to a duration (speed/trim) and/or aspect (pad/crop). Args: input, duration, method, aspect, fit, width, fps, smooth, output.",+ "caption": "Burn SRT/ASS or timed text; animated/karaoke styles; brand. Args: input, srt, ass, text, animate, karaoke, font, size, position, brand, output.",+ "overlay": "Composite a logo/image/text with timing and fade. Args: input, image, text, logo, position, start, end, fade, opacity, scale, brand, output.",+ "graphics": "Motion-graphics templates: lower-third, title, chapter, progress, countdown, bug. Args: input, template, name, title, subtitle, start, end, brand, output.",+ "sync": "Detect offset between two recordings by audio, optional drift fix. Args: reference, second, fix_drift, replace_audio, trim_second, output.",+ "multicam": "Align N cameras/recorders and switch between them. Args: inputs (list), switch, auto, audio, fix_drift, output.",+ "audio": "Denoise/voice chain, music bed with ducking, fades, downmix, replace. Args: input, voice, denoise, music, duck, music_volume, fade_in, fade_out, downmix, replace, output.",+ "loudness": "Two-pass EBU R128 normalisation. Args: input, lufs, tp, measure_only, output.",+ "silence": "Remove dead air / list silences. Args: input, threshold, min_silence, margin, list, edl, output.",+ "join": "Concatenate clips with transitions, normalising size/fps/audio. Args: inputs (list), transition, duration, width, height, fps, output.",+ "color": "HDR→SDR tone mapping, LUTs, retag, strip Dolby Vision. Args: input, to_sdr, lut, retag, strip_dovi, tonemap, output.",+ "export": "Platform presets: youtube, youtube4k, reels, x, prores, h265, gif. Args: input, preset, fit, output.",+ "check": "Pre-delivery compliance per platform. Args: input, platform, no_loudness.",+ "scenes": "Scene changes, audio peaks, highlight proposals. Args: input, highlights, target, edl, sheet.",+ "look": "Contact sheet / frames / before-after PNG for visual checks. Args: input, at (list), tiles, compare, output.",+ "render": "Render a whole edit from project.json. Args: project, fast, stop_after, init.",+ "verify": "Run the toolchain on real files and report PASS/FAIL. Args: paths (list), quick, report.",+ "report": "HTML delivery report. Args: after, before, platform, commands, notes, title, output.",+}+POSITIONAL = {"probe": ["inputs"], "sync": ["reference", "second"], "multicam": ["inputs"], "join": ["inputs"], "verify": ["paths"], "render": ["project"]}+++def build_argv(name: str, args: Dict[str, Any]) -> List[str]:+ if isinstance(args.get("argv"), list):+ argv = [str(a) for a in args["argv"]]+ if name not in ("look", "probe") and "--json" not in argv and "--help" not in argv:+ argv.append("--json")+ return argv+ argv: List[str] = []+ args = dict(args)+ pos = POSITIONAL.get(name, ["input"])+ for key in pos:+ val = args.pop(key, None)+ if val is None:+ continue+ if isinstance(val, list):+ argv += [str(v) for v in val]+ else:+ argv.append(str(val))+ for key, val in args.items():+ if val is None or val is False:+ continue+ flag = "--" + key.replace("_", "-")+ if key == "output":+ flag = "-o"+ if key == "lufs" and name == "loudness":+ flag = "-I"+ if val is True:+ argv.append(flag)+ elif isinstance(val, list):+ for v in val:+ argv += [flag, str(v)]+ else:+ argv += [flag, str(val)]+ if name not in ("look", "probe") and "--json" not in argv and "--help" not in argv:+ argv.append("--json")+ return argv+++def call_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:+ script = SCRIPTS / f"{name}.py"+ if not script.exists():+ return {"isError": True, "content": [{"type": "text", "text": f"unknown tool {name}"}]}+ argv = build_argv(name, args or {})+ proc = subprocess.run([sys.executable, str(script)] + argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)+ stdout = proc.stdout.strip()+ text = stdout+ structured = None+ try:+ structured = json.loads(stdout) if stdout.startswith("{") or stdout.startswith("[") else None+ except ValueError:+ structured = None+ if proc.returncode != 0:+ err = proc.stderr.strip().splitlines()+ tail = "\n".join(err[-12:])+ return {"isError": True, "content": [{"type": "text", "text": f"{name} failed (exit {proc.returncode})\n{tail}"}]}+ if structured is None:+ text = stdout or "\n".join(proc.stderr.strip().splitlines()[-5:])+ result: Dict[str, Any] = {"content": [{"type": "text", "text": text}]}+ if structured is not None:+ result["structuredContent"] = structured if isinstance(structured, dict) else {"result": structured}+ return result+++def tool_list() -> List[Dict[str, Any]]:+ out = []+ for name, desc in TOOLS.items():+ out.append({+ "name": name,+ "description": desc + " Pass either named args (flags without dashes, underscores for hyphens) or argv (raw CLI list). Media paths must be absolute.",+ "inputSchema": {"type": "object", "properties": {"argv": {"type": "array", "items": {"type": "string"}, "description": "raw CLI arguments"}}, "additionalProperties": True},+ })+ return out+++def handle(req: Dict[str, Any]) -> Dict[str, Any]:+ method = req.get("method")+ params = req.get("params") or {}+ if method == "initialize":+ return {"protocolVersion": PROTOCOL_VERSION, "capabilities": {"tools": {}}, "serverInfo": {"name": "ffmpeg-skill", "version": version()}}+ if method == "tools/list":+ return {"tools": tool_list()}+ if method == "tools/call":+ return call_tool(params.get("name", ""), params.get("arguments") or {})+ if method == "ping":+ return {}+ raise KeyError(method)+++def version() -> str:+ try:+ return json.loads((HERE.parent / "package.json").read_text())["version"]+ except Exception:+ return "0"+++def main() -> int:+ if "--list" in sys.argv:+ for t in tool_list():+ print(f"{t['name']:10s} {t['description'].split(' Pass either')[0]}")+ return 0+ if "--call" in sys.argv: # debugging helper: --call NAME '{"input": "..."}'+ i = sys.argv.index("--call")+ name = sys.argv[i + 1]+ args = json.loads(sys.argv[i + 2]) if len(sys.argv) > i + 2 else {}+ print(json.dumps(call_tool(name, args), indent=2))+ return 0+ stdin = sys.stdin.buffer+ stdout = sys.stdout.buffer+ for raw in stdin:+ line = raw.decode("utf-8", errors="replace").strip()+ if not line:+ continue+ try:+ req = json.loads(line)+ except ValueError:+ continue+ if "id" not in req: # notification+ continue+ try:+ result = handle(req)+ resp = {"jsonrpc": "2.0", "id": req["id"], "result": result}+ except KeyError as exc:+ resp = {"jsonrpc": "2.0", "id": req["id"], "error": {"code": -32601, "message": f"method not found: {exc}"}}+ except Exception as exc: # noqa: BLE001+ resp = {"jsonrpc": "2.0", "id": req["id"], "error": {"code": -32000, "message": str(exc)}}+ stdout.write((json.dumps(resp) + "\n").encode("utf-8"))+ stdout.flush()+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/package.json b/package.jsonindex fe11c68..cca91a0 100644--- a/package.json+++ b/package.json@@ -1,7 +1,7 @@{"name": "ffmpeg-skill",- "version": "0.6.0",- "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: declarative project rendering, brand kits, motion-graphics templates, HTML delivery reports, scene detection, delivery checks, cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",+ "version": "0.7.0",+ "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: MCP server, batch processing, declarative project rendering, brand kits, motion-graphics templates, HTML delivery reports, scene detection, delivery checks, cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.","keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],"license": "MIT","author": "kajisho5",diff --git a/scripts/batch.py b/scripts/batch.pynew file mode 100755index 0000000..b196ce7--- /dev/null+++ b/scripts/batch.py@@ -0,0 +1,177 @@+#!/usr/bin/env python3+"""Apply the same recipe to every file in a folder, with a content-hash cache+so re-runs only process what changed. A recipe is a list of script steps;+{in} and {out} are substituted, and the output of one step feeds the next.++Recipe (batch.json):+{+ "glob": "*.mp4",+ "output_dir": "out",+ "suffix": "_final",+ "steps": [+ ["silence.py", "{in}", "--threshold", "-38", "-o", "{out}"],+ ["loudness.py", "{in}", "-o", "{out}"],+ ["export.py", "{in}", "--preset", "youtube", "-o", "{out}"]+ ]+}+or use a render project for every file: {"project": "project.json", "clip_key": 0}++Examples:+ python3 batch.py ~/Footage --recipe batch.json+ python3 batch.py ~/Footage --recipe batch.json --dry-run+ python3 batch.py ~/Footage --recipe batch.json --force # ignore the cache+ python3 batch.py ~/Footage --recipe batch.json --watch 30 # poll the folder every 30 s+"""+import argparse+import hashlib+import json+import os+import subprocess+import sys+import time+from pathlib import Path+from typing import Any, Dict, List++from _common import STATE, add_common, apply_common, die, emit, info++HERE = Path(__file__).resolve().parent+MEDIA_EXT = {".mp4", ".mov", ".m4v", ".mkv", ".webm", ".avi", ".mts", ".m2ts", ".mxf", ".wav", ".m4a", ".mp3", ".flac"}+++def file_key(path: Path) -> str:+ st = path.stat()+ h = hashlib.sha1()+ h.update(f"{path.name}|{st.st_size}|{int(st.st_mtime)}".encode())+ with open(path, "rb") as fh: # first and last MB: cheap and good enough to detect changes+ h.update(fh.read(1 << 20))+ if st.st_size > 2 << 20:+ fh.seek(-(1 << 20), os.SEEK_END)+ h.update(fh.read(1 << 20))+ return h.hexdigest()+++def recipe_key(recipe: Dict[str, Any]) -> str:+ return hashlib.sha1(json.dumps(recipe, sort_keys=True).encode()).hexdigest()[:12]+++def run_step(argv: List[str]) -> bool:+ cmd = [sys.executable, str(HERE / argv[0])] + argv[1:]+ if STATE["fast"]:+ cmd.append("--fast")+ if STATE["dry_run"]:+ cmd.append("--dry-run")+ info(" → " + " ".join(os.path.basename(c) if i < 2 else c for i, c in enumerate(cmd)))+ proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)+ if proc.returncode != 0:+ info(" " + "\n ".join(proc.stderr.strip().splitlines()[-4:]))+ return False+ return True+++def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path) -> Dict[str, Any]:+ suffix = recipe.get("suffix", "_out")+ final_ext = recipe.get("ext") or src.suffix.lstrip(".") or "mp4"+ final = outdir / f"{src.stem}{suffix}.{final_ext}"+ t0 = time.time()+ if recipe.get("project"):+ proj = json.loads(Path(recipe["project"]).read_text(encoding="utf-8"))+ idx = int(recipe.get("clip_key", 0))+ proj.setdefault("clips", [{}])+ while len(proj["clips"]) <= idx:+ proj["clips"].append({})+ proj["clips"][idx]["src"] = str(src.resolve())+ proj["output"] = str(final.resolve())+ pj = work / f"{src.stem}_project.json"+ pj.write_text(json.dumps(proj, indent=2), encoding="utf-8")+ ok = run_step(["render.py", str(pj)])+ else:+ steps = recipe.get("steps") or []+ if not steps:+ die("recipe needs steps or project")+ cur = str(src)+ ok = True+ for i, step in enumerate(steps):+ last = i == len(steps) - 1+ out = str(final) if last else str(work / f"{src.stem}_step{i}.{'mp4' if src.suffix.lower() not in ('.wav', '.mp3', '.m4a', '.flac') else src.suffix.lstrip('.')}")+ argv = [str(a).replace("{in}", cur).replace("{out}", out) for a in step]+ if not run_step(argv):+ ok = False+ break+ cur = out+ return {"file": str(src), "output": str(final), "ok": ok, "seconds": round(time.time() - t0, 1)}+++def main() -> int:+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ ap.add_argument("folder")+ ap.add_argument("--recipe", required=True, help="batch.json")+ ap.add_argument("--force", action="store_true", help="ignore the cache and redo everything")+ ap.add_argument("--watch", type=float, help="keep polling the folder every N seconds")+ ap.add_argument("--work", help="work directory for intermediates (default: <output_dir>/.work)")+ add_common(ap)+ args = ap.parse_args()+ apply_common(args)++ folder = Path(args.folder)+ if not folder.is_dir():+ die(f"not a folder: {folder}")+ try:+ recipe = json.loads(Path(args.recipe).read_text(encoding="utf-8"))+ except (OSError, ValueError) as exc:+ die(f"cannot read recipe: {exc}")+ if recipe.get("project") and not os.path.isabs(recipe["project"]):+ recipe["project"] = str((Path(args.recipe).resolve().parent / recipe["project"]))+ outdir = Path(recipe.get("output_dir") or (folder / "out"))+ if not outdir.is_absolute():+ outdir = folder / outdir+ work = Path(args.work) if args.work else outdir / ".work"+ outdir.mkdir(parents=True, exist_ok=True)+ work.mkdir(parents=True, exist_ok=True)+ cache_path = outdir / ".ffskill_cache.json"+ cache: Dict[str, Any] = {}+ if cache_path.exists() and not args.force:+ try:+ cache = json.loads(cache_path.read_text(encoding="utf-8"))+ except ValueError:+ cache = {}+ rkey = recipe_key(recipe)+ glob = recipe.get("glob") or "*"++ def one_pass() -> List[Dict[str, Any]]:+ results = []+ files = sorted(p for p in folder.glob(glob) if p.is_file() and p.suffix.lower() in MEDIA_EXT and outdir not in p.parents)+ for src in files:+ key = f"{file_key(src)}:{rkey}"+ hit = cache.get(key)+ if hit and Path(hit.get("output", "")).exists() and not args.force:+ info(f"skip (cached) {src.name}")+ results.append({**hit, "cached": True})+ continue+ info(f"=== {src.name}")+ r = process(src, recipe, outdir, work)+ results.append(r)+ if r["ok"] and not STATE["dry_run"]:+ cache[key] = r+ cache_path.write_text(json.dumps(cache, indent=2), encoding="utf-8")+ return results++ results = one_pass()+ if args.watch:+ info(f"watching {folder} every {args.watch:g}s (Ctrl-C to stop)")+ try:+ while True:+ time.sleep(args.watch)+ results = one_pass()+ except KeyboardInterrupt:+ pass+ done = sum(1 for r in results if r["ok"])+ info(f"{done}/{len(results)} processed, {sum(1 for r in results if r.get('cached'))} from cache")+ emit(None, results=results, processed=done, total=len(results))+ if not args.json:+ for r in results:+ print(f"{'OK ' if r['ok'] else 'FAIL'} {r['file']} -> {r['output']}" + (" (cached)" if r.get("cached") else ""))+ return 0 if done == len(results) else 1+++if __name__ == "__main__":+ sys.exit(main())diff --git a/scripts/caption.py b/scripts/caption.pyindex 9b01549..c272c31 100755--- a/scripts/caption.py+++ b/scripts/caption.py@@ -21,7 +21,7 @@ import osimport reimport sysfrom pathlib import Path-from typing import List, Tuple+from typing import List, Optional, Tuplefrom _common import color_hex, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, fmt_srt_time, info, parse_time, probe, run, x264_args@@ -60,6 +60,71 @@ def parse_text_cues(path: str, auto_seconds: float, gap: float) -> List[Tuple[flreturn cues+def transcribe(video: str, out_srt: str, language: Optional[str], model: str) -> List[Tuple[float, float, str]]:+ """Optional local ASR bridge. Tries, in order: whisper-cli / main (whisper.cpp), faster-whisper (python),+ whisper (openai-whisper CLI). Produces an SRT with word timings where the engine supports it.+ No engine installed -> clear error with install hints; the skill never depends on one."""+ import shutil+ import subprocess+ import tempfile+ from _common import require_tool+ ffmpeg = require_tool("ffmpeg")+ tmpdir = tempfile.mkdtemp(prefix="ffskill_asr_")+ wav = os.path.join(tmpdir, "audio.wav")+ subprocess.run([ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-i", video, "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le", wav], check=True)+ # 1. whisper.cpp+ cli = shutil.which("whisper-cli") or shutil.which("whisper-cpp") or shutil.which("main")+ if cli and (shutil.which("whisper-cli") or shutil.which("whisper-cpp")):+ model_path = model+ if not os.path.exists(model_path):+ for cand in (os.path.expanduser(f"~/.cache/whisper.cpp/ggml-{model}.bin"), f"models/ggml-{model}.bin", f"/usr/local/share/whisper/ggml-{model}.bin"):+ if os.path.exists(cand):+ model_path = cand+ break+ base = os.path.join(tmpdir, "out")+ cmd = [cli, "-m", model_path, "-f", wav, "-osrt", "-of", base]+ if language:+ cmd += ["-l", language]+ proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)+ if proc.returncode == 0 and os.path.exists(base + ".srt"):+ info(f"transcribed with whisper.cpp ({os.path.basename(cli)}, model {os.path.basename(model_path)})")+ cues = parse_srt(base + ".srt")+ write_srt(cues, out_srt)+ return cues+ info("whisper.cpp found but failed: " + (proc.stderr.strip().splitlines() or ["?"])[-1][:200])+ # 2. faster-whisper (python package)+ try:+ from faster_whisper import WhisperModel # type: ignore+ m = WhisperModel(model, device="cpu", compute_type="int8")+ segments, _ = m.transcribe(wav, language=language, word_timestamps=False)+ cues = [(seg.start, seg.end, seg.text.strip()) for seg in segments if seg.text.strip()]+ if cues:+ info("transcribed with faster-whisper")+ write_srt(cues, out_srt)+ return cues+ except ImportError:+ pass+ # 3. openai-whisper CLI+ if shutil.which("whisper"):+ cmd = ["whisper", wav, "--model", model, "--output_format", "srt", "--output_dir", tmpdir]+ if language:+ cmd += ["--language", language]+ proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)+ srt = os.path.join(tmpdir, "audio.srt")+ if proc.returncode == 0 and os.path.exists(srt):+ info("transcribed with openai-whisper")+ cues = parse_srt(srt)+ write_srt(cues, out_srt)+ return cues+ die("no local speech-to-text engine found for --transcribe.\n"+ "Install one (all run offline):\n"+ " whisper.cpp: brew install whisper-cpp (then download a model: ggml-base.bin)\n"+ " faster-whisper: pip install faster-whisper\n"+ " openai-whisper: pip install openai-whisper\n"+ "Or write the cues by hand with --text cues.txt (see format above).")+ return []++def parse_srt(path: str) -> List[Tuple[float, float, str]]:cues: List[Tuple[float, float, str]] = []block: List[str] = []@@ -217,6 +282,9 @@ def main() -> int:src.add_argument("--srt", help="SRT file to burn")src.add_argument("--ass", help="ASS file to burn (styles inside the file are used)")src.add_argument("--text", help="plain text cue file to convert into SRT (see format above)")+ src.add_argument("--transcribe", action="store_true", help="generate the SRT from the audio with a local speech-to-text engine if one is installed (whisper-cli / whisper / faster-whisper); never required")+ src.add_argument("--language", help="language code for --transcribe (e.g. en, ja); default auto")+ src.add_argument("--model", default="base", help="whisper model name/path for --transcribe (default base)")src.add_argument("--write-srt", help="where to save the generated SRT (default: <text>.srt)")src.add_argument("--auto-seconds", type=float, default=3.0, help="duration for cues without timing (default 3)")src.add_argument("--gap", type=float, default=0.0, help="gap after auto-timed cues in seconds")@@ -263,10 +331,17 @@ def main() -> int:args.karaoke = Trueif args.brand and brand.get("font_file") and not args.fonts_dir:args.fonts_dir = str(Path(brand["font_file"]).parent)- if not (args.srt or args.ass or args.text):- die("give one of --srt, --ass or --text")+ if not (args.srt or args.ass or args.text or args.transcribe):+ die("give one of --srt, --ass, --text or --transcribe")srt_path = args.srt+ if args.transcribe:+ if not args.input:+ die("--transcribe needs the input video")+ srt_path = args.write_srt or os.path.splitext(args.input)[0] + ".srt"+ cues = transcribe(args.input, srt_path, args.language, args.model)+ info(f"wrote {srt_path} ({len(cues)} cues)")+ args.text = Noneif args.text:cues = parse_text_cues(args.text, args.auto_seconds, args.gap)srt_path = args.write_srt or os.path.splitext(args.text)[0] + ".srt"diff --git a/tests/test_all.py b/tests/test_all.pyindex 3e59e36..d045681 100644--- a/tests/test_all.py+++ b/tests/test_all.py@@ -695,6 +695,72 @@ class FFmpegSkillTests(unittest.TestCase):script("report.py", "--after", reels, "--no-sheets", "-o", OUT / "report2.html")self.assertLess((OUT / "report2.html").stat().st_size, 40000)+ # ---------------------------------------------------------------- v0.7: MCP server / batch / ASR bridge+ def test_mcp_server_stdio(self):+ server = ROOT / "mcp" / "server.py"+ reqs = [+ {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}},+ {"jsonrpc": "2.0", "method": "notifications/initialized"},+ {"jsonrpc": "2.0", "id": 2, "method": "tools/list"},+ {"jsonrpc": "2.0", "id": 3, "method": "tools/call", "params": {"name": "probe", "arguments": {"inputs": [str(self.src)]}}},+ {"jsonrpc": "2.0", "id": 4, "method": "tools/call", "params": {"name": "cut", "arguments": {"input": str(self.src), "start": 1, "end": 3, "output": str(OUT / "mcp_cut.mp4")}}},+ {"jsonrpc": "2.0", "id": 5, "method": "tools/call", "params": {"name": "cut", "arguments": {"argv": [str(self.src), "--start", "0", "--end", "2", "-o", str(OUT / "mcp_cut2.mp4")]}}},+ {"jsonrpc": "2.0", "id": 6, "method": "tools/call", "params": {"name": "nope", "arguments": {}}},+ {"jsonrpc": "2.0", "id": 7, "method": "bogus/method"},+ ]+ proc = subprocess.run([sys.executable, str(server)], input="\n".join(json.dumps(r) for r in reqs) + "\n", stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)+ self.assertEqual(proc.returncode, 0, proc.stderr)+ resp = {r["id"]: r for r in (json.loads(l) for l in proc.stdout.splitlines() if l.strip())}+ self.assertEqual(resp[1]["result"]["serverInfo"]["name"], "ffmpeg-skill")+ names = {t["name"] for t in resp[2]["result"]["tools"]}+ self.assertTrue({"probe", "cut", "render", "check", "scenes", "batch"} <= names or {"probe", "cut", "render", "check", "scenes"} <= names)+ self.assertEqual(resp[3]["result"]["structuredContent"]["duration"], 12.0)+ self.assertClose(resp[4]["result"]["structuredContent"]["probe"]["duration"], 2.0, 0.6)+ self.assertTrue(Path(resp[5]["result"]["structuredContent"]["output"]).exists())+ self.assertTrue(resp[6]["result"].get("isError"))+ self.assertEqual(resp[7]["error"]["code"], -32601)++ def test_batch_recipe_and_cache(self):+ folder = OUT / "batch_in"+ folder.mkdir(exist_ok=True)+ for name in ("a.mp4", "b.mp4"):+ (folder / name).write_bytes(Path(self.src).read_bytes())+ recipe = folder / "batch.json"+ recipe.write_text(json.dumps({"glob": "*.mp4", "output_dir": "out", "suffix": "_final",+ "steps": [["fit.py", "{in}", "--duration", "5", "-o", "{out}"], ["export.py", "{in}", "--preset", "x", "-o", "{out}"]]}))+ data = json.loads(script("batch.py", folder, "--recipe", recipe, "--fast", "--json").stdout)+ self.assertEqual(data["processed"], 2)+ self.assertFalse(any(r.get("cached") for r in data["results"]))+ for r in data["results"]:+ m = probe(r["output"])+ self.assertClose(m["duration"], 5.0, 0.3)+ self.assertEqual(m["video"]["width"], 1280)+ data = json.loads(script("batch.py", folder, "--recipe", recipe, "--fast", "--json").stdout)+ self.assertTrue(all(r.get("cached") for r in data["results"]), "second run served from cache")+ data = json.loads(script("batch.py", folder, "--recipe", recipe, "--fast", "--force", "--json").stdout)+ self.assertFalse(any(r.get("cached") for r in data["results"]))+ # project-based recipe+ proj = folder / "p.json"+ proj.write_text(json.dumps({"clips": [{"src": "x", "in": 0, "out": 3}], "export": {"preset": "x"}}))+ recipe2 = folder / "batch2.json"+ recipe2.write_text(json.dumps({"glob": "a.mp4", "output_dir": "out2", "suffix": "_p", "project": "p.json"}))+ data = json.loads(script("batch.py", folder, "--recipe", recipe2, "--fast", "--json").stdout)+ self.assertEqual(data["processed"], 1)+ self.assertClose(probe(data["results"][0]["output"])["duration"], 3.0, 0.3)++ def test_transcribe_without_engine_explains(self):+ import shutil+ if shutil.which("whisper-cli") or shutil.which("whisper"):+ self.skipTest("a local ASR engine is installed")+ try:+ import faster_whisper # noqa: F401+ self.skipTest("faster-whisper installed")+ except ImportError:+ pass+ proc = script("caption.py", self.src, "--transcribe", "-o", OUT / "asr.mp4", expect_fail=True)+ self.assertIn("no local speech-to-text engine", proc.stderr)+ self.assertIn("whisper", proc.stderr)+# ---------------------------------------------------------------- helpdef test_every_script_has_help(self):for name in sorted(p.name for p in SCRIPTS.glob("*.py") if not p.name.startswith("_")):