Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
fit/color/graphics/overlay: stop silently dropping subtitle/data streams (#92)
Shogo Kajiyama11cb2af221a4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 09f9caa..84f61f1 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,52 @@(nothing yet)+## 0.12.1 — Stream-preservation audit: subtitle/data streams no longer silently dropped by picture-only edits++Prompted by an external review pushing back that "feature-complete" for this project now means+proving reliability, not adding tools. Audited all 28 tools against a real fixture carrying+video + audio + subtitle + chapters, by actually running each tool and `ffprobe`-ing its output+rather than reading the code and guessing (an ad hoc pass, not itself checked in as a test). The+checked-in regression test below covers the four tools this pass actually changed, against a+narrower existing fixture (video + audio + two subtitle tracks, no chapters).++- **`fit.py`, `color.py` (`--correct`/`--lut`/`--to-sdr`), `graphics.py`, `overlay.py`: kept the+ source's subtitle/data streams instead of silently dropping them.** Each of these builds an+ explicit `-map` list naming only the video and audio streams it re-encodes; a source with an+ embedded subtitle track (or a data stream) lost it with no signal to the caller, even though+ the operation never touched it. `chapters` already survived regardless (`-map_chapters`+ defaults independently of `-map`) — this was specifically about subtitle/data. Each of the four+ now tries `-map 0:s? -map 0:d? -c:s copy -c:d copy` alongside its existing maps first (a no-op+ via `?` when the source has none), falling back to the original video+audio-only command only+ if that combined attempt fails (e.g. a subtitle codec that can't be stream-copied into a+ changed output container) — the same fallback shape `color.py --retag` already used for this+ in 0.12.0. `--json` gains `dropped_non_av_streams` (`true` only when that fallback was actually+ needed), matching the field name `--retag` already introduced.+- **Deliberately left as-is, tracked in [#91](https://github.com/kajisho5/ffmpeg-skill/issues/91):**+ `caption.py`'s burn mode (whether a pre-existing embedded subtitle should coexist with a newly+ *burned-in* one is a real design question, not a clear-cut preservation fix); `audio.py` when+ its output is a video container; `join.py`/`multicam.py`/`sync.py`, which combine multiple+ separate input files or replace the audio track outright — "which input's subtitle survives"+ has no single correct answer the way a single-input picture/colour edit does, matching the+ existing "different problem shape" carve-out already used for the 0.12.0 `--audio-stream`+ extension. Attachments (`-map 0:t?`, e.g. embedded ASS fonts) are also not covered yet — no+ tool here currently reads/writes ASS-with-fonts end to end, so the risk is theoretical for now.+- Test: `test_picture_only_edits_keep_the_sources_subtitle_streams` runs all four fixed tools+ against `c_subbed.mkv` (two real embedded SRT tracks, already used by other tests) and asserts+ BOTH subtitle tracks survive and `dropped_non_av_streams` is `false`.+- **Two real regressions caught in code review before this shipped, both fixed and covered by a+ new test:** `overlay.py`'s `--image`/`--video` branches combined the pre-existing `-shortest`+ with the new subtitle map -- a subtitle ending before the main video could truncate the WHOLE+ output to the subtitle's length (reproduced: 6s video, 1s subtitle -> 1.04s output). `-shortest`+ is now used only when the source's duration is unknown; the existing `-t <duration>` (already+ there for FFmpeg 7+ precision) is used alone whenever it's known, since it only bounds the main+ input. Separately, `fit.py --method speed` retimes video/audio (`setpts`/`atempo`) but a+ stream-copied subtitle keeps its original timestamps, so it would silently desync from the+ now-faster/slower picture; `fit.py` now skips subtitle preservation specifically when changing+ speed and reports `dropped_non_av_streams: true` honestly instead. `probe()` gains an additive+ `data_streams` count (alongside the existing `subtitle_streams`) so that determination also+ catches a data-only stream with no subtitle track.+## 0.12.0 — 2026-09-07 — Hardening pass: stream/input safety, contract-vs-implementation drift, SKILL.md/eval consistencyA hardening-focused release: no new tools, no new features. Everything here closes a gap betweendiff --git a/docs/contract.md b/docs/contract.mdindex 84417f3..ac74026 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:| Field | Meaning | Changes when ||---|---|---|| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |-| `skill.version` | the npm / package.json version (`0.12.0`) | any release |+| `skill.version` | the npm / package.json version (`0.12.1`) | any release |A release that adds a tool or a flag keeps `contract_version`; a breaking change to theToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`@@ -39,7 +39,7 @@ drifting silently.```json{"contract_version": "1.0",- "skill": {"id": "ffmpeg-skill", "version": "0.12.0", "execution_mode": "local", "kind": "execution",+ "skill": {"id": "ffmpeg-skill", "version": "0.12.1", "execution_mode": "local", "kind": "execution","entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},diff --git a/package.json b/package.jsonindex b55e9c5..9c118ff 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "ffmpeg-skill",- "version": "0.12.0",+ "version": "0.12.1","description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 28 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.","keywords": ["ffmpeg",diff --git a/scripts/_common.py b/scripts/_common.pyindex 45e4dc8..26f3401 100755--- a/scripts/_common.py+++ b/scripts/_common.py@@ -269,6 +269,20 @@ def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True) -> subprreturn _run_captured(list(cmd), check)+def run_keeping_subtitles(cmd: List[str], output: str) -> bool:+ """Run an ffmpeg command that already maps its video/audio, trying first to also+ stream-copy any subtitle/data streams the source has (`-map 0:s?`/`0:d?` are no-ops when+ there are none). A source whose subtitle codec cannot be copied into the target container+ (e.g. a container change) makes that first attempt fail; retry the same command without the+ extra maps rather than let a tool that never touched subtitles start hard-failing because of+ them. `cmd` is the full argv *without* the output path. Returns True only when the+ retry-without-subtitles path was actually needed (i.e. subtitle/data streams were dropped)."""+ if run(cmd + ["-map", "0:s?", "-map", "0:d?", "-c:s", "copy", "-c:d", "copy", output], check=False).returncode == 0:+ return False+ run(cmd + [output])+ return True++def _run_captured(cmd: List[str], check: bool) -> subprocess.CompletedProcess:"""Plain run with stdout/stderr captured."""proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)@@ -387,7 +401,7 @@ def probe(path: str, role: str = "input") -> Dict[str, Any]:return {"file": path, "dry_run": True, "format": None, "duration": 0.0, "size_bytes": 0, "bitrate": None,"video": {"codec": None, "width": 0, "height": 0, "fps": 0.0, "pix_fmt": None, "hdr": False,"color_transfer": None, "color_primaries": None, "rotation": 0, "variable_frame_rate_suspected": False},- "audio": {"codec": None, "channels": 0, "sample_rate": 0}, "subtitle_streams": 0}+ "audio": {"codec": None, "channels": 0, "sample_rate": 0}, "subtitle_streams": 0, "data_streams": 0}die(f"input not found: {path}")ffprobe = require_tool("ffprobe")proc = run(@@ -405,6 +419,7 @@ def probe(path: str, role: str = "input") -> Dict[str, Any]:video = next((s for s in streams if s.get("codec_type") == "video" and s.get("disposition", {}).get("attached_pic", 0) == 0), None)audio = next((s for s in streams if s.get("codec_type") == "audio"), None)subs = [s for s in streams if s.get("codec_type") == "subtitle"]+ data_stream_count = sum(1 for s in streams if s.get("codec_type") in ("data", "attachment"))duration = _to_float(fmt.get("duration"))if duration is None and video:@@ -423,6 +438,7 @@ def probe(path: str, role: str = "input") -> Dict[str, Any]:"video": None,"audio": None,"subtitle_streams": len(subs),+ "data_streams": data_stream_count,# every subtitle stream in file order: index n here is `-map 0:s:n`"subtitle_stream_details": [{"index": n,diff --git a/scripts/color.py b/scripts/color.pyindex eb90d3d..3700921 100755--- a/scripts/color.py+++ b/scripts/color.py@@ -19,7 +19,7 @@ import osimport sysfrom typing import List-from _common import add_common, analyze_levels, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, info, probe, run, x264_args+from _common import add_common, analyze_levels, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, info, probe, run, run_keeping_subtitles, x264_argsTONEMAPS = ["hable", "mobius", "reinhard", "bt2390", "clip", "linear", "gamma"]@@ -212,16 +212,16 @@ def main() -> int:tag = "lut"cmd = ffmpeg_base() + ["-i", args.input, "-vf", vf, "-map", "0:v:0", "-map", f"0:a:{args.audio_stream}?"]- cmd += x264_args(args.crf, args.preset) + cfr_args(meta) + (aac_args() if has_audio else []) + [output]- run(cmd)+ cmd += x264_args(args.crf, args.preset) + cfr_args(meta) + (aac_args() if has_audio else [])+ dropped_streams = run_keeping_subtitles(cmd, output)r = probe(output, role="output")info(f"wrote {output} ({r['duration']:.3f}s, {r['video']['width']}x{r['video']['height']}, "f"{r['video']['color_transfer']}/{r['video']['color_primaries']}, {tag})")+ extra = {"dropped_non_av_streams": dropped_streams}if measurements is not None:measurements["output"] = analyze_levels(output)- emit(output, measurements=measurements)- else:- emit(output)+ extra["measurements"] = measurements+ emit(output, **extra)return 0diff --git a/scripts/fit.py b/scripts/fit.pyindex 08ab169..046c429 100755--- a/scripts/fit.py+++ b/scripts/fit.py@@ -37,7 +37,7 @@ import sysfrom fractions import Fractionfrom typing import List-from _common import video_args, STATE, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, ffmpeg_base, info, parse_time, probe, run, x264_args+from _common import video_args, STATE, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, x264_argsASPECT_PRESETS = {"16:9": Fraction(16, 9), "9:16": Fraction(9, 16), "1:1": Fraction(1, 1), "4:5": Fraction(4, 5), "4:3": Fraction(4, 3), "21:9": Fraction(21, 9)}@@ -225,15 +225,24 @@ def main() -> int:cmd += aac_args()else:cmd += ["-an"]- cmd += post + [output]- run(cmd)+ cmd += post+ if abs(factor - 1.0) > 1e-4:+ # A subtitle/data stream stream-copied by run_keeping_subtitles keeps the source's+ # original timestamps; --method speed retimes video (setpts) and audio (atempo) but has+ # no equivalent way to retime a copied subtitle track, so it would desync from the+ # now-faster/slower picture. Drop them here rather than ship a captions track that lies+ # about when a line is spoken.+ run(cmd + [output])+ dropped_streams = bool(meta.get("subtitle_streams") or meta.get("data_streams"))+ else:+ dropped_streams = run_keeping_subtitles(cmd, output)result = probe(output, role="output")msg = f"wrote {output} ({result['duration']:.3f}s, {result['video']['width']}x{result['video']['height']})"if abs(factor - 1.0) > 1e-4:msg += f", speed {factor:.3f}x"info(msg)- emit(output)+ emit(output, dropped_non_av_streams=dropped_streams)return 0diff --git a/scripts/graphics.py b/scripts/graphics.pyindex 24eab9b..7ffd1b4 100755--- a/scripts/graphics.py+++ b/scripts/graphics.py@@ -21,7 +21,7 @@ import argparseimport sysfrom typing import List, Optional-from _common import aac_args, add_common, apply_common, cfr_args, color_hex, default_output, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, load_brand, parse_time, probe, run, video_args+from _common import aac_args, add_common, apply_common, cfr_args, color_hex, default_output, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, load_brand, parse_time, probe, run, run_keeping_subtitles, video_argsTEMPLATES = ["lower-third", "title", "chapter", "progress", "countdown", "bug"]@@ -163,11 +163,10 @@ def main() -> int:cmd += ["-vf", ",".join(filters), "-map", "0:v:0", "-map", f"0:a:{args.audio_stream}?"]cmd += video_args(meta, args.crf, args.preset) + cfr_args(meta)cmd += aac_args() if meta.get("audio") else ["-an"]- cmd.append(output)- run(cmd)+ dropped_streams = run_keeping_subtitles(cmd, output)r = probe(output, role="output")info(f"wrote {output} ({r['duration']:.3f}s, {args.template})")- emit(output, template=args.template)+ emit(output, template=args.template, dropped_non_av_streams=dropped_streams)return 0diff --git a/scripts/overlay.py b/scripts/overlay.pyindex a94e666..15250b4 100755--- a/scripts/overlay.py+++ b/scripts/overlay.py@@ -24,7 +24,7 @@ import argparseimport sysfrom typing import List, Optional-from _common import STATE, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, x264_args+from _common import STATE, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, x264_argsPOS = {"top-left": ("{m}", "{m}"),@@ -193,12 +193,19 @@ def main() -> int:# -loop 1 turns the still into a timed stream so fade/enable expressions see real timestampscmd = ffmpeg_base() + ["-i", args.input, "-loop", "1", "-i", args.image]fc = f"[1:v]{','.join(chain)},setpts=PTS-STARTPTS[ov];[0:v][ov]{ov}[out]"- cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?", "-shortest"]- # -shortest alone is not exact on FFmpeg 7+: the muxer keeps up to shortest_buf_duration (10 s)- # of the looped still after the video ended, and the file came out 2 s long on 8.1 / 9.0.- # The output must be as long as the main input, so say so explicitly.+ cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?"]if meta.get("duration"):+ # An explicit -t is exact and, unlike -shortest, only bounds the *main* input's+ # streams -- a preserved subtitle/data stream that ends earlier (run_keeping_subtitles)+ # must not be allowed to cut the whole output short via -shortest's "stop at whichever+ # mapped stream finishes first" semantics.cmd += ["-t", f"{meta['duration']:.3f}"]+ else:+ # No known duration to bound by -t (e.g. probe found no video duration): -shortest is+ # the only thing stopping the looped still from running forever. FFmpeg 7+'s+ # shortest_buf_duration slack (up to 10s) is an accepted imprecision here since there is+ # no better bound available.+ cmd += ["-shortest"]elif args.video:pip_meta = probe(args.video)if not pip_meta.get("video"):@@ -219,9 +226,13 @@ def main() -> int:ov += f":enable='{enable}'"cmd = ffmpeg_base() + ["-i", args.input, "-i", args.video]fc = f"[1:v]{','.join(chain)}[ov];[0:v][ov]{ov}[out]"- cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?", "-shortest"]+ cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?"]if meta.get("duration"):+ # See the --image branch above: -t (exact, bounds only the main input) instead of+ # -shortest (would also stop at a preserved subtitle/data stream that ends earlier).cmd += ["-t", f"{meta['duration']:.3f}"]+ else:+ cmd += ["-shortest"]else:x, y = position_exprs(args.position, args.margin, text_mode=True)opts = [f"text='{escape_drawtext(args.text)}'", f"fontsize={args.font_size}", f"x={x}", f"y={y}",@@ -245,12 +256,11 @@ def main() -> int:cmd += video_args(meta, args.crf, args.preset) + cfr_args(meta)cmd += aac_args() if meta.get("audio") else ["-an"]- cmd.append(output)- run(cmd)+ dropped_streams = run_keeping_subtitles(cmd, output)if not STATE.dry_run:result = probe(output, role="output")info(f"wrote {output} ({result['duration']:.3f}s)")- emit(output)+ emit(output, dropped_non_av_streams=dropped_streams)return 0diff --git a/tests/test_contract.py b/tests/test_contract.pyindex ecc726a..7c9e55a 100644--- a/tests/test_contract.py+++ b/tests/test_contract.py@@ -85,6 +85,10 @@ class ContractTests(unittest.TestCase):"-c:v", "copy", "-c:a", "copy", "-c:s", "srt","-metadata:s:s:0", "language=eng", "-metadata:s:s:0", "title=English","-metadata:s:s:1", "language=jpn", "-metadata:s:s:1", "title=Japanese", cls.subbed)+ cls.data_stream = OUT / "c_data_stream.mov"+ ffmpeg("-f", "lavfi", "-i", "testsrc2=size=320x180:rate=30", "-f", "lavfi", "-i", f"aevalsrc='{TONE}':s=48000",+ "-t", "3", "-timecode", "00:00:00:00", "-c:v", "libx264", "-preset", "veryfast", "-pix_fmt", "yuv420p",+ "-c:a", "aac", cls.data_stream)cls.garbage = OUT / "c_garbage.mp4"cls.garbage.write_bytes(bytes((i * 7919) % 256 for i in range(200_000)))cls.empty = OUT / "c_empty.mp4"@@ -359,6 +363,53 @@ class ContractTests(unittest.TestCase):self.assertEqual(meta["video"]["height"], 0)self.assertEqual(meta["video"]["fps"], 0.0)+ def test_picture_only_edits_keep_the_sources_subtitle_streams(self):+ """fit.py/color.py/graphics.py/overlay.py used to build an explicit, selective -map+ list naming only video+audio, silently dropping any subtitle track the source had --+ found by an ad hoc audit of all 28 tools for stream preservation (#91), not itself+ checked in. Each now tries run_keeping_subtitles() first (stream-copies subtitle/data+ alongside the re-encoded picture) before falling back to the original video+audio-only+ command. c_subbed.mkv (built in setUpClass) has two real SRT subtitle tracks; every tool+ below must keep BOTH (a partial loss -- e.g. only one surviving -- is still a real+ regression this asserts against), and --json must report dropped_non_av_streams: false+ since nothing here should need the fallback."""+ cases = [+ ("fit", [self.subbed, "--width", "480", "-o", self.out("keepsub_fit.mkv"), "--json"]),+ ("color", [self.subbed, "--correct", "--exposure", "0.3", "-o", self.out("keepsub_color.mkv"), "--json"]),+ ("graphics", [self.subbed, "--template", "lower-third", "--name", "X", "-o", self.out("keepsub_graphics.mkv"), "--json"]),+ ("overlay", [self.subbed, "--text", "hi", "-o", self.out("keepsub_overlay.mkv"), "--json"]),+ ]+ for name, args in cases:+ doc = json.loads(tool(name, *args).stdout)+ self.assertFalse(doc["dropped_non_av_streams"], name)+ kinds = sh("ffprobe", "-v", "error", "-show_entries", "stream=codec_type", "-of", "csv=p=0", doc["output"]).stdout.split()+ self.assertEqual(kinds.count("subtitle"), 2, f"{name}: expected both source subtitle tracks, got {kinds}")++ def test_overlay_image_keeping_a_short_subtitle_does_not_truncate_the_output(self):+ """Real regression caught in review of #91's fix, before it shipped: overlay.py's+ --image branch used -shortest (needed to stop the looped still running forever) alongside+ -t <duration> (an FFmpeg-7+-precision belt-and-suspenders). Once run_keeping_subtitles()+ also mapped the source's subtitle track, -shortest's "-stop at whichever mapped stream+ ends first" semantics meant a subtitle that ends early (c_subbed.mkv's covers only 0-2s+ of its 6s video) silently truncated the WHOLE output to ~2s -- reproduced directly before+ the fix (6s in, ~1s out). Fixed by only falling back to -shortest when no duration is+ known at all; -t alone (exact, bounds only the main input) is used whenever it is."""+ doc = json.loads(tool("overlay", self.subbed, "--image", self.logo, "-o", self.out("keepsub_overlay_img.mkv"), "--json").stdout)+ result = doc["probe"]+ self.assertGreater(result["duration"], 5.0, f"output was truncated to the subtitle's length: {result['duration']}s")++ def test_fit_speed_change_drops_subtitles_instead_of_desyncing_them(self):+ """A stream-copied subtitle keeps the source's original timestamps; --method speed+ retimes video (setpts) and audio (atempo) but has no way to retime a copied subtitle+ track along with them, so keeping it would silently desync captions from the now-faster+ or -slower picture (caught in review of #91's fix, before it shipped). fit.py must not+ call run_keeping_subtitles() when it's changing speed -- dropping the subtitle track is+ the honest outcome, reported via dropped_non_av_streams: true, not a silently-wrong one."""+ doc = json.loads(tool("fit", self.subbed, "--duration", "2", "-o", self.out("speed_fit.mkv"), "--json").stdout)+ self.assertTrue(doc["dropped_non_av_streams"])+ kinds = sh("ffprobe", "-v", "error", "-show_entries", "stream=codec_type", "-of", "csv=p=0", doc["output"]).stdout+ self.assertNotIn("subtitle", kinds)+def test_changelog_mentions_every_closed_issue_since_last_tag(self):"""A merged fix can land after CHANGELOG.md's current-version section was alreadywritten (this happened for real: #77's fix, PR #88, merged after the 0.12.0 section was@@ -1022,6 +1073,14 @@ class ContractTests(unittest.TestCase):doc2 = self._run_structured("probe", {"inputs": [str(self.src)]})self.assertEqual(doc2["subtitle_streams"], 0)self.assertEqual(doc2["subtitle_stream_details"], [])+ # data_streams (added alongside run_keeping_subtitles(), #91 review): a plain fixture+ # with no data/attachment stream reports 0, same additive-field convention as+ # subtitle_streams above.+ self.assertEqual(doc2["data_streams"], 0)+ # positive case: c_data_stream.mov has a real data stream (a -timecode track, the+ # standard way to get one in mov/mp4) -- confirms detection isn't just "always 0".+ doc3 = self._run_structured("probe", {"inputs": [str(self.data_stream)]})+ self.assertEqual(doc3["data_streams"], 1)def test_color_overlay_caption_export_check_look_render_via_contract(self):doc = self._run_structured("color", {"input": str(self.hdr), "to_sdr": True, "fast": True, "output": str(self.out("sdr.mp4"))})