Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

fix: settle the ffmpeg exit-code and subtitle-track policies (#191)

claude[bot]43e994c5f492
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9b62abb..ab1ea82 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,7 @@
## Unreleased
-(nothing yet)
+- Policies settled after the boundary sweep. An ffmpeg failure now always exits 1 (ffmpeg's own code, which varied by build and stage, is `ffmpeg_returncode` in the JSON). `audio.py`, `loudness.py` and `proxy.py` keep a subtitle/data track the container can hold, as `fit`/`color`/`overlay` do; `join.py`, `broll.py` and `speedramp.py`, whose timeline moves, report `dropped_non_av_streams: true` when the source had one. The rule is written down in `docs/design-decisions.md`.
## 1.4.14
diff --git a/docs/contract.md b/docs/contract.md
index a6910c8..f32621a 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -46,7 +46,7 @@ For the whole of 1.x:
| Tool ids (`ffmpeg-skill/<name>`) and script names | never removed or renamed |
| CLI arguments (`argparse` dests, flags, positionals) | never removed, renamed, or made newly required; new optional arguments may be added |
| `--json` output keys, and the keys of `contract --json` / `doctor --json` | never removed or given a different type; new keys may be added |
-| Exit codes (0 success, 1 failure, 2 unknown/undecidable in `doctor`) | unchanged |
+| Exit codes (0 success, 1 failure incl. ffmpeg failures, 2 unknown/undecidable in `doctor`, 124 timeout, 127 missing tool, 128+signal interrupted) | unchanged |
| `contract_version` (`1.0`) | unchanged; a ToolSpec shape change is a major |
| MCP `tools/list` names and `inputSchema` property names | derived from the above, so covered by the same promise |
| Behaviour of a tool for the same input and arguments | may change only to fix a defect or to track an FFmpeg change, and every such change gets a CHANGELOG line |
@@ -318,7 +318,8 @@ before, and, when `--json` was given, on stdout:
`message` carries the script's own reason (missing input, ffprobe failure, the last
stderr lines of ffmpeg, the verification that failed); an optional `error.hint` names the
flag change that would make a retry meaningful (never a diagnosis of the media); `commands` lists what was planned
-or run so the caller can retry or report without re-deriving the command. `code` is a
+or run so the caller can retry or report without re-deriving the command; `kind: ffmpeg`
+failures add `ffmpeg_returncode` (ffmpeg's own exit code; the process exits 1). `code` is a
purely additive, statically-mapped relabelling of `kind` (never a new distinction `kind`
doesn't already make) for a caller that wants a stable enum instead of matching `kind`
strings. `retryable` is currently always `false`: none of the kinds are distinguishable
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index f7ae023..91bb7d3 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -63,6 +63,14 @@ exists. When a decision changes, edit the entry in the same PR.
success**, so a failed run never costs the caller the file that was there (FFmpeg 5.x truncates
the output before a filter error). The temp name `.<stem>.ffskill-<pid><ext>` is expected in the
output directory during a run. Code: `_common._stage_existing_output()`.
+- **Subtitle/data tracks are kept where the picture's timeline is untouched and dropped where
+ it is retimed; both cases report `dropped_non_av_streams`.** Tools that re-encode the picture
+ or audio in place (`fit`, `color`, `graphics`, `overlay`, `audio`, `loudness`, `proxy`,
+ `deinterlace`, `denoise`, `freeze --mode extend`) try `run_keeping_subtitles()` first and
+ report `false` unless the container refused the track. Tools that move the timeline
+ (`cut` re-encode, `pad --start`, `freeze --mode insert`, `fit --method speed`, `speedramp`,
+ `broll`, `join`) do not copy a track whose cues would fire at the wrong time and report
+ `true` when the source had one. Code: `_common.run_keeping_subtitles()` and each tool's `emit`.
- **`caption.py` leaves its `.srt`/`.ass` sidecar next to the output by design**; it is a
deliverable (the subtitle file), not an intermediate.
- **`examples/out/` and `tests/out/` are not tracked**; local demo output can be large. Nothing
@@ -90,6 +98,11 @@ exists. When a decision changes, edit the entry in the same PR.
release workflow with a PAT (the ruleset blocks the built-in token) and must not trigger a
second run. Never quote that marker in a PR body: a squash merge copies it into the merge
commit and skips every workflow (`references/process-pitfalls.md`).
+- **An ffmpeg failure exits 1, whatever ffmpeg's own exit code was.** ffmpeg's code varies by
+ build and by the failing stage (1, 69, 218, 234, a negative signal number), and 124/127/
+ 130/143 are reserved for timeout, missing tool and interrupts; passing the raw code through
+ made the process exit code depend on the ffmpeg build. The raw code is in the JSON failure
+ document as `ffmpeg_returncode`. Code: `_common._fail()`.
- **`retryable` is always `false` in failure documents.** No failure kind is distinguishable
today from a deterministic one that would fail identically on a blind retry, so the field never
invites a retry loop. Code: `_common.ERROR_RETRYABLE`.
diff --git a/references/scripts.md b/references/scripts.md
index c2fe796..c631d54 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -281,6 +281,8 @@ rather than a continuous curve. `START`/`END` take seconds or `mm:ss`;
0.5 = half speed). Picking exactly where a ramp should ease in or out is a
judgement call for the calling agent, made concrete here as the segment
boundaries it supplies.
+A subtitle/data track in the source is not carried into the retimed/concatenated
+output; the result says so with `dropped_non_av_streams: true`.
### loop.py — repeat a clip
```
@@ -308,6 +310,8 @@ overlap or run past A's end, and B must have enough material from `--from`.
`--audio a` (default) keeps A's audio untouched and stream-copied; `b` replaces
it inside each window with B's; `mix` plays both. The output's length is
verified against A's.
+A subtitle/data track in the source is not carried into the retimed/concatenated
+output; the result says so with `dropped_non_av_streams: true`.
### metadata.py — chapter markers and container tags, streams copied
```
@@ -362,6 +366,8 @@ channel layout (the widest clip's -- a 5.1 clip keeps 5.1 -- or `--channels`;
silent track generated for clips without audio), then chains `xfade` +
`acrossfade`. Output length = sum of clips − transition × (n−1). Clips must be
longer than 2 × the transition. Use `--transition none` for a plain cut.
+A subtitle/data track in the source is not carried into the retimed/concatenated
+output; the result says so with `dropped_non_av_streams: true`.
### render.py — the whole edit in one project.json
```
@@ -596,7 +602,8 @@ standard talking-head chain. `--duck` uses a sidechain compressor keyed by the
speech so music dips under dialogue and swells in pauses. `--downmix` uses the
ITU centre/LFE weights for 5.1/7.1 → stereo. `--mono` averages a stereo pair,
leaves a 1-channel input untouched and downmixes >2 channels through
-swresample. Video is always stream-copied.
+swresample. Video is always stream-copied, and so is a subtitle/data track
+when the container can hold it (`dropped_non_av_streams` says when it could not).
Run `loudness.py` after this for final levels.
### loudness.py — EBU R128 normalisation
@@ -604,7 +611,8 @@ Run `loudness.py` after this for final levels.
loudness.py INPUT [-I -14] [--tp -1] [--lra 11] [--measure-only] [-o OUT]
```
Two-pass `loudnorm`: measure, then apply with measured values (linear mode when
-the true-peak ceiling allows). Video is stream-copied; audio becomes AAC in
+the true-peak ceiling allows). Video and any subtitle/data track are
+stream-copied (`dropped_non_av_streams` reports a track the container refused); audio becomes AAC in
video containers or the codec matching the extension (.wav → PCM, .flac, .mp3).
The written file is measured again: a lossy encoder can push peaks past the
ceiling loudnorm held (ffmpeg's AAC at 192k turned one transient from -2.4 to
@@ -629,6 +637,7 @@ proxy.py INPUT [--width W | --scale F] [--crf N] [--fps N] [--no-audio] [-o OUT]
Not a delivery preset: resizes to `--width` (default 640) or by `--scale`
factor, re-encodes at a proxy-grade `--crf` (default 30) with the fastest
x264/x265 preset, keeps the source's own dynamic range (HDR stays HDR;
-run `color.py --to-sdr` first if SDR is wanted). Only executes the spec
+run `color.py --to-sdr` first if SDR is wanted) and keeps a subtitle/data
+track when the container can hold it (`dropped_non_av_streams`). Only executes the spec
given — does not decide which asset to proxy or what for.
diff --git a/scripts/_common.py b/scripts/_common.py
index e122864..13ed474 100755
--- a/scripts/_common.py
+++ b/scripts/_common.py
@@ -398,8 +398,12 @@ def _cleanup_partial_output(cmd: Sequence[str]) -> None:
def _fail(cmd: Sequence[str], returncode: int, stderr: str) -> None:
# Partial-output cleanup already ran in the caller (_run_captured/_run_with_progress) for
# every failed ffmpeg invocation, not just this check=True path -- see _cleanup_partial_output.
+ # The process exit code is always 1 for an ffmpeg failure: ffmpeg's own code (1, 69, 218, 234,
+ # a negative signal number...) varies by build and by the failing stage, and 124/127/130/143
+ # are reserved for timeout, missing tool and interrupts. The raw code is kept in the JSON
+ # document as `ffmpeg_returncode` for a caller that wants it. docs/design-decisions.md.
tail = "\n".join(stderr.strip().splitlines()[-15:])
- die(f"command failed ({returncode}): {cmd[0]}\n{tail}", code=returncode or 1, kind="ffmpeg")
+ die(f"command failed ({returncode}): {cmd[0]}\n{tail}", code=1, kind="ffmpeg", ffmpeg_returncode=returncode)
def _check_no_overwrite_input(cmd: Sequence[str]) -> None:
diff --git a/scripts/audio.py b/scripts/audio.py
index 6a36712..a832cfb 100755
--- a/scripts/audio.py
+++ b/scripts/audio.py
@@ -20,7 +20,7 @@ import argparse
import sys
from typing import List
-from _common import STATE, add_common, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, run, fmt_secs
+from _common import STATE, add_common, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, run, run_keeping_subtitles, fmt_secs
VOICE_CHAIN = "highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2"
@@ -226,8 +226,11 @@ def main() -> int:
if not keep_video:
# audio-only outputs: a looped music bed is infinite, -shortest ends the run with the main track
cmd.append("-shortest")
- cmd.append(output)
- run(cmd)
+ if keep_video:
+ dropped_streams = run_keeping_subtitles(cmd, output)
+ else:
+ dropped_streams = bool(has_video and (meta.get("subtitle_streams") or meta.get("data_streams")))
+ run(cmd + [output])
r = probe(output, role="output")
a = r["audio"]
if r.get("video") and audio_out and not STATE.dry_run:
@@ -235,7 +238,8 @@ def main() -> int:
info(f"wrote {output} ({fmt_secs(r['duration'])}, audio {a['codec']} {a['channels']}ch {a['sample_rate']}Hz"
+ (", video stream-copied" if has_video and not audio_out else ", video dropped" if has_video else "") + ")")
emit(output, video=bool(has_video and not audio_out), audio_stream=args.audio_stream,
- dynamics=[f for f in (args.gate and "agate", args.compress and "acompressor", args.limit and "alimiter") if f])
+ dynamics=[f for f in (args.gate and "agate", args.compress and "acompressor", args.limit and "alimiter") if f],
+ dropped_non_av_streams=dropped_streams)
return 0
diff --git a/scripts/broll.py b/scripts/broll.py
index 1e42320..144b39d 100755
--- a/scripts/broll.py
+++ b/scripts/broll.py
@@ -145,7 +145,8 @@ def main() -> int:
if not STATE.dry_run and dur_a and abs((result.get("duration") or 0.0) - dur_a) > max(0.1, 1.5 / fps):
die(f"output is {fmt_secs(result.get('duration'))} but the A-roll is {dur_a:.3f}s -- a cutaway must not change the length", kind="output")
info(f"wrote {output} ({result.get('duration', 0):.3f}s, {len(cutaways)} cutaway(s), audio={args.audio})")
- emit(output, cutaways=[{"insert": c["path"], "at": c["at"], "end": c["at"] + c["length"], "from": c["from"]} for c in cutaways], audio=args.audio)
+ emit(output, cutaways=[{"insert": c["path"], "at": c["at"], "end": c["at"] + c["length"], "from": c["from"]} for c in cutaways], audio=args.audio,
+ dropped_non_av_streams=bool(meta_a.get("subtitle_streams") or meta_a.get("data_streams")))
return 0
diff --git a/scripts/join.py b/scripts/join.py
index f828b74..af92d6f 100755
--- a/scripts/join.py
+++ b/scripts/join.py
@@ -206,7 +206,8 @@ def main() -> int:
expected = sum(durs) - d * (n - 1)
r = probe(output, role="output")
info(f"wrote {output} ({fmt_secs(r['duration'])}, expected ~{expected:.3f}s, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
- emit(output, mode="video", clips=n, transition=args.transition, expected_duration=round(expected, 3))
+ emit(output, mode="video", clips=n, transition=args.transition, expected_duration=round(expected, 3),
+ dropped_non_av_streams=any(m.get("subtitle_streams") or m.get("data_streams") for m in metas))
return 0
diff --git a/scripts/loudness.py b/scripts/loudness.py
index f79dd0f..adb4288 100755
--- a/scripts/loudness.py
+++ b/scripts/loudness.py
@@ -18,7 +18,7 @@ import os
import re
import sys
-from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, dry_run_input_pending
+from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, run_keeping_subtitles, dry_run_input_pending
@@ -90,7 +90,10 @@ def main() -> int:
bitrate_pinned = args.audio_bitrate is not None
bitrate = args.audio_bitrate or "192k"
+ dropped_streams = False
+
def encode(tp: float, bitrate: str) -> None:
+ nonlocal dropped_streams
af = (
f"loudnorm=I={args.lufs}:TP={tp}:LRA={args.lra}"
f":measured_I={stats['input_i']}:measured_TP={stats['input_tp']}:measured_LRA={stats['input_lra']}"
@@ -101,6 +104,8 @@ def main() -> int:
cmd += ["-vn"] + audio_codec_for(output, bitrate)
else:
cmd += ["-map", "0:v:0", "-map", "0:a:0", "-c:v", "copy", "-c:a", "aac", "-b:a", bitrate]
+ dropped_streams = run_keeping_subtitles(cmd, output)
+ return
cmd.append(output)
run(cmd)
@@ -149,7 +154,7 @@ def main() -> int:
f"the encoder overshoots more than the loudnorm ceiling can absorb at this bitrate",
kind="verification", output=output, result=result,
hint="raise --audio-bitrate (e.g. 256k) or deliver a lossless format (wav/flac) and let the platform encode")
- emit(output, result=result)
+ emit(output, result=result, dropped_non_av_streams=dropped_streams)
return 0
diff --git a/scripts/proxy.py b/scripts/proxy.py
index 191acaf..3b4040f 100644
--- a/scripts/proxy.py
+++ b/scripts/proxy.py
@@ -25,7 +25,7 @@ Examples:
import argparse
import sys
-from _common import add_common, apply_common, cfr_args, default_output, die, emit, ffmpeg_base, info, probe, run, video_args, fmt_secs
+from _common import add_common, apply_common, cfr_args, default_output, die, emit, ffmpeg_base, info, probe, run_keeping_subtitles, video_args, fmt_secs
def even(n: float) -> int:
@@ -67,14 +67,13 @@ def main() -> int:
cmd = ffmpeg_base() + ["-i", args.input, "-vf", f"scale={out_w}:-2"]
cmd += video_args(meta, args.crf, "veryfast")
cmd += cfr_args(meta, args.fps)
- cmd += ["-c:a", "aac", "-b:a", "96k"] if has_audio else ["-an"]
- cmd.append(output)
- run(cmd)
+ cmd += ["-map", "0:v:0"] + (["-map", "0:a:0", "-c:a", "aac", "-b:a", "96k"] if has_audio else ["-an"])
+ dropped_streams = run_keeping_subtitles(cmd, output)
result = probe(output)
v = result["video"]
info(f"wrote {output} ({fmt_secs(result['duration'])}, {v['width']}x{v['height']}, {v['codec']}, crf {args.crf})")
- emit(output)
+ emit(output, dropped_non_av_streams=dropped_streams)
return 0
diff --git a/scripts/speedramp.py b/scripts/speedramp.py
index 9b9e24c..aa72b5e 100644
--- a/scripts/speedramp.py
+++ b/scripts/speedramp.py
@@ -115,7 +115,7 @@ def main() -> int:
result = probe(output, role="output")
v = result["video"]
info(f"wrote {output} ({fmt_secs(result['duration'])}, {v['width']}x{v['height']}, {len(segments)} speed segments)")
- emit(output)
+ emit(output, dropped_non_av_streams=bool(meta.get("subtitle_streams") or meta.get("data_streams")))
return 0
diff --git a/tests/test_contract.py b/tests/test_contract.py
index c82de26..0aa70e3 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -1692,6 +1692,41 @@ class ContractTests(unittest.TestCase):
proc = tool("fit", self.src, "--aspect", "1:1", "--dry-run", "--json", "-o", ro / "x.mp4", check=False)
self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input")
+ def test_settled_policies_exit_code_and_subtitle_tracks(self):
+ """docs/design-decisions.md: an ffmpeg failure exits 1 whatever ffmpeg returned (the raw code
+ travels as ffmpeg_returncode); tools that leave the timeline alone keep a subtitle track
+ (audio/loudness/proxy joined fit/color/overlay), tools that move it say so."""
+ bad = self.out("policy_bad.cube")
+ bad.write_text('TITLE "x"\nLUT_3D_SIZE 2\ngarbage\n')
+ proc = tool("color", self.src, "--lut", bad, "--json", "-o", self.out("policy_lut.mp4"), check=False)
+ self.assertEqual(proc.returncode, 1, proc.stderr[-300:])
+ doc = json.loads(proc.stdout)
+ self.assertEqual((doc["status"], doc["error"]["kind"], doc["exit_code"]), ("failed", "ffmpeg", 1))
+ self.assertIsInstance(doc["ffmpeg_returncode"], int)
+ self.assertNotEqual(doc["ffmpeg_returncode"], 0)
+ keep = [
+ ("audio", [self.subbed, "--gain", "3", "-o", self.out("policy_audio.mkv"), "--json"]),
+ ("loudness", [self.subbed, "-o", self.out("policy_loud.mkv"), "--json"]),
+ ("proxy", [self.subbed, "--width", "160", "-o", self.out("policy_proxy.mkv"), "--json"]),
+ ]
+ for name, args in keep:
+ doc = json.loads(tool(name, *args).stdout)
+ self.assertFalse(doc["dropped_non_av_streams"], name)
+ kinds = sh("ffprobe", "-v", "error", "-show_entries", "stream=codec_type", "-of", "csv=p=0", doc["output"]).stdout.split()
+ self.assertEqual(kinds.count("subtitle"), 2, f"{name}: expected both source subtitle tracks, got {kinds}")
+ drop = [
+ ("join", [self.subbed, self.subbed, "--duration", "0.3", "-o", self.out("policy_join.mp4"), "--json"]),
+ ("broll", [self.subbed, "--insert", self.src, "--at", "1", "--duration", "1", "-o", self.out("policy_broll.mp4"), "--json"]),
+ ("speedramp", [self.subbed, "--segment", "0-3:2", "--segment", "3-6:1", "-o", self.out("policy_ramp.mp4"), "--json"]),
+ ]
+ for name, args in drop:
+ doc = json.loads(tool(name, *args, "--fast").stdout)
+ self.assertTrue(doc["dropped_non_av_streams"], name)
+ self.assertEqual(doc["probe"]["subtitle_streams"], 0, name)
+ # a plain source reports false, so the key is a fact and not a constant
+ doc = json.loads(tool("speedramp", self.src, "--segment", "0-6:2", "-o", self.out("policy_ramp_plain.mp4"), "--json", "--fast").stdout)
+ self.assertFalse(doc["dropped_non_av_streams"])
+
def test_boundary_sweep_p3_regressions(self):
"""Sweep P3s: export's HDR warning reaches --json (F11); graphics refuses a frame too small
for its templates (F16); insert refuses a video where it wants a still (F17); look builds