Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
contract --json: add reencodes_video/reencodes_audio per tool (#36)
Shogo Kajiyama42a6b6a6c4f1
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 67f93db..ce207a9 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,7 @@## Unreleased — FFmpeg 8+ / Windows compatibility for caption and color+- **`contract --json`: `reencodes_video`/`reencodes_audio` per tool.** Each of the 21 tools now declares, per stream type, `"always"` / `"never"` / `"conditional"` (with a `reencode_note` for the conditional ones), read from what each script's own encode/copy args actually do. Surfaces a fact that wasn't documented anywhere: `fit`, `caption`, `overlay`, `graphics`, `color`, `join`, `multicam` and `silence` always transcode audio to AAC alongside a video filter — there is no `-c:a copy` path in this codebase for a tool that also re-encodes video, so a caller cannot assume the original audio codec survives a picture-only edit. `caption.py`'s docstring now says plainly that burn-in is the only mode (no soft-subtitle mux) and always re-encodes both streams. Additive contract field; no tool's behaviour changed.- **SKILL.md: explicit "what this skill does and does not decide".** Added a section naming what belongs to a production agent (approval, which cut is right), another skill (thumbnail composition), or nobody in this skill (content understanding, judging a highlight's interest beyond a measured proxy) -- and an explicit rule against ever falling back to a raw `ffmpeg`/`filter_complex` invocation when a request needs something none of the 21 scripts expose. The trigger description (frontmatter) is intentionally left as-is: it stays broad on purpose (any video/audio file touch), since narrowing it risks under-triggering on requests that do belong here; what changed is what the skill does once triggered. `evals/agent_prompts_24.json` gains one refusal case (`r05-no-raw-ffmpeg`) asking directly for a raw `-filter_complex` command.- **`check.py`: plain-language `reason` on the less obvious FAILs.** Every row already had `fix` (the command that resolves it); video codec, pixel format, HDR colour and loudness FAILs now also carry `reason` ("QuickTime and iOS commonly reject video that isn't 8-bit 4:2:0") for a caller reporting the result to someone who doesn't already know why the spec value matters. Empty on PASS rows and every other check. Additive, no existing field changed.- **`cut.py --json`: full requested-vs-actual provenance.** `expected_duration`/`duration_error_ms`/`precision`/`reencoded` already existed (0.9.1); added `requested_start`/`requested_end` (or `requested_segments` for `--segments`), `requested_duration`, `output_duration`, `duration_delta_seconds` (seconds-unit alias of `duration_error_ms`) and `mode` (`copy`/`accurate`/`hybrid` — "hybrid" means a lossless cut silently re-encoded because the keyframe snap exceeded `--tolerance`) and `keyframe_snapped`. Additive only, no existing field changed. Intended for a downstream repo (an editing skill, an agent) that wants this in its own provenance/audit trail without re-deriving it from `reencoded`+`precision`.diff --git a/docs/contract.md b/docs/contract.mdindex fe068b5..104d67d 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -64,6 +64,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:| `produces_artifact` | writes a file (media, PNG, HTML, EDL) || `verification` | `{required, tools}`: which tools to run on the output afterwards || `requires_visual_verification` | the picture changed; run `ffmpeg-skill/look` and inspect the PNG |+| `reencodes_video`, `reencodes_audio` | `"always"` / `"never"` / `"conditional"`, meaning *when that stream is present in the input* — not whether the tool touches the file at all. `"conditional"` tools (`cut`, `export`, `render`, `batch`, `verify`) carry a `reencode_note` explaining what it depends on. Several visual tools (`fit`, `caption`, `overlay`, `graphics`, `color`, `join`, `multicam`, `silence`) are `"always"` on audio too: this codebase never mixes `-c:v` re-encode with `-c:a copy` in one call, so a caller cannot assume the original audio codec survives just because only the picture changed || `audio_only` | accepts an audio-only input (WAV, MP3, M4A, FLAC, OGG, Opus) || `video_required` | refuses an input without a video stream ("input has no video stream") || | `join` has `audio_only: true` and `video_required: false` since 0.9.1: audio-only inputs are joined as audio (no `look` needed then); mixing audio and video inputs is refused |diff --git a/scripts/_contract.py b/scripts/_contract.pyindex 6862595..e003f51 100644--- a/scripts/_contract.py+++ b/scripts/_contract.py@@ -146,6 +146,38 @@ DRY_RUN_NOTES = {"verify": "not supported: the flag is accepted but the steps run and outputs are written",}+# Whether a tool re-encodes each stream *when that stream is present in the input* -- not whether+# the tool touches the file at all. "always"/"never" are unconditional given that stream exists;+# "conditional" means it depends on flags or on how far a lossless attempt misses (see "note").+# Read from each script's actual encode/copy args, not from role or intent, since several tools+# (fit, caption, overlay, graphics, color, join, multicam, silence) always transcode audio to AAC+# alongside a video filter even though the audio itself is untouched content -- there is no+# "-c:a copy while re-encoding video" path in this codebase, so a soft-subtitle-style passthrough+# of the original audio codec never happens on those tools.+REENCODE_META: Dict[str, Dict[str, str]] = {+ "probe": dict(video="never", audio="never", note="analysis only, no artifact"),+ "cut": dict(video="conditional", audio="conditional", note="lossless -c copy preferred; re-encodes on --accurate, a VFR source, or a keyframe snap past --tolerance (see cut.py --json: mode, keyframe_snapped)"),+ "fit": dict(video="always", audio="always", note="always re-encodes to AAC when audio is present, even if only --fps or --aspect was asked for"),+ "caption": dict(video="always", audio="always", note="burn-in only: no soft-subtitle mux path exists, so captions always cost a full re-encode of both streams"),+ "overlay": dict(video="always", audio="always"),+ "graphics": dict(video="always", audio="always"),+ "sync": dict(video="never", audio="conditional", note="video is never touched; audio is copied or re-encoded depending on --trim-second / --replace-audio / --fix-drift"),+ "multicam": dict(video="always", audio="always"),+ "audio": dict(video="never", audio="always", note="video stream is always -c:v copy when present; this tool's job is the audio"),+ "loudness": dict(video="never", audio="always"),+ "silence": dict(video="always", audio="always", note="removing gaps requires cutting on non-keyframe boundaries"),+ "join": dict(video="always", audio="always"),+ "color": dict(video="always", audio="always"),+ "export": dict(video="conditional", audio="conditional", note="--preset copy is -c:v copy -c:a copy (no re-encode); every other preset re-encodes both"),+ "check": dict(video="never", audio="never", note="read-only, no artifact"),+ "scenes": dict(video="never", audio="never", note="analysis only; --sheet renders a new contact-sheet PNG, not a re-encode of the source"),+ "look": dict(video="never", audio="never", note="renders a new contact-sheet/frame PNG, not a re-encode of the source"),+ "render": dict(video="conditional", audio="conditional", note="delegated: depends on which stages a project.json runs and how each one behaves"),+ "batch": dict(video="conditional", audio="conditional", note="delegated: depends on which script each recipe step runs"),+ "verify": dict(video="conditional", audio="conditional", note="delegated: runs cut/fit/caption/export/loudness/color internally as checks"),+ "report": dict(video="never", audio="never", note="measures via look/check; produces an HTML report, not a re-encoded artifact"),+}+IDEMPOTENCY = {"bit_exact": "same inputs and flags give byte-identical output","content_equivalent": "same inputs and flags give the same media content; bytes may differ between encoder builds",@@ -506,6 +538,11 @@ def tool_spec(name: str, version: str) -> Dict[str, Any]:"mutates_input": False,"produces_artifact": meta["produces_artifact"],"verification": {"required": bool(meta["verify"]), "tools": [f"{SKILL_ID}/{t}" for t in meta["verify"]]},+ # a tool with no declared entry reports "conditional" rather than guessing "always" or+ # "never" -- same "unknown is not missing" principle as doctor's capability detection+ "reencodes_video": REENCODE_META.get(name, {}).get("video", "conditional"),+ "reencodes_audio": REENCODE_META.get(name, {}).get("audio", "conditional"),+ **({"reencode_note": REENCODE_META[name]["note"]} if REENCODE_META.get(name, {}).get("note") else {}),"requires_visual_verification": meta["visual"],"audio_only": meta["audio_only"],"video_required": meta["video_required"],diff --git a/scripts/caption.py b/scripts/caption.pyindex 3197615..9843ba5 100755--- a/scripts/caption.py+++ b/scripts/caption.py@@ -4,6 +4,12 @@Styling (font, size, colour, outline, position) applies to SRT input vialibass force_style. ASS files carry their own styles and are rendered as-is.+This only burns subtitles into the picture. There is no soft-subtitle (mux a+subtitle stream, toggleable by the player) path -- every call re-encodes the+whole video and its audio (see contract --json: reencodes_video/reencodes_audio+are both "always" for this tool), even for a source that only needed the+subtitle track added.+Text-to-SRT input format (one cue per line, blank lines ignored):0:00-0:03 Hello and welcome00:00:03.500 --> 00:00:06 Second line | with a manual line breakdiff --git a/tests/test_contract.py b/tests/test_contract.pyindex ee7893a..f79547c 100644--- a/tests/test_contract.py+++ b/tests/test_contract.py@@ -262,6 +262,17 @@ class ContractTests(unittest.TestCase):skill = (ROOT / "SKILL.md").read_text(encoding="utf-8")self.assertIn("Look: not needed", skill)+ def test_reencodes_video_and_audio_declared_for_every_tool(self):+ for t in self.contract["tools"]:+ self.assertIn(t["reencodes_video"], ("always", "never", "conditional"), t["name"])+ self.assertIn(t["reencodes_audio"], ("always", "never", "conditional"), t["name"])+ # a handful of the least intuitive ones, checked against what the scripts actually do+ self.assertEqual((self.tools["cut"]["reencodes_video"], self.tools["cut"]["reencodes_audio"]), ("conditional", "conditional"))+ self.assertEqual((self.tools["export"]["reencodes_video"], self.tools["export"]["reencodes_audio"]), ("conditional", "conditional"))+ self.assertEqual((self.tools["caption"]["reencodes_video"], self.tools["caption"]["reencodes_audio"]), ("always", "always"))+ self.assertEqual((self.tools["loudness"]["reencodes_video"], self.tools["loudness"]["reencodes_audio"]), ("never", "always"))+ self.assertEqual((self.tools["probe"]["reencodes_video"], self.tools["probe"]["reencodes_audio"]), ("never", "never"))+def test_original_preservation_and_roles(self):for t in self.contract["tools"]:self.assertFalse(t["mutates_input"], t["name"])