Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
doctor: report GPU-backed encoders present in the ffmpeg build (#52) (#59)
Shogo Kajiyama9cc8652acfe0
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7470356..e2fedda 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,7 @@## Unreleased+- **`doctor --json`: `gpu_encoders`.** No tool here uses GPU-accelerated encoding — every tool assumes CPU x264/x265 — but `doctor` had no way to answer "is GPU encoding available on this machine" at all, unlike every other capability it already reports `yes`/`no`/`unknown` for. `gpu_encoders` reports GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) present in this ffmpeg *build*, read from the same `-encoders` listing `doctor` already parses — `{"status": "parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. Deliberately build-presence only: proving a real GPU/driver will accept a job would need an actual encode, which `doctor`'s introspection never runs (matching its existing 10s-timeout, listing-only philosophy). Purely informational — no tool declares or requires a GPU encoder, so this field never affects `ok` or any tool's `usable`. The human-readable `doctor` output gets one line naming what's present (or "none"). Closes [#52](https://github.com/kajisho5/ffmpeg-skill/issues/52).- **`caption.py --mode mux`: soft subtitle stream instead of burn-in.** Every caption call previously re-encoded both streams to render pixels (`reencodes_video`/`reencodes_audio`: `"always"`) even when the caller only wanted a subtitle track added, not the picture changed. `--mode mux` (new; `--mode burn` stays the default) copies video and audio untouched (`-c:v copy -c:a copy`) and adds the SRT as a separate, player-toggleable subtitle stream — `reencodes_video`/`reencodes_audio` are now `"conditional"` with a note explaining the split. Only takes a plain SRT (`--srt`/`--text`/`--transcribe`), not `--ass`, `--animate` or `--karaoke`: styling and animation render pixels, so they have no soft-subtitle equivalent and are refused with a pointer to `--mode burn`. The subtitle codec is picked from the output container (`mov_text` for `.mp4`/`.m4v`/`.mov`, `srt` for `.mkv`, `webvtt` for `.webm`); an unrecognized container is refused rather than guessed at. `contract --json` gains three new optional capabilities (`encoder:mov_text`/`encoder:webvtt`/`encoder:srt`, each `"when"`-gated to the matching output container) so `doctor` can report them honestly; the existing required capabilities (`encoder:libx264`, `encoder:aac`, `filter:subtitles`) are unchanged, since `--mode burn` is still the default and doctor's usability model doesn't vary by flag. Closes [#51](https://github.com/kajisho5/ffmpeg-skill/issues/51) — timecode-aware cue timing and explicit multi-audio-track selection were split out to [#54](https://github.com/kajisho5/ffmpeg-skill/issues/54) and [#55](https://github.com/kajisho5/ffmpeg-skill/issues/55) to keep this change reviewable.- **Test: the prose tool count in README/`docs/contract.md`/`package.json` is now checked against the real tool list.** 0.11.0 was cut to fix exactly this drift (README said 28 twice and 22 once; `package.json` said 21) by hand, with nothing to stop it recurring. `tests/test_contract.py`'s `test_docs_tool_count_matches_the_real_tool_list` scans all three for `"<N> tools"` wording and fails if any number doesn't match `scripts/`'s actual public-tool count, so the next tool added/removed without updating every mention fails CI instead of drifting silently. `contract_version`/`skill.version` were already split (0.9.0) and already documented as "additive keeps `contract_version`, breaking bumps it" — `docs/contract.md` now also repeats the tagged-version pin guidance next to that table, since `capability_map` makes this the most cross-repo-facing part of the contract. Closes [#50](https://github.com/kajisho5/ffmpeg-skill/issues/50).- **Tests: `overlay.py --image` on an audio-less video is now covered.** Every existing overlay test used adiff --git a/README.md b/README.mdindex d4a5740..822d437 100644--- a/README.md+++ b/README.md@@ -252,11 +252,13 @@ The contract is generated from the code that runs, not maintained beside it. For```bashnpx ffmpeg-skill doctor # human-readable-npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools+npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools / gpu_encoders````doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.+`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`.+## FFmpeg compatibilityThe tools need FFmpeg 5.0 or later. The capability parser has been run against the listings of these builds:diff --git a/docs/contract.md b/docs/contract.mdindex 9459d26..4fe6e11 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -211,6 +211,15 @@ capabilities by hand to answer "can I run `caption.py` on this machine right nowpassing overall does not mean every tool is usable (a plain Homebrew `ffmpeg` on macOS is `ok`for tools that don't need `subtitles`/`drawtext`/`zscale`, but `caption.usable` is `"no"`).+`doctor`'s `gpu_encoders` field reports GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`,+`vaapi`, `amf`) present in this ffmpeg *build*, read from `-encoders` alone — `{"status":+"parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. It proves the build shipped the+capability, not that the GPU/driver on this machine will accept a job (that needs a real+encode, which this introspection never runs). No tool declares or requires a GPU encoder, so+`gpu_encoders` never affects `ok` or any tool's `usable` — it exists purely so a caller can ask+the same honest yes/no/unknown question about GPU support that filter/encoder detection already+answers for everything else, without a tool here needing to use one.+## InvocationStructured arguments are the canonical way to call a tool, on the CLI or through MCP.diff --git a/scripts/_contract.py b/scripts/_contract.pyindex 962a895..fcc9607 100644--- a/scripts/_contract.py+++ b/scripts/_contract.py@@ -436,6 +436,28 @@ def _ff_list(binary: str, flag: str) -> List[str]:return _ff_listing(binary, flag)["names"]+# GPU-backed encoders are named `<codec>_<backend>` by every ffmpeg build (h264_nvenc,+# hevc_videotoolbox, av1_qsv, h264_vaapi, hevc_amf, ...); recognised by the backend suffix so a+# new codec in a future ffmpeg build needs no change here.+_GPU_ENCODER_SUFFIXES = ("_nvenc", "_videotoolbox", "_qsv", "_vaapi", "_amf")+++def _gpu_encoders(encoders_listing: Dict[str, Any]) -> Dict[str, Any]:+ """GPU-backed encoders this ffmpeg BUILD was compiled with, read from `-encoders` alone.++ This proves the build carries e.g. h264_nvenc; it does NOT prove the GPU/driver on this+ machine will accept a job -- that would require actually running an encode, which doctor's+ introspection deliberately never does beyond listing/-version (see doctor()'s own docstring).+ A caller that needs to know "will a GPU encode actually work here" has to try one; this only+ answers "did this ffmpeg build even ship the capability."+ """+ status = encoders_listing["status"]+ if status != "parsed":+ return {"status": status, "detail": encoders_listing["detail"], "present": []}+ present = sorted(n for n in encoders_listing["names"] if n.endswith(_GPU_ENCODER_SUFFIXES))+ return {"status": "parsed", "present": present}++def _version_line(binary: str) -> Optional[str]:exe = shutil.which(binary)if not exe:@@ -478,6 +500,13 @@ def doctor() -> Dict[str, Any]:that would prove it could not be read (unparsed output, ffmpeg failure); it is never folded into`missing` (a filter that exists is not reported absent) nor into `available` (a failed detectionis not a pass). `ok` is true only when nothing required is missing or unknown.++ `gpu_encoders` is a separate, honest answer to a question none of the required/optional+ capabilities above ask: which GPU-backed encoders (nvenc, videotoolbox, qsv, vaapi, amf) this+ ffmpeg BUILD carries, from `-encoders` alone. No tool here requires or uses one -- every tool+ still assumes CPU x264/x265 -- so `gpu_encoders` never affects `ok` or any tool's `usable`. It+ only proves the build shipped the capability, never that the GPU/driver on this machine will+ actually accept a job (that needs a real encode, which this introspection never runs)."""listings = {"encoders": _ff_listing("ffmpeg", "-encoders"),@@ -530,6 +559,7 @@ def doctor() -> Dict[str, Any]:"errors": errors,"ok": not missing_required and not unknown_required,"tools": _tool_usability(state),+ "gpu_encoders": _gpu_encoders(listings["encoders"]),}@@ -848,6 +878,9 @@ def main() -> int:not_usable = sorted(name for name, t in d["tools"].items() if t["usable"] != "yes")if not_usable and d["ok"]:print(f"note: overall 'ok' means nothing REQUIRED BY EVERY TOOL is missing -- {len(not_usable)} tool(s) still can't run today: {', '.join(not_usable)} (see doctor --json .tools for why)")+ gpu = d["gpu_encoders"]+ if gpu["status"] == "parsed":+ print(f"GPU-backed encoders in this build: {', '.join(gpu['present']) or 'none'} (no tool here uses one yet; this build-presence check does not prove the GPU/driver will accept a job)")for err in d["errors"]:print(f"detection error: {err}", file=sys.stderr)if d["ok"]:diff --git a/tests/test_contract.py b/tests/test_contract.pyindex 09578db..d808d9d 100644--- a/tests/test_contract.py+++ b/tests/test_contract.py@@ -933,6 +933,32 @@ class DoctorDetectionTests(unittest.TestCase):for key in ("available", "missing", "missing_optional", "unknown", "detection", "detected_by"):self.assertIn(key, contract["capabilities"])+ def test_gpu_encoders_are_reported_from_the_build_alone(self):+ """gpu_encoders answers only "did this ffmpeg build ship the capability" -- never affects ok/usable."""+ # the real macOS capture carries VideoToolbox (h264/hevc), the Windows capture carries nvenc/qsv/amf/vaapi+ d_mac, _ = self._doctor("ffmpeg_filters_8.1.2_macos.txt", encoders="ffmpeg_encoders_8.1.2_macos.txt", bsfs="ffmpeg_bsfs_8.1.2_macos.txt")+ self.assertEqual(d_mac["gpu_encoders"]["status"], "parsed")+ self.assertIn("h264_videotoolbox", d_mac["gpu_encoders"]["present"])+ self.assertIn("hevc_videotoolbox", d_mac["gpu_encoders"]["present"])+ self.assertNotIn("h264_nvenc", d_mac["gpu_encoders"]["present"], "macOS build has no nvenc")+ d_win, _ = self._doctor("ffmpeg_filters_9.0.1_windows.txt", encoders="ffmpeg_encoders_9.0.1_windows.txt", bsfs="ffmpeg_bsfs_9.0.1_windows.txt")+ self.assertEqual(d_win["gpu_encoders"]["status"], "parsed")+ self.assertGreater(len(d_win["gpu_encoders"]["present"]), 0)+ for name in d_win["gpu_encoders"]["present"]:+ self.assertTrue(name.endswith(("_nvenc", "_videotoolbox", "_qsv", "_vaapi", "_amf")), name)+ # no tool declares or requires a GPU encoder, so its presence/absence never affects ok or any tool's usability+ # (the Windows gyan.dev capture is a full build with nothing missing, unlike the macOS one above)+ self.assertTrue(d_win["ok"])+ self.assertEqual(d_win["tools"]["export"]["usable"], "yes")+ declared = set(_contract.required_capabilities()["required"] + _contract.required_capabilities()["optional"])+ self.assertFalse(any(c.startswith("encoder:") and c[8:] in d_win["gpu_encoders"]["present"] for c in declared),+ "no tool declares a GPU encoder as required/optional -- gpu_encoders is purely informational")+ # an unreadable encoder listing (fed the filters-garbage fixture as "encoders" to force+ # unparsed output) is reported honestly by status, never silently claimed as an empty pass+ d_fail, _ = self._doctor("ffmpeg_filters_6.1.txt", encoders="ffmpeg_filters_garbage.txt")+ self.assertNotEqual(d_fail["gpu_encoders"]["status"], "parsed")+ self.assertEqual(d_fail["gpu_encoders"]["present"], [])+if __name__ == "__main__":unittest.main(verbosity=2)