Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

doctor: report GPU-backed encoders present in the ffmpeg build (#52) (#59)

Shogo Kajiyama9cc8652acfe0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7470356..e2fedda 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,7 @@
## Unreleased
+- **`doctor --json`: `gpu_encoders`.** No tool here uses GPU-accelerated encoding — every tool assumes CPU x264/x265 — but `doctor` had no way to answer "is GPU encoding available on this machine" at all, unlike every other capability it already reports `yes`/`no`/`unknown` for. `gpu_encoders` reports GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) present in this ffmpeg *build*, read from the same `-encoders` listing `doctor` already parses — `{"status": "parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. Deliberately build-presence only: proving a real GPU/driver will accept a job would need an actual encode, which `doctor`'s introspection never runs (matching its existing 10s-timeout, listing-only philosophy). Purely informational — no tool declares or requires a GPU encoder, so this field never affects `ok` or any tool's `usable`. The human-readable `doctor` output gets one line naming what's present (or "none"). Closes [#52](https://github.com/kajisho5/ffmpeg-skill/issues/52).
- **`caption.py --mode mux`: soft subtitle stream instead of burn-in.** Every caption call previously re-encoded both streams to render pixels (`reencodes_video`/`reencodes_audio`: `"always"`) even when the caller only wanted a subtitle track added, not the picture changed. `--mode mux` (new; `--mode burn` stays the default) copies video and audio untouched (`-c:v copy -c:a copy`) and adds the SRT as a separate, player-toggleable subtitle stream — `reencodes_video`/`reencodes_audio` are now `"conditional"` with a note explaining the split. Only takes a plain SRT (`--srt`/`--text`/`--transcribe`), not `--ass`, `--animate` or `--karaoke`: styling and animation render pixels, so they have no soft-subtitle equivalent and are refused with a pointer to `--mode burn`. The subtitle codec is picked from the output container (`mov_text` for `.mp4`/`.m4v`/`.mov`, `srt` for `.mkv`, `webvtt` for `.webm`); an unrecognized container is refused rather than guessed at. `contract --json` gains three new optional capabilities (`encoder:mov_text`/`encoder:webvtt`/`encoder:srt`, each `"when"`-gated to the matching output container) so `doctor` can report them honestly; the existing required capabilities (`encoder:libx264`, `encoder:aac`, `filter:subtitles`) are unchanged, since `--mode burn` is still the default and doctor's usability model doesn't vary by flag. Closes [#51](https://github.com/kajisho5/ffmpeg-skill/issues/51) — timecode-aware cue timing and explicit multi-audio-track selection were split out to [#54](https://github.com/kajisho5/ffmpeg-skill/issues/54) and [#55](https://github.com/kajisho5/ffmpeg-skill/issues/55) to keep this change reviewable.
- **Test: the prose tool count in README/`docs/contract.md`/`package.json` is now checked against the real tool list.** 0.11.0 was cut to fix exactly this drift (README said 28 twice and 22 once; `package.json` said 21) by hand, with nothing to stop it recurring. `tests/test_contract.py`'s `test_docs_tool_count_matches_the_real_tool_list` scans all three for `"<N> tools"` wording and fails if any number doesn't match `scripts/`'s actual public-tool count, so the next tool added/removed without updating every mention fails CI instead of drifting silently. `contract_version`/`skill.version` were already split (0.9.0) and already documented as "additive keeps `contract_version`, breaking bumps it" — `docs/contract.md` now also repeats the tagged-version pin guidance next to that table, since `capability_map` makes this the most cross-repo-facing part of the contract. Closes [#50](https://github.com/kajisho5/ffmpeg-skill/issues/50).
- **Tests: `overlay.py --image` on an audio-less video is now covered.** Every existing overlay test used a
diff --git a/README.md b/README.md
index d4a5740..822d437 100644
--- a/README.md
+++ b/README.md
@@ -252,11 +252,13 @@ The contract is generated from the code that runs, not maintained beside it. For
```bash
npx ffmpeg-skill doctor # human-readable
-npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools
+npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools / gpu_encoders
```
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
+`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`.
+
## FFmpeg compatibility
The tools need FFmpeg 5.0 or later. The capability parser has been run against the listings of these builds:
diff --git a/docs/contract.md b/docs/contract.md
index 9459d26..4fe6e11 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -211,6 +211,15 @@ capabilities by hand to answer "can I run `caption.py` on this machine right now
passing overall does not mean every tool is usable (a plain Homebrew `ffmpeg` on macOS is `ok`
for tools that don't need `subtitles`/`drawtext`/`zscale`, but `caption.usable` is `"no"`).
+`doctor`'s `gpu_encoders` field reports GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`,
+`vaapi`, `amf`) present in this ffmpeg *build*, read from `-encoders` alone — `{"status":
+"parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. It proves the build shipped the
+capability, not that the GPU/driver on this machine will accept a job (that needs a real
+encode, which this introspection never runs). No tool declares or requires a GPU encoder, so
+`gpu_encoders` never affects `ok` or any tool's `usable` — it exists purely so a caller can ask
+the same honest yes/no/unknown question about GPU support that filter/encoder detection already
+answers for everything else, without a tool here needing to use one.
+
## Invocation
Structured arguments are the canonical way to call a tool, on the CLI or through MCP.
diff --git a/scripts/_contract.py b/scripts/_contract.py
index 962a895..fcc9607 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -436,6 +436,28 @@ def _ff_list(binary: str, flag: str) -> List[str]:
return _ff_listing(binary, flag)["names"]
+# GPU-backed encoders are named `<codec>_<backend>` by every ffmpeg build (h264_nvenc,
+# hevc_videotoolbox, av1_qsv, h264_vaapi, hevc_amf, ...); recognised by the backend suffix so a
+# new codec in a future ffmpeg build needs no change here.
+_GPU_ENCODER_SUFFIXES = ("_nvenc", "_videotoolbox", "_qsv", "_vaapi", "_amf")
+
+
+def _gpu_encoders(encoders_listing: Dict[str, Any]) -> Dict[str, Any]:
+ """GPU-backed encoders this ffmpeg BUILD was compiled with, read from `-encoders` alone.
+
+ This proves the build carries e.g. h264_nvenc; it does NOT prove the GPU/driver on this
+ machine will accept a job -- that would require actually running an encode, which doctor's
+ introspection deliberately never does beyond listing/-version (see doctor()'s own docstring).
+ A caller that needs to know "will a GPU encode actually work here" has to try one; this only
+ answers "did this ffmpeg build even ship the capability."
+ """
+ status = encoders_listing["status"]
+ if status != "parsed":
+ return {"status": status, "detail": encoders_listing["detail"], "present": []}
+ present = sorted(n for n in encoders_listing["names"] if n.endswith(_GPU_ENCODER_SUFFIXES))
+ return {"status": "parsed", "present": present}
+
+
def _version_line(binary: str) -> Optional[str]:
exe = shutil.which(binary)
if not exe:
@@ -478,6 +500,13 @@ def doctor() -> Dict[str, Any]:
that would prove it could not be read (unparsed output, ffmpeg failure); it is never folded into
`missing` (a filter that exists is not reported absent) nor into `available` (a failed detection
is not a pass). `ok` is true only when nothing required is missing or unknown.
+
+ `gpu_encoders` is a separate, honest answer to a question none of the required/optional
+ capabilities above ask: which GPU-backed encoders (nvenc, videotoolbox, qsv, vaapi, amf) this
+ ffmpeg BUILD carries, from `-encoders` alone. No tool here requires or uses one -- every tool
+ still assumes CPU x264/x265 -- so `gpu_encoders` never affects `ok` or any tool's `usable`. It
+ only proves the build shipped the capability, never that the GPU/driver on this machine will
+ actually accept a job (that needs a real encode, which this introspection never runs).
"""
listings = {
"encoders": _ff_listing("ffmpeg", "-encoders"),
@@ -530,6 +559,7 @@ def doctor() -> Dict[str, Any]:
"errors": errors,
"ok": not missing_required and not unknown_required,
"tools": _tool_usability(state),
+ "gpu_encoders": _gpu_encoders(listings["encoders"]),
}
@@ -848,6 +878,9 @@ def main() -> int:
not_usable = sorted(name for name, t in d["tools"].items() if t["usable"] != "yes")
if not_usable and d["ok"]:
print(f"note: overall 'ok' means nothing REQUIRED BY EVERY TOOL is missing -- {len(not_usable)} tool(s) still can't run today: {', '.join(not_usable)} (see doctor --json .tools for why)")
+ gpu = d["gpu_encoders"]
+ if gpu["status"] == "parsed":
+ print(f"GPU-backed encoders in this build: {', '.join(gpu['present']) or 'none'} (no tool here uses one yet; this build-presence check does not prove the GPU/driver will accept a job)")
for err in d["errors"]:
print(f"detection error: {err}", file=sys.stderr)
if d["ok"]:
diff --git a/tests/test_contract.py b/tests/test_contract.py
index 09578db..d808d9d 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -933,6 +933,32 @@ class DoctorDetectionTests(unittest.TestCase):
for key in ("available", "missing", "missing_optional", "unknown", "detection", "detected_by"):
self.assertIn(key, contract["capabilities"])
+ def test_gpu_encoders_are_reported_from_the_build_alone(self):
+ """gpu_encoders answers only "did this ffmpeg build ship the capability" -- never affects ok/usable."""
+ # the real macOS capture carries VideoToolbox (h264/hevc), the Windows capture carries nvenc/qsv/amf/vaapi
+ d_mac, _ = self._doctor("ffmpeg_filters_8.1.2_macos.txt", encoders="ffmpeg_encoders_8.1.2_macos.txt", bsfs="ffmpeg_bsfs_8.1.2_macos.txt")
+ self.assertEqual(d_mac["gpu_encoders"]["status"], "parsed")
+ self.assertIn("h264_videotoolbox", d_mac["gpu_encoders"]["present"])
+ self.assertIn("hevc_videotoolbox", d_mac["gpu_encoders"]["present"])
+ self.assertNotIn("h264_nvenc", d_mac["gpu_encoders"]["present"], "macOS build has no nvenc")
+ d_win, _ = self._doctor("ffmpeg_filters_9.0.1_windows.txt", encoders="ffmpeg_encoders_9.0.1_windows.txt", bsfs="ffmpeg_bsfs_9.0.1_windows.txt")
+ self.assertEqual(d_win["gpu_encoders"]["status"], "parsed")
+ self.assertGreater(len(d_win["gpu_encoders"]["present"]), 0)
+ for name in d_win["gpu_encoders"]["present"]:
+ self.assertTrue(name.endswith(("_nvenc", "_videotoolbox", "_qsv", "_vaapi", "_amf")), name)
+ # no tool declares or requires a GPU encoder, so its presence/absence never affects ok or any tool's usability
+ # (the Windows gyan.dev capture is a full build with nothing missing, unlike the macOS one above)
+ self.assertTrue(d_win["ok"])
+ self.assertEqual(d_win["tools"]["export"]["usable"], "yes")
+ declared = set(_contract.required_capabilities()["required"] + _contract.required_capabilities()["optional"])
+ self.assertFalse(any(c.startswith("encoder:") and c[8:] in d_win["gpu_encoders"]["present"] for c in declared),
+ "no tool declares a GPU encoder as required/optional -- gpu_encoders is purely informational")
+ # an unreadable encoder listing (fed the filters-garbage fixture as "encoders" to force
+ # unparsed output) is reported honestly by status, never silently claimed as an empty pass
+ d_fail, _ = self._doctor("ffmpeg_filters_6.1.txt", encoders="ffmpeg_filters_garbage.txt")
+ self.assertNotEqual(d_fail["gpu_encoders"]["status"], "parsed")
+ self.assertEqual(d_fail["gpu_encoders"]["present"], [])
+
if __name__ == "__main__":
unittest.main(verbosity=2)