Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

feat: opt-in Apple VideoToolbox encoding (--hw) (#311)

Shogo Kajiyama61d94baef1a8
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9587456..bd0e9b8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,10 @@
## Unreleased
-(nothing yet)
+- feat: opt-in Apple VideoToolbox encoding. `--hw`/`--no-hw` on every re-encoding tool and `export.py` (two booleans in the contract and MCP: `hw`, `no_hw`); `FFMPEG_SKILL_HW=1` makes it the default for everything but `export.py`'s delivery presets (an export under it says so in `hw`, `requested: false`); `render.py`/`batch.py --hw` make it explicit for every stage, and `waveform.py` passes its choice to the caption/title stage that writes its file. h264/hevc (HDR Main10 with the source's tags and HDR10 side data)/prores; AV1 stays on SVT-AV1; an `export.py` preset keeps its own format on the GPU (an h264/h265 preset stays 8-bit BT.709, `prores` keeps the source's tags). CRF maps to `-q:v` by SSIM against x264/x265 `medium` (`tests/bench_vt.py`): SDR H.264 18/23/28 → 75/64/53, SDR HEVC → 78/68/58, and HDR its own curve, 63/55/47, because its CPU line is x265 Main10 at CRF+2; matched quality costs 1.1–2.9× the CPU bytes on SDR and 1.9–3.5× on HDR. BT.709 tags go through `h264_metadata`/`hevc_metadata`, so FFmpeg ≥7.1 never converts an untagged source. Apple Silicon only (the hardware, also under a Rosetta Python). Only a job VideoToolbox itself refuses (the encoder could not open, or its compression session failed, in FFmpeg 5.1 to master's wordings) is re-encoded on the CPU, with the odd-dimension retry a CPU run gets; any other failure fails once, names its encoder and carries `encoder`/`hw`. Every result names its `encoder`; `hw` says what was asked and what ran (`requested`, `source`, `used` — null under `--dry-run` — `fallback`, `notes`); a GPU encode adds a top-level `notes` line that its quality is not CRF-equivalent, and one chosen by `FFMPEG_SKILL_HW=1` notes the size cost and `--no-hw`. `render.py` carries its stages' and an executed plan's `hw`, `batch.py` its items' (each `results` row names its `encoder`), and both caches key on the GPU setting. `doctor --json` gains `hw`. Built by [@rymalia](https://github.com/rymalia) in #304, including the VideoToolbox quality bench (`tests/bench_vt.py`) and its fitted curves.
+- fix: a `--dry-run` that needs the encoder list (`--codec av1`/`prores`, `--hw`) reads it from `ffprobe -encoders` and no longer runs `ffmpeg -encoders`, keeping the promise that a writing tool's dry run runs no ffmpeg; a real run keeps reading `ffmpeg -encoders`, the build that encodes (on a mixed install, a Homebrew ffprobe next to a static ffmpeg, ffprobe's list is another build's).
+- fix(batch): a cache entry whose output a later run rewrote (another recipe for the same file) is dropped, so switching back re-processes the item instead of serving the other run's file as cached.
+- fix(waveform): `commands` lists the title/caption stages' command lines too, as `render.py`'s does, so it holds every ffmpeg command the run planned or ran.
## 2.5.1
diff --git a/README.md b/README.md
index 23f2a71..88973b9 100644
--- a/README.md
+++ b/README.md
@@ -188,7 +188,7 @@ These are the rules the skill file gives the agent and the code enforces.
## Tools
-42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
+42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8) and, with `export.py`, `--hw`/`--no-hw` (Apple VideoToolbox on Apple Silicon, opt-in; see `doctor` below), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
**Analysis and inspection**
@@ -356,7 +356,7 @@ npx ffmpeg-skill doctor --json # available / missing / missing_optional / unkn
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor` also reports fonts: the default drawtext family, and `fonts.scripts` — one `available`/`missing`/`unknown` per writing system (ja, zh, ko, ar, he, hi, th, ru, el) with the file it would use — so "can this machine render Korean captions" is answered before the job, not after. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
-`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`. GPU-accelerated encoding stays deliberately off the roadmap until there's a real-hardware-verified design for it (build-presence alone is not proof a job will succeed) — not a promised feature, just an honest "not yet, and not without proof it actually works."
+`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). It never affects `ok` or any tool's `usable`. The one GPU path a tool uses is Apple VideoToolbox, opt-in: `--hw` on every re-encoding tool (h264/hevc/prores on the GPU, AV1 stays on SVT-AV1), `FFMPEG_SKILL_HW=1` to make it this machine's default for everything except `export.py`'s delivery presets (those take `--hw` explicitly, or `render.py --hw` for a whole project). It runs only on Apple Silicon and never by default; it falls back to the CPU encoder — reported in the result's `hw.fallback`/`hw.notes` — only when VideoToolbox itself refuses a job (any other ffmpeg failure is reported as it happened, not retried); every result names the `encoder` that actually ran (`batch.py`, which encodes nothing itself: each `results` row), and a GPU encode adds a `notes` line that its quality is not CRF-equivalent. Measured on an M4 Max, 2–7× faster than x264/x265 `medium`; at matched SSIM (`tests/bench_vt.py`) 1.1–2.9× the bytes on SDR and 1.9–3.5× on HDR, which is why it is a flag and not the default (`docs/design-decisions.md`). `doctor --json`'s `hw` says whether this machine can run it (`platform_ok`) and whether it is the default here (`default_on`).
## Gotchas and best practices
diff --git a/SKILL.md b/SKILL.md
index fa9543d..d23e38d 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -7,7 +7,7 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script about to run is the cheapest full flag list. A reference file costs as much as this one; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end). MCP: `tools/list` shows only the core 12 by default; the other 30 (per this table) stay callable by name via `tools/call`, described by `contract --json`; set `FFMPEG_SKILL_MCP_FULL=1` for all 42.
-Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (status/output/verified plus a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
+Shared flags (every script): `--dry-run`; `--json` (output path, its probe, the commands run); `--json-brief` (status/output/verified + a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress` (percent/ETA); `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (a dry run `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--hw` (Apple Silicon GPU, big files; `FFMPEG_SKILL_HW=1` = default, not export), `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe, skipping artifacts/side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per tool: `contract --json` `dry_run`.
@@ -16,7 +16,7 @@ Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `chec
0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok`/`usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
1. **Probe what you must plan from.** Run `probe.py` on each input you plan from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts.
-3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
+3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json.**
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix only when the request implies the answer, else state the choice and its cost. Mention WARNs; do not chase them.
6. **Verify the output.** Confirm duration, resolution, fps and audio match the request, from the writing tool's `--json`/`--json-brief` probe or `probe.py`, and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
@@ -216,4 +216,4 @@ One line each; open the linked `references/gotchas.md` section when the job is i
- `--fit crop` 16:9→9:16 throws away 70% of the width; "60 seconds" by speed or by trim are different answers — say which and why. -> [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
- TikTok/Reels cover the bottom fifth and right column with their own UI — templates keep text out of those zones; `look.py --safe tiktok` shows them. -> [#platform-safe-zones](references/gotchas.md#platform-safe-zones)
- `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before trusting picks. -> [#highlights](references/gotchas.md#highlights)
-- Re-encodes use x264 `medium`; chain 3+ in one `render.py` project. -> [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
+- CPU re-encodes: x264 `medium`; chain 3+ in one `render.py` project. -> [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
diff --git a/docs/contract.md b/docs/contract.md
index 32da811..da1e84d 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -216,6 +216,15 @@ Main10 for HDR), so the flags add no behaviour to a caller that does not pass th
encoder each value needs is listed under the tool's optional capabilities (`--codec hevc` and
so on). `export.py` refuses `--codec`: its presets decide the codec.
+`--hw` / `--no-hw` are on the same tools and on `export.py`, marked `common`;
+`render.py` and `batch.py` take them for every stage they run. They are two booleans in the
+input schema (`hw`, `no_hw`; MCP sends `--hw` for `hw: true` and `--no-hw` for `no_hw: true`),
+mutually exclusive. `--hw` moves an h264/hevc/prores encode to Apple VideoToolbox (Apple Silicon
+only; AV1 stays on the CPU); `--no-hw` keeps it on the CPU even when `FFMPEG_SKILL_HW=1` makes the
+GPU this machine's default. Neither changes anything for a caller that passes neither on a
+machine without the variable. What ran is in the result's `encoder` and `hw` (see the keys added
+after 2.2.2, and `doctor`'s `gpu_encoders` below).
+
`--plan FILE` (1.6) is a dry run that also writes a plan document: `{"plan_version": 1,
"tool", "argv", "cwd", "inputs": [{"path", "size", "sha256_head_tail"}], "commands",
"output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
@@ -237,11 +246,17 @@ gives you:
| Hint | Tools |
|---|---|
-| `bit_exact` | probe, check, scenes, look |
+| `bit_exact` | probe, check, scenes, look, metadata, background |
| `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
| `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
| `environment_dependent` | verify |
+The hints describe the CPU encoders. Under `--hw`, or `FFMPEG_SKILL_HW=1` on the machine, a tool
+that encodes video runs VideoToolbox, whose bytes are not promised identical from run to run:
+treat such a result as `content_equivalent` at best. That includes `background`, whose hint stays
+`bit_exact` within 2.x. The variable belongs to the machine, so the same flags can give x264 bytes
+on one machine and VideoToolbox bytes on another; `encoder` in the result says which.
+
## `provides`
`provides` lists these 42 tools by a cross-repository Capability id, for
@@ -339,7 +354,8 @@ Names: `ffmpeg`, `ffprobe`, `encoder:<name>`, `filter:<name>`, `bsf:<name>`,
`available`, `missing` and `missing_optional` are added from `doctor`, which reads
`ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper. Pass `--static`
to omit detection. Nothing from the environment other than those lists and the
-ffmpeg/ffprobe/python versions is printed; no environment variables, no paths.
+ffmpeg/ffprobe/python versions is printed; no environment variables, no paths (`doctor`'s
+`hw.default_on` is a boolean derived from `FFMPEG_SKILL_HW`, never its value).
`external:whisper` is **optional** for two tools since 1.17: `caption.py`
(`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requires
@@ -380,9 +396,14 @@ for tools that don't need `subtitles`/`drawtext`/`zscale`, but `caption.usable`
"parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. It proves the build shipped the
capability, not that the GPU/driver on this machine will accept a job (that needs a real
encode, which this introspection never runs). No tool declares or requires a GPU encoder, so
-`gpu_encoders` never affects `ok` or any tool's `usable` — it exists purely so a caller can ask
-the same honest yes/no/unknown question about GPU support that filter/encoder detection already
-answers for everything else, without a tool here needing to use one.
+`gpu_encoders` never affects `ok` or any tool's `usable`. `--hw` (and
+`FFMPEG_SKILL_HW=1`, which `export.py`'s delivery presets ignore) puts h264/hevc/prores encodes on
+VideoToolbox on Apple Silicon only, never by default. A job VideoToolbox refuses (the encoder could
+not open, or its compression session failed) is re-encoded on the CPU and said so in the result's
+`hw.fallback`/`hw.notes`; any other failure is not retried. When `FFMPEG_SKILL_HW=1` rather than `--hw` chose the GPU and the GPU ran,
+`hw.notes` also says so, with the file-size cost and `--no-hw` as the remedy for a final
+deliverable. `doctor`'s `hw` field is `{platform_ok, default_on}` — normalised
+booleans, never the environment value.
`doctor`'s `fonts` field reports whether the default drawtext font (`caption.py`'s
`--animate`/`--karaoke`, `graphics.py`'s templates — `BRAND_DEFAULTS["font"]`, `"DejaVu Sans"`)
@@ -493,7 +514,7 @@ Per-tool keys added in 1.17, all additive:
| `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
| `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
| `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
-| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
+| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
Per-tool keys added in 1.17.1, all additive:
@@ -524,6 +545,7 @@ Per-tool keys added after 2.2.2, all additive:
| `silent` | `join.py` (every run) | `[{index, path, peak_db, at, end}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `at`/`end` are where the input sits in the output, in seconds (a transition overlaps its neighbours). An input named by `--allow-silent N[,N]` (1-based) is a planned pause: always joined, listed with `intended: true`, no warning or note, whatever `--on-silent` says. `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
| `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
| `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
+| `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |
`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index 6b89920..bdd4cd0 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -825,3 +825,127 @@ not a new file format this tool would have to maintain.
cut), the picture's otherwise (the tail is trimmed). Tests:
`test_join_none_holds_a_picture_shorter_than_its_sound`,
`test_join_none_trims_a_sound_tail_under_a_frame`.
+
+## Unreleased — the GPU
+
+- **`--hw` is opt-in, and a delivery preset needs it explicitly.** Measured first on an M4 Max
+ (FFmpeg 9.0, three 1080p SDR clips) and re-measured with `tests/bench_vt.py` on an M3 (FFmpeg
+ 9.0.2, SSIM against x264/x265 `medium` at CRF 18/23/28, iPhone footage included): VideoToolbox
+ is 2–7× faster but needs 1.1–2.9× the bytes for the same SSIM on SDR and 1.9–3.5× on HDR. A draft or an
+ intermediate is the place for speed; a file that is uploaded is the place for bytes. So
+ `FFMPEG_SKILL_HW=1` changes every re-encoding tool's default but not `export.py`'s delivery
+ presets, and `render.py --hw` is the one switch that puts a whole project, export included, on
+ the GPU. It is never switched on by anything else (not `--fast`, not a detected GPU). An
+ `export.py` run under the variable says in `hw` (`requested: false`, `source: env`, a note) that
+ the preset stayed on the CPU and that `--hw` moves it. Code: `_common.runner.apply_common`,
+ `add_hw_orchestrator_args`. Tests: `HwResolutionTests`, `test_export_preset_needs_an_explicit_hw`
+ (a dry run with VideoToolbox stood in, so it runs on every CI runner),
+ `test_the_env_default_reaches_the_other_re_encoding_tools`.
+- **CRF maps to VideoToolbox `-q:v` on the safe side, with a separate HDR curve.**
+ `vt_quality` takes, at each CRF, the highest `-q:v` whose SSIM matched the CPU encode across
+ `tests/bench_vt.py`'s clips (synthetic CG, fractal and grain, iPhone 17 Pro SDR and HLG at 30
+ and 60 fps), so a GPU encode never measures below the CPU one; the other clips pay in bytes.
+ HDR has its own curve (63 − 1.6 per CRF step against SDR H.264's 75 − 2.2 and HEVC's 78 − 2.0).
+ Its CPU line is x265 Main10 at CRF+2, and the Main10 VideoToolbox line needs ~11–15 less
+ `-q:v` for that SSIM. On the first fit's SDR curve, HLG phone footage came out at 6–8× x265's
+ bytes at a higher SSIM (0.998 against 0.989). The split is on `bt2020_or_hdr`, a probe fact,
+ and only the HEVC line takes it: an export preset's H.264 line on an HDR source replaces x264
+ at the same CRF. The HDR clips' matches spread by 6.6 at CRF 18 and ~10 at 23/28, 60 fps
+ lowest. One curve set by the 30 fps clip covers them, and the 60 fps clips pay 2.6–3.5× x265's
+ bytes; a content class or a bitrate ceiling was not worth that. Three rows match within 0.3
+ `-q:v`, inside VideoToolbox's run-to-run noise, so "never below" means within noise. SSIM is the
+ metric the first fit used: x264/x265 psy tuning lowers their SSIM at equal visual quality, so a
+ matched `-q:v` leans low. Frames are paired by index (`settb=1/1000,setpts=N`), because
+ `setpts=N/FRAME_RATE/TB` rounds in a 1/1000 or 1/600 time base and pairs two frames in three
+ with a neighbour. Tests: `test_quality_mapping_is_monotonic_and_bounded`,
+ `test_the_curves_keep_the_measured_values`, `test_an_hdr_source_gets_the_hdr_quality_curve`,
+ `test_an_h264_line_on_an_hdr_source_keeps_the_h264_curve`.
+- **`--quality` on the GPU is an approximation of the CRF encode, said so, and `verified` does not
+ change meaning.** `-q:v` is not a CRF: VideoToolbox has no rate-distortion model that x264/x265's
+ CRF promises a quality from, and the curves above are a fit at three CRFs on a handful of clips,
+ pinned to the measured values, not derived. A content type the bench did not hold can land above
+ or below the CPU encode's quality, and the bytes differ by 1.1–3.5×. So every VideoToolbox
+ encode adds a top-level `notes` line naming the encoder and its `-q:v`, saying it is not
+ CRF-equivalent and that `--no-hw` gives the CRF encode. `verified` stays what it is for every
+ tool, the output's measured properties (written, probed, the tool's own measured steps: loudness,
+ duration, a platform check); none of them reads the encoder line or assumes x264/CRF, so a GPU
+ file verifies exactly as the same file from the CPU would. Tests:
+ `test_a_gpu_encode_notes_that_its_quality_is_not_crf_equivalent`,
+ `test_verified_is_the_measured_output_whatever_the_encoder`,
+ `test_the_curves_keep_the_measured_values`.
+- **VideoToolbox BT.709 tags go through a bitstream filter.** The ≥7.1 reason `bt709_tag_args`
+ uses encoder VUI parameters holds for VideoToolbox too, and it has no `-x264-params`; the
+ `-colorspace` output options put a real matrix conversion on an untagged source (24 dB PSNR,
+ tag-neutral). `h264_metadata`/`hevc_metadata` write the VUI after encoding (49.9 dB). Test:
+ `test_hw_encode_reports_itself_and_keeps_an_untagged_source_unconverted`.
+- **A job VideoToolbox refuses is re-encoded on the CPU, and the result says so; no other failure
+ is retried.** VideoToolbox has hard limits (H.264 stops at 4096 wide) that listing the encoder
+ cannot reveal. `run()` swaps the recorded VideoToolbox arguments back to the CPU line they
+ replaced and retries once; `hw.used: false`, `hw.fallback: true` and `hw.notes` (with the line
+ ffmpeg printed) report it, and `encoder` names what really ran. It retries only when stderr shows
+ VideoToolbox's own failure, read from the FFmpeg 5.1–8.0 and master sources: "Error while opening
+ encoder" on a line naming a `*_videotoolbox` encoder (6.0+ log contexts `[vost#N:M/<encoder>]`,
+ 8.0+ `[enc:<encoder>]`; 5.1's line names no encoder, so there it counts beside an error the
+ encoder logged in its own `[<encoder> @ …]` context), or the encoder's compression-session
+ failures in that context ("create compression session", "prepare encoder", "encode frame",
+ "encoding frame", matched without case because master drops the "Error: " prefix). A bad filter
+ graph, a missing font (7.0+ then prints "Could not open encoder before EOF", which names the
+ encoder but means it never got a frame) or a full disk is the same failure on the CPU: retrying
+ ran the job twice and reported the GPU as its cause. Those fail once, and the failure document
+ names the encoder of the command that ran, says it was not retried, and carries `encoder`/`hw`;
+ when the CPU retry fails too, the message and stderr are the CPU command's. Code:
+ `_common.runner.videotoolbox_failure`, `run`, `_fail`. Tests: `VideoToolboxFailureTests`,
+ `test_a_videotoolbox_open_failure_is_re_encoded_on_the_cpu_and_reported`,
+ `test_a_failure_that_is_not_videotoolbox_s_is_not_retried_and_names_the_command_that_ran`,
+ `test_when_the_cpu_retry_fails_too_the_error_is_the_cpu_command_s` (a fake ffmpeg, every CI
+ runner); `test_a_job_videotoolbox_refuses_falls_back_to_the_cpu_and_says_so` (real hardware).
+ The CPU retry gets the odd-dimension retry a CPU-only run gets
+ (`test_an_odd_sized_job_videotoolbox_refused_gets_the_even_scale_on_the_cpu`), and on 5.1 the
+ encoder's non-fatal property lines do not count as its refusal beside another stream's open
+ failure (`NOT_VT`, the 5.1 audio-encoder case).
+- **The encoder list is `ffmpeg -encoders`; `ffprobe -encoders` only under `--dry-run`.** The
+ AV1/ProRes refusals and the VideoToolbox check must describe the binary that encodes, and on a
+ mixed install (a Homebrew ffprobe next to a static ffmpeg) ffprobe's list is another build's. A
+ dry run promises never to run ffmpeg, so it reads ffprobe's, as `ffmpeg_version()` always does;
+ the real run re-decides against ffmpeg's. Code: `_common.runner.ffmpeg_encoders`. Tests:
+ `EncoderListTests`, `test_hw_dry_run_runs_no_ffmpeg`.
+- **HDR10 side data survives VideoToolbox** (mastering display, content light level), measured
+ on FFmpeg 9.0 — no note is raised for it. Test: `test_hw_hdr10_side_data_survives`.
+- **An `export.py` preset on the GPU writes the preset's format, and its CPU fallback is the
+ preset's own line.** A fixed h264/h265 preset is 8-bit with BT.709 tags whatever the source (on
+ an HDR source it says so in a note, as without `--hw`), so its VideoToolbox line is built as for
+ an SDR source, on the SDR curve, rather than as the Main10 HDR line the re-encoding tools use;
+ otherwise `--hw` changed the delivered format and a GPU→CPU fallback changed it back. The CPU
+ line recorded for that fallback carries the BT.709 tags only where `export.py` adds them (not
+ `prores`, a master that keeps the source's). Code: `_common.decision.hw_preset_video`. Tests:
+ `test_an_h265_preset_on_an_hdr_source_stays_the_preset_s_8_bit_bt709_format`,
+ `test_the_prores_preset_s_cpu_fallback_keeps_the_source_s_tags`.
+- **`--hw` and `--no-hw` are two booleans, not one dest.** The contract's `input_schema` keeps one
+ property per argparse dest, and the MCP transport turns `true` into the property's flag, so a
+ store_true/store_false pair on one dest published only `--no-hw`, under the name `hw`. Each flag
+ has its own dest (`hw`, `no_hw`, mutually exclusive), `hw_flag()` reads the pair back, and the
+ generator refuses any parser whose actions share a dest. Tests:
+ `test_every_flag_of_every_parser_is_published_once_under_its_own_name`.
+- **Under `--dry-run`, `hw.used` is null; `encoder` is the planned encoder.** `used` says what ran,
+ and a dry run ran nothing (the same rule as `waveform.py`'s `silent`). `encoder` keeps naming the
+ encoder of the planned command, as `commands` lists planned commands, and the
+ `FFMPEG_SKILL_HW` note still says what the GPU will cost. Test:
+ `test_a_dry_run_reports_the_planned_encoder_and_no_used`.
+- **A tool that runs other tools passes the GPU choice on and reports its stages.** `render.py`
+ and `batch.py` export `--hw`/`--no-hw` to their stages; `waveform.py` passes its explicit flag to
+ the caption/title stage, which writes its deliverable, and folds that stage's commands and `hw`
+ into its own result. `render.py` (stages and an executed plan) and `batch.py` (items, into its
+ top-level `hw`, `used` null) carry what their children reported, a failed stage's `encoder` and
+ `hw` included. `batch.py`'s item cache, like `render.py --cache`, keys on the GPU setting (the
+ suffix is empty when every step is on the CPU, so older caches still hit), and an entry whose
+ output a later run rewrote is dropped. Tests: `test_batch_never_serves_a_cached_item_across_the_gpu_setting`,
+ `test_batch_s_top_level_hw_carries_its_items`, `test_a_hw_plan_executed_by_render_reports_hw`,
+ `test_render_s_failed_stage_keeps_its_encoder_and_the_earlier_stages_fallback`,
+ `test_waveform_passes_its_gpu_choice_to_the_stage_that_writes_the_file`,
+ `test_waveform_s_no_hw_overrides_the_variable_for_its_stage_too`.
+- **Apple Silicon is the hardware, not the interpreter.** An x86_64 Python under Rosetta reports
+ `platform.machine()` as `x86_64` on an M-series Mac; `sysctl -n hw.optional.arm64` still reads 1
+ there. Whether `-q:v` works is the ffmpeg binary's build, and an x86_64 ffmpeg that refuses it is
+ re-encoded on the CPU by the fallback above, with ffmpeg's line in the note. Code:
+ `_common.runner.apple_silicon`; `doctor`'s `hw.platform_ok` uses the same check. Test:
+ `test_hw_runs_on_apple_silicon_hardware_under_a_rosetta_python`.
diff --git a/docs/roadmap.md b/docs/roadmap.md
index 9485aa2..13e7695 100644
--- a/docs/roadmap.md
+++ b/docs/roadmap.md
@@ -616,9 +616,19 @@ truth-up. No further action was needed there.
issue #143.
- ffmpeg 8 / 9: filter and encoder fixtures refreshed, `bt709_tag_args` and the colour
negotiation path re-verified on each; a compatibility table in `references/devices.md`.
-- `--hwaccel auto` (opt-in): videotoolbox / vaapi / nvenc for previews (`--fast`) only,
- never for the final encode unless `--hwaccel final` is given, with the encoder named in the
- result so a difference is traceable.
+- GPU encoding -- **shipped differently than planned, after 2.5.1 (#304).** The plan was
+ `--hwaccel auto` over videotoolbox / vaapi / nvenc, for `--fast` previews only unless
+ `--hwaccel final`. What ships is Apple VideoToolbox only (Apple Silicon; h264/hevc/prores,
+ AV1 stays on SVT-AV1), as `--hw`/`--no-hw` on every re-encoding tool and `export.py`. It is
+ never automatic and not tied to `--fast`: `--hw` asks for it on one call, `FFMPEG_SKILL_HW=1`
+ makes it a machine's default for every re-encoding tool except `export.py`'s delivery presets,
+ which take only an explicit `--hw` (or `render.py`/`batch.py --hw`, explicit for every stage).
+ A job VideoToolbox itself refuses is re-encoded on the CPU; nothing else is retried. Every
+ result names the `encoder` that ran and carries `hw` (`requested`, `source`, `used`,
+ `fallback`, `notes`), and a GPU encode notes that its quality is not CRF-equivalent: `--quality`
+ maps to `-q:v` by an approximate SSIM fit, at 1.1-3.5x the bytes. vaapi / nvenc / qsv are not
+ planned: each needs a real-hardware-verified quality mapping and refusal list, as VideoToolbox
+ got (`tests/bench_vt.py`, docs/design-decisions.md "Unreleased -- the GPU").
- Eval 20 on the Windows and macOS runners; it also grades whether agents quote the metrics
rather than re-probe.
diff --git a/references/gotchas.md b/references/gotchas.md
index 6ed5877..9b4a126 100644
--- a/references/gotchas.md
+++ b/references/gotchas.md
@@ -267,7 +267,13 @@ the contact sheet (`--sheet`) before treating the picks as final.
Anything chained by hand through three re-encodes should be one `render.py`
project instead, so the plan is one file and the user can change one number.
Re-encodes use x264 `medium`; for long files add `--preset veryfast` to
-intermediates and keep the default for the final export.
+intermediates and keep the default for the final export. On Apple Silicon,
+`--hw` (or `FFMPEG_SKILL_HW=1` for the machine) moves h264/hevc/prores encodes to
+VideoToolbox: 2–7× faster, 1.1–2.9× the bytes at the same quality (1.9–3.5× on HDR). The variable
+leaves `export.py`'s delivery presets on the CPU, which is the usual split (GPU
+intermediates, CPU delivery); `render.py --hw` puts the export on the GPU too.
+Check `encoder`/`hw` in the result: a job the GPU refuses (H.264 wider than 4096)
+was re-encoded on the CPU.
## Audio-only files
diff --git a/references/scripts.md b/references/scripts.md
index 1a35fdd..4581b18 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -1,6 +1,6 @@
# Script reference
-Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither flag; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
+Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
## Time grammar (every time-taking flag, 1.9)
diff --git a/scripts/_common/decision.py b/scripts/_common/decision.py
index 349c9e5..cb6e6c7 100644
--- a/scripts/_common/decision.py
+++ b/scripts/_common/decision.py
@@ -16,7 +16,7 @@ from pathlib import Path
from typing import Any, Dict, List, Optional, Sequence, Tuple
from _common.color import bt709_tag_args, _sdr_bt709
from _common.emit import die
-from _common.runner import CODECS, STATE, ffmpeg_encoders
+from _common.runner import CODECS, STATE, ffmpeg_encoders, ffmpeg_version, hw_platform_reason
def pad_filters(out_w: int, out_h: int, fill: str, color: str, blur: int, darken: float = 0.0) -> str:
@@ -277,7 +277,126 @@ def cfr_args(meta: Optional[Dict[str, Any]], fps: Optional[float] = None) -> Lis
return ["-fps_mode", "cfr", "-r", f"{rate:g}"]
+# ------------------------------------------------------- VideoToolbox (--hw)
+#
+# CRF -> VideoToolbox -q:v (1-100, higher = better): at each CRF the highest -q:v whose SSIM matched
+# the CPU line's (x264/x265 medium) across the clips of tests/bench_vt.py -- synthetic CG, fractal
+# and grain plus iPhone 17 Pro 1080p (M3, FFmpeg 9.0.2), so a GPU encode never measures below the
+# CPU one. HDR has its own curve: its CPU line is x265 Main10 at CRF+2, and the Main10 VideoToolbox
+# line needs ~11-15 less -q:v for that SSIM (the SDR curve wrote HLG phone footage at 6-8x x265's
+# bytes). Matched quality costs 1.1-2.9x the bytes of x264/x265 on SDR and 1.9-3.5x on HDR --
+# which is why --hw is opt-in and delivery presets stay on the CPU (docs/design-decisions.md).
+
+
+def vt_quality(codec: str, crf: int, hdr: bool = False) -> int:
+ base, slope = (63.0, 1.6) if hdr else (75.0, 2.2) if codec == "h264" else (78.0, 2.0)
+ return max(1, min(100, int(round(base - slope * (crf - 18)))))
+
+
+def _vt_bt709(codec: str) -> List[str]:
+ """BT.709 tags for a VideoToolbox SDR encode. From FFmpeg 7.1 the -colorspace output options
+ drive a real matrix conversion on an untagged source (see bt709_tag_args), and VideoToolbox has
+ no -x264-params VUI route; a bitstream filter writes the VUI after encoding, which the filter
+ graph never sees (measured: 49.9 dB PSNR tag-neutral vs 23.8 dB through the output options).
+ It is used on every version, not only >= 7.1: it is tag-only everywhere, and a git build of
+ 7.1 reads as (7, 0) from its libavutil major, which would otherwise take the converting path."""
+ return ["-bsf:v", f"{codec}_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1"]
+
+
+def _vt_args(codec: str, crf: int, meta: Optional[Dict[str, Any]], keep_bt709: bool) -> Optional[List[str]]:
+ """VideoToolbox encoder options for `codec`, or None (with a note in STATE.hw_notes) when the
+ request stays on the CPU: no VideoToolbox encoder for the codec, this machine cannot, or the
+ ffmpeg build lacks it."""
+ name = {"h264": "h264_videotoolbox", "hevc": "hevc_videotoolbox", "prores": "prores_videotoolbox"}.get(codec)
+ reason = None if name else f"no VideoToolbox encoder for {codec}"
+ reason = reason or hw_platform_reason()
+ if not reason and name not in ffmpeg_encoders():
+ reason = f"this ffmpeg build has no {name}"
+ if reason:
+ note = f"{reason}; encoded on the CPU"
+ if note not in STATE.hw_notes:
+ STATE.hw_notes.append(note)
+ return None
+ v = (meta or {}).get("video") or {}
+ hdr = bool(v.get("bt2020_or_hdr"))
+ hdr_tags = ["-colorspace", v.get("color_space") or "bt2020nc", "-color_primaries", v.get("color_primaries") or "bt2020",
+ "-color_trc", v.get("color_transfer") or "arib-std-b67"]
+ if codec == "prores":
+ return ["-c:v", name, "-profile:v", "hq", "-pix_fmt", "p210le"] + (hdr_tags if hdr else [])
+ # The HDR curve was fitted on the Main10 HEVC line; an 8-bit H.264 line (an export preset on
+ # an HDR source) replaces x264 at the same CRF and keeps the H.264 curve.
+ q = str(vt_quality(codec, crf, hdr and codec == "hevc"))
+ if codec == "h264":
+ return ["-c:v", name, "-q:v", q, "-profile:v", "high", "-pix_fmt", "yuv420p", "-movflags", "+faststart"] + \
+ (_vt_bt709("h264") if keep_bt709 else [])
+ if hdr:
+ # Measured on FFmpeg 9.0: HDR10 mastering-display and content-light side data carry through
+ # hevc_videotoolbox exactly as through libx265 (test_hw_hdr10_side_data_survives).
+ return ["-c:v", name, "-q:v", q, "-profile:v", "main10", "-pix_fmt", "p010le", "-tag:v", "hvc1"] + hdr_tags + ["-movflags", "+faststart"]
+ return ["-c:v", name, "-q:v", q, "-pix_fmt", "yuv420p", "-tag:v", "hvc1", "-movflags", "+faststart"] + \
+ (_vt_bt709("hevc") if keep_bt709 else [])
+
+
+def _maybe_hw(codec: str, crf: int, meta: Optional[Dict[str, Any]], keep_bt709: bool, cpu: List[str]) -> List[str]:
+ """`cpu` unchanged unless --hw / $FFMPEG_SKILL_HW is on and VideoToolbox can take `codec`; the
+ swap is remembered so run() can put the CPU line back if the GPU refuses the job."""
+ if not STATE.hw:
+ return cpu
+ vt = _vt_args(codec, crf, meta, keep_bt709)
+ if vt is None:
+ return cpu
+ # a copy: a caller that edits the returned line in place (export.py strips -movflags) must
+ # not edit the recorded slice with it, or restate_last_swap() never sees the line as built
+ STATE.hw_swaps.append((list(vt), cpu))
+ return vt
+
+
+_CPU_TO_CODEC = {"libx264": "h264", "libx265": "hevc", "prores_ks": "prores"}
+
+
+def hw_preset_video(video: List[str], meta: Optional[Dict[str, Any]], *, bt709: bool) -> List[str]:
+ """export.py's fixed-preset encoder line moved to VideoToolbox (same codec, the preset's -crf
+ mapped to -q:v, its -r kept); unchanged -- with the reason noted -- when that cannot apply.
+
+ `bt709`: whether export.py tags this preset's CPU line BT.709 (every fixed preset but prores,
+ whose master keeps the source's tags). The VideoToolbox line writes the format the CPU preset
+ writes: an h264/hevc preset is 8-bit with BT.709 tags whatever the source (on an HDR source it
+ says so in a note, as without --hw), so it is built as for an SDR source -- not the Main10 HDR
+ line, which would make the GPU file and its CPU fallback different formats."""
+ enc = video[video.index("-c:v") + 1] if "-c:v" in video else None
+ codec = _CPU_TO_CODEC.get(enc or "")
+ if codec is None:
+ return video
+ crf = int(video[video.index("-crf") + 1]) if "-crf" in video else 18
+ vt = _vt_args(codec, crf, meta if codec == "prores" else None, bt709)
+ if vt is None:
+ return video
+ vt = strip_movflags(vt) # export adds it
+ if "-r" in video:
+ vt += ["-r", video[video.index("-r") + 1]]
+ # the CPU line export would have written: the preset's own options plus the BT.709 tags export
+ # adds to it (only to the CPU line), so a GPU->CPU fallback is tagged exactly like a CPU run
+ STATE.hw_swaps.append((list(vt), list(video) + (bt709_tag_args(enc) if bt709 else [])))
+ return vt
+
+
+def strip_movflags(args: List[str]) -> List[str]:
+ return [a for i, a in enumerate(args) if not (a == "-movflags" or (i and args[i - 1] == "-movflags"))]
+
+
+def restate_last_swap(before: List[str], after: List[str]) -> None:
+ """A caller that edits an encoder line after encoder_args() returned it (export.py strips
+ -movflags) restates the recorded swap, so run()'s GPU->CPU fallback still finds the slice."""
+ if STATE.hw_swaps and STATE.hw_swaps[-1][0] == before:
+ STATE.hw_swaps[-1] = (list(after), strip_movflags(STATE.hw_swaps[-1][1]))
+
+
def encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str, Any]] = None, keep_bt709: bool = True) -> List[str]:
+ """encoder options for `codec` (_cpu_encoder_args), on VideoToolbox when --hw asks and it can."""
+ return _maybe_hw(codec, crf, meta, keep_bt709, _cpu_encoder_args(codec, crf, preset, meta, keep_bt709))
+
+
+def _cpu_encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str, Any]] = None, keep_bt709: bool = True) -> List[str]:
"""The one place that turns (--codec, --quality, --preset, source) into encoder options.
h264 -> x264 8-bit BT.709 (refuses HDR: 8-bit H.264 cannot carry it); hevc -> x265, Main10
@@ -340,7 +459,7 @@ def x264_args(crf: int = 18, preset: str = "medium", keep_bt709: bool = True) ->
(color.py's --to-sdr path builds its own H.264 line; the flag still has to reach it)."""
if STATE.codec and STATE.codec != "h264":
return encoder_args(STATE.codec, crf, preset, None, keep_bt709)
- return _x264_raw(crf, preset, keep_bt709)
+ return _maybe_hw("h264", crf, None, keep_bt709, _x264_raw(crf, preset, keep_bt709))
def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium") -> List[str]:
@@ -356,6 +475,10 @@ def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "med
v = (meta or {}).get("video") or {}
if not v.get("bt2020_or_hdr"):
return x264_args(crf, preset)
+ return _maybe_hw("hevc", crf, meta, True, _x265_hdr_default(v, crf, preset))
+
+
+def _x265_hdr_default(v: Dict[str, Any], crf: int, preset: str) -> List[str]:
cs = v.get("color_space") or "bt2020nc"
prim = v.get("color_primaries") or "bt2020"
trc = v.get("color_transfer") or "arib-std-b67"
diff --git a/scripts/_common/emit.py b/scripts/_common/emit.py
index 262fb42..bba88c2 100644
--- a/scripts/_common/emit.py
+++ b/scripts/_common/emit.py
@@ -9,7 +9,7 @@ import json
import math
import os
import sys
-from typing import Any, Dict, List, Optional, Sequence
+from typing import Any, Dict, List, Optional, Sequence, Tuple
from _common.runner import Context, ERROR_CODE, ERROR_RETRYABLE, STATE
@@ -28,6 +28,14 @@ def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]
ctx.plan = None # a failed run plans nothing (the exit hook must not write a plan for it)
STATE.plan = None # the hook falls back to STATE when nothing passed a ctx; a failed run plans nothing there either
sys.stderr.write(f"error: {msg}\n" + (f"hint: {hint}\n" if hint else ""))
+ if kind == "ffmpeg" and "hw" not in extra and (ctx.hw or ctx.hw_stages or ctx.hw_env_ignored):
+ # a tool's own ffmpeg failure (cut.py runs run(check=False) and dies itself; render.py
+ # re-raises a stage's) carries `encoder` and `hw` as run()'s _fail() does: under --hw /
+ # $FFMPEG_SKILL_HW two encoders may have run, and a caller must know whose failure it is
+ enc = extra.get("encoder") or _video_encoders(ctx.commands)[0]
+ if enc and "encoder" not in extra:
+ extra["encoder"] = enc
+ extra["hw"] = hw_report(ctx, enc)
if ctx.json:
doc: Dict[str, Any] = {
"status": "failed", "exit_code": code,
@@ -63,6 +71,138 @@ def _set_current_ctx(ctx: "Context") -> None:
_CURRENT_CTX = ctx
+ENV_HW_NOTE = ("VideoToolbox chosen by FFMPEG_SKILL_HW=1 (machine default; ~1.1-3.5x the bytes of x264/x265 "
+ "at matched quality); rerun with --no-hw for a final deliverable")
+ENV_NOT_FOR_DELIVERY_NOTE = ("FFMPEG_SKILL_HW=1 is not applied to export.py's delivery presets (a delivered file is where "
+ "VideoToolbox's larger files cost most); pass --hw to put this export on the GPU")
+
+
+def _video_encoders(commands: Sequence[str]) -> "Tuple[Optional[str], Optional[str]]":
+ """(the last video encoder the command lines name, or `copy` for a stream copy; that encoder's
+ command line). A later copy (loudness.py after export's encode) keeps the encode before it."""
+ import re as _re
+ enc, line, copied = None, None, False
+ for c in commands:
+ for name in _re.findall(r"(?:^|\s)-(?:c:v|vcodec|codec:v)\s+(\S+)", c):
+ if name == "copy":
+ copied = True
+ else:
+ enc, line = name, c
+ if _re.search(r"(?:^|\s)-(?:c|codec)\s+copy(?:\s|$)", c):
+ copied = True
+ if enc is None and copied:
+ return "copy", None
+ return enc, line
+
+
+def hw_report(ctx: "Context", enc: Optional[str]) -> Dict[str, Any]:
+ """The `hw` object for a run whose last video encoder was `enc` (None: this process recorded
+ no encode -- batch.py, whose stages are child processes). `requested`: VideoToolbox was asked
+ for this run's encodes (by --hw, by $FFMPEG_SKILL_HW where it applies, or for a stage);
+ `source`: flag|env; `used`: the encoder that ran last is VideoToolbox; `fallback`: a job
+ VideoToolbox refused was re-encoded on the CPU, here or in a stage; `notes`: why. Under
+ --dry-run `used` is null and `fallback` false: nothing ran."""
+ stages = list(ctx.hw_stages)
+ requested = bool(ctx.hw) or any(st.get("requested") for st in stages)
+ source = ctx.hw_source if ctx.hw else next((st.get("source") for st in stages if st.get("requested")), None)
+ if not requested and ctx.hw_env_ignored:
+ source = "env"
+ # a stage that reported its source without asking (export.py under the variable: requested
+ # false, source env) still names where the request came from
+ source = source or next((st.get("source") for st in stages if st.get("source")), None)
+ vt = enc is not None and enc.endswith("_videotoolbox")
+ # `used` is what ran; a dry run ran nothing (the planned encoder is `encoder`), so null --
+ # as waveform.py's `silent` is null when nothing was measured
+ used = None if enc is None or ctx.dry_run else vt
+ notes = list(ctx.hw_notes)
+ if vt and source == "env" and not any(n.endswith(ENV_HW_NOTE) for n in notes):
+ # the machine default, not this call, chose the GPU: say what it costs and how to opt out
+ # (once: a stage that ran on it already said so)
+ notes.append(ENV_HW_NOTE)
+ if ctx.hw_env_ignored and enc not in (None, "copy"):
+ notes.append(ENV_NOT_FOR_DELIVERY_NOTE)
+ return {"requested": requested, "source": source, "used": used,
+ "fallback": bool(ctx.hw_fallback) or any(st.get("fallback") for st in stages), "notes": notes}
+
+
+def hw_quality_note(enc: Optional[str], line: Optional[str]) -> Optional[str]:
+ """The `notes` line every VideoToolbox encode carries: its quality is not the CRF encode's."""
+ import re as _re
+ if not enc or not enc.endswith("_videotoolbox"):
+ return None
+ if enc == "prores_videotoolbox":
+ return ("prores_videotoolbox is not prores_ks: the same 422 HQ profile from a different encoder, so size and detail "
+ "differ from the CPU encode; `verified` covers the output's measured properties, not parity with the CPU "
+ "encode; --no-hw for prores_ks")
+ m = _re.search(r"(?:^|\s)-q:v\s+(\d+)", line or "")
+ return (f"{enc}{' -q:v ' + m.group(1) if m else ''} is not CRF-equivalent: --quality (or the preset's CRF) is mapped "
+ "to -q:v by an approximate SSIM fit (docs/design-decisions.md), so size and detail differ from the x264/x265 "
+ "encode (~1.1-3.5x the bytes at matched SSIM); `verified` covers the output's measured properties, not quality "
+ "parity with the CPU encode; --no-hw for the CRF encode")
+
+
+def _encoder_report(ctx: "Context") -> Dict[str, Any]:
+ """`encoder`: the last video encoder this run's ffmpeg commands name (after any GPU->CPU
+ fallback, since run() rewrites the recorded command), `copy` for a stream copy; absent when no
+ command encoded video. `hw` (hw_report): present whenever VideoToolbox was asked for (--hw or
+ $FFMPEG_SKILL_HW, a stage's, or the variable a delivery preset does not take)."""
+ enc, _line = _video_encoders(ctx.commands)
+ out: Dict[str, Any] = {"encoder": enc} if enc else {}
+ if ctx.hw or ctx.hw_stages or ctx.hw_env_ignored:
+ out["hw"] = hw_report(ctx, enc)
+ return out
+
+
+def _hw_notes_for(ctx: "Context") -> List[str]:
+ """Top-level `notes` lines for the GPU encodes this result stands on: this tool's last encode,
+ and (render.py) every stage that ran on VideoToolbox."""
+ enc, line = _video_encoders(ctx.commands)
+ out: List[str] = []
+ for note in [hw_quality_note(enc, line)] + [st.get("quality_note") for st in ctx.hw_stages]:
+ if note and note not in out:
+ out.append(note)
+ return out
+
+
+def absorb_stage_hw(stage: str, doc: Any, ctx: "Optional[Context]" = None) -> None:
+ """render.py runs each stage as a child process: carry the stage's `hw` facts (fell back, and
+ why; ran on VideoToolbox, with its quality note) into this result, which records only the
+ command lines."""
+ ctx = ctx or STATE
+ hw = doc.get("hw") if isinstance(doc, dict) else None
+ if not isinstance(hw, dict):
+ return
+ for n in hw.get("notes") or []:
+ # once: this process may have said it already (waveform.py's own encode), or another stage
+ if n not in ctx.hw_notes and not any(existing.endswith(": " + n) for existing in ctx.hw_notes):
+ ctx.hw_notes.append(f"{stage}: {n}")
+ enc, line = _video_encoders([str(c) for c in (doc.get("commands") or [])])
+ quality = hw_quality_note(enc, line) if hw.get("used") else None
+ ctx.hw_stages.append({"stage": stage, "requested": bool(hw.get("requested")), "source": hw.get("source"),
+ "used": hw.get("used"), "fallback": bool(hw.get("fallback")), "quality_note": quality})
+
+
+def steps_encoder_report(steps: Sequence["Tuple[str, Any]"]) -> Dict[str, Any]:
+ """batch.py's per-item `encoder`, `hw` and `notes`, from the --json documents of the steps
+ (child processes) that made the item: the last encoder a step ran (a later stream copy keeps
+ the encode before it), and their GPU facts as render.py's result carries its stages'."""
+ ctx = Context()
+ ctx.dry_run = STATE.dry_run
+ enc: Optional[str] = None
+ for name, doc in steps:
+ step_enc = doc.get("encoder") if isinstance(doc, dict) else None
+ if step_enc and (step_enc != "copy" or enc is None):
+ enc = step_enc
+ absorb_stage_hw(name, doc, ctx)
+ out: Dict[str, Any] = {"encoder": enc} if enc else {}
+ if ctx.hw_stages:
+ out["hw"] = hw_report(ctx, enc)
+ quality = [st["quality_note"] for st in ctx.hw_stages if st.get("quality_note")]
+ if quality:
+ out["notes"] = list(dict.fromkeys(quality))
+ return out
+
+
def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any) -> None:
"""Final stdout line: the output path, or a JSON document with --json.
@@ -88,7 +228,11 @@ def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any
steps.insert(0, {"step": "exists", "ok": True})
doc["verified"] = not ctx.dry_run and bool(steps) and all(s.get("ok") for s in steps)
doc["verification"] = steps
+ doc.update(_encoder_report(ctx))
doc.update(extra)
+ hw_notes = [n for n in _hw_notes_for(ctx) if n not in (doc.get("notes") or [])]
+ if hw_notes:
+ doc["notes"] = list(doc.get("notes") or []) + hw_notes
if ctx.plan:
doc["plan"] = write_plan(ctx.plan, output, extra, ctx=ctx)
print_json(_brief(doc, meta) if ctx.json_brief else doc)
diff --git a/scripts/_common/runner.py b/scripts/_common/runner.py
index d2786aa..0edc5d4 100644
--- a/scripts/_common/runner.py
+++ b/scripts/_common/runner.py
@@ -155,7 +155,8 @@ class Context:
makes it obvious what run()/emit() depend on and lets tests reset it with ``STATE.reset()``.
"""
- __slots__ = ("dry_run", "json", "json_brief", "progress", "fast", "duration_hint", "commands", "timeout", "overwrite", "written", "preexisting", "plan", "plan_written", "plan_inputs", "codec")
+ __slots__ = ("dry_run", "json", "json_brief", "progress", "fast", "duration_hint", "commands", "timeout", "overwrite", "written", "preexisting", "plan", "plan_written", "plan_inputs", "codec",
+ "hw", "hw_source", "hw_notes", "hw_swaps", "hw_fallback", "hw_env_ignored", "hw_stages")
def __init__(self) -> None:
self.reset()
@@ -176,6 +177,13 @@ class Context:
self.plan_written = False # write_plan() ran (emit or the exit hook), so the hook does not write twice
self.plan_inputs: List[str] = [] # side inputs (srt/ass/lut/font files) a tool named through escape_filter_path
self.codec: Optional[str] = None # --codec: encoder for the re-encode (None = x264 for SDR, x265 for HDR)
+ self.hw = False # --hw / $FFMPEG_SKILL_HW: VideoToolbox encoders where they apply
+ self.hw_source: Optional[str] = None # "flag" (--hw/--no-hw given) or "env" (FFMPEG_SKILL_HW), None = off by default
+ self.hw_notes: List[str] = [] # why an encode stayed on (or fell back to) the CPU, for the result document
+ self.hw_swaps: List[Tuple[List[str], List[str]]] = [] # (VideoToolbox args, the CPU args they replaced) for run()'s fallback
+ self.hw_fallback = False # run() re-encoded a job VideoToolbox refused on the CPU (or a stage reported one)
+ self.hw_env_ignored = False # $FFMPEG_SKILL_HW=1 was set but this tool's delivery preset does not take it (export.py)
+ self.hw_stages: List[Dict[str, Any]] = [] # render.py: the `hw` facts each stage (a child process) reported
STATE = Context()
@@ -206,6 +214,31 @@ def add_common(ap: "argparse.ArgumentParser", codec: bool = True) -> None:
help="video encoder for the re-encode: h264 (x264, the default for SDR), hevc (x265, the default for HDR), av1 (SVT-AV1 or libaom), prores (422 HQ, needs a .mov/.mkv output); HDR sources keep their colour on hevc/av1/prores")
g.add_argument("--quality", type=int, default=ap._defaults["crf"], metavar="N",
help="encoder quality on the CRF scale (default %(default)s; lower = better; 18 visually lossless for x264/x265, up to 63 for av1); ignored by prores")
+ if (codec and "crf" in ap._defaults) or not codec:
+ _add_hw_flags(g, "encode on the GPU with Apple VideoToolbox (h264/hevc/prores; Apple Silicon only): several times faster, "
+ "larger files at the same quality; falls back to the CPU encoder, reported, where it cannot apply"
+ + ("" if codec else ". The only way to put a delivery preset on the GPU: $FFMPEG_SKILL_HW does not apply here"),
+ "encode on the CPU (x264/x265/SVT-AV1/prores_ks) even when $FFMPEG_SKILL_HW=1")
+ ap.set_defaults(_hw_env=bool(codec))
+
+
+def _add_hw_flags(g: Any, hw_help: str, no_hw_help: str) -> None:
+ """--hw and --no-hw, each its own boolean (dests `hw` and `no_hw`). They are not one dest
+ with store_true/store_false: the contract's input_schema keeps one property per dest and the
+ MCP transport turns `true` into the flag, so a shared dest published --no-hw under the name
+ `hw` and an MCP `hw: true` ran on the CPU. hw_flag() reads the pair back as True/False/None."""
+ hw = g.add_mutually_exclusive_group()
+ hw.add_argument("--hw", action="store_true", help=hw_help)
+ hw.add_argument("--no-hw", action="store_true", help=no_hw_help)
+
+
+def hw_flag(args: "argparse.Namespace") -> Optional[bool]:
+ """--hw -> True, --no-hw -> False, neither (or a tool without the flags) -> None."""
+ if getattr(args, "hw", False):
+ return True
+ if getattr(args, "no_hw", False):
+ return False
+ return None
def apply_common(args: "argparse.Namespace") -> None:
@@ -228,6 +261,25 @@ def apply_common(args: "argparse.Namespace") -> None:
if STATE.fast and getattr(args, "preset", None) in X264_PRESETS:
args.preset = "veryfast"
STATE.codec = getattr(args, "codec", None) or None
+ STATE.hw_notes, STATE.hw_swaps, STATE.hw_fallback, STATE.hw_env_ignored, STATE.hw_stages = [], [], False, False, []
+ flag = hw_flag(args)
+ if flag is not None:
+ STATE.hw, STATE.hw_source = bool(flag), "flag"
+ if getattr(args, "_hw_orchestrator", False):
+ # render.py / batch.py --hw|--no-hw: every stage they run inherits it as an explicit
+ # choice -- export.py's delivery presets included, which $FFMPEG_SKILL_HW never reaches
+ os.environ[HW_ENV] = "1" if flag else "0"
+ os.environ[HW_FORCED_ENV] = "1"
+ elif hasattr(args, "hw") and os.environ.get(HW_FORCED_ENV) == "1":
+ STATE.hw, STATE.hw_source = env_hw(), "flag"
+ elif hasattr(args, "hw") and getattr(args, "_hw_env", False) and env_hw():
+ STATE.hw, STATE.hw_source = True, "env"
+ else:
+ STATE.hw, STATE.hw_source = False, None
+ # export.py (a delivery preset decides the encoder): the machine default is not applied,
+ # and the result says so rather than leaving a caller to wonder why the GPU was not used
+ STATE.hw_env_ignored = (hasattr(args, "hw") and env_hw() and not getattr(args, "_hw_env", False)
+ and not getattr(args, "_hw_orchestrator", False))
quality = getattr(args, "quality", None)
if quality is not None:
top = 63 if STATE.codec == "av1" else 51
@@ -354,15 +406,48 @@ def _cleanup_partial_output(cmd: Sequence[str]) -> None:
pass
-def _fail(cmd: Sequence[str], returncode: int, stderr: str) -> None:
+def _fail(cmd: Sequence[str], returncode: int, stderr: str, ctx: "Optional[Context]" = None, retried: bool = False) -> None:
# Partial-output cleanup already ran in the caller (_run_captured/_run_with_progress) for
# every failed ffmpeg invocation, not just this check=True path -- see _cleanup_partial_output.
# The process exit code is always 1 for an ffmpeg failure: ffmpeg's own code (1, 69, 218, 234,
# a negative signal number...) varies by build and by the failing stage, and 124/127/130/143
# are reserved for timeout, missing tool and interrupts. The raw code is kept in the JSON
# document as `ffmpeg_returncode` for a caller that wants it. docs/design-decisions.md.
+ #
+ # `cmd` is the command that ran last: after run()'s GPU->CPU retry (`retried`), the CPU one.
+ # Under --hw / $FFMPEG_SKILL_HW the message names its video encoder and the document carries
+ # `encoder` and `hw`, because two encoders may have run and a caller must know whose stderr
+ # this is.
+ ctx = ctx or STATE
tail = "\n".join(stderr.strip().splitlines()[-15:])
- die(f"command failed ({returncode}): {cmd[0]}\n{tail}", code=1, kind="ffmpeg", ffmpeg_returncode=returncode)
+ what, extra = str(cmd[0]), {}
+ enc = video_encoder_of(cmd) if ctx.hw else None
+ if enc:
+ on_gpu = enc.endswith("_videotoolbox")
+ why = videotoolbox_failure(stderr) if on_gpu else None
+ if not on_gpu and retried:
+ what += f" (video encoder {enc}, the CPU retry after VideoToolbox refused the job)"
+ elif on_gpu and why is None:
+ what += f" (video encoder {enc}; not retried on the CPU: ffmpeg reported no VideoToolbox error)"
+ else:
+ what += f" (video encoder {enc})"
+ report = hw_report(ctx, enc)
+ if on_gpu:
+ report["notes"].append(
+ "the VideoToolbox encode failed and was not retried on the CPU: " +
+ ("ffmpeg reported no encoder-open or compression-session error, so the cause is not the GPU (see the message)"
+ if why is None else f"VideoToolbox refused it ({why}) and no CPU encoder line was recorded to retry with"))
+ extra = {"encoder": enc, "hw": report}
+ die(f"command failed ({returncode}): {what}\n{tail}", code=1, kind="ffmpeg", ffmpeg_returncode=returncode, **extra)
+
+
+def video_encoder_of(cmd: Sequence[str]) -> Optional[str]:
+ """The video encoder an ffmpeg argv names last (-c:v / -vcodec / -codec:v), or None."""
+ enc = None
+ for i, a in enumerate(cmd[:-1]):
+ if a in ("-c:v", "-vcodec", "-codec:v"):
+ enc = str(cmd[i + 1])
+ return enc
def _check_no_overwrite_input(cmd: Sequence[str]) -> None:
@@ -621,6 +706,126 @@ def _stage_existing_output(cmd: Sequence[str]) -> Tuple[List[str], Optional[str]
return list(cmd[:-1]) + [tmp], output, tmp
+HW_ENV = "FFMPEG_SKILL_HW"
+# set only by render.py/batch.py --hw/--no-hw for the stages they run: "the outer command chose
+# explicitly", so a child resolves $FFMPEG_SKILL_HW as a flag (export.py included), not a default
+HW_FORCED_ENV = "_FFMPEG_SKILL_HW_EXPLICIT"
+
+
+def add_hw_orchestrator_args(ap: "argparse.ArgumentParser") -> None:
+ """--hw / --no-hw on a tool that runs other tools (render.py, batch.py) and encodes nothing itself."""
+ _add_hw_flags(ap, "run every stage's encode on Apple VideoToolbox, the final delivery export included (Apple Silicon; "
+ "falls back to the CPU, reported, where it cannot apply). Without it, $FFMPEG_SKILL_HW=1 still "
+ "puts the intermediate stages on the GPU and leaves the delivery export on the CPU",
+ "every stage on the CPU encoders, whatever $FFMPEG_SKILL_HW says")
+ ap.set_defaults(_hw_orchestrator=True)
+
+
+def env_hw() -> bool:
+ """$FFMPEG_SKILL_HW asks for VideoToolbox by default (1/true/yes/on); anything else is off."""
+ return os.environ.get(HW_ENV, "").strip().lower() in ("1", "true", "yes", "on")
+
+
+def hw_platform_reason() -> Optional[str]:
+ """None when this machine can take a VideoToolbox constant-quality encode, else why not.
+ `-q:v` on VideoToolbox is Apple-Silicon-only in FFmpeg: an Intel Mac lists the encoders but
+ rejects the quality setting, so the machine check is macOS AND Apple Silicon hardware."""
+ if platform.system() != "Darwin":
+ return "VideoToolbox is macOS-only"
+ if not apple_silicon():
+ return "VideoToolbox constant-quality encoding needs Apple Silicon"
+ return None
+
+
+_APPLE_SILICON: Optional[bool] = None
+
+
+def apple_silicon() -> bool:
+ """Whether this Mac's hardware is Apple Silicon -- not this interpreter's architecture: an
+ x86_64 Python under Rosetta reports platform.machine() "x86_64" on an M-series Mac, where
+ `sysctl -n hw.optional.arm64` still reads 1 (an Intel Mac has no such key). What decides -q:v
+ is the ffmpeg binary's own build; an x86_64 ffmpeg there refuses it, and run() re-encodes that
+ job on the CPU with ffmpeg's line in the note. Read once."""
+ global _APPLE_SILICON
+ if _APPLE_SILICON is None:
+ _APPLE_SILICON = platform.machine() == "arm64"
+ if not _APPLE_SILICON and platform.system() == "Darwin":
+ try:
+ out = subprocess.run([shutil.which("sysctl") or "/usr/sbin/sysctl", "-n", "hw.optional.arm64"],
+ stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, encoding="utf-8",
+ errors="replace", timeout=10).stdout
+ _APPLE_SILICON = out.strip() == "1"
+ except (OSError, subprocess.SubprocessError):
+ _APPLE_SILICON = False
+ return _APPLE_SILICON
+
+
+# What FFmpeg prints when a VideoToolbox encoder cannot take a job, read from the sources of 5.1,
+# 6.0, 6.1, 7.0, 7.1, 8.0 and master (fftools/ffmpeg*.c, libavcodec/videotoolboxenc.c):
+# - the CLI's encoder-open failure, "Error while opening encoder", on every version. From 6.0 the
+# line carries the output stream's log context, which names the encoder ("[vost#0:0/
+# h264_videotoolbox @ 0x...]", from 8.0 followed by "[enc:h264_videotoolbox @ 0x...]"). 5.1
+# prints "Error initializing output stream 0:0 -- Error while opening encoder for output stream
+# #0:0 ...", which names no encoder, so there it counts only beside an error line the
+# VideoToolbox encoder logged in its own context;
+# - the encoder's own session failures, logged in its codec context ("[h264_videotoolbox @
+# 0x...]"): "cannot create compression session", "cannot prepare encoder", "cannot encode
+# frame", "Error encoding frame". Each of them fails the encode. Matched without case and
+# without the "Error: " prefix, which master drops ("Cannot create compression session").
+# Deliberately not counted, so the job is not run twice and the GPU is not blamed:
+# - "Could not open encoder before EOF" (7.0+). It names the encoder, but it follows a filter
+# graph that failed before the first frame -- a missing font, a bad filter argument -- so the
+# encoder never got as far as opening;
+# - the encoder's ERROR-level lines that do not fail the encode ("Error setting profile/level
+# property ... Output will be encoded using a supported profile/level combination", the
+# frames_before/frames_after/entropy/realtime/pixel-aspect properties), which a job that fails
+# for another reason (a full disk: "Error submitting a packet to the muxer") can still carry.
+_VT_ENCODER_RE = re.compile(r"(?:h264|hevc|prores)_videotoolbox")
+_VT_OWN_CONTEXT_RE = re.compile(r"\[(?:h264|hevc|prores)_videotoolbox @ [^\]]*\]")
+_VT_SESSION_RE = re.compile(r"create compression session|prepare encoder|encode frame|encoding frame", re.I)
+# the encoder's own ERROR-level lines that do not fail the encode (listed above)
+_VT_NONFATAL_RE = re.compile(r"profile/level|entropy|frames_before|frames_after|realtime|pixel aspect|aspect ratio", re.I)
+_STREAM_CONTEXT_RE = re.compile(r"\[[a-z]ost#\d+:\d+")
+_ENCODER_OPEN_FAILED = "Error while opening encoder"
+
+
+def videotoolbox_failure(stderr: str) -> Optional[str]:
+ """The stderr line that shows a VideoToolbox encoder refused the job (could not open, or its
+ compression session failed), with the log contexts' pointers dropped; None when the failure
+ is something else. Only such a failure is retried on the CPU (see the notes above)."""
+ lines = [ln.strip() for ln in (stderr or "").splitlines() if ln.strip()]
+ own = [ln for ln in lines if _VT_OWN_CONTEXT_RE.search(ln)]
+ # the lines that can explain a refusal: not the non-fatal property errors, which a job that
+ # failed elsewhere (5.1: an audio encoder that could not open) still carries
+ fatal = [ln for ln in own if not _VT_NONFATAL_RE.search(ln)]
+ found = next((ln for ln in own if _VT_SESSION_RE.search(ln)), None)
+ if found is None:
+ for ln in lines:
+ if _ENCODER_OPEN_FAILED not in ln:
+ continue
+ if _VT_ENCODER_RE.search(ln): # 6.0+: the line names the encoder that failed to open
+ found = fatal[-1] if fatal else ln
+ break
+ if fatal and not _STREAM_CONTEXT_RE.search(ln): # 5.1: no encoder named; the encoder's own error line is
+ found = fatal[-1]
+ break
+ return re.sub(r" @ 0x[0-9a-fA-F]+", "", found)[:200] if found else None
+
+
+def _hw_fallback(cmd: List[str], ctx: "Context") -> Optional[List[str]]:
+ """`cmd` with every VideoToolbox encoder line this process built swapped back to the CPU line
+ it replaced, or None when it has none -- for run() when the GPU refused an encode."""
+ out, changed = list(cmd), False
+ for vt, cpu in ctx.hw_swaps:
+ n = len(vt)
+ for i in range(len(out) - n + 1):
+ if out[i:i + n] == vt:
+ out[i:i + n] = cpu
+ changed = True
+ break
+ return out if changed else None
+
+
def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True, ctx: "Optional[Context]" = None) -> subprocess.CompletedProcess:
"""Run a command, echoing it to stderr unless quiet. Exits on failure when check=True.
@@ -648,19 +853,50 @@ def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True, ctx: "Op
flush_drawtext_textfiles(cmd)
with _OutputLock(cmd[-1] if is_ffmpeg else "-"):
exec_cmd, final, tmp = _stage_existing_output(cmd) if is_ffmpeg else (list(cmd), None, None)
- proc = _execute(exec_cmd)
- if proc.returncode != 0 and is_ffmpeg:
- retry = _odd_dimension_retry(exec_cmd, proc.stderr or "")
+ def even_retry(proc: subprocess.CompletedProcess, ran: List[str]) -> "Tuple[subprocess.CompletedProcess, List[str]]":
+ """An attempt that failed on odd dimensions, run again with an even scale: after the
+ first attempt, and after a GPU->CPU retry (VideoToolbox may refuse an odd-sized job
+ for another reason before x264 ever reports the size)."""
+ if proc.returncode == 0 or not is_ffmpeg:
+ return proc, ran
+ retry = _odd_dimension_retry(ran, proc.stderr or "")
if retry is not None:
info("source has odd dimensions; scaling to even before encoding (yuv420p needs it)")
ctx.commands[-1] = _cmdline(retry[:-1] + [cmd[-1]])
- proc = _execute(retry)
- elif "not divisible by 2" in (proc.stderr or ""):
+ return _execute(retry), retry
+ if "not divisible by 2" in (proc.stderr or ""):
die("the source has odd dimensions (width or height not divisible by 2) and this tool's filter graph "
"cannot pad them itself; make them even first, e.g. fit.py --width/--height, then retry",
kind="input")
+ return proc, ran
+
+ # `tried` is the last command run: the CPU fallback swaps its encoder line in
+ proc, tried = even_retry(_execute(exec_cmd), exec_cmd)
+ retried = False
+ if proc.returncode != 0 and is_ffmpeg and ctx.hw_swaps:
+ cpu_cmd = _hw_fallback(tried, ctx)
+ if cpu_cmd is not None:
+ # Only a failure VideoToolbox itself reported is retried on the CPU: a bad filter
+ # graph, a missing font or a full disk would fail the same way twice and be
+ # reported as the GPU's fault (videotoolbox_failure() has the strings and why).
+ # A failure that is not one stays a failure of this command: with check=True,
+ # _fail() names it; with check=False the caller's own retry decides (a success
+ # after that must not carry a note about it).
+ why = videotoolbox_failure(proc.stderr or "")
+ if why is not None:
+ reason = f"VideoToolbox refused the encode: {why}"
+ info(reason + "; retrying on the CPU encoder")
+ ctx.hw_notes.append(reason + "; re-encoded on the CPU")
+ ctx.hw_fallback = True
+ ctx.commands[-1] = _cmdline(cpu_cmd[:-1] + [cmd[-1]])
+ if not quiet:
+ # echoed like the first attempt, so a caller that reads the "$ " lines
+ # (render.py's stages) records the command that really ran
+ info("$ " + ctx.commands[-1], ctx=ctx)
+ proc, tried = even_retry(_execute(cpu_cmd), cpu_cmd)
+ retried = True
if proc.returncode != 0 and check:
- _fail(exec_cmd, proc.returncode, proc.stderr or "")
+ _fail(tried, proc.returncode, proc.stderr or "", ctx=ctx, retried=retried)
if final and tmp:
if proc.returncode == 0:
try:
@@ -1005,13 +1241,22 @@ def flush_drawtext_textfiles(cmd: "Sequence[str]") -> "List[str]":
def ffmpeg_encoders() -> set:
- """Names from `ffmpeg -encoders`, read once; empty when ffmpeg is missing. Used only to pick
- an AV1 encoder and to refuse --codec av1 / prores before ffmpeg would."""
+ """Names from `ffmpeg -encoders` -- the build that will run the encode -- read once; empty when
+ it cannot be read. Used to pick an AV1 encoder, to refuse --codec av1 / prores before ffmpeg
+ would, and to see whether --hw's VideoToolbox encoders are built in.
+
+ Under --dry-run only, the list comes from `ffprobe -encoders` instead: a dry run promises
+ never to run ffmpeg (docs/contract.md: ffmpeg_execution "none") while ffprobe always may, as in
+ ffmpeg_version(). That is the one case for ffprobe. On a mixed install (a Homebrew ffprobe next
+ to a static ffmpeg) its list is another build's, so a real run never uses it: the AV1/ProRes
+ refusals and the VideoToolbox check must describe the binary that encodes, and a plan that
+ read ffprobe's list is re-decided against ffmpeg's when it runs."""
global _ENCODERS
if _ENCODERS is None:
_ENCODERS = set()
+ tool = "ffprobe" if STATE.dry_run else "ffmpeg"
try:
- out = subprocess.run([shutil.which("ffmpeg") or "ffmpeg", "-hide_banner", "-encoders"], stdout=subprocess.PIPE,
+ out = subprocess.run([shutil.which(tool) or tool, "-hide_banner", "-encoders"], stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True, encoding="utf-8", errors="replace", timeout=PROBE_TIMEOUT).stdout
_ENCODERS = set(re.findall(r"^\s*[VAS][.\w]{5}\s+(\S+)", out, re.M))
except (OSError, subprocess.SubprocessError):
@@ -1043,4 +1288,4 @@ def read_text_or_die(path: str, flag: str) -> str:
# emit needs runner's Context/STATE/ERROR_CODE, so the two form a cycle that has to be cut
# somewhere: by the time this line runs every name emit reads from runner above is defined, and
# every name runner reads from emit is only ever read inside a function body, never at import.
-from _common.emit import _plan_at_exit, die, info # noqa: E402
+from _common.emit import _plan_at_exit, die, hw_report, info # noqa: E402
diff --git a/scripts/_contract.py b/scripts/_contract.py
index ae7c867..47edb42 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -295,7 +295,8 @@ REENCODE_META: Dict[str, Dict[str, str]] = {
}
IDEMPOTENCY = {
- "bit_exact": "same inputs and flags give byte-identical output",
+ "bit_exact": "same inputs and flags give byte-identical output (with the CPU encoders: under --hw / $FFMPEG_SKILL_HW a "
+ "VideoToolbox encode is content_equivalent at best)",
"content_equivalent": "same inputs and flags give the same media content; bytes may differ between encoder builds",
"cached": "re-runs skip inputs whose content hash and recipe are unchanged",
"environment_dependent": "output includes timings or machine state and differs between runs",
@@ -348,7 +349,7 @@ def input_schema(parser: argparse.ArgumentParser) -> Dict[str, Any]:
props: Dict[str, Any] = {}
required: List[str] = []
positional: List[str] = []
- common = {"dry_run", "json", "progress", "fast", "timeout", "overwrite", "plan", "codec", "quality"}
+ common = {"dry_run", "json", "progress", "fast", "timeout", "overwrite", "plan", "codec", "quality", "hw", "no_hw"}
for action in parser._actions:
if isinstance(action, argparse._HelpAction):
continue
@@ -371,6 +372,13 @@ def input_schema(parser: argparse.ArgumentParser) -> Dict[str, Any]:
required.append(action.dest)
if action.dest in common:
prop["common"] = True
+ if action.dest in props:
+ # one property per dest: a second action (a store_false --no-x beside a store_true --x)
+ # would replace the first, and the contract and MCP would publish only the second flag
+ # under the first one's name -- an MCP `x: true` would then send --no-x
+ first = props[action.dest]["cli"]
+ raise RuntimeError(f"{parser.prog}: {'/'.join(action.option_strings) or action.dest} shares dest {action.dest!r} "
+ f"with {'/'.join(first) if isinstance(first, list) else 'a positional'}; give each flag its own dest")
props[action.dest] = prop
groups = [g for g in getattr(parser, "_mutually_exclusive_groups", []) if g._group_actions]
schema: Dict[str, Any] = {"type": "object", "properties": props, "required": required, "positional": positional, "additionalProperties": False}
@@ -380,8 +388,9 @@ def input_schema(parser: argparse.ArgumentParser) -> Dict[str, Any]:
return schema
-def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
- """What the tool prints on stdout with --json (keys observed in the implementation)."""
+def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str, Any]:
+ """What the tool prints on stdout with --json (keys observed in the implementation). `hw`: the
+ tool takes --hw, so its `notes` can also carry a VideoToolbox encode's quality line."""
if name == "probe":
return {"type": "object", "description": "one probe document, or an array of them for several inputs",
"properties": {"file": {"type": "string"}, "format": {"type": "string"}, "duration": {"type": "number"}, "size_bytes": {"type": "integer"},
@@ -389,6 +398,8 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
base = {"status": {"enum": ["completed"]}, "output": {"type": ["string", "null"], "description": "path written, or null"},
"dry_run": {"type": "boolean"}, "commands": {"type": "array", "items": {"type": "string"}, "description": "every ffmpeg command line planned or run"},
"probe": {"type": "object", "description": "probe of the output when a file was written"},
+ "encoder": {"type": "string", "description": "the video encoder the last ffmpeg command used (after any GPU->CPU fallback; under --dry-run the planned one; render.py and waveform.py: their stages' commands included), `copy` for a stream copy; absent when no command encoded video"},
+ "hw": {"type": "object", "description": "present when VideoToolbox was asked for (--hw, $FFMPEG_SKILL_HW, a render.py stage, a batch.py item) and on export.py when $FFMPEG_SKILL_HW=1 is set but its delivery preset does not take it: {requested: whether this run's encodes were asked onto VideoToolbox (false for that export), source: flag|env, used: true|false whether the encoder that ran last is VideoToolbox, null under --dry-run (nothing ran) and when this process ran no encode itself (batch.py: its stages are child processes; its top-level hw gathers the items' requested/source/fallback/notes and each results row carries its item's encoder and hw), fallback: a job VideoToolbox refused (encoder open or compression session failure; nothing else is retried) was re-encoded on the CPU, notes: why an encode stayed on or fell back to the CPU, and, when $FFMPEG_SKILL_HW chose the GPU and it ran, what that costs and that --no-hw opts out}. A VideoToolbox encode also adds a top-level notes line: its quality is not CRF-equivalent. A kind: ffmpeg failure under --hw / $FFMPEG_SKILL_HW carries encoder and hw too"},
"plan": {"type": "string", "description": "with --plan FILE: the plan document written (the run itself is a dry run)"},
"verified": {"type": "boolean", "description": "true only when the artifact was written, probed, and every self-check the tool ran (verification) met its target; false under --dry-run"},
"verification": {"type": "array", "items": {"type": "object", "properties": {"step": {"type": "string"}, "ok": {"type": "boolean"}}}, "description": "what the tool itself verified: probe, plus loudness (loudness.py, export platform presets) or check (render)"}}
@@ -481,6 +492,11 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
"audio": {"type": "object", "description": "what the mix was built from: voice (null | light | medium | strong), stereo_widen, effects/effects_volume, and with --music the music_volume plus duck (null when --duck was not given, else the threshold in dB and linear, ratio, attack_ms, release_ms, amount_db actually used)"}}
props = dict(base)
props.update(extra)
+ notes = props.get("notes")
+ if hw and isinstance(notes, dict) and notes.get("description"):
+ # a tool-specific condition is not the only one: emit() adds the GPU line to any tool
+ props["notes"] = dict(notes, description=notes["description"] + "; also, on a VideoToolbox encode (--hw / "
+ "$FFMPEG_SKILL_HW), a line that its quality is not CRF-equivalent (see hw)")
required = ["status", "output", "dry_run", "commands"]
return {"type": "object", "properties": props, "required": required, "additionalProperties": True}
@@ -656,6 +672,14 @@ def _whisper_available() -> bool:
return importlib.util.find_spec("faster_whisper") is not None or importlib.util.find_spec("whisper") is not None
+def _hw_default() -> Dict[str, Any]:
+ """--hw's machine facts, normalised (never the raw environment value): whether VideoToolbox
+ constant-quality encoding can run here (the tools' own check: Apple Silicon hardware, also
+ under a Rosetta Python), and whether $FFMPEG_SKILL_HW makes it the default."""
+ from _common.runner import env_hw, hw_platform_reason
+ return {"platform_ok": hw_platform_reason() is None, "default_on": env_hw()}
+
+
def _default_font() -> str:
from _common import BRAND_DEFAULTS
return str(BRAND_DEFAULTS["font"])
@@ -872,6 +896,7 @@ def doctor() -> Dict[str, Any]:
"ok": not missing_required and not unknown_required,
"tools": _tool_usability(state),
"gpu_encoders": _gpu_encoders(listings["encoders"]),
+ "hw": _hw_default(),
"fonts": _fonts_capability(probe=True),
}
@@ -1101,7 +1126,7 @@ def tool_spec(name: str, version: str) -> Dict[str, Any]:
"inputs": list(meta["inputs"]),
"outputs": list(meta["outputs"]),
"input_schema": schema,
- "output_schema": output_schema(name, meta),
+ "output_schema": output_schema(name, meta, hw="hw" in schema["properties"]),
"supports_dry_run": supports_dry_run,
"dry_run": {"supported": supports_dry_run,
"ffmpeg_execution": "full" if not supports_dry_run else "analysis_only" if name in DRY_RUN_ANALYSIS else "none",
diff --git a/scripts/batch.py b/scripts/batch.py
index 45f5371..3a254c7 100755
--- a/scripts/batch.py
+++ b/scripts/batch.py
@@ -33,6 +33,8 @@ from pathlib import Path
from typing import Any, Dict, List, Optional
from _common import STATE, add_common, apply_common, child_args, die, emit, info, run_tool, read_text_or_die, MEDIA_EXT as _MEDIA_EXT
+from _common.runner import HW_FORCED_ENV, add_hw_orchestrator_args, env_hw
+from _common.emit import absorb_stage_hw, steps_encoder_report
HERE = Path(__file__).resolve().parent
MEDIA_EXT = {e for e in _MEDIA_EXT if e not in (".png", ".jpg", ".jpeg", ".webp")} # one list (_common); a batch walks media, not stills
@@ -73,6 +75,18 @@ def recipe_key(recipe: Dict[str, Any]) -> str:
return hashlib.sha1((json.dumps(recipe, sort_keys=True) + "\0" + project_content).encode()).hexdigest()[:12]
+def gpu_key() -> str:
+ """The GPU setting the steps resolve, as a suffix for the item cache key: a VideoToolbox file is
+ not the CPU's bytes, so a --hw output is never served to a --no-hw run or the other way round
+ (render.py's cache_key folds the same two variables in). Read after apply_common(), which has
+ exported --hw/--no-hw to the steps. '' when every step encodes on the CPU -- no flag and no
+ $FFMPEG_SKILL_HW, or --no-hw -- so the keys of a cache written before --hw existed still hit."""
+ if not env_hw():
+ return ""
+ # the variable alone leaves export.py's delivery presets on the CPU; --hw puts them on the GPU
+ return ":hw" if os.environ.get(HW_FORCED_ENV) == "1" else ":hw-env"
+
+
JOBS_CAP = 8 # beyond this, concurrent encodes contend for the same cores and memory
_LOG = threading.local()
@@ -157,6 +171,7 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
pj = work / f"{src.stem}_project.json"
pj.write_text(json.dumps(proj, indent=2), encoding="utf-8")
ok, _doc = run_step(["render.py", str(pj)], budget())
+ step_docs = [("render", _doc)]
cut_reencoded: List[bool] = []
else:
steps = recipe.get("steps") or []
@@ -165,11 +180,13 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
cur = str(src)
ok = True
cut_reencoded = []
+ step_docs = []
for i, step in enumerate(steps):
last = i == len(steps) - 1
out = str(final) if last else str(work / f"{src.stem}_step{i}.{'mp4' if src.suffix.lower() not in ('.wav', '.mp3', '.m4a', '.flac') else src.suffix.lstrip('.')}")
argv = [str(a).replace("{in}", cur).replace("{out}", out) for a in step]
step_ok, doc = run_step(argv, budget())
+ step_docs.append((Path(str(argv[0])).stem if argv else "step", doc))
# only cut.py's own doc carries `reencoded` -- true if ANY range this call cut needed
# the tolerance-triggered hybrid re-encode fallback (cut.py ORs its per-segment results
# into one top-level field; it doesn't report which range, so this is per cut.py call,
@@ -183,6 +200,9 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
result: Dict[str, Any] = {"file": str(src), "output": str(final), "ok": ok, "seconds": round(time.time() - t0, 1)}
if cut_reencoded:
result["cut_reencoded"] = cut_reencoded
+ # the encoder each item really ran and its GPU facts (--hw / FFMPEG_SKILL_HW: on VideoToolbox,
+ # fell back and why): this process encodes nothing itself, so its top-level hw.used is null
+ result.update(steps_encoder_report(step_docs))
return result
@@ -199,6 +219,7 @@ def main() -> int:
"Default 1, which is 1.16's behaviour exactly.")
ap.add_argument("--work", help="work directory for intermediates (default: <output_dir>/.work)")
add_common(ap)
+ add_hw_orchestrator_args(ap)
args = ap.parse_args()
apply_common(args)
@@ -240,7 +261,7 @@ def main() -> int:
cache = json.loads(cache_path.read_text(encoding="utf-8"))
except ValueError:
cache = {}
- rkey = recipe_key(recipe)
+ rkey = recipe_key(recipe) + gpu_key()
glob = recipe.get("glob") or "*"
# --jobs: every item is itself an ffmpeg that already threads across cores, so beyond a few
@@ -303,6 +324,10 @@ def main() -> int:
# write is already atomic: two finishers could otherwise serialise from two different
# snapshots and lose an entry.
with cache_lock:
+ # this run just rewrote r["output"]: an entry under another key (another recipe,
+ # the other GPU setting) that names the same file now describes bytes that are gone
+ for stale in [k for k, v in cache.items() if k != key and isinstance(v, dict) and v.get("output") == r["output"]]:
+ del cache[stale]
cache[key] = r
# write_text isn't atomic -- a process killed mid-write (or a --watch loop racing
# a concurrent manual run) could leave a truncated file that json.loads() above
@@ -507,6 +532,10 @@ def main() -> int:
results = one_pass()
except KeyboardInterrupt:
info("watch stopped")
+ # the items' GPU facts (asked for, fell back and why) reach the top-level `hw`, as render.py's
+ # stages' do; `used` stays null there, because this process encoded nothing itself
+ for r in results:
+ absorb_stage_hw(Path(r["file"]).name, r)
done = sum(1 for r in results if r["ok"])
info(f"{done}/{len(results)} processed, {sum(1 for r in results if r.get('cached'))} from cache")
if not args.json:
diff --git a/scripts/export.py b/scripts/export.py
index 2197fc5..e8f996a 100755
--- a/scripts/export.py
+++ b/scripts/export.py
@@ -37,6 +37,7 @@ from pathlib import Path
from typing import Dict, List
from _common import STATE, add_common, apply_common, bt709_tag_args, child_args, emit, cfr_args, default_output, die, encoder_args, ffmpeg_base, info, probe, run, run_tool, validate_color, pad_filters, add_pad_fill_args, fmt_secs
+from _common.decision import hw_preset_video, restate_last_swap
from check import SPECS as PLATFORMS, measure_loudness
from _platforms import ALIASES as _ALIASES, PLATFORMS as PLATFORM_TABLE, resolve as resolve_platform
# Frame and duration limit come from the one platform table (scripts/_platforms.py) rather than
@@ -190,20 +191,28 @@ def main() -> int:
"veryfast" if STATE.fast else "medium", meta)
# encoder_args() already applied --fast (its own preset scale per encoder: SVT-AV1 counts
# 1..12, not x264's names) and appends +faststart, which this tool adds again for mp4
+ as_built = list(video)
while "-movflags" in video:
i = video.index("-movflags")
del video[i:i + 2]
+ restate_last_swap(as_built, video)
if args.crf is not None and "-crf" in video:
video[video.index("-crf") + 1] = str(args.crf)
if STATE.fast and "-preset" in video and not p.get("codec"):
video[video.index("-preset") + 1] = "veryfast"
+ # a fixed preset's line is tagged BT.709 (prores, an editing master, keeps the source's tags)
+ tag_bt709 = args.preset not in ("prores",) and not p.get("codec")
+ if STATE.hw and not p.get("codec") and video:
+ # --hw (never $FFMPEG_SKILL_HW here) puts a fixed preset on VideoToolbox: the same codec,
+ # its CRF mapped to -q:v, the preset's frame rate kept; tags come with the VT line itself
+ video = hw_preset_video(video, meta, bt709=tag_bt709)
cmd += video
if args.preset != "copy":
# a stream copy can't be frame-rate-conformed or retagged without decoding it — that would
# no longer be a copy, and would silently mislabel colour the agent never actually looked at
if "-r" not in video:
cmd += cfr_args(meta)
- if args.preset not in ("prores",) and not p.get("codec"):
+ if tag_bt709 and not video[video.index("-c:v") + 1].endswith("_videotoolbox"):
cmd += bt709_tag_args(video[video.index("-c:v") + 1])
if out_ext == "mp4":
cmd += ["-movflags", "+faststart"]
diff --git a/scripts/render.py b/scripts/render.py
index e87525a..aecbad3 100755
--- a/scripts/render.py
+++ b/scripts/render.py
@@ -65,6 +65,8 @@ from typing import Any, Dict, List, Optional, Sequence
from export import PRESETS, PLATFORM_OF
from _platforms import PLATFORMS, caption_defaults, resolve as resolve_platform
from _common import STATE, add_common, aspect_ratio, brand_caption_style, load_brand, apply_common, child_args, die, emit, info, probe, require_tool, run, run_tool, place_output, refuse_output_is_input, _check_existing_output, _check_output_path, fingerprint, PLAN_VERSION, ffmpeg_version
+from _common.runner import HW_ENV, HW_FORCED_ENV, add_hw_orchestrator_args
+from _common.emit import absorb_stage_hw
import subprocess
from _contract import CONTRACT_VERSION
from batch import file_key
@@ -354,6 +356,7 @@ def render_pack(names: List[str], args) -> int:
if line.startswith("$ ") or line.startswith("[dry-run]"):
STATE.commands.append(line[2:] if line.startswith("$ ") else line)
STATE.commands.extend(str(c) for c in (doc.get("commands") or []))
+ absorb_stage_hw(name, doc)
out = doc.get("output") or dest_out
chk = doc.get("check") or {}
ok = proc.returncode == 0 and doc.get("status") == "completed"
@@ -541,16 +544,29 @@ def sh(script: str, *argv: Any, extra: List[str] = None, stage: str = None) -> s
# needs (a timeout inside audio.py is a timeout, not an "input" error of render.py).
err = doc.get("error") or {}
extra_fields = {"hint": err["hint"]} if err.get("hint") else {}
+ extra_fields.update(_stage_failure_hw(Path(script).stem, doc))
die(f"{script} failed: {err.get('message') or (proc.stderr.strip().splitlines() or ['?'])[-1][:300]}",
code=int(doc.get("exit_code") or 1), kind=err.get("kind") or "input", stage=script, **extra_fields)
_LAST_DOC.clear()
_LAST_DOC.update(doc if isinstance(doc, dict) else {})
+ absorb_stage_hw(Path(script).stem, doc) # a stage's GPU facts (ran on VideoToolbox, fell back and why) reach this result
out_path = str(doc.get("output") or "")
if key:
cache_store(stage, key, out_path or (dest or ""), time.time() - started)
return out_path
+def _stage_failure_hw(stage: str, doc: Any) -> Dict[str, Any]:
+ """A failed stage's GPU facts, for render's own failure document: the stage's `hw` joins the
+ earlier stages' (a fallback before the failure is not lost), and its `encoder` -- the command
+ that failed -- is forwarded. die() then adds `hw` for a kind: ffmpeg failure, as a tool's own
+ run() failure carries it."""
+ if not isinstance(doc, dict):
+ return {}
+ absorb_stage_hw(stage, doc)
+ return {"encoder": doc["encoder"]} if doc.get("encoder") else {}
+
+
# ------------------------------------------------------------------ the stage cache (1.17)
#
# Opt-in only: --cache DIR. There is no default directory -- a cache that appears on someone's
@@ -623,6 +639,10 @@ def cache_key(stage: str, script: str, argv: "Sequence[Any]", inputs: "Sequence[
"inputs": [{"hash": _content_hash(p)} for p in inputs],
"child": [a for a in child_args() if a != "--dry-run"],
"codec": STATE.codec, "ext": Path(dest).suffix if dest else None,
+ # what a stage's encoder resolves from: $FFMPEG_SKILL_HW (inherited) and whether the outer
+ # command made it explicit (--hw/--no-hw, which reaches export.py too). A GPU encode is not
+ # the CPU's bytes, so a --hw artifact is never served to a CPU run or the other way round.
+ "hw": [os.environ.get(HW_ENV, "").strip().lower() in ("1", "true", "yes", "on"), os.environ.get(HW_FORCED_ENV) == "1"],
"ffmpeg": CACHE.get("ffmpeg"), "skill": SKILL_VERSION, "contract": CONTRACT_VERSION,
}
return hashlib.sha1(json.dumps(payload, sort_keys=True, ensure_ascii=False).encode()).hexdigest()
@@ -724,7 +744,8 @@ def execute_plan(plan: Dict[str, Any], path: str) -> int:
if proc.returncode != 0 or doc.get("status") != "completed":
err = doc.get("error") or {}
die(f"{tool} failed while executing the plan: {err.get('message') or proc.stderr.strip()[-300:]}",
- kind=err.get("kind", "ffmpeg"), plan=path, tool=tool)
+ kind=err.get("kind", "ffmpeg"), plan=path, tool=tool, **_stage_failure_hw(tool, doc))
+ absorb_stage_hw(tool, doc) # the tool's GPU facts (fell back and why, ran on VideoToolbox) reach this result
output = doc.get("output") or plan.get("output")
check_result = None
exit_code = 0
@@ -863,6 +884,7 @@ def main() -> int:
"or 'all' its directory is where the pack is written")
tpl.add_argument("--write-project", metavar="FILE", help="write the filled project.json for editing and stop (no render)")
add_common(ap)
+ add_hw_orchestrator_args(ap)
args = ap.parse_args()
apply_common(args)
diff --git a/scripts/waveform.py b/scripts/waveform.py
index 31bf4a7..7b62011 100644
--- a/scripts/waveform.py
+++ b/scripts/waveform.py
@@ -38,6 +38,7 @@ from _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platform
from _common import (add_common, apply_common, aac_args, default_output, die, emit, ffmpeg_base,
info, load_brand, pad_filters, probe, run, STATE, validate_color, video_args, X264_PRESETS,
fmt_secs, child_args, run_tool, measured_level_dbfs)
+from _common.emit import absorb_stage_hw
WAVEFORM_MODES = ["point", "line", "p2p", "cline"]
@@ -307,20 +308,30 @@ def _child(script_name: str, argv: "list") -> None:
"""Run one of this skill's own tools as a second process, so the code path it owns (the ASS
generator, the drawtext template) stays the only one there is. The shared flags (--overwrite,
--timeout, --fast, --dry-run) reach it through child_args(), as render.py's stages get them,
- and a failure is re-raised with the child's own kind, exit code and hint."""
+ and a failure is re-raised with the child's own kind, exit code and hint.
+
+ The GPU choice reaches it too: an explicit --hw/--no-hw is passed on (the child would
+ otherwise resolve $FFMPEG_SKILL_HW for itself, and the last stage writes the deliverable),
+ and the child's commands and `hw` facts join this result, so `encoder`/`hw` describe the
+ file delivered rather than the intermediate render."""
here = os.path.dirname(os.path.abspath(__file__))
- cmd = [os.path.join(here, script_name)] + [str(a) for a in argv] + child_args() + ["--json"]
+ hw = (["--hw"] if STATE.hw else ["--no-hw"]) if STATE.hw_source == "flag" else []
+ cmd = [os.path.join(here, script_name)] + [str(a) for a in argv] + hw + child_args() + ["--json"]
info("-> " + " ".join(os.path.basename(c) if c.endswith(".py") else str(c) for c in cmd[:-1]))
proc = run_tool(cmd)
+ try:
+ doc = json.loads(proc.stdout.strip() or "{}")
+ except ValueError:
+ doc = {}
+ if not isinstance(doc, dict):
+ doc = {}
+ STATE.commands.extend(str(c) for c in (doc.get("commands") or []))
+ absorb_stage_hw(os.path.splitext(script_name)[0], doc)
if proc.returncode != 0:
- try:
- doc = json.loads(proc.stdout.strip() or "{}")
- except ValueError:
- doc = {}
- if not isinstance(doc, dict):
- doc = {}
err = doc.get("error") or {}
extra = {"hint": err["hint"]} if err.get("hint") else {}
+ if doc.get("encoder"):
+ extra["encoder"] = doc["encoder"]
die(f"{script_name} failed: {err.get('message') or (proc.stderr.strip().splitlines() or ['?'])[-1][:300]}",
code=int(doc.get("exit_code") or 1), kind=err.get("kind") or "input", stage=script_name, **extra)
diff --git a/tests/_fixtures.py b/tests/_fixtures.py
index 1de4ffc..337317b 100644
--- a/tests/_fixtures.py
+++ b/tests/_fixtures.py
@@ -20,9 +20,18 @@ ROOT = Path(__file__).resolve().parent.parent
SCRIPTS = ROOT / "scripts"
HERE = Path(__file__).resolve().parent
OUT = Path(os.environ.get("OUT", ROOT / "tests" / "out"))
+
sys.path.insert(0, str(SCRIPTS))
sys.path.insert(0, str(HERE))
from _common import escape_filter_path # noqa: E402
+from _common.runner import HW_ENV, HW_FORCED_ENV # noqa: E402
+
+# The host's own GPU default must not reach the suite (every tool subprocess inherits os.environ):
+# a machine with FFMPEG_SKILL_HW=1 would put every encode on VideoToolbox. Tests that exercise it
+# opt in with an explicit env. The names come from the runner, so this scrub and the one in
+# test_contract.py cannot drift from what the tools read.
+for _k in (HW_ENV, HW_FORCED_ENV):
+ os.environ.pop(_k, None)
TONES = ("0.6*sin(2*PI*440*t)*gt(sin(2*PI*0.37*t)\\,0.3)+0.4*sin(2*PI*880*t)*gt(sin(2*PI*0.53*t+1)\\,0.6)"
"+0.3*sin(2*PI*220*t)*gt(sin(2*PI*0.21*t+2)\\,0.7)")
diff --git a/tests/bench_vt.py b/tests/bench_vt.py
new file mode 100644
index 0000000..b130c08
--- /dev/null
+++ b/tests/bench_vt.py
@@ -0,0 +1,220 @@
+#!/usr/bin/env python3
+"""Quality benchmark for --hw: which VideoToolbox -q:v matches the CPU encode's SSIM at a CRF.
+
+For each clip and codec, encodes the CPU line the skill writes (x264 / x265 medium, from
+decision._cpu_encoder_args, so HDR gets x265 Main10 at CRF+2) at each --crf, then sweeps
+VideoToolbox -q:v over a grid with the same line --hw writes (decision._vt_args). Each encode's
+SSIM is measured against the source with frames paired by index (settb=1/1000,setpts=N): iPhone
+timestamps jitter and a re-encode lands on a clean grid, so the ssim filter's timestamp pairing
+compares neighbours. N/FRAME_RATE/TB is not enough: in a time base that does not divide the frame
+duration (1/1000 at 30 fps, 1/600 at 29.97) it rounds, and two frames in three pair with a
+neighbour (measured: 27 dB for an encode that is 47 dB). The q whose SSIM matches the CPU encode is interpolated from the sweep, and
+the table reports it beside vt_quality()'s current answer, with bytes for both encoders.
+
+VideoToolbox output is not bit-deterministic, so an SSIM can move in the fourth decimal between
+runs; compare the fitted q, not a hash. SSIM is the
+original fit's metric, kept for comparability; x264/x265 psy tuning lowers their SSIM at equal
+visual quality, so a matched q leans low and the byte ratio reads kind to VideoToolbox. HDR rows
+match x265 at CRF+2 (the CPU line HDR gets), so fit them apart from SDR hevc. Timings include
+decode and process start: rough ratios, one run each. The VFR conform the scripts add is left out: both encoders
+read the same frames, which keeps the index pairing exact.
+
+Clips: three synthetic 1080p30 stand-ins (CG test pattern, a detailed fractal zoom, grainy
+gradients) plus any --clip given; each is cut to --seconds.
+
+Usage:
+ python3 tests/bench_vt.py --clip ~/Footage/clip.mov --json vt.json
+ python3 tests/bench_vt.py --no-synthetic --clip a.mov --crf 23 --hwdec
+"""
+import argparse
+import json
+import math
+import os
+import re
+import subprocess
+import sys
+import tempfile
+import time
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / "scripts"))
+from _common import decision, probe # noqa: E402
+
+SYNTHETIC = {
+ "cg-testsrc2": "testsrc2=size=1920x1080:rate=30",
+ "fractal-zoom": "mandelbrot=size=1920x1080:rate=30:maxiter=200",
+ "grain-gradient": "gradients=size=1920x1080:rate=30:speed=0.02,noise=alls=10:allf=t+u",
+}
+DEFAULT_GRID = list(range(4, 101, 4))
+
+
+def sh(cmd):
+ return subprocess.run([str(c) for c in cmd], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+
+
+def encode(src, seconds, vargs, out, vf=None, pre=()):
+ cmd = ["ffmpeg", "-v", "error", "-y", *pre, "-t", f"{seconds}", "-i", src, "-map", "0:v:0", "-an"]
+ cmd += (["-vf", vf] if vf else []) + list(vargs) + [out]
+ t0 = time.monotonic()
+ r = sh(cmd)
+ took = time.monotonic() - t0
+ if r.returncode != 0:
+ raise RuntimeError(f"encode failed: {' '.join(map(str, cmd))}\n{r.stderr[-800:]}")
+ return os.path.getsize(out), took
+
+
+def frames(path, seconds=None):
+ """Video frames ffmpeg decodes from `path` (its first `seconds`, when given)."""
+ r = sh(["ffmpeg", "-v", "info", *(["-t", f"{seconds}"] if seconds else []), "-i", path, "-map", "0:v:0", "-f", "null", "-"])
+ m = re.findall(r"frame=\s*(\d+)", r.stderr)
+ if not m:
+ raise RuntimeError(f"could not count frames of {path}:\n{r.stderr[-400:]}")
+ return int(m[-1])
+
+
+def ssim(ref, dist, seconds, fmt, ref_frames):
+ """All-plane SSIM of `dist` against the first `seconds` of `ref`, frames paired by index. The
+ ssim filter repeats the shorter input's last frame, so a dropped frame would lower the score
+ with no error (measured: 0.893 for a 0.99 encode 30 frames short); a count mismatch fails."""
+ got = frames(dist)
+ if got != ref_frames:
+ raise RuntimeError(f"{dist} has {got} frames, the reference {ref_frames}: SSIM would pair the wrong frames")
+ graph = f"[0:v]settb=1/1000,setpts=N,format={fmt}[a];[1:v]settb=1/1000,setpts=N,format={fmt}[b];[a][b]ssim=shortest=1"
+ r = sh(["ffmpeg", "-v", "info", "-t", f"{seconds}", "-i", ref, "-i", dist, "-lavfi", graph, "-f", "null", "-"])
+ m = re.search(r"SSIM .*All:([0-9.]+)", r.stderr)
+ if not m:
+ raise RuntimeError(f"no SSIM for {dist}:\n{r.stderr[-800:]}")
+ return float(m.group(1))
+
+
+def db(s):
+ return -10 * math.log10(max(1e-9, 1 - s))
+
+
+def matched_q(curve, target):
+ """-q:v where the sweep's SSIM first crosses `target`, interpolated linearly in SSIM dB (near
+ 1.0 raw SSIM barely moves between grid points), with the VideoToolbox bytes there; (None, None)
+ off the grid. VideoToolbox is not deterministic, so a curve can dip: bench_clip flags it."""
+ pts = sorted(curve, key=lambda p: p["q"])
+ t = db(target)
+ for lo, hi in zip(pts, pts[1:]):
+ a, b = db(lo["ssim"]), db(hi["ssim"])
+ if a <= t <= b and b > a:
+ f = (t - a) / (b - a)
+ return round(lo["q"] + f * (hi["q"] - lo["q"]), 1), round(lo["bytes"] + f * (hi["bytes"] - lo["bytes"]))
+ return None, None
+
+
+def bench_clip(name, src, meta, args, tmp):
+ v = meta["video"]
+ hdr = bool(v.get("bt2020_or_hdr"))
+ fmt = "yuv420p10le" if hdr else "yuv420p"
+ seconds = min(args.seconds, float(meta.get("duration") or args.seconds))
+ source_bytes = (meta.get("bitrate") or 0) * seconds / 8
+ codecs = ["hevc"] if hdr else args.codec
+ ref_frames = frames(src, seconds)
+ out = {"clip": name, "width": v.get("width"), "height": v.get("height"), "fps": v.get("fps"),
+ "hdr": hdr, "transfer": v.get("color_transfer"), "seconds": seconds,
+ "source_bytes": round(source_bytes), "codecs": {}}
+ for codec in codecs:
+ vt_line = decision._vt_args(codec, 18, meta, True)
+ if vt_line is None:
+ raise RuntimeError(f"no VideoToolbox {codec} here: {decision.STATE.hw_notes}")
+ qi = vt_line.index("-q:v") + 1
+ curve = []
+ for q in sorted(set(args.grid) | {decision.vt_quality(codec, crf, hdr) for crf in args.crf}):
+ line = list(vt_line)
+ line[qi] = str(q)
+ path = os.path.join(tmp, f"vt_{q}.mp4")
+ size, took = encode(src, seconds, line, path)
+ curve.append({"q": q, "bytes": size, "ssim": ssim(src, path, seconds, fmt, ref_frames), "time": round(took, 2)})
+ os.remove(path)
+ print(f" {name} {codec} vt q{q}: {size / 1e6:.2f} MB ssim {curve[-1]['ssim']:.5f}", file=sys.stderr)
+ rows = []
+ for crf in args.crf:
+ cpu_line = decision._cpu_encoder_args(codec, crf, "medium", meta, True)
+ path = os.path.join(tmp, f"cpu_{crf}.mp4")
+ size, took = encode(src, seconds, cpu_line, path)
+ target = ssim(src, path, seconds, fmt, ref_frames)
+ os.remove(path)
+ q, vt_bytes = matched_q(curve, target)
+ now = decision.vt_quality(codec, crf, hdr)
+ now_pt = next((p for p in curve if p["q"] == now), None)
+ rows.append({"crf": crf, "cpu_crf": int(cpu_line[cpu_line.index("-crf") + 1]), "cpu_bytes": size,
+ "cpu_ssim": target, "cpu_time": round(took, 2), "matched_q": q, "matched_vt_bytes": vt_bytes,
+ "vt_quality_now": now, "now_ssim": now_pt and now_pt["ssim"], "now_bytes": now_pt and now_pt["bytes"]})
+ print(f" {name} {codec} crf{crf}: cpu {size / 1e6:.2f} MB ssim {target:.5f} -> q {q} (now {now})", file=sys.stderr)
+ entry = {"curve": curve, "rows": rows}
+ if args.hwdec:
+ entry["hwdec"] = hwdec_timing(src, seconds, vt_line, tmp)
+ entry["monotonic"] = all(a["ssim"] <= b["ssim"] for a, b in zip(curve, curve[1:]))
+ out["codecs"][codec] = entry
+ return out
+
+
+def hwdec_timing(src, seconds, vt_line, tmp):
+ """VideoToolbox encode through a CPU scale, decoding on the CPU and then on the GPU."""
+ path = os.path.join(tmp, "hwdec.mp4")
+ res = {}
+ for label, pre in (("cpu_decode", ()), ("vt_decode", ("-hwaccel", "videotoolbox"))):
+ _, took = encode(src, seconds, vt_line, path, vf="scale=1280:-2", pre=pre)
+ res[label] = round(took, 2)
+ os.remove(path)
+ return res
+
+
+def table(results):
+ lines = ["| Clip | Codec | CRF | CPU MB | CPU SSIM | matched q | VT MB at match | now q | VT MB now | SSIM now |",
+ "|---|---|---|---|---|---|---|---|---|---|"]
+ for r in results:
+ for codec, e in r["codecs"].items():
+ for row in e["rows"]:
+ mb = lambda b: f"{b / 1e6:.2f}" if b else "-" # noqa: E731
+ q = row["matched_q"] if row["matched_q"] is not None else "off grid"
+ now_ssim = f"{row['now_ssim']:.4f}" if row["now_ssim"] else "-"
+ lines.append(f"| {r['clip']} | {codec} | {row['crf']} ({row['cpu_crf']}) | {mb(row['cpu_bytes'])} | "
+ f"{row['cpu_ssim']:.4f} | {q} | {mb(row['matched_vt_bytes'])} | {row['vt_quality_now']} | "
+ f"{mb(row['now_bytes'])} | {now_ssim} |")
+ return "\n".join(lines)
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ ap.add_argument("--clip", action="append", default=[], help="a real clip to add (repeatable)")
+ ap.add_argument("--no-synthetic", action="store_true", help="leave out the three generated clips")
+ ap.add_argument("--seconds", type=float, default=6.0)
+ ap.add_argument("--crf", type=int, nargs="+", default=[18, 23, 28])
+ ap.add_argument("--codec", nargs="+", default=["h264", "hevc"], choices=["h264", "hevc"], help="SDR codecs (HDR is hevc)")
+ ap.add_argument("--grid", type=int, nargs="+", default=DEFAULT_GRID, help="VideoToolbox -q:v values to sweep")
+ ap.add_argument("--hwdec", action="store_true", help="also time -hwaccel videotoolbox decode (a CPU scale between)")
+ ap.add_argument("--json")
+ args = ap.parse_args()
+ decision.STATE.hw = True
+ results = []
+ with tempfile.TemporaryDirectory(prefix="ffskill_bvt_") as tmp:
+ clips = []
+ if not args.no_synthetic:
+ for name, graph in SYNTHETIC.items():
+ path = os.path.join(tmp, f"{name}.mkv")
+ r = sh(["ffmpeg", "-v", "error", "-y", "-f", "lavfi", "-i", graph, "-t", f"{args.seconds}",
+ "-c:v", "ffv1", "-pix_fmt", "yuv420p", path])
+ if r.returncode != 0:
+ print(f"could not generate {name}: {r.stderr[-400:]}", file=sys.stderr)
+ return 1
+ clips.append((name, path))
+ clips += [(Path(c).stem, os.path.expanduser(c)) for c in args.clip]
+ if not clips:
+ print("no clips: pass --clip or drop --no-synthetic")
+ return 1
+ for name, path in clips:
+ print(f"{name} ...", file=sys.stderr)
+ results.append(bench_clip(name, path, probe(path), args, tmp))
+ if args.json: # write as we go: a long run keeps what it measured
+ Path(args.json).write_text(json.dumps(results, indent=2))
+ print(table(results))
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.json
index f17118c..4689475 100644
--- a/tests/fixtures/mcp_tools.json
+++ b/tests/fixtures/mcp_tools.json
@@ -71,7 +71,9 @@
"fps",
"gradient",
"height",
+ "hw",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -90,8 +92,10 @@
"fast",
"folder",
"force",
+ "hw",
"jobs",
"json_brief",
+ "no_hw",
"overwrite",
"plan",
"recipe",
@@ -113,9 +117,11 @@
"end",
"fast",
"from_",
+ "hw",
"input",
"insert",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"pad_color",
@@ -153,6 +159,7 @@
"fps",
"gap",
"highlight_color",
+ "hw",
"input",
"json_brief",
"karaoke",
@@ -166,6 +173,7 @@
"min_size",
"mode",
"model",
+ "no_hw",
"offset",
"outline",
"outline_color",
@@ -228,6 +236,7 @@
"force",
"gain",
"gamma",
+ "hw",
"input",
"json_brief",
"levels_in_black",
@@ -237,6 +246,7 @@
"lift",
"lut",
"lut_strength",
+ "no_hw",
"output",
"overwrite",
"peak",
@@ -263,8 +273,10 @@
"fast",
"fps",
"height",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -306,9 +318,11 @@
"duration",
"end",
"fast",
+ "hw",
"input",
"json_brief",
"min_confidence",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -332,9 +346,11 @@
"codec",
"dry_run",
"fast",
+ "hw",
"input",
"json_brief",
"mode",
+ "no_hw",
"only_interlaced",
"output",
"overwrite",
@@ -357,10 +373,12 @@
"dry_run",
"fast",
"fps",
+ "hw",
"input",
"json_brief",
"luma_spatial",
"luma_temporal",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -380,9 +398,11 @@
"dry_run",
"fast",
"fit",
+ "hw",
"input",
"json_brief",
"list",
+ "no_hw",
"no_scale",
"normalize",
"output",
@@ -413,10 +433,12 @@
"fps",
"from_center",
"height",
+ "hw",
"input",
"json_brief",
"max_speed",
"method",
+ "no_hw",
"output",
"overwrite",
"pad_blur",
@@ -441,9 +463,11 @@
"dry_run",
"fast",
"hold",
+ "hw",
"input",
"json_brief",
"mode",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -472,11 +496,13 @@
"fast",
"font",
"font_file",
+ "hw",
"input",
"json_brief",
"lang",
"margin",
"name",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -517,9 +543,11 @@
"font_size",
"fps",
"gap",
+ "hw",
"inputs",
"json_brief",
"label",
+ "no_hw",
"output",
"overwrite",
"pad",
@@ -541,8 +569,10 @@
"fast",
"fps",
"height",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"pan",
@@ -569,9 +599,11 @@
"fit",
"fps",
"height",
+ "hw",
"inputs",
"json_brief",
"list",
+ "no_hw",
"on_missing",
"on_silent",
"output",
@@ -619,8 +651,10 @@
"dry_run",
"duration",
"fast",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -698,10 +732,12 @@
"fix_drift",
"fps",
"height",
+ "hw",
"inputs",
"json_brief",
"max_offset",
"min_shot",
+ "no_hw",
"offsets_only",
"output",
"overwrite",
@@ -741,11 +777,13 @@
"font_color",
"font_file",
"font_size",
+ "hw",
"image",
"input",
"json_brief",
"logo",
"margin",
+ "no_hw",
"opacity",
"output",
"overwrite",
@@ -772,8 +810,10 @@
"dry_run",
"end",
"fast",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -809,9 +849,11 @@
"dry_run",
"fast",
"fps",
+ "hw",
"input",
"json_brief",
"no_audio",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -834,9 +876,11 @@
"fast",
"fps",
"height",
+ "hw",
"input",
"json_brief",
"mode",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -862,12 +906,14 @@
"fast",
"fit",
"from_stage",
+ "hw",
"image",
"init",
"json_brief",
"keep",
"list_templates",
"logo",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -911,9 +957,11 @@
"codec",
"dry_run",
"fast",
+ "hw",
"input",
"json_brief",
"no_audio",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -964,7 +1012,9 @@
"fast",
"fps",
"height",
+ "hw",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"pattern",
@@ -992,6 +1042,7 @@
"filler_list",
"filler_pad",
"filler_words",
+ "hw",
"input",
"json_brief",
"list",
@@ -999,6 +1050,7 @@
"max_cuts",
"min_keep",
"min_silence",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -1019,8 +1071,10 @@
"codec",
"dry_run",
"fast",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -1042,10 +1096,12 @@
"fps",
"h_fov",
"height",
+ "hw",
"input",
"input_projection",
"interp",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"pitch",
@@ -1069,8 +1125,10 @@
"crop",
"dry_run",
"fast",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -1096,8 +1154,10 @@
"fill_color",
"fit",
"fps",
+ "hw",
"input",
"json_brief",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -1118,9 +1178,11 @@
"fast",
"fine_ms",
"fix_drift",
+ "hw",
"json_brief",
"max_offset",
"more_sources",
+ "no_hw",
"output",
"overwrite",
"plan",
@@ -1166,10 +1228,12 @@
"fast",
"fps",
"height",
+ "hw",
"image",
"image_fit",
"input",
"json_brief",
+ "no_hw",
"on_silent",
"opacity",
"output",
diff --git a/tests/test_accel.py b/tests/test_accel.py
new file mode 100644
index 0000000..2b05874
--- /dev/null
+++ b/tests/test_accel.py
@@ -0,0 +1,1102 @@
+#!/usr/bin/env python3
+"""Tests for Apple VideoToolbox encoding (--hw).
+
+ python3 tests/test_accel.py # this group alone
+ python3 tests/test_all.py # every group
+
+The pure-logic cases run everywhere. The real VideoToolbox encodes run only on Apple Silicon with
+an ffmpeg that lists the encoders.
+"""
+import argparse
+import contextlib
+import importlib
+import io
+import json
+import os
+import platform
+import shutil
+import subprocess
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from _fixtures import MediaFixtures, OUT, SCRIPTS, script, sh # noqa: E402
+from _common import STATE, add_common, apply_common, ffmpeg_encoders # noqa: E402
+import _common # noqa: E402
+from _common import decision, runner # noqa: E402
+
+emit_module = importlib.import_module("_common.emit") # the module; `_common.emit` is the function
+
+
+def _vt_here() -> bool:
+ return (platform.system() == "Darwin" and platform.machine() == "arm64"
+ and {"h264_videotoolbox", "hevc_videotoolbox"} <= ffmpeg_encoders())
+
+
+def _parse(argv, codec=True, orchestrator=False):
+ ap = argparse.ArgumentParser()
+ ap.add_argument("input", nargs="?")
+ ap.add_argument("-o", "--output")
+ if codec and not orchestrator:
+ ap.set_defaults(crf=18)
+ add_common(ap, codec=codec)
+ if orchestrator:
+ runner.add_hw_orchestrator_args(ap)
+ args = ap.parse_args(argv)
+ apply_common(args)
+ return args
+
+
+def require_ffmpeg_or_skip(*binaries):
+ """As test_contract.py's: locally a skip, in CI (CI=true) a broken install step."""
+ missing = [b for b in binaries if not shutil.which(b)]
+ if not missing:
+ return
+ msg = f"{'/'.join(missing)} not on PATH"
+ if os.environ.get("CI"):
+ raise AssertionError(f"{msg} -- in CI this is a broken install step, not a reason to skip")
+ raise unittest.SkipTest(msg)
+
+
+# What FFmpeg really prints when a VideoToolbox encoder cannot open, one shape per wording
+# (fftools/ffmpeg*.c and libavcodec/videotoolboxenc.c of each version; pointers shortened).
+VT_OPEN_FAILED = {
+ "5.1": "[h264_videotoolbox @ 0x7f9e1c004a00] Error: cannot create compression session: -12902\n"
+ "[h264_videotoolbox @ 0x7f9e1c004a00] Try -allow_sw 1. The hardware encoder may be busy, or not supported.\n"
+ "Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe "
+ "incorrect parameters such as bit_rate, rate, width or height\n",
+ "6.0": "[h264_videotoolbox @ 0x14f6063b0] Error: cannot create compression session: -12908\n"
+ "[vost#0:0/h264_videotoolbox @ 0x14f605f40] Error initializing output stream: Error while opening encoder "
+ "for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height\n",
+ "6.1": "[h264_videotoolbox @ 0x600002d0c000] Error: cannot create compression session: -12902\n"
+ "[h264_videotoolbox @ 0x600002d0c000] Try -allow_sw 1. The hardware encoder may be busy, or not supported.\n"
+ "[vost#0:0/h264_videotoolbox @ 0x600002d0c1e0] Error while opening encoder - maybe incorrect parameters "
+ "such as bit_rate, rate, width or height.\n",
+ "7.1": "[hevc_videotoolbox @ 0x13a604a40] Error: -q:v qscale not available for encoder. Use -b:v bitrate instead.\n"
+ "[vost#0:0/hevc_videotoolbox @ 0x13a6046a0] Error while opening encoder - maybe incorrect parameters such "
+ "as bit_rate, rate, width or height.\n"
+ "[vf#0:0 @ 0x13a604e30] Error sending frames to consumers: Generic error in an external library\n",
+ "8.0": "[vost#0:0/h264_videotoolbox @ 0x600003a1c0f0] [enc:h264_videotoolbox @ 0x600003a1c1e0] Error while opening "
+ "encoder - maybe incorrect parameters such as bit_rate, rate, width or height.\n"
+ "[vf#0:0 @ 0x600003a1c3c0] Error sending frames to consumers: Generic error in an external library\n",
+ "master": "[prores_videotoolbox @ 0x12e604a40] Cannot create compression session: -12903\n"
+ "[vost#0:0/prores_videotoolbox @ 0x12e6046a0] [enc:prores_videotoolbox @ 0x12e604960] Error while opening "
+ "encoder - maybe incorrect parameters such as bit_rate, rate, width or height.\n",
+ "mid-stream": "[h264_videotoolbox @ 0x7fa1] Error encoding frame: -12911\n"
+ "[vost#0:0/h264_videotoolbox @ 0x7fa2] Error submitting video frame to the encoder\n",
+}
+# Failures of a VideoToolbox job that are not VideoToolbox's: retrying them on the CPU would run
+# the job twice and blame the GPU.
+NOT_VT = {
+ "missing font (7.x: the encoder never opened)":
+ "[Parsed_drawtext_0 @ 0x6000] Cannot find a valid font for the family Sans\n"
+ "[AVFilterGraph @ 0x6001] Error initializing filters\n"
+ "[vost#0:0/h264_videotoolbox @ 0x6002] Could not open encoder before EOF\n"
+ "[vost#0:0/h264_videotoolbox @ 0x6002] Task finished with error code: -22 (Invalid argument)\n",
+ "bad filter graph (6.1)":
+ "[AVFilterGraph @ 0x7000] No such filter: 'scalee'\n"
+ "Error reinitializing filters!\nFailed to inject frame into filter network: Invalid argument\n",
+ "full disk after a non-fatal encoder error line":
+ "[h264_videotoolbox @ 0x8000] Error setting profile/level property: -12900. Output will be encoded using a "
+ "supported profile/level combination.\n"
+ "[vost#0:0/h264_videotoolbox @ 0x8001] Error submitting a packet to the muxer: No space left on device\n"
+ "[out#0/mp4 @ 0x8002] Error writing trailer: No space left on device\n",
+ "the audio encoder failed to open (6.1+)":
+ "[h264_videotoolbox @ 0x9000] Error setting entropy property: -12900\n"
+ "[aost#0:1/aac @ 0x9001] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, "
+ "width or height.\n",
+ "the audio encoder failed to open (5.1: no stream context, a non-fatal VideoToolbox line before it)":
+ "[h264_videotoolbox @ 0x9000] Error setting entropy property: -12900\n"
+ "Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe "
+ "incorrect parameters such as bit_rate, rate, width or height\n",
+ "a missing input": "a.mp4: No such file or directory\n",
+}
+
+
+class FakeFfmpeg:
+ """A stand-in ffmpeg for run(): an `ffmpeg` (POSIX sh) or `ffmpeg.cmd` (Windows) wrapper that
+ runs a Python script with this interpreter, so the same fake works on every CI runner. Each
+ call is logged; a command naming a *_videotoolbox encoder gets the `vt` behaviour, any other
+ the `cpu` one: {"rc", "stderr"}, or {"real": "<ffmpeg path>"} to hand the command to a real
+ ffmpeg (the CPU encode then writes a real, probe-able file)."""
+
+ def __init__(self, directory: Path, behaviour):
+ self.dir = Path(directory)
+ self.log = self.dir / "calls.jsonl"
+ conf = self.dir / "fake_ffmpeg.json"
+ conf.write_text(json.dumps(behaviour), encoding="utf-8")
+ body = self.dir / "fake_ffmpeg.py"
+ body.write_text(
+ "import json, subprocess, sys\n"
+ f"conf = json.load(open({str(conf)!r}, encoding='utf-8'))\n"
+ "args = sys.argv[1:]\n"
+ f"with open({str(self.log)!r}, 'a', encoding='utf-8') as fh:\n"
+ " fh.write(json.dumps(args) + '\\n')\n"
+ "do = conf['vt' if any(a.endswith('_videotoolbox') for a in args) else 'cpu']\n"
+ "if 'real' in do:\n"
+ " sys.exit(subprocess.run([do['real']] + args).returncode)\n"
+ "sys.stderr.write(do.get('stderr', ''))\n"
+ "sys.exit(do.get('rc', 0))\n", encoding="utf-8")
+ if os.name == "nt":
+ self.path = self.dir / "ffmpeg.cmd"
+ # text mode writes the CRLF line ends cmd.exe expects; run() starts a .cmd directly
+ # (CreateProcess runs it through cmd.exe), and the arguments here have no cmd metacharacters
+ self.path.write_text(f'@"{sys.executable}" "{body}" %*\n@exit /b %ERRORLEVEL%\n', encoding="utf-8")
+ else:
+ self.path = self.dir / "ffmpeg"
+ self.path.write_text(f'#!/bin/sh\nexec "{sys.executable}" "{body}" "$@"\n', encoding="utf-8")
+ self.path.chmod(0o755)
+
+ def calls(self):
+ if not self.log.exists():
+ return []
+ return [json.loads(ln) for ln in self.log.read_text(encoding="utf-8").splitlines() if ln.strip()]
+
+
+def _vt_available():
+ """VideoToolbox as an Apple Silicon build would report it, on any runner: the platform check
+ passes and the encoder list has the VideoToolbox (and CPU) encoders."""
+ return (mock.patch.object(decision, "hw_platform_reason", return_value=None),
+ mock.patch.object(decision, "ffmpeg_encoders", return_value={
+ "h264_videotoolbox", "hevc_videotoolbox", "prores_videotoolbox", "libx264", "libx265", "prores_ks"}))
+
+
+class HwResolutionTests(unittest.TestCase):
+ """--hw / --no-hw / $FFMPEG_SKILL_HW resolve the same way in every tool."""
+
+ def setUp(self):
+ self._env = mock.patch.dict(os.environ, {}, clear=False)
+ self._env.start()
+ for k in (runner.HW_ENV, runner.HW_FORCED_ENV):
+ os.environ.pop(k, None)
+
+ def tearDown(self):
+ self._env.stop()
+ STATE.reset()
+
+ def test_off_by_default(self):
+ _parse(["in.mp4"])
+ self.assertFalse(STATE.hw)
+ self.assertIsNone(STATE.hw_source)
+
+ def test_env_turns_it_on_and_no_hw_turns_it_off(self):
+ os.environ[runner.HW_ENV] = "1"
+ _parse(["in.mp4"])
+ self.assertEqual((STATE.hw, STATE.hw_source), (True, "env"))
+ _parse(["in.mp4", "--no-hw"])
+ self.assertEqual((STATE.hw, STATE.hw_source), (False, "flag"))
+
+ def test_export_style_tools_ignore_the_env_default(self):
+ """A delivery preset (codec=False: export.py) is on the GPU only when asked explicitly."""
+ os.environ[runner.HW_ENV] = "1"
+ _parse(["in.mp4"], codec=False)
+ self.assertFalse(STATE.hw)
+ _parse(["in.mp4", "--hw"], codec=False)
+ self.assertEqual((STATE.hw, STATE.hw_source), (True, "flag"))
+
+ def test_orchestrator_flag_reaches_export_style_children_as_explicit(self):
+ """render.py/batch.py --hw is an explicit choice for every stage, export.py included;
+ their --no-hw overrides a machine default for every stage."""
+ _parse(["p.json", "--hw"], orchestrator=True)
+ self.assertEqual((os.environ.get(runner.HW_ENV), os.environ.get(runner.HW_FORCED_ENV)), ("1", "1"))
+ _parse(["in.mp4"], codec=False) # a child export.py
+ self.assertEqual((STATE.hw, STATE.hw_source), (True, "flag"))
+ _parse(["p.json", "--no-hw"], orchestrator=True)
+ _parse(["in.mp4"]) # a child fit.py
+ self.assertEqual((STATE.hw, STATE.hw_source), (False, "flag"))
+
+ def test_tools_without_an_encoder_never_pick_it_up(self):
+ os.environ[runner.HW_ENV] = "1"
+ ap = argparse.ArgumentParser()
+ ap.add_argument("input")
+ add_common(ap) # no crf default: an analysis tool
+ args = ap.parse_args(["in.mp4"])
+ self.assertFalse(hasattr(args, "hw"))
+ apply_common(args)
+ self.assertFalse(STATE.hw)
+
+
+class VtArgsTests(unittest.TestCase):
+ def tearDown(self):
+ STATE.reset()
+
+ def test_quality_mapping_is_monotonic_and_bounded(self):
+ for codec, hdr in (("h264", False), ("hevc", False), ("hevc", True)):
+ qs = [decision.vt_quality(codec, crf, hdr) for crf in range(0, 52)]
+ self.assertEqual(qs, sorted(qs, reverse=True))
+ self.assertTrue(all(1 <= q <= 100 for q in qs))
+
+ def test_an_hdr_source_gets_the_hdr_quality_curve(self):
+ """HLG phone footage at the SDR curve's -q:v came out 6-8x x265's bytes at a higher SSIM
+ (tests/bench_vt.py); the Main10 line takes its own, lower curve, chosen from the source."""
+ STATE.hw = True
+ hdr = {"video": {"bt2020_or_hdr": True, "color_transfer": "arib-std-b67"}}
+ sdr = {"video": {"bt2020_or_hdr": False}}
+ with mock.patch.object(decision, "hw_platform_reason", return_value=None), \
+ mock.patch.object(decision, "ffmpeg_encoders", return_value={"hevc_videotoolbox"}):
+ for crf in (18, 23, 28):
+ q_hdr = decision._vt_args("hevc", crf, hdr, True)
+ q_sdr = decision._vt_args("hevc", crf, sdr, True)
+ q_hdr, q_sdr = int(q_hdr[q_hdr.index("-q:v") + 1]), int(q_sdr[q_sdr.index("-q:v") + 1])
+ self.assertEqual(q_hdr, decision.vt_quality("hevc", crf, True))
+ self.assertEqual(q_sdr, decision.vt_quality("hevc", crf))
+ self.assertLess(q_hdr, q_sdr - 10, crf)
+
+ def test_the_curves_keep_the_measured_values(self):
+ """The CRF 18/23/28 values tests/bench_vt.py measured (the highest -q:v that matched the CPU
+ encode's SSIM on every clip); a slope typo moves one of them."""
+ self.assertEqual([decision.vt_quality("h264", c) for c in (18, 23, 28)], [75, 64, 53])
+ self.assertEqual([decision.vt_quality("hevc", c) for c in (18, 23, 28)], [78, 68, 58])
+ self.assertEqual([decision.vt_quality("hevc", c, True) for c in (18, 23, 28)], [63, 55, 47])
+
+ def test_an_h264_line_on_an_hdr_source_keeps_the_h264_curve(self):
+ """An export preset's x264 line on an HLG source: the VideoToolbox H.264 line replaces x264
+ at the same CRF, so the HDR (Main10, CRF+2) curve does not apply."""
+ STATE.hw = True
+ hdr = {"video": {"bt2020_or_hdr": True, "color_transfer": "arib-std-b67"}}
+ with mock.patch.object(decision, "hw_platform_reason", return_value=None), \
+ mock.patch.object(decision, "ffmpeg_encoders", return_value={"h264_videotoolbox"}):
+ line = decision.hw_preset_video(["-c:v", "libx264", "-preset", "slow", "-crf", "18", "-pix_fmt", "yuv420p"], hdr, bt709=True)
+ self.assertEqual(line[line.index("-q:v") + 1], "75")
+
+ def test_av1_and_an_intel_mac_stay_on_the_cpu_with_a_note(self):
+ STATE.hw = True
+ self.assertIsNone(decision._vt_args("av1", 30, None, True))
+ with mock.patch.object(decision, "hw_platform_reason", return_value="VideoToolbox constant-quality encoding needs Apple Silicon"):
+ cpu = ["-c:v", "libx264", "-crf", "18"]
+ self.assertEqual(decision._maybe_hw("h264", 18, None, True, cpu), cpu)
+ self.assertTrue(any("av1" in n for n in STATE.hw_notes))
+ self.assertTrue(any("Apple Silicon" in n for n in STATE.hw_notes))
+
+ def test_run_can_put_the_cpu_line_back(self):
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ cmd = ["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", "out.mp4"]
+ self.assertEqual(runner._hw_fallback(cmd, STATE), ["ffmpeg", "-i", "a.mp4", "-c:v", "libx264", "-crf", "18", "out.mp4"])
+ self.assertIsNone(runner._hw_fallback(["ffmpeg", "-i", "a", "b"], STATE))
+
+ def test_export_preset_keeps_its_frame_rate_and_leaves_faststart_to_export(self):
+ STATE.hw = True
+ with mock.patch.object(decision, "hw_platform_reason", return_value=None), \
+ mock.patch.object(decision, "ffmpeg_encoders", return_value={"h264_videotoolbox"}):
+ vt = decision.hw_preset_video(["-c:v", "libx264", "-preset", "medium", "-crf", "20", "-profile:v", "high",
+ "-pix_fmt", "yuv420p", "-r", "30"], None, bt709=True)
+ self.assertEqual(vt[vt.index("-c:v") + 1], "h264_videotoolbox")
+ self.assertEqual(vt[vt.index("-r") + 1], "30")
+ self.assertNotIn("-movflags", vt)
+ self.assertNotIn("-preset", vt)
+
+ def test_render_cache_key_changes_with_the_gpu_setting(self):
+ """A VideoToolbox artifact is never served to a CPU run (and the other way round)."""
+ import render
+ with mock.patch.dict(os.environ, {runner.HW_ENV: "0"}):
+ os.environ.pop(runner.HW_FORCED_ENV, None)
+ cpu = render.cache_key("fit", "fit.py", ["--height", "720"], [])
+ with mock.patch.dict(os.environ, {runner.HW_ENV: "1"}):
+ gpu = render.cache_key("fit", "fit.py", ["--height", "720"], [])
+ with mock.patch.dict(os.environ, {runner.HW_ENV: "1", runner.HW_FORCED_ENV: "1"}):
+ forced = render.cache_key("fit", "fit.py", ["--height", "720"], [])
+ self.assertEqual(len({cpu, gpu, forced}), 3)
+
+
+class HwReviewRegressionTests(unittest.TestCase):
+ """Cases a review of the first --hw implementation found."""
+
+ def tearDown(self):
+ STATE.reset()
+
+ def _vt_ok(self):
+ return (mock.patch.object(decision, "hw_platform_reason", return_value=None),
+ mock.patch.object(decision, "ffmpeg_encoders", return_value={"h264_videotoolbox", "hevc_videotoolbox", "prores_videotoolbox"}))
+
+ def test_export_edits_to_a_vt_line_keep_the_fallback_findable(self):
+ """export.py strips -movflags from encoder_args()'s line; the swap must follow the edit."""
+ STATE.hw = True
+ a, b = self._vt_ok()
+ with a, b:
+ built = decision.encoder_args("hevc", 20, "medium", {"video": {"bt2020_or_hdr": True, "color_transfer": "smpte2084"}})
+ stripped = decision.strip_movflags(built)
+ decision.restate_last_swap(built, stripped)
+ cmd = ["ffmpeg", "-i", "in.mov"] + stripped + ["-movflags", "+faststart", "out.mp4"]
+ back = runner._hw_fallback(cmd, STATE)
+ self.assertIsNotNone(back)
+ self.assertIn("libx265", back)
+
+ def test_export_preset_fallback_is_tagged_like_a_cpu_run(self):
+ STATE.hw = True
+ a, b = self._vt_ok()
+ with a, b:
+ decision.hw_preset_video(["-c:v", "libx264", "-preset", "slow", "-crf", "18", "-profile:v", "high", "-pix_fmt", "yuv420p"], None, bt709=True)
+ cpu = STATE.hw_swaps[-1][1]
+ self.assertTrue("-x264-params" in cpu or "-colorspace" in cpu, cpu)
+
+ def test_vt_bt709_tags_come_from_a_bitstream_filter_on_every_version(self):
+ """On FFmpeg >= 7.1 the -colorspace output options insert a real matrix conversion on an
+ untagged source, so VideoToolbox gets its tags from a bitstream filter instead. A git build
+ of 7.1 reads as (7, 0), so the tag path must not change with the version guess either."""
+ for codec in ("h264", "hevc"):
+ for v in ((6, 1), (7, 0), (7, 1), (9, 0)):
+ with mock.patch.object(decision, "ffmpeg_version", return_value=v):
+ tags = decision._vt_bt709(codec)
+ self.assertEqual(tags[0], "-bsf:v", (codec, v))
+ self.assertNotIn("-colorspace", tags)
+ self.assertNotIn("-x264-params", tags)
+
+ def test_run_retries_a_refused_videotoolbox_encode_on_the_cpu_and_reports_it(self):
+ """run() itself: a failed VT encode is re-run once with the recorded CPU line, the note and
+ the recorded command follow, and the result reports the CPU encoder."""
+ import importlib
+ emit = importlib.import_module("_common.emit")
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["6.1"])
+ ok = subprocess.CompletedProcess([], 0, "", "")
+ with tempfile.TemporaryDirectory() as d, \
+ mock.patch.object(runner, "_execute", side_effect=[refused, ok]) as execute:
+ out = str(Path(d) / "o.mp4")
+ proc = runner.run(["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", out], quiet=True)
+ self.assertEqual(proc.returncode, 0)
+ self.assertEqual(execute.call_count, 2)
+ retried = execute.call_args_list[1][0][0]
+ self.assertIn("libx264", retried)
+ self.assertNotIn("h264_videotoolbox", retried)
+ self.assertIn("libx264", STATE.commands[-1])
+ self.assertTrue(any("VideoToolbox refused" in n and "cannot create compression session" in n for n in STATE.hw_notes),
+ STATE.hw_notes)
+ rep = emit._encoder_report(STATE)
+ self.assertEqual(rep["encoder"], "libx264")
+ self.assertFalse(rep["hw"]["used"])
+ self.assertTrue(rep["hw"]["fallback"])
+
+ def test_the_cpu_fallback_keeps_the_even_dimension_scale_of_a_retry(self):
+ """An odd-sized source under --hw: the encode is retried with an even scale, and when
+ VideoToolbox refuses that retry the CPU line is swapped into the retry, not into the
+ first command (x264 then failed on the same odd frame)."""
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ odd = subprocess.CompletedProcess([], 1, "", "width not divisible by 2 (641x359)\n")
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["8.0"])
+ ok = subprocess.CompletedProcess([], 0, "", "")
+ with tempfile.TemporaryDirectory() as d, \
+ mock.patch.object(runner, "_execute", side_effect=[odd, refused, ok]) as execute:
+ out = str(Path(d) / "o.mp4")
+ proc = runner.run(["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", out], quiet=True)
+ self.assertEqual(proc.returncode, 0)
+ cpu = execute.call_args_list[2][0][0]
+ self.assertIn("libx264", cpu)
+ self.assertTrue(any("scale" in a for a in cpu), cpu)
+ self.assertIn("scale", STATE.commands[-1])
+
+ def test_encoder_report_keeps_the_encode_behind_a_later_copy(self):
+ import importlib
+ emit = importlib.import_module("_common.emit")
+ STATE.commands = ["ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4", "ffmpeg -i b.mp4 -c:v copy -af loudnorm c.mp4"]
+ STATE.hw, STATE.hw_source = True, "flag"
+ rep = emit._encoder_report(STATE)
+ self.assertEqual(rep["encoder"], "h264_videotoolbox")
+ self.assertTrue(rep["hw"]["used"])
+ STATE.commands = ["ffmpeg -ss 1 -i a.mp4 -c copy -t 2 b.mp4"]
+ self.assertEqual(emit._encoder_report(STATE)["encoder"], "copy")
+ STATE.commands = [] # batch.py: the stages are child processes it does not record
+ self.assertIsNone(emit._encoder_report(STATE)["hw"]["used"])
+
+ def test_an_env_chosen_gpu_encode_says_how_to_get_the_cpu_one(self):
+ """FFMPEG_SKILL_HW=1 puts every tool on VideoToolbox, whose files are larger at the same
+ quality; a result says so only when the environment chose the GPU and the GPU ran."""
+ import importlib
+ emit = importlib.import_module("_common.emit")
+ gpu, cpu = "ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4", "ffmpeg -i a -c:v libx264 -crf 18 b.mp4"
+ for hw, source, command, noted in [(True, "env", gpu, True), (True, "flag", gpu, False),
+ (True, "env", cpu, False), (False, None, cpu, False)]:
+ with self.subTest(hw=hw, source=source, command=command):
+ STATE.hw, STATE.hw_source, STATE.hw_notes, STATE.commands = hw, source, [], [command]
+ rep = emit._encoder_report(STATE)
+ if not hw:
+ self.assertNotIn("hw", rep)
+ continue
+ notes = rep["hw"]["notes"]
+ self.assertEqual(any("FFMPEG_SKILL_HW=1" in n and "--no-hw" in n for n in notes), noted, notes)
+ self.assertEqual(STATE.hw_notes, [], "the report adds the note; the run's own list is untouched")
+
+ def test_hw_dry_run_runs_no_ffmpeg(self):
+ """The dry-run promise: --hw's encoder check reads the build through ffprobe, never ffmpeg.
+ (On a machine without VideoToolbox the check is never reached; EncoderListTests pins the
+ ffprobe-only-in-a-dry-run rule itself on every runner.)"""
+ require_ffmpeg_or_skip("ffprobe")
+ with tempfile.TemporaryDirectory() as d:
+ fake = FakeFfmpeg(Path(d), {"vt": {"rc": 1, "stderr": "called\n"}, "cpu": {"rc": 1, "stderr": "called\n"}})
+ env = dict(os.environ, PATH=f"{d}{os.pathsep}{os.environ.get('PATH', '')}")
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "fit.py"), str(Path(d) / "pending.mp4"), "--height", "360", "--hw",
+ "--dry-run", "--json", "-o", str(Path(d) / "o.mp4")], env=env, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace")
+ self.assertEqual(proc.returncode, 0, proc.stderr)
+ self.assertEqual(fake.calls(), [])
+
+
+class VideoToolboxFailureTests(unittest.TestCase):
+ """Which ffmpeg failures run() retries on the CPU: only VideoToolbox's own (review of #304)."""
+
+ def test_every_ffmpeg_version_s_videotoolbox_open_failure_is_recognised(self):
+ for version, stderr in VT_OPEN_FAILED.items():
+ with self.subTest(version=version):
+ line = runner.videotoolbox_failure(stderr)
+ self.assertIsNotNone(line, stderr)
+ self.assertIn("videotoolbox", line)
+ self.assertNotIn("0x", line, "log-context pointers are dropped from the note")
+
+ def test_a_failure_that_is_not_videotoolbox_s_is_not(self):
+ for case, stderr in NOT_VT.items():
+ with self.subTest(case=case):
+ self.assertIsNone(runner.videotoolbox_failure(stderr))
+
+
+class EncoderListTests(unittest.TestCase):
+ """ffmpeg_encoders() reads the build that encodes (`ffmpeg -encoders`); `ffprobe -encoders`
+ only under --dry-run, which promises never to run ffmpeg (review of #304: on a mixed install
+ ffprobe's list is another build's, and the AV1/ProRes checks read it too)."""
+
+ def setUp(self):
+ runner._ENCODERS = None
+
+ def tearDown(self):
+ runner._ENCODERS = None
+ STATE.reset()
+
+ def _read(self, dry_run):
+ STATE.dry_run = dry_run
+ listing = " V....D libsvtav1 SVT-AV1\n V....D h264_videotoolbox VideoToolbox H.264 Encoder\n"
+ with mock.patch.object(runner.shutil, "which", side_effect=lambda name: os.path.join("bin", name)), \
+ mock.patch.object(runner.subprocess, "run",
+ return_value=subprocess.CompletedProcess([], 0, listing, "")) as call:
+ names = runner.ffmpeg_encoders()
+ return os.path.basename(call.call_args[0][0][0]), names
+
+ def test_a_real_run_reads_ffmpeg_s_own_list(self):
+ tool, names = self._read(dry_run=False)
+ self.assertEqual(tool, "ffmpeg")
+ self.assertEqual(names, {"libsvtav1", "h264_videotoolbox"})
+
+ def test_only_a_dry_run_reads_ffprobe_s(self):
+ tool, _names = self._read(dry_run=True)
+ self.assertEqual(tool, "ffprobe")
+
+
+class _RealFfmpegCase(unittest.TestCase):
+ """A 1 s source made with the real ffmpeg, a work directory, and no GPU variables leaking in."""
+
+ @classmethod
+ def setUpClass(cls):
+ require_ffmpeg_or_skip("ffmpeg", "ffprobe")
+ cls.real_ffmpeg = shutil.which("ffmpeg")
+ cls.tmp = tempfile.TemporaryDirectory()
+ cls.src = Path(cls.tmp.name) / "src.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=320x240:rate=30",
+ "-t", "1", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", cls.src)
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.tmp.cleanup()
+
+ def setUp(self):
+ STATE.reset()
+ self._env = mock.patch.dict(os.environ, {}, clear=False)
+ self._env.start()
+ for k in (runner.HW_ENV, runner.HW_FORCED_ENV):
+ os.environ.pop(k, None)
+ self.work = Path(tempfile.mkdtemp(dir=self.tmp.name))
+
+ def tearDown(self):
+ self._env.stop()
+ STATE.reset()
+
+ def _tool(self, module_name, *argv, env=None):
+ """Run a tool's main() in this process (so _vt_available() applies) and parse its --json."""
+ import importlib
+ module = importlib.import_module(module_name)
+ out, err = io.StringIO(), io.StringIO()
+ a, b = _vt_available()
+ with a, b, mock.patch.dict(os.environ, env or {}), \
+ mock.patch.object(sys, "argv", [module_name + ".py"] + [str(x) for x in argv] + ["--json"]), \
+ contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
+ STATE.reset()
+ try:
+ module.main()
+ except SystemExit as e:
+ self.assertEqual(e.code, 0, err.getvalue())
+ return json.loads(out.getvalue())
+
+
+class HwEverywhereTests(_RealFfmpegCase):
+ """The --hw decisions on every CI runner, VideoToolbox or not: the platform check and the
+ encoder list are stood in for (_vt_available), the encodes are a fake ffmpeg that fails the
+ VideoToolbox command the way FFmpeg does and hands the CPU one to the real ffmpeg."""
+
+ def test_export_preset_needs_an_explicit_hw(self):
+ """docs/design-decisions.md: FFMPEG_SKILL_HW=1 leaves export.py's delivery presets on x264 (a
+ dry run: the encoder choice), says so, and --hw -- or render.py --hw's explicit marker --
+ puts the same preset on VideoToolbox, so the platform stand-in is not what kept it off."""
+ out = self.work / "x.mp4"
+ doc = self._tool("export", self.src, "--preset", "x", "--dry-run", "-o", out, env={runner.HW_ENV: "1"})
+ self.assertEqual(doc["encoder"], "libx264")
+ self.assertNotIn("videotoolbox", " ".join(doc["commands"]))
+ self.assertEqual({k: doc["hw"][k] for k in ("requested", "source", "used", "fallback")},
+ {"requested": False, "source": "env", "used": None, "fallback": False}, "a dry run: nothing ran")
+ self.assertTrue(any("--hw" in n and "FFMPEG_SKILL_HW=1" in n for n in doc["hw"]["notes"]), doc["hw"])
+ self.assertFalse(any("CRF-equivalent" in n for n in doc.get("notes") or []))
+ for argv, env in ((["--hw"], {}), ([], {runner.HW_ENV: "1", runner.HW_FORCED_ENV: "1"})):
+ with self.subTest(argv=argv, env=env):
+ doc = self._tool("export", self.src, "--preset", "x", "--dry-run", "-o", out, *argv, env=env)
+ self.assertEqual(doc["encoder"], "h264_videotoolbox", "the planned encoder")
+ self.assertIsNone(doc["hw"]["used"], "a dry run ran nothing")
+ self.assertTrue(any("not CRF-equivalent" in n for n in doc["notes"]), doc.get("notes"))
+
+ def test_the_env_default_reaches_the_other_re_encoding_tools(self):
+ doc = self._tool("fit", self.src, "--height", "120", "--dry-run", "-o", self.work / "f.mp4", env={runner.HW_ENV: "1"})
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertEqual((doc["hw"]["requested"], doc["hw"]["source"], doc["hw"]["used"]), (True, "env", None))
+ self.assertTrue(any("--no-hw" in n for n in doc["hw"]["notes"]))
+ doc = self._tool("fit", self.src, "--height", "120", "--dry-run", "-o", self.work / "f.mp4")
+ self.assertEqual(doc["encoder"], "libx264", "never automatic: no flag, no variable, no GPU")
+ self.assertNotIn("hw", doc)
+
+ def test_a_batch_row_names_the_encoder_its_steps_ran(self):
+ """batch.py encodes nothing itself (its top-level hw.used is null); each item's row carries
+ the encoder its steps ran, and `hw` only when VideoToolbox was asked for."""
+ folder = self.work / "batch"
+ folder.mkdir()
+ (folder / "a.mp4").write_bytes(self.src.read_bytes())
+ recipe = folder / "batch.json"
+ recipe.write_text(json.dumps({"glob": "*.mp4", "output_dir": "out", "suffix": "_b",
+ "steps": [["fit.py", "{in}", "--height", "120", "-o", "{out}"]]}), encoding="utf-8")
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "batch.py"), str(folder), "--recipe", str(recipe), "--fast", "--json"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace")
+ self.assertEqual(proc.returncode, 0, proc.stderr)
+ row = json.loads(proc.stdout)["results"][0]
+ self.assertEqual(row["encoder"], "libx264")
+ self.assertNotIn("hw", row)
+
+ def _run_encode(self, fake):
+ """What a re-encoding tool does: the encoder line from video_args() (VideoToolbox under
+ --hw, the swap recorded), one ffmpeg command, run(), emit()."""
+ STATE.hw, STATE.hw_source, STATE.json = True, "flag", True
+ a, b = _vt_available()
+ with a, b:
+ video = decision.video_args(_common.probe(str(self.src)), 23, "ultrafast")
+ self.assertEqual(video[video.index("-c:v") + 1], "h264_videotoolbox")
+ out = self.work / "o.mp4"
+ cmd = [str(fake.path), "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-i", str(self.src), "-t", "0.5"] + video + ["-an", str(out)]
+ stdout, stderr = io.StringIO(), io.StringIO()
+ with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
+ try:
+ runner.run(cmd)
+ emit_module.emit(str(out))
+ code = 0
+ except SystemExit as e:
+ code = e.code
+ return code, json.loads(stdout.getvalue()), stderr.getvalue()
+
+ def test_a_videotoolbox_open_failure_is_re_encoded_on_the_cpu_and_reported(self):
+ fake = FakeFfmpeg(self.work, {"vt": {"rc": 187, "stderr": VT_OPEN_FAILED["6.1"]}, "cpu": {"real": self.real_ffmpeg}})
+ code, doc, _err = self._run_encode(fake)
+ self.assertEqual(code, 0, doc)
+ calls = fake.calls()
+ self.assertEqual(len(calls), 2, calls)
+ self.assertIn("h264_videotoolbox", calls[0])
+ self.assertIn("libx264", calls[1])
+ self.assertNotIn("h264_videotoolbox", calls[1])
+ self.assertTrue(doc["verified"], "verified is the probe of what the CPU wrote")
+ self.assertEqual(doc["probe"]["video"]["codec"], "h264")
+ self.assertEqual(doc["encoder"], "libx264")
+ self.assertIn("libx264", doc["commands"][-1])
+ self.assertEqual((doc["hw"]["used"], doc["hw"]["fallback"]), (False, True))
+ self.assertTrue(any("VideoToolbox refused" in n and "cannot create compression session" in n
+ for n in doc["hw"]["notes"]), doc["hw"])
+ self.assertFalse(any("CRF-equivalent" in n for n in doc.get("notes") or []), "the file is the CPU encode")
+
+ def test_a_failure_that_is_not_videotoolbox_s_is_not_retried_and_names_the_command_that_ran(self):
+ fake = FakeFfmpeg(self.work, {"vt": {"rc": 234, "stderr": NOT_VT["missing font (7.x: the encoder never opened)"]},
+ "cpu": {"real": self.real_ffmpeg}})
+ code, doc, _err = self._run_encode(fake)
+ self.assertEqual(code, 1)
+ self.assertEqual(len(fake.calls()), 1, "a missing font is not the GPU's: one run, no CPU retry")
+ self.assertEqual(doc["error"]["kind"], "ffmpeg")
+ self.assertIn("h264_videotoolbox", doc["error"]["message"])
+ self.assertIn("not retried on the CPU", doc["error"]["message"])
+ self.assertIn("Cannot find a valid font", doc["error"]["message"])
+ self.assertIn("h264_videotoolbox", doc["commands"][-1])
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertEqual((doc["hw"]["used"], doc["hw"]["fallback"]), (True, False))
+ self.assertFalse(any("VideoToolbox refused" in n for n in doc["hw"]["notes"]), doc["hw"])
+
+ def test_when_the_cpu_retry_fails_too_the_error_is_the_cpu_command_s(self):
+ fake = FakeFfmpeg(self.work, {"vt": {"rc": 187, "stderr": VT_OPEN_FAILED["5.1"]},
+ "cpu": {"rc": 1, "stderr": "[libx264 @ 0x55d] broken CPU encode\n"}})
+ code, doc, _err = self._run_encode(fake)
+ self.assertEqual(code, 1)
+ self.assertEqual(len(fake.calls()), 2)
+ message = doc["error"]["message"]
+ self.assertIn("libx264, the CPU retry after VideoToolbox refused the job", message)
+ self.assertIn("broken CPU encode", message)
+ self.assertNotIn("compression session", message, "the stderr is the command's that ran last")
+ self.assertIn("libx264", doc["commands"][-1])
+ self.assertEqual((doc["encoder"], doc["hw"]["used"], doc["hw"]["fallback"]), ("libx264", False, True))
+
+
+class HwResultTests(unittest.TestCase):
+ """What a result says about a GPU encode (review of #304: two encoders, one quality scale)."""
+
+ def tearDown(self):
+ STATE.reset()
+
+ def _doc(self, commands, hw=True):
+ STATE.reset()
+ STATE.hw, STATE.hw_source, STATE.json, STATE.dry_run = hw, ("flag" if hw else None), True, True
+ STATE.commands = list(commands)
+ out = io.StringIO()
+ with contextlib.redirect_stdout(out):
+ emit_module.emit("planned.mp4")
+ return json.loads(out.getvalue())
+
+ def test_a_gpu_encode_notes_that_its_quality_is_not_crf_equivalent(self):
+ doc = self._doc(["ffmpeg -i a.mp4 -c:v h264_videotoolbox -q:v 64 -pix_fmt yuv420p out.mp4"])
+ notes = [n for n in doc["notes"] if "not CRF-equivalent" in n]
+ self.assertEqual(len(notes), 1, doc["notes"])
+ self.assertIn("h264_videotoolbox -q:v 64", notes[0])
+ self.assertIn("--no-hw", notes[0])
+ doc = self._doc(["ffmpeg -i a.mov -c:v prores_videotoolbox -profile:v hq out.mov"])
+ self.assertTrue(any(n.startswith("prores_videotoolbox is not prores_ks") for n in doc["notes"]), doc["notes"])
+ for commands, hw in ((["ffmpeg -i a.mp4 -c:v libx264 -crf 23 out.mp4"], True),
+ (["ffmpeg -i a.mp4 -c:v libx264 -crf 23 out.mp4"], False)):
+ self.assertNotIn("notes", self._doc(commands, hw))
+
+ def test_verified_is_the_measured_output_whatever_the_encoder(self):
+ """`verified` is the probe (and the tool's own measured steps), never a property of the
+ encoder line: the same file verifies the same under a VideoToolbox command line."""
+ require_ffmpeg_or_skip("ffmpeg", "ffprobe")
+ with tempfile.TemporaryDirectory() as d:
+ out = Path(d) / "o.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=160x120:rate=30",
+ "-t", "0.3", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", out)
+ docs = []
+ for enc in ("h264_videotoolbox -q:v 64", "libx264 -crf 23"):
+ STATE.reset()
+ STATE.hw, STATE.hw_source, STATE.json = True, "flag", True
+ STATE.commands = [f"ffmpeg -i a.mp4 -c:v {enc} {out}"]
+ buf = io.StringIO()
+ with contextlib.redirect_stdout(buf):
+ emit_module.emit(str(out))
+ docs.append(json.loads(buf.getvalue()))
+ for doc in docs:
+ self.assertTrue(doc["verified"])
+ self.assertEqual(doc["verification"], [{"step": "probe", "ok": True}])
+ self.assertEqual(docs[0]["probe"], docs[1]["probe"])
+
+ def test_a_batch_item_s_steps_report_their_gpu_facts_once(self):
+ fit = {"encoder": "h264_videotoolbox", "commands": ["ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4"],
+ "hw": {"requested": True, "source": "env", "used": True, "fallback": False, "notes": [emit_module.ENV_HW_NOTE]}}
+ loud = {"encoder": "copy", "commands": ["ffmpeg -i b.mp4 -c:v copy -af loudnorm c.mp4"]}
+ rep = emit_module.steps_encoder_report([("fit", fit), ("loudness", loud)])
+ self.assertEqual(rep["encoder"], "h264_videotoolbox", "a later stream copy keeps the encode before it")
+ self.assertEqual({k: rep["hw"][k] for k in ("requested", "source", "used", "fallback")},
+ {"requested": True, "source": "env", "used": True, "fallback": False})
+ self.assertEqual(sum(n.endswith(emit_module.ENV_HW_NOTE) for n in rep["hw"]["notes"]), 1, rep["hw"]["notes"])
+ self.assertTrue(any("not CRF-equivalent" in n for n in rep["notes"]))
+ self.assertEqual(emit_module.steps_encoder_report([("fit", {"encoder": "libx264"})]), {"encoder": "libx264"})
+
+ def test_render_carries_a_stage_s_fallback_and_gpu_quality_note(self):
+ """render.py runs each stage as a child process and keeps only its command lines; the
+ stage's `hw` facts (fell back and why, ran on VideoToolbox) reach render's own result."""
+ STATE.reset()
+ STATE.json = True
+ fit = {"encoder": "libx264", "commands": ["ffmpeg -i a -c:v libx264 -crf 18 b.mp4"],
+ "hw": {"requested": True, "source": "env", "used": False, "fallback": True,
+ "notes": ["VideoToolbox refused the encode: [h264_videotoolbox] Error: cannot create compression session: -12902; re-encoded on the CPU"]}}
+ cap = {"encoder": "h264_videotoolbox", "commands": ["ffmpeg -i b.mp4 -c:v h264_videotoolbox -q:v 75 c.mp4"],
+ "hw": {"requested": True, "source": "env", "used": True, "fallback": False, "notes": [emit_module.ENV_HW_NOTE]}}
+ exp = {"encoder": "libx264", "commands": ["ffmpeg -i c.mp4 -c:v libx264 -crf 18 d.mp4"],
+ "hw": {"requested": False, "source": "env", "used": False, "fallback": False, "notes": [emit_module.ENV_NOT_FOR_DELIVERY_NOTE]}}
+ for stage, doc in (("fit", fit), ("caption", cap), ("export", exp)):
+ emit_module.absorb_stage_hw(stage, doc)
+ STATE.commands += doc["commands"]
+ out = io.StringIO()
+ with contextlib.redirect_stdout(out):
+ emit_module.emit(None) # a real run's document (no file here to probe)
+ doc = json.loads(out.getvalue())
+ self.assertEqual(doc["encoder"], "libx264")
+ self.assertEqual({k: doc["hw"][k] for k in ("requested", "source", "used", "fallback")},
+ {"requested": True, "source": "env", "used": False, "fallback": True})
+ self.assertTrue(any(n.startswith("fit: VideoToolbox refused") for n in doc["hw"]["notes"]), doc["hw"])
+ self.assertTrue(any(n.startswith("caption: ") and "FFMPEG_SKILL_HW=1" in n for n in doc["hw"]["notes"]))
+ self.assertTrue(any("h264_videotoolbox -q:v 75 is not CRF-equivalent" in n for n in doc["notes"]), doc.get("notes"))
+
+
+class TakeoverReviewTests(unittest.TestCase):
+ """Cases the reviews of the #304 takeover found, at the unit level."""
+
+ def setUp(self):
+ STATE.reset()
+
+ def tearDown(self):
+ STATE.reset()
+
+ def test_the_prores_preset_s_cpu_fallback_keeps_the_source_s_tags(self):
+ """export.py tags every fixed preset's CPU line BT.709 except prores, whose master keeps
+ the source's tags; the CPU line recorded for a refused VideoToolbox job must be the one
+ export would have written (it tagged an HLG master BT.709 and still verified)."""
+ STATE.hw = True
+ prores = ["-c:v", "prores_ks", "-profile:v", "3", "-vendor", "apl0", "-pix_fmt", "yuv422p10le"]
+ hdr = {"video": {"bt2020_or_hdr": True, "color_space": "bt2020nc", "color_primaries": "bt2020",
+ "color_transfer": "arib-std-b67"}}
+ a, b = _vt_available()
+ with a, b:
+ vt = decision.hw_preset_video(list(prores), hdr, bt709=False)
+ self.assertEqual(vt[vt.index("-c:v") + 1], "prores_videotoolbox")
+ self.assertEqual(STATE.hw_swaps[-1][1], prores)
+
+ def test_an_h265_preset_on_an_hdr_source_stays_the_preset_s_8_bit_bt709_format(self):
+ """The CPU h265 preset writes 8-bit with BT.709 tags whatever the source (and notes that
+ it did not convert HDR); the VideoToolbox line must write the same format, not the Main10
+ HLG line, so the GPU file and its CPU fallback are the same kind of file."""
+ STATE.hw = True
+ h265 = ["-c:v", "libx265", "-preset", "medium", "-crf", "24", "-pix_fmt", "yuv420p", "-tag:v", "hvc1"]
+ hdr = {"video": {"bt2020_or_hdr": True, "color_space": "bt2020nc", "color_primaries": "bt2020",
+ "color_transfer": "arib-std-b67"}}
+ a, b = _vt_available()
+ with a, b:
+ vt = decision.hw_preset_video(list(h265), hdr, bt709=True)
+ self.assertEqual(vt[vt.index("-c:v") + 1], "hevc_videotoolbox")
+ self.assertEqual(vt[vt.index("-pix_fmt") + 1], "yuv420p")
+ self.assertNotIn("main10", vt)
+ self.assertNotIn("arib-std-b67", vt)
+ self.assertIn("hevc_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1", vt)
+ self.assertEqual(vt[vt.index("-q:v") + 1], str(decision.vt_quality("hevc", 24)), "the SDR HEVC curve: x265 8-bit at CRF 24")
+
+ def test_export_s_in_place_movflags_edit_is_restated_in_the_recorded_swap(self):
+ """export.py deletes -movflags from encoder_args()'s list in place; the recorded VideoToolbox
+ slice is a copy, so restate_last_swap() sees the line as built and the CPU fallback line
+ loses its -movflags too (export adds +faststart once, after the encoder line)."""
+ STATE.hw = True
+ a, b = _vt_available()
+ with a, b:
+ video = decision.encoder_args("hevc", 20, "medium", {"video": {"bt2020_or_hdr": True, "color_transfer": "smpte2084"}})
+ as_built = list(video)
+ while "-movflags" in video: # export.py's own edit, in place
+ i = video.index("-movflags")
+ del video[i:i + 2]
+ decision.restate_last_swap(as_built, video)
+ vt, cpu = STATE.hw_swaps[-1]
+ self.assertEqual(vt, video)
+ self.assertNotIn("-movflags", cpu)
+ back = runner._hw_fallback(["ffmpeg", "-i", "in.mov"] + video + ["-movflags", "+faststart", "out.mp4"], STATE)
+ self.assertIn("libx265", back)
+ self.assertEqual(back.count("-movflags"), 1, back)
+
+ def test_an_odd_sized_job_videotoolbox_refused_gets_the_even_scale_on_the_cpu(self):
+ """VideoToolbox refused the job before anything reported the odd size; the CPU retry then
+ fails with x264's "not divisible by 2", and gets the even scale a CPU-only run gets."""
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["6.1"])
+ odd = subprocess.CompletedProcess([], 1, "", "[libx264 @ 0x1] width not divisible by 2 (321x241)\n")
+ ok = subprocess.CompletedProcess([], 0, "", "")
+ with tempfile.TemporaryDirectory() as d, \
+ mock.patch.object(runner, "_execute", side_effect=[refused, odd, ok]) as execute:
+ proc = runner.run(["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", str(Path(d) / "o.mp4")], quiet=True)
+ self.assertEqual(proc.returncode, 0)
+ self.assertEqual(execute.call_count, 3)
+ last = execute.call_args_list[2][0][0]
+ self.assertIn("libx264", last)
+ self.assertTrue(any(runner.EVEN_SCALE in a for a in last), last)
+ self.assertIn("libx264", STATE.commands[-1])
+ self.assertIn(runner.EVEN_SCALE, STATE.commands[-1])
+ self.assertTrue(STATE.hw_fallback)
+
+ def test_a_dry_run_reports_the_planned_encoder_and_no_used(self):
+ """`used` says what ran; under --dry-run nothing did, so it is null while `encoder` names
+ the planned encoder. The FFMPEG_SKILL_HW note still tells a planner what the GPU costs."""
+ STATE.hw, STATE.hw_source, STATE.dry_run = True, "env", True
+ STATE.commands = ["ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4"]
+ rep = emit_module._encoder_report(STATE)
+ self.assertEqual(rep["encoder"], "h264_videotoolbox")
+ self.assertIsNone(rep["hw"]["used"])
+ self.assertFalse(rep["hw"]["fallback"])
+ self.assertTrue(any(n.endswith(emit_module.ENV_HW_NOTE) for n in rep["hw"]["notes"]))
+ STATE.dry_run = False
+ self.assertTrue(emit_module._encoder_report(STATE)["hw"]["used"])
+ STATE.dry_run = True # batch.py's rows: the steps' documents, re-read
+ rows = emit_module.steps_encoder_report([("fit", {"encoder": "h264_videotoolbox", "hw": {
+ "requested": True, "source": "env", "used": None, "fallback": False, "notes": []}})])
+ self.assertIsNone(rows["hw"]["used"])
+
+ def test_an_export_only_row_names_the_variable_as_its_source(self):
+ """A batch row whose only step is export.py under FFMPEG_SKILL_HW=1 (requested false,
+ source env): the row's `source` is the stage's, not null."""
+ exp = {"encoder": "libx264", "hw": {"requested": False, "source": "env", "used": False, "fallback": False,
+ "notes": [emit_module.ENV_NOT_FOR_DELIVERY_NOTE]}}
+ rep = emit_module.steps_encoder_report([("export", exp)])
+ self.assertEqual((rep["hw"]["requested"], rep["hw"]["source"]), (False, "env"))
+
+ def test_a_tool_s_own_ffmpeg_failure_carries_encoder_and_hw(self):
+ """cut.py runs run(check=False) and dies itself (kind ffmpeg): under --hw that failure carries
+ `encoder` and `hw` like run()'s own, and without --hw it carries neither (unchanged)."""
+ for hw in (True, False):
+ with self.subTest(hw=hw):
+ STATE.reset()
+ STATE.hw, STATE.hw_source, STATE.json = hw, ("flag" if hw else None), True
+ STATE.commands = ["ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4" if hw else "ffmpeg -i a -c:v libx264 b.mp4"]
+ out = io.StringIO()
+ with contextlib.redirect_stdout(out), contextlib.redirect_stderr(io.StringIO()), \
+ self.assertRaises(SystemExit):
+ emit_module.die("ffmpeg failed:\nboom", kind="ffmpeg")
+ doc = json.loads(out.getvalue())
+ if hw:
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertEqual((doc["hw"]["requested"], doc["hw"]["used"]), (True, True))
+ else:
+ self.assertNotIn("encoder", doc)
+ self.assertNotIn("hw", doc)
+ STATE.reset()
+ STATE.hw, STATE.hw_source, STATE.json = True, "flag", True
+ out = io.StringIO()
+ with contextlib.redirect_stdout(out), contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit):
+ emit_module.die("bad input", kind="input")
+ self.assertNotIn("hw", json.loads(out.getvalue()), "only a kind: ffmpeg failure carries it")
+
+ def test_render_s_failed_stage_keeps_its_encoder_and_the_earlier_stages_fallback(self):
+ """render.py re-raises a failed stage: the stage's `encoder` and `hw`, and a fallback an
+ earlier stage reported, reach render's own failure document."""
+ import render
+ STATE.json = True
+ emit_module.absorb_stage_hw("fit", {"encoder": "libx264", "hw": {
+ "requested": True, "source": "flag", "used": False, "fallback": True,
+ "notes": ["VideoToolbox refused the encode: [h264_videotoolbox] Error: cannot create compression session: -12902; re-encoded on the CPU"]}})
+ failed = {"status": "failed", "exit_code": 1, "commands": ["ffmpeg -i b.mp4 -c:v h264_videotoolbox -q:v 75 c.mp4"],
+ "error": {"kind": "ffmpeg", "message": "command failed (1): ffmpeg (video encoder h264_videotoolbox; not retried on the CPU: ...)"},
+ "encoder": "h264_videotoolbox",
+ "hw": {"requested": True, "source": "flag", "used": True, "fallback": False, "notes": []}}
+ proc = subprocess.CompletedProcess([], 1, json.dumps(failed), "")
+ out = io.StringIO()
+ with mock.patch.object(render, "run_tool", return_value=proc), contextlib.redirect_stdout(out), \
+ contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit):
+ render.sh("caption.py", "b.mp4", "-o", "c.mp4")
+ doc = json.loads(out.getvalue())
+ self.assertEqual(doc["error"]["kind"], "ffmpeg")
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertEqual((doc["hw"]["requested"], doc["hw"]["used"], doc["hw"]["fallback"]), (True, True, True))
+ self.assertTrue(any(n.startswith("fit: VideoToolbox refused") for n in doc["hw"]["notes"]), doc["hw"])
+
+ def test_hw_runs_on_apple_silicon_hardware_under_a_rosetta_python(self):
+ """An x86_64 Python under Rosetta reports platform.machine() "x86_64" on an M-series Mac;
+ the hardware (sysctl hw.optional.arm64) decides, and doctor's platform_ok agrees."""
+ import _contract
+
+ def sysctl(argv, **_kw):
+ self.assertEqual(argv[1:], ["-n", "hw.optional.arm64"])
+ return subprocess.CompletedProcess(argv, 0, answer, "")
+ for answer, expected in (("1\n", None), ("", "VideoToolbox constant-quality encoding needs Apple Silicon")):
+ with self.subTest(sysctl=answer), \
+ mock.patch("platform.system", return_value="Darwin"), \
+ mock.patch("platform.machine", return_value="x86_64"), \
+ mock.patch.object(runner.subprocess, "run", side_effect=sysctl), \
+ mock.patch.object(runner, "_APPLE_SILICON", None):
+ self.assertEqual(runner.hw_platform_reason(), expected)
+ runner._APPLE_SILICON = None
+ self.assertEqual(_contract._hw_default()["platform_ok"], expected is None)
+ with mock.patch("platform.system", return_value="Linux"), mock.patch.object(runner, "_APPLE_SILICON", None):
+ self.assertEqual(runner.hw_platform_reason(), "VideoToolbox is macOS-only")
+
+
+class HwChildStageTests(_RealFfmpegCase):
+ """The tools that run other tools pass the GPU choice on and report what their stages did
+ (reviews of the #304 takeover). Real ffmpeg; VideoToolbox is stood in only in-process."""
+
+ def _recipe(self, folder):
+ folder.mkdir()
+ (folder / "a.mp4").write_bytes(self.src.read_bytes())
+ recipe = folder / "batch.json"
+ recipe.write_text(json.dumps({"glob": "*.mp4", "output_dir": "out", "suffix": "_b",
+ "steps": [["fit.py", "{in}", "--height", "120", "-o", "{out}"]]}), encoding="utf-8")
+ return recipe
+
+ def test_batch_never_serves_a_cached_item_across_the_gpu_setting(self):
+ """The item cache key folds in the GPU setting, as render.py's does: --no-hw after --hw
+ (the remedy an env-chosen GPU row prints) re-encodes; and a run that overwrote an output
+ drops the other setting's entry, so switching back re-encodes too."""
+ folder = self.work / "cache"
+ recipe = self._recipe(folder)
+
+ def run(*flags):
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "batch.py"), str(folder), "--recipe", str(recipe),
+ "--fast", "--overwrite", "--json", *flags],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace")
+ self.assertEqual(proc.returncode, 0, proc.stderr)
+ return json.loads(proc.stdout)
+ cached = [bool(run(*f)["results"][0].get("cached")) for f in (["--hw"], ["--no-hw"], ["--hw"], ["--hw"], [])]
+ self.assertEqual(cached, [False, False, False, True, False])
+
+ def test_batch_s_top_level_hw_carries_its_items(self):
+ """batch.py's own `hw` aggregates its rows: an item's fallback shows at the top level, and
+ FFMPEG_SKILL_HW=1 alone (no flag) still gives a top-level `hw`, with `used` null."""
+ import batch
+ folder = self.work / "agg"
+ recipe = self._recipe(folder)
+ step = {"status": "completed", "encoder": "libx264", "commands": ["ffmpeg -i a -c:v libx264 -crf 18 b.mp4"],
+ "hw": {"requested": True, "source": "flag", "used": False, "fallback": True,
+ "notes": ["VideoToolbox refused the encode: [h264_videotoolbox] Error: cannot create compression session: -12902; re-encoded on the CPU"]}}
+ env_step = {"status": "completed", "encoder": "h264_videotoolbox", "commands": ["ffmpeg -i a -c:v h264_videotoolbox -q:v 75 b.mp4"],
+ "hw": {"requested": True, "source": "env", "used": True, "fallback": False, "notes": [emit_module.ENV_HW_NOTE]}}
+ for argv, env, doc, expected in ((["--hw"], {}, step, {"requested": True, "source": "flag", "used": None, "fallback": True}),
+ ([], {runner.HW_ENV: "1"}, env_step, {"requested": True, "source": "env", "used": None, "fallback": False})):
+ with self.subTest(argv=argv, env=env):
+ out = io.StringIO()
+ with mock.patch.object(batch, "run_step", return_value=(True, doc)), mock.patch.dict(os.environ, env), \
+ mock.patch.object(sys, "argv", ["batch.py", str(folder), "--recipe", str(recipe), "--force", "--json"] + argv), \
+ contextlib.redirect_stdout(out), contextlib.redirect_stderr(io.StringIO()):
+ STATE.reset()
+ batch.main()
+ res = json.loads(out.getvalue())
+ self.assertEqual({k: res["hw"][k] for k in expected}, expected, res["hw"])
+ self.assertEqual(res["results"][0]["hw"]["fallback"], expected["fallback"])
+ if expected["fallback"]:
+ self.assertTrue(any(n.startswith("a.mp4: fit: VideoToolbox refused") for n in res["hw"]["notes"]), res["hw"])
+
+ def test_a_hw_plan_executed_by_render_reports_hw(self):
+ """`render.py plan.json` runs the planned tool as a child: its `hw` facts reach the result,
+ as a direct `fit.py --hw` reports them."""
+ plan = self.work / "plan.json"
+ out = self.work / "planned.mp4"
+ subprocess.run([sys.executable, str(SCRIPTS / "fit.py"), str(self.src), "--height", "120", "--hw", "--fast",
+ "--plan", str(plan), "-o", str(out)], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "render.py"), str(plan), "--json"],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace")
+ self.assertEqual(proc.returncode, 0, proc.stderr)
+ doc = json.loads(proc.stdout)
+ self.assertEqual((doc["hw"]["requested"], doc["hw"]["source"]), (True, "flag"), doc.get("hw"))
+ self.assertEqual(doc["encoder"], doc["tool_result"]["encoder"])
+ self.assertEqual(doc["hw"]["used"], doc["tool_result"]["hw"]["used"])
+
+ def test_a_cut_ffmpeg_failure_under_hw_names_its_encoder(self):
+ """cut.py --accurate under --hw: its own kind: ffmpeg failure carries `encoder` and `hw`."""
+ fake = FakeFfmpeg(self.work, {"vt": {"rc": 1, "stderr": "[enc @ 0x55d] broken encode\n"},
+ "cpu": {"rc": 1, "stderr": "[enc @ 0x55d] broken encode\n"}})
+ env = dict(os.environ, PATH=f"{fake.dir}{os.pathsep}{os.environ.get('PATH', '')}")
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "cut.py"), str(self.src), "--start", "0.1", "--end", "0.7",
+ "--accurate", "--hw", "--json", "-o", str(self.work / "c.mp4")], env=env,
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace")
+ self.assertEqual(proc.returncode, 1, proc.stderr)
+ doc = json.loads(proc.stdout)
+ self.assertEqual(doc["error"]["kind"], "ffmpeg")
+ self.assertIn(doc["encoder"], ("libx264", "h264_videotoolbox"))
+ self.assertEqual((doc["hw"]["requested"], doc["hw"]["source"]), (True, "flag"))
+
+ def _waveform(self, *argv, env=None):
+ """waveform.py in this process (its own encode sees VideoToolbox stood in), its caption.py
+ stage a real child: (the document, the child's argv, the child's own document)."""
+ import waveform
+ wav = self.work / "tone.wav"
+ if not wav.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "sine=f=440:d=2", "-c:a", "pcm_s16le", wav)
+ srt = self.work / "s.srt"
+ srt.write_text("1\n00:00:00,000 --> 00:00:01,500\nHello\n", encoding="utf-8")
+ seen = []
+
+ def spy(cmd, **kw):
+ proc = runner.run_tool(cmd, **kw)
+ seen.append((list(cmd), json.loads(proc.stdout)))
+ return proc
+ out = io.StringIO()
+ a, b = _vt_available()
+ with a, b, mock.patch.object(waveform, "run_tool", side_effect=spy), mock.patch.dict(os.environ, env or {}), \
+ mock.patch.object(sys, "argv", ["waveform.py", str(wav), "--width", "320", "--height", "180", "--srt", str(srt),
+ "--dry-run", "--json", "-o", str(self.work / "w.mp4")] + list(argv)), \
+ contextlib.redirect_stdout(out), contextlib.redirect_stderr(io.StringIO()):
+ STATE.reset()
+ waveform.main()
+ (child_argv, child_doc), = seen
+ return json.loads(out.getvalue()), child_argv, child_doc
+
+ def test_waveform_passes_its_gpu_choice_to_the_stage_that_writes_the_file(self):
+ """caption.py writes waveform.py's deliverable: --hw reaches it, and the result's
+ `encoder` is the encoder of the file delivered, not of the intermediate render."""
+ doc, child_argv, child_doc = self._waveform("--hw")
+ self.assertIn("--hw", child_argv)
+ self.assertEqual(child_doc["hw"]["requested"], True)
+ self.assertEqual(doc["encoder"], child_doc["encoder"])
+ self.assertEqual(doc["commands"][-1], child_doc["commands"][-1])
+
+ def test_waveform_s_no_hw_overrides_the_variable_for_its_stage_too(self):
+ doc, child_argv, child_doc = self._waveform("--no-hw", env={runner.HW_ENV: "1"})
+ self.assertIn("--no-hw", child_argv)
+ self.assertNotIn("hw", child_doc, "the caption stage ran on the CPU, as asked")
+ self.assertNotIn("hw", doc)
+ self.assertEqual(doc["encoder"], "libx264")
+
+
[email protected](_vt_here(), "VideoToolbox encoders need Apple Silicon and an ffmpeg that lists them")
+class VtEncodeTests(MediaFixtures):
+ """Real VideoToolbox encodes, probed."""
+
+ def test_hw_encode_reports_itself_and_keeps_an_untagged_source_unconverted(self):
+ src = OUT / "vt_untagged.mkv"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=s=640x360:d=1",
+ "-pix_fmt", "yuv420p", "-c:v", "libx264", "-crf", "0", src)
+ out = OUT / "vt_fit.mp4"
+ doc = json.loads(script("fit.py", src, "--height", "360", "--hw", "--json", "-o", out).stdout)
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertEqual(doc["hw"], {"requested": True, "source": "flag", "used": True, "fallback": False, "notes": []})
+ self.assertTrue(any("h264_videotoolbox -q:v" in n and "not CRF-equivalent" in n for n in doc["notes"]), doc.get("notes"))
+ v = doc["probe"]["video"]
+ self.assertEqual((v["color_space"], v["color_primaries"], v["color_transfer"]), ("bt709", "bt709", "bt709"))
+ neutral = "setparams=colorspace=unknown:color_primaries=unknown:color_trc=unknown"
+ proc = subprocess.run(["ffmpeg", "-hide_banner", "-i", str(out), "-i", str(src), "-lavfi",
+ f"[0:v]{neutral}[a];[1:v]{neutral}[b];[a][b]psnr", "-f", "null", "-"],
+ stderr=subprocess.PIPE, text=True)
+ psnr = float(proc.stderr.split("average:")[1].split()[0])
+ self.assertGreater(psnr, 40, "the BT.709 tag turned into a colour conversion (~24 dB)")
+
+ def test_hw_hdr10_side_data_survives(self):
+ """PQ + mastering-display + content-light metadata come through hevc_videotoolbox."""
+ src = OUT / "vt_hdr10_md.mp4"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=s=640x360:d=1",
+ "-c:v", "libx265", "-pix_fmt", "yuv420p10le", "-tag:v", "hvc1", "-x265-params",
+ "log-level=error:hdr10=1:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):max-cll=1000,400",
+ "-bsf:v", "hevc_metadata=colour_primaries=9:transfer_characteristics=16:matrix_coefficients=9", src)
+ out = OUT / "vt_hdr10_out.mp4"
+ doc = json.loads(script("fit.py", src, "--height", "360", "--hw", "--json", "-o", out).stdout)
+ self.assertEqual(doc["encoder"], "hevc_videotoolbox")
+ self.assertEqual(doc["probe"]["video"]["color_transfer"], "smpte2084")
+ side = sh("ffprobe", "-v", "error", "-select_streams", "v", "-read_intervals", "%+#1", "-show_frames",
+ "-show_entries", "frame_side_data=side_data_type", out).stdout
+ self.assertIn("Mastering display metadata", side)
+ self.assertIn("Content light level metadata", side)
+
+ def test_a_job_videotoolbox_refuses_falls_back_to_the_cpu_and_says_so(self):
+ """H.264 on VideoToolbox stops at 4096 wide; an 8K frame is re-encoded on x264, reported.
+ Half a second is enough for the real refusal (the full clip took ~26 s of x264 at 8K)."""
+ out = OUT / "vt_8k.mp4"
+ doc = json.loads(script("fit.py", self.src, "--duration", "0.5", "--method", "trim", "--width", "8192",
+ "--height", "4608", "--hw", "--fast", "--json", "-o", out).stdout)
+ self.assertEqual(doc["encoder"], "libx264")
+ self.assertFalse(doc["hw"]["used"])
+ self.assertTrue(doc["hw"]["fallback"])
+ self.assertTrue(any("VideoToolbox refused" in n for n in doc["hw"]["notes"]), doc["hw"])
+
+ def test_export_with_hw_really_runs_on_videotoolbox(self):
+ """An explicit --hw export really runs on VideoToolbox and keeps the preset's frame rate
+ (the env-default half of the decision is pinned on every runner by
+ HwEverywhereTests.test_export_preset_needs_an_explicit_hw)."""
+ env = dict(os.environ, FFMPEG_SKILL_HW="1")
+ doc = json.loads(script("export.py", self.src, "--preset", "x", "--dry-run", "--json",
+ "-o", OUT / "vt_x_env.mp4", env=env).stdout)
+ self.assertEqual(doc["encoder"], "libx264")
+ doc = json.loads(script("export.py", self.src, "--preset", "x", "--hw", "--json",
+ "-o", OUT / "vt_x_hw.mp4").stdout)
+ self.assertEqual(doc["encoder"], "h264_videotoolbox")
+ self.assertTrue(doc["hw"]["used"])
+ self.assertEqual(round(doc["probe"]["video"]["fps"]), 30)
+
+
+if __name__ == "__main__":
+ unittest.main(verbosity=2)
diff --git a/tests/test_all.py b/tests/test_all.py
index 6839778..6286273 100644
--- a/tests/test_all.py
+++ b/tests/test_all.py
@@ -4,8 +4,8 @@
python3 tests/test_all.py # or: python3 -m unittest tests/test_all.py
The cases themselves live one module per tool group -- test_analysis, test_editing, test_audio,
-test_picture, test_delivery, test_orchestration -- over the fixtures in _fixtures.py. This file
-imports all six so that the entry point every CI job, npm test and release_check.sh already runs
+test_picture, test_delivery, test_orchestration, test_accel -- over the fixtures in _fixtures.py. This file
+imports all seven so that the entry point every CI job, npm test and release_check.sh already runs
keeps running exactly the same set; each group module is also runnable on its own.
"""
import sys
@@ -14,6 +14,7 @@ from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
+import test_accel # noqa: E402
import test_analysis # noqa: E402
import test_audio # noqa: E402
import test_delivery # noqa: E402
@@ -21,11 +22,11 @@ import test_editing # noqa: E402
import test_orchestration # noqa: E402
import test_picture # noqa: E402
-MODULES = (test_analysis, test_editing, test_audio, test_picture, test_delivery, test_orchestration)
+MODULES = (test_analysis, test_editing, test_audio, test_picture, test_delivery, test_orchestration, test_accel)
def load_tests(loader, tests, pattern):
- """unittest calls this instead of scanning this module: the cases are in the six group
+ """unittest calls this instead of scanning this module: the cases are in the seven group
modules, and naming them here keeps the set (and the order groups run in) explicit."""
suite = unittest.TestSuite()
for module in MODULES:
diff --git a/tests/test_contract.py b/tests/test_contract.py
index 6ab9fc3..ad17b8d 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -25,6 +25,14 @@ CORPUS = ROOT / "tests" / "corpus"
sys.path.insert(0, str(SCRIPTS))
sys.path.insert(0, str(ROOT / "mcp"))
import _contract # noqa: E402
+from _common.runner import HW_ENV, HW_FORCED_ENV # noqa: E402
+
+# The host's own GPU default must not reach the suite (every tool subprocess inherits os.environ):
+# a machine with FFMPEG_SKILL_HW=1 would put every encode on VideoToolbox. Tests that exercise it
+# opt in with an explicit env. The names come from the runner, so this scrub and the one in
+# tests/_fixtures.py cannot drift from what the tools read.
+for _k in (HW_ENV, HW_FORCED_ENV):
+ os.environ.pop(_k, None)
import _common # noqa: E402
import server as mcp_server # noqa: E402
@@ -421,6 +429,65 @@ class ContractTests(unittest.TestCase):
self.assertEqual(self.tools["export"]["input_schema"]["properties"]["crf"]["type"], "integer")
self.assertEqual(self.tools["export"]["input_schema"]["properties"]["preset"]["enum"], sorted(self.tools["export"]["input_schema"]["properties"]["preset"]["enum"]))
+ def test_every_flag_of_every_parser_is_published_once_under_its_own_name(self):
+ """Review of #304: --hw (store_true) and --no-hw (store_false) once shared the dest `hw`.
+ input_schema keeps one property per dest, so the contract published only --no-hw, typed
+ string, under `hw`; mutually_exclusive read [["hw", "hw"]]; and an MCP `hw: true` sent
+ --no-hw. Every option string of every parser must sit in exactly one property's cli, and
+ the generator refuses a parser whose actions share a dest."""
+ import argparse
+ for name, spec in self.tools.items():
+ with self.subTest(tool=name):
+ parser = _contract._capture_parser(SCRIPTS / f"{name}.py")
+ published = [f for p in spec["input_schema"]["properties"].values() if p["cli"] != "positional" for f in p["cli"]]
+ for action in parser._actions:
+ for flag in action.option_strings:
+ if not isinstance(action, argparse._HelpAction):
+ self.assertEqual(published.count(flag), 1, f"{name}: {flag}")
+ for group in spec["input_schema"].get("mutually_exclusive", []):
+ self.assertEqual(len(group), len(set(group)), f"{name}: {group}")
+ ap = argparse.ArgumentParser(prog="t")
+ ap.add_argument("--x", dest="x", action="store_true")
+ ap.add_argument("--no-x", dest="x", action="store_false")
+ with self.assertRaisesRegex(RuntimeError, "shares dest 'x'"):
+ _contract.input_schema(ap)
+ for name in ("fit", "export", "render", "batch"):
+ props = self.tools[name]["input_schema"]["properties"]
+ self.assertEqual((props["hw"]["type"], props["hw"]["cli"]), ("boolean", ["--hw"]), name)
+ self.assertEqual((props["no_hw"]["type"], props["no_hw"]["cli"]), ("boolean", ["--no-hw"]), name)
+ self.assertIn(["hw", "no_hw"], self.tools[name]["input_schema"]["mutually_exclusive"])
+ on = mcp_server.build_argv("fit", {"input": "a.mp4", "hw": True})
+ self.assertIn("--hw", on)
+ self.assertNotIn("--no-hw", on)
+ self.assertIn("--no-hw", mcp_server.build_argv("fit", {"input": "a.mp4", "no_hw": True}))
+ self.assertNotIn("--hw", mcp_server.build_argv("render", {"project": "p.json", "no_hw": True}))
+ self.assertIn("--hw", mcp_server.build_argv("render", {"project": "p.json", "hw": True}))
+
+ def required_lists(node):
+ if isinstance(node, dict):
+ if isinstance(node.get("required"), list):
+ yield node["required"]
+ for v in node.values():
+ yield from required_lists(v)
+ elif isinstance(node, list):
+ for v in node:
+ yield from required_lists(v)
+ for t in _contract.mcp_tools():
+ for req in required_lists(t["inputSchema"]):
+ self.assertEqual(len(req), len(set(req)), f"{t['name']}: JSON Schema `required` items must be unique: {req}")
+
+ def test_a_conditional_notes_description_also_names_the_gpu_line(self):
+ """emit() adds the VideoToolbox quality line to `notes` on any tool that takes --hw, so a
+ tool whose `notes` description names its own condition (loop: "--boomerang ... the audio
+ was dropped") must not read as if that were the only one."""
+ for name, spec in self.tools.items():
+ notes = spec["output_schema"]["properties"].get("notes") or {}
+ if "hw" in spec["input_schema"]["properties"] and notes.get("description"):
+ with self.subTest(tool=name):
+ self.assertIn("VideoToolbox", notes["description"])
+ self.assertIn("VideoToolbox", self.tools["loop"]["output_schema"]["properties"]["notes"]["description"])
+ self.assertIn("VideoToolbox", self.contract["idempotency_hints"]["bit_exact"])
+
def test_response_schema(self):
for t in self.contract["tools"]:
schema = t["output_schema"]