Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

fix: line-break, speech-language and VideoToolbox fallback bugs (#315)

Shogo Kajiyama146dee9d13d4
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8e380fe..1ac9944 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,14 @@
## Unreleased
-(nothing yet)
+### Fixed
+
+- **Wrapped titles on Windows.** The text file drawtext reads is now written with bare line feeds. Text mode wrote each kept line break as CRLF there, and drawtext from FFmpeg 6.1 draws that as a blank line plus a line break, so a wrapped hook, title or meme caption came out at twice its height. Trailing newlines are dropped (drawtext ignores one and draws a second as a blank line), and every route normalises CRLF and CR to LF before it lays out a label.
+- **`graphics.py` lays out by line count.** The `title` template put its accent bar and subtitle for one line of text, so a two-line title was struck through and covered; `lower-third` printed a two-line name over its title; the `hook` band stayed at 30% of the short side whatever the wrapped title needed, so a 9:16 hook of four or more lines ran out of it onto the bare video. Each now follows the number of lines. A one-line label draws as before; the hook band grows only when the wrapped text needs more than the base band holds.
+- **`silence.py --filler --transcribe` uses the transcript's language.** With the default `--filler-lang auto` it always took the English list, so French or Japanese speech was transcribed correctly and its fillers stayed in the cut. The language the engine decoded picks the list: whisper.cpp's `result.language`, faster-whisper's detected language and openai-whisper's `language`, when nothing was named. A transcript in a language with no built-in list is refused (`kind: input`), as `--filler-lang` names it, with `--filler-words FILE` as the way forward; the English list is not applied to speech it does not know.
+- **Speech language handling.** `--language auto` is no language (it was treated as a non-English one, which skipped Parakeet or refused on a Parakeet-only machine); `en-US`, `eng` and `English` reach Whisper as `en`, and a regional code (`ja-JP`) as its primary language (`ja`). Quantised English-only models (`ggml-base.en-q5_1.bin`, `tiny.en-q8_0`) count as English-only for routing, so a multilingual Whisper is preferred on undetected speech. Known gap: with whisper.cpp as the only engine and an English-only model, undetected speech is still decoded as English and the run adds no note.
+- **A VideoToolbox refusal is reported by its outcome.** `hw.fallback` is true only when the CPU retry succeeded. A failure whose CPU retry also failed names it ("the CPU retry failed too") and leaves `fallback` false; a retry that times out keeps the neutral "retrying on the CPU encoder" line. A job of many encodes, or a process that ran the same refusal twice, names it once with its latest outcome.
+- **Contract and docs.** `doctor`'s plain-text summary no longer says no tool uses a GPU encoder; `output_schema` publishes `notes` (the VideoToolbox quality line) for every tool that takes `--hw`; the dry-run note for `silence.py` and `execution.model_downloads.when` say that `--filler --transcribe` runs the speech engine under `--dry-run`; `docs/contract.md` lists `loop.py`'s `boomerang`/`notes` and `cropdetect` as `environment_dependent`; the advice to pass `--hw` for an export is given only where `--hw` can run; the test-suite description counts all nine group modules; a test that needs no speech engine skips when any speech engine is installed.
## 2.8.0
diff --git a/README.md b/README.md
index cfe69c3..cae8a84 100644
--- a/README.md
+++ b/README.md
@@ -401,7 +401,7 @@ FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on th
**What is tested where.** The contract and the test suite (`tests/test_contract.py`,
`tests/test_all.py`, which aggregates one module per tool group — `test_analysis.py`,
`test_editing.py`, `test_audio.py`, `test_picture.py`, `test_delivery.py`,
-`test_orchestration.py` — over the shared footage in `tests/_fixtures.py`) run on Linux, macOS
+`test_orchestration.py`, `test_accel.py`, `test_asr.py`, `test_cut_copy.py` — over the shared footage in `tests/_fixtures.py`) run on Linux, macOS
and Windows on every pull request, minus the handful of POSIX-shim tests listed under
[Development](#development). The real-device media corpus
(`tests/corpus.py`) has been run on Linux and macOS; the full corpus has **not** been run on
diff --git a/docs/contract.md b/docs/contract.md
index 2a6ce2b..40ca5d5 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -247,7 +247,7 @@ runs the tool with the planned `argv`, then the verify steps (a `check` step may
### Repeatability
No tool keeps state or uses randomness. `deterministic_inputs` is `false` only for
-`verify`, whose output includes timings. `idempotency_hint` says what "same inputs"
+`verify` and `cropdetect`, whose output includes timings or machine state. `idempotency_hint` says what "same inputs"
gives you:
| Hint | Tools |
@@ -255,7 +255,7 @@ gives you:
| `bit_exact` | probe, check, scenes, look, metadata, background |
| `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
| `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
-| `environment_dependent` | verify |
+| `environment_dependent` | verify, cropdetect |
The hints describe the CPU encoders. Under `--hw`, or `FFMPEG_SKILL_HW=1` on the machine, a tool
that encodes video runs VideoToolbox, whose bytes are not promised identical from run to run:
@@ -568,12 +568,13 @@ Per-tool keys added after 2.2.2, all additive:
| key | tool | what it holds |
|---|---|---|
+| `boomerang`, `notes` | `loop.py` | `boomerang` is `true` for `--boomerang` (frames 0..N-1 then N-2..1, repeated, so neither turnaround frame is shown twice); `notes` says the audio was dropped when the source had any (a boomerang is silent) |
| `sdr_path`, `notes` | `color.py --to-sdr` | `sdr_path` is `"tonemap"` for PQ / HLG / Dolby Vision input (and `--force` on an untagged file) or `"gamut"` for BT.2020 primaries on an SDR transfer, which is converted to BT.709 without a tone map (`--tonemap`/`--peak`/`--desat` do not apply); `notes` says which path was taken and why |
| `silent` | `join.py` (every run) | `[{index, path, peak_db, at, end}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `at`/`end` are where the input sits in the output, in seconds (a transition overlaps its neighbours). An input named by `--allow-silent N[,N]` (1-based) is a planned pause: always joined, listed with `intended: true`, no warning or note, whatever `--on-silent` says. `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
| `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
| `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
| `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |
-| `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and `null` when the engine identified it itself; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |
+| `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and, when nothing was named, the language the engine decoded where it reports one (whisper.cpp's word-timing JSON, faster-whisper, openai-whisper's word-timing JSON), else `null`; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |
| `notes` | `silence.py --filler --transcribe` (new for `silence.py`; `caption.py` already had it) | present when `--engine auto` ran the English-only Parakeet model on English *assumed* (no language given or detectable, and no Whisper engine installed or none that succeeded): the transcript is wrong if the speech is not English. `caption.py --transcribe` puts the same line in its `notes` |
| `reencode_reason`, `segment_precision`, `least_exact_precision`, `start_snapped` | `cut.py` (every run) | `reencode_reason`: why anything was re-encoded, distinct values in first-seen order, `[]` for a clean copy: `requested` (`--accurate`), `codec` (`--codec`), `vfr`, `vfr_inconclusive`, `pcm_container`, `copy_failed`, `tolerance`, `concat_fallback` (a `--segments` join that could not stay a stream copy, which now also sets `reencoded: true` and `mode: hybrid`; it reported `copy` before). `segment_precision`: each `--segments` segment's precision (`null` for one segment); `least_exact_precision`: the least exact of them, on every run. `precision` and `keyframe_snapped` keep their meaning (`keyframe_snapped` is `precision == "packet"`, a stream copy end to end); `least_exact_precision` differs from `precision` only for an audio-only join of copied and re-encoded parts. `start_snapped`: measured, `true` when a copied segment's picture starts more than one frame from its requested start (or its start could not be measured: an unknown start is never claimed exact), `false` when every start is where asked or everything re-encoded, `null` under `--dry-run`. `requested_segments`/`requested_duration` keep their meaning, the request clamped to the media's duration: a segment ended with the video (less than a frame of sound trimmed) shows in `duration_delta_seconds` and a `notes` line |
| `edit_list`, `stored_preroll_seconds`, `av_start_skew_seconds`, `notes` | `cut.py` (every run) | `edit_list`: `true` only for a single-segment `.mp4`/`.m4v`/`.mov` stream copy run with `--edit-list`, which keeps the MP4 edit list the copy writes instead of `-avoid_negative_ts make_zero` (the default, unchanged): the keyframe's pre-roll is stored but hidden, so the picture starts at `--start`. `stored_preroll_seconds`: with `edit_list`, the seconds decoded from the keyframe the copy's seek lands on (measured with the seek ffmpeg makes) up to the first presented frame; `null` otherwise. Under `--edit-list` a copy is judged by its video's length (a `notes` line says so when the container is longer), and one that starts where asked but ends past `--tolerance` re-encodes (`tolerance`) without a `lossless_alternative`. `av_start_skew_seconds`: the output's audio start minus its video start (`null` when either is missing or under `--dry-run`); on a stream copy, when it is more than max(2 frames, 0.1 s) away from the source's own audio/video offset where the copy's picture starts (the keyframe it began from; `--start` under `--edit-list`), a `notes` line names `--accurate`, and `--edit-list` for a single `.mp4`/`.mov` copy whose start snapped. A re-encode carries no such note. The skew is reported, never repaired. `notes`: what the run did that a caller should know, in plain sentences (`[]` when nothing) |
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index aca277e..4f6f852 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -893,12 +893,19 @@ not a new file format this tool would have to maintain.
encoder but means it never got a frame) or a full disk is the same failure on the CPU: retrying
ran the job twice and reported the GPU as its cause. Those fail once, and the failure document
names the encoder of the command that ran, says it was not retried, and carries `encoder`/`hw`;
- when the CPU retry fails too, the message and stderr are the CPU command's. Code:
+ when the CPU retry fails too, the message and stderr are the CPU command's, and the note says
+ the retry failed too instead of claiming a re-encode. `hw.fallback` is true only when the CPU
+ retry succeeded (it means "re-encoded on the CPU"); a retry that times out keeps the neutral
+ "retrying on the CPU encoder" line. The note is one line per refusal, the latest outcome
+ (`_set_refusal_note`), so a job of many encodes names it once. Code:
`_common.runner.videotoolbox_failure`, `run`, `_fail`. Tests: `VideoToolboxFailureTests`,
`test_a_videotoolbox_open_failure_is_re_encoded_on_the_cpu_and_reported`,
`test_a_failure_that_is_not_videotoolbox_s_is_not_retried_and_names_the_command_that_ran`,
- `test_when_the_cpu_retry_fails_too_the_error_is_the_cpu_command_s` (a fake ffmpeg, every CI
- runner); `test_a_job_videotoolbox_refuses_falls_back_to_the_cpu_and_says_so` (real hardware).
+ `test_when_the_cpu_retry_fails_too_the_error_is_the_cpu_command_s`,
+ `test_a_failed_cpu_retry_does_not_claim_a_re_encode_and_a_repeat_refusal_is_noted_once`,
+ `test_a_cpu_retry_that_times_out_leaves_no_claim_that_it_re_encoded` (fake ffmpeg, every CI runner);
+ `test_a_job_videotoolbox_refuses_falls_back_to_the_cpu_and_says_so` (real hardware, confirmed on an
+ Apple M3 with FFmpeg 9.0.2: 54 tests OK, 0 skipped).
The CPU retry gets the odd-dimension retry a CPU-only run gets
(`test_an_odd_sized_job_videotoolbox_refused_gets_the_even_scale_on_the_cpu`), and on 5.1 the
encoder's non-fatal property lines do not count as its refusal beside another stream's open
@@ -963,7 +970,9 @@ not a new file format this tool would have to maintain.
that reads `verified`. "Goes first" means a Whisper engine that detects the language: whisper.cpp
is passed `-l auto` when nothing named the language (its own default is `-l en`, which decoded
every unnamed language as English, in 2.5.1 too), else the named or detected one, and an
- English-only Whisper model (`base.en`, `ggml-*.en.bin`) does not count as one.
+ English-only Whisper model (`base.en`, `ggml-*.en.bin`, quantised too: `ggml-base.en-q5_1.bin`) does
+ not count as one for routing. Known gap: with whisper.cpp as the only engine and such a model, undetected
+ speech is decoded as English and the run adds no note.
A named `--engine parakeet-*` runs as asked, with no note, and so does a Parakeet model named
with `--model` or `PARAKEET_MODEL`: a multilingual (v3) one assumes nothing and gets no note; an
English-only one named with `--model` gets the note, pointing at a v3 model rather than at a
diff --git a/references/scripts.md b/references/scripts.md
index 093df35..e29ebde 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -617,7 +617,7 @@ Three refusals, all `kind: input`, all before any encode:
Results: `filler.removed_count`, `filler.removed_seconds`, `filler.removed`
(one entry per span), `filler.lang`, `filler.list`, `filler.word_timings` and
-`filler.warnings`. With `--transcribe`, `filler.source` is `whisper:ENGINE` or
+`filler.warnings`; `--filler-lang auto` refuses (`kind: input`) a transcript in a language with no built-in list, so pass `--filler-words FILE` for it. With `--transcribe`, `filler.source` is `whisper:ENGINE` or
`parakeet:ENGINE` and `filler.transcription` says which engine, model, language
and routing made the words; a top-level `notes` line says when `--engine auto`
ran the English-only Parakeet model on English assumed (no `--filler-lang`, and
diff --git a/scripts/_common/asr.py b/scripts/_common/asr.py
index c6fc54f..9812ef0 100644
--- a/scripts/_common/asr.py
+++ b/scripts/_common/asr.py
@@ -301,6 +301,7 @@ def _skipped_parakeet(route: "Route", language: Optional[str], language_flag: st
def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int,
ffmpeg: str, shutil, subprocess, wanted: str = "auto") -> Transcription:
from _common import run_analysis, STATE
+ language = engine_language(language)
wav = os.path.join(tmpdir, "audio.wav")
# A wav in our own temp dir: a measurement input for the engine, not a deliverable, so it
# is not a run() call (no --dry-run gate, not recorded), but it keeps the time limit and
@@ -360,12 +361,14 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
import threading
result: list = []
crashed: list = []
+ heard: list = [] # the language faster-whisper decoded, when it names one
def work() -> None:
try:
m = WhisperModel(model, device="cpu", compute_type="int8")
- segments, _ = m.transcribe(wav, language=lang, word_timestamps=False)
+ segments, decoded = m.transcribe(wav, language=lang, word_timestamps=False)
result.extend((seg.start, seg.end, seg.text.strip()) for seg in segments if seg.text.strip())
+ heard.append(getattr(decoded, "language", None))
except Exception as exc: # noqa: BLE001 -- a model that would not load or run: a failed engine
crashed.append(exc)
@@ -388,7 +391,8 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
info("transcribed with faster-whisper")
write_srt(cues, out_srt)
return Transcription(cues=cues, engine="faster-whisper",
- facts=dict(facts, engine="faster-whisper", model=model, language=lang))
+ facts=dict(facts, engine="faster-whisper", model=model,
+ language=lang or _heard_language(heard)))
except ImportError:
pass
# 3. openai-whisper CLI
@@ -508,6 +512,23 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
return [w for w in out if w["word"].strip()]
+def _whisper_json_language(path: str) -> Optional[str]:
+ """The language a Whisper engine's JSON says it decoded: whisper.cpp's `result.language`, or
+ openai-whisper's top-level `language`. None when the file has neither."""
+ try:
+ doc = json.loads(Path(path).read_text(encoding="utf-8")) or {}
+ lang = str(((doc.get("result") or {}).get("language")) or doc.get("language") or "").strip().lower()
+ except (OSError, ValueError, AttributeError):
+ return None
+ return lang or None
+
+
+def _heard_language(heard: list) -> Optional[str]:
+ """The language an in-process engine decoded (faster-whisper's info.language), lower-cased, or None."""
+ lang = next((str(h).strip().lower() for h in heard if h), "")
+ return lang or None
+
+
def _words_from_openai_whisper_json(path: str) -> "List[Dict[str, Any]]":
"""openai-whisper --word_timestamps True --output_format json: segments[].words[]."""
try:
@@ -644,6 +665,22 @@ def _is_english(language: Optional[str]) -> bool:
return (language or "").lower().split("-")[0].split("_")[0] in ("en", "eng", "english")
+def engine_language(language: Optional[str]) -> Optional[str]:
+ """`--language` as the engines take it. "auto" (caption.py's documented default, spelled out)
+ is no language: None, the undetected route. Every spelling of English (en-US, eng, English)
+ is "en", which Parakeet accepts as English and Whisper accepts as a language code; any other
+ value is its two-letter primary language (ja-JP -> ja)."""
+ named = (language or "").strip()
+ if not named or named.lower() == "auto":
+ return None
+ if _is_english(named):
+ return "en"
+ # Whisper takes the two-letter code: a regional or script subtag (ja-JP, pt-BR, zh-Hant) names the
+ # same language to it, and an ISO 639-2 code (jpn) is passed as given
+ primary = named.replace("_", "-").split("-")[0].lower()
+ return primary if len(primary) == 2 else named
+
+
def _whisper_cpp_cli(shutil) -> Optional[str]:
cli = shutil.which("whisper-cli") or shutil.which("whisper-cpp")
if not cli:
@@ -675,7 +712,7 @@ def detect_language(wav: str, shutil, subprocess) -> Optional[str]:
if not cli:
return None
model = next((p for p in (_whisper_cpp_model(m) for m in ("tiny", "base", "small", "tiny.en"))
- if os.path.exists(p) and not p.endswith(".en.bin")), None)
+ if os.path.exists(p) and not _english_only_whisper_model(p)), None)
if not model:
return None
proc = _asr_run([cli, "-m", model, "-f", wav, "-dl", "-l", "auto"], subprocess, "whisper.cpp language detection")
@@ -756,9 +793,10 @@ def _names_parakeet_model(name: str) -> bool:
def _english_only_whisper_model(name: str) -> bool:
- """An English-only Whisper model: tiny.en, base.en, ... or a ggml-*.en.bin file."""
+ """An English-only Whisper model: tiny.en, base.en, ... or a ggml-*.en.bin file, in any
+ quantisation (ggml-base.en-q5_1.bin, tiny.en-q8_0) or variant (ggml-small.en-tdrz.bin)."""
base = os.path.basename(name).lower()
- return (base[:-4] if base.endswith(".bin") else base).endswith(".en")
+ return re.search(r"\.en(-[a-z0-9_]+)*$", base[:-4] if base.endswith(".bin") else base) is not None
def _whisper_ready(shutil, model: Optional[str]) -> bool:
@@ -913,6 +951,7 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
from _common import require_tool, run_analysis, STATE
ffmpeg = require_tool("ffmpeg")
wanted = requested_engine(engine)
+ language = engine_language(language)
parakeet_model = None if model == "base" else model
tmpdir = _tempfile.mkdtemp(prefix="ffskill_asrw_")
try:
@@ -962,7 +1001,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
words = _words_from_whisper_cpp_json(base + ".json")
info(f"word timings from whisper.cpp ({len(words)} words)")
return Transcription(words=words, engine="whisper.cpp",
- facts=dict(facts, engine="whisper.cpp", model=model_path, language=lang))
+ facts=dict(facts, engine="whisper.cpp", model=model_path,
+ language=lang or _whisper_json_language(base + ".json")))
line = _failure_line(proc)
info("whisper.cpp found but produced no word-timing JSON: " + line)
tried.append(_failed("whisper.cpp", "found but produced no word-timing JSON: " + line))
@@ -976,11 +1016,13 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
import threading
collected: list = []
crashed: list = []
+ heard: list = []
def work() -> None:
try:
m = WhisperModel(model, device="cpu", compute_type="int8")
- segments, _ = m.transcribe(wav, language=lang, word_timestamps=True)
+ segments, decoded = m.transcribe(wav, language=lang, word_timestamps=True)
+ heard.append(getattr(decoded, "language", None))
for seg in segments:
for w in (getattr(seg, "words", None) or []):
collected.append({"word": str(w.word).strip(),
@@ -1002,7 +1044,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
else:
info(f"word timings from faster-whisper ({len(collected)} words)")
return Transcription(words=[w for w in collected if w["word"]], engine="faster-whisper",
- facts=dict(facts, engine="faster-whisper", model=model, language=lang))
+ facts=dict(facts, engine="faster-whisper", model=model,
+ language=lang or _heard_language(heard)))
except ImportError:
pass
@@ -1018,7 +1061,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
words = _words_from_openai_whisper_json(doc)
info(f"word timings from openai-whisper ({len(words)} words)")
return Transcription(words=words, engine="openai-whisper",
- facts=dict(facts, engine="openai-whisper", model=model, language=lang))
+ facts=dict(facts, engine="openai-whisper", model=model,
+ language=lang or _whisper_json_language(doc)))
line = _failure_line(proc)
info("openai-whisper found but produced no word-timing JSON: " + line)
tried.append(_failed("openai-whisper", "found but produced no word-timing JSON: " + line))
diff --git a/scripts/_common/drawtext.py b/scripts/_common/drawtext.py
index 0953787..925b1c0 100644
--- a/scripts/_common/drawtext.py
+++ b/scripts/_common/drawtext.py
@@ -115,7 +115,7 @@ def drawtext_text_opts(text: str, tmpdir: "Optional[str]" = None) -> str:
`]`, `;` and `\` all reach the picture verbatim -- 1.15 fixes `overlay.py --text "it's 100%
done"` losing both characters. A newline is kept (`\r\n` and `\r` become `\n`): drawtext
draws it as a line break, and graphics.py's wrap puts them there on purpose (stripping them
- ran a wrapped title off both edges of the frame). A tab, vertical tab or form
+ ran a wrapped title off both edges of the frame); trailing newlines are dropped. A tab, vertical tab or form
feed becomes a space, as the wrap measures it; other control characters are stripped (that
dates from the inline-escape route, where they broke the graph parser).
@@ -127,6 +127,9 @@ def drawtext_text_opts(text: str, tmpdir: "Optional[str]" = None) -> str:
"""
cleaned = (text or "").replace("\r\n", "\n").replace("\r", "\n")
cleaned = re.sub(r"[\x00-\x08\x0e-\x1f\x7f]", "", re.sub(r"[\t\x0b\x0c]", " ", cleaned))
+ # drawtext ignores one trailing newline and draws a second as a blank line, which would
+ # centre the block half a line too high; a label that is only newlines stays as it is
+ cleaned = cleaned.rstrip("\n") or cleaned
import hashlib
name = "t_" + hashlib.sha256(cleaned.encode("utf-8")).hexdigest()[:16] + ".txt"
if tmpdir is None:
diff --git a/scripts/_common/emit.py b/scripts/_common/emit.py
index bba88c2..81baa9e 100644
--- a/scripts/_common/emit.py
+++ b/scripts/_common/emit.py
@@ -73,8 +73,17 @@ def _set_current_ctx(ctx: "Context") -> None:
ENV_HW_NOTE = ("VideoToolbox chosen by FFMPEG_SKILL_HW=1 (machine default; ~1.1-3.5x the bytes of x264/x265 "
"at matched quality); rerun with --no-hw for a final deliverable")
-ENV_NOT_FOR_DELIVERY_NOTE = ("FFMPEG_SKILL_HW=1 is not applied to export.py's delivery presets (a delivered file is where "
- "VideoToolbox's larger files cost most); pass --hw to put this export on the GPU")
+ENV_NOT_FOR_DELIVERY_BASE = ("FFMPEG_SKILL_HW=1 is not applied to export.py's delivery presets (a delivered file is where "
+ "VideoToolbox's larger files cost most)")
+ENV_NOT_FOR_DELIVERY_NOTE = ENV_NOT_FOR_DELIVERY_BASE + "; pass --hw to put this export on the GPU"
+
+
+def env_not_for_delivery_note() -> str:
+ """The note for an export the variable left on the CPU: the advice to pass --hw only on a
+ machine that can take it (the same advice on a Linux box would send the caller to a flag that
+ then reports the CPU encode)."""
+ from _common import decision # the same name the --hw decisions (and their tests) stand the platform in on
+ return ENV_NOT_FOR_DELIVERY_NOTE if decision.hw_platform_reason() is None else ENV_NOT_FOR_DELIVERY_BASE
def _video_encoders(commands: Sequence[str]) -> "Tuple[Optional[str], Optional[str]]":
@@ -120,7 +129,7 @@ def hw_report(ctx: "Context", enc: Optional[str]) -> Dict[str, Any]:
# (once: a stage that ran on it already said so)
notes.append(ENV_HW_NOTE)
if ctx.hw_env_ignored and enc not in (None, "copy"):
- notes.append(ENV_NOT_FOR_DELIVERY_NOTE)
+ notes.append(env_not_for_delivery_note())
return {"requested": requested, "source": source, "used": used,
"fallback": bool(ctx.hw_fallback) or any(st.get("fallback") for st in stages), "notes": notes}
diff --git a/scripts/_common/runner.py b/scripts/_common/runner.py
index 3862f53..6dcc8ee 100644
--- a/scripts/_common/runner.py
+++ b/scripts/_common/runner.py
@@ -885,9 +885,10 @@ def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True, ctx: "Op
why = videotoolbox_failure(proc.stderr or "")
if why is not None:
reason = f"VideoToolbox refused the encode: {why}"
+ # said before the retry, without a claim about its outcome: a retry that never
+ # returns (a timeout exits from inside _execute) leaves this line, which is true
+ _set_refusal_note(ctx, reason, "; retrying on the CPU encoder")
info(reason + "; retrying on the CPU encoder")
- ctx.hw_notes.append(reason + "; re-encoded on the CPU")
- ctx.hw_fallback = True
ctx.commands[-1] = _cmdline(cpu_cmd[:-1] + [cmd[-1]])
if not quiet:
# echoed like the first attempt, so a caller that reads the "$ " lines
@@ -895,6 +896,11 @@ def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True, ctx: "Op
info("$ " + ctx.commands[-1], ctx=ctx)
proc, tried = even_retry(_execute(cpu_cmd), cpu_cmd)
retried = True
+ # the outcome replaces the line above: a failure document must not claim a
+ # re-encode that did not happen, and hw.fallback is the same claim as a flag
+ ok = proc.returncode == 0
+ _set_refusal_note(ctx, reason, "; re-encoded on the CPU" if ok else "; the CPU retry failed too")
+ ctx.hw_fallback = ctx.hw_fallback or ok
if proc.returncode != 0 and check:
_fail(tried, proc.returncode, proc.stderr or "", ctx=ctx, retried=retried)
if final and tmp:
@@ -910,6 +916,14 @@ def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True, ctx: "Op
return proc
+def _set_refusal_note(ctx: "Context", reason: str, outcome: str) -> None:
+ """One note per refusal: the latest outcome for `reason` replaces any earlier line for it, so a
+ job of many encodes names it once, and a process that ran an earlier job with the same refusal
+ does not keep that job's outcome beside this one."""
+ ctx.hw_notes[:] = [n for n in ctx.hw_notes if not n.startswith(reason)]
+ ctx.hw_notes.append(reason + outcome)
+
+
def _execute(exec_cmd: List[str]) -> subprocess.CompletedProcess:
"""One attempt, never exiting on failure (run() decides after its retries)."""
if STATE.progress and _is_ffmpeg(exec_cmd) and exec_cmd[-1] != "-":
@@ -1257,7 +1271,9 @@ def flush_drawtext_textfiles(cmd: "Sequence[str]") -> "List[str]":
# symlink guard is belt and braces there and unavailable on Windows
flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC | getattr(os, "O_NOFOLLOW", 0)
fd = os.open(path, flags, 0o600)
- with os.fdopen(fd, "w", encoding="utf-8") as fh:
+ # newline="\n": text mode would write each kept line break as CRLF on Windows, and
+ # drawtext from FFmpeg 6.1 draws that as a blank line plus a line break
+ with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as fh:
fh.write(body)
written.append(path)
return written
diff --git a/scripts/_contract.py b/scripts/_contract.py
index 4a4c33c..22727c6 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -235,7 +235,7 @@ DRY_RUN_ANALYSIS = {
"scenes": "scene and audio-peak measurement runs; --sheet and --edl are not written",
"report": "probe, loudness and contact-sheet measurements run; the HTML is not written",
"cropdetect": "the cropdetect filter runs over the sampled windows to measure bars; this tool never writes a file regardless of --dry-run",
- "silence": "silencedetect runs so the reported silences and keep ranges are real; the cut output is not written",
+ "silence": "silencedetect runs so the reported silences and keep ranges are real; the cut output is not written. --filler --transcribe also runs the speech engine (its word timings are what the filler spans are planned on), which may download a model on first use",
"loudness": "the loudnorm measurement pass runs so input_i and the planned pass-2 command are real; the normalised output is not written",
"check": "read-only tool; the loudness measurement (or, when it is skipped, the audio row's volumedetect) and the --content decode run under --dry-run too, so every row is present",
"stabilize": "vidstabdetect (pass 1, into a temp file) runs; the stabilised output (pass 2) is not written",
@@ -518,10 +518,15 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
props = dict(base)
props.update(extra)
notes = props.get("notes")
- if hw and isinstance(notes, dict) and notes.get("description"):
- # a tool-specific condition is not the only one: emit() adds the GPU line to any tool
- props["notes"] = dict(notes, description=notes["description"] + "; also, on a VideoToolbox encode (--hw / "
- "$FFMPEG_SKILL_HW), a line that its quality is not CRF-equivalent (see hw)")
+ if hw:
+ # emit() adds the VideoToolbox quality line to `notes` on every tool that takes --hw, so every
+ # such tool publishes `notes`, and a tool-specific condition (loop's "--boomerang ... the audio
+ # was dropped", export's own lines) is not the only one it can carry
+ gpu = "a line that its quality is not CRF-equivalent (on a VideoToolbox encode: --hw / $FFMPEG_SKILL_HW; see hw)"
+ if isinstance(notes, dict) and notes.get("description"):
+ props["notes"] = dict(notes, description=notes["description"] + "; also " + gpu)
+ else:
+ props["notes"] = dict(notes or {"type": "array", "items": {"type": "string"}}, description=gpu[0].upper() + gpu[1:])
required = ["status", "output", "dry_run", "commands"]
return {"type": "object", "properties": props, "required": required, "additionalProperties": True}
@@ -840,8 +845,9 @@ def doctor() -> Dict[str, Any]:
`gpu_encoders` is a separate, honest answer to a question none of the required/optional
capabilities above ask: which GPU-backed encoders (nvenc, videotoolbox, qsv, vaapi, amf) this
- ffmpeg BUILD carries, from `-encoders` alone. No tool here requires or uses one -- every tool
- still assumes CPU x264/x265 -- so `gpu_encoders` never affects `ok` or any tool's `usable`. It
+ ffmpeg BUILD carries, from `-encoders` alone. No tool requires one -- every tool runs on CPU
+ x264/x265 unless `--hw` / FFMPEG_SKILL_HW=1 puts it on VideoToolbox (Apple Silicon only) -- so
+ `gpu_encoders` never affects `ok` or any tool's `usable`. It
only proves the build shipped the capability, never that the GPU/driver on this machine will
actually accept a job (that needs a real encode, which this introspection never runs).
@@ -1439,7 +1445,7 @@ def build(detect: bool = True) -> Dict[str, Any]:
# first time it runs; faster-whisper is a library, so its fetch is made from the skill's own
# process. A caller that sandboxes the network must know that before a first --transcribe.
"model_downloads": {
- "when": "first use of a speech engine (--transcribe) whose model is not cached yet; media is never uploaded",
+ "when": "first use of a speech engine (--transcribe; silence.py --filler --transcribe runs it under --dry-run too, caption.py --transcribe does not) whose model is not cached yet; media is never uploaded",
"in_process": {"faster-whisper": "Hugging Face (Systran/faster-whisper-<model>)"},
"child_process": {"parakeet-mlx": "Hugging Face (mlx-community/parakeet-*)",
"openai-whisper": "OpenAI (its model checkpoint)"},
@@ -1513,7 +1519,7 @@ def main() -> int:
print(f"note: overall 'ok' means nothing REQUIRED BY EVERY TOOL is missing -- {len(not_usable)} tool(s) still can't run today: {', '.join(not_usable)} (see doctor --json .tools for why)")
gpu = d["gpu_encoders"]
if gpu["status"] == "parsed":
- print(f"GPU-backed encoders in this build: {len(gpu['present'])} (no tool here uses one; names in doctor --json)")
+ print(f"GPU-backed encoders in this build: {len(gpu['present'])} (only --hw / FFMPEG_SKILL_HW=1 use one, VideoToolbox on Apple Silicon; names in doctor --json)")
print(_fonts_summary_line(d["fonts"]))
print("full detail: doctor --json (capability lists, per-tool `usable`, fix hints)")
for err in d["errors"]:
diff --git a/scripts/graphics.py b/scripts/graphics.py
index 1c99296..ae4bdfe 100755
--- a/scripts/graphics.py
+++ b/scripts/graphics.py
@@ -125,6 +125,10 @@ def main() -> int:
for _attr in ("name", "title", "subtitle", "text", "top", "bottom"):
_v = getattr(args, _attr, None)
if isinstance(_v, str):
+ # one line-break form for every route: libass draws a bare CR as nothing, drawtext
+ # ignores one trailing newline, and lines_of() counts the rest -- so the layout and the
+ # drawn lines agree only if the text is normalised here, once
+ _v = _v.replace("\r\n", "\n").replace("\r", "\n").rstrip("\n")
setattr(args, _attr, _v if _v.strip() else None)
args.platform = resolve_platform(args.platform)
@@ -295,6 +299,15 @@ def main() -> int:
slice_overlong=True, sliced=sliced_atoms) if para.strip() else [para])
return "\n".join(out)
+ def lines_of(text):
+ """How many lines drawtext and libass draw for `text` (a trailing newline is not one)."""
+ return max(1, len(str(text or "").replace("\r\n", "\n").replace("\r", "\n").rstrip("\n").split("\n")))
+
+ def block_h(text, size_px):
+ """The height the layouts below give a label: `size_px` for the first line, as they always
+ have, and a line pitch of 1.2 x the size for each further one."""
+ return size_px + (lines_of(text) - 1) * int(size_px * 1.2)
+
def add_text(text, drawtext, *, target=None, **el):
"""One line of template text: a drawtext filter on the old route, an ASS element on the
new one. The geometry is computed identically either way."""
@@ -314,7 +327,8 @@ def main() -> int:
h1 = int(base * 0.055)
h2 = int(base * 0.038)
pad = int(base * 0.02)
- bar_h = h1 + (h2 + pad if args.title else 0) + pad * 2
+ name_h = block_h(args.name, h1)
+ bar_h = name_h + (block_h(args.title, h2) + pad if args.title else 0) + pad * 2
bar_w = int(base * 0.62)
y0 = H - m_bottom - bar_h
# slide in from the left over 0.4 s, slide out over 0.3 s (overlay evaluates x per frame)
@@ -328,7 +342,7 @@ def main() -> int:
x_rest, x_off = m_left + tx_pad, -bar_w + tx_pad
draws = []
for text, fs, colour, ty in ((args.name, h1, text_c, y0 + pad),
- (args.title, h2, primary, y0 + pad + h1 + pad // 2)):
+ (args.title, h2, primary, y0 + pad + name_h + pad // 2)):
if not text:
continue
draws.append(f"drawtext={drawtext_text_opts(text)}:{fo}:fontsize={fs}:"
@@ -355,17 +369,23 @@ def main() -> int:
h1 = int(base * 0.11)
h2 = int(base * 0.045)
filters.append(f"drawbox=x=0:y=0:w=iw:h=ih:color={ff_color(bg, 0.55)}:t=fill:{en}")
+ # the title is centred as a block, so the bar and the subtitle sit below its last line:
+ # `title_h` is the block's height, as the one-line layout has always taken h1; a further
+ # subtitle line hangs below the first, whose position the one-line layout fixes
+ title_h = block_h(args.title, h1)
+ sub_y = title_h // 2 + int(base * 0.03)
+ sub_extra = block_h(args.subtitle, h2) - h2 if args.subtitle else 0
add_text(args.title,
f"drawtext={drawtext_text_opts(args.title)}:{fo}:fontsize={h1}:fontcolor={ff_color(text_c)}{drawtext_center_align()}:x=(w-text_w)/2:y=(h-text_h)/2-{h2 if args.subtitle else 0}:alpha='{fade_a}':{en}",
size=h1, color=text_c, font=ass_font_family(), align=5, x=W / 2,
y=H / 2 - (h2 if args.subtitle else 0), outline=max(1.0, h1 / 20.0),
outline_color="000000", start=s, end=e, fade=(300, 300))
- filters.append(f"drawbox=x=(iw-{int(base * 0.12)})/2:y=(ih)/2+{h1 // 2 + (0 if args.subtitle else 0)}:w={int(base * 0.12)}:h={max(2, int(base * 0.006))}:color={ff_color(primary)}:t=fill:{en}")
+ filters.append(f"drawbox=x=(iw-{int(base * 0.12)})/2:y=(ih)/2+{title_h // 2}:w={int(base * 0.12)}:h={max(2, int(base * 0.006))}:color={ff_color(primary)}:t=fill:{en}")
if args.subtitle:
add_text(args.subtitle,
- f"drawtext={drawtext_text_opts(args.subtitle)}:{fo}:fontsize={h2}:fontcolor={ff_color(primary)}{drawtext_center_align()}:x=(w-text_w)/2:y=(h-text_h)/2+{h1 // 2 + int(base * 0.03)}:alpha='{fade_a}':{en}",
+ f"drawtext={drawtext_text_opts(args.subtitle)}:{fo}:fontsize={h2}:fontcolor={ff_color(primary)}{drawtext_center_align()}:x=(w-text_w)/2:y=(h-text_h)/2+{sub_y + sub_extra // 2}:alpha='{fade_a}':{en}",
size=h2, color=primary, font=ass_font_family(), align=5, x=W / 2,
- y=H / 2 + h1 // 2 + int(base * 0.03), outline=max(1.0, h2 / 20.0),
+ y=H / 2 + sub_y + sub_extra // 2, outline=max(1.0, h2 / 20.0),
outline_color="000000", start=s, end=e, fade=(300, 300))
elif args.template in ("chapter", "bug"):
@@ -439,10 +459,13 @@ def main() -> int:
hen = f"enable='between(t,{s:.3f},{he:.3f})'"
h1 = int(base * 0.085)
bar_h = max(3, int(base * 0.01))
- band_h = int(base * 0.30)
+ hook_title = wrapped(args.title, h1)
+ # the base band (30% of the short side) holds two lines; a longer wrapped title (a 9:16
+ # hook is four or five) grows the band to the text plus 0.7 of a line of margin, instead of
+ # drawing white text on the bare video. The band is clipped at the frame edges.
+ band_h = min(H, max(int(base * 0.30), block_h(hook_title, h1) + int(h1 * 0.7)))
y0 = (H - band_h) // 2
filters.append(f"drawbox=x=0:y={y0}:w=iw:h={band_h}:color={ff_color(bg, 0.78)}:t=fill:{hen}")
- hook_title = wrapped(args.title, h1)
add_text(hook_title,
f"drawtext={drawtext_text_opts(hook_title)}:{fo}:fontsize={h1}:fontcolor={ff_color(text_c)}"
f"{drawtext_center_align()}:x=(w-text_w)/2:y=(h-text_h)/2:{hen}",
diff --git a/scripts/silence.py b/scripts/silence.py
index 1d37694..b9424e3 100755
--- a/scripts/silence.py
+++ b/scripts/silence.py
@@ -141,9 +141,38 @@ def resolve_filler(args, meta):
"pass the result with --words.", kind="input")
lang = args.filler_lang
+ reported_lang = lang
if lang == "auto":
- lang = str((raw or {}).get("language") or "").lower()[:2] if isinstance(raw, dict) else ""
- lang = lang if lang in FILLER_WORDS else "en"
+ # the language the transcript says it is in: a --words document's own field, else what the
+ # engine was told or detected for this run (`--transcribe`). A language that is known and has
+ # no built-in list is refused, as --filler-lang names it: the English list would cut real
+ # words of that language (Norwegian "er" is "is"), and a transcript with no language at all
+ # (or one that says "auto", which is no language either) keeps the English list it always had
+ # the ISO 639-2 spellings of the languages that have a built-in list (the mux tag uses them)
+ iso3 = {"eng": "en", "spa": "es", "deu": "de", "ger": "de", "fra": "fr", "fre": "fr",
+ "ita": "it", "jpn": "ja", "por": "pt"}
+
+ def _code(value) -> str:
+ value = str(value or "").strip().lower().replace("_", "-")
+ if value in ("", "auto"):
+ return ""
+ # a regional tag names its primary language (ja-JP is ja); a three-letter code that is not
+ # in the table stays as it is (est is Estonian, not the Spanish "es"); a word such as
+ # "english" is cut to its two-letter code
+ primary = value.split("-")[0]
+ if primary in iso3:
+ return iso3[primary]
+ return primary[:2] if len(primary) > 3 else primary
+ spoken = _code((raw or {}).get("language")) if isinstance(raw, dict) else ""
+ spoken = spoken or _code(transcription.get("language") or transcription.get("detected_language"))
+ if spoken and spoken not in FILLER_WORDS and not args.filler_words:
+ die(f"the transcript is in {spoken}, which has no built-in filler list. The languages with one are "
+ f"{', '.join(sorted(FILLER_WORDS))}; pass --filler-words FILE with your own list for {spoken}, or "
+ "--filler-lang for a language that has one.", kind="input")
+ lang = spoken if spoken in FILLER_WORDS else "en"
+ # with a custom list the run goes ahead, and the result names the language it heard, not the
+ # English list it did not use
+ reported_lang = spoken or lang
listname = "builtin"
if args.filler_words:
wordlist = set(_word_list(read_text_or_die(args.filler_words, "--filler-words")))
@@ -174,7 +203,7 @@ def resolve_filler(args, meta):
warnings.append(f"--filler-extra {', '.join(extra_markers)}: a discourse marker, not a "
"disfluency -- this will cut real sentences.")
block = {
- "lang": lang, "source": source, "engine": engine,
+ "lang": reported_lang, "source": source, "engine": engine,
**({"transcription": transcription} if engine and transcription else {}),
"words": sorted(wordlist), "removed": [dict(s) for s in spans],
"removed_count": len(spans),
diff --git a/tests/_fixtures.py b/tests/_fixtures.py
index f0b94be..cfe5bfa 100644
--- a/tests/_fixtures.py
+++ b/tests/_fixtures.py
@@ -206,7 +206,7 @@ def _families_for(script):
return []
proc = subprocess.run(["fc-list", f":lang={FC_LANG[script]}", "family"], stdout=subprocess.PIPE, text=True)
return [f for line in proc.stdout.splitlines() for f in line.split(",")]
-# One build per process, not one per group: setUpClass runs for each of the six group classes and
+# One build per process, not one per group: setUpClass runs for each of the nine group classes and
# the footage below takes the best part of a minute to encode. The first class through builds it
# and records every attribute it set; the rest are handed the same paths.
_BUILT = {}
diff --git a/tests/test_accel.py b/tests/test_accel.py
index 2b05874..7a10763 100644
--- a/tests/test_accel.py
+++ b/tests/test_accel.py
@@ -370,6 +370,63 @@ class HwReviewRegressionTests(unittest.TestCase):
self.assertFalse(rep["hw"]["used"])
self.assertTrue(rep["hw"]["fallback"])
+ def test_a_failed_cpu_retry_does_not_claim_a_re_encode_and_a_repeat_refusal_is_noted_once(self):
+ """The note is written from the retry's outcome: a document for a job whose CPU retry also
+ failed must not say it was re-encoded, and a job of many encodes names the refusal once."""
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["6.1"])
+ broken = subprocess.CompletedProcess([], 1, "", "broken")
+ ok = subprocess.CompletedProcess([], 0, "", "")
+ with tempfile.TemporaryDirectory() as d:
+ out = str(Path(d) / "o.mp4")
+ cmd = ["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", out]
+ with mock.patch.object(runner, "_execute", side_effect=[refused, broken]):
+ proc = runner.run(cmd, quiet=True, check=False)
+ self.assertEqual(proc.returncode, 1)
+ self.assertFalse(STATE.hw_fallback, "fallback means a job was re-encoded on the CPU; this one was not")
+ self.assertEqual(len(STATE.hw_notes), 1)
+ self.assertIn("the CPU retry failed too", STATE.hw_notes[0])
+ self.assertNotIn("re-encoded on the CPU", STATE.hw_notes[0])
+ STATE.hw_notes, STATE.hw_fallback = [], False
+ with mock.patch.object(runner, "_execute", side_effect=[refused, ok, refused, ok]):
+ runner.run(cmd, quiet=True)
+ runner.run(cmd, quiet=True)
+ self.assertTrue(STATE.hw_fallback)
+ self.assertEqual(len(STATE.hw_notes), 1, STATE.hw_notes)
+ self.assertTrue(STATE.hw_notes[0].endswith("re-encoded on the CPU"))
+
+ def test_a_cpu_retry_that_times_out_leaves_no_claim_that_it_re_encoded(self):
+ """A timeout exits from inside the retry, before any outcome exists: the note stays the
+ neutral "retrying" line and fallback stays false."""
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["6.1"])
+ with tempfile.TemporaryDirectory() as d:
+ cmd = ["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", str(Path(d) / "o.mp4")]
+ with mock.patch.object(runner, "_execute", side_effect=[refused, SystemExit(124)]), \
+ self.assertRaises(SystemExit):
+ runner.run(cmd, quiet=True)
+ self.assertFalse(STATE.hw_fallback)
+ self.assertEqual(len(STATE.hw_notes), 1, STATE.hw_notes)
+ self.assertTrue(STATE.hw_notes[0].endswith("; retrying on the CPU encoder"), STATE.hw_notes)
+
+ def test_a_later_success_replaces_an_earlier_failed_retry_for_the_same_refusal(self):
+ STATE.hw, STATE.hw_source = True, "flag"
+ STATE.hw_swaps = [(["-c:v", "h264_videotoolbox", "-q:v", "75"], ["-c:v", "libx264", "-crf", "18"])]
+ refused = subprocess.CompletedProcess([], 1, "", VT_OPEN_FAILED["6.1"])
+ broken = subprocess.CompletedProcess([], 1, "", "broken")
+ ok = subprocess.CompletedProcess([], 0, "", "")
+ with tempfile.TemporaryDirectory() as d:
+ cmd = ["ffmpeg", "-i", "a.mp4", "-c:v", "h264_videotoolbox", "-q:v", "75", str(Path(d) / "o.mp4")]
+ with mock.patch.object(runner, "_execute", side_effect=[refused, broken]):
+ runner.run(cmd, quiet=True, check=False)
+ with mock.patch.object(runner, "_execute", side_effect=[refused, ok]):
+ runner.run(cmd, quiet=True)
+ self.assertTrue(STATE.hw_fallback)
+ self.assertEqual(len(STATE.hw_notes), 1, STATE.hw_notes)
+ self.assertTrue(STATE.hw_notes[0].endswith("re-encoded on the CPU"), STATE.hw_notes)
+
def test_the_cpu_fallback_keeps_the_even_dimension_scale_of_a_retry(self):
"""An odd-sized source under --hw: the encode is retried with an even scale, and when
VideoToolbox refuses that retry the CPU line is swapped into the retry, not into the
@@ -641,7 +698,8 @@ class HwEverywhereTests(_RealFfmpegCase):
self.assertIn("broken CPU encode", message)
self.assertNotIn("compression session", message, "the stderr is the command's that ran last")
self.assertIn("libx264", doc["commands"][-1])
- self.assertEqual((doc["encoder"], doc["hw"]["used"], doc["hw"]["fallback"]), ("libx264", False, True))
+ self.assertEqual((doc["encoder"], doc["hw"]["used"], doc["hw"]["fallback"]), ("libx264", False, False))
+ self.assertTrue(any(n.endswith("the CPU retry failed too") for n in doc["hw"]["notes"]), doc["hw"])
class HwResultTests(unittest.TestCase):
@@ -705,6 +763,20 @@ class HwResultTests(unittest.TestCase):
self.assertTrue(any("not CRF-equivalent" in n for n in rep["notes"]))
self.assertEqual(emit_module.steps_encoder_report([("fit", {"encoder": "libx264"})]), {"encoder": "libx264"})
+ def test_the_export_note_advises_hw_only_where_hw_can_run(self):
+ """export.py under FFMPEG_SKILL_HW=1 stays on the CPU and says so; "pass --hw" is advice for
+ a machine that can take the GPU, not for a Linux box that would report the CPU encode."""
+ STATE.reset()
+ STATE.hw_env_ignored = True
+ with mock.patch.object(decision, "hw_platform_reason", return_value=None):
+ notes = emit_module.hw_report(STATE, "libx264")["notes"]
+ self.assertEqual(notes, [emit_module.ENV_NOT_FOR_DELIVERY_NOTE])
+ self.assertIn("pass --hw", notes[0])
+ with mock.patch.object(decision, "hw_platform_reason", return_value="VideoToolbox is macOS-only"):
+ notes = emit_module.hw_report(STATE, "libx264")["notes"]
+ self.assertEqual(notes, [emit_module.ENV_NOT_FOR_DELIVERY_BASE])
+ self.assertNotIn("--hw", notes[0])
+
def test_render_carries_a_stage_s_fallback_and_gpu_quality_note(self):
"""render.py runs each stage as a child process and keeps only its command lines; the
stage's `hw` facts (fell back and why, ran on VideoToolbox) reach render's own result."""
diff --git a/tests/test_asr.py b/tests/test_asr.py
index 1d601da..aeb5ab3 100644
--- a/tests/test_asr.py
+++ b/tests/test_asr.py
@@ -283,8 +283,9 @@ class _BridgeHarness:
return calls[0]
-def _faster_whisper_with_words():
- """A faster_whisper that loads and returns one segment with word timings."""
+def _faster_whisper_with_words(detected=None):
+ """A faster_whisper that loads and returns one segment with word timings; `detected` is the
+ language its transcribe() info reports (faster-whisper's info.language)."""
mod = type(sys)("faster_whisper")
class Word:
@@ -301,7 +302,7 @@ def _faster_whisper_with_words():
def transcribe(self, wav, language=None, word_timestamps=False):
mod.languages.append(language)
- return iter([Segment()]), None
+ return iter([Segment()]), type("Info", (), {"language": detected})()
mod.WhisperModel = WhisperModel
mod.languages = []
return mod
@@ -533,6 +534,60 @@ class WhisperLanguageTests(_SrtDir, _BridgeHarness, unittest.TestCase):
run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "fr", "base"), fake=fake)
self.assertEqual((self._l(fake), run.facts["language"]), ("fr", "fr"))
+ def test_auto_and_every_spelling_of_english_reach_the_engines_as_the_engines_take_them(self):
+ """`--language auto` is the documented default spelled out, not a language; en-US, eng and
+ English are English, and Whisper takes "en" where it would reject the others."""
+ for given, passed in (("auto", None), ("AUTO", None), ("", None), ("en-US", "en"), ("eng", "en"),
+ ("English", "en"), ("EN", "en"), ("fr", "fr"), ("FR", "fr"),
+ ("ja-JP", "ja"), ("pt_BR", "pt"), ("zh-Hant", "zh"), ("jpn", "jpn")):
+ self.assertEqual(asr.engine_language(given), passed, given)
+ fake = _FakeEngines(["whisper-cli"])
+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "eng", "base"), fake=fake)
+ self.assertEqual((self._l(fake), run.facts["language"]), ("en", "en"))
+ fake = _FakeEngines(["whisper-cli"])
+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4", "auto"), fake=fake)
+ self.assertEqual((self._l(fake), run.facts["language"]), ("auto", None))
+ # an installed Parakeet takes --language auto as the undetected route, not as "not English"
+ fake = _FakeEngines(["parakeet-mlx"])
+ with mock.patch.object(asr, "detect_language", return_value=None):
+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "auto", "base"), fake=fake)
+ self.assertEqual((run.engine, run.facts["routing"]), ("parakeet-mlx", asr.ROUTING_ASSUMED_ENGLISH))
+
+ def test_the_language_an_engine_decoded_is_reported_when_nothing_was_named(self):
+ """faster-whisper's detected language and openai-whisper's JSON `language` are read back the way
+ whisper.cpp's are: silence.py --filler picks its word list from it."""
+ fw = _faster_whisper_with_words(detected="JA")
+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4"), faster_whisper=fw)
+ self.assertEqual((run.engine, run.facts["language"]), ("faster-whisper", "ja"))
+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"),
+ faster_whisper=_faster_whisper_with_words(detected="fr"))
+ self.assertEqual((run.engine, run.facts["language"]), ("faster-whisper", "fr"))
+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4", "de"),
+ faster_whisper=_faster_whisper_with_words(detected="ja"))
+ self.assertEqual(run.facts["language"], "de", "a language that was named is the one reported")
+ with tempfile.TemporaryDirectory() as d:
+ doc = Path(d) / "audio.json"
+ doc.write_text(json.dumps({"language": "no", "segments": []}), encoding="utf-8")
+ self.assertEqual(asr._whisper_json_language(str(doc)), "no")
+
+ def test_the_language_whisper_cpp_decoded_is_read_back_from_its_json(self):
+ """With -l auto nobody named the language and the detector may be absent: whisper.cpp's own
+ `result.language` is what lets a caller (silence.py --filler) pick the right word list."""
+ with tempfile.TemporaryDirectory() as d:
+ doc = Path(d) / "out.json"
+ doc.write_text(json.dumps({"result": {"language": "FR"}, "transcription": []}), encoding="utf-8")
+ self.assertEqual(asr._whisper_json_language(str(doc)), "fr")
+ doc.write_text(json.dumps({"transcription": []}), encoding="utf-8")
+ self.assertIsNone(asr._whisper_json_language(str(doc)))
+ self.assertIsNone(asr._whisper_json_language(str(Path(d) / "missing.json")))
+
+ def test_a_quantised_english_only_model_is_still_english_only(self):
+ for name in ("ggml-base.en.bin", "base.en", "ggml-base.en-q5_1.bin", "tiny.en-q8_0", "/m/ggml-small.en-q5_1.bin",
+ "ggml-small.en-tdrz.bin"):
+ self.assertTrue(asr._english_only_whisper_model(name), name)
+ for name in ("ggml-base.bin", "base", "ggml-large-v3-q5_0.bin", "ggml-medium-q8_0.bin", "/models.en/ggml-base.bin"):
+ self.assertFalse(asr._english_only_whisper_model(name), name)
+
def test_an_english_only_whisper_model_is_no_engine_for_any_language(self):
cli = mock.Mock(which=lambda n: "/x/whisper-cli" if n == "whisper-cli" else None)
openai = mock.Mock(which=lambda n: "/x/whisper" if n == "whisper" else None)
diff --git a/tests/test_contract.py b/tests/test_contract.py
index d869758..d498272 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -501,10 +501,12 @@ class ContractTests(unittest.TestCase):
tool whose `notes` description names its own condition (loop: "--boomerang ... the audio
was dropped") must not read as if that were the only one."""
for name, spec in self.tools.items():
- notes = spec["output_schema"]["properties"].get("notes") or {}
- if "hw" in spec["input_schema"]["properties"] and notes.get("description"):
- with self.subTest(tool=name):
- self.assertIn("VideoToolbox", notes["description"])
+ if "hw" not in spec["input_schema"]["properties"]:
+ continue
+ notes = spec["output_schema"]["properties"].get("notes")
+ with self.subTest(tool=name):
+ self.assertIsNotNone(notes, "a --hw tool publishes notes: emit() adds the GPU line to it")
+ self.assertIn("VideoToolbox", notes.get("description", ""))
self.assertIn("VideoToolbox", self.tools["loop"]["output_schema"]["properties"]["notes"]["description"])
self.assertIn("VideoToolbox", self.contract["idempotency_hints"]["bit_exact"])
diff --git a/tests/test_editing.py b/tests/test_editing.py
index 6a413a2..63d10c0 100644
--- a/tests/test_editing.py
+++ b/tests/test_editing.py
@@ -2017,6 +2017,37 @@ class EditingTests(MediaFixtures):
for span in fil["removed"]:
self.assertLess(span["start"], span["end"])
+ def test_filler_lang_auto_follows_the_language_the_engine_transcribed(self):
+ """--transcribe with the default --filler-lang auto took the English list whatever the
+ engine heard: French speech was transcribed right and its "euh" stayed in the cut. The
+ language the engine reports picks the list; one with no built-in list says so."""
+ words = ("[{'word': 'Bonjour', 'start': 0.2, 'end': 0.5}, {'word': 'euh', 'start': 0.6, 'end': 0.8},"
+ " {'word': 'tout', 'start': 1.0, 'end': 1.3}]")
+
+ def run(spoken, expect_fail=False):
+ shim = OUT / f"asr_shim_{spoken}"
+ shim.mkdir(exist_ok=True)
+ (shim / "sitecustomize.py").write_text(
+ "import sys, os\n"
+ "sys.path.insert(0, os.environ['FFSKILL_SCRIPTS'])\n"
+ "import _common.asr as asr\n"
+ "def fake(video, language=None, model='base', audio_stream=0, engine=None, **kw):\n"
+ f" return asr.Transcription(words={words}, engine='whisper.cpp', facts={{'language': {spoken!r}}})\n"
+ "asr.transcribe_words_result = fake\n", encoding="utf-8")
+ env = dict(os.environ, PYTHONPATH=str(shim), FFSKILL_SCRIPTS=str(SCRIPTS))
+ r = sh(sys.executable, SCRIPTS / "silence.py", self._gappy(), "--filler",
+ "--transcribe", "--filler-list", "--json", env=env, expect_fail=expect_fail)
+ return json.loads(r.stdout)
+ fr = run("fr")["filler"]
+ self.assertEqual((fr["lang"], fr["removed_words"], fr["warnings"]), ("fr", ["euh"], []))
+ en = run("en")["filler"]
+ self.assertEqual((en["lang"], en["removed_words"]), ("en", []))
+ # a language with no built-in list is refused, not cut with the English list ("er" is "is" in Norwegian)
+ no = run("no", expect_fail=True)
+ self.assertEqual(no["error"]["kind"], "input")
+ self.assertIn("no built-in filler list", no["error"]["message"])
+ self.assertIn("--filler-words", no["error"]["message"])
+
def test_filler_transcribe_refuses_when_the_engine_gives_no_word_timings(self):
"""An engine that runs but whose build has no word timestamps must say exactly that and
name --words -- not report an empty removal as a success."""
@@ -2061,6 +2092,50 @@ class EditingTests(MediaFixtures):
self.assertIn("--transcribe", err["message"])
self.assertFalse(out.exists())
+ def test_filler_auto_language_in_a_words_file_is_no_language(self):
+ """A --words document that says "auto" names no language: it keeps the English list, as a
+ document with no language field does, rather than being refused as the language "au"."""
+ doc = OUT / "words_lang_auto.json"
+ doc.write_text(json.dumps({"language": "auto", "words": [
+ {"word": "So", "start": 0.2, "end": 0.5}, {"word": "um", "start": 0.6, "end": 0.8}]}), encoding="utf-8")
+ fil = json.loads(script("silence.py", self._gappy(), "--filler", "--words", doc, "--filler-list",
+ "--json").stdout)["filler"]
+ self.assertEqual((fil["lang"], fil["removed_words"]), ("en", ["um"]))
+
+ def test_filler_iso_639_2_codes_name_the_language_with_a_built_in_list(self):
+ """A --words document may say "jpn" (ISO 639-2): that is Japanese, which has a built-in list,
+ not the two letters "jp" that have none."""
+ doc = OUT / "words_lang_jpn.json"
+ doc.write_text(json.dumps({"language": "jpn", "words": [
+ {"word": "えー", "start": 0.2, "end": 0.5}, {"word": "今日", "start": 0.6, "end": 0.8}]}), encoding="utf-8")
+ fil = json.loads(script("silence.py", self._gappy(), "--filler", "--words", doc, "--filler-list",
+ "--json").stdout)["filler"]
+ self.assertEqual((fil["lang"], fil["removed_words"]), ("ja", ["えー"]))
+
+ def test_an_unlisted_three_letter_code_is_refused_not_read_as_its_two_letter_prefix(self):
+ """est is Estonian: it has no built-in list, so it is refused. Cutting it to "es" would run the
+ Spanish list on Estonian speech."""
+ doc = OUT / "words_lang_est.json"
+ doc.write_text(json.dumps({"language": "est", "words": [
+ {"word": "tere", "start": 0.2, "end": 0.5}]}), encoding="utf-8")
+ r = script("silence.py", self._gappy(), "--filler", "--words", doc, "--filler-list", "--json",
+ expect_fail=True)
+ err = json.loads(r.stdout)["error"]
+ self.assertEqual(err["kind"], "input")
+ self.assertIn("est", err["message"])
+
+ def test_filler_custom_list_reports_the_language_it_heard(self):
+ """With --filler-words the run goes ahead on a language with no built-in list, and says which
+ language the transcript is in rather than the English default it did not use."""
+ doc = OUT / "words_lang_no.json"
+ doc.write_text(json.dumps({"language": "no", "words": [
+ {"word": "er", "start": 0.2, "end": 0.5}, {"word": "sant", "start": 0.6, "end": 0.8}]}), encoding="utf-8")
+ custom = OUT / "filler_words_no.txt"
+ custom.write_text("er\n", encoding="utf-8")
+ fil = json.loads(script("silence.py", self._gappy(), "--filler", "--words", doc, "--filler-words", custom,
+ "--filler-list", "--json").stdout)["filler"]
+ self.assertEqual((fil["lang"], fil["removed_words"]), ("no", ["er"]))
+
def test_filler_refuses_a_segment_only_transcript(self):
doc = OUT / "segments_only.json"
doc.write_text(json.dumps({"language": "en", "segments": [
diff --git a/tests/test_picture.py b/tests/test_picture.py
index 64b658c..31ed262 100644
--- a/tests/test_picture.py
+++ b/tests/test_picture.py
@@ -13,6 +13,7 @@ import re
import shutil
import subprocess
import sys
+import tempfile
import unittest
import unittest.mock
from pathlib import Path
@@ -1716,7 +1717,7 @@ class PictureTests(MediaFixtures):
def test_transcribe_without_engine_explains(self):
import shutil
- if shutil.which("whisper-cli") or shutil.which("whisper"):
+ if any(shutil.which(e) for e in ("whisper-cli", "whisper-cpp", "whisper", "parakeet-mlx", "parakeet-cli")):
self.skipTest("a local ASR engine is installed")
try:
import faster_whisper # noqa: F401
@@ -2972,12 +2973,31 @@ class MultiLineDrawtextTests(unittest.TestCase):
def test_the_drawn_text_keeps_its_line_breaks_and_loses_other_control_characters(self):
import _common
for text, drawn in (("A\nB", "A\nB"), ("A\r\nB", "A\nB"), ("A\rB", "A\nB"), ("A\x07B\x00", "AB"),
- ("tab\there\x0cnow", "tab here now")):
+ ("tab\there\x0cnow", "tab here now"),
+ # drawtext ignores one trailing newline and draws a second as a blank line
+ ("A\nB\n", "A\nB"), ("A\nB\n\n", "A\nB"), ("\n", "\n")):
opts = _common.drawtext_text_opts(text, tmpdir=str(OUT))
path = re.search(r"textfile=(.+?\.txt)", opts.replace("\\", "")).group(1)
pending = {os.path.normpath(k): v for k, v in _common._DRAWTEXT_PENDING.items()}
self.assertEqual(pending[os.path.normpath(path)], drawn, repr(text))
+ def test_the_text_file_is_written_with_bare_line_feeds_on_every_platform(self):
+ """Text mode writes each line break as CRLF on Windows, and drawtext from FFmpeg 6.1 draws
+ that as a blank line plus a line break (a wrapped title came out at twice its height)."""
+ import _common
+ OUT.mkdir(parents=True, exist_ok=True)
+ tmp = tempfile.mkdtemp(dir=str(OUT))
+ opts = _common.drawtext_text_opts("one\ntwo\nthree", tmpdir=tmp)
+ path = os.path.join(tmp, re.search(r"(t_[0-9a-f]+\.txt)", opts).group(1))
+ # text mode would write os.linesep, which is CRLF on Windows and invisible on this runner, so
+ # the file is opened with newline="\n" and that argument is what this checks
+ with unittest.mock.patch("os.fdopen", wraps=os.fdopen) as fdopen:
+ written = _common.flush_drawtext_textfiles(["ffmpeg", "-vf", f"drawtext={opts}", "out.mp4"])
+ self.assertEqual(fdopen.call_args.kwargs.get("newline"), "\n", fdopen.call_args)
+ self.assertEqual([os.path.normpath(w) for w in written], [os.path.normpath(path)])
+ with open(path, "rb") as fh:
+ self.assertEqual(fh.read(), b"one\ntwo\nthree")
+
def test_centred_lines_are_aligned_where_ffmpeg_can(self):
"""drawtext's text_align arrived in FFmpeg 6.1; 5.x and 6.0 would reject it, and centre
the block as a whole with its lines left-aligned."""
@@ -3038,6 +3058,144 @@ class MultiLineDrawtextTests(unittest.TestCase):
self.assertEqual(self._bands(self._gray(out, 3.0), top), [], "no text after the card's --duration")
+class LineAwareLayoutTests(unittest.TestCase):
+ """2.5.1 kept a newline in a title, but the title, lower-third and hook layouts were sized for
+ one line of text: a two-line title was struck through by its own accent bar and covered by its
+ subtitle, a two-line name was printed over its title, and a long 9:16 hook ran out of its band
+ onto the bare video. The layouts now follow the line count; one line is laid out as before."""
+
+ def _need_font(self):
+ if not default_font_file("DejaVu Sans") and not default_font_file(""):
+ self.skipTest("no concrete font file on this machine for drawtext")
+
+ def _background(self, name, size, colour):
+ OUT.mkdir(parents=True, exist_ok=True)
+ bg = OUT / f"{name}.mp4"
+ if not bg.exists():
+ sh("ffmpeg", "-y", "-v", "error", "-f", "lavfi", "-i", f"color=c={colour}:s={size}:r=30:d=3",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", str(bg))
+ return bg
+
+ def _rgb(self, path, w, h, at=1.5):
+ raw = subprocess.run(["ffmpeg", "-v", "error", "-ss", str(at), "-i", str(path), "-frames:v", "1",
+ "-f", "rawvideo", "-pix_fmt", "rgb24", "-"], stdout=subprocess.PIPE, check=True).stdout
+ self.assertEqual(len(raw), w * h * 3)
+ return raw
+
+ @staticmethod
+ def _runs(raw, w, h, pred, x0=0, x1=None, gap=4):
+ """(first row, last row) of each run of rows holding a pixel of columns x0..x1 that satisfies pred."""
+ x1 = w if x1 is None else x1
+ rows = [y for y in range(h) if any(pred(*raw[(y * w + x) * 3:(y * w + x) * 3 + 3]) for x in range(x0, x1))]
+ out = []
+ for y in rows:
+ if out and y - out[-1][1] <= gap:
+ out[-1][1] = y
+ else:
+ out.append([y, y])
+ return out
+
+ def test_a_two_line_title_with_a_two_line_subtitle_does_not_overlap_its_accent_bar(self):
+ self._need_font()
+ W, H = 1280, 720
+ bg = self._background("t2_bg_blue", f"{W}x{H}", "0x305070")
+ out = OUT / "t2_title.mp4"
+ script("graphics.py", bg, "--template", "title", "--title", "Hello\nWorld", "--subtitle", "sub one\nsub two",
+ "-o", out)
+ raw = self._rgb(out, W, H)
+ white = self._runs(raw, W, H, lambda r, g, b: r > 235 and g > 235 and b > 235)
+ yellow = self._runs(raw, W, H, lambda r, g, b: r > 230 and 170 < g < 235 and b < 90, gap=2)
+ self.assertEqual(len(white), 2, f"two title lines, got {white}")
+ self.assertEqual(len(yellow), 3, f"the accent bar, then two subtitle lines, got {yellow}")
+ bar, sub1, sub2 = yellow
+ self.assertLessEqual(bar[1] - bar[0], 6, f"the bar is a thin rule, got {bar}")
+ self.assertLess(white[1][1], bar[0], "the title ends above the bar")
+ self.assertLess(bar[1], sub1[0], "the first subtitle line starts below the bar")
+ self.assertLess(sub1[1], sub2[0], "the subtitle lines do not overlap")
+
+ def test_a_one_line_title_is_laid_out_as_it_always_was(self):
+ """The bar and subtitle of a one-line title sit where 2.5.0 put them (1280x720: bar at y=399)."""
+ self._need_font()
+ W, H = 1280, 720
+ bg = self._background("t2_bg_blue", f"{W}x{H}", "0x305070")
+ proc = script("graphics.py", bg, "--template", "title", "--title", "One line", "--subtitle", "plain sub",
+ "-o", OUT / "t2_title1.mp4", "--dry-run", "--json")
+ cmd = " ".join(json.loads(proc.stdout)["commands"])
+ self.assertIn("drawbox=x=(iw-86)/2:y=(ih)/2+39:", cmd)
+ self.assertIn("y=(h-text_h)/2+60:", cmd)
+
+ def test_a_two_line_name_and_title_fit_in_the_lower_third_bar(self):
+ self._need_font()
+ W, H = 1280, 720
+ bg = self._background("t2_bg_blue", f"{W}x{H}", "0x305070")
+ out = OUT / "t2_lower.mp4"
+ script("graphics.py", bg, "--template", "lower-third", "--name", "Jane\nDoe", "--title", "CEO\nand founder",
+ "-o", out)
+ raw = self._rgb(out, W, H)
+ # the bar is the dark plate right of the accent stripe; the text lies inside it
+ plate = self._runs(raw, W, H, lambda r, g, b: r < 40 and g < 45 and b < 50, x0=60, x1=62, gap=2)
+ self.assertEqual(len(plate), 1, plate)
+ # the two name lines can merge into one run on FFmpeg 5.1 (their rows are close): the name is
+ # checked as a block, the title as its two lines
+ white = self._runs(raw, W, H, lambda r, g, b: r > 235 and g > 235 and b > 235, x0=60)
+ yellow = self._runs(raw, W, H, lambda r, g, b: r > 230 and 170 < g < 235 and b < 90, x0=60, gap=2)
+ self.assertEqual(len(yellow), 2, yellow)
+ self.assertTrue(white, "the name is drawn")
+ self.assertGreater(white[-1][1] - white[0][0], 39, f"a two-line name is taller than one line, got {white}")
+ self.assertLess(white[-1][1], yellow[0][0], "the name ends above the title")
+ self.assertGreaterEqual(white[0][0], plate[0][0])
+ self.assertLessEqual(yellow[-1][1], plate[0][1])
+
+ def test_a_trailing_newline_is_not_a_second_line(self):
+ """'One line\\n' is one line: the frame drawn for it is the frame drawn for 'One line'. Before the
+ normalisation the block was counted as two lines and the bar was placed for the second."""
+ self._need_font()
+ W, H = 1280, 720
+ bg = self._background("t2_bg_blue", f"{W}x{H}", "0x305070")
+ frames = []
+ for i, title in enumerate(("One line", "One line\n")):
+ out = OUT / f"t2_trailing_{i}.mp4"
+ script("graphics.py", bg, "--template", "title", "--title", title, "--subtitle", "sub",
+ "-o", out)
+ frames.append(self._rgb(out, W, H))
+ self.assertEqual(frames[0], frames[1], "a trailing newline changed the layout")
+
+ def test_a_crlf_title_on_the_libass_route_draws_both_lines_above_the_bar(self):
+ """libass drops a bare CR's line, so a CRLF title drew one line while the bar was placed for
+ two: the line breaks are normalised once for every route, and both lines are drawn."""
+ self._need_font()
+ W, H = 1280, 720
+ bg = self._background("t2_bg_blue", f"{W}x{H}", "0x305070")
+ out = OUT / "t2_title_crlf_ass.mp4"
+ script("graphics.py", bg, "--template", "title", "--title", "Hello\r\nWorld", "--subtitle", "sub",
+ "--text-render", "ass", "-o", out)
+ raw = self._rgb(out, W, H)
+ white = self._runs(raw, W, H, lambda r, g, b: r > 235 and g > 235 and b > 235)
+ yellow = self._runs(raw, W, H, lambda r, g, b: r > 230 and 170 < g < 235 and b < 90, gap=2)
+ self.assertEqual(len(white), 2, f"both title lines are drawn, got {white}")
+ self.assertLess(white[1][1], yellow[0][0], "the title ends above the accent bar")
+
+ def test_a_long_hook_title_stays_inside_its_band(self):
+ self._need_font()
+ W, H = 1080, 1920
+ bg = self._background("t2_bg_grey_portrait", f"{W}x{H}", "0x909090")
+ out = OUT / "t2_hook.mp4"
+ script("graphics.py", bg, "--template", "hook", "--title",
+ "How I went from zero to a million followers in just ninety days with no budget", "-o", out)
+ raw = self._rgb(out, W, H)
+ band = self._runs(raw, W, H, lambda r, g, b: r < 100, x0=2, x1=4, gap=2)
+ band = [r for r in band if r[0] > 40] # not the progress bar along the top edge
+ self.assertEqual(len(band), 1, band)
+ # rows separated by less than the run gap merge (a descender touching the next line's ascender
+ # does so on FFmpeg 5.1), so this checks where the block starts and ends, not how many runs
+ text = self._runs(raw, W, H, lambda r, g, b: r > 235 and g > 235 and b > 235, x0=0, x1=W)
+ text = [r for r in text if r[0] > 40]
+ self.assertTrue(text, "the title is drawn")
+ self.assertGreater(text[-1][1] - text[0][0], 3 * 91, f"a multi-line title, got {text}")
+ self.assertGreater(text[0][0], band[0][0], "the first line starts inside the band")
+ self.assertLess(text[-1][1], band[0][1], "the last line ends inside the band")
+
+
class WrapReadabilityTests(unittest.TestCase):
"""1.15: no one-character orphan line, and a balanced break for spaced scripts."""