Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
feat: Parakeet speech engines for --transcribe (#312)
Shogo Kajiyama231130bf69de
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex afd2868..2a23f93 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,12 @@## Unreleased-(nothing yet)+- feat: Parakeet speech engines, built by [@rymalia](https://github.com/rymalia) in #305. `parakeet-mlx` and `parakeet.cpp` (`parakeet-cli` + a `.gguf`) for `caption.py --transcribe` and `silence.py --filler --transcribe`; `--engine` / `FFMPEG_SKILL_ASR_ENGINE` pick one, and a named engine runs as asked. `auto` runs Parakeet for English speech (an English `--language`, else whisper.cpp's language detector) and Whisper otherwise. Speech whose language nobody named and nothing here could detect goes to a Whisper engine that can run; Parakeet runs on English assumed only when there is none or every one failed, and the result's top-level `notes` then says the transcript is wrong if the speech is not English (new for `silence.py`). Results carry `transcription` (engine, model, language, routing) in every caption `--mode`; `silence.py`'s `filler.source` is `parakeet:ENGINE` for them. A Parakeet engine whose output is not a readable transcript is a failed run and the next engine is tried; its own empty answer is the `no_speech` refusal. `transcribe_result()` / `transcribe_words_result()` return what a call used as a `Transcription`, and `transcribe()` / `transcribe_words()` keep their return shapes. New optional contract capability `external:parakeet`.+- fix: whisper.cpp is told to detect the language (`-l auto`) when `--language` / `--filler-lang` is not given. It was given no `-l`, and its own default is English, so `caption.py --transcribe` and `silence.py --filler --transcribe` decoded speech in any other language as English. The result's `transcription.language` stays `null` for a language the engine identified itself.+- fix: the "no local speech-to-text engine found" refusal, with its install lines, now only means no engine was found. A whisper.cpp that was installed but failed, with nothing after it, got that refusal; now the refusal names each engine that was found and why it did not transcribe (`kind: input`, `reason: "engine_failed"` or `"english_only"`, `engines[].detail` with the engine's own error line).+- fix: a faster-whisper that raises while loading or running its model (an offline first run, a broken install) is a failed engine: `caption.py --transcribe` logs "faster-whisper found but failed" and tries the next engine instead of refusing with "faster-whisper found no speech", and `silence.py --filler --transcribe` can fall back to Parakeet. The exception used to be swallowed in the engine's thread. An engine that ran and found no segment is still the `no_speech` refusal.+- docs: parakeet-mlx downloads its default model (`mlx-community/parakeet-tdt-0.6b-v2`) from Hugging Face on first use, as faster-whisper does, and openai-whisper downloads its model from OpenAI. The install hint, README, docs/contract.md and references/scripts.md now say so, and that the audio never leaves the machine (inference is local). The skill's own code opens no connection, but faster-whisper runs inside the skill's process, so its first-run download is made from that process. The refusal's "all run offline" is gone: a first run that downloads is not offline.+- feat(contract): `execution.model_downloads` says which speech engine fetches its model on first use, from where, and in which process (faster-whisper in the skill's own; parakeet-mlx and openai-whisper in their own child processes; whisper.cpp and parakeet.cpp never). `execution.network: false` keeps its meaning, the skill's own code, and no longer reads as a promise that a first `--transcribe` is network-free.## 2.6.0diff --git a/README.md b/README.mdindex 88973b9..6ae843b 100644--- a/README.md+++ b/README.md@@ -82,7 +82,7 @@ An agent that "knows FFmpeg" still guesses: it assumes a frame rate, picks a cod- **Structured tools, not shell strings.** Each operation is a script with typed arguments. Nothing runs through a shell; no filter graph is accepted from the caller.- **A contract the agent can read.** `contract --json` states, for every tool, what it takes, what it writes, which FFmpeg components it needs and how the result is verified. The MCP surface is derived from it.- **Verification after execution.** The result is probed, checked against the destination's spec and, when the picture changed, looked at as a contact sheet.-- **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required — by `caption.py --transcribe` and, since 1.17, by `silence.py --filler --transcribe`; both take a transcript you already have instead, and both refuse with the install lines rather than guessing.+- **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a speech engine (Parakeet or a whisper) is installed, never required — by `caption.py --transcribe` and, since 1.17, by `silence.py --filler --transcribe`; both take a transcript you already have instead, and both refuse with the install lines rather than guessing. The engine runs on your machine and the audio is never uploaded; parakeet-mlx, faster-whisper and openai-whisper download their model weights the first time they run (parakeet-mlx's default from Hugging Face).## Quick start@@ -371,7 +371,7 @@ The short list for humans. The agent-facing version, with the reasoning, is the- **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened. Since 1.16.1 a Thai run and a katakana word are never broken inside (Thai writes no space inside a phrase and there is no dictionary: the break goes where you put a space or `|`), and a line wider than the safe width is reported as `overlong` with the fix named.- **Caption size fitted to the cue (1.17, reachable from the templates since 1.17.1).** At a platform caption size a line holds about six em, so an ordinary sentence needs four lines and `--max-lines 2` used to cut it into consecutive cues — half the sentence arriving late. `caption.py --fit-size` (default `auto`) walks the size down until every cue fits, *before* laying the cues out, with a legibility floor of 4.5 % of the frame height (`--min-size`, default 13 ASS units). An explicit `--size` or a `brand.json` size is a statement about the look and is never overridden — which in 1.17.0 also silenced the fitter on every `render.py --template` run, since a template fills the size from the platform table; 1.17.1 marks that size as the default it is (`"fit_size": "on"` in the filled project), so the type shrinks and no cue is split. `--fit-size off` restores that earlier behaviour byte for byte, and the caption text is still never rewritten to make it fit.- **Beat-synced cuts (1.17).** `scenes.py --beats` reports the measured grid — tempo, beat times, and a confidence built from how far the winning autocorrelation lag stands above the others and how many onsets land on it. `cut.py --snap beats` (and a `"snap"` block in a `render.py` project) moves in/out points to the nearest beat within `--snap-tolerance` — and only onto the grid points a measured onset actually marks, never onto the regular grid's continuation through a passage with no music in it. Below `--min-confidence` it **refuses**: a cut point may move to a measured beat and may not appear from one, so speech and ambience get an honest "no steady pulse here" instead of an invented grid.-- **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the three installs.+- **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.- **Throughput (1.17).** `batch.py --jobs N` runs several files at once, capped at `min(N, cpu_count, 8)` and sharing one `--timeout` budget rather than one per item; the per-item table keeps its order. `render.py --cache DIR` reuses stages whose inputs and arguments did not change, so swapping an export preset re-runs export only. The cache is opt-in with no default directory, and the ffmpeg, skill and contract versions are part of every key, so a cache is never reused across them.- **Audiogram (1.16).** `waveform.py --image cover.png` (or `render.py --template audiogram`) puts the waveform over a still plate for an episode that has no picture, with `--platform` for the frame, `--title` and burnt-in captions. The image is a local file you give: nothing is fetched and no cover art is ever invented.- **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.diff --git a/SKILL.md b/SKILL.mdindex d23e38d..e858d2a 100644--- a/SKILL.md+++ b/SKILL.md@@ -32,7 +32,7 @@ Ask a short question only when the answer changes the output materially and the- **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four and names a template. No destination and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where.- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, say which. Ask when the content is a talk (trimming loses words) and the change is large.-- **Captions** without a text source: `--transcribe` if a local whisper exists, else ask for the text or a timed file; never invent dialogue.+- **Captions** without a text source: `--transcribe` if a local engine exists, else ask for the text or a timed file; never invent dialogue.- **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once, reuse it.- **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.- **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of guessing centre, or ask which edge to keep.@@ -97,7 +97,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,| "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` || "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux`; repeat `--srt file:lang` for several languages, `.mkv` for more than two || "the captions are tiny / three lines on a Short", "don't chop the sentence" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` keeps `--size` and splits instead, `--min-size` the floor). Keep the template's `--max-lines` (2 on a vertical) and let the size drop; raising it to dodge a shrink stacks two words per line. A word still wider than the column at the floor is sliced at the edge (hyphen preferred), never rewritten, automatic; `broken_inside_word` in the JSON counts it |-| "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper; else `--text`) |+| "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (Parakeet/whisper; else `--text`) || "TikTok-style captions with the words popping" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` || "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` || "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |diff --git a/docs/contract.md b/docs/contract.mdindex 78e4b64..d4f3e61 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -123,7 +123,11 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},- "execution": {"shell": false, "arbitrary_executables": false, "network": false, "input_mutation": false}+ "execution": {"shell": false, "arbitrary_executables": false, "network": false,+ "model_downloads": {"when": "...", "in_process": {"faster-whisper": "..."},+ "child_process": {"parakeet-mlx": "...", "openai-whisper": "..."},+ "none": ["whisper.cpp", "parakeet.cpp"], "offline": "..."},+ "input_mutation": false}}```@@ -349,10 +353,10 @@ A project may now carry `"template"` (the name it was filled from) and `"frame":## CapabilitiesNames: `ffmpeg`, `ffprobe`, `encoder:<name>`, `filter:<name>`, `bsf:<name>`,-`external:whisper`. `capabilities.required` is the union of every tool's required list;+`external:whisper`, `external:parakeet`. `capabilities.required` is the union of every tool's required list;`optional` the union of the conditional ones. With detection (the default)`available`, `missing` and `missing_optional` are added from `doctor`, which reads-`ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper. Pass `--static`+`ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper or Parakeet. Pass `--static`to omit detection. Nothing from the environment other than those lists and theffmpeg/ffprobe/python versions is printed; no environment variables, no paths (`doctor`'s`hw.default_on` is a boolean derived from `FFMPEG_SKILL_HW`, never its value).@@ -360,8 +364,29 @@ ffmpeg/ffprobe/python versions is printed; no environment variables, no paths (``external:whisper` is **optional** for two tools since 1.17: `caption.py`(`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requiresit — both take a transcript the caller already has (`--srt`/`--words`), and-both refuse with the same three install lines when asked to make one with no+both refuse with the same install lines when asked to make one with noengine present. Whisper is never a dependency of this skill.+`external:parakeet` is the same kind of optional capability for the same two flags:+parakeet-mlx, or parakeet-cli with a `.gguf` model it can find. Either engine family+satisfies `--transcribe`; `--engine auto` (the default, or `FFMPEG_SKILL_ASR_ENGINE`) runs+Parakeet for English speech and Whisper for every other language. When no `--language` is given+and no detector can name it, a Whisper engine that can run takes it (whisper.cpp is passed+`-l auto`: its own default is English). Parakeet then runs only on English *assumed* -- when no+Whisper engine is installed, or every one failed, or `--model` names an English-only Parakeet+model -- and the result's top-level `notes` says the transcript is wrong if the speech is not+English; a multilingual (v3) Parakeet model assumes nothing and gets no note. The result's+`transcription` says which engine, model and routing decision produced the cues (in every+`--mode`, `mux` included).+Every engine transcribes on the machine running the skill, and the audio is never uploaded. The+skill's own code opens no network connection (`execution.network: false`), but an engine may+fetch its model weights on first use: parakeet-mlx downloads its default+`mlx-community/parakeet-tdt-0.6b-v2` from Hugging Face and openai-whisper its model from OpenAI,+each in its own child process. faster-whisper is a Python library the skill runs inside its own+process, so its first-run download from Hugging Face is made by the skill's process;+`HF_HUB_OFFLINE=1` keeps it (and parakeet-mlx) on the cached copy after that. parakeet.cpp and+whisper.cpp read a model file the user downloaded. `execution.model_downloads` says the same in the+contract, engine by engine, so a caller that sandboxes the network can tell before a first+`--transcribe` that faster-whisper's fetch comes from the skill's own process.`doctor` has three states per capability. `available` and `missing` come from a listingthat was read; `unknown` means the listing that would prove the capability could not be@@ -546,11 +571,19 @@ Per-tool keys added after 2.2.2, all additive:| `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line || `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected || `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |+| `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and `null` when the engine identified it itself; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |+| `notes` | `silence.py --filler --transcribe` (new for `silence.py`; `caption.py` already had it) | present when `--engine auto` ran the English-only Parakeet model on English *assumed* (no language given or detectable, and no Whisper engine installed or none that succeeded): the transcript is wrong if the speech is not English. `caption.py --transcribe` puts the same line in its `notes` |`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ranand produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,`reason: "no_speech"` and `engine` in the failure document — distinct from the "no local-speech-to-text engine found" refusal, which now only means no engine was found.+speech-to-text engine found" refusal, which now only means no engine was found. When `--engine+auto` found engines and none transcribed, the refusal is `kind: input` with `reason:+"engine_failed"` (one ran and failed) or `"english_only"` (the only engines found were Parakeet+engines auto passed over for a language that is not English), `engine` (the first one it names)+and `engines: [{engine, reason, detail}]`, where `detail` carries an engine's own error line. A+named `--engine` that is not installed is `kind: missing_tool`; one that is installed and failed+is `kind: input`, `reason: "engine_failed"`, with `engine` and `detail`.`check.py` also gains an informational `subtitles` row on **every** platform:`PASS` when every soft subtitle stream carries a language tag, `WARN` when onediff --git a/docs/design-decisions.md b/docs/design-decisions.mdindex bdd4cd0..e76c106 100644--- a/docs/design-decisions.md+++ b/docs/design-decisions.md@@ -949,3 +949,60 @@ not a new file format this tool would have to maintain.re-encoded on the CPU by the fallback above, with ffmpeg's line in the note. Code:`_common.runner.apple_silicon`; `doctor`'s `hw.platform_ok` uses the same check. Test:`test_hw_runs_on_apple_silicon_hardware_under_a_rosetta_python`.++## Unreleased — a second speech engine++- **`auto` picks Parakeet only for English.** The default Parakeet model (tdt-0.6b-v2) is+ English-only and transcribes other speech as English-shaped nonsense. A named `--language`+ decides; without one whisper.cpp's detector (the smallest multilingual ggml model, a second+ or less) decides. When neither can name the language, a Whisper engine that can run goes first+ (Parakeet only if every Whisper engine fails): a fast English transcript is not worth a+ confidently wrong one. Only when no Whisper engine is installed, or every one failed, does+ Parakeet run on English assumed, and then the result's top-level `notes` says the transcript is+ wrong if the speech is not English -- `transcription.routing` alone was too quiet for a caller+ that reads `verified`. "Goes first" means a Whisper engine that detects the language: whisper.cpp+ is passed `-l auto` when nothing named the language (its own default is `-l en`, which decoded+ every unnamed language as English, in 2.5.1 too), else the named or detected one, and an+ English-only Whisper model (`base.en`, `ggml-*.en.bin`) does not count as one.+ A named `--engine parakeet-*` runs as asked, with no note, and so does a Parakeet model named+ with `--model` or `PARAKEET_MODEL`: a multilingual (v3) one assumes nothing and gets no note; an+ English-only one named with `--model` gets the note, pointing at a v3 model rather than at a+ Whisper engine, which cannot load it. Measured on 8 min of LibriSpeech+ (contributor's figures, Apple Silicon): parakeet-mlx v2 2.8% WER at ~120× real time, whisper+ large-v3-turbo 2.4% at ~39×. Tests: `ParakeetRoutingTests`, `ParakeetEngineTests`,+ `UndetectedLanguageTests`, `WhisperLanguageTests`, `AutoRoutingFallbackTests`.+- **A Whisper engine that failed is a failure, not silence.** Undetected speech goes to Whisper+ first, so a Whisper failure has to read as one for Parakeet to be tried after it. A+ faster-whisper whose model would not load (an offline first run) raised inside its worker+ thread, the exception was swallowed and the run was refused as "found no speech"; it now logs+ "found but failed" and the next engine is tried, as a failed whisper.cpp always was. An engine+ that ran and returned no segment is still `no_speech` (`AsrNoSpeechTests`). For word timings,+ undetected speech gives every Whisper engine its turn before Parakeet, as captions do; with no+ Parakeet waiting, the first Whisper engine that ran and failed is still the one silence.py's+ "no word-level timings" refusal names, as in 2.5.1. Tests: `WhisperFailureTests`,+ `AutoRoutingFallbackTests`.+- **"No engine found" means none was found.** Its install lines are for a machine with no engine;+ 2.5.1 also gave it when an installed whisper.cpp failed, and auto would have given it after+ passing over an installed Parakeet engine for French. When engines were found and none+ transcribed, the refusal names each and why (`kind: input`, `reason` `engine_failed` or+ `english_only`, `engines[].detail` with the engine's own error line). A named `--engine` is `kind: missing_tool`+ only when it is not installed; an installed one that failed is `kind: input`, `engine_failed`.+ The line comes from stderr, or for parakeet-mlx from its stdout, where it prints its errors.+ Test: `EngineRefusalTests`.+- **The transcribe call returns what it used; nothing is left in module state.**+ `transcribe_result()` / `transcribe_words_result()` return a `Transcription` (cues, words,+ engine, the `transcription` facts, notes), so two transcriptions in one process never read each+ other's engine or words. `transcribe()` and `transcribe_words()` keep their old return shapes+ for existing callers. Test: `TranscriptionStateTests`.+- **The skill's code downloads nothing; an engine may fetch its model once.** parakeet-mlx's+ default `mlx-community/parakeet-tdt-0.6b-v2` comes from Hugging Face on first use, as+ faster-whisper's does, and openai-whisper's from OpenAI. The audio never leaves the machine and+ inference is local, so "no cloud" stays true. `execution.network: false` is about the skill's+ own code: parakeet-mlx and openai-whisper fetch in their own child processes, but faster-whisper+ is a library the skill runs in its own process, so its first-run fetch is made from that process+ (`HF_HUB_OFFLINE=1` keeps it on the cached copy afterwards). Loading it with `local_files_only`+ by default would break the first run 2.5.1 users rely on. The install hint, docs/contract.md and+ references/scripts.md say so, and the contract's `execution.model_downloads` names+ each engine's source and process, so `network: false` cannot be read as "a first+ `--transcribe` is network-free". Tests: `InstallHintTests`,+ `test_the_network_claim_matches_where_each_engine_runs`.diff --git a/references/scripts.md b/references/scripts.mdindex 4581b18..18abcb5 100644--- a/references/scripts.md+++ b/references/scripts.md@@ -490,7 +490,8 @@ each shorter clip's last frame (with silence) out to the longest.silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it][--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]- [--transcribe] [--filler-list] [--max-cuts 400]+ [--transcribe] [--engine auto|parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper]+ [--filler-list] [--max-cuts 400]```Runs `silencedetect`, keeps `--margin` seconds of air around speech, dropsgaps shorter than `--min-silence`, and re-encodes once with `select`/`aselect`@@ -533,7 +534,10 @@ Three refusals, all `kind: input`, all before any encode:- `--filler` with neither `--words` nor `--transcribe` → names both flags.- `--transcribe` with no engine on PATH → the same message `caption.py` gives,- with the three install lines.+ with its install lines (Parakeet and whisper engines). Engines that were found+ but did not transcribe are named instead, with why (`reason: engine_failed`,+ or `english_only` for a Parakeet engine auto passed over for `--filler-lang`+ that is not English; see `caption.py --transcribe` below).- `--transcribe` where the engine runs but its build produces no word-leveltimings → names that engine, says some builds do not support word timestamps,and points at `--words`. (`--transcribe` drives whichever engine is installed@@ -546,7 +550,11 @@ Three refusals, all `kind: input`, all before any encode:Results: `filler.removed_count`, `filler.removed_seconds`, `filler.removed`(one entry per span), `filler.lang`, `filler.list`, `filler.word_timings` and-`filler.warnings`. The existing `removed_seconds` keeps exactly the meaning it+`filler.warnings`. With `--transcribe`, `filler.source` is `whisper:ENGINE` or+`parakeet:ENGINE` and `filler.transcription` says which engine, model, language+and routing made the words; a top-level `notes` line says when `--engine auto`+ran the English-only Parakeet model on English assumed (no `--filler-lang`, and+no Whisper engine that could take it). The existing `removed_seconds` keeps exactly the meaning ithas always had — the seconds of **silence** this run removed, the figure thesame run would report without `--filler` — and `removed_seconds_total` is theadditive sibling covering everything that went. A filler word quiet enough to@@ -928,12 +936,38 @@ above the cap is clamped with a note, not refused, and both `jobs` and- `--watch` composes: each pass is parallel, passes are sequential.### caption.py --transcribe — optional local speech-to-text-If `whisper-cli` (whisper.cpp), `faster-whisper` or `whisper` is installed,-`caption.py input.mp4 --transcribe [--language ja] [--model base]` writes the-SRT from the audio and burns it (combine with `--animate pop --karaoke`).-Nothing is downloaded and nothing is required: without an engine it prints-install hints and the user can supply `--text` cues instead. Always tell the-user which engine was used, and treat the transcript as a draft to review.+If `parakeet-mlx`, `parakeet-cli` (parakeet.cpp, with a `.gguf`), `whisper-cli` (whisper.cpp),+`faster-whisper` or `whisper` is installed,+`caption.py input.mp4 --transcribe [--language ja] [--model base] [--engine auto]` writes the+SRT from the audio and burns it (combine with `--animate pop --karaoke`). `--engine auto` (default,+or `FFMPEG_SKILL_ASR_ENGINE`) runs Parakeet for English speech -- an English `--language`, else+English from whisper.cpp's language detector -- and Whisper for any other language. When neither+can name the language, a Whisper engine that can run goes first; Parakeet runs on English+*assumed* only when no Whisper engine is installed or every one failed, and the result's+top-level `notes` then says so: that transcript is wrong for non-English speech, so pass `--language`+(`--filler-lang` for `silence.py`) when the speech may not be English.+`--engine parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper` forces one (a+missing one is `kind: missing_tool`; an installed one that fails is `kind: input`, `reason:+engine_failed`, with its own error line in `detail`). Without a language whisper.cpp is passed+`-l auto` (its own default is English), else the named or detected language. `--model` is the+whisper model; a Parakeet engine takes it only when it names a Parakeet model+(`mlx-community/parakeet-*` or a `.gguf`), else `PARAKEET_MODEL` / `PARAKEET_CPP_MODEL`, else the+English `tdt-0.6b-v2`. A non-English `--language` on an English-only Parakeet model is refused; a+multilingual (v3) model assumes no language, so it gets no English-assumed note. The result's+`transcription` names the engine, model and routing. `silence.py --filler --transcribe` takes+the same `--engine`.+Nothing is required: without an engine it prints install hints and the user can supply `--text`+cues instead. When engines are installed but none transcribed, the refusal names each and why+instead (`reason: engine_failed`, or `english_only` for a Parakeet engine auto passed over for+another language). The skill's own code downloads nothing, and every engine transcribes on this+machine -- the audio is never uploaded. An engine may fetch its *model weights* the+first time it runs: parakeet-mlx downloads `mlx-community/parakeet-tdt-0.6b-v2` (or the `--model` /+`PARAKEET_MODEL` repo) from Hugging Face into its cache; faster-whisper (also from Hugging Face)+and openai-whisper (from OpenAI) fetch theirs on first use too. faster-whisper is a library that+runs inside the skill's own process, so its download is made from that process.+`HF_HUB_OFFLINE=1` keeps the Hugging Face ones on the cached copy afterwards. parakeet.cpp and+whisper.cpp only read a model file you downloaded. Always tell the user which engine was used,+and treat the transcript as a draft to review.### MCP server — the toolkit for any MCP client`python3 mcp/server.py` speaks MCP over stdio; each script is a tool takingdiff --git a/scripts/_common/asr.py b/scripts/_common/asr.pyindex 788e956..c6fc54f 100644--- a/scripts/_common/asr.py+++ b/scripts/_common/asr.py@@ -1,9 +1,11 @@"""The optional local speech-to-text bridge, and the SRT the rest of the skill reads and writes.-Whisper is never required. Nothing here runs unless a caller asked for a transcript:+No speech engine is ever required. Nothing here runs unless a caller asked for a transcript:`caption.py --transcribe` and `silence.py --filler --transcribe` share this one engine probe, sothe "no engine found" message, its install lines and its exit code are stated once rather than-copied per tool. This module is neither ffprobe nor a decision, which is why it is its own file.+copied per tool. What a call used (engine, model, routing, words, notes) comes back in its+Transcription, never in module state. This module is neither ffprobe nor a decision, which is+why it is its own file."""from __future__ import annotations@@ -11,20 +13,67 @@ import jsonimport osimport refrom pathlib import Path-from typing import Any, Dict, List, Optional, Tuple+from typing import Any, Dict, List, NamedTuple, Optional, Tuplefrom _common.decision import fmt_srt_time, parse_timefrom _common.emit import die, infofrom _common.runner import read_text_or_die-# The three engines this skill knows how to drive, and how to install each -- one string, so-# every tool that needs a transcript refuses in the same words.-ASR_ENGINES = ("whisper.cpp", "faster-whisper", "openai-whisper")+# The engines this skill knows how to drive, and how to install each -- one string, so every+# tool that needs a transcript refuses in the same words. The two Parakeet engines are+# English-first (the default v2 model is English-only): under `--engine auto` they run only for+# English speech and the Whisper engines take every other language.+PARAKEET_ENGINES = ("parakeet-mlx", "parakeet.cpp")+WHISPER_ENGINES = ("whisper.cpp", "faster-whisper", "openai-whisper")+ASR_ENGINES = PARAKEET_ENGINES + WHISPER_ENGINES+ENGINE_CHOICES = ("auto",) + ASR_ENGINES+ASR_ENGINE_ENV = "FFMPEG_SKILL_ASR_ENGINE"+PARAKEET_MLX_DEFAULT_MODEL = "mlx-community/parakeet-tdt-0.6b-v2"+# Every engine transcribes on this machine: the audio is never uploaded. Three of them fetch+# their model weights the first time they run, which the hint says, so "offline" is never+# promised for a first run that is not. The skill's own code opens no connection, but+# faster-whisper is a library run inside this process, so its first-run fetch is made from here.ASR_INSTALL_HINT = (- "Install one (all run offline):\n"+ "Install one (each transcribes on this machine; the audio never leaves it):\n"+ " parakeet-mlx: uv tool install parakeet-mlx (Apple Silicon; English; the first run\n"+ " downloads the model from Hugging Face, " + PARAKEET_MLX_DEFAULT_MODEL + ")\n"+ " parakeet.cpp: parakeet-cli from github.com/mudler/parakeet.cpp releases, plus a\n"+ " tdt-0.6b-v2 .gguf in ~/.cache/parakeet.cpp/ (English)\n"" whisper.cpp: brew install whisper-cpp (then download a model: ggml-base.bin)\n"- " faster-whisper: pip install faster-whisper\n"- " openai-whisper: pip install openai-whisper")+ " faster-whisper: pip install faster-whisper (the first run downloads its model)\n"+ " openai-whisper: pip install openai-whisper (the first run downloads its model)")+++class Transcription:+ """What one transcription produced, and what produced it.++ transcribe_result() and transcribe_words_result() return one instead of leaving it in module+ state, so two transcriptions in one process (mcp/server.py, batch.py, a test) can never read+ each other's engine, words or routing.++ cues [(start, end, text)]: what transcribe_result() wrote to the SRT (a Parakeet word+ run fills them too; a whisper word run leaves them [])+ words [{word, start, end}]: transcribe_words_result()'s word timings; from+ transcribe_result(), a Parakeet engine's word timings for --karaoke ([] for a+ whisper engine, whose word timings are read from a JSON next to the SRT instead)+ engine the engine that produced them, or None+ facts the result document's `transcription` block, in its key order:+ {routing, detected_language?, engine, model, language}+ notes sentences for the result document's top-level `notes`+ """++ def __init__(self, cues: Optional[List[Tuple[float, float, str]]] = None,+ words: "Optional[List[Dict[str, Any]]]" = None, engine: Optional[str] = None,+ facts: Optional[Dict[str, Any]] = None, notes: Optional[List[str]] = None) -> None:+ self.cues: List[Tuple[float, float, str]] = list(cues or [])+ self.words: "List[Dict[str, Any]]" = [dict(w) for w in (words or [])]+ self.engine = engine+ self.facts: Dict[str, Any] = dict(facts or {})+ self.notes: List[str] = list(notes or [])++ def __repr__(self) -> str:+ return (f"Transcription(engine={self.engine!r}, cues={len(self.cues)}, words={len(self.words)}, "+ f"facts={self.facts!r}, notes={self.notes!r})")def parse_srt(path: str) -> List[Tuple[float, float, str]]:@@ -61,6 +110,67 @@ def die_no_speech(engine: str, video: str) -> None:hint="check that the input (and --audio-stream) carries the speech, or write the cues by hand with --text")+ENGINE_FAILED_REASON = "engine_failed"+ENGLISH_ONLY_REASON = "english_only"+++def die_engine_failed(engine: str, video: str, tried: "List[Dict[str, str]]", installed: bool) -> None:+ """A named --engine that did not transcribe. kind missing_tool only when it is not installed;+ one that is installed and failed is kind input, `reason: "engine_failed"`, with the engine's+ own last line in `detail` -- a JSON caller never sees the log line above it."""+ if not installed:+ die(f"--engine {engine} is not installed" + (+ " (parakeet-cli and a .gguf model it can find: --model, PARAKEET_CPP_MODEL or ~/.cache/parakeet.cpp/)"+ if engine == "parakeet.cpp" else ""),+ kind="missing_tool", hint="install it (see --help), or use --engine auto to use whichever engine is installed")+ detail = next((t["detail"] for t in tried if t["engine"] == engine), "failed")+ die(f"--engine {engine} could not transcribe {video}: {engine} {detail}", kind="input",+ reason=ENGINE_FAILED_REASON, engine=engine, detail=detail,+ hint="fix the engine (its own message is in detail), or use --engine auto to fall back to another installed engine")+++def die_engines_failed(tried: "List[Dict[str, str]]", video: str, flag: str, alternative: str) -> None:+ """--engine auto found engines and none transcribed: name each and why. Not die_no_engine():+ its install lines would tell the caller to install what is installed. kind input;+ `reason` is "english_only" when the only engines found were Parakeet engines auto passed over+ for a language that is not English, else "engine_failed"; `engines` lists them all."""+ failed = [t for t in tried if t["reason"] != ENGLISH_ONLY_REASON]+ if failed:+ hint = "fix the engine that failed (its own message is in engines[].detail)"+ else:+ hint = ("install a Whisper engine for this language (see --help), or name a Parakeet engine with --engine "+ "and give it a multilingual (v3) model")+ die(f"no installed speech-to-text engine could transcribe {video} for {flag}:\n"+ + "\n".join(f" {t['engine']} {t['detail']}" for t in tried) + "\n" + alternative,+ kind="input", reason=ENGINE_FAILED_REASON if failed else ENGLISH_ONLY_REASON,+ engine=(failed or tried)[0]["engine"], engines=[dict(t) for t in tried], hint=hint)+++def _failed(engine: str, detail: str) -> "Dict[str, str]":+ return {"engine": engine, "reason": ENGINE_FAILED_REASON, "detail": detail}+++def _failure_line(proc: Any, errors_on_stdout: bool = False) -> str:+ """The line that says why an engine run failed, for the log and the failure document.++ Its last stderr line, else its last stdout line. parakeet-mlx (`errors_on_stdout`) prints its+ errors to stdout through rich -- "Error loading model ...", wrapped onto continuation lines --+ while stderr may hold only a download's progress bars, so its last error line comes first."""+ if errors_on_stdout:+ lines = [ln.strip() for ln in (proc.stdout or "").splitlines()]+ for i in range(len(lines) - 1, -1, -1):+ if "error" in lines[i].lower():+ end = i + 1+ while end < len(lines) and lines[end]:+ end += 1+ return " ".join(lines[i:end])[:200]+ for text in (proc.stderr, proc.stdout):+ lines = [ln.strip() for ln in (text or "").splitlines() if ln.strip()]+ if lines:+ return lines[-1][:200]+ return "?"++def _engine_cues(srt: str, engine: str, video: str) -> List[Tuple[float, float, str]]:"""The cues of an SRT an engine wrote; an SRT with no cue in it is die_no_speech()."""try:@@ -85,79 +195,179 @@ def write_srt(cues: List[Tuple[float, float, str]], path: str) -> None:fh.write(f"{i}\n{fmt_srt_time(s)} --> {fmt_srt_time(e)}\n{t}\n\n")-def transcribe(video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int = 0) -> List[Tuple[float, float, str]]:- """Optional local ASR bridge. Tries, in order: whisper-cli / main (whisper.cpp), faster-whisper (python),- whisper (openai-whisper CLI). Produces an SRT with word timings where the engine supports it.- No engine installed -> clear error with install hints; the skill never depends on one."""+def transcribe_result(video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int = 0,+ engine: Optional[str] = None) -> Transcription:+ """Optional local ASR bridge: the SRT written to `out_srt`, and a Transcription saying what+ produced it. `engine` (else $FFMPEG_SKILL_ASR_ENGINE, else auto) picks the engine; auto tries+ Parakeet (parakeet-mlx, parakeet.cpp) for English speech, then whisper-cli / main+ (whisper.cpp), faster-whisper (python), whisper (openai-whisper CLI). No engine installed ->+ clear error with install hints; the skill never depends on one."""import shutilimport subprocessimport tempfile- from _common import require_tool, run_analysis, STATE+ from _common import require_toolffmpeg = require_tool("ffmpeg")+ wanted = requested_engine(engine)tmpdir = tempfile.mkdtemp(prefix="ffskill_asr_")try:- return _transcribe_in(tmpdir, video, out_srt, language, model, audio_stream, ffmpeg, shutil, subprocess)+ return _transcribe_in(tmpdir, video, out_srt, language, model, audio_stream, ffmpeg, shutil, subprocess, wanted)finally:shutil.rmtree(tmpdir, ignore_errors=True)-def _asr_run(cmd: List[str], subprocess, name: str) -> "subprocess.CompletedProcess":+def transcribe(video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int = 0,+ engine: Optional[str] = None) -> List[Tuple[float, float, str]]:+ """transcribe_result()'s cues alone: the call shape caption.transcribe has always had. A caller+ that reports which engine ran (caption.py does) calls transcribe_result() instead."""+ return transcribe_result(video, out_srt, language, model, audio_stream, engine).cues+++def _asr_run(cmd: List[str], subprocess, name: str, env: "Optional[Dict[str, str]]" = None) -> "subprocess.CompletedProcess":"""Run a speech-to-text engine under the same wall-clock limit as an ffmpeg call."""from _common import STATE, dielimit = STATE.timeout or Nonetry:- return subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace", timeout=limit)+ return subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace",+ timeout=limit, **({"env": env} if env is not None else {}))except subprocess.TimeoutExpired:die(f"{name} exceeded the {limit:.0f} s time limit and was killed; raise --timeout for a long recording",code=124, kind="timeout")return None # unreachable+def assumed_english_note(language_flag: str = "--language", whisper_failed: bool = False,+ parakeet_model: bool = False) -> str:+ """The top-level `notes` line for a transcript the English-only Parakeet model made from speech+ whose language nobody named and nothing here could detect. The fix it names: the Whisper+ engine that failed (`whisper_failed`), a multilingual Parakeet model when --model named a+ Parakeet one (`parakeet_model`), else a Whisper engine to install."""+ if whisper_failed:+ fix = "or fix the Whisper engine that failed (the log names it)"+ elif parakeet_model:+ fix = "or name a multilingual (v3) Parakeet model with --model"+ else:+ fix = "or install a Whisper engine with a multilingual model, which --engine auto then uses for speech it cannot identify"+ return (f"English was assumed: no {language_flag} was given and the spoken language could not be "+ "detected here (that needs whisper.cpp with a multilingual ggml model), so the English-only "+ "Parakeet model made this transcript. If the speech is not English the transcript is wrong: "+ f"name the language with {language_flag}, " + fix)+++def _parakeet_attempt(engines: "List[str]", facts: Dict[str, Any], wanted: str, language: Optional[str],+ wav: str, tmpdir: str, model: Optional[str], shutil, subprocess, video: str,+ language_flag: str, need_words: bool = False,+ tried: "Optional[List[Dict[str, str]]]" = None) -> Optional[Tuple[Transcription, str]]:+ """(Transcription, the model's file name) from the first of `engines` that transcribes, or None.++ Under --engine auto with no language named or detected, the English-only model ran on an+ assumption: the Transcription then carries assumed_english_note(), and a warning is logged. A+ multilingual (v3) model assumed nothing, so it gets no note and its routing drops "assumed+ English". `need_words`: an engine whose cues read but whose word times do not is passed over+ (silence.py --filler can cut nothing on cues), unless it was asked for by name. An engine that+ ran and did not transcribe is added to `tried`."""+ for eng in engines:+ got = run_parakeet(eng, wav, tmpdir, model, language, shutil, subprocess, video, tried)+ if got and need_words and not got[1] and wanted not in PARAKEET_ENGINES:+ # cues read but no word times (a caption could use them; this caller cannot); a+ # named engine keeps its own "no word-level timings" refusal instead+ info(f"{eng} gave cues but no readable word timings; trying the next engine")+ if tried is not None:+ tried.append(_failed(eng, "gave cues but no readable word timings"))+ got = None+ if not got:+ continue+ cues, words, chosen = got+ run = Transcription(cues=cues, words=words, engine=eng,+ facts=dict(facts, engine=eng, model=chosen, language=language or facts.get("detected_language")))+ if wanted == "auto" and not language and not facts.get("detected_language"):+ if _parakeet_multilingual(chosen):+ run.facts["routing"] = run.facts["routing"].replace(", assumed English", "; the Parakeet model is multilingual")+ else:+ run.notes.append(assumed_english_note(language_flag, whisper_failed=facts.get("routing") == ROUTING_LAST_RESORT,+ parakeet_model=facts.get("routing") == ROUTING_PARAKEET_MODEL))+ info("warning: " + run.notes[-1])+ return run, os.path.basename(chosen)+ return None+++def _skipped_parakeet(route: "Route", language: Optional[str], language_flag: str) -> "List[Dict[str, str]]":+ """The installed Parakeet engines --engine auto passed over for a language that is not English."""+ what = f"{language_flag} {language}" if language else f"the detected language {route.facts.get('detected_language')}"+ return [{"engine": e, "reason": ENGLISH_ONLY_REASON,+ "detail": f"is installed, but --engine auto runs Parakeet only for English speech, and {what} is not English"}+ for e in route.skipped]++def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int,- ffmpeg: str, shutil, subprocess) -> List[Tuple[float, float, str]]:- from _common import run_analysis, STATE, die+ ffmpeg: str, shutil, subprocess, wanted: str = "auto") -> Transcription:+ from _common import run_analysis, STATEwav = os.path.join(tmpdir, "audio.wav")# A wav in our own temp dir: a measurement input for the engine, not a deliverable, so it# is not a run() call (no --dry-run gate, not recorded), but it keeps the time limit and# reports an unreadable input as kind ffmpeg instead of a CalledProcessError traceback.run_analysis([ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-i", video,"-map", f"0:a:{audio_stream}", "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le", wav])+ # every engine found that did not transcribe, and why: the refusal names them rather than+ # telling the caller to install what is installed+ tried: "List[Dict[str, str]]" = []++ def parakeet(engines: "List[str]", facts: Dict[str, Any]) -> Optional[Transcription]:+ done = _parakeet_attempt(engines, facts, wanted, language, wav, tmpdir, model, shutil, subprocess, video, "--language",+ tried=tried)+ if not done:+ return None+ run, model_name = done+ info(f"transcribed with {run.engine} (model {model_name})")+ write_srt(run.cues, out_srt)+ return run++ # 0. Parakeet (English), when auto-routing picks it or it was asked for by name+ route = parakeet_route(wanted, language, wav, shutil, subprocess, model)+ done = parakeet(route.first, route.facts)+ if done:+ return done+ tried.extend(_skipped_parakeet(route, language, "--language"))+ if wanted in PARAKEET_ENGINES:+ die_engine_failed(wanted, video, tried, _parakeet_available(wanted, shutil, model))+ facts = route.facts+ if route.first:+ # auto chose Parakeet and no Parakeet engine transcribed it: the routing says why Whisper ran+ facts = dict(facts, routing=facts["routing"] + ROUTING_PARAKEET_FAILED)+ # the language each Whisper engine is told: the named one, else the one auto's detector named+ lang = language or facts.get("detected_language")# 1. whisper.cpp- cli = shutil.which("whisper-cli") or shutil.which("whisper-cpp")- if not cli:- # older whisper.cpp builds ship the binary as plain `main`; accept it only when it lives- # in a directory that names whisper, so an unrelated /usr/bin/main is never run- main_bin = shutil.which("main")- if main_bin and "whisper" in os.path.dirname(os.path.realpath(main_bin)).lower():- cli = main_bin+ cli = _whisper_cpp_cli(shutil) if wanted in ("auto", "whisper.cpp") else Noneif cli:- model_path = model- if not os.path.exists(model_path):- for cand in (os.path.expanduser(f"~/.cache/whisper.cpp/ggml-{model}.bin"), f"models/ggml-{model}.bin", f"/usr/local/share/whisper/ggml-{model}.bin"):- if os.path.exists(cand):- model_path = cand- break+ model_path = _whisper_cpp_model(model)base = os.path.join(tmpdir, "out")- cmd = [cli, "-m", model_path, "-f", wav, "-osrt", "-of", base]- if language:- cmd += ["-l", language]+ # whisper.cpp's own default is -l en: without "auto" it decodes any speech as English+ cmd = [cli, "-m", model_path, "-f", wav, "-osrt", "-of", base, "-l", lang or "auto"]proc = _asr_run(cmd, subprocess, "whisper.cpp")if proc.returncode == 0 and os.path.exists(base + ".srt"):info(f"transcribed with whisper.cpp ({os.path.basename(cli)}, model {os.path.basename(model_path)})")cues = _engine_cues(base + ".srt", "whisper.cpp", video)write_srt(cues, out_srt)- return cues- info("whisper.cpp found but failed: " + (proc.stderr.strip().splitlines() or ["?"])[-1][:200])+ return Transcription(cues=cues, engine="whisper.cpp",+ facts=dict(facts, engine="whisper.cpp", model=model_path, language=lang))+ line = _failure_line(proc)+ info("whisper.cpp found but failed: " + line)+ tried.append(_failed("whisper.cpp", "found but failed: " + line))# 2. faster-whisper (python package)try:+ if wanted not in ("auto", "faster-whisper"):+ raise ImportErrorfrom faster_whisper import WhisperModel # type: ignoreimport threadingresult: list = []+ crashed: list = []def work() -> None:- m = WhisperModel(model, device="cpu", compute_type="int8")- segments, _ = m.transcribe(wav, language=language, word_timestamps=False)- result.extend((seg.start, seg.end, seg.text.strip()) for seg in segments if seg.text.strip())+ try:+ m = WhisperModel(model, device="cpu", compute_type="int8")+ segments, _ = m.transcribe(wav, language=lang, word_timestamps=False)+ result.extend((seg.start, seg.end, seg.text.strip()) for seg in segments if seg.text.strip())+ except Exception as exc: # noqa: BLE001 -- a model that would not load or run: a failed engine+ crashed.append(exc)# An in-process engine gets the same wall-clock limit as the CLI engines and ffmpeg.t = threading.Thread(target=work, daemon=True)@@ -165,28 +375,50 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[strt.join(STATE.timeout or None)if t.is_alive():die(f"faster-whisper exceeded the {STATE.timeout:.0f} s time limit; raise --timeout for a long recording", code=124, kind="timeout")- cues = list(result)- if not cues:- die_no_speech("faster-whisper", video)- info("transcribed with faster-whisper")- write_srt(cues, out_srt)- return cues+ if crashed:+ # it failed (a model it could not fetch or load), which is not "found no speech":+ # move on to the next engine, as a failed whisper.cpp does+ line = str(crashed[0])[:200] or type(crashed[0]).__name__+ info("faster-whisper found but failed: " + line)+ tried.append(_failed("faster-whisper", "found but failed: " + line))+ else:+ cues = list(result)+ if not cues:+ die_no_speech("faster-whisper", video)+ info("transcribed with faster-whisper")+ write_srt(cues, out_srt)+ return Transcription(cues=cues, engine="faster-whisper",+ facts=dict(facts, engine="faster-whisper", model=model, language=lang))except ImportError:pass# 3. openai-whisper CLI- if shutil.which("whisper"):+ if wanted in ("auto", "openai-whisper") and shutil.which("whisper"):cmd = ["whisper", wav, "--model", model, "--output_format", "srt", "--output_dir", tmpdir]- if language:- cmd += ["--language", language]+ if lang:+ cmd += ["--language", lang]proc = _asr_run(cmd, subprocess, "openai-whisper")srt = os.path.join(tmpdir, "audio.srt")if proc.returncode == 0 and os.path.exists(srt):info("transcribed with openai-whisper")cues = _engine_cues(srt, "openai-whisper", video)write_srt(cues, out_srt)- return cues+ return Transcription(cues=cues, engine="openai-whisper",+ facts=dict(facts, engine="openai-whisper", model=model, language=lang))+ line = _failure_line(proc)+ info("openai-whisper found but failed: " + line)+ tried.append(_failed("openai-whisper", "found but failed: " + line))+ # 4. Parakeet after all, when the language was never known and every Whisper engine failed+ if route.last:+ info("no Whisper engine transcribed it; trying Parakeet with English assumed")+ done = parakeet(list(route.last), {"routing": ROUTING_LAST_RESORT})+ if done:+ return done+ if wanted != "auto":+ die_engine_failed(wanted, video, tried, any(t["engine"] == wanted for t in tried))+ if tried:+ die_engines_failed(tried, video, "--transcribe", "Or write the cues by hand with --text cues.txt (see format above).")die_no_engine("Or write the cues by hand with --text cues.txt (see format above).")- return []+ return Transcription()def die_no_engine(alternative: str, flag: str = "--transcribe") -> None:@@ -293,22 +525,395 @@ def _words_from_openai_whisper_json(path: str) -> "List[Dict[str, Any]]":return [w for w in out if w["word"]]+# ------------------------------------------------------- Parakeet engines and routing+++def _words_from_parakeet_mlx_json(path: str) -> "List[Dict[str, Any]]":+ """parakeet-mlx --output-format json: sentences[].tokens[] of SUB-word pieces, times in seconds.++ A piece with a leading space starts a word; a bare " " piece is a separator, so the piece+ after it starts a word too (" m" "is" "ter" " " "Q" "u" "il" "ter" -> "mister", "Quilter").+ """+ try:+ doc = json.loads(Path(path).read_text(encoding="utf-8"))+ except (OSError, ValueError):+ return []+ out: "List[Dict[str, Any]]" = []+ for sent in _dicts(doc.get("sentences")) if isinstance(doc, dict) else []:+ brk = True # every sentence starts a new word+ for tok in _dicts(sent.get("tokens")):+ text = str(tok.get("text") or "")+ try:+ start, end = float(tok["start"]), float(tok["end"])+ except (KeyError, TypeError, ValueError):+ continue+ if not text.strip():+ brk = True+ continue+ if brk or text.startswith(" ") or not out:+ out.append({"word": text.strip(), "start": start, "end": end})+ else:+ out[-1]["word"] += text+ out[-1]["end"] = end+ brk = False+ return [w for w in out if w["word"]]+++def _cues_from_parakeet_mlx_json(path: str) -> List[Tuple[float, float, str]]:+ """parakeet-mlx's own sentence segmentation, as cues."""+ try:+ doc = json.loads(Path(path).read_text(encoding="utf-8"))+ except (OSError, ValueError):+ return []+ cues: List[Tuple[float, float, str]] = []+ for sent in _dicts(doc.get("sentences")) if isinstance(doc, dict) else []:+ try:+ text = str(sent.get("text") or "").strip()+ if text:+ cues.append((float(sent["start"]), float(sent["end"]), text))+ except (KeyError, TypeError, ValueError):+ continue+ return cues+++def _dicts(value: Any) -> "List[Dict[str, Any]]":+ """The dict items of a JSON list; anything else (a number, a string, null) is no items. The+ engines' JSON is parsed defensively: a wrong shape yields no words, never a traceback."""+ return [v for v in value if isinstance(v, dict)] if isinstance(value, list) else []+++def _words_from_parakeet_cpp_json(text: str) -> "List[Dict[str, Any]]":+ """parakeet-cli transcribe --json: {"words": [{"w", "start", "end", "conf"}]}, times in seconds."""+ try:+ doc = json.loads(text)+ except ValueError:+ return []+ out: "List[Dict[str, Any]]" = []+ for w in _dicts(doc.get("words")) if isinstance(doc, dict) else []:+ try:+ word = str(w.get("w") or w.get("word") or "").strip()+ if word:+ out.append({"word": word, "start": float(w["start"]), "end": float(w["end"])})+ except (AttributeError, KeyError, TypeError, ValueError):+ continue+ return out+++# Cue boundaries for an engine that gives words but no sentences (parakeet.cpp): the same limits+# parakeet-mlx's own segmentation is run with, so both Parakeet engines cut cues alike.+CUE_MAX_SECONDS = 7.0+CUE_MAX_CHARS = 84+CUE_GAP_SECONDS = 0.8+++def cues_from_words(words: "List[Dict[str, Any]]") -> List[Tuple[float, float, str]]:+ """Group timed words into cues: a cue ends after sentence-final punctuation, before a pause of+ CUE_GAP_SECONDS or more, or before it would pass CUE_MAX_SECONDS / CUE_MAX_CHARS."""+ cues: List[Tuple[float, float, str]] = []+ cur: "List[Dict[str, Any]]" = []++ def flush() -> None:+ if cur:+ cues.append((cur[0]["start"], cur[-1]["end"], " ".join(w["word"] for w in cur)))+ cur.clear()++ for w in words:+ if cur:+ text_len = len(" ".join(x["word"] for x in cur)) + 1 + len(w["word"])+ if (w["start"] - cur[-1]["end"] >= CUE_GAP_SECONDS or w["end"] - cur[0]["start"] > CUE_MAX_SECONDS+ or text_len > CUE_MAX_CHARS):+ flush()+ cur.append(w)+ if w["word"][-1:] in ".?!":+ flush()+ flush()+ return cues+++def requested_engine(engine: Optional[str]) -> str:+ """--engine, else $FFMPEG_SKILL_ASR_ENGINE, else "auto"; an unknown env value is refused."""+ name = engine or os.environ.get(ASR_ENGINE_ENV, "").strip() or "auto"+ if name not in ENGINE_CHOICES:+ die(f"{ASR_ENGINE_ENV}={name!r} is not a speech engine this skill drives (one of {', '.join(ENGINE_CHOICES)})",+ kind="input", hint=f"unset {ASR_ENGINE_ENV} or set it to one of {', '.join(ENGINE_CHOICES)}")+ return name+++def _is_english(language: Optional[str]) -> bool:+ # "eng" too: caption.py's --language is also the mux track tag, where ISO 639-2 is natural+ return (language or "").lower().split("-")[0].split("_")[0] in ("en", "eng", "english")+++def _whisper_cpp_cli(shutil) -> Optional[str]:+ cli = shutil.which("whisper-cli") or shutil.which("whisper-cpp")+ if not cli:+ # older whisper.cpp builds ship the binary as plain `main`; accept it only when it lives+ # in a directory that names whisper, so an unrelated /usr/bin/main is never run+ main_bin = shutil.which("main")+ if main_bin and "whisper" in os.path.dirname(os.path.realpath(main_bin)).lower():+ cli = main_bin+ return cli+++def _whisper_cpp_model(model: str) -> str:+ if os.path.exists(model):+ return model+ for cand in (os.path.expanduser(f"~/.cache/whisper.cpp/ggml-{model}.bin"), f"models/ggml-{model}.bin",+ f"/usr/local/share/whisper/ggml-{model}.bin"):+ if os.path.exists(cand):+ return cand+ return model+++def detect_language(wav: str, shutil, subprocess) -> Optional[str]:+ """The spoken language of `wav` from whisper.cpp's detector (-dl), or None when it cannot tell.++ The smallest installed ggml model is used: detection reads the first 30 s once, and a small+ model answers it in well under a second on the GPU.+ """+ cli = _whisper_cpp_cli(shutil)+ if not cli:+ return None+ model = next((p for p in (_whisper_cpp_model(m) for m in ("tiny", "base", "small", "tiny.en"))+ if os.path.exists(p) and not p.endswith(".en.bin")), None)+ if not model:+ return None+ proc = _asr_run([cli, "-m", model, "-f", wav, "-dl", "-l", "auto"], subprocess, "whisper.cpp language detection")+ m = re.search(r"auto-detected language:\s*([a-z]{2,3})", (proc.stderr or "") + (proc.stdout or ""))+ return m.group(1) if m else None+++ROUTING_ASSUMED_ENGLISH = "auto: language not detectable here and no Whisper engine, assumed English"+ROUTING_UNDETECTED_WHISPER = "auto: language not detectable here, Whisper first (Parakeet is English-only)"+ROUTING_LAST_RESORT = "auto: language not detectable here and no Whisper engine transcribed it, assumed English"+ROUTING_PARAKEET_MODEL = "auto: --model names a Parakeet model and the language is not detectable here, assumed English"+# appended to the routing auto chose Parakeet for, when no Parakeet engine transcribed it and a+# Whisper engine did+ROUTING_PARAKEET_FAILED = ", no Parakeet engine transcribed it"+++class Route(NamedTuple):+ """parakeet_route()'s answer: which Parakeet engines run, and when."""+ first: List[str] # Parakeet engines to try before any Whisper engine+ facts: Dict[str, Any] # the routing facts for the result document's `transcription`+ last: Tuple[str, ...] = () # Parakeet engines to try once every Whisper engine has failed+ skipped: Tuple[str, ...] = () # installed Parakeet engines auto passed over: the language is not English+++def parakeet_route(engine: str, language: Optional[str], wav: str, shutil, subprocess,+ model: Optional[str] = None, whisper_model: Optional[str] = None) -> Route:+ """Route(the Parakeet engines to try first, routing facts for the result document, the ones to+ try last, the installed ones passed over).++ `auto` tries Parakeet only for English: an explicit --language, else whisper.cpp's detector.+ When neither can say, the English-only model is not trusted with it: a Whisper engine that+ can run (`whisper_model`, default `model`) goes first and Parakeet only after every Whisper+ engine failed; with no Whisper engine (or a --model that names a Parakeet model, which no+ Whisper engine can load), Parakeet runs on English assumed and the caller's result carries+ assumed_english_note(). A named Parakeet engine always runs; with a non-English --language it+ needs a multilingual (v3) model, which _parakeet_model_for() checks.+ """+ if engine in PARAKEET_ENGINES:+ return Route([engine], {"routing": "requested"})+ if engine != "auto":+ return Route([], {"routing": "requested"})+ installed = tuple(e for e in PARAKEET_ENGINES if _parakeet_available(e, shutil, model))+ if not installed:+ return Route([], {"routing": "auto: no Parakeet engine installed"})+ if language:+ if _is_english(language):+ return Route(list(PARAKEET_ENGINES), {"routing": "auto: --language is English"})+ info(f"{' and '.join(installed)} passed over: --engine auto runs Parakeet only for English speech (--language {language})")+ return Route([], {"routing": f"auto: --language {language} is not English"}, (), installed)+ detected = detect_language(wav, shutil, subprocess)+ if detected is None:+ wmodel = whisper_model if whisper_model is not None else model+ if _names_parakeet_model(wmodel or "base"):+ return Route(list(PARAKEET_ENGINES), {"routing": ROUTING_PARAKEET_MODEL})+ if _whisper_ready(shutil, wmodel):+ info("the spoken language could not be detected here; a Whisper engine takes it (Parakeet's model is English-only)")+ return Route([], {"routing": ROUTING_UNDETECTED_WHISPER}, PARAKEET_ENGINES)+ return Route(list(PARAKEET_ENGINES), {"routing": ROUTING_ASSUMED_ENGLISH})+ if _is_english(detected):+ return Route(list(PARAKEET_ENGINES), {"routing": "auto: detected English", "detected_language": detected})+ info(f"{' and '.join(installed)} passed over: --engine auto runs Parakeet only for English speech (detected {detected})")+ return Route([], {"routing": f"auto: detected {detected}, not English", "detected_language": detected}, (), installed)+++def _module_importable(name: str) -> bool:+ """`import name` would find something, without importing it (faster-whisper pulls in ctranslate2)."""+ import importlib.util+ import sys+ try:+ return importlib.util.find_spec(name) is not None+ except (ImportError, ValueError): # a module already in sys.modules with no __spec__ (a stub)+ return sys.modules.get(name) is not None+++def _names_parakeet_model(name: str) -> bool:+ """`name` is a Parakeet model (an mlx-community/parakeet-* repo or a .gguf), which no Whisper engine loads."""+ return "parakeet" in name.lower() or name.lower().endswith(".gguf")+++def _english_only_whisper_model(name: str) -> bool:+ """An English-only Whisper model: tiny.en, base.en, ... or a ggml-*.en.bin file."""+ base = os.path.basename(name).lower()+ return (base[:-4] if base.endswith(".bin") else base).endswith(".en")+++def _whisper_ready(shutil, model: Optional[str]) -> bool:+ """A Whisper engine that --engine auto would run with `model` on speech in any language:+ whisper.cpp with that model on disk, faster-whisper importable, or the openai-whisper CLI. A+ Parakeet model name is no Whisper model, and an English-only one (base.en, ggml-*.en.bin)+ handles English alone, so both answer False."""+ name = (model or "base")+ if _names_parakeet_model(name) or _english_only_whisper_model(name):+ return False+ if _whisper_cpp_cli(shutil):+ path = _whisper_cpp_model(name)+ if os.path.exists(path) and not _english_only_whisper_model(path):+ return True+ return _module_importable("faster_whisper") or bool(shutil.which("whisper"))+++def _parakeet_available(engine: str, shutil, model: Optional[str] = None) -> bool:+ if engine == "parakeet-mlx":+ return bool(shutil.which("parakeet-mlx"))+ gguf = _parakeet_cpp_gguf(model)+ return bool(shutil.which("parakeet-cli")) and gguf is not None and os.path.isfile(gguf)+++def _parakeet_cpp_gguf(model: Optional[str]) -> Optional[str]:+ """--model when it is a .gguf, else $PARAKEET_CPP_MODEL, else an English tdt-0.6b-v2 .gguf in+ ~/.cache/parakeet.cpp/ (f16 first, then q8_0, then any other quantisation)."""+ if model and model.lower().endswith(".gguf"):+ return os.path.expanduser(model)+ env = os.environ.get("PARAKEET_CPP_MODEL", "").strip()+ if env:+ return os.path.expanduser(env)+ cache = Path(os.path.expanduser("~/.cache/parakeet.cpp"))+ found = sorted(cache.glob("tdt-0.6b-v2-*.gguf")) if cache.is_dir() else []+ for pref in ("-f16.gguf", "-q8_0.gguf"):+ for p in found:+ if p.name.endswith(pref):+ return str(p)+ return str(found[0]) if found else None+++def _parakeet_model_for(engine: str, model: Optional[str], language: Optional[str]) -> str:+ """The model the Parakeet engine runs: --model when it names a Parakeet model, else the+ engine's own environment variable, else the English v2 default. A non-English --language on a+ model without "v3" in its name is refused: v2 would transcribe it as English gibberish."""+ if engine == "parakeet-mlx":+ chosen = model if model and "parakeet" in model.lower() and not model.lower().endswith(".gguf") else \+ (os.environ.get("PARAKEET_MODEL", "").strip() or PARAKEET_MLX_DEFAULT_MODEL)+ else:+ chosen = _parakeet_cpp_gguf(model) or ""+ if language and not _is_english(language) and not _parakeet_multilingual(chosen):+ die(f"{engine} with model {os.path.basename(chosen) or '?'} is English-only; --language {language} needs a multilingual (v3) Parakeet model",+ kind="input", hint="pass --model with a parakeet-tdt-0.6b-v3 model, or --engine auto / whisper.cpp for this language")+ return chosen+++def _parakeet_multilingual(model: str) -> bool:+ """A multilingual Parakeet model (tdt-0.6b-v3, 25 European languages) rather than the English-only v2."""+ return "v3" in os.path.basename(model or "").lower()+++def run_parakeet(engine: str, wav: str, tmpdir: str, model: Optional[str], language: Optional[str], shutil, subprocess,+ video: str, tried: "Optional[List[Dict[str, str]]]" = None,+ ) -> Optional[Tuple[List[Tuple[float, float, str]], "List[Dict[str, Any]]", str]]:+ """(cues, words, model) from one Parakeet engine, or None when it is not installed or failed+ (an info line says which, and a failed run is added to `tried`), so the caller moves on to the+ next engine. Output that yields no words is only "no speech" when it is the engine's own empty+ answer; anything else it could not be read, which is a failed run too."""+ if not _parakeet_available(engine, shutil, model):+ info(f"{engine} not available" + (" (no readable .gguf: --model, PARAKEET_CPP_MODEL, or tdt-0.6b-v2 in ~/.cache/parakeet.cpp)"+ if engine == "parakeet.cpp" and shutil.which("parakeet-cli") else ""))+ return None+ chosen = _parakeet_model_for(engine, model, language)++ def failed(detail: str) -> None:+ info(f"{engine} {detail}")+ if tried is not None:+ tried.append(_failed(engine, detail))++ if engine == "parakeet-mlx":+ outdir = os.path.join(tmpdir, "pmlx")+ cmd = ["parakeet-mlx", wav, "--model", chosen, "--output-format", "json", "--output-dir", outdir,+ "--max-duration", f"{CUE_MAX_SECONDS:g}", "--silence-gap", f"{CUE_GAP_SECONDS:g}"]+ # the JSON is looked for as <outdir>/audio.json: a host PARAKEET_OUTPUT_TEMPLATE would name it otherwise+ env = {k: v for k, v in os.environ.items() if k != "PARAKEET_OUTPUT_TEMPLATE"}+ proc = _asr_run(cmd, subprocess, "parakeet-mlx", env=env)+ doc = os.path.join(outdir, os.path.splitext(os.path.basename(wav))[0] + ".json")+ if proc.returncode != 0 or not os.path.exists(doc):+ failed("found but failed: " + _failure_line(proc, errors_on_stdout=True))+ return None+ cues, words = _cues_from_parakeet_mlx_json(doc), _words_from_parakeet_mlx_json(doc)+ try:+ raw, key = Path(doc).read_text(encoding="utf-8", errors="replace"), "sentences"+ except OSError:+ raw, key = "", "sentences"+ else:+ cmd = ["parakeet-cli", "transcribe", "--model", chosen, "--input", wav, "--json"]+ proc = _asr_run(cmd, subprocess, "parakeet.cpp")+ if proc.returncode != 0:+ failed("found but failed: " + _failure_line(proc))+ return None+ words = _words_from_parakeet_cpp_json(proc.stdout)+ cues = cues_from_words(words)+ raw, key = proc.stdout or "", "words"+ if not cues:+ if _heard_nothing(raw, key):+ die_no_speech(engine, video)+ failed(f"ran but wrote output this skill cannot read as a transcript ({(raw.strip() or 'nothing')[:80]!r})")+ return None+ return cues, words, chosen+++def _heard_nothing(text: str, key: str) -> bool:+ """True when an engine's JSON is its well-formed empty answer, {key: []}: what parakeet-cli+ ("words") and parakeet-mlx ("sentences") both write for silence (measured on 3 s of it).+ Anything else that yields no words -- not JSON, the wrong shape, entries that do not parse --+ is output that could not be read, not a finding that nobody spoke."""+ try:+ doc = json.loads(text)+ except ValueError:+ return False+ return isinstance(doc, dict) and doc.get(key) == []++def transcribe_words(video: str, language: "Optional[str]" = None, model: str = "base",- audio_stream: int = 0) -> "Tuple[List[Dict[str, Any]], Optional[str]]":- """([{word, start, end}, ...], the engine that produced them) from a local whisper.+ audio_stream: int = 0, engine: "Optional[str]" = None) -> "Tuple[List[Dict[str, Any]], Optional[str]]":+ """transcribe_words_result()'s (words, engine) alone: the call shape this function has had+ since 1.17. A caller that reports which engine ran (silence.py does) calls+ transcribe_words_result() instead."""+ run = transcribe_words_result(video, language, model, audio_stream, engine)+ return run.words, run.engine+++def transcribe_words_result(video: str, language: "Optional[str]" = None, model: str = "base",+ audio_stream: int = 0, engine: "Optional[str]" = None,+ language_flag: str = "--filler-lang") -> Transcription:+ """[{word, start, end}, ...] from a local speech engine, as a Transcription naming the engine.- Drives whichever engine is installed with ITS word-timestamp option -- whisper.cpp+ `engine` (else $FFMPEG_SKILL_ASR_ENGINE, else auto) picks it; auto tries Parakeet for English+ speech first (parakeet-mlx sub-word tokens merged into words, parakeet.cpp `--json` words),+ then drives whichever whisper is installed with ITS word-timestamp option -- whisper.cpp`--output-json-full`, faster-whisper `word_timestamps=True`, openai-whisper- `--word_timestamps True` -- and returns the words it measured. ([], engine) when the engine- ran but its build produced no word-level timings, so the caller can refuse naming that engine- instead of pretending the audio had no words in it. No engine at all raises through- die_no_engine(), the same refusal caption.py gives.+ `--word_timestamps True` -- and returns the words it measured. Words [] with the engine set+ when the engine ran but its build produced no word-level timings, so the caller can refuse+ naming that engine instead of pretending the audio had no words in it. No engine at all+ raises through die_no_engine(), the same refusal caption.py gives. `language_flag` is the+ caller's flag for the language, named in assumed_english_note()."""import shutil as _shutilimport subprocess as _subprocessimport tempfile as _tempfilefrom _common import require_tool, run_analysis, STATEffmpeg = require_tool("ffmpeg")+ wanted = requested_engine(engine)+ parakeet_model = None if model == "base" else modeltmpdir = _tempfile.mkdtemp(prefix="ffskill_asrw_")try:wav = os.path.join(tmpdir, "audio.wav")@@ -316,47 +921,72 @@ def transcribe_words(video: str, language: "Optional[str]" = None, model: str ="-map", f"0:a:{audio_stream}", "-vn", "-ac", "1", "-ar", "16000","-c:a", "pcm_s16le", wav])+ tried: "List[Dict[str, str]]" = [] # engines found that gave no words, and why++ def parakeet(engines: "List[str]", facts: Dict[str, Any]) -> Optional[Transcription]:+ done = _parakeet_attempt(engines, facts, wanted, language, wav, tmpdir, parakeet_model,+ _shutil, _subprocess, video, language_flag, need_words=True, tried=tried)+ if not done:+ return None+ run, model_name = done+ info(f"word timings from {run.engine} ({len(run.words)} words, model {model_name})")+ return run++ # 0. Parakeet (English), when auto-routing picks it or it was asked for by name+ route = parakeet_route(wanted, language, wav, _shutil, _subprocess, parakeet_model, whisper_model=model)+ done = parakeet(route.first, route.facts)+ if done:+ return done+ tried.extend(_skipped_parakeet(route, language, language_flag))+ if wanted in PARAKEET_ENGINES:+ die_engine_failed(wanted, video, tried, _parakeet_available(wanted, _shutil, parakeet_model))+ facts = route.facts+ if route.first:+ facts = dict(facts, routing=facts["routing"] + ROUTING_PARAKEET_FAILED)+ lang = language or facts.get("detected_language")+ failed: Optional[str] = None # the first Whisper engine that ran and wrote no word-timing JSON+ # Undetected speech (Parakeet waits in route.last) was routed to Whisper because some+ # Whisper engine can take it, so each one gets its turn; otherwise the first that ran+ # and failed is the one the caller's refusal names, as before.+ every_whisper = bool(route.last)+# 1. whisper.cpp- cli = _shutil.which("whisper-cli") or _shutil.which("whisper-cpp")- if not cli:- main_bin = _shutil.which("main")- if main_bin and "whisper" in os.path.dirname(os.path.realpath(main_bin)).lower():- cli = main_bin+ cli = _whisper_cpp_cli(_shutil) if wanted in ("auto", "whisper.cpp") else Noneif cli:- model_path = model- if not os.path.exists(model_path):- for cand in (os.path.expanduser(f"~/.cache/whisper.cpp/ggml-{model}.bin"),- f"models/ggml-{model}.bin",- f"/usr/local/share/whisper/ggml-{model}.bin"):- if os.path.exists(cand):- model_path = cand- break+ model_path = _whisper_cpp_model(model)base = os.path.join(tmpdir, "out")- cmd = [cli, "-m", model_path, "-f", wav, "--output-json-full", "-of", base]- if language:- cmd += ["-l", language]+ # whisper.cpp's own default is -l en: without "auto" it decodes any speech as English+ cmd = [cli, "-m", model_path, "-f", wav, "--output-json-full", "-of", base, "-l", lang or "auto"]proc = _asr_run(cmd, _subprocess, "whisper.cpp")if proc.returncode == 0 and os.path.exists(base + ".json"):words = _words_from_whisper_cpp_json(base + ".json")info(f"word timings from whisper.cpp ({len(words)} words)")- return words, "whisper.cpp"- info("whisper.cpp found but produced no word-timing JSON: "- + (proc.stderr.strip().splitlines() or ["?"])[-1][:200])- return [], "whisper.cpp"+ return Transcription(words=words, engine="whisper.cpp",+ facts=dict(facts, engine="whisper.cpp", model=model_path, language=lang))+ line = _failure_line(proc)+ info("whisper.cpp found but produced no word-timing JSON: " + line)+ tried.append(_failed("whisper.cpp", "found but produced no word-timing JSON: " + line))+ failed = "whisper.cpp"# 2. faster-whispertry:+ if (failed and not every_whisper) or wanted not in ("auto", "faster-whisper"):+ raise ImportErrorfrom faster_whisper import WhisperModel # type: ignoreimport threadingcollected: list = []+ crashed: list = []def work() -> None:- m = WhisperModel(model, device="cpu", compute_type="int8")- segments, _ = m.transcribe(wav, language=language, word_timestamps=True)- for seg in segments:- for w in (getattr(seg, "words", None) or []):- collected.append({"word": str(w.word).strip(),- "start": float(w.start), "end": float(w.end)})+ try:+ m = WhisperModel(model, device="cpu", compute_type="int8")+ segments, _ = m.transcribe(wav, language=lang, word_timestamps=True)+ for seg in segments:+ for w in (getattr(seg, "words", None) or []):+ collected.append({"word": str(w.word).strip(),+ "start": float(w.start), "end": float(w.end)})+ except Exception as exc: # noqa: BLE001 -- a model that would not load or run+ crashed.append(exc)t = threading.Thread(target=work, daemon=True)t.start()@@ -364,29 +994,52 @@ def transcribe_words(video: str, language: "Optional[str]" = None, model: str =if t.is_alive():die(f"faster-whisper exceeded the {STATE.timeout:.0f} s time limit; raise ""--timeout for a long recording", code=124, kind="timeout")- info(f"word timings from faster-whisper ({len(collected)} words)")- return [w for w in collected if w["word"]], "faster-whisper"+ if crashed:+ line = str(crashed[0])[:200] or type(crashed[0]).__name__+ info("faster-whisper found but failed: " + line)+ tried.append(_failed("faster-whisper", "found but failed: " + line))+ failed = failed or "faster-whisper"+ else:+ info(f"word timings from faster-whisper ({len(collected)} words)")+ return Transcription(words=[w for w in collected if w["word"]], engine="faster-whisper",+ facts=dict(facts, engine="faster-whisper", model=model, language=lang))except ImportError:pass# 3. openai-whisper CLI- if _shutil.which("whisper"):+ if (not failed or every_whisper) and wanted in ("auto", "openai-whisper") and _shutil.which("whisper"):cmd = ["whisper", wav, "--model", model, "--word_timestamps", "True","--output_format", "json", "--output_dir", tmpdir]- if language:- cmd += ["--language", language]+ if lang:+ cmd += ["--language", lang]proc = _asr_run(cmd, _subprocess, "openai-whisper")doc = os.path.join(tmpdir, "audio.json")if proc.returncode == 0 and os.path.exists(doc):words = _words_from_openai_whisper_json(doc)info(f"word timings from openai-whisper ({len(words)} words)")- return words, "openai-whisper"- info("openai-whisper found but produced no word-timing JSON: "- + (proc.stderr.strip().splitlines() or ["?"])[-1][:200])- return [], "openai-whisper"+ return Transcription(words=words, engine="openai-whisper",+ facts=dict(facts, engine="openai-whisper", model=model, language=lang))+ line = _failure_line(proc)+ info("openai-whisper found but produced no word-timing JSON: " + line)+ tried.append(_failed("openai-whisper", "found but produced no word-timing JSON: " + line))+ failed = failed or "openai-whisper"+ # 4. Parakeet after all, when the language was never known and no Whisper engine gave words+ if route.last:+ info("no Whisper engine produced word timings; trying Parakeet with English assumed")+ done = parakeet(list(route.last), {"routing": ROUTING_LAST_RESORT})+ if done:+ return done+ if failed and not route.last:+ # it ran: the caller refuses naming it ("no word-level timings"), not "no engine".+ # After a last-resort Parakeet run, the refusal below names every engine tried.+ return Transcription(engine=failed, facts=facts)+ if wanted != "auto":+ die_engine_failed(wanted, video, tried, False)+ if tried:+ die_engines_failed(tried, video, "--filler --transcribe", "or pass --words with a transcript you already have.")die_no_engine("or pass --words with a transcript you already have.",flag="--filler --transcribe")- return [], None+ return Transcription()finally:_shutil.rmtree(tmpdir, ignore_errors=True)diff --git a/scripts/_contract.py b/scripts/_contract.pyindex 47edb42..f90ab43 100644--- a/scripts/_contract.py+++ b/scripts/_contract.py@@ -54,6 +54,8 @@ ROLES = {# filter:<name> `ffmpeg -filters`# bsf:<name> `ffmpeg -bsfs`# external:whisper a local whisper engine (whisper.cpp / faster-whisper / openai-whisper)+# external:parakeet a local Parakeet engine (parakeet-mlx, or parakeet-cli with a .gguf model);+# English. Either external engine satisfies --transcribe.# "optional" entries name the flag or condition under which the capability is needed.FF = ["ffmpeg", "ffprobe"]X264 = "encoder:libx264"@@ -145,7 +147,7 @@ TOOL_META: Dict[str, Dict[str, Any]] = {required=FF + [X264], optional=[],video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),"caption": dict(role="execution", inputs=["video asset", "SRT/ASS file or timed text (--text)"], outputs=["video artifact with burnt-in captions (--mode burn)", "video artifact with one or several language-tagged soft subtitle streams (--mode mux, --srt file:lang repeated)", "generated .srt / .ass sidecar"],- required=FF + [X264, AAC, "filter:subtitles"], optional=[{"capability": "filter:ass", "when": "--animate / --karaoke"}, HDR_X265, {"capability": "external:whisper", "when": "--transcribe"},+ required=FF + [X264, AAC, "filter:subtitles"], optional=[{"capability": "filter:ass", "when": "--animate / --karaoke"}, HDR_X265, {"capability": "external:whisper", "when": "--transcribe"}, {"capability": "external:parakeet", "when": "--transcribe (English; either engine family satisfies it)"},{"capability": "encoder:mov_text", "when": "--mode mux with a .mp4/.m4v/.mov output"}, {"capability": "encoder:webvtt", "when": "--mode mux with a .webm output"}, {"capability": "encoder:srt", "when": "--mode mux with a .mkv output"}],video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),"overlay": dict(role="execution", inputs=["video asset", "image (--image / --logo), text (--text), or a second video (--video) to composite"], outputs=["video artifact with the overlay composited"],@@ -172,7 +174,8 @@ TOOL_META: Dict[str, Dict[str, Any]] = {video_required=False, audio_only=True, visual=False, verify=["probe", "check"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),"silence": dict(role="analysis_and_execution", inputs=["video or audio asset"], outputs=["silence list JSON (--list)", "artifact with silences removed", "EDL text (--edl)"],required=FF + ["filter:silencedetect"], optional=[{"capability": X264, "when": "removing silences from a video"}, HDR_X265, {"capability": AAC, "when": "removing silences from a video"},- {"capability": "external:whisper", "when": "--filler --transcribe"}] + AUDIO_OUT,+ {"capability": "external:whisper", "when": "--filler --transcribe"},+ {"capability": "external:parakeet", "when": "--filler --transcribe (English; either engine family satisfies it)"}] + AUDIO_OUT,video_required=False, audio_only=True, visual=False, verify=["probe"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),"join": dict(role="execution", inputs=["two or more video assets, or two or more audio-only assets"], outputs=["concatenated video artifact", "concatenated audio artifact (audio-only inputs, audio output extension)"],required=FF + ["filter:xfade", "filter:acrossfade"],@@ -409,7 +412,8 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str"notes": {"type": "array", "items": {"type": "string"}, "description": "present when no --platform was named: youtube was assumed and judgement rows are WARN"},"checks": {"type": "array", "items": {"type": "object", "properties": {"check": {"type": "string", "description": "row name; --content adds black, frozen and silence; audio FAILs when the track is present but silent (peak <= -50 dBFS)"}, "status": {"enum": ["PASS", "WARN", "FAIL"]}, "value": {}, "expected": {}, "fix": {"type": "string"}, "kind": {"enum": ["format", "judgement"]}}}}}elif name == "caption":- extra = {"caption": {"type": "object", "description": "cue layout: shifted / wrapped / rebalanced / split / extended / dropped counts, plus wrap ('phrase' or 'measured'), phrase_breaks (1.16), broken_inside_word -- atoms hard-sliced at the column edge because they did not fit alone even at the size floor (1.18.4) -- and overlong -- now residual: a single character alone wider than the column. Burn mode: cues_burned -- non-blank cues that overlap [0, duration] and are drawn -- and cues_outside -- non-blank cues wholly outside it (2.2.6); a burn with no visible cue is refused, kind input"},+ extra = {"transcription": {"type": "object", "description": "with --transcribe: {engine, model, language, routing (why auto chose it), detected_language when auto detected it}; when auto ran the English-only Parakeet model on English assumed, notes says so"},+ "caption": {"type": "object", "description": "cue layout: shifted / wrapped / rebalanced / split / extended / dropped counts, plus wrap ('phrase' or 'measured'), phrase_breaks (1.16), broken_inside_word -- atoms hard-sliced at the column edge because they did not fit alone even at the size floor (1.18.4) -- and overlong -- now residual: a single character alone wider than the column. Burn mode: cues_burned -- non-blank cues that overlap [0, duration] and are drawn -- and cues_outside -- non-blank cues wholly outside it (2.2.6); a burn with no visible cue is refused, kind input"},"tracks": {"type": "array", "description": "--mode mux: one entry per subtitle stream in the output ({index, file, language, title, codec, default, cues, kept_from_input}); a stream the input already carried has file null and kept_from_input true (1.16)"},"subtitle_tracks": {"type": "integer", "description": "--mode mux: how many subtitle streams the output carries"},"emoji": {"type": "object", "description": "how the emoji in the text were drawn (mode, overlays, missing)"},@@ -430,7 +434,9 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str"speech": {"type": "array", "description": "--speech (1.18): [{time, speech_music_ratio}]"}}elif name == "silence":extra = {"silences": {"type": "array"}, "keep": {"type": "array"}, "input_duration": {"type": "number"}, "kept_duration": {"type": "number"}, "removed_seconds": {"type": "number"},- "speech_aware": {"type": "object", "description": "--speech-aware (1.18): {min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}"}}+ "speech_aware": {"type": "object", "description": "--speech-aware (1.18): {min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}"},+ "filler": {"type": "object", "description": "--filler: {lang, source (whisper-json:FILE | whisper:ENGINE | parakeet:ENGINE), engine, transcription (with --transcribe: {engine, model, language, routing}), removed, removed_count, ...}"},+ "notes": {"type": "array", "items": {"type": "string"}, "description": "--filler --transcribe under --engine auto: present when the English-only Parakeet model ran on English assumed (no --filler-lang, no language detected, no Whisper engine installed or none that succeeded); the transcript is wrong if the speech is not English"}}elif name == "sync":extra = {"reference": {"type": "string"}, "second": {"type": "string"}, "offset_seconds": {"type": "number"}, "confidence": {"type": "number"}, "meaning": {"type": "string"}, "drift": {"type": "object"},"sources": {"type": "array", "description": "1.18: [{path, offset_s, confidence, drift_ppm}], one per SOURCE; the only per-source shape once more than one SOURCE is given"}}@@ -672,6 +678,13 @@ def _whisper_available() -> bool:return importlib.util.find_spec("faster_whisper") is not None or importlib.util.find_spec("whisper") is not None+def _parakeet_available() -> bool:+ """parakeet-mlx on PATH, or parakeet-cli with a model it can find (PARAKEET_CPP_MODEL or a+ .gguf in ~/.cache/parakeet.cpp) -- the same test the speech bridge applies before running it."""+ from _common.asr import _parakeet_available as runtime_check # the bridge's own test, not a copy of it+ return any(runtime_check(e, shutil) for e in ("parakeet-mlx", "parakeet.cpp"))++def _hw_default() -> Dict[str, Any]:"""--hw's machine facts, normalised (never the raw environment value): whether VideoToolboxconstant-quality encoding can run here (the tools' own check: Apple Silicon hardware, also@@ -870,6 +883,8 @@ def doctor() -> Dict[str, Any]:state[cap] = _from("bsfs", cap[4:])elif cap == "external:whisper":state[cap] = "available" if _whisper_available() else "missing"+ elif cap == "external:parakeet":+ state[cap] = "available" if _parakeet_available() else "missing"else:state[cap] = "missing"available = sorted(c for c, st in state.items() if st == "available")@@ -997,7 +1012,15 @@ def _capability_fix_hint(cap: str) -> str:return (f"no installed font covers {LANGUAGE_NAMES.get(script, script)} text on this machine; "f"{FONT_INSTALL_HINT} (doctor --json .fonts.scripts lists every script)")if cap == "external:whisper":- return "install a local whisper (whisper-cli, whisper-cpp, faster-whisper or openai-whisper) for --transcribe"+ return ("install a local whisper (whisper-cli, whisper-cpp, faster-whisper or openai-whisper) for --transcribe; "+ "faster-whisper and openai-whisper download their model on first use, and every engine transcribes "+ "on this machine (the audio is never uploaded)")+ if cap == "external:parakeet":+ from _common.asr import PARAKEET_MLX_DEFAULT_MODEL+ return ("for English --transcribe, install parakeet-mlx (uv tool install parakeet-mlx; its first run downloads "+ f"{PARAKEET_MLX_DEFAULT_MODEL} from Hugging Face, then it transcribes on this machine and the audio is "+ "never uploaded) or parakeet-cli with a tdt-0.6b-v2 .gguf in ~/.cache/parakeet.cpp; a local whisper "+ "covers every language")return f"'{cap}' is not available; see docs/contract.md"@@ -1362,7 +1385,7 @@ def build(detect: bool = True) -> Dict[str, Any]:version = skill_version()tools = [tool_spec(n, version) for n in public_tools()]wanted = required_capabilities()- caps: Dict[str, Any] = {"required": wanted["required"], "optional": wanted["optional"], "naming": "ffmpeg | ffprobe | encoder:<name> | filter:<name> | bsf:<name> | external:whisper"}+ caps: Dict[str, Any] = {"required": wanted["required"], "optional": wanted["optional"], "naming": "ffmpeg | ffprobe | encoder:<name> | filter:<name> | bsf:<name> | external:whisper | external:parakeet"}if detect:d = doctor()caps.update({"available": d["available"], "missing": d["missing"], "missing_optional": d["missing_optional"],@@ -1389,8 +1412,21 @@ def build(detect: bool = True) -> Dict[str, Any]:"execution": {"shell": False,"arbitrary_executables": False,- "subprocess": "argv list only: [python3, scripts/<tool>.py, ...] and [ffmpeg|ffprobe, ...] resolved from PATH",+ "subprocess": "argv list only: [python3, scripts/<tool>.py, ...] and [ffmpeg|ffprobe, ...] resolved from PATH; "+ "with --transcribe, a local speech engine's CLI (whisper.cpp's whisper-cli / whisper-cpp / main, "+ "openai-whisper's whisper, parakeet-mlx, parakeet-cli) resolved from PATH","network": False,+ # `network` is about the skill's own code. A speech engine may fetch its model weights the+ # first time it runs; faster-whisper is a library, so its fetch is made from the skill's own+ # process. A caller that sandboxes the network must know that before a first --transcribe.+ "model_downloads": {+ "when": "first use of a speech engine (--transcribe) whose model is not cached yet; media is never uploaded",+ "in_process": {"faster-whisper": "Hugging Face (Systran/faster-whisper-<model>)"},+ "child_process": {"parakeet-mlx": "Hugging Face (mlx-community/parakeet-*)",+ "openai-whisper": "OpenAI (its model checkpoint)"},+ "none": ["whisper.cpp", "parakeet.cpp"],+ "offline": "HF_HUB_OFFLINE=1 keeps faster-whisper and parakeet-mlx on the cached copy",+ },"input_mutation": False,},"invocation": {diff --git a/scripts/caption.py b/scripts/caption.pyindex 7a20349..70714b2 100755--- a/scripts/caption.py+++ b/scripts/caption.py@@ -49,6 +49,8 @@ from _common import emoji_filter_chain, EMOJI_ASSET_HINT, emoji_asset_for, emoji# The ASR bridge and the SRT reader/writer live in _common.asr since 1.17 (silence.py --filler# shares them). They stay caption.py's public names -- every caller and test that reached for# caption.parse_srt / caption.transcribe / caption.whisper_word_timings before 1.17 still does.+from _common import asr as _asr+from _common.asr import ENGINE_CHOICESfrom _common import (ASR_INSTALL_HINT, die_no_engine, parse_srt, transcribe, whisper_word_timings,write_srt)from _common import (SAFE_WIDTH_FRACTION, ORPHAN_MIN_EM, WRAP_MODES, wrap_text, wrap_variants, best_break,@@ -901,7 +903,7 @@ def main() -> int:"them; .mkv is the honest multi-track container and stores the code you give verbatim")src.add_argument("--ass", help="ASS file to burn (styles inside the file are used)")src.add_argument("--text", help="plain text cue file to convert into SRT (see format above)")- src.add_argument("--transcribe", action="store_true", help="generate the SRT from the audio with a local speech-to-text engine if one is installed (whisper-cli / whisper / faster-whisper); never required")+ src.add_argument("--transcribe", action="store_true", help="generate the SRT from the audio with a local speech-to-text engine if one is installed (parakeet-mlx / parakeet.cpp / whisper-cli / whisper / faster-whisper); never required")src.add_argument("--language", "--lang", help="language code (e.g. en, ja, zh, ko): the language for --transcribe (default auto), ""the tag on the subtitle stream with --mode mux, and the hint that says whether Han-only ""text is Chinese, Japanese or Korean when a font is picked by script")@@ -914,7 +916,12 @@ def main() -> int:"(default: none, so no player burns in a language the viewer did not ask for)")src.add_argument("--offset", default="0", help="shift every cue by TIME (seconds, mm:ss, hh:mm:ss.ms or ""hh:mm:ss:ff; a leading - shifts earlier); works for --text, --srt and --ass")- src.add_argument("--model", default="base", help="whisper model name/path for --transcribe (default base)")+ src.add_argument("--model", default="base",+ help="speech model for --transcribe: a whisper model name/path (default base), or a Parakeet "+ "model (an mlx-community/parakeet-* repo or a .gguf) for the Parakeet engines")+ src.add_argument("--engine", choices=ENGINE_CHOICES, default=None,+ help="speech engine for --transcribe (default: $FFMPEG_SKILL_ASR_ENGINE, else auto = Parakeet "+ "for English speech when installed, else whisper.cpp / faster-whisper / openai-whisper)")src.add_argument("--write-srt", help="where to save the generated SRT (default: <text>.srt)")src.add_argument("--auto-seconds", type=float, default=3.0, help="duration for cues without timing (default 3)")src.add_argument("--gap", type=float, default=0.0, help="gap after auto-timed cues in seconds")@@ -1241,8 +1248,13 @@ def main() -> int:cues = []info(f"[dry-run] would transcribe {args.input} and write {srt_path}")else:- cues = transcribe(args.input, srt_path, args.language, args.model, args.audio_stream)- args._word_timings = whisper_word_timings(srt_path)+ # the call's own result, not module state: engine, routing and words belong to this run+ asr_run = _asr.transcribe_result(args.input, srt_path, args.language, args.model, args.audio_stream,+ engine=args.engine)+ cues = asr_run.cues+ args._asr = dict(asr_run.facts)+ args._asr_notes = list(asr_run.notes)+ args._word_timings = whisper_word_timings(srt_path) or sorted((w["start"], w["end"], w["word"]) for w in asr_run.words)cues, changed = lay_out(cues)if changed:write_srt(cues, srt_path)@@ -1279,7 +1291,8 @@ def main() -> int:# WRITING waits for a real run (the rule every side file in this tool follows), which is why# the cues are kept in hand below for the font sample and the ASS generator.planned_cues: Optional[List[Tuple[float, float, str]]] = None- side_notes: List[str] = []+ # an English-only transcript made on an assumption says so first (asr.assumed_english_note)+ side_notes: List[str] = list(getattr(args, "_asr_notes", None) or [])ass_sample_path = args.assif args.srt and not (args.text or args.transcribe) and os.path.exists(srt_path or ""):adjusted, changed = lay_out(parse_srt(srt_path))@@ -1416,7 +1429,8 @@ def main() -> int:result = probe(output, role="output")info(f"wrote {output} ({fmt_secs(result.get('duration'))}, mux, {len(added)} "f"subtitle track(s) added, codec {codec})")- emit(output, tracks=tracks, subtitle_tracks=total, **({"notes": notes} if notes else {}))+ emit(output, tracks=tracks, subtitle_tracks=total, **({"notes": notes} if notes else {}),+ **({"transcription": args._asr} if getattr(args, "_asr", None) else {}))return 0# A font that covers the text, before anything is rendered: non-Latin cues in a Latin-only@@ -1582,6 +1596,8 @@ def main() -> int:info("caption text unchanged: the cues were burned exactly as given (line breaks, timing ""and type size only)")extra["caption"] = dict(caption_stats)+ if getattr(args, "_asr", None):+ extra["transcription"] = args._asrif emoji_plan:notes = list(extra.get("notes") or [])if emoji_plan["mode"] == "mono":diff --git a/scripts/silence.py b/scripts/silence.pyindex 48d9901..1d37694 100755--- a/scripts/silence.py+++ b/scripts/silence.py@@ -20,7 +20,8 @@ from typing import List, Tuple# `detect` moved into _common/probe.py in 1.16.0 so metadata.py --auto-chapters can measure the# same silences without importing this tool; the body is unchanged and the name still lives here.from _common import (filler_spans, FILLER_WORDS, FILLER_AMBIGUOUS, FILLER_DISCOURSE_MARKERS,- FILLER_PAD, transcribe_words, read_text_or_die)+ FILLER_PAD, read_text_or_die)+from _common import asr as _asrfrom _common import detect_silences as detect, STATE, video_args, add_common, apply_common, audio_codec_for, cfr_args, default_output, die, emit, ffmpeg_base, info, is_audio_output, print_json, probe, run, X264_PRESETS, measured_level_dbfs, fmt_secs@@ -99,7 +100,7 @@ def resolve_filler(args, meta):die("--filler needs word timings: pass --words transcript.json (a whisper JSON with word ""timestamps) or --transcribe. There is no way to find a filler word without them -- ""cutting the short quiet blips instead would remove real speech.", kind="input")- source, engine, raw = None, None, None+ source, engine, raw, transcription = None, None, None, {}if args.words:try:raw = json.loads(read_text_or_die(args.words, "--words"))@@ -118,14 +119,20 @@ def resolve_filler(args, meta):'{"words": [{"word": ..., "start": ..., "end": ...}]} (or the same inside ''"segments").', kind="input")else:- # No pre-check for an installed engine here: transcribe_words() probes for one and+ # No pre-check for an installed engine here: transcribe_words_result() probes for one and# raises the same die_no_engine() refusal when there is none. Two places deciding "is# whisper here" is two places to disagree.# The engine is driven with ITS word-timestamp option (whisper.cpp --output-json-full,# faster-whisper word_timestamps=True, openai-whisper --word_timestamps True). An SRT# cannot answer this question: a cue has a start and an end, a word does not.- words, engine = transcribe_words(args.input, args.filler_lang if args.filler_lang != "auto" else None)- source = f"whisper:{engine}" if engine else "whisper"+ # The call's own result, not module state: engine, routing and notes belong to this run.+ asr_run = _asr.transcribe_words_result(args.input, args.filler_lang if args.filler_lang != "auto" else None,+ engine=args.engine)+ words, engine = asr_run.words, asr_run.engine+ transcription = dict(asr_run.facts)+ args._asr_notes = list(asr_run.notes)+ family = "parakeet" if engine in _asr.PARAKEET_ENGINES else "whisper"+ source = f"{family}:{engine}" if engine else "whisper"if not words:die(f"{engine or 'the local engine'} ran but produced no word-level timings, so there ""is nothing for --filler to cut on. Some builds do not support word timestamps. "@@ -168,6 +175,7 @@ def resolve_filler(args, meta):"disfluency -- this will cut real sentences.")block = {"lang": lang, "source": source, "engine": engine,+ **({"transcription": transcription} if engine and transcription else {}),"words": sorted(wordlist), "removed": [dict(s) for s in spans],"removed_count": len(spans),"removed_seconds": round(sum(s["end"] - s["start"] for s in spans), 3),@@ -235,8 +243,11 @@ def main() -> int:fil.add_argument("--words", metavar="FILE",help="a whisper JSON with word-level timings, for --filler")fil.add_argument("--transcribe", action="store_true",- help="produce the word timings with a local whisper (never required; the same "+ help="produce the word timings with a local speech engine, Parakeet or whisper (never required; the same ""bridge caption.py uses)")+ fil.add_argument("--engine", choices=_asr.ENGINE_CHOICES, default=None,+ help="speech engine for --transcribe (default: $FFMPEG_SKILL_ASR_ENGINE, else auto = Parakeet "+ "for English speech when installed, else whisper)")fil.add_argument("--filler-list", action="store_true",help="report what --filler would remove and write nothing")fil.add_argument("--max-cuts", type=int, default=400,@@ -311,6 +322,10 @@ def main() -> int:f"({filler_info['lang']}, list {filler_info['list']})")for warning in filler_info.get("warnings") or []:info("warning: " + warning)+ if getattr(args, "_asr_notes", None):+ # an English-only transcript made on an assumption (asr.assumed_english_note): the+ # bridge already logged the warning; the result document carries it at the top level+ summary["notes"] = list(args._asr_notes)if len(keeps) > args.max_cuts:die(f"{len(keeps)} keep ranges is above --max-cuts {args.max_cuts}: the filter graph grows ""with every range and a graph this size is slow and fragile. Raise --max-cuts if you "diff --git a/tests/_fixtures.py b/tests/_fixtures.pyindex 337317b..f0b94be 100644--- a/tests/_fixtures.py+++ b/tests/_fixtures.py@@ -21,6 +21,11 @@ SCRIPTS = ROOT / "scripts"HERE = Path(__file__).resolve().parentOUT = Path(os.environ.get("OUT", ROOT / "tests" / "out"))+# The host's own defaults must not reach the suite (every tool subprocess inherits os.environ):+# FFMPEG_SKILL_ASR_ENGINE or a PARAKEET_* setting would change which speech engine a --transcribe+# test drives, or where parakeet-mlx writes. Tests that exercise those opt in with an explicit env.+for _k in ("FFMPEG_SKILL_ASR_ENGINE", "PARAKEET_MODEL", "PARAKEET_CPP_MODEL", "PARAKEET_OUTPUT_TEMPLATE"):+ os.environ.pop(_k, None)sys.path.insert(0, str(SCRIPTS))sys.path.insert(0, str(HERE))from _common import escape_filter_path # noqa: E402diff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.jsonindex 4689475..3b43a28 100644--- a/tests/fixtures/mcp_tools.json+++ b/tests/fixtures/mcp_tools.json@@ -151,6 +151,7 @@"emoji_assets","emoji_max","emoji_scale",+ "engine","fast","fit_size","fit_size_scope",@@ -1034,6 +1035,7 @@"codec","dry_run","edl",+ "engine","fast","filler","filler_extra",diff --git a/tests/test_all.py b/tests/test_all.pyindex 6286273..4c37d00 100644--- a/tests/test_all.py+++ b/tests/test_all.py@@ -4,7 +4,7 @@python3 tests/test_all.py # or: python3 -m unittest tests/test_all.pyThe cases themselves live one module per tool group -- test_analysis, test_editing, test_audio,-test_picture, test_delivery, test_orchestration, test_accel -- over the fixtures in _fixtures.py. This file+test_picture, test_delivery, test_orchestration, test_accel, test_asr -- over the fixtures in _fixtures.py. This fileimports all seven so that the entry point every CI job, npm test and release_check.sh already runskeeps running exactly the same set; each group module is also runnable on its own."""@@ -16,13 +16,14 @@ sys.path.insert(0, str(Path(__file__).resolve().parent))import test_accel # noqa: E402import test_analysis # noqa: E402+import test_asr # noqa: E402import test_audio # noqa: E402import test_delivery # noqa: E402import test_editing # noqa: E402import test_orchestration # noqa: E402import test_picture # noqa: E402-MODULES = (test_analysis, test_editing, test_audio, test_picture, test_delivery, test_orchestration, test_accel)+MODULES = (test_analysis, test_editing, test_audio, test_picture, test_delivery, test_orchestration, test_accel, test_asr)def load_tests(loader, tests, pattern):diff --git a/tests/test_asr.py b/tests/test_asr.pynew file mode 100644index 0000000..1d601da--- /dev/null+++ b/tests/test_asr.py@@ -0,0 +1,976 @@+#!/usr/bin/env python3+"""Tests for the Parakeet speech engines.++ python3 tests/test_asr.py # this group alone+ python3 tests/test_all.py # every group++The Parakeet engines are driven through fake binaries on a PATH that holds nothing else, and a+faster_whisper the host may have installed is hidden from those runs, so these cases never depend on+what the host has installed.+"""+import json+import os+import platform+import shutil+import stat+import subprocess+import sys+import tempfile+import textwrap+import unittest+from pathlib import Path+from unittest import mock++sys.path.insert(0, str(Path(__file__).resolve().parent))+from _fixtures import MediaFixtures, OUT, script, sh # noqa: E402+from _common import asr # noqa: E402+++def _assumed(doc):+ """The result document's top-level notes that say English was assumed."""+ return [n for n in doc.get("notes") or [] if n.startswith("English was assumed")]+++class ParakeetCppModelTests(unittest.TestCase):+ """parakeet.cpp runs only with a .gguf it can find: an explicit --model is enough, for a+ forced engine and for --engine auto alike."""++ def test_parakeet_cpp_runs_with_only_an_explicit_model(self):+ with tempfile.TemporaryDirectory() as d, mock.patch.dict(os.environ, {"HOME": d}):+ os.environ.pop("PARAKEET_CPP_MODEL", None)+ gguf = Path(d) / "tdt-0.6b-v3-f16.gguf"+ gguf.write_bytes(b"GGUF")+ which = mock.Mock(which=lambda n: "/x/parakeet-cli" if n == "parakeet-cli" else None)+ self.assertFalse(asr._parakeet_available("parakeet.cpp", which))+ self.assertTrue(asr._parakeet_available("parakeet.cpp", which, str(gguf)))+ self.assertEqual(asr._parakeet_model_for("parakeet.cpp", str(gguf), "de"), str(gguf))+ # --engine auto must see it too, not only a forced --engine parakeet.cpp+ self.assertEqual(asr.parakeet_route("auto", "en", "a.wav", which, subprocess)[0], [])+ self.assertEqual(asr.parakeet_route("auto", "en", "a.wav", which, subprocess, str(gguf))[0], list(asr.PARAKEET_ENGINES))+++MLX_DOC = {"text": "So um we start. Here.", "sentences": [+ {"text": "So um we start.", "start": 0.2, "end": 1.6, "tokens": [+ {"text": " So", "start": 0.2, "end": 0.4}, {"text": " u", "start": 0.5, "end": 0.6}, {"text": "m", "start": 0.6, "end": 0.7},+ {"text": " ", "start": 0.7, "end": 0.8}, {"text": "we", "start": 0.9, "end": 1.0}, {"text": " start", "start": 1.1, "end": 1.5},+ {"text": ".", "start": 1.5, "end": 1.6}]},+ {"text": "Here.", "start": 2.0, "end": 2.4, "tokens": [{"text": " Here", "start": 2.0, "end": 2.3}, {"text": ".", "start": 2.3, "end": 2.4}]}]}+CPP_DOC = {"text": "So um we start. Here.", "frame_sec": 0.08, "words": [+ {"w": "So", "start": 0.2, "end": 0.4, "conf": 0.99}, {"w": "um", "start": 0.5, "end": 0.7, "conf": 0.9},+ {"w": "we", "start": 0.9, "end": 1.0, "conf": 0.99}, {"w": "start.", "start": 1.1, "end": 1.6, "conf": 0.99},+ {"w": "Here.", "start": 2.0, "end": 2.4, "conf": 0.99}]}+++class ParakeetParsingTests(unittest.TestCase):+ def test_mlx_subword_tokens_become_words(self):+ with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:+ json.dump(MLX_DOC, fh)+ words = asr._words_from_parakeet_mlx_json(fh.name)+ self.assertEqual([w["word"] for w in words], ["So", "um", "we", "start.", "Here."])+ self.assertEqual((words[1]["start"], words[1]["end"]), (0.5, 0.7))+ self.assertEqual(asr._cues_from_parakeet_mlx_json(fh.name), [(0.2, 1.6, "So um we start."), (2.0, 2.4, "Here.")])+ os.unlink(fh.name)++ def test_cpp_words_and_the_cues_built_from_them(self):+ words = asr._words_from_parakeet_cpp_json(json.dumps(CPP_DOC))+ self.assertEqual(len(words), 5)+ self.assertEqual(asr.cues_from_words(words), [(0.2, 1.6, "So um we start."), (2.0, 2.4, "Here.")])++ def test_malformed_engine_output_yields_no_words_instead_of_crashing(self):+ """Garbage, the wrong shape, and words missing fields are skipped, never raised."""+ for text in ("not json", "[]", "null", json.dumps({"words": "x"}), json.dumps({"words": 5}),+ json.dumps({"words": [5, None, []]})):+ self.assertEqual(asr._words_from_parakeet_cpp_json(text), [], text)+ mixed = {"words": [{"w": "ok", "start": 0.1, "end": 0.3}, {"w": "no-times"}, {"start": 1, "end": 2},+ {"w": "bad", "start": "x", "end": 1}, "str", {"w": "fine", "start": 1.0, "end": 1.2}]}+ self.assertEqual([w["word"] for w in asr._words_from_parakeet_cpp_json(json.dumps(mixed))], ["ok", "fine"])+ with tempfile.TemporaryDirectory() as d:+ p = Path(d) / "a.json"+ for body in ("{truncated", "[1, 2]", json.dumps({"sentences": [{"text": "no times"}]}),+ json.dumps({"sentences": 5}), json.dumps({"sentences": [5, None]}),+ json.dumps({"sentences": [{"tokens": 5}]}), json.dumps({"sentences": [{"tokens": [5, None]}]})):+ p.write_text(body)+ self.assertEqual(asr._cues_from_parakeet_mlx_json(str(p)), [], body)+ self.assertEqual(asr._words_from_parakeet_mlx_json(str(p)), [], body)+ self.assertEqual(asr._words_from_parakeet_mlx_json(str(Path(d) / "missing.json")), [])++ def test_cues_split_on_a_pause_and_on_length(self):+ words = [{"word": "a", "start": 0.0, "end": 0.2}, {"word": "b", "start": 1.5, "end": 1.7}]+ self.assertEqual(len(asr.cues_from_words(words)), 2)+ # 0.5 s apart, 0.4 s long: w13 ends at 6.9 s, w14 would stretch the cue to 7.4 s (> 7 s)+ long = [{"word": f"w{i}", "start": i * 0.5, "end": i * 0.5 + 0.4} for i in range(20)]+ self.assertEqual(asr.cues_from_words(long), [+ (0.0, 6.9, " ".join(f"w{i}" for i in range(14))),+ (7.0, 9.9, " ".join(f"w{i}" for i in range(14, 20)))])+++class ParakeetRoutingTests(unittest.TestCase):+ def test_auto_routes_by_language(self):+ which = mock.Mock(side_effect=lambda n: "/x/" + n if n == "parakeet-mlx" else None)+ sh_ = mock.Mock(which=which)+ self.assertEqual(asr.parakeet_route("auto", "en", "a.wav", sh_, subprocess)[0], ["parakeet-mlx", "parakeet.cpp"])+ # ISO 639-2, the spelling caption.py's --language also tags a muxed track with+ self.assertEqual(asr.parakeet_route("auto", "eng", "a.wav", sh_, subprocess)[0], ["parakeet-mlx", "parakeet.cpp"])+ self.assertEqual(asr.parakeet_route("auto", "fr", "a.wav", sh_, subprocess)[0], [])+ self.assertEqual(asr.parakeet_route("auto", "fr", "a.wav", sh_, subprocess).skipped, ("parakeet-mlx",))+ self.assertEqual(asr.parakeet_route("whisper.cpp", None, "a.wav", sh_, subprocess)[0], [])+ self.assertEqual(asr.parakeet_route("parakeet.cpp", "de", "a.wav", sh_, subprocess)[0], ["parakeet.cpp"])+ with mock.patch.object(asr, "detect_language", return_value="ja"):+ self.assertEqual(asr.parakeet_route("auto", None, "a.wav", sh_, subprocess)[0], [])+ with mock.patch.object(asr, "detect_language", return_value=None), \+ mock.patch.object(asr, "_whisper_ready", return_value=False):+ route = asr.parakeet_route("auto", None, "a.wav", sh_, subprocess)+ self.assertEqual(route.first, ["parakeet-mlx", "parakeet.cpp"])+ self.assertEqual(route.last, ())+ self.assertIn("assumed English", route.facts["routing"])++ def test_english_only_model_refuses_another_language(self):+ with mock.patch.dict(os.environ, {}, clear=False):+ os.environ.pop("PARAKEET_MODEL", None)+ with self.assertRaises(SystemExit):+ asr._parakeet_model_for("parakeet-mlx", None, "fr")+ self.assertEqual(asr._parakeet_model_for("parakeet-mlx", None, "eng"), asr.PARAKEET_MLX_DEFAULT_MODEL)+ self.assertIn("v3", asr._parakeet_model_for("parakeet-mlx", "mlx-community/parakeet-tdt-0.6b-v3", "fr"))+ self.assertEqual(asr._parakeet_model_for("parakeet-mlx", "large-v3-turbo", "en"), asr.PARAKEET_MLX_DEFAULT_MODEL)++ def test_unknown_engine_in_the_environment_is_refused(self):+ with mock.patch.dict(os.environ, {asr.ASR_ENGINE_ENV: "vosk"}):+ with self.assertRaises(SystemExit):+ asr.requested_engine(None)++ def test_engine_flag_beats_the_environment_which_beats_auto(self):+ with mock.patch.dict(os.environ, {}, clear=False):+ os.environ.pop(asr.ASR_ENGINE_ENV, None)+ self.assertEqual(asr.requested_engine(None), "auto")+ os.environ[asr.ASR_ENGINE_ENV] = "parakeet.cpp"+ self.assertEqual(asr.requested_engine(None), "parakeet.cpp")+ self.assertEqual(asr.requested_engine("whisper.cpp"), "whisper.cpp")+++class UndetectedLanguageTests(unittest.TestCase):+ """Review of #305: with no --language and nothing to detect one, auto handed the speech to the+ English-only Parakeet model and the only trace was `transcription.routing`. A Whisper engine+ that can run now goes first; only with none does Parakeet run on English assumed, and then+ the caller gets a top-level `notes` line saying the transcript may be wrong."""++ def setUp(self):+ self.mlx_only = mock.Mock(which=lambda n: "/x/parakeet-mlx" if n == "parakeet-mlx" else None)++ def test_a_runnable_whisper_goes_first_and_parakeet_waits(self):+ with mock.patch.object(asr, "detect_language", return_value=None), \+ mock.patch.object(asr, "_whisper_ready", return_value=True):+ route = asr.parakeet_route("auto", None, "a.wav", self.mlx_only, subprocess)+ self.assertEqual(route.first, [])+ self.assertEqual(route.last, asr.PARAKEET_ENGINES)+ self.assertEqual(route.facts, {"routing": asr.ROUTING_UNDETECTED_WHISPER})++ def test_only_parakeet_runs_on_english_assumed(self):+ with mock.patch.object(asr, "detect_language", return_value=None), \+ mock.patch.object(asr, "_whisper_ready", return_value=False):+ route = asr.parakeet_route("auto", None, "a.wav", self.mlx_only, subprocess)+ self.assertEqual(route.first, list(asr.PARAKEET_ENGINES))+ self.assertEqual(route.facts, {"routing": asr.ROUTING_ASSUMED_ENGLISH})++ def test_a_named_language_or_a_detected_one_never_asks_for_whisper(self):+ with mock.patch.object(asr, "_whisper_ready", side_effect=AssertionError("not consulted")):+ self.assertEqual(asr.parakeet_route("auto", "en", "a.wav", self.mlx_only, subprocess).first,+ list(asr.PARAKEET_ENGINES))+ with mock.patch.object(asr, "detect_language", return_value="en"):+ self.assertEqual(asr.parakeet_route("auto", None, "a.wav", self.mlx_only, subprocess).last, ())+ # a named engine runs as asked+ self.assertEqual(asr.parakeet_route("parakeet-mlx", None, "a.wav", self.mlx_only, subprocess),+ asr.Route(["parakeet-mlx"], {"routing": "requested"}))++ def test_whisper_ready_means_an_engine_auto_would_run(self):+ with tempfile.TemporaryDirectory() as d, mock.patch.dict(sys.modules, {"faster_whisper": None}):+ model = Path(d) / "ggml-base.bin"+ cli_only = mock.Mock(which=lambda n: "/x/whisper-cli" if n == "whisper-cli" else None)+ self.assertFalse(asr._whisper_ready(cli_only, str(Path(d) / "missing.bin")), "whisper-cli with no model")+ model.write_bytes(b"")+ self.assertTrue(asr._whisper_ready(cli_only, str(model)))+ self.assertFalse(asr._whisper_ready(cli_only, "mlx-community/parakeet-tdt-0.6b-v3"), "a Parakeet model")+ self.assertFalse(asr._whisper_ready(cli_only, str(Path(d) / "tdt-0.6b-v2-f16.gguf")))+ self.assertTrue(asr._whisper_ready(mock.Mock(which=lambda n: "/x/whisper" if n == "whisper" else None), "base"))+ self.assertFalse(asr._whisper_ready(self.mlx_only, "base"))+ stub = type(sys)("faster_whisper") # a module with no __spec__, as a test stub is+ with mock.patch.dict(sys.modules, {"faster_whisper": stub}):+ self.assertTrue(asr._whisper_ready(self.mlx_only, "base"))++ def test_the_note_names_the_callers_flag(self):+ self.assertIn("--filler-lang", asr.assumed_english_note("--filler-lang"))+ self.assertIn("install a Whisper engine", asr.assumed_english_note())+ self.assertIn("fix the Whisper engine", asr.assumed_english_note(whisper_failed=True))+++class _FakeEngines:+ """shutil / subprocess stand-ins for the bridge: `which` answers only for `names`, and `run`+ writes what the named engine would have written where its arguments say. An engine in `fail`+ exits 1 the way the real one does: parakeet-mlx prints its error to stdout (through rich, with+ a download's progress bar on stderr), the others to stderr. `calls` records (argv, env)."""++ MLX_ERROR = ("Error loading model mlx-community/parakeet-tdt-0.6b-v2: cannot reach\n"+ "huggingface.co (offline)\n\nparakeet-tdt-0.6b-v2 transcription complete.\n")++ def __init__(self, names, fail=()):+ self.names = set(names)+ self.fail = set(fail)+ self.calls = []++ def which(self, name):+ return f"/x/{name}" if name in self.names or name in ("ffmpeg", "ffprobe") else None++ def ran(self, exe):+ """The argv of every run of `exe`."""+ return [c for c, _ in self.calls if os.path.basename(c[0]) == exe]++ def run(self, cmd, **kw):+ self.calls.append((list(cmd), kw.get("env")))+ exe = os.path.basename(cmd[0])+ if exe in self.fail:+ if exe == "parakeet-mlx":+ return subprocess.CompletedProcess(cmd, 1, self.MLX_ERROR, "Fetching 5 files: 0%| | 0/5\n")+ return subprocess.CompletedProcess(cmd, 1, "", f"{exe}: failed to load model\n")+ if exe == "parakeet-mlx":+ out = cmd[cmd.index("--output-dir") + 1]+ os.makedirs(out, exist_ok=True)+ # the real CLI names its file by --output-template, else $PARAKEET_OUTPUT_TEMPLATE+ template = (kw["env"] if kw.get("env") is not None else os.environ).get("PARAKEET_OUTPUT_TEMPLATE") or "{filename}"+ name = template.format(filename="audio", parent=out, date="20261009", index="1")+ Path(out, name + ".json").write_text(json.dumps(MLX_DOC), encoding="utf-8")+ elif exe == "whisper-cli" and "-osrt" in cmd:+ Path(cmd[cmd.index("-of") + 1] + ".srt").write_text(+ "1\n00:00:00,000 --> 00:00:01,500\nBonjour tout le monde\n", encoding="utf-8")+ elif exe == "whisper-cli" and "--output-json-full" in cmd:+ Path(cmd[cmd.index("-of") + 1] + ".json").write_text(json.dumps({"transcription": [{"tokens": [+ {"text": " Bonjour", "offsets": {"from": 0, "to": 600}},+ {"text": " euh", "offsets": {"from": 700, "to": 900}}]}]}), encoding="utf-8")+ return subprocess.CompletedProcess(cmd, 0, "", "")+++class _BridgeHarness:+ """Runs the real bridge in process against _FakeEngines: no binary, no ffmpeg, every OS."""++ def _with(self, engines, call, faster_whisper=None, fail=(), env=None, fake=None):+ import _common+ fake = fake or _FakeEngines(engines, fail)+ stack = [mock.patch.object(_common, "run_analysis", lambda *a, **k: None),+ mock.patch.object(shutil, "which", fake.which),+ mock.patch.object(subprocess, "run", fake.run),+ # None makes `import faster_whisper` fail; a stub module stands in for an install+ mock.patch.dict(sys.modules, {"faster_whisper": faster_whisper}),+ # a host ggml model in ~/.cache/whisper.cpp must not change the model named below+ mock.patch.object(asr, "_whisper_cpp_model", lambda m: m),+ mock.patch.dict(os.environ, dict({"PARAKEET_MODEL": "", "PARAKEET_CPP_MODEL": "", asr.ASR_ENGINE_ENV: ""},+ **(env or {})))]+ for p in stack:+ p.start()+ try:+ return call()+ finally:+ for p in reversed(stack):+ p.stop()++ def _refusal(self, engines, call, **kw):+ """The one die() a bridge call ends in: (message, kind, extra fields)."""+ calls = []++ def fake_die(msg, code=1, kind="input", **extra):+ calls.append((msg, kind, extra))+ raise SystemExit(code)+ with mock.patch.object(asr, "die", fake_die), self.assertRaises(SystemExit):+ self._with(engines, call, **kw)+ self.assertEqual(len(calls), 1, calls)+ return calls[0]+++def _faster_whisper_with_words():+ """A faster_whisper that loads and returns one segment with word timings."""+ mod = type(sys)("faster_whisper")++ class Word:+ def __init__(self, word, start, end):+ self.word, self.start, self.end = word, start, end++ class Segment:+ start, end, text = 0.0, 0.9, " Bonjour euh"+ words = [Word(" Bonjour", 0.0, 0.6), Word(" euh", 0.7, 0.9)]++ class WhisperModel:+ def __init__(self, *a, **k):+ pass++ def transcribe(self, wav, language=None, word_timestamps=False):+ mod.languages.append(language)+ return iter([Segment()]), None+ mod.WhisperModel = WhisperModel+ mod.languages = []+ return mod+++def _crashing_faster_whisper():+ """A faster_whisper whose model cannot be loaded -- what an offline first run looks like."""+ mod = type(sys)("faster_whisper")++ class WhisperModel:+ def __init__(self, *a, **k):+ raise RuntimeError("cannot fetch Systran/faster-whisper-base: offline")+ mod.WhisperModel = WhisperModel+ return mod+++class WhisperFailureTests(_BridgeHarness, unittest.TestCase):+ """Undetected speech goes to Whisper first, so a Whisper engine that fails must read as a+ failure: a faster-whisper whose model would not load used to be swallowed in its thread and+ reported as "found no speech", which also kept Parakeet from ever being tried."""++ def test_a_crashed_faster_whisper_is_not_no_speech(self):+ """...and not "no engine found" either: it was found. The refusal names it and its own error."""+ with tempfile.TemporaryDirectory() as d:+ msg, kind, extra = self._refusal([], lambda: asr.transcribe_result("talk.mp4", os.path.join(d, "a.srt"), None, "base"),+ faster_whisper=_crashing_faster_whisper())+ self.assertEqual((kind, extra.get("reason"), extra.get("engine")), ("input", asr.ENGINE_FAILED_REASON, "faster-whisper"))+ self.assertNotIn("no local speech-to-text engine found", msg)+ self.assertNotIn("pip install", msg, "no install line for an engine that is installed")+ self.assertIn("faster-whisper found but failed: cannot fetch Systran/faster-whisper-base: offline", msg)+ self.assertEqual(extra["engines"], [{"engine": "faster-whisper", "reason": asr.ENGINE_FAILED_REASON,+ "detail": "found but failed: cannot fetch Systran/faster-whisper-base: offline"}])++ def test_parakeet_runs_last_after_a_crashed_faster_whisper(self):+ with tempfile.TemporaryDirectory() as d:+ run = self._with(["parakeet-mlx"], lambda: asr.transcribe_result("talk.mp4", os.path.join(d, "a.srt"), None, "base"),+ faster_whisper=_crashing_faster_whisper())+ self.assertEqual(run.engine, "parakeet-mlx")+ self.assertEqual(run.facts["routing"], asr.ROUTING_LAST_RESORT)+ self.assertEqual(run.notes, [asr.assumed_english_note(whisper_failed=True)])+ words = self._with(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4"),+ faster_whisper=_crashing_faster_whisper())+ self.assertEqual((words.engine, len(words.words)), ("parakeet-mlx", 5))+ self.assertEqual(words.notes, [asr.assumed_english_note("--filler-lang", whisper_failed=True)])++ def test_a_crashed_faster_whisper_word_run_is_still_refused_by_name_without_parakeet(self):+ """No Parakeet to fall back on: silence.py keeps its "<engine> ran but produced no+ word-level timings" refusal, naming faster-whisper, as before."""+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4"), faster_whisper=_crashing_faster_whisper())+ self.assertEqual((run.engine, run.words), ("faster-whisper", []))++ def test_a_word_run_whose_last_resort_parakeet_also_fails_names_every_engine(self):+ """Parakeet was tried after the Whisper engine failed: the refusal must name both, not+ hand back the Whisper failure as if Parakeet had never run (CodeRabbit on #312)."""+ msg, kind, extra = self._refusal(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4"),+ faster_whisper=_crashing_faster_whisper(), fail=("parakeet-mlx",))+ self.assertEqual((kind, extra.get("reason")), ("input", asr.ENGINE_FAILED_REASON))+ self.assertEqual([e["engine"] for e in extra["engines"]], ["faster-whisper", "parakeet-mlx"])+ self.assertIn("--filler --transcribe", msg)+++class TranscriptionStateTests(_BridgeHarness, unittest.TestCase):+ """Review of #305: the engine, routing and Parakeet word timings were handed back through+ module-level LAST_RUN / LAST_WORDS, which every call in one process shared (mcp/server.py,+ batch.py, a test). The calls now return them; nothing is kept in the module."""++ def test_two_transcriptions_in_one_process_do_not_share_state(self):+ self.assertFalse(hasattr(asr, "LAST_RUN") or hasattr(asr, "LAST_WORDS"),+ "module-level hand-off state is back")+ with tempfile.TemporaryDirectory() as d:+ srt = os.path.join(d, "a.srt")+ first = self._with(["parakeet-mlx"], lambda: asr.transcribe_result("talk.mp4", srt, "en", "base"))+ snapshot = (first.engine, [dict(w) for w in first.words], dict(first.facts), list(first.cues))+ self.assertEqual(first.engine, "parakeet-mlx")+ self.assertEqual([w["word"] for w in first.words], ["So", "um", "we", "start.", "Here."])+ self.assertEqual(first.facts, {"routing": "auto: --language is English", "engine": "parakeet-mlx",+ "model": asr.PARAKEET_MLX_DEFAULT_MODEL, "language": "en"})++ # a whisper caption run after it: no Parakeet words, model or routing carried over+ second = self._with(["whisper-cli"], lambda: asr.transcribe_result("talk.mp4", srt, None, "base",+ engine="whisper.cpp"))+ self.assertEqual(second.engine, "whisper.cpp")+ self.assertEqual(second.words, [])+ self.assertEqual(second.cues, [(0.0, 1.5, "Bonjour tout le monde")])+ self.assertEqual(second.facts, {"routing": "requested", "engine": "whisper.cpp", "model": "base", "language": None})+ self.assertEqual(second.notes, [])++ # a word run in between and after: its own words, engine and facts only+ third = self._with(["whisper-cli", "parakeet-mlx"],+ lambda: asr.transcribe_words_result("talk.mp4", "fr", engine="whisper.cpp"))+ self.assertEqual([w["word"] for w in third.words], ["Bonjour", "euh"])+ self.assertEqual(third.facts["language"], "fr")++ # and the earlier results are still exactly what their own calls returned+ self.assertEqual((first.engine, first.words, first.facts, first.cues), snapshot)+ self.assertEqual(second.words, [])++ def test_the_old_call_shapes_still_answer(self):+ """transcribe() -> cues and transcribe_words() -> (words, engine): what callers had."""+ with tempfile.TemporaryDirectory() as d:+ cues = self._with(["parakeet-mlx"], lambda: asr.transcribe("talk.mp4", os.path.join(d, "a.srt"), "en", "base"))+ self.assertEqual(cues, [(0.2, 1.6, "So um we start."), (2.0, 2.4, "Here.")])+ words, engine = self._with(["parakeet-mlx"], lambda: asr.transcribe_words("talk.mp4", "en"))+ self.assertEqual((len(words), engine), (5, "parakeet-mlx"))+++class _SrtDir:+ def setUp(self):+ self._dir = tempfile.TemporaryDirectory()+ self.srt = os.path.join(self._dir.name, "a.srt")++ def tearDown(self):+ self._dir.cleanup()+++class EngineRefusalTests(_SrtDir, _BridgeHarness, unittest.TestCase):+ """Review of #305: "no local speech-to-text engine found", with its install lines, was the refusal+ whenever nothing transcribed -- also when an installed engine had failed (a whisper.cpp with+ nothing after it, as in 2.5.1) or when auto had passed over an installed Parakeet engine for a+ language that is not English. It now means only that no engine was found; the others name each+ engine and why. A named engine is kind missing_tool only when it is not installed."""++ def test_nothing_installed_is_still_no_engine(self):+ msg, kind, _ = self._refusal([], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"))+ self.assertTrue(msg.startswith("no local speech-to-text engine found for --transcribe."), msg)+ self.assertEqual(kind, "input")+ msg, kind, _ = self._refusal([], lambda: asr.transcribe_words_result("talk.mp4"))+ self.assertTrue(msg.startswith("no local speech-to-text engine found for --filler --transcribe."), msg)++ def test_parakeet_passed_over_for_another_language_is_named(self):+ msg, kind, extra = self._refusal(["parakeet-mlx"], lambda: asr.transcribe_result("talk.mp4", self.srt, "fr", "base"))+ self.assertEqual((kind, extra["reason"], extra["engine"]), ("input", asr.ENGLISH_ONLY_REASON, "parakeet-mlx"))+ self.assertNotIn("no local speech-to-text engine found", msg)+ self.assertNotIn("uv tool install parakeet-mlx", msg, "it is installed")+ self.assertIn("parakeet-mlx is installed, but --engine auto runs Parakeet only for English speech, "+ "and --language fr is not English", msg)+ self.assertIn("Whisper engine", extra["hint"])+ msg, _, extra = self._refusal(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4", "fr"))+ self.assertEqual(extra["reason"], asr.ENGLISH_ONLY_REASON)+ self.assertIn("--filler-lang fr is not English", msg)+ with mock.patch.object(asr, "detect_language", return_value="ja"):+ msg, _, _ = self._refusal(["parakeet-mlx"], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"))+ self.assertIn("the detected language ja is not English", msg)++ def test_a_failed_whisper_cpp_with_nothing_after_it_is_named(self):+ """2.5.1 told the caller to install whisper.cpp when whisper.cpp itself had failed."""+ msg, kind, extra = self._refusal(["whisper-cli"], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"),+ fail=["whisper-cli"])+ self.assertEqual((kind, extra["reason"], extra["engine"]), ("input", asr.ENGINE_FAILED_REASON, "whisper.cpp"))+ self.assertIn("whisper.cpp found but failed: whisper-cli: failed to load model", msg)+ self.assertNotIn("no local speech-to-text engine found", msg)++ def test_a_named_engine_that_is_installed_and_failed_is_not_missing_tool(self):+ """kind missing_tool (DEPENDENCY_MISSING, "install it") only when the engine is not there; the+ engine's own error is in the document -- parakeet-mlx prints it to stdout, not stderr."""+ msg, kind, extra = self._refusal(+ ["parakeet-mlx"], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base", engine="parakeet-mlx"),+ fail=["parakeet-mlx"])+ self.assertEqual((kind, extra["reason"], extra["engine"]), ("input", asr.ENGINE_FAILED_REASON, "parakeet-mlx"))+ self.assertEqual(extra["detail"], "found but failed: Error loading model mlx-community/parakeet-tdt-0.6b-v2: "+ "cannot reach huggingface.co (offline)")+ self.assertIn(extra["detail"], msg)+ _, kind, extra = self._refusal(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4", engine="parakeet-mlx"),+ fail=["parakeet-mlx"])+ self.assertEqual((kind, extra["reason"]), ("input", asr.ENGINE_FAILED_REASON))+ _, kind, extra = self._refusal(+ ["whisper-cli"], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base", engine="whisper.cpp"),+ fail=["whisper-cli"])+ self.assertEqual((kind, extra["engine"]), ("input", "whisper.cpp"))+ self.assertIn("failed to load model", extra["detail"])++ def test_a_named_engine_that_is_not_installed_is_missing_tool(self):+ for engine in asr.ASR_ENGINES:+ msg, kind, _ = self._refusal([], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base", engine=engine))+ self.assertEqual(kind, "missing_tool", engine)+ self.assertIn(f"--engine {engine} is not installed", msg)+ _, kind, _ = self._refusal([], lambda: asr.transcribe_words_result("talk.mp4", engine=engine))+ self.assertEqual(kind, "missing_tool", engine)++ def test_the_failure_line_reads_parakeet_mlx_errors_from_stdout(self):+ proc = subprocess.CompletedProcess([], 1, _FakeEngines.MLX_ERROR, "Fetching 5 files: 0%|\n")+ self.assertEqual(asr._failure_line(proc, errors_on_stdout=True),+ "Error loading model mlx-community/parakeet-tdt-0.6b-v2: cannot reach huggingface.co (offline)")+ self.assertEqual(asr._failure_line(proc), "Fetching 5 files: 0%|")+ self.assertEqual(asr._failure_line(subprocess.CompletedProcess([], 1, "only stdout\n", "")), "only stdout")+ self.assertEqual(asr._failure_line(subprocess.CompletedProcess([], 1, "", "")), "?")+++class WhisperLanguageTests(_SrtDir, _BridgeHarness, unittest.TestCase):+ """Review of #305: whisper.cpp was given -l only with --language, and its own default is -l en,+ so it decoded any speech as English -- on main for every run without --language, on the route+ that sends undetected speech to Whisper because Parakeet is English-only, and after auto's+ detector had named the language. It is now told the language, or "auto"."""++ @staticmethod+ def _l(fake):+ argv = fake.ran("whisper-cli")[-1]+ return argv[argv.index("-l") + 1]++ def test_no_language_asks_whisper_cpp_to_detect_it(self):+ for call in (lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"),+ lambda: asr.transcribe_words_result("talk.mp4")):+ fake = _FakeEngines(["whisper-cli"])+ run = self._with([], call, fake=fake)+ self.assertEqual((run.engine, self._l(fake)), ("whisper.cpp", "auto"))+ self.assertIsNone(run.facts["language"], "whisper.cpp detected it; nothing named it")++ def test_the_detected_language_is_passed_on(self):+ fake = _FakeEngines(["whisper-cli", "parakeet-mlx"])+ with mock.patch.object(asr, "detect_language", return_value="ja"):+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, None, "base"), fake=fake)+ self.assertEqual(self._l(fake), "ja")+ self.assertEqual(run.facts, {"routing": "auto: detected ja, not English", "detected_language": "ja",+ "engine": "whisper.cpp", "model": "base", "language": "ja"})+ self.assertEqual(fake.ran("parakeet-mlx"), [])+ fw = _faster_whisper_with_words()+ run = self._with(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4"), faster_whisper=fw)+ self.assertEqual((run.engine, fw.languages, run.facts["language"]), ("faster-whisper", ["ja"], "ja"))++ def test_undetected_speech_routed_to_whisper_lets_whisper_cpp_detect(self):+ fake = _FakeEngines(["whisper-cli", "parakeet-mlx"])+ with mock.patch.object(asr, "detect_language", return_value=None), \+ mock.patch.object(asr, "_whisper_ready", return_value=True):+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4"), fake=fake)+ self.assertEqual((run.facts["routing"], self._l(fake)), (asr.ROUTING_UNDETECTED_WHISPER, "auto"))++ def test_a_named_language_is_passed_as_given(self):+ fake = _FakeEngines(["whisper-cli"])+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "fr", "base"), fake=fake)+ self.assertEqual((self._l(fake), run.facts["language"]), ("fr", "fr"))++ def test_an_english_only_whisper_model_is_no_engine_for_any_language(self):+ cli = mock.Mock(which=lambda n: "/x/whisper-cli" if n == "whisper-cli" else None)+ openai = mock.Mock(which=lambda n: "/x/whisper" if n == "whisper" else None)+ with tempfile.TemporaryDirectory() as d, mock.patch.dict(sys.modules, {"faster_whisper": None}):+ en = Path(d) / "ggml-base.en.bin"+ en.write_bytes(b"")+ self.assertFalse(asr._whisper_ready(cli, str(en)))+ self.assertFalse(asr._whisper_ready(openai, "base.en"))+ self.assertTrue(asr._whisper_ready(openai, "base"))+++class AutoRoutingFallbackTests(_SrtDir, _BridgeHarness, unittest.TestCase):+ """Review of #305: what --engine auto does when its first choice fails, and what it then says."""++ def test_whisper_after_a_failed_parakeet_says_why_it_ran(self):+ """The routing said why Parakeet was chosen, next to engine whisper.cpp."""+ fake = _FakeEngines(["parakeet-mlx", "whisper-cli"], fail=["parakeet-mlx"])+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "en", "base"), fake=fake)+ self.assertEqual(run.engine, "whisper.cpp")+ self.assertEqual(run.facts["routing"], "auto: --language is English, no Parakeet engine transcribed it")+ words = self._with(["parakeet-mlx", "whisper-cli"], lambda: asr.transcribe_words_result("talk.mp4", "en"),+ fail=["parakeet-mlx"])+ self.assertEqual((words.engine, words.facts["routing"]),+ ("whisper.cpp", "auto: --language is English" + asr.ROUTING_PARAKEET_FAILED))++ def test_undetected_speech_reaches_faster_whisper_after_a_failed_whisper_cpp(self):+ """_whisper_ready() counted faster-whisper, but the word path stopped at the failed+ whisper.cpp and ran Parakeet on English assumed, saying no Whisper engine transcribed it."""+ fake = _FakeEngines(["whisper-cli", "parakeet-mlx"], fail=["whisper-cli"])+ run = self._with([], lambda: asr.transcribe_words_result("talk.mp4"), faster_whisper=_faster_whisper_with_words(),+ fake=fake)+ self.assertEqual(run.engine, "faster-whisper")+ self.assertEqual([w["word"] for w in run.words], ["Bonjour", "euh"])+ self.assertEqual(run.facts["routing"], asr.ROUTING_UNDETECTED_WHISPER)+ self.assertEqual((run.notes, fake.ran("parakeet-mlx")), ([], []))++ def test_without_parakeet_a_failed_whisper_cpp_word_run_is_named_as_before(self):+ """No routing decision rests on faster-whisper here: the engine that ran and failed is the+ one silence.py's "no word-level timings" refusal names, as in 2.5.1."""+ run = self._with(["whisper-cli"], lambda: asr.transcribe_words_result("talk.mp4"),+ faster_whisper=_faster_whisper_with_words(), fail=["whisper-cli"])+ self.assertEqual((run.engine, run.words), ("whisper.cpp", []))++ def test_a_multilingual_parakeet_model_assumes_no_english(self):+ v3 = "mlx-community/parakeet-tdt-0.6b-v3"+ run = self._with(["parakeet-mlx", "whisper-cli"], lambda: asr.transcribe_result("talk.mp4", self.srt, None, v3))+ self.assertEqual((run.engine, run.facts["model"]), ("parakeet-mlx", v3))+ self.assertEqual(run.facts["routing"], "auto: --model names a Parakeet model and the language is not detectable "+ "here; the Parakeet model is multilingual")+ self.assertEqual(run.notes, [])+ # the model from PARAKEET_MODEL, no Whisper engine: nothing assumed either+ run = self._with(["parakeet-mlx"], lambda: asr.transcribe_words_result("talk.mp4"), env={"PARAKEET_MODEL": v3})+ self.assertEqual(run.facts["model"], v3)+ self.assertNotIn("assumed English", run.facts["routing"])+ self.assertEqual(run.notes, [])++ def test_an_english_only_parakeet_model_named_with_model_points_at_the_model(self):+ """Not "no Whisper engine ... install one": a Whisper engine cannot load a Parakeet model."""+ run = self._with(["parakeet-mlx", "whisper-cli"],+ lambda: asr.transcribe_result("talk.mp4", self.srt, None, asr.PARAKEET_MLX_DEFAULT_MODEL))+ self.assertEqual(run.facts["routing"], asr.ROUTING_PARAKEET_MODEL)+ self.assertEqual(run.notes, [asr.assumed_english_note(parakeet_model=True)])+ self.assertNotIn("install a Whisper engine", run.notes[0])++ def test_a_host_output_template_does_not_hide_parakeet_mlx_output(self):+ """parakeet-mlx names its file by $PARAKEET_OUTPUT_TEMPLATE; the bridge reads audio.json."""+ fake = _FakeEngines(["parakeet-mlx"])+ run = self._with([], lambda: asr.transcribe_result("talk.mp4", self.srt, "en", "base"), fake=fake,+ env={"PARAKEET_OUTPUT_TEMPLATE": "{filename}_{date}"})+ self.assertEqual(run.engine, "parakeet-mlx")+ (_, env), = fake.calls+ self.assertNotIn("PARAKEET_OUTPUT_TEMPLATE", env)+++class InstallHintTests(unittest.TestCase):+ """Review of #305: parakeet-mlx fetches its default model from Hugging Face on first use. The+ hints say so, and say the audio stays on the machine, so "no cloud" stays a true sentence."""++ def test_the_refusal_hint_names_the_first_run_download_and_keeps_audio_local(self):+ hint = asr.ASR_INSTALL_HINT+ line = next(ln for ln in hint.splitlines() if ln.strip().startswith("parakeet-mlx:"))+ rest = hint[hint.index(line):].split("parakeet.cpp:")[0]+ self.assertIn("first run", rest)+ self.assertIn("Hugging Face", rest)+ self.assertIn(asr.PARAKEET_MLX_DEFAULT_MODEL, rest)+ self.assertIn("never leaves", hint.splitlines()[0])+ self.assertNotIn("offline", hint, "a first run that downloads is not offline")++ def test_the_capability_fix_hint_says_the_same(self):+ """_capability_fix_hint() is the `fix` doctor gives for a missing *required* capability; no+ tool requires a speech engine today, so doctor lists external:* under missing_optional+ without it. The text is still kept true for the day one does."""+ import _contract+ fix = _contract._capability_fix_hint("external:parakeet")+ for words in ("first run", "Hugging Face", asr.PARAKEET_MLX_DEFAULT_MODEL, "never uploaded"):+ self.assertIn(words, fix)++ def test_the_network_claim_matches_where_each_engine_runs(self):+ """Review of #305: "the skill itself opens no network connection" was false for faster-whisper,+ a library the skill runs in its own process, whose first run downloads its model. And+ "doctor's fix line" was named as a place that says so, which doctor never shows for an+ optional capability."""+ import _contract+ root = Path(__file__).resolve().parent.parent+ contract_md = (root / "docs" / "contract.md").read_text(encoding="utf-8")+ decisions = (root / "docs" / "design-decisions.md").read_text(encoding="utf-8")+ self.assertNotIn("skill itself opens no network connection", contract_md)+ self.assertIn("faster-whisper is a Python library the skill runs inside its own process", " ".join(contract_md.split()))+ self.assertNotIn("(the skill's own process) stay true", decisions)+ for text in (decisions, (root / "CHANGELOG.md").read_text(encoding="utf-8")):+ self.assertNotIn("doctor's fix line", text)+ self.assertNotIn("`doctor`'s fix line", text)+ # the engines it runs are listed where the contract says what it runs+ execution = _contract.build(detect=False)["execution"]+ ran = execution["subprocess"]+ for exe in ("whisper-cli", "whisper-cpp", "parakeet-mlx", "parakeet-cli", "whisper"):+ self.assertIn(exe, ran)+ # ...and the machine-readable contract says which engine downloads, from which process: a+ # caller that read only `network: false` took a first faster-whisper run for network-free+ # (CodeRabbit on #312)+ downloads = execution["model_downloads"]+ self.assertEqual(set(downloads["in_process"]), {"faster-whisper"})+ self.assertEqual(set(downloads["child_process"]), {"parakeet-mlx", "openai-whisper"})+ self.assertEqual(set(downloads["none"]), {"whisper.cpp", "parakeet.cpp"})+ self.assertIn("model_downloads", contract_md)++ def test_the_reference_page_no_longer_says_nothing_is_downloaded(self):+ root = Path(__file__).resolve().parent.parent+ page = (root / "references" / "scripts.md").read_text(encoding="utf-8")+ section = page[page.index("### caption.py --transcribe"):]+ section = section[:section.index("\n### ", 5)]+ self.assertNotIn("Nothing is downloaded", section)+ for words in ("Hugging Face", "never uploaded", "first time it runs"):+ self.assertIn(words, section)++[email protected](platform.system() == "Windows", "the fake engines are #! scripts on a POSIX PATH shim; the routing "+ "they drive is covered on every OS by the in-process tests above")+class ParakeetEngineTests(MediaFixtures):+ """caption.py / silence.py drive fake Parakeet binaries on a PATH that holds only them and ffmpeg."""++ @classmethod+ def setUpClass(cls):+ super().setUpClass()+ cls.bin = Path(tempfile.mkdtemp(prefix="ffskill_fakeasr_"))+ for tool in ("ffmpeg", "ffprobe", "fc-match", "fc-list"):+ if shutil.which(tool):+ os.symlink(shutil.which(tool), cls.bin / tool)+ mlx = """+ import json, os, sys+ a = sys.argv[1:]; out = a[a.index("--output-dir") + 1]; os.makedirs(out, exist_ok=True)+ json.dump(DOC, open(os.path.join(out, os.path.splitext(os.path.basename(a[0]))[0] + ".json"), "w"))+ """+ cpp = """+ import json, sys+ assert sys.argv[1] == "transcribe" and "--json" in sys.argv+ print(json.dumps(DOC))+ """+ for name, body, doc in (("parakeet-mlx", mlx, MLX_DOC), ("parakeet-cli", cpp, CPP_DOC)):+ p = cls.bin / name+ p.write_text(f"#!{sys.executable}\nDOC = {doc!r}\n" + textwrap.dedent(body))+ p.chmod(p.stat().st_mode | stat.S_IXUSR)+ cls.gguf = cls.bin / "tdt-0.6b-v2-f16.gguf"+ cls.gguf.write_bytes(b"GGUF")+ # A PATH hides the host's engine binaries, but not a Python module: the tools would import a+ # faster_whisper the host has installed (a Mac where Parakeet is used often has one) and+ # Whisper would then go first. This sitecustomize makes `import faster_whisper` fail in+ # every subprocess, as on CI.+ cls.shim = Path(tempfile.mkdtemp(prefix="ffskill_nofw_"))+ (cls.shim / "sitecustomize.py").write_text("import sys\nsys.modules['faster_whisper'] = None\n", encoding="utf-8")++ @classmethod+ def tearDownClass(cls):+ shutil.rmtree(cls.bin, ignore_errors=True)+ shutil.rmtree(cls.shim, ignore_errors=True)+ super().tearDownClass()++ def env(self, **extra):+ e = dict(os.environ, PATH=f"{self.bin}{os.pathsep}/usr/bin{os.pathsep}/bin", PARAKEET_CPP_MODEL=str(self.gguf),+ PYTHONPATH=os.pathsep.join(p for p in (str(self.shim), os.environ.get("PYTHONPATH")) if p))+ e.update(extra)+ return e++ def test_a_host_faster_whisper_is_hidden_from_the_tools(self):+ """With a faster_whisper importable on the host (a stub package on PYTHONPATH stands in for+ one), the tools still cannot import it."""+ stub = Path(tempfile.mkdtemp(prefix="ffskill_fwstub_"))+ try:+ (stub / "faster_whisper").mkdir()+ (stub / "faster_whisper" / "__init__.py").write_text("class WhisperModel:\n pass\n", encoding="utf-8")+ env = self.env()+ env["PYTHONPATH"] += os.pathsep + str(stub)+ probe = sh(sys.executable, "-c", "import sys; sys.path.insert(0, sys.argv[1]); from _common import asr; "+ "print(asr._module_importable('faster_whisper'))", str(Path(__file__).resolve().parent.parent / "scripts"),+ env=env)+ finally:+ shutil.rmtree(stub, ignore_errors=True)+ self.assertEqual(probe.stdout.strip(), "False")++ def test_caption_auto_picks_parakeet_for_english_and_reports_it(self):+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--language", "en", "--fast", "--json",+ "-o", OUT / "pk_auto.mp4", env=self.env()).stdout)+ self.assertEqual(doc["transcription"]["engine"], "parakeet-mlx")+ self.assertEqual(doc["transcription"]["model"], asr.PARAKEET_MLX_DEFAULT_MODEL)+ self.assertIn("So um we start.", (OUT / "pk_auto.srt").read_text())+ self.assertFalse(_assumed(doc), "English was named, not assumed")++ def test_caption_engine_flag_picks_parakeet_cpp(self):+ """One end-to-end parakeet.cpp run; flag-over-env precedence is a unit test above."""+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--engine", "parakeet.cpp", "--fast", "--json",+ "-o", OUT / "pk_cpp.mp4", env=self.env()).stdout)+ self.assertEqual(doc["transcription"]["engine"], "parakeet.cpp")+ self.assertIn("So um we start.", (OUT / "pk_cpp.srt").read_text())+ # a named engine runs as asked, with no language: no routing decision, so no warning+ self.assertEqual(doc["transcription"]["routing"], "requested")+ self.assertFalse(_assumed(doc))++ def test_caption_mux_reports_the_transcription_too(self):+ """--mode mux wrote a soft subtitle track from the transcript but left `transcription` out of+ its result, so a caller could not tell which engine made it."""+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--engine", "parakeet.cpp", "--mode", "mux",+ "--json", "-o", OUT / "pk_mux.mp4", env=self.env()).stdout)+ self.assertEqual(doc["subtitle_tracks"], 1)+ self.assertEqual(doc["transcription"]["engine"], "parakeet.cpp")++ def test_caption_takes_the_engine_from_the_environment(self):+ """$FFMPEG_SKILL_ASR_ENGINE reaches caption.py through its real parser (no --engine given)."""+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--fast", "--json", "-o", OUT / "pk_cpp_env.mp4",+ env=self.env(FFMPEG_SKILL_ASR_ENGINE="parakeet.cpp")).stdout)+ self.assertEqual(doc["transcription"]["engine"], "parakeet.cpp")++ def _run_with_mlx_writing(self, body, tool, *args, expect_fail=False):+ """`tool` with a parakeet-mlx that exits 0 after writing `body` as its JSON."""+ fake = Path(tempfile.mkdtemp(prefix="ffskill_oddasr_"))+ try:+ for name in os.listdir(self.bin):+ if name != "parakeet-mlx":+ os.symlink(self.bin / name, fake / name)+ mlx = fake / "parakeet-mlx"+ mlx.write_text(f"#!{sys.executable}\nimport os, sys\na = sys.argv[1:]; out = a[a.index('--output-dir') + 1]\n"+ "os.makedirs(out, exist_ok=True)\n"+ "open(os.path.join(out, os.path.splitext(os.path.basename(a[0]))[0] + '.json'), 'w')"+ f".write({body!r})\n")+ mlx.chmod(0o755)+ proc = script(tool, self.src, *args, "--json", env=self.env(PATH=str(fake)), expect_fail=expect_fail)+ finally:+ shutil.rmtree(fake, ignore_errors=True)+ return json.loads(proc.stdout)++ def test_unreadable_engine_output_falls_through_instead_of_claiming_silence(self):+ """parakeet-mlx exited 0 but wrote something that is not a transcript: that is a failed+ run, so auto moves on to parakeet.cpp instead of reporting "no speech" in the video."""+ doc = self._run_with_mlx_writing('{"sentences": 5}', "caption.py", "--transcribe", "--fast", "-o", OUT / "pk_garbage.mp4")+ self.assertEqual(doc["transcription"]["engine"], "parakeet.cpp")++ def test_filler_words_fall_through_an_engine_with_sentences_but_no_word_times(self):+ """Sentences whose text and times read but whose tokens do not: cues for a caption, but no+ word timings, which is all silence.py --filler needs -- so the next engine is tried."""+ body = json.dumps({"text": "So um we start.", "sentences": [{"text": "So um we start.", "start": 0.2, "end": 1.6,+ "tokens": [{"text": " So"}, 5]}]})+ doc = self._run_with_mlx_writing(body, "silence.py", "--filler", "--transcribe", "--filler-list")+ self.assertEqual(doc["filler"]["source"], "parakeet:parakeet.cpp")+ self.assertEqual([r["word"] for r in doc["filler"]["removed"]], ["um"])++ def test_a_named_engine_with_no_word_times_is_refused_by_name(self):+ """--engine parakeet-mlx ran and gave no word timings: say that, not "install it"."""+ body = json.dumps({"text": "So um we start.", "sentences": [{"text": "So um we start.", "start": 0.2, "end": 1.6,+ "tokens": [{"text": " So"}, 5]}]})+ doc = self._run_with_mlx_writing(body, "silence.py", "--filler", "--transcribe", "--filler-list",+ "--engine", "parakeet-mlx", expect_fail=True)+ self.assertEqual(doc["error"]["kind"], "input")+ self.assertIn("parakeet-mlx ran but produced no word-level timings", doc["error"]["message"])++ def test_an_engine_that_heard_nothing_still_reports_no_speech(self):+ """What both real engines write for 3 s of silence (measured): an empty list. That is an+ answer, not a failure, so it is still the no-speech refusal and nothing else is tried."""+ doc = self._run_with_mlx_writing('{"text": "", "sentences": []}', "caption.py", "--transcribe", "--fast",+ "-o", OUT / "pk_silent.mp4", expect_fail=True)+ self.assertEqual((doc["error"]["kind"], doc.get("reason"), doc.get("engine")), ("input", "no_speech", "parakeet-mlx"))++ def test_auto_falls_through_a_failing_parakeet_mlx_to_parakeet_cpp(self):+ """parakeet-mlx is installed but crashes: auto moves on to the next Parakeet engine."""+ broken = Path(tempfile.mkdtemp(prefix="ffskill_brokenasr_"))+ try:+ for name in os.listdir(self.bin):+ if name != "parakeet-mlx":+ os.symlink(self.bin / name, broken / name)+ ran = broken / "mlx-ran"+ mlx = broken / "parakeet-mlx"+ mlx.write_text(f"#!/bin/sh\n: > '{ran}'\necho 'Metal device lost' >&2\nexit 3\n")+ mlx.chmod(0o755)+ # only the fakes and ffmpeg on PATH: no host whisper can detect a language or take over,+ # so auto assumes English and the order is parakeet-mlx, then parakeet.cpp+ doc = json.loads(script("silence.py", self.src, "--filler", "--transcribe", "--filler-list", "--json",+ env=self.env(PATH=str(broken))).stdout)+ mlx_ran = ran.exists()+ finally:+ shutil.rmtree(broken, ignore_errors=True)+ self.assertTrue(mlx_ran, "the crashing parakeet-mlx was never run, so nothing fell through")+ self.assertEqual(doc["filler"]["source"], "parakeet:parakeet.cpp")+ self.assertEqual([r["word"] for r in doc["filler"]["removed"]], ["um"])++ def _with_whisper_cli(self, fail):+ """A PATH holding the fake Parakeet engines, ffmpeg and a fake whisper-cli whose language+ detector names nothing; it transcribes ("Bonjour tout le monde") unless `fail`. HOME+ holds the ggml-base.bin it is run with. Returns (PATH dir, HOME dir); the caller removes both."""+ d = Path(tempfile.mkdtemp(prefix="ffskill_whisperasr_"))+ home = Path(tempfile.mkdtemp(prefix="ffskill_whisperhome_"))+ for name in os.listdir(self.bin):+ os.symlink(self.bin / name, d / name)+ (home / ".cache" / "whisper.cpp").mkdir(parents=True)+ (home / ".cache" / "whisper.cpp" / "ggml-base.bin").write_bytes(b"")+ cli = d / "whisper-cli"+ cli.write_text(f"#!{sys.executable}\nFAIL = {fail!r}\nARGS = {str(home / 'whisper-args.json')!r}\n" + textwrap.dedent("""+ import json, sys+ a = sys.argv[1:]+ if "-dl" not in a:+ json.dump(a, open(ARGS, "w"))+ if "-dl" in a:+ print("whisper_full_with_state: no language line in this build", file=sys.stderr)+ sys.exit(0)+ if FAIL:+ print("whisper_init_from_file_with_params: failed to load model", file=sys.stderr)+ sys.exit(1)+ of = a[a.index("-of") + 1]+ if "-osrt" in a:+ open(of + ".srt", "w").write("1\\n00:00:00,000 --> 00:00:01,500\\nBonjour tout le monde\\n")+ if "--output-json-full" in a:+ json.dump({"transcription": [{"tokens": [{"text": " Bonjour", "offsets": {"from": 0, "to": 600}},+ {"text": " euh", "offsets": {"from": 700, "to": 900}}]}]},+ open(of + ".json", "w"))+ """))+ cli.chmod(0o755)+ return d, home++ def test_no_language_and_only_parakeet_says_english_was_assumed(self):+ """Review of #305: a Mac with parakeet-mlx and no Whisper captioning speech nobody named.+ Parakeet still runs, but the result says, at the top level, that English was assumed."""+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--fast", "--json",+ "-o", OUT / "pk_assumed.mp4", env=self.env(PATH=str(self.bin))).stdout)+ self.assertEqual(doc["transcription"]["engine"], "parakeet-mlx")+ self.assertEqual(doc["transcription"]["routing"], asr.ROUTING_ASSUMED_ENGLISH)+ self.assertEqual(len(_assumed(doc)), 1, doc.get("notes"))+ self.assertIn("--language", _assumed(doc)[0])+ self.assertTrue(doc["verified"], "the render itself is still verified; the note carries the doubt")+ fil = json.loads(script("silence.py", self.src, "--filler", "--transcribe", "--filler-list", "--json",+ env=self.env(PATH=str(self.bin))).stdout)+ self.assertEqual(fil["filler"]["source"], "parakeet:parakeet-mlx")+ self.assertEqual(len(_assumed(fil)), 1, fil.get("notes"))+ self.assertIn("--filler-lang", _assumed(fil)[0])++ def test_no_language_goes_to_whisper_before_parakeet(self):+ """The same machine with whisper.cpp installed but no language it can detect: Whisper,+ which handles any language, takes the speech; the English-only model is not trusted with it."""+ d, home = self._with_whisper_cli(fail=False)+ args = home / "whisper-args.json"+ try:+ env = self.env(PATH=str(d), HOME=str(home))+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--mode", "mux", "--json",+ "-o", OUT / "pk_whisperfirst.mp4", env=env).stdout)+ caption_args = json.loads(args.read_text())+ fil = json.loads(script("silence.py", self.src, "--filler", "--transcribe", "--filler-list", "--json",+ env=env).stdout)+ filler_args = json.loads(args.read_text())+ finally:+ shutil.rmtree(d, ignore_errors=True)+ shutil.rmtree(home, ignore_errors=True)+ self.assertEqual(doc["transcription"], {"routing": asr.ROUTING_UNDETECTED_WHISPER, "engine": "whisper.cpp",+ "model": str(home / ".cache" / "whisper.cpp" / "ggml-base.bin"),+ "language": None})+ # whisper.cpp's own default is -l en: "handles any language" only holds when it is told to detect+ for argv in (caption_args, filler_args):+ self.assertEqual(argv[argv.index("-l") + 1], "auto", argv)+ self.assertIn("Bonjour tout le monde", (OUT / "pk_whisperfirst.srt").read_text())+ self.assertFalse(_assumed(doc))+ self.assertEqual(fil["filler"]["source"], "whisper:whisper.cpp")+ self.assertEqual(fil["filler"]["transcription"]["routing"], asr.ROUTING_UNDETECTED_WHISPER)+ self.assertNotIn("notes", fil)++ def test_parakeet_runs_last_when_every_whisper_engine_fails(self):+ """Whisper went first and failed: Parakeet is still better than refusing, on English+ assumed, and the note says which fix applies."""+ d, home = self._with_whisper_cli(fail=True)+ try:+ env = self.env(PATH=str(d), HOME=str(home))+ doc = json.loads(script("caption.py", self.src, "--transcribe", "--mode", "mux", "--json",+ "-o", OUT / "pk_lastresort.mp4", env=env).stdout)+ fil = json.loads(script("silence.py", self.src, "--filler", "--transcribe", "--filler-list", "--json",+ env=env).stdout)+ finally:+ shutil.rmtree(d, ignore_errors=True)+ shutil.rmtree(home, ignore_errors=True)+ self.assertEqual(doc["transcription"]["engine"], "parakeet-mlx")+ self.assertEqual(doc["transcription"]["routing"], asr.ROUTING_LAST_RESORT)+ self.assertEqual(len(_assumed(doc)), 1, doc.get("notes"))+ self.assertIn("fix the Whisper engine", _assumed(doc)[0])+ self.assertEqual(fil["filler"]["source"], "parakeet:parakeet-mlx")+ self.assertEqual(fil["filler"]["transcription"]["routing"], asr.ROUTING_LAST_RESORT)+ self.assertEqual(len(_assumed(fil)), 1, fil.get("notes"))++ def test_caption_another_language_skips_parakeet(self):+ """Parakeet is installed and auto passes it over for French: the refusal says that, not+ "no local speech-to-text engine found" with a line telling the caller to install it."""+ proc = script("caption.py", self.src, "--transcribe", "--language", "fr", "--fast", "--json",+ "-o", OUT / "pk_fr.mp4", env=self.env(PATH=str(self.bin)), expect_fail=True)+ doc = json.loads(proc.stdout)+ self.assertEqual((doc["error"]["kind"], doc["reason"], doc["engine"]), ("input", asr.ENGLISH_ONLY_REASON, "parakeet-mlx"))+ self.assertNotIn("no local speech-to-text engine found", doc["error"]["message"])+ self.assertIn("--language fr is not English", doc["error"]["message"])+ self.assertEqual([e["engine"] for e in doc["engines"]], ["parakeet-mlx", "parakeet.cpp"])++ def test_a_named_engine_that_failed_says_why_in_the_document(self):+ """parakeet-mlx prints its errors to stdout and exits 1: the failure document carries that+ line (a JSON caller never sees the log), and the kind is not missing_tool -- it is installed."""+ broken = Path(tempfile.mkdtemp(prefix="ffskill_mlxerr_"))+ try:+ for name in os.listdir(self.bin):+ if name != "parakeet-mlx":+ os.symlink(self.bin / name, broken / name)+ mlx = broken / "parakeet-mlx"+ mlx.write_text("#!/bin/sh\necho 'Fetching 5 files: 0%|' >&2\n"+ "echo 'Error loading model mlx-community/parakeet-tdt-0.6b-v2: Metal device lost'\nexit 1\n")+ mlx.chmod(0o755)+ proc = script("caption.py", self.src, "--transcribe", "--engine", "parakeet-mlx", "--fast", "--json",+ "-o", OUT / "pk_mlxerr.mp4", env=self.env(PATH=str(broken)), expect_fail=True)+ finally:+ shutil.rmtree(broken, ignore_errors=True)+ doc = json.loads(proc.stdout)+ self.assertEqual((doc["error"]["kind"], doc["reason"], doc["engine"]), ("input", asr.ENGINE_FAILED_REASON, "parakeet-mlx"))+ self.assertNotEqual(doc["error"]["code"], "DEPENDENCY_MISSING")+ self.assertEqual(doc["detail"], "found but failed: Error loading model mlx-community/parakeet-tdt-0.6b-v2: Metal device lost")++ def test_silence_filler_words_from_parakeet(self):+ doc = json.loads(script("silence.py", self.src, "--filler", "--transcribe", "--filler-list", "--json",+ "--engine", "parakeet.cpp", env=self.env()).stdout)+ self.assertEqual(doc["filler"]["source"], "parakeet:parakeet.cpp")+ self.assertEqual([r["word"] for r in doc["filler"]["removed"]], ["um"])+++if __name__ == "__main__":+ unittest.main(verbosity=2)diff --git a/tests/test_contract.py b/tests/test_contract.pyindex ad17b8d..3b15252 100644--- a/tests/test_contract.py+++ b/tests/test_contract.py@@ -33,6 +33,12 @@ from _common.runner import HW_ENV, HW_FORCED_ENV # noqa: E402# tests/_fixtures.py cannot drift from what the tools read.for _k in (HW_ENV, HW_FORCED_ENV):os.environ.pop(_k, None)++# The host's own speech defaults must not reach the suite either: FFMPEG_SKILL_ASR_ENGINE or a+# PARAKEET_* setting would change which speech engine a --transcribe test drives, or where+# parakeet-mlx writes. Tests that exercise those opt in with an explicit env.+for _k in ("FFMPEG_SKILL_ASR_ENGINE", "PARAKEET_MODEL", "PARAKEET_CPP_MODEL", "PARAKEET_OUTPUT_TEMPLATE"):+ os.environ.pop(_k, None)import _common # noqa: E402import server as mcp_server # noqa: E402@@ -262,6 +268,20 @@ class ContractTests(unittest.TestCase):f"docs/roadmap.md says the released version is {m.group(1)!r}, "f"but package.json is {pkg['version']!r} -- update the roadmap sentence")+ def test_contract_md_names_only_environment_variables_the_code_reads(self):+ """docs/contract.md is the authority on flags; a review of #305 found its `cache` row naming+ FFMPEG_SKILL_HW, carried over from an unmerged branch, which nothing in this tree read.+ Every FFMPEG_SKILL_* variable it names must be read by a script, the MCP server or the+ installer."""+ text = (ROOT / "docs" / "contract.md").read_text(encoding="utf-8")+ code = "".join(p.read_text(encoding="utf-8", errors="replace")+ for d in ("scripts", "mcp", "bin") for p in sorted((ROOT / d).rglob("*"))+ if p.suffix in (".py", ".js") and "__pycache__" not in p.parts)+ named = sorted(set(re.findall(r"\bFFMPEG_SKILL_[A-Z0-9_]*[A-Z0-9]", text)))+ self.assertTrue(named, "docs/contract.md names no FFMPEG_SKILL_* variable at all")+ self.assertEqual([v for v in named if v not in code], [],+ "docs/contract.md names environment variables nothing in scripts/, mcp/ or bin/ reads")+def test_docs_failure_json_example_keys_match_the_real_error_shape(self):"""docs/contract.md's illustrative failure-JSON example listed only kind/message undererror for years after "code"/"retryable" were added to the real die() output (Hardeningdiff --git a/tests/test_editing.py b/tests/test_editing.pyindex 5440ee2..ff21592 100644--- a/tests/test_editing.py+++ b/tests/test_editing.py@@ -1896,14 +1896,13 @@ class EditingTests(MediaFixtures):(shim / "sitecustomize.py").write_text("import sys, os\n""sys.path.insert(0, os.environ['FFSKILL_SCRIPTS'])\n"- "import _common.asr as asr, _common as c\n"- "def fake(video, language=None, model='base', audio_stream=0):\n"- " return ([{'word': 'So', 'start': 0.2, 'end': 0.5},\n"+ "import _common.asr as asr\n"+ "def fake(video, language=None, model='base', audio_stream=0, engine=None, **kw):\n"+ " return asr.Transcription(words=[{'word': 'So', 'start': 0.2, 'end': 0.5},\n"" {'word': 'um', 'start': 0.6, 'end': 0.8},\n"" {'word': 'this', 'start': 1.0, 'end': 1.3},\n"- " {'word': 'uh', 'start': 2.2, 'end': 2.45}], 'faster-whisper')\n"- "asr.transcribe_words = fake\n"- "c.transcribe_words = fake\n",+ " {'word': 'uh', 'start': 2.2, 'end': 2.45}], engine='faster-whisper')\n"+ "asr.transcribe_words_result = fake\n",encoding="utf-8")env = dict(os.environ, PYTHONPATH=str(shim), FFSKILL_SCRIPTS=str(SCRIPTS))r = sh(sys.executable, SCRIPTS / "silence.py", self._gappy(), "--filler",@@ -1923,11 +1922,10 @@ class EditingTests(MediaFixtures):(shim / "sitecustomize.py").write_text("import sys, os\n""sys.path.insert(0, os.environ['FFSKILL_SCRIPTS'])\n"- "import _common.asr as asr, _common as c\n"- "def fake(video, language=None, model='base', audio_stream=0):\n"- " return ([], 'whisper.cpp')\n"- "asr.transcribe_words = fake\n"- "c.transcribe_words = fake\n",+ "import _common.asr as asr\n"+ "def fake(video, language=None, model='base', audio_stream=0, engine=None, **kw):\n"+ " return asr.Transcription(engine='whisper.cpp')\n"+ "asr.transcribe_words_result = fake\n",encoding="utf-8")env = dict(os.environ, PYTHONPATH=str(shim), FFSKILL_SCRIPTS=str(SCRIPTS))out = OUT / "filler_nowords.mp4"diff --git a/tests/test_picture.py b/tests/test_picture.pyindex 6185847..64b658c 100644--- a/tests/test_picture.py+++ b/tests/test_picture.py@@ -3601,5 +3601,38 @@ class AsrNoSpeechTests(unittest.TestCase):self.assertNotIn("no local speech-to-text engine", call[0])[email protected](platform.system() == "Windows", "the fake whisper-cli is a #!/bin/sh script on a POSIX PATH shim")+class WhisperDefaultModelTests(unittest.TestCase):+ """A whisper.cpp install made the way the refusal hint says (brew + ggml-base.bin) must be the+ model caption.py --transcribe runs by default: the default became large-v3-turbo with the+ Parakeet engines, and whisper-cli was handed a bare model name it could not open."""++ def test_the_default_model_is_the_one_the_install_hint_names(self):+ import tempfile+ from _common import asr+ self.assertIn("ggml-base.bin", asr.ASR_INSTALL_HINT)+ with tempfile.TemporaryDirectory() as d:+ home, bin_dir = Path(d, "home"), Path(d, "bin")+ (home / ".cache" / "whisper.cpp").mkdir(parents=True)+ model = home / ".cache" / "whisper.cpp" / "ggml-base.bin"+ model.write_bytes(b"")+ bin_dir.mkdir()+ seen = Path(d, "args.txt")+ # a whisper-cli that records its arguments and writes one cue where -of says+ cli = bin_dir / "whisper-cli"+ cli.write_text('#!/bin/sh\nprintf "%s\\n" "$@" > "' + str(seen) + '"\n'+ 'while [ $# -gt 0 ]; do [ "$1" = "-of" ] && of="$2"; shift; done\n'+ 'printf "1\\n00:00:00,000 --> 00:00:01,000\\nhello\\n" > "$of.srt"\n', encoding="utf-8")+ cli.chmod(0o755)+ clip = Path(d, "talk.mp4")+ sh("ffmpeg", "-y", "-v", "error", "-f", "lavfi", "-i", "color=c=black:s=320x180:r=30:d=2",+ "-f", "lavfi", "-i", "sine=d=2", "-c:v", "libx264", "-c:a", "aac", "-shortest", clip)+ env = dict(os.environ, HOME=str(home), PATH=f"{bin_dir}{os.pathsep}{os.environ['PATH']}",+ FFMPEG_SKILL_ASR_ENGINE="whisper.cpp")+ script("caption.py", clip, "--transcribe", "-o", Path(d, "out.mp4"), env=env)+ args = seen.read_text(encoding="utf-8").split("\n")+ self.assertEqual(args[args.index("-m") + 1], str(model))++if __name__ == "__main__":unittest.main(verbosity=2)