Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

feat: catch valid-but-empty content (silent TTS, invisible captions, -Infinity JSON) (#294)

Shogo Kajiyama581d18ed2e11
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f041c9b..305d871 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,23 @@
## Unreleased
-(nothing yet)
+- feat(join): `--on-silent warn|fail|skip` (default `warn`) and `--silence-threshold DB` (default -50 dBFS). The preflight measures every input with audio (one volumedetect pass on a real run; `--dry-run` measures nothing and says so) and treats a peak at or below the threshold as silent -- the trace of a TTS step that wrote a valid but empty wav, which the 2.2.0 missing/empty/unreadable checks let through. `warn` joins it and names it under the new `silent: [{index, path, peak_db}]` key and in `notes`; `fail` refuses it in the same `kind: input` document as the other problems (`silent (peak -91.0 dBFS)`); `skip` drops it into `skipped`. Inputs without an audio stream are never silent. The join's own ffmpeg command is unchanged, and render.py forwards nothing new.
+- fix: `audio.py` no longer passes a silent `--music` / `--replace` / `--effects` file, or a silent
+ voice under `--duck`, as verified. Each is peak-measured in the 2.2.4 preflight (real runs; a dry run measures nothing);
+ new `--on-silent warn|fail` (default warn: `silent: [{flag, path, peak_db}]` + note; fail:
+ reason `silent (peak X dBFS)` in the one refusal) and `--silence-threshold` (default -50 dBFS).
+- fix: no `--json` document contains `-Infinity` / `NaN` any more. `print_json` writes
+ non-finite floats as `"-inf"` / `"inf"` / `null` (as `loudness.py` already did) with
+ `allow_nan=False`; `export.py` on silent audio reports `loudness.lufs: "-inf"`,
+ `silent: true`, and a note saying "output audio is silent" instead of recommending loudness.py.
+- fix(caption): a burn whose cues are never visible is refused with `kind: input` instead of reporting `verified: true` -- every cue outside `[0, duration]`, every cue blank, or `--ass` with no `Dialogue` lines; applies to `waveform.py --srt` too. The `caption` block reports `cues_burned` and `cues_outside`, and cues partly outside get a warning note
+- fix(graphics, overlay): whitespace-only `--title`, `--name`, `--text` (and the other text flags) count as missing and are refused instead of burning an empty graphic
+- feat(check): `--content` (opt-in) adds `black` (blackdetect share: WARN > 10%, FAIL >= 95%), `frozen` (freezedetect longest span: WARN > max(3 s, 30%), FAIL when the whole video is frozen) and `silence` (silencedetect share at -50 dB: WARN > 50%, FAIL >= 95%) rows from one decode pass. Without it the row set is unchanged. render.py's `check` section takes `"content": true` to forward it.
+- fix(check): the `audio` row no longer PASSes "present" for a silent track: FAIL under a platform/`--lufs` loudness target, WARN with none (a muted screen recording is legitimate), when the track is silent (peak <= -50 dBFS) -- read off the loudness pass, or from one volumedetect pass when `--no-loudness` or a spec without a loudness target skips it.
+- fix(caption): the "re-run without --dry-run to produce X_adjusted.srt" note appears only on dry runs
+- fix(join): a segment shorter than 2 frames (audio-only: 0.05 s) and a path listed twice no longer join unremarked: new `short_segments: [{index, path, duration}]` and `duplicates: [{path, indices}]` keys (always present, `[]` when none) plus notes. Warnings only; the join command is unchanged.
+- fix(waveform): silent input audio no longer renders a flat-line audiogram without a word. A real run peak-measures the input; `--on-silent warn|fail` (default warn) and `--silence-threshold` (default -50 dBFS) as in `audio.py`; new `silent` key (`null` under `--dry-run`) and a note.
+- fix(asr): a speech engine that ran and found no speech says `<engine> found no speech in <input>` (`kind: input`, `reason: "no_speech"`, `engine`) instead of faster-whisper's "no local speech-to-text engine found" or whisper.cpp / openai-whisper's "no cues found in /tmp/ffskill_asr_*/audio.srt".
## 2.2.5
diff --git a/SKILL.md b/SKILL.md
index 93e3f41..278ea2b 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -23,7 +23,7 @@ Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `chec
7. **Keep the user's originals.** Never overwrite the source; write new files next to input or where asked. An existing output path is refused (`kind: input`); `--overwrite` is the one way to say "yes, replace it".
8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` scaling a clip to the first clip's frame, `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG (4x3 for whole-clip layout). Not finished until `Look:` names that PNG — a probe cannot see a caption on a face. Audio-only: `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
- **Mechanical (this skill's own job to verify and report):** the specified text/logo is at the specified position, subtitles appear at the specified timestamps, dimensions are even. Letterboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
- - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, which belongs to the calling agent — say what you see in one line and let them judge it.
+ - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, the calling agent's call — say what you see in one line; they judge.
With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, without stalling for a capability that isn't there.
## Before you run anything: what to ask, what to assume
@@ -136,7 +136,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
| "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied; in a project: `"chapters"`). `--auto-chapters` proposes them from measured pauses/scene cuts (`Chapter N`) |
| "set the title / artist / comment" | `metadata.py episode.mp4 --title "Episode 12" --artist "Studio"` |
| "put these videos in a 4x2 grid" | `grid.py t1.mp4 ... t8.mp4 --cols 4 --rows 2` |
-| "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` (TTS parts: `--list parts.txt`, `--on-missing skip`) |
+| "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` (TTS: `--list parts.txt`, `--on-missing skip`, `--on-silent fail`) |
| "several changes to one edit", 3+ steps | `render.py --init project.json`, edit, `render.py project.json` |
| "I changed one stage, don't redo the rest" | `render.py project.json --cache DIR` — identical stages reused (`--from STAGE` starts there) |
| "open it in Premiere / Resolve / Final Cut" | `render.py project.json --export-timeline edit.fcpxml\|.edl\|.otio` — renders nothing; read `not_exported`; no project? `--write-project` |
diff --git a/docs/contract.md b/docs/contract.md
index 45780cb..0a72a0c 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -186,6 +186,11 @@ The workflow in `SKILL.md` is "probe first, verify last". The contract states it
(fit, caption, overlay, graphics, color, join, multicam, render). Audio-only tools and
audio-only inputs never need `look`; the report line is `Look: not needed`. `check`
rows carry `kind: format` (fix it) or `kind: judgement` (decide with the user).
+The `audio` row FAILs (WARN when no loudness target applies) when the track is present but silent (peak at or below -50 dBFS, from
+the loudness pass or, when loudness is skipped, one volumedetect pass). `check --content`
+(opt-in; render.py `"check": {"content": true}`) adds `black`, `frozen` and `silence` rows
+from one decode pass; without it the row set is unchanged. Thresholds:
+`docs/design-decisions.md` ("check.py --content").
### Dry run
@@ -441,6 +446,20 @@ runs a shell, evaluates strings, or executes anything other than the named scrip
## JSON output
+Non-finite numbers never reach a `--json` document (2.2.6): every tool prints through one
+writer that turns `-inf` / `inf` into the strings `"-inf"` / `"inf"` (the spelling
+`loudness.py` already used for a silent input) and NaN into `null`, then serialises with
+`allow_nan=False`, so a strict JSON parser reads every document. A silent file's loudness is
+therefore `"lufs": "-inf"`, never `-Infinity`.
+
+`audio.py` (2.2.6) measures the whole-file peak of every `--music` / `--replace` / `--effects`
+file and, under `--duck`, of the voice: at or below `--silence-threshold` (default -50 dBFS)
+the track is silent. `--on-silent warn` (default) mixes it and adds `silent: [{flag, path,
+peak_db}]` plus a `notes` line; `--on-silent fail` names it in the single `kind: input`
+refusal's `problems` with reason `"silent (peak X dBFS)"`. A real run measures every file
+that exists. `export.py`'s `loudness` gains `silent: true` for a silent output, whose note
+says "output audio is silent" instead of recommending `loudness.py`.
+
Per-tool keys added in 1.13: `audio` (`audio.py`) reports the mix it built — the
`--voice` level, `stereo_widen`, whether an `--effects` bed was mixed, and with
`--music` the `music_volume` plus a `duck` object naming the threshold (dB and
@@ -477,6 +496,7 @@ Per-tool keys added in 1.17.1, all additive:
| key | tool | what it holds |
|---|---|---|
| `caption` | `render.py` | the caption stage's own block, forwarded verbatim from `caption.py` (the cue-layout counts plus the fit-size keys above), so a template run can be read for `split` and `size_used` without re-running the stage. `null` when the project has no captions stage — and also when the captions stage came from the `--cache` (a cache hit carries no stage document, so a second `render.py … --cache DIR` run reports `caption: null` while `stages_done` still lists `captions`). `caption.py --mode mux` writes no `caption` block at all |
+| `cues_burned`, `cues_outside` | `caption.py` | siblings inside the `caption` block, **burn mode only** (2.2.6): `cues_burned` counts the non-blank cues that overlap `[0, duration]` of the video and are drawn, `cues_outside` the non-blank cues wholly outside it (each also named in `notes`). A burn where none is visible — every cue after the end, every cue blank, an `--ass` with no `Dialogue` lines — is refused with `kind: input` instead of reporting `verified: true`; `waveform.py --srt` passes that refusal on with the same kind |
| `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |
@@ -497,6 +517,14 @@ Per-tool keys added after 2.2.2, all additive:
| key | tool | what it holds |
|---|---|---|
| `sdr_path`, `notes` | `color.py --to-sdr` | `sdr_path` is `"tonemap"` for PQ / HLG / Dolby Vision input (and `--force` on an untagged file) or `"gamut"` for BT.2020 primaries on an SDR transfer, which is converted to BT.709 without a tone map (`--tonemap`/`--peak`/`--desat` do not apply); `notes` says which path was taken and why |
+| `silent` | `join.py` (every run) | `[{index, path, peak_db}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
+| `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
+| `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
+
+`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
+and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
+`reason: "no_speech"` and `engine` in the failure document — distinct from the "no local
+speech-to-text engine found" refusal, which now only means no engine was found.
`check.py` also gains an informational `subtitles` row on **every** platform:
`PASS` when every soft subtitle stream carries a language tag, `WARN` when one
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index d54f596..da94ae3 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -58,6 +58,17 @@ exists. When a decision changes, edit the entry in the same PR.
`join.preflight()`, `join.main()`, `join.unpending_length()`.
Tests: `test_join_dry_run_plans_on_pending_segments`,
`test_render_dry_run_joins_a_trimmed_and_an_untrimmed_audio_clip`.
+- **`join.py --on-silent` defaults to `warn`, not `fail`, at a -50 dBFS peak.** Deliberate
+ silence exists: a beat of room tone between lines, a music bed's silent intro, a clip whose
+ camera audio is to be replaced later. Refusing those by default would break joins that worked
+ in 2.2.5, so the default only names the input (`silent`, `notes`, stderr) and a pipeline that
+ knows every segment must speak -- TTS lines -- opts into `fail` or `skip`. The measure is the
+ whole-file peak, not the mean or loudness: a failed TTS file is digital silence (volumedetect
+ reports -91 dBFS) and any real speech peaks far above -50 dBFS even when quiet, while a noisy
+ room-tone take can still peak above it and is then (correctly) not called silent; a mean or
+ LUFS gate would flag quiet but real narration. An input with no audio stream is never silent
+ (the join adds silence for it on purpose), and an unmeasurable level is left to the join.
+ Code: `join.find_silent()`, `join.main()`. Test: `test_join_on_silent_names_a_silent_segment`.
- **`verify` accepts `--dry-run` and ignores it.** Its job is to run the tools for real.
Contract: `_contract.DRY_RUN_NOTES["verify"]`.
@@ -725,6 +736,21 @@ not a new file format this tool would have to maintain.
checklist.
+## Unreleased — captions that can be seen
+
+- **A burn that draws nothing is a refusal, not a warning.** `caption.py` counts the non-blank
+ cues overlapping `[0, duration]`; none (a cue file timed for a longer cut, all-blank text, an
+ `--ass` without `Dialogue` lines) is `kind: input`, because the output would be the input
+ re-encoded and `verified: true` would be false at the layer a caller reads. Some cues outside
+ is only a warning with `cues_outside`: trimming a clip out of a longer transcript is normal.
+ With no probed duration every non-blank cue counts as drawn. `--mode mux` is not checked; a
+ soft track draws nothing by itself. Tests: `test_caption_refuses_cues_that_are_never_visible`,
+ `test_caption_reports_cues_burned_and_outside`, `test_waveform_srt_outside_the_render_is_refused_as_input`.
+- **Whitespace-only text flags are missing, not text.** `graphics.py` maps a blank
+ `--title`/`--name`/`--subtitle`/`--text`/`--top`/`--bottom` to unset before the template's own
+ "needs --title" check; non-blank text is drawn exactly as given (not stripped). `overlay.py
+ --text " "` is refused. Test: `test_graphics_and_overlay_refuse_blank_text`.
+
## 2.2.4 — the first bad input, named
- **Every bad extra input is refused together, before ffmpeg runs.** The same rule 2.2.0/2.2.1
@@ -738,3 +764,53 @@ not a new file format this tool would have to maintain.
changed. Tests: `test_audio_beds_without_audio_are_refused_together`,
`test_render_real_run_names_every_unreadable_clip_source`, `test_sync_too_short_names_the_file`,
`test_multicam_too_short_names_the_file`.
+
+
+## 2.2.6 — silence is reported, not verified
+
+- **A silent track warns by default, and refuses only on request.** `audio.py` measures the
+ whole-file peak of each `--music` / `--replace` / `--effects` file and, under `--duck`, the
+ voice. Default `--on-silent warn` still mixes it (a deliberately silent placeholder bed is a
+ real use, and join.py's `--on-silent` takes the same default and -50 dBFS peak threshold) but
+ names it under `silent` and in a note; `--on-silent fail` makes it one more `problems` entry in
+ the 2.2.4 refusal. Peak rather than mean: a sparse effects track with a quiet mean is not
+ silent. Tests: `test_audio_silent_beds_warn_by_default_and_fail_on_request`,
+ `test_audio_duck_under_a_silent_voice_is_reported`.
+- **Non-finite numbers are strings, not `-Infinity`.** `print_json` maps -inf/inf to `"-inf"` /
+ `"inf"` and NaN to `null` and serialises with `allow_nan=False`: a JSON-valid document beats a
+ numeric type for a value that has no JSON number, and `"-inf"` is what `loudness.py` already
+ reported. A silent export's note says the audio is silent; loudness.py cannot fix that, so it
+ is neither recommended nor run by `--normalize`. Test:
+ `test_audio_every_tool_json_on_silent_input_parses_strictly`.
+- **`check.py --content` is opt-in, and its thresholds are loose.** The black / frozen /
+ silence rows cost a full decode of the file (one pass, blackdetect + freezedetect +
+ silencedetect), where the default rows cost an ffprobe and one audio pass; and a black
+ intro, a held title card or a silent B-roll montage are real edits, so they would be noise on
+ every delivery. Thresholds: `black` WARN above 10% of the duration, FAIL at >= 95% (all
+ black, allowing blackdetect's last-frame shortfall); `frozen` WARN when the longest frozen
+ span exceeds max(3 s, 30% of the duration) -- a 3 s title hold is normal, a stuck third of a
+ short is not -- FAIL at >= 95% (the whole video frozen); `silence` WARN above 50% below
+ -50 dB, FAIL at >= 95%. Only FAIL means "almost certainly a broken render". The default row
+ set is byte-identical without the flag. Tests: `test_check_content_rows`,
+ `test_check_default_rows_unchanged_without_content`.
+- **`check.py`'s `audio` row flags a silent track even without `--content`: FAIL under a loudness target, WARN with none (a muted screen recording is legitimate).** A present
+ stream at or below -50 dBFS peak (2.2.6's threshold) is the empty-TTS / muted-export case
+ that "present" used to pass; it costs nothing when the loudness pass ran (its true peak is
+ read) and one volumedetect pass when loudness is skipped. Test:
+ `test_check_audio_row_fails_on_silent_track`.
+
+## 2.3.0 — short, repeated and silent inputs
+
+- **A short or repeated join input warns; it never refuses.** A clip under 2 frames (audio-only:
+ 0.05 s) is almost always a failed earlier step, but a repeated clip (an intro used twice, a
+ loop) is a real edit, and the frame count cannot tell a deliberate one-frame flash from a bug.
+ Both are named under `short_segments` / `duplicates` and in notes; `verified` and the ffmpeg
+ command are unchanged. Test: `test_join_warns_on_short_segments_and_duplicates`.
+- **A silent waveform input warns by default, like audio.py.** `waveform.py --on-silent`
+ takes audio.py's default (`warn`) and -50 dBFS peak threshold; `verified` stays about the
+ render (size, rate, duration) so the top-level and `audiogram.verified` never disagree, and
+ `silent` is the key a caller reads. `--dry-run` measures nothing and says `null`. Test:
+ `test_waveform_silent_input_warns_or_fails`.
+- **"Found no speech" is not "no engine".** An engine that ran and returned no cue is refused
+ as `kind: input`, `reason: "no_speech"`, naming the engine and the caller's input rather than
+ the engine's deleted temporary SRT. Tests: `AsrNoSpeechTests`.
diff --git a/references/scripts.md b/references/scripts.md
index 67787be..fbf3253 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -1,6 +1,6 @@
# Script reference
-Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither flag; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
+Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither flag; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
## Time grammar (every time-taking flag, 1.9)
@@ -312,7 +312,7 @@ waveform.py INPUT [--style waveform|spectrum] [--width W] [--height H] [--fps N]
[--image PATH] [--image-fit cover|contain|blur]
[--position bottom|centre|top|strip] [--vis-height FRAC] [--opacity 0..1]
[--platform NAME] [--srt FILE | --text FILE] [--title TEXT] [--brand brand.json]
- [-o OUT]
+ [--on-silent warn|fail] [--silence-threshold -50] [-o OUT]
```
Renders the input's audio as a video: `--style waveform` (default, FFmpeg's
`showwaves`) draws amplitude over time; `--style spectrum` (`showspectrum`)
@@ -343,7 +343,11 @@ give an image or a colour. The title and caption stages receive `--overwrite`, `
`--srt`/`--text` file is refused (`kind: input`) before anything is encoded. Without any of
these flags the command line is byte-identical to 1.15's. Every run's result carries an `audiogram` object (style,
background, image, position, vis_height, platform, captions, title, stages,
-verified). `render.py --template audiogram` is the one-call form; it is
+verified). A real run peak-measures the input audio: at or below `--silence-threshold`
+(default -50 dBFS) the render is a flat line, so `--on-silent warn` (default) renders it with
+`silent: true` and a note and `fail` refuses (`kind: input`) before encoding; `silent` is
+`null` under `--dry-run`. An `--image` ffprobe reports as 0x0 is refused as undecodable.
+`render.py --template audiogram` is the one-call form; it is
deliberately not part of `--template all`.
### freeze.py — hold a frame for N seconds
@@ -556,7 +560,7 @@ writes the resulting cut list exactly as it does today.
```
join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
[--duration 0.5] [--width W --height H] [--fps N] [--fit pad|crop] [-o OUT]
-join.py --list parts.txt [--on-missing fail|skip] [...]
+join.py --list parts.txt [--on-missing fail|skip] [--on-silent warn|fail|skip] [--silence-threshold -50] [...]
```
`--list FILE` (2.2) reads the clips from a file, one per line in order, paths relative to the
file (blank lines and `#` comments ignored; ffmpeg's `file 'x.wav'` lines accepted), for the
@@ -572,6 +576,16 @@ in for its streams (any audio extension the skill reads, `.aiff` and `.caf` incl
picture), so a pending `.mp4` next to measured audio is refused as the mix a real run would
refuse; `expected_duration` is null while any input is pending, and `notes` names the planned
frame, rate or xfade offsets that rest on a pending input's unmeasured placeholder.
+Every existing input with an audio stream is also measured (one volumedetect pass; not under `--dry-run`
+too): a peak at or below `--silence-threshold` (default -50 dBFS) is silent, the trace of a TTS
+call that wrote a valid but empty file. `--on-silent warn` (default) joins it and names it under
+`silent: [{index, path, peak_db}]` and in `notes`; `fail` refuses it in the same `kind: input`
+document (`silent (peak -91.0 dBFS)`); `skip` leaves it out under `skipped`. A clip without an
+audio stream is never silent.
+Two warnings never refuse or change the command: `short_segments: [{index, path, duration}]`
+names a clip shorter than 2 frames at the join's fps (audio-only: 0.05 s), and
+`duplicates: [{path, indices}]` a path listed more than once (a repeat can be intended); both
+are always present (`[]` when none) and noted.
Normalises every clip to one frame size, fps, `yuv420p`, 48 kHz and one
channel layout (the widest clip's -- a 5.1 clip keeps 5.1 -- or `--channels`;
silent track generated for clips without audio), then chains `xfade` +
@@ -838,7 +852,7 @@ measurements that combine with each other and with `--beats`/`--highlights`:
### check.py — pre-delivery compliance
```
-check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--json]
+check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--content] [--json]
[--max-duration S] [--aspect 9:16] [--lufs -14] [--tp -1] [--max-mb N]
```
PASS/WARN/FAIL per check with the script that fixes it. Run it as the final
@@ -855,6 +869,14 @@ minimum height, fps, codecs, size, LUFS, true peak, SDR-only) come from the one
in `scripts/_platforms.py`, which `export.py` and the `render.py` templates read too -- so the
loudness a preset normalises to and the loudness this tool checks are the same value by
construction, not by two lists agreeing.
+The `audio` row FAILs (under a loudness target; WARN without one) on a present-but-silent track (peak <= -50 dBFS, the join/audio
+`--silence-threshold` default): measured from the loudness pass, or with one volumedetect
+pass when `--no-loudness` (or a spec with no loudness target) skips it.
+`--content` (opt-in, one decode pass) adds three rows: `black` (share of the duration that is
+black: WARN > 10%, FAIL >= 95%), `frozen` (longest frozen span: WARN > max(3 s, 30% of the
+duration), FAIL when it covers >= 95%, i.e. the whole video) and `silence` (share below
+-50 dBFS: WARN > 50%, FAIL >= 95%). A render project enables it with `"check": {"platform":
+"reels", "content": true}`.
### batch.py — same recipe over a folder, cached
```
@@ -1125,6 +1147,16 @@ the honest sentence for a report, made automatic: only the line breaks, the
timing and the type size ever move, and those do not count as a change. The key
is burn mode only; `--mode mux` never touches the text and omits it.
+**Cues that can be seen (2.2.6).** A burn counts the cues it actually draws:
+`cues_burned` (non-blank, overlapping `[0, duration]`) and `cues_outside`
+(non-blank, wholly after the end or before 0) in the `caption` block, with a
+warning note naming the outside count. When no cue is visible — the only cue
+starts after the video ends, every cue is blank, or an `--ass` file has no
+`Dialogue` lines — the run is refused with `kind: input` (`no cue falls inside
+the video (0–3.0 s); first cue starts at 10.0 s`, `every cue is blank`), dry
+runs included. `waveform.py --srt` passes the refusal on. `graphics.py` and
+`overlay.py` treat whitespace-only `--title`/`--name`/`--text` as missing.
+
Under `--dry-run`/`--plan` on an input that does not exist yet there is no
geometry to measure. With `--platform` the destination's own frame is used —
that frame *is* what the real run will have, so the planned `FontSize` is the
@@ -1400,6 +1432,10 @@ threshold ducks on quieter speech, a shorter release brings the bed back faster.
Every `--music` / `--effects` / `--replace` file is checked before ffmpeg runs (under
`--dry-run` too): missing, empty, unreadable or with no audio stream, all of them are named in
one refusal (`kind: input`, `problems: [{flag, path, reason}]`).
+Each of those files, and under `--duck` the voice, is also measured (whole-file peak, under
+not under `--dry-run`): at or below `--silence-threshold` (default -50 dBFS) it is silent.
+`--on-silent warn` (default) mixes it anyway and reports `silent: [{flag, path, peak_db}]`
+plus a note; `--on-silent fail` adds it to the same refusal with reason `silent (peak X dBFS)`.
`--stereo-widen 0..1` widens the stereo image (`extrastereo=m=1+2*amount`) and
needs a real stereo source: it scales the side signal (L−R), so a mono track
duplicated to two channels has nothing to scale. A 1-channel input is refused
diff --git a/scripts/_common/asr.py b/scripts/_common/asr.py
index ba51689..788e956 100644
--- a/scripts/_common/asr.py
+++ b/scripts/_common/asr.py
@@ -50,6 +50,28 @@ def parse_srt(path: str) -> List[Tuple[float, float, str]]:
return cues
+NO_SPEECH_REASON = "no_speech"
+
+
+def die_no_speech(engine: str, video: str) -> None:
+ """An engine ran and heard nothing: kind input, `reason: "no_speech"`, naming the engine and
+ the input -- not the "no engine found" refusal (one was found) and not "no cues found in"
+ the engine's own temporary SRT (a path the caller never gave and that is already deleted)."""
+ die(f"{engine} found no speech in {video}", kind="input", reason=NO_SPEECH_REASON, engine=engine,
+ hint="check that the input (and --audio-stream) carries the speech, or write the cues by hand with --text")
+
+
+def _engine_cues(srt: str, engine: str, video: str) -> List[Tuple[float, float, str]]:
+ """The cues of an SRT an engine wrote; an SRT with no cue in it is die_no_speech()."""
+ try:
+ text = Path(srt).read_text(encoding="utf-8", errors="replace")
+ except OSError:
+ text = ""
+ if "-->" not in text:
+ die_no_speech(engine, video)
+ return parse_srt(srt)
+
+
def write_srt(cues: List[Tuple[float, float, str]], path: str) -> None:
with open(path, "w", encoding="utf-8") as fh:
for i, (s, e, t) in enumerate(cues, 1):
@@ -122,7 +144,7 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
proc = _asr_run(cmd, subprocess, "whisper.cpp")
if proc.returncode == 0 and os.path.exists(base + ".srt"):
info(f"transcribed with whisper.cpp ({os.path.basename(cli)}, model {os.path.basename(model_path)})")
- cues = parse_srt(base + ".srt")
+ cues = _engine_cues(base + ".srt", "whisper.cpp", video)
write_srt(cues, out_srt)
return cues
info("whisper.cpp found but failed: " + (proc.stderr.strip().splitlines() or ["?"])[-1][:200])
@@ -144,10 +166,11 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
if t.is_alive():
die(f"faster-whisper exceeded the {STATE.timeout:.0f} s time limit; raise --timeout for a long recording", code=124, kind="timeout")
cues = list(result)
- if cues:
- info("transcribed with faster-whisper")
- write_srt(cues, out_srt)
- return cues
+ if not cues:
+ die_no_speech("faster-whisper", video)
+ info("transcribed with faster-whisper")
+ write_srt(cues, out_srt)
+ return cues
except ImportError:
pass
# 3. openai-whisper CLI
@@ -159,7 +182,7 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
srt = os.path.join(tmpdir, "audio.srt")
if proc.returncode == 0 and os.path.exists(srt):
info("transcribed with openai-whisper")
- cues = parse_srt(srt)
+ cues = _engine_cues(srt, "openai-whisper", video)
write_srt(cues, out_srt)
return cues
die_no_engine("Or write the cues by hand with --text cues.txt (see format above).")
diff --git a/scripts/_common/emit.py b/scripts/_common/emit.py
index eaca3ed..262fb42 100644
--- a/scripts/_common/emit.py
+++ b/scripts/_common/emit.py
@@ -243,8 +243,21 @@ def write_plan(path: str, output: Optional[str], extra: Dict[str, Any], ctx: "Op
return path
+def json_safe(obj: Any) -> Any:
+ """Replace non-finite floats with the strings loudness.py already reports ("-inf" / "inf")
+ and NaN with null: json.dumps would otherwise write -Infinity / NaN, which no strict JSON
+ parser accepts (a silent file measures -inf LUFS)."""
+ if isinstance(obj, float) and not math.isfinite(obj):
+ return None if obj != obj else ("inf" if obj > 0 else "-inf")
+ if isinstance(obj, dict):
+ return {k: json_safe(v) for k, v in obj.items()}
+ if isinstance(obj, (list, tuple)):
+ return [json_safe(v) for v in obj]
+ return obj
+
+
def print_json(obj: Any) -> None:
- sys.stdout.write(json.dumps(obj, indent=2, ensure_ascii=False) + "\n")
+ sys.stdout.write(json.dumps(json_safe(obj), indent=2, ensure_ascii=False, allow_nan=False) + "\n")
# Deferred for the same reason as runner's import of this module: probe needs die() from here, and
diff --git a/scripts/_common/probe.py b/scripts/_common/probe.py
index fb61d07..cb10c41 100644
--- a/scripts/_common/probe.py
+++ b/scripts/_common/probe.py
@@ -343,11 +343,15 @@ def keyframes_near(path: str, t: float, window: float = 5.0) -> List[float]:
return sorted(set(out))
-def measured_level_dbfs(path: str, seconds: float = 120.0) -> Optional[Dict[str, float]]:
+def measured_level_dbfs(path: str, seconds: Optional[float] = 120.0,
+ audio_stream: Optional[int] = None) -> Optional[Dict[str, float]]:
"""Mean and peak level of the first `seconds` of audio (volumedetect), in dBFS; None if unmeasurable.
- Cheap enough to run once as a hint when a threshold-based tool found nothing."""
+ Cheap enough to run once as a hint when a threshold-based tool found nothing. seconds=None
+ measures the whole file; audio_stream picks the Nth audio stream (default: ffmpeg's choice)."""
ffmpeg = require_tool("ffmpeg")
- proc = run_analysis([ffmpeg, "-hide_banner", "-nostdin", "-t", f"{seconds:.0f}", "-i", path, "-vn",
+ head = ["-t", f"{seconds:.0f}"] if seconds is not None else []
+ pick = ["-map", f"0:a:{audio_stream}"] if audio_stream is not None else []
+ proc = run_analysis([ffmpeg, "-hide_banner", "-nostdin", *head, "-i", path, *pick, "-vn",
"-af", "volumedetect", "-f", "null", "-"], check=False)
m_mean = re.search(r"mean_volume:\s*(-?[0-9.]+) dB", proc.stderr)
m_max = re.search(r"max_volume:\s*(-?[0-9.]+) dB", proc.stderr)
@@ -463,6 +467,68 @@ def detect_silences(path: str, threshold: float, min_silence: float) -> "List[Tu
return silences
+# 2.3.0's silence threshold: a track whose whole-file peak is at or below this is digital
+# silence (join.py / audio.py --silence-threshold default, check.py's audio row).
+SILENT_PEAK_DBFS = -50.0
+
+# check.py --content thresholds (docs/design-decisions.md "check.py --content").
+CONTENT_BLACK_PIX_TH = 0.10 # blackdetect pixel threshold (share of the luma range)
+CONTENT_MIN_SPAN = 0.1 # blackdetect minimum run, seconds
+CONTENT_FREEZE_NOISE = "-60dB" # freezedetect noise tolerance
+CONTENT_FREEZE_MIN = 0.5 # freezedetect minimum run, seconds
+CONTENT_SILENCE_MIN = 0.5 # silencedetect minimum run, seconds
+
+_BLACK_RE = re.compile(r"black_start:\s*([0-9.]+)\s+black_end:\s*([0-9.]+)")
+_FREEZE_RE = re.compile(r"lavfi\.freezedetect\.freeze_(start|end):\s*([0-9.]+)")
+
+
+def detect_content(path: str, duration: float, *, video: bool = True, audio: bool = True) -> Dict[str, Any]:
+ """One decode pass of blackdetect + freezedetect (video) and silencedetect (audio, at
+ SILENT_PEAK_DBFS). Returns {"black": [(s, e)], "frozen": [(s, e)], "silence": [(s, e)]}
+ with open-ended spans closed at `duration`; a key is absent when its stream is. {} when the
+ input is a pending dry-run placeholder."""
+ if dry_run_input_pending(path) or not (video or audio):
+ return {}
+ ffmpeg = require_tool("ffmpeg")
+ cmd = [ffmpeg, "-hide_banner", "-nostdin", "-i", path]
+ if video:
+ cmd += ["-map", "0:v:0", "-vf", f"blackdetect=d={CONTENT_MIN_SPAN}:pix_th={CONTENT_BLACK_PIX_TH},"
+ f"freezedetect=n={CONTENT_FREEZE_NOISE}:d={CONTENT_FREEZE_MIN}"]
+ if audio:
+ cmd += ["-map", "0:a:0", "-af", f"silencedetect=noise={SILENT_PEAK_DBFS:g}dB:d={CONTENT_SILENCE_MIN}"]
+ proc = run_analysis(cmd + ["-f", "null", "-"], check=False, record=True)
+ if proc.returncode != 0:
+ die(f"content detection failed:\n{proc.stderr.strip()[-800:]}", kind="ffmpeg")
+ end = max(duration, 0.0)
+ out: Dict[str, Any] = {}
+ if video:
+ out["black"] = [(float(a), min(float(b), end) if end else float(b)) for a, b in _BLACK_RE.findall(proc.stderr)]
+ frozen: "List[Tuple[float, float]]" = []
+ start = None
+ for kind, val in _FREEZE_RE.findall(proc.stderr):
+ if kind == "start":
+ start = float(val)
+ elif start is not None:
+ frozen.append((start, float(val)))
+ start = None
+ if start is not None:
+ frozen.append((start, end))
+ out["frozen"] = frozen
+ if audio:
+ sil: "List[Tuple[float, float]]" = []
+ start = None
+ for kind, val in SIL_RE.findall(proc.stderr):
+ if kind == "start":
+ start = float(val)
+ elif start is not None:
+ sil.append((start, min(float(val), end) if end else float(val)))
+ start = None
+ if start is not None:
+ sil.append((start, end))
+ out["silence"] = sil
+ return out
+
+
SCORE_RE = re.compile(r"frame:(\d+)\s+pts:\d+\s+pts_time:([0-9.]+)")
diff --git a/scripts/_contract.py b/scripts/_contract.py
index 5d6dd5e..bfa070f 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -196,7 +196,9 @@ TOOL_META: Dict[str, Dict[str, Any]] = {
{"capability": "filter:loudnorm", "when": "preset youtube / youtube4k / reels / x with audio: the written file is measured against the platform's loudness target (result `loudness`)"}],
video_required=True, audio_only=False, visual=False, verify=["probe", "check"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
"check": dict(role="verification", inputs=["media artifact"], outputs=["compliance rows JSON on stdout (no file)"],
- required=["ffprobe"], optional=[{"capability": "ffmpeg", "when": "loudness rows (default)"}, {"capability": "filter:loudnorm", "when": "loudness rows (default)"}],
+ required=["ffprobe"], optional=[{"capability": "ffmpeg", "when": "loudness rows (default); the audio row's volumedetect silence test when loudness is skipped"}, {"capability": "filter:loudnorm", "when": "loudness rows (default)"},
+ {"capability": "filter:blackdetect", "when": "--content (black row)"}, {"capability": "filter:freezedetect", "when": "--content (frozen row)"},
+ {"capability": "filter:silencedetect", "when": "--content (silence row)"}],
video_required=False, audio_only=True, visual=False, verify=[], produces_artifact=False, idempotency="bit_exact", deterministic=True),
"scenes": dict(role="analysis", inputs=["video asset"], outputs=["scene / audio-peak / highlight JSON on stdout", "EDL text (--edl)", "per-scene contact sheet PNG (--sheet)"],
required=FF + ["filter:scdet"], optional=[{"capability": "filter:drawtext", "when": "--sheet"}, {"capability": "filter:tile", "when": "--sheet"},
@@ -231,7 +233,7 @@ DRY_RUN_ANALYSIS = {
"cropdetect": "the cropdetect filter runs over the sampled windows to measure bars; this tool never writes a file regardless of --dry-run",
"silence": "silencedetect runs so the reported silences and keep ranges are real; the cut output is not written",
"loudness": "the loudnorm measurement pass runs so input_i and the planned pass-2 command are real; the normalised output is not written",
- "check": "read-only tool; the loudness measurement runs under --dry-run too, so every row is present",
+ "check": "read-only tool; the loudness measurement (or, when it is skipped, the audio row's volumedetect) and the --content decode run under --dry-run too, so every row is present",
"stabilize": "vidstabdetect (pass 1, into a temp file) runs; the stabilised output (pass 2) is not written",
}
DRY_RUN_NOTES = {
@@ -394,9 +396,9 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
if name == "check":
extra = {"platform": {"type": "string"}, "ok": {"type": "boolean"}, "failed": {"type": "integer"}, "warnings": {"type": "integer"},
"notes": {"type": "array", "items": {"type": "string"}, "description": "present when no --platform was named: youtube was assumed and judgement rows are WARN"},
- "checks": {"type": "array", "items": {"type": "object", "properties": {"check": {"type": "string"}, "status": {"enum": ["PASS", "WARN", "FAIL"]}, "value": {}, "expected": {}, "fix": {"type": "string"}, "kind": {"enum": ["format", "judgement"]}}}}}
+ "checks": {"type": "array", "items": {"type": "object", "properties": {"check": {"type": "string", "description": "row name; --content adds black, frozen and silence; audio FAILs when the track is present but silent (peak <= -50 dBFS)"}, "status": {"enum": ["PASS", "WARN", "FAIL"]}, "value": {}, "expected": {}, "fix": {"type": "string"}, "kind": {"enum": ["format", "judgement"]}}}}}
elif name == "caption":
- extra = {"caption": {"type": "object", "description": "cue layout: shifted / wrapped / rebalanced / split / extended / dropped counts, plus wrap ('phrase' or 'measured'), phrase_breaks (1.16), broken_inside_word -- atoms hard-sliced at the column edge because they did not fit alone even at the size floor (1.18.4) -- and overlong -- now residual: a single character alone wider than the column"},
+ extra = {"caption": {"type": "object", "description": "cue layout: shifted / wrapped / rebalanced / split / extended / dropped counts, plus wrap ('phrase' or 'measured'), phrase_breaks (1.16), broken_inside_word -- atoms hard-sliced at the column edge because they did not fit alone even at the size floor (1.18.4) -- and overlong -- now residual: a single character alone wider than the column. Burn mode: cues_burned -- non-blank cues that overlap [0, duration] and are drawn -- and cues_outside -- non-blank cues wholly outside it (2.2.6); a burn with no visible cue is refused, kind input"},
"tracks": {"type": "array", "description": "--mode mux: one entry per subtitle stream in the output ({index, file, language, title, codec, default, cues, kept_from_input}); a stream the input already carried has file null and kept_from_input true (1.16)"},
"subtitle_tracks": {"type": "integer", "description": "--mode mux: how many subtitle streams the output carries"},
"emoji": {"type": "object", "description": "how the emoji in the text were drawn (mode, overlays, missing)"},
@@ -408,6 +410,7 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
"notes": {"type": "array", "items": {"type": "string"}}}
elif name == "waveform":
extra = {"audiogram": {"type": "object", "description": "{style, background ('image' or 'color'), image, position, vis_height, platform, captions, title, stages, verified} -- present on every run, so a plain waveform answers background 'color' (1.16)"},
+ "silent": {"type": ["boolean", "null"], "description": "true when the input audio's whole-file peak is at or below --silence-threshold (default -50 dBFS) and --on-silent warn (default) rendered the flat line anyway, with a notes line; false when audible; null under --dry-run (nothing measured). --on-silent fail refuses as kind input instead; verified is unaffected"},
"notes": {"type": "array", "items": {"type": "string"}}}
elif name == "scenes":
extra = {"file": {"type": "string"}, "duration": {"type": "number"}, "scene_count": {"type": "integer"}, "scenes": {"type": "array"}, "audio_peaks": {"type": "array"},
@@ -437,7 +440,7 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
elif name == "report":
extra = {"report": {"type": "string"}, "check": {"type": ["object", "null"]}}
elif name == "export":
- extra = {"loudness": {"type": "object", "description": "platform presets with audio: the written file's lufs/tp against the platform's target_lufs/target_tp, ok true when inside the spec; normalized true when --normalize ran loudness.py on the file"},
+ extra = {"loudness": {"type": "object", "description": "platform presets with audio: the written file's lufs/tp against the platform's target_lufs/target_tp, ok true when inside the spec; normalized true when --normalize ran loudness.py on the file; silent true (lufs \"-inf\", ok false, no normalize) when the written audio is silent"},
"notes": {"type": "array", "items": {"type": "string"}}}
elif name == "loudness":
extra = {"measured": {"type": "object", "description": "the loudnorm measurement of the input (input_i, input_tp, input_lra, input_thresh, target_offset); with --measure-only it is the whole result"},
@@ -461,12 +464,17 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
"sample_rate": {"type": "integer", "description": "audio mode only"}, "channels": {"type": "integer", "description": "audio mode only"},
"video": {"type": "boolean", "description": "false in audio mode: the output has no video stream"},
"skipped": {"type": "array", "description": "inputs --on-missing skip left out: [{index, path, reason}] ([] when none were)"},
+ "silent": {"type": "array", "description": "inputs whose audio peak is at or below --silence-threshold (default -50 dBFS) and that --on-silent warn (default) joined anyway: [{index, path, peak_db}] ([] otherwise; fail refuses them under problems, skip lists them under skipped); an input with no audio stream is never silent"},
+ "short_segments": {"type": "array", "description": "warning, never a refusal: measured inputs shorter than 2 frames at the join's fps (audio-only join: shorter than 0.05 s), [{index, path, duration}] ([] when none), each also in notes; a pending --dry-run input is not measured"},
+ "duplicates": {"type": "array", "description": "warning, never a refusal (a repeat can be intended): a path (compared resolved) listed more than once and joined each time, [{path, indices}] with 0-based list positions ([] when none), also in notes"},
"pending": {"type": "array", "description": "--dry-run: inputs that do not exist yet, planned on as an earlier step's output and never also under skipped: [{index, path}] ([] otherwise)"},
- "notes": {"type": "array", "items": {"type": "string"}, "description": "--dry-run with pending inputs: which ones, what a real run does if one is still missing (including a skip that leaves fewer than two inputs), whether the mode came from the extensions, and which planned numbers are the unmeasured stub's placeholders (a pending first clip's frame and rate, xfade offsets after a pending clip)"}}
+ "notes": {"type": "array", "items": {"type": "string"}, "description": "--on-silent warn: which silent inputs were joined anyway; short_segments / duplicates: which inputs and why; --dry-run with pending inputs: which ones, what a real run does if one is still missing (including a skip that leaves fewer than two inputs), whether the mode came from the extensions, and which planned numbers are the unmeasured stub's placeholders (a pending first clip's frame and rate, xfade offsets after a pending clip)"}}
elif name == "audio":
extra = {"video": {"type": "boolean", "description": "true when the input's video stream was copied; false for an audio output extension (extraction)"},
"audio_stream": {"type": "integer", "description": "which input audio stream was processed (--audio-stream)"},
"dynamics": {"type": "array", "items": {"enum": ["agate", "acompressor", "alimiter"]}, "description": "typed dynamics filters applied, in graph order"},
+ "silent": {"type": "array", "description": "--on-silent warn (default), only when a track was silent: [{flag, path, peak_db}] for each --music/--replace/--effects file, or under --duck the voice (flag \"input\"), whose whole-file peak is at or below --silence-threshold (default -50 dBFS); it was mixed anyway. --on-silent fail refuses instead, reason \"silent (peak X dBFS)\" in problems"},
+ "notes": {"type": "array", "items": {"type": "string"}, "description": "present with `silent`: which tracks were silent and how to refuse them"},
"audio": {"type": "object", "description": "what the mix was built from: voice (null | light | medium | strong), stereo_widen, effects/effects_volume, and with --music the music_volume plus duck (null when --duck was not given, else the threshold in dB and linear, ratio, attack_ms, release_ms, amount_db actually used)"}}
props = dict(base)
props.update(extra)
diff --git a/scripts/audio.py b/scripts/audio.py
index 0d0f699..1aac190 100755
--- a/scripts/audio.py
+++ b/scripts/audio.py
@@ -25,7 +25,7 @@ import sys
from pathlib import Path
from typing import Any, Dict, List
-from _common import STATE, add_common, dry_run_input_pending, require_tool, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, run, run_keeping_subtitles, fmt_secs
+from _common import STATE, add_common, dry_run_input_pending, require_tool, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, measured_level_dbfs, probe, run, run_keeping_subtitles, fmt_secs
VOICE_CHAIN = "highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2"
@@ -129,6 +129,30 @@ def preflight_beds(args: argparse.Namespace) -> List[Dict[str, Any]]:
return problems
+def find_silent_tracks(args: argparse.Namespace, problems: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+ """Every extra file (and, under --duck, the voice track that keys the ducking) whose whole-file
+ peak is at or below --silence-threshold dBFS: a silent --music / --replace / --effects bed
+ used to be mixed and reported as verified, and a silent voice leaves --duck with nothing to
+ duck on. Files already named in `problems`, and files an earlier stage has not written yet
+ (--dry-run), are not measured; a real file is measured under --dry-run too, like the preflight."""
+ bad = {p["path"] for p in problems}
+ tracks = [(f, v, None) for f, v in (("--replace", args.replace), ("--music", args.music),
+ ("--effects", args.effects)) if v and v not in bad]
+ if args.duck and not args.replace:
+ tracks.append(("input", args.input, args.audio_stream))
+ silent: List[Dict[str, Any]] = []
+ if STATE.dry_run:
+ # a writing tool's dry run runs no ffmpeg (docs/contract.md); the real run measures
+ return silent
+ for flag, path, stream in tracks:
+ if dry_run_input_pending(path) or not os.path.isfile(path):
+ continue
+ level = measured_level_dbfs(path, seconds=None, audio_stream=stream)
+ if level is not None and level["peak_dbfs"] <= args.silence_threshold:
+ silent.append({"flag": flag, "path": path, "peak_db": level["peak_dbfs"]})
+ return silent
+
+
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("input")
@@ -188,6 +212,11 @@ def main() -> int:
dyn.add_argument("--gate-knee", type=float, help="knee dB 1..8 (default 2.83)")
ap.add_argument("--audio-stream", type=int, default=0, help="which audio stream of the input to process, 0-based in file order (probe lists them under audio_streams)")
ap.add_argument("--bitrate", default="192k")
+ ap.add_argument("--on-silent", choices=["warn", "fail"], default="warn",
+ help="a --music/--replace/--effects file (or, under --duck, the voice) whose peak is at or below "
+ "--silence-threshold: warn (default: mix it, report it under `silent`) or fail (refuse before ffmpeg runs)")
+ ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DBFS",
+ help="peak level in dBFS at or below which a track counts as silent (default -50)")
add_common(ap)
args = ap.parse_args()
apply_common(args)
@@ -216,11 +245,23 @@ def main() -> int:
die(f"--stereo-widen must be 0..1 (0 = untouched, 1 = maximum), got {args.stereo_widen:g}")
problems = preflight_beds(args)
+ silent = find_silent_tracks(args, problems)
+ if args.on_silent == "fail":
+ problems += [{"flag": t["flag"], "path": t["path"], "reason": f"silent (peak {t['peak_db']:g} dBFS)"} for t in silent]
+ silent = []
if problems:
die(f"{len(problems)} of the extra audio files cannot be mixed: "
+ "; ".join(f"{p['flag']} {p['path']}: {p['reason']}" for p in problems),
kind="input", problems=problems,
- hint="each --music / --effects / --replace file must be a readable file with an audio stream")
+ hint="each --music / --effects / --replace file must be a readable file with an audio stream"
+ + (f" and a peak above {args.silence_threshold:g} dBFS (--on-silent warn mixes a silent one anyway)"
+ if args.on_silent == "fail" else ""))
+ notes: List[str] = []
+ if silent:
+ notes.append("silent track(s) mixed anyway: " + "; ".join(
+ f"{t['flag']} {t['path']} peaks at {t['peak_db']:g} dBFS" for t in silent)
+ + f" (at or below --silence-threshold {args.silence_threshold:g}); pass --on-silent fail to refuse instead")
+ info("warning: " + notes[-1])
meta = probe(args.input)
dur = meta.get("duration") or 0.0
@@ -376,7 +417,8 @@ def main() -> int:
if args.duck else None)
emit(output, audio=audio_block, video=bool(has_video and not audio_out), audio_stream=args.audio_stream,
dynamics=[f for f in (args.gate and "agate", args.compress and "acompressor", args.limit and "alimiter") if f],
- dropped_non_av_streams=dropped_streams)
+ dropped_non_av_streams=dropped_streams,
+ **({"silent": silent, "notes": notes} if silent else {}))
return 0
diff --git a/scripts/caption.py b/scripts/caption.py
index 6522ffb..e7b89a2 100755
--- a/scripts/caption.py
+++ b/scripts/caption.py
@@ -491,6 +491,53 @@ def parse_ass_dialogue(path: str) -> str:
return "\n".join(text)
+def parse_ass_cues(path: str) -> List[Tuple[float, float, str]]:
+ """(start, end, drawn text) of every Dialogue line of an ASS file, override blocks stripped."""
+ def secs(stamp: str) -> float:
+ h, m, rest = stamp.strip().split(":")
+ return int(h) * 3600 + int(m) * 60 + float(rest)
+
+ cues: List[Tuple[float, float, str]] = []
+ # the [Events] Format line names the columns; Text is always last (libass reads it the same
+ # way), and a file without one gets the standard ten
+ names = ["layer", "start", "end", "style", "name", "marginl", "marginr", "marginv", "effect", "text"]
+ for line in read_text_or_die(path, "--ass").lstrip("\ufeff").splitlines():
+ if line.startswith("Format:") and "text" in line.lower():
+ names = [n.strip().lower() for n in line.partition(":")[2].split(",")]
+ continue
+ if not line.startswith("Dialogue:"):
+ continue
+ fields = line.partition(":")[2].split(",", len(names) - 1)
+ if len(fields) < len(names) or "start" not in names or "end" not in names:
+ continue
+ try:
+ start, end = secs(fields[names.index("start")]), secs(fields[names.index("end")])
+ except (ValueError, IndexError):
+ continue
+ text = re.sub(r"\{[^}]*\}", "", fields[-1]).replace("\\N", "\n").replace("\\n", "\n").replace("\\h", " ")
+ cues.append((start, end, text))
+ return cues
+
+
+def cue_text_visible(text: str) -> bool:
+ return bool((text or "").strip())
+
+
+def count_visible_cues(cues, duration) -> Tuple[int, int]:
+ """(cues drawn inside [0, duration], non-blank cues entirely outside it). Blank cues count in
+ neither. With no known duration every non-blank cue is taken as drawn."""
+ burned = outside = 0
+ dur = float(duration) if duration else None
+ for start, end, text in cues:
+ if not cue_text_visible(text):
+ continue
+ if end > 0 and (dur is None or start < dur):
+ burned += 1
+ else:
+ outside += 1
+ return burned, outside
+
+
def shift_ass_file(src: str, dst: str, offset: float) -> int:
"""Copy an ASS file with every Dialogue start/end moved by `offset` seconds."""
def shift(stamp: str) -> str:
@@ -1247,8 +1294,8 @@ def main() -> int:
srt_path = new_srt
planned_cues = adjusted
side_notes.append(f"the burned subtitles are {new_srt}, the adjusted copy of "
- f"{os.path.basename(args.srt)} this run writes (--offset/--max-lines/--min-duration); "
- "re-run this command without --dry-run to produce it")
+ f"{os.path.basename(args.srt)} this run writes (--offset/--max-lines/--min-duration)"
+ + ("; re-run this command without --dry-run to produce it" if STATE.dry_run else ""))
if args.ass and args.offset and os.path.exists(args.ass):
shifted = os.path.splitext(output)[0] + "_offset.ass"
if STATE.dry_run:
@@ -1258,8 +1305,8 @@ def main() -> int:
info(f"wrote {shifted} ({n} cues shifted by {args.offset:+g} s)")
args.ass = shifted
side_notes.append(f"the burned subtitles are {shifted}, the offset copy of "
- f"{os.path.basename(ass_sample_path)} this run writes; re-run this command "
- "without --dry-run to produce it")
+ f"{os.path.basename(ass_sample_path)} this run writes"
+ + ("; re-run this command without --dry-run to produce it" if STATE.dry_run else ""))
# a side file this run has planned but (under --dry-run) not written is still the file the
# command names, so its absence must not be reported as a missing input
planned_only = STATE.dry_run and planned_cues is not None
@@ -1436,6 +1483,44 @@ def main() -> int:
if args.ass:
if not generated_ass and not os.path.exists(args.ass) and not planned_ass:
die(f"ASS file not found: {args.ass}")
+ # 2.2.6: a burn whose cues are all blank, or all outside the video, draws nothing -- and used
+ # to report verified. Count the cues that can actually be seen; none is a refusal.
+ if generated_ass:
+ burn_cues: Optional[List[Tuple[float, float, str]]] = cues_for_ass
+ elif args.ass and not os.path.exists(ass_sample_path):
+ # a dry run's pending --ass (an earlier step writes it): nothing to count yet
+ burn_cues = None
+ elif args.ass:
+ if planned_ass: # dry run: the offset copy is not written yet, shift the caller's file
+ burn_cues = [(s + args.offset, e + args.offset, t) for s, e, t in parse_ass_cues(ass_sample_path)]
+ else:
+ burn_cues = parse_ass_cues(args.ass)
+ if not burn_cues:
+ die(f"no Dialogue lines found in {ass_sample_path} -- nothing would be drawn", kind="input")
+ elif planned_cues is not None:
+ burn_cues = planned_cues
+ elif args.text or args.transcribe:
+ burn_cues = cues or None # empty under a --transcribe dry run: nothing to count yet
+ elif srt_path and os.path.exists(srt_path):
+ burn_cues = parse_srt(srt_path)
+ else:
+ burn_cues = None
+ if burn_cues:
+ burned, outside = count_visible_cues(burn_cues, meta.get("duration"))
+ if not burned and not outside:
+ die("every cue is blank -- nothing would be drawn", kind="input")
+ if not burned:
+ first = min(s for s, _e, t in burn_cues if cue_text_visible(t))
+ die(f"no cue falls inside the video (0\u2013{float(meta['duration']):.1f} s); first cue "
+ f"starts at {first:.1f} s -- check --offset or the cue file", kind="input")
+ if outside:
+ msg = (f"{outside} cue(s) fall outside the video (0\u2013{float(meta['duration']):.1f} s) "
+ "and are not drawn")
+ info(f"warning: {msg}")
+ side_notes.append(msg)
+ caption_stats["cues_burned"] = burned
+ caption_stats["cues_outside"] = outside
+ if args.ass:
vf = f"ass={escape_filter_path(args.ass)}"
if args.fonts_dir:
vf += f":fontsdir={escape_filter_path(args.fonts_dir)}"
diff --git a/scripts/check.py b/scripts/check.py
index 5d1371e..3d2dd07 100755
--- a/scripts/check.py
+++ b/scripts/check.py
@@ -18,6 +18,12 @@ Examples:
python3 check.py reel.mp4 --platform reels --json
python3 check.py spot.mov --platform broadcast
python3 check.py clip.mp4 --platform custom --max-duration 30 --aspect 1:1 --lufs -16
+ python3 check.py final.mp4 --platform reels --content
+
+--content (opt-in) adds three content rows from one decode pass: black (share of the
+duration that is black: WARN > 10%, FAIL >= 95%), frozen (longest frozen span: WARN >
+max(3 s, 30% of the duration), FAIL when the whole video is frozen) and silence (share
+below -50 dBFS: WARN > 50%, FAIL >= 95%). Without it the row set is unchanged.
"""
import argparse
import json
@@ -27,6 +33,7 @@ from fractions import Fraction
from typing import Any, Dict, List
from _platforms import PLATFORMS, PLATFORM_CHOICES, spec_of, resolve as resolve_platform
+from _common.probe import SILENT_PEAK_DBFS, detect_content, measured_level_dbfs
from _common import STATE, add_common, apply_common, die, emit, info, probe, require_tool, run, run_analysis, dry_run_input_pending
# The one delivery table (scripts/_platforms.py): check.py's rows, export.py's presets and the
@@ -52,6 +59,17 @@ def measure_loudness(path: str) -> Dict[str, float]:
return {}
+# --content thresholds (docs/design-decisions.md, "check.py --content").
+BLACK_WARN, BLACK_FAIL = 0.10, 0.95 # share of the duration that is black
+FROZEN_WARN_S, FROZEN_WARN_SHARE = 3.0, 0.30 # longest frozen span: WARN above max(3 s, 30%)
+FROZEN_FAIL = 0.95 # a frozen span covering this share = whole video frozen
+SILENCE_WARN, SILENCE_FAIL = 0.50, 0.95 # share of the duration below SILENT_PEAK_DBFS
+
+
+def _span_total(spans: Any) -> float:
+ return sum(max(0.0, e - s) for s, e in spans)
+
+
def aspect_name(w: int, h: int) -> str:
f = Fraction(w, h)
for name, target in (("16:9", Fraction(16, 9)), ("9:16", Fraction(9, 16)), ("1:1", Fraction(1)), ("4:5", Fraction(4, 5)), ("4:3", Fraction(4, 3)), ("21:9", Fraction(21, 9))):
@@ -70,6 +88,8 @@ def main() -> int:
ap.add_argument("--tp", type=float, help="override true-peak ceiling")
ap.add_argument("--max-mb", type=float, help="override max file size in MB")
ap.add_argument("--no-loudness", action="store_true", help="skip the loudness measurement (faster)")
+ ap.add_argument("--content", action="store_true",
+ help="also check the content (one decode pass): black, frozen and silence rows")
add_common(ap)
args = ap.parse_args()
apply_common(args)
@@ -193,10 +213,24 @@ def main() -> int:
"by number and the viewer has to guess") if untagged else "")
if a:
+ audio_row = next(r for r in rows if r["check"] == "audio")
+
+ def silent_audio(peak: float) -> None:
+ # a present-but-silent track is the failed-TTS / muted-export case: "present" is not a pass
+ # FAIL under a platform with a loudness target (it would fail loudness anyway); WARN
+ # otherwise: a muted screen recording or a b-roll clip is a legitimate silent track
+ audio_row.update(status="FAIL" if spec["lufs"] is not None else "WARN", value=audio_row["value"] + f", silent (peak {peak:.1f} dB)",
+ expected=f"present, peak above {SILENT_PEAK_DBFS:g} dBFS",
+ fix="audio.py --replace with the real track, or re-export from the master",
+ reason="the audio track exists but carries no sound: viewers hear nothing")
+ loudness_ran = False
if a.get("sample_rate") and a["sample_rate"] not in (44100, 48000):
row("sample rate", "WARN", a["sample_rate"], "44100 or 48000", "loudness.py --sample-rate 48000")
if not args.no_loudness and spec["lufs"] is not None:
lm = measure_loudness(args.input)
+ loudness_ran = bool(lm)
+ if lm and lm["tp"] <= SILENT_PEAK_DBFS:
+ silent_audio(lm["tp"])
if lm:
diff = abs(lm["lufs"] - spec["lufs"])
row("loudness", "PASS" if diff <= spec["lufs_tol"] else "FAIL", f"{lm['lufs']:.1f} LUFS", f"{spec['lufs']:g} ± {spec['lufs_tol']:g} LUFS", f"loudness.py -I {spec['lufs']:g} for speech or music; leave ambience/near-silence (<= -40 LUFS) alone and say so",
@@ -210,11 +244,39 @@ def main() -> int:
row("loudness", "WARN", "could not measure", f"{spec['lufs']:g} ± {spec['lufs_tol']:g} LUFS", "re-run check.py, or verify loudness manually",
reason="ffmpeg's loudness measurement didn't produce a readable result -- this was not actually checked")
row("true peak", "WARN", "could not measure", f"<= {spec['tp']:g} dBTP", "re-run check.py, or verify true peak manually")
+ if not loudness_ran and not dry_run_input_pending(args.input):
+ # loudness skipped (--no-loudness, or a spec with no target): one cheap volumedetect pass
+ lv = measured_level_dbfs(args.input, seconds=None)
+ if lv and lv["peak_dbfs"] <= SILENT_PEAK_DBFS:
+ silent_audio(lv["peak_dbfs"])
elif args.platform in ("podcast",):
row("audio", "FAIL", "none", "audio stream", "audio.py --replace")
else:
row("audio", "WARN", "none", "audio stream", "audio.py --replace (silent uploads are often rejected)")
+ if args.content:
+ cd = detect_content(args.input, dur, video=bool(v), audio=bool(a))
+ span = dur if dur > 0 else 0.0
+ if "black" in cd and span:
+ share = _span_total(cd["black"]) / span
+ row("black", "FAIL" if share >= BLACK_FAIL else ("WARN" if share > BLACK_WARN else "PASS"),
+ f"{share * 100:.0f}% black", f"<= {BLACK_WARN * 100:.0f}% black",
+ "look.py the black spans; trim them with cut.py or re-export from the master",
+ reason="an all-black or mostly black picture is usually a failed render or a missing clip")
+ if "frozen" in cd and span:
+ longest = max((e - s for s, e in cd["frozen"]), default=0.0)
+ limit = max(FROZEN_WARN_S, FROZEN_WARN_SHARE * span)
+ row("frozen", "FAIL" if longest >= FROZEN_FAIL * span else ("WARN" if longest > limit else "PASS"),
+ f"longest {longest:.1f}s", f"<= {limit:.1f}s frozen",
+ "look.py the frozen span: a still title card is fine, a stuck decode is not",
+ reason="a picture that never changes is usually a stuck frame, not the edit")
+ if "silence" in cd and span:
+ share = _span_total(cd["silence"]) / span
+ row("silence", "FAIL" if share >= SILENCE_FAIL else ("WARN" if share > SILENCE_WARN else "PASS"),
+ f"{share * 100:.0f}% silent", f"<= {SILENCE_WARN * 100:.0f}% below {SILENT_PEAK_DBFS:g} dBFS",
+ "silence.py to cut the gaps, or audio.py --replace when the track is empty",
+ reason="most of the running time has no audible sound")
+
failed = [r for r in rows if r["status"] == "FAIL"]
warned = [r for r in rows if r["status"] == "WARN"]
notes: List[str] = []
diff --git a/scripts/export.py b/scripts/export.py
index 86b5935..2197fc5 100755
--- a/scripts/export.py
+++ b/scripts/export.py
@@ -30,6 +30,7 @@ Examples:
"""
import argparse
import json
+import math
import os
import sys
from pathlib import Path
@@ -227,7 +228,14 @@ def main() -> int:
ok = abs(m["lufs"] - spec["lufs"]) <= spec["lufs_tol"] and m["tp"] <= spec["tp"]
extra["loudness"] = {"lufs": m["lufs"], "tp": m["tp"], "target_lufs": spec["lufs"], "target_tp": spec["tp"], "ok": ok}
extra["verification"] = [{"step": "loudness", "ok": ok, "platform": platform}]
- if not ok and args.normalize:
+ if not math.isfinite(m["lufs"]):
+ # a silent output measures -inf LUFS: no gain reaches the target, so recommending
+ # (or running, under --normalize) loudness.py only returns "nothing to normalise"
+ extra["loudness"]["silent"] = True
+ notes.append(f"output audio is silent (integrated loudness -inf, {platform} expects {spec['lufs']:g} LUFS); "
+ "check the source's audio or give the export a real track (audio.py --replace)")
+ info("warning: " + notes[-1])
+ elif not ok and args.normalize:
# Eval 7: every platform job ran export -> loudness.py -> export again (two video
# encodes). The levels pass only re-encodes audio, so do it here on the written
# file and the caller gets one export that meets the spec.
diff --git a/scripts/graphics.py b/scripts/graphics.py
index 40afb20..f6d30db 100755
--- a/scripts/graphics.py
+++ b/scripts/graphics.py
@@ -120,6 +120,12 @@ def main() -> int:
add_common(ap)
args = ap.parse_args()
apply_common(args)
+ # 2.2.6: whitespace-only text is no text -- stripped before the "needs --title" checks, which
+ # used to pass on " " and burn an empty bar
+ for _attr in ("name", "title", "subtitle", "text", "top", "bottom"):
+ _v = getattr(args, _attr, None)
+ if isinstance(_v, str):
+ setattr(args, _attr, _v if _v.strip() else None)
args.platform = resolve_platform(args.platform)
if args.platform and not PLATFORMS[args.platform].get("frame"):
diff --git a/scripts/join.py b/scripts/join.py
index 95b7374..bc0fb72 100755
--- a/scripts/join.py
+++ b/scripts/join.py
@@ -27,13 +27,19 @@ refused with every problem named at once (kind input), never discovered one run
Under --dry-run a segment that does not exist yet is an earlier step's output: it is planned
on and named under `pending` (never `skipped`), whichever --on-missing was given, and its
extension stands in for what it will hold (an audio extension: no picture).
+
+Every input with an audio stream is also measured (one volumedetect pass): a segment whose peak
+is at or below --silence-threshold (default -50 dBFS) is silent -- the trace of a TTS call that
+wrote a valid but empty wav. --on-silent warn (default) joins it and names it under `silent`;
+fail refuses it with the other input problems; skip drops it into `skipped`. A clip with no
+audio stream is never silent.
"""
import argparse
import os
import sys
from typing import Any, Dict, List, Optional, Tuple
-from _common import STATE, AUDIO_MEDIA_EXT, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, dry_run_input_pending, emit, ffmpeg_base, info, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
+from _common import STATE, AUDIO_MEDIA_EXT, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, dry_run_input_pending, emit, ffmpeg_base, info, measured_level_dbfs, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
TRANSITIONS = ["fade", "dissolve", "wipeleft", "wiperight", "wipeup", "wipedown", "slideleft", "slideright",
"circleopen", "circleclose", "fadeblack", "fadewhite", "smoothleft", "smoothright", "radial", "none"]
@@ -43,12 +49,23 @@ STATE_SKIPPED: List[Dict[str, Any]] = []
# --dry-run: inputs that do not exist yet, planned on as an earlier step's output
STATE_PENDING: List[Dict[str, Any]] = []
STATE_NOTES: List[str] = []
+# --on-silent warn: inputs whose audio peak is at or below --silence-threshold, joined anyway
+STATE_SILENT: List[Dict[str, Any]] = []
+# warnings, never refusals: a clip shorter than MIN_SEGMENT_FRAMES frames (audio-only: shorter than
+# MIN_AUDIO_SEGMENT seconds) and a path listed more than once (repeating a clip can be intentional)
+STATE_SHORT: List[Dict[str, Any]] = []
+STATE_DUPES: List[Dict[str, Any]] = []
+# original 0-based list position of each input still in args.inputs (after any skip)
+STATE_KEPT: List[int] = []
+MIN_SEGMENT_FRAMES = 2
+MIN_AUDIO_SEGMENT = 0.05
def reported() -> Dict[str, Any]:
"""The input bookkeeping both success documents carry: `skipped` is what the join left out,
`pending` what a dry run planned on without it existing; `notes` says the same in words."""
- extra: Dict[str, Any] = {"skipped": list(STATE_SKIPPED), "pending": list(STATE_PENDING)}
+ extra: Dict[str, Any] = {"skipped": list(STATE_SKIPPED), "pending": list(STATE_PENDING), "silent": list(STATE_SILENT),
+ "short_segments": list(STATE_SHORT), "duplicates": list(STATE_DUPES)}
if STATE_NOTES:
extra["notes"] = list(STATE_NOTES)
return extra
@@ -77,6 +94,29 @@ def clip_length(meta: Dict[str, Any], fps: float) -> float:
return v
+def find_duplicates(paths: List[str]) -> List[Dict[str, Any]]:
+ """Every path (compared by its resolved absolute form) listed more than once, with the 0-based
+ list positions it appears at. A warning, not a refusal: repeating a clip can be intentional."""
+ seen: Dict[str, List[int]] = {}
+ for i, p in enumerate(paths):
+ seen.setdefault(os.path.realpath(p), []).append(i)
+ return [{"path": paths[ix[0]], "indices": ix} for ix in seen.values() if len(ix) > 1]
+
+
+def note_short_segments(paths: List[str], metas: List[dict], minimum: float, unit: str) -> None:
+ """Record every measured input shorter than `minimum` seconds (a pending dry-run stub is not
+ measured): such a clip joins as a flash nobody sees -- usually a failed earlier step."""
+ for p, m, k in zip(paths, metas, STATE_KEPT or range(len(paths))):
+ dur = m.get("duration") or 0.0
+ if m.get("dry_run") or dur >= minimum:
+ continue
+ STATE_SHORT.append({"index": k, "path": p, "duration": round(dur, 4)})
+ if STATE_SHORT:
+ listing = ", ".join(f"#{s['index'] + 1} {s['path']} ({s['duration']:g}s)" for s in STATE_SHORT)
+ info(f"warning: {len(STATE_SHORT)} input(s) shorter than {unit}: {listing}")
+ STATE_NOTES.append(f"{len(STATE_SHORT)} input(s) are shorter than {unit} and were joined anyway: {listing}")
+
+
def expected_text(expected: Optional[float]) -> str:
"""unpending_length() as the stderr summary line puts it."""
return f"expected ~{expected:.3f}s" if expected is not None else "expected length unknown until the pending inputs exist"
@@ -87,6 +127,7 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
n = len(args.inputs)
durs = [m.get("duration") or 0.0 for m in metas]
d = args.duration if args.transition != "none" else 0.0
+ note_short_segments(args.inputs, metas, MIN_AUDIO_SEGMENT, f"{MIN_AUDIO_SEGMENT:g}s")
for p, dur in zip(args.inputs, durs):
if d and dur <= d * 2 and not STATE.dry_run:
die(f"{p} is only {dur:.2f}s, too short for a {d:.2f}s crossfade; shorten --duration")
@@ -164,12 +205,13 @@ def read_list(path: str) -> List[str]:
return entries
-def preflight(paths: List[str]) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]:
+def preflight(paths: List[str], audible: Optional[List[int]] = None) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]:
"""Every problem with every input, found before anything runs: missing, empty (0 bytes,
the usual trace of a TTS or download step that failed after creating its file), or not
readable as media. One ffprobe per file; nothing is decoded. Under --dry-run a file that
does not exist yet is not a problem but pending (the second list, with a note): in a
- planned pipeline it is an earlier step's output, and the plan keeps it."""
+ planned pipeline it is an earlier step's output, and the plan keeps it. The index of every
+ readable input with an audio stream is appended to `audible` (for find_silent())."""
ffprobe = require_tool("ffprobe")
problems: List[Dict[str, Any]] = []
pending: List[Dict[str, Any]] = []
@@ -192,9 +234,30 @@ def preflight(paths: List[str]) -> Tuple[List[Dict[str, Any]], List[Dict[str, An
if proc.returncode != 0 or not kinds & {"audio", "video"}:
first = ((proc.stderr or "").strip().splitlines() or ["no audio or video stream"])[-1]
problems.append({"index": i, "path": p, "reason": f"unreadable: {first}"})
+ elif "audio" in kinds and audible is not None:
+ audible.append(i)
return problems, pending
+def find_silent(paths: List[str], audible: List[int], threshold: float) -> List[Dict[str, Any]]:
+ """The audible inputs whose whole-file peak (volumedetect max_volume) is at or below
+ `threshold` dBFS: one decode of the audio per input. Not under --dry-run: join is a writing
+ tool, and a dry run of it (or of render.py, which joins through it) runs no ffmpeg -- the plan
+ notes that silence is measured on the real run. An input whose level cannot be measured is
+ left to the join itself."""
+ silent: List[Dict[str, Any]] = []
+ if STATE.dry_run:
+ if audible:
+ STATE_NOTES.append("silence is not measured under --dry-run; a real run measures every input with audio "
+ "for --on-silent")
+ return silent
+ for i in audible:
+ lv = measured_level_dbfs(paths[i], seconds=86400)
+ if lv is not None and lv["peak_dbfs"] <= threshold:
+ silent.append({"index": i, "path": paths[i], "peak_db": lv["peak_dbfs"]})
+ return silent
+
+
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("inputs", nargs="*", help="two or more clips in order (or --list FILE)")
@@ -203,6 +266,12 @@ def main() -> int:
ap.add_argument("--on-missing", choices=["fail", "skip"], default="fail",
help="a missing, empty or unreadable input: fail (default; every problem named at once) or skip "
"(join the rest; each skipped input is reported under `skipped`)")
+ ap.add_argument("--on-silent", choices=["warn", "fail", "skip"], default="warn",
+ help="an input whose audio peak is at or below --silence-threshold: warn (default; join it, "
+ "name it under `silent`), fail (refuse with the other input problems) or skip (leave it out, "
+ "under `skipped`); a clip with no audio stream is never silent")
+ ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DB",
+ help="peak level in dBFS at or below which an input counts as silent (default -50)")
ap.add_argument("-o", "--output", help="output file (default: <first>_joined.mp4)")
ap.add_argument("--transition", choices=TRANSITIONS, default="fade", help="transition between clips (default fade)")
ap.add_argument("--duration", type=float, default=0.5, help="transition length in seconds (default 0.5)")
@@ -228,19 +297,52 @@ def main() -> int:
args.inputs = read_list(args.list)
if not args.inputs:
die("give at least two clips (or --list FILE)")
+ if args.silence_threshold >= 0:
+ die(f"--silence-threshold is a peak level in dBFS and must be negative, got {args.silence_threshold:g}")
skipped: List[Dict[str, Any]] = []
- problems, pending = preflight(args.inputs)
- if problems and args.on_missing == "fail":
- listing = "; ".join(f"#{p['index'] + 1} {p['path']}: {p['reason']}" for p in problems)
- die(f"{len(problems)} of {len(args.inputs)} inputs cannot be joined: {listing}",
- hint="fix or regenerate them, or pass --on-missing skip to join the rest", problems=problems)
- if problems:
- bad = {p["index"] for p in problems}
- skipped = problems
+ orig_inputs = list(args.inputs)
+ audible: List[int] = []
+ problems, pending = preflight(args.inputs, audible)
+ silent = find_silent(args.inputs, audible, args.silence_threshold)
+ silent_problems = [{"index": s["index"], "path": s["path"], "reason": f"silent (peak {s['peak_db']:.1f} dBFS)"}
+ for s in silent]
+ refused = sorted((problems if args.on_missing == "fail" else [])
+ + (silent_problems if args.on_silent == "fail" else []), key=lambda p: p["index"])
+ if refused:
+ listing = "; ".join(f"#{p['index'] + 1} {p['path']}: {p['reason']}" for p in refused)
+ ways = []
+ if args.on_missing == "fail" and problems:
+ ways.append("--on-missing skip")
+ if args.on_silent == "fail" and silent:
+ ways.append("--on-silent skip (or warn)")
+ die(f"{len(refused)} of {len(args.inputs)} inputs cannot be joined: {listing}",
+ hint="fix or regenerate them, or pass " + " and ".join(ways) + " to join the rest", problems=refused)
+ dropped = sorted((problems if args.on_missing == "skip" else [])
+ + (silent_problems if args.on_silent == "skip" else []), key=lambda p: p["index"])
+ if dropped:
+ bad = {p["index"] for p in dropped}
+ skipped = dropped
args.inputs = [p for i, p in enumerate(args.inputs) if i not in bad]
- for p in problems:
+ for p in dropped:
info(f"skipping #{p['index'] + 1} {p['path']}: {p['reason']}")
+ if silent and args.on_silent == "warn":
+ STATE_SILENT[:] = silent
+ listing = ", ".join(f"#{s['index'] + 1} {s['path']} (peak {s['peak_db']:.1f} dBFS)" for s in silent)
+ info(f"warning: {len(silent)} input(s) silent at or below {args.silence_threshold:g} dBFS, joined anyway: {listing}")
+ STATE_NOTES.append(f"{len(silent)} input(s) have no audible sound (peak at or below {args.silence_threshold:g} dBFS) "
+ f"and were joined anyway: {listing}; pass --on-silent fail or skip to refuse or drop them")
STATE_SKIPPED[:] = skipped
+ STATE_KEPT[:] = [i for i in range(len(args.inputs) + len(skipped)) if i not in {p["index"] for p in skipped}]
+ dupes = [dd for dd in find_duplicates(orig_inputs)
+ if len([i for i in dd["indices"] if i in STATE_KEPT]) > 1]
+ for dd in dupes:
+ dd["indices"] = [i for i in dd["indices"] if i in STATE_KEPT]
+ STATE_DUPES[:] = dupes
+ if dupes:
+ listing = "; ".join(f"{dd['path']} at #" + ", #".join(str(i + 1) for i in dd["indices"]) for dd in dupes)
+ info(f"warning: input(s) listed more than once: {listing}")
+ STATE_NOTES.append(f"{len(dupes)} input(s) are listed more than once and joined each time: {listing}; "
+ "fine if the repeat is intended")
STATE_PENDING[:] = pending
if pending:
# a dry run cannot know whether the file will be there: it plans on it, and says what a
@@ -305,6 +407,7 @@ def main() -> int:
fps = args.fps or first.get("fps") or 30.0
fps = round(fps) if abs(fps - round(fps)) < 0.02 else fps
w, h = w - (w % 2), h - (h % 2)
+ note_short_segments(args.inputs, metas, MIN_SEGMENT_FRAMES / fps, f"{MIN_SEGMENT_FRAMES} frames at {fps:g} fps")
durs = [m.get("duration") or 0.0 for m in metas]
d = args.duration if args.transition != "none" else 0.0
# a crossfaded join gives each clip one length for both streams, so the offsets that place
diff --git a/scripts/overlay.py b/scripts/overlay.py
index d6defec..c0ef259 100755
--- a/scripts/overlay.py
+++ b/scripts/overlay.py
@@ -154,6 +154,8 @@ def main() -> int:
args.scale = int(brand.get("logo_scale", 160))
if args.opacity == 1.0:
args.opacity = float(brand.get("logo_opacity", 1.0))
+ if args.text is not None and not args.text.strip():
+ die("--text is blank -- give the words to draw", kind="input")
if not (args.image or args.text or args.video):
die("give --image, --text, --logo or --video")
if args.brand:
diff --git a/scripts/render.py b/scripts/render.py
index 7b7f6ea..1f7f550 100755
--- a/scripts/render.py
+++ b/scripts/render.py
@@ -29,7 +29,7 @@ Project format (all keys optional except clips):
"loudness": {"lufs": -14, "tp": -1},
"fit": {"duration": 60},
"export": {"preset": "reels", "normalize": true}, (default for platform presets; false opts out)
- "check": {"platform": "reels"},
+ "check": {"platform": "reels"}, ("content": true adds check.py --content's black/frozen/silence rows)
"chapters": "chapters.txt" (or [{"at": "0:00", "title": "Intro"}, ...])
}
@@ -149,7 +149,7 @@ OBJECT_KEYS: Dict[str, frozenset] = {
"loudness": frozenset({"lufs", "tp"}),
"fit": frozenset({"duration", "method", "aspect", "fit", "width", "height", "fps", "smooth"}),
"export": frozenset({"preset", "fit", "crf", "normalize"}),
- "check": frozenset({"platform"}),
+ "check": frozenset({"platform", "content"}),
}
# Typos difflib cannot see: a clip is trimmed with in/out, not the start/end that time a title.
NEAR_KEYS: Dict[str, Dict[str, str]] = {"clips[]": {"start": "in", "end": "out", "from": "in", "to": "out"},
@@ -1341,7 +1341,8 @@ def main() -> int:
check_result = None
exit_code = 0
if ck and ck.get("platform") and not STATE.dry_run:
- proc = run_tool([str(HERE / "check.py"), output, "--platform", ck["platform"], "--json"] + child_args())
+ proc = run_tool([str(HERE / "check.py"), output, "--platform", ck["platform"], "--json"]
+ + (["--content"] if ck.get("content") is True else []) + child_args())
try:
check_result = json.loads(proc.stdout)
except ValueError:
diff --git a/scripts/waveform.py b/scripts/waveform.py
index d96d37f..31bf4a7 100644
--- a/scripts/waveform.py
+++ b/scripts/waveform.py
@@ -37,7 +37,7 @@ import sys
from _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platform
from _common import (add_common, apply_common, aac_args, default_output, die, emit, ffmpeg_base,
info, load_brand, pad_filters, probe, run, STATE, validate_color, video_args, X264_PRESETS,
- fmt_secs, child_args, run_tool)
+ fmt_secs, child_args, run_tool, measured_level_dbfs)
WAVEFORM_MODES = ["point", "line", "p2p", "cline"]
@@ -56,6 +56,11 @@ def main() -> int:
ap.add_argument("--split-channels", action="store_true", help="draw each channel in its own lane instead of summing to one")
ap.add_argument("--audio-stream", type=int, default=0,
help="which audio stream of the input to render, 0-based in file order (default 0)")
+ ap.add_argument("--on-silent", choices=["warn", "fail"], default="warn",
+ help="input audio whose peak is at or below --silence-threshold (the render would be a flat "
+ "line): warn (default: render it, report `silent: true`) or fail (refuse before ffmpeg runs)")
+ ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DBFS",
+ help="peak level in dBFS at or below which the input counts as silent (default -50)")
ag = ap.add_argument_group("audiogram (1.16)",
"The visualisation over a picture, for an episode that has no video. The image is a "
"local file you give: this skill has no network access and never invents cover art.")
@@ -136,6 +141,19 @@ def main() -> int:
audio_streams = meta.get("audio_streams") or []
if audio_streams and not (0 <= args.audio_stream < len(audio_streams)):
die(f"--audio-stream {args.audio_stream}: input has {len(audio_streams)} audio stream(s), 0..{len(audio_streams) - 1}")
+ if args.silence_threshold >= 0:
+ die(f"--silence-threshold is a peak level in dBFS and must be negative, got {args.silence_threshold:g}")
+ # silent input draws a flat line that used to be reported as verified; a writing tool's dry
+ # run runs no ffmpeg (docs/contract.md), so only the real run measures
+ silent_peak = None
+ if not STATE.dry_run:
+ level = measured_level_dbfs(args.input, seconds=None, audio_stream=args.audio_stream)
+ if level is not None and level["peak_dbfs"] <= args.silence_threshold:
+ silent_peak = level["peak_dbfs"]
+ if args.on_silent == "fail":
+ die(f"{args.input} is silent (audio stream {args.audio_stream} peaks at {silent_peak:g} dBFS, at or below "
+ f"--silence-threshold {args.silence_threshold:g}): the visualisation would be a flat line",
+ kind="input", hint="check the audio, or pass --on-silent warn to render it anyway")
output = args.output or default_output(args.input, "waveform")
# The visualisation's own band. Without --image/--position it fills the frame, which is what
@@ -243,6 +261,10 @@ def main() -> int:
notes.append(f"the visualisation band and {args.platform}'s bottom safe zone overlap by "
f"{overlap}px: the captions or the app's own UI will sit over the waveform "
f"(lower --vis-height to {max(0.05, (args.height - safe_px) / args.height):.2f})")
+ if silent_peak is not None:
+ info(f"warning: {args.input} is silent (peak {silent_peak:g} dBFS); the visualisation is a flat line")
+ notes.append(f"the input audio is silent (peak {silent_peak:g} dBFS, at or below --silence-threshold "
+ f"{args.silence_threshold:g}): the visualisation is a flat line; pass --on-silent fail to refuse instead")
duration_ok = True
if not STATE.dry_run and meta.get("duration") and result.get("duration"):
duration_ok = abs(float(result["duration"]) - float(meta["duration"])) <= 0.05
@@ -259,7 +281,7 @@ def main() -> int:
notes.append(f"the render is {float(v['fps']):g} fps against the {args.fps:g} fps asked for")
# reported on every run, not only an audiogram one: a caller that keys on `audiogram.background`
# should not have to guess whether the key exists (`"color"` is the plain-waveform answer).
- extra = {"audiogram": {
+ extra = {"silent": None if STATE.dry_run else silent_peak is not None, "audiogram": {
"style": args.style,
"background": "image" if args.image else "color",
"image": args.image,
diff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.json
index c4c3e23..4247b8d 100644
--- a/tests/fixtures/mcp_tools.json
+++ b/tests/fixtures/mcp_tools.json
@@ -45,10 +45,12 @@
"music_fade_out",
"music_loop",
"music_volume",
+ "on_silent",
"output",
"overwrite",
"plan",
"replace",
+ "silence_threshold",
"stereo",
"stereo_widen",
"timeout",
@@ -193,6 +195,7 @@
"properties": [
"argv",
"aspect",
+ "content",
"dry_run",
"fast",
"input",
@@ -569,6 +572,7 @@
"json_brief",
"list",
"on_missing",
+ "on_silent",
"output",
"overwrite",
"pad_color",
@@ -576,6 +580,7 @@
"preset",
"quality",
"sample_rate",
+ "silence_threshold",
"timeout",
"transition",
"width"
@@ -1163,6 +1168,7 @@
"image_fit",
"input",
"json_brief",
+ "on_silent",
"opacity",
"output",
"overwrite",
@@ -1171,6 +1177,7 @@
"position",
"preset",
"quality",
+ "silence_threshold",
"split_channels",
"srt",
"style",
diff --git a/tests/test_analysis.py b/tests/test_analysis.py
index cc23971..6c52e92 100644
--- a/tests/test_analysis.py
+++ b/tests/test_analysis.py
@@ -244,6 +244,63 @@ class AnalysisTests(MediaFixtures):
self.assertEqual(h["bit_depth"], 10)
self.assertEqual(h["codec"], "hevc")
+ # ---------------------------------------------------------------- check --content
+ def _content_clip(self, name, video, audio, seconds=6):
+ path = OUT / name
+ if not path.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", video,
+ "-f", "lavfi", "-i", audio, "-t", str(seconds), "-r", "10", "-pix_fmt", "yuv420p",
+ "-c:v", "libx264", "-c:a", "aac", "-shortest", path)
+ return path
+
+ def _rows(self, path, *extra):
+ proc = subprocess.run([sys.executable, str(SCRIPTS / "check.py"), str(path), "--platform", "custom",
+ "--json", *extra], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ return {r["check"]: r for r in json.loads(proc.stdout)["checks"]}
+
+ def test_check_content_rows(self):
+ tone = "sine=frequency=440:sample_rate=48000"
+ normal = self._content_clip("content_normal.mp4", "testsrc2=size=160x90:rate=10", tone)
+ black = self._content_clip("content_black.mp4", "color=black:size=160x90:rate=10", tone)
+ frozen = self._content_clip("content_frozen.mp4", "color=gray:size=160x90:rate=10,drawbox=x=20:y=20:w=40:h=40:color=red:t=fill", tone)
+ # 2 s moving then 4 s held: the held span (4 s) exceeds max(3 s, 30% of 6 s) but is not the whole clip
+ part = OUT / "content_part_frozen.mp4"
+ if not part.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i",
+ "testsrc2=size=160x90:rate=10,trim=duration=2,tpad=stop_mode=clone:stop_duration=4",
+ "-f", "lavfi", "-i", tone, "-t", "6", "-pix_fmt", "yuv420p", "-c:v", "libx264", "-c:a", "aac", part)
+ silent = self._content_clip("content_silent.mp4", "testsrc2=size=160x90:rate=10", "anullsrc=r=48000:cl=mono")
+ # 1.5 s black then 4.5 s picture: 25% black -> WARN; tone for 2 s then 4 s silence -> 67% silent -> WARN
+ mixed = OUT / "content_mixed.mp4"
+ if not mixed.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i",
+ "testsrc2=size=160x90:rate=10,drawbox=t=fill:color=black:enable='lt(t,1.5)'",
+ "-f", "lavfi", "-i", tone + ",volume=enable='gte(t,2)':volume=0", "-t", "6",
+ "-pix_fmt", "yuv420p", "-c:v", "libx264", "-c:a", "aac", mixed)
+ r = self._rows(normal, "--content")
+ self.assertEqual([r[k]["status"] for k in ("black", "frozen", "silence")], ["PASS"] * 3, r)
+ self.assertEqual(self._rows(black, "--content")["black"]["status"], "FAIL")
+ self.assertEqual(self._rows(frozen, "--content")["frozen"]["status"], "FAIL")
+ self.assertEqual(self._rows(part, "--content")["frozen"]["status"], "WARN")
+ self.assertEqual(self._rows(silent, "--content")["silence"]["status"], "FAIL")
+ r = self._rows(mixed, "--content")
+ self.assertEqual((r["black"]["status"], r["silence"]["status"]), ("WARN", "WARN"), r)
+
+ def test_check_default_rows_unchanged_without_content(self):
+ clip = self._content_clip("content_black.mp4", "color=black:size=160x90:rate=10",
+ "sine=frequency=440:sample_rate=48000")
+ rows = self._rows(clip, "--no-loudness")
+ self.assertEqual(list(rows), ["duration", "aspect", "vfr", "colour", "audio", "subtitles"])
+ self.assertEqual(rows["audio"]["status"], "PASS")
+
+ def test_check_audio_row_fails_on_silent_track(self):
+ silent = self._content_clip("content_silent.mp4", "testsrc2=size=160x90:rate=10", "anullsrc=r=48000:cl=mono")
+ # FAIL under a loudness target; WARN with none (a muted screen recording is legitimate)
+ for extra, status in ((["--lufs", "-14"], "FAIL"), (["--platform", "custom", "--no-loudness"], "WARN")):
+ row = self._rows(silent, *extra)["audio"]
+ self.assertEqual(row["status"], status, (extra, row))
+ self.assertIn("silent", row["value"])
+
def test_check_podcast_reports_chapters_and_channel_count(self):
"""1.13: two informational podcast rows. Neither may FAIL a delivery, and neither may
appear for a platform that does not care."""
diff --git a/tests/test_audio.py b/tests/test_audio.py
index 192d9dd..ed11e29 100644
--- a/tests/test_audio.py
+++ b/tests/test_audio.py
@@ -6,6 +6,7 @@
"""
import json
import shlex
+import subprocess
import sys
import unittest
@@ -546,6 +547,40 @@ class AudiogramTests(MediaFixtures):
self.assertFalse(doc["verified"])
self.assertFalse(doc["audiogram"]["verified"])
+ def test_waveform_silent_input_warns_or_fails(self):
+ """A silent input renders a flat line that used to be reported as verified. The real run
+ measures the input's peak (same -50 dBFS --silence-threshold and --on-silent warn|fail as
+ audio.py): warn (default) renders it with `silent: true` and a note, fail refuses as kind
+ input before ffmpeg runs; --dry-run measures nothing (`silent: null`)."""
+ src = OUT / "wave_silent.wav"
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "anullsrc=d=1", "-t", "1", src)
+ doc = json.loads(script("waveform.py", src, "--width", "128", "--height", "128", "--json",
+ "-o", OUT / "wave_silent.mp4").stdout)
+ self.assertIs(doc["silent"], True)
+ self.assertTrue(any("silent" in n for n in doc["notes"]))
+ proc = script("waveform.py", src, "--on-silent", "fail", "--width", "128", "--height", "128", "--json",
+ "-o", OUT / "wave_silent_fail.mp4", expect_fail=True)
+ doc = json.loads(proc.stdout)
+ self.assertEqual((doc["error"]["kind"], doc["commands"]), ("input", []))
+ self.assertIn("silent", doc["error"]["message"])
+ self.assertFalse((OUT / "wave_silent_fail.mp4").exists())
+ doc = json.loads(script("waveform.py", src, "--dry-run", "--json", "-o", OUT / "wave_silent_dry.mp4").stdout)
+ self.assertIsNone(doc["silent"])
+ doc = json.loads(script("waveform.py", self.mic, "--width", "128", "--height", "128", "--json",
+ "-o", OUT / "wave_loud.mp4").stdout)
+ self.assertIs(doc["silent"], False)
+
+ def test_audiogram_refuses_a_zero_by_zero_image(self):
+ """A truncated PNG that ffprobe reports as 0x0 is an input refusal naming the image."""
+ full = self._plate(64, 64)
+ trunc = OUT / "ag_trunc.png"
+ trunc.write_bytes(full.read_bytes()[:30])
+ proc = script("waveform.py", self.mic, "--image", trunc, "-o", OUT / "ag_trunc.mp4",
+ "--json", expect_fail=True)
+ doc = json.loads(proc.stdout)
+ self.assertEqual(doc["error"]["kind"], "input")
+ self.assertIn(str(trunc), doc["error"]["message"])
+
def test_audiogram_refuses_an_image_ffmpeg_cannot_decode(self):
"""Spec 2.6: a file that is not a decodable image is an input refusal naming it, before
ffmpeg is ever started -- not a raw ffmpeg failure."""
@@ -628,6 +663,18 @@ class AudiogramTests(MediaFixtures):
self.assertEqual(doc["commands"], [], "nothing was encoded")
self.assertFalse((OUT / "wf_nosrt_vis.mp4").exists())
+ def test_waveform_srt_outside_the_render_is_refused_as_input(self):
+ """2.2.6: caption.py refuses a cue file none of whose cues falls inside the video; the
+ audiogram's caption stage passes that refusal on as kind input."""
+ srt = OUT / "wf_late.srt"
+ srt.write_text("1\n00:10:00,000 --> 00:10:02,000\nLate\n", encoding="utf-8")
+ out = OUT / "wf_late.mp4"
+ proc = script("waveform.py", self.mic, "--width", "320", "--height", "180", "--srt", srt,
+ "--json", "-o", out, expect_fail=True)
+ doc = json.loads(proc.stdout)
+ self.assertEqual(doc["error"]["kind"], "input")
+ self.assertIn("no cue falls inside the video", doc["error"]["message"])
+
def test_waveform_failed_caption_stage_keeps_child_kind_and_cleans_up(self):
"""A caption stage that fails re-raises caption.py's own kind (here: an existing output
without --overwrite is an input refusal), and the _vis intermediate is removed."""
@@ -668,6 +715,88 @@ class AudiogramTests(MediaFixtures):
self.assertIn(str(pics[1]), doc["error"]["message"])
self.assertEqual(doc["commands"], [])
+ # ------------------------------------------------------------ silent tracks (2.2.6)
+ def _silent_files(self):
+ wav = OUT / "silent_bed.wav"
+ if not wav.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", "anullsrc=r=48000:cl=stereo", "-t", "3", wav)
+ clip = OUT / "silent_voice_clip.mp4"
+ if not clip.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc=d=3:s=160x120",
+ "-f", "lavfi", "-i", "anullsrc=r=48000:cl=stereo", "-t", "3", "-c:v", "libx264", "-c:a", "aac", clip)
+ return wav, clip
+
+ def test_audio_silent_beds_warn_by_default_and_fail_on_request(self):
+ """A silent --music / --effects / --replace file used to be mixed and reported as verified.
+ Default warn: mixed, named under `silent` and in a note. --on-silent fail: named in the
+ single input refusal, before ffmpeg runs. A dry run measures nothing (a writing tool's dry
+ run runs no ffmpeg), so it completes and the real run is the one that refuses."""
+ wav, _ = self._silent_files()
+ for flag in ("--music", "--effects", "--replace"):
+ doc = json.loads(script("audio.py", self.src, flag, wav, "--json",
+ "-o", OUT / "silent_bed_out.mp4").stdout)
+ self.assertEqual(doc["status"], "completed")
+ self.assertEqual([(t["flag"], t["path"]) for t in doc["silent"]], [(flag, str(wav))])
+ self.assertLessEqual(doc["silent"][0]["peak_db"], -50)
+ self.assertTrue(any("silent" in n for n in doc["notes"]))
+ proc = script("audio.py", self.src, flag, wav, "--on-silent", "fail", "--json",
+ "-o", OUT / "silent_bed_out.mp4", expect_fail=True)
+ err = json.loads(proc.stdout)
+ self.assertEqual(err["error"]["kind"], "input")
+ self.assertEqual(len(err["problems"]), 1)
+ self.assertEqual(err["problems"][0]["flag"], flag)
+ self.assertRegex(err["problems"][0]["reason"], r"^silent \(peak -?[0-9.]+ dBFS\)$")
+ self.assertEqual(err["commands"], [])
+ doc = json.loads(script("audio.py", self.src, flag, wav, "--on-silent", "fail", "--json", "--dry-run",
+ "-o", OUT / "silent_bed_out.mp4").stdout)
+ self.assertEqual(doc["status"], "completed")
+ # a real bed is never reported, and a lower threshold lets the silent one through
+ doc = json.loads(script("audio.py", self.src, "--music", self.mic, "--json", "--dry-run",
+ "-o", OUT / "silent_bed_out.mp4").stdout)
+ self.assertNotIn("silent", doc)
+ doc = json.loads(script("audio.py", self.src, "--music", wav, "--silence-threshold", "-120",
+ "--on-silent", "fail", "--json", "-o", OUT / "silent_bed_out.mp4").stdout)
+ self.assertEqual(doc["status"], "completed")
+
+ def test_audio_duck_under_a_silent_voice_is_reported(self):
+ _, clip = self._silent_files()
+ doc = json.loads(script("audio.py", clip, "--music", self.mic, "--duck", "--json",
+ "-o", OUT / "silent_duck_out.mp4").stdout)
+ self.assertEqual([t["flag"] for t in doc["silent"]], ["input"])
+ proc = script("audio.py", clip, "--music", self.mic, "--duck", "--on-silent", "fail", "--json",
+ "-o", OUT / "silent_duck_out.mp4", expect_fail=True)
+ self.assertEqual([p["flag"] for p in json.loads(proc.stdout)["problems"]], ["input"])
+
+ def test_audio_every_tool_json_on_silent_input_parses_strictly(self):
+ """A silent file measures -inf LUFS; json.dumps wrote it as -Infinity (export.py's
+ `loudness.lufs`), which no strict JSON parser reads. Every --json document must parse
+ with NaN/Infinity refused, and export's note must say the audio is silent."""
+ _, clip = self._silent_files()
+
+ def strict(text):
+ def refuse(c):
+ raise ValueError(f"non-standard JSON constant {c}")
+ return json.loads(text, parse_constant=refuse)
+
+ runs = [("probe.py", clip), ("check.py", clip, "--platform", "youtube"),
+ ("loudness.py", clip, "--measure-only"), ("loudness.py", clip, "-o", OUT / "silent_ln.mp4"),
+ ("export.py", clip, "--preset", "youtube", "-o", OUT / "silent_export.mp4"),
+ ("audio.py", clip, "--fade-in", "0.5", "-o", OUT / "silent_audio.mp4"),
+ ("silence.py", clip, "-o", OUT / "silent_cut.mp4")]
+ for name, *args in runs:
+ extra = ["--overwrite"] if "-o" in args else []
+ proc = subprocess.run([sys.executable, str(SCRIPTS / name), *map(str, args), "--json", *extra],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ doc = strict(proc.stdout) # success or refusal, either document must be strict JSON
+ self.assertIsInstance(doc, dict, name)
+ if name == "export.py":
+ self.assertEqual(doc["loudness"]["lufs"], "-inf")
+ self.assertTrue(doc["loudness"]["silent"])
+ self.assertFalse(doc["loudness"]["ok"])
+ self.assertTrue(any(n.startswith("output audio is silent") for n in doc["notes"]))
+ self.assertFalse(any("run loudness.py" in n for n in doc["notes"]))
+
if __name__ == "__main__":
unittest.main(verbosity=2)
diff --git a/tests/test_editing.py b/tests/test_editing.py
index 2967c6c..d885ba0 100644
--- a/tests/test_editing.py
+++ b/tests/test_editing.py
@@ -1063,6 +1063,96 @@ class EditingTests(MediaFixtures):
"--json", "-o", d / "clean.wav").stdout)
self.assertEqual(doc["skipped"], [])
+ def test_join_warns_on_short_segments_and_duplicates(self):
+ """A segment shorter than 2 frames (audio-only: 0.05 s) and a path listed twice used to
+ join as verified with nothing said. Both are warnings, not refusals (repeating a clip can
+ be intended): `short_segments` [{index, path, duration}] and `duplicates` [{path,
+ indices}], always present, plus a note each. The join command is unchanged."""
+ d = OUT / "join_short_dup"
+ d.mkdir(exist_ok=True)
+ ff = ("ffmpeg", "-y", "-hide_banner", "-loglevel", "error")
+ sh(*ff, "-f", "lavfi", "-i", "testsrc=s=64x64:r=25:d=1", "-f", "lavfi", "-i", "sine=d=1",
+ "-shortest", "-c:v", "libx264", "-c:a", "aac", "-pix_fmt", "yuv420p", d / "a.mp4")
+ sh(*ff, "-f", "lavfi", "-i", "testsrc=s=64x64:r=25", "-f", "lavfi", "-i", "sine",
+ "-frames:v", "1", "-t", "0.04", "-c:v", "libx264", "-c:a", "aac", "-pix_fmt", "yuv420p", d / "b.mp4")
+ sh(*ff, "-f", "lavfi", "-i", "sine=d=1", d / "a.wav")
+ sh(*ff, "-f", "lavfi", "-i", "sine=d=0.02", d / "b.wav")
+ doc = json.loads(script("join.py", d / "a.mp4", d / "b.mp4", d / "a.mp4", "--transition", "none",
+ "--json", "-o", d / "v.mp4").stdout)
+ self.assertEqual((doc["status"], doc["clips"]), ("completed", 3))
+ self.assertEqual([(s["index"], Path(s["path"]).name) for s in doc["short_segments"]], [(1, "b.mp4")])
+ self.assertLess(doc["short_segments"][0]["duration"], 2 / 25)
+ self.assertEqual([(Path(x["path"]).name, x["indices"]) for x in doc["duplicates"]], [("a.mp4", [0, 2])])
+ self.assertTrue(any("more than once" in n for n in doc["notes"]))
+ self.assertTrue(any("shorter than 2 frames" in n for n in doc["notes"]))
+ doc = json.loads(script("join.py", d / "a.wav", d / "b.wav", "--transition", "none",
+ "--json", "-o", d / "o.wav").stdout)
+ self.assertEqual([Path(s["path"]).name for s in doc["short_segments"]], ["b.wav"])
+ self.assertEqual(doc["duplicates"], [])
+ # clean: both keys present and empty
+ doc = json.loads(script("join.py", d / "a.wav", d / "a.wav", "--transition", "none",
+ "--json", "-o", d / "c.wav").stdout)
+ self.assertEqual(doc["short_segments"], [])
+ self.assertEqual(len(doc["duplicates"]), 1)
+ doc = json.loads(script("join.py", d / "a.mp4", d / "a.mp4", "--transition", "none", "--dry-run",
+ "--json", "-o", d / "dry.mp4").stdout)
+ self.assertEqual(doc["short_segments"], [])
+ shutil.rmtree(d, ignore_errors=True)
+
+ def test_join_on_silent_names_a_silent_segment(self):
+ """Reported from the same VPS shorts pipeline: TTS sometimes writes a valid but fully
+ silent wav, which passes the missing/empty/unreadable preflight, and the video shipped
+ with no narration. Every input with audio is measured (peak dBFS): --on-silent warn
+ (default) joins it and names it under `silent`, fail refuses it in the one kind-input
+ document, skip leaves it out under `skipped`. A clip without audio is never silent."""
+ d = OUT / "join_silent"
+ d.mkdir(exist_ok=True)
+ for i in (1, 3):
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", f"sine=frequency={300 * i}:sample_rate=24000", "-t", "1.5", d / f"line{i}.wav")
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", "anullsrc=r=24000:cl=mono", "-t", "1.5", d / "line2.wav")
+ # a quiet but audible line: about -40 dBFS peak (sine is -18 dBFS)
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", "sine=frequency=500:sample_rate=24000", "-t", "1.5", "-af", "volume=-22dB", d / "quiet.wav")
+ (d / "parts.txt").write_text("line1.wav\nline2.wav\nline3.wav\n", encoding="utf-8")
+ # warn (default): joined, named, verified untouched
+ doc = json.loads(script("join.py", "--list", d / "parts.txt", "--transition", "none", "--json",
+ "-o", d / "warn.wav").stdout)
+ self.assertEqual((doc["status"], doc["clips"], doc["verified"]), ("completed", 3, True))
+ self.assertEqual([(s["index"], Path(s["path"]).name) for s in doc["silent"]], [(1, "line2.wav")])
+ self.assertLessEqual(doc["silent"][0]["peak_db"], -50)
+ self.assertTrue(any("line2.wav" in n for n in doc["notes"]))
+ self.assertTrue((d / "warn.wav").exists())
+ # fail: one kind-input refusal naming it, nothing written
+ proc = script("join.py", "--list", d / "parts.txt", "--transition", "none", "--on-silent", "fail",
+ "--json", "-o", d / "fail.wav", expect_fail=True)
+ doc = json.loads(proc.stdout)
+ self.assertEqual((doc["error"]["kind"], doc["commands"]), ("input", []))
+ self.assertEqual([Path(p["path"]).name for p in doc["problems"]], ["line2.wav"])
+ self.assertTrue(doc["problems"][0]["reason"].startswith("silent (peak -"))
+ self.assertFalse((d / "fail.wav").exists())
+ # skip: joined without it, listed under skipped
+ doc = json.loads(script("join.py", "--list", d / "parts.txt", "--transition", "none", "--on-silent", "skip",
+ "--json", "-o", d / "skip.wav").stdout)
+ self.assertEqual((doc["clips"], doc["silent"]), (2, []))
+ self.assertEqual([Path(p["path"]).name for p in doc["skipped"]], ["line2.wav"])
+ self.assertAlmostEqual(doc["probe"]["duration"], 3.0, delta=0.05)
+ # threshold: -40 dBFS is audible at the default, silent at -35
+ doc = json.loads(script("join.py", d / "line1.wav", d / "quiet.wav", "--transition", "none",
+ "--json", "-o", d / "q1.wav").stdout)
+ self.assertEqual(doc["silent"], [])
+ doc = json.loads(script("join.py", d / "line1.wav", d / "quiet.wav", "--transition", "none",
+ "--silence-threshold", "-35", "--json", "-o", d / "q2.wav").stdout)
+ self.assertEqual([Path(s["path"]).name for s in doc["silent"]], ["quiet.wav"])
+ # a clip with no audio stream is never silent
+ for n in (1, 2):
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i",
+ "testsrc2=size=160x120:rate=25", "-t", "1", "-an", "-pix_fmt", "yuv420p", d / f"mute{n}.mp4")
+ doc = json.loads(script("join.py", d / "mute1.mp4", d / "mute2.mp4", "--transition", "none", "--on-silent", "fail",
+ "--json", "-o", d / "mute.mp4").stdout)
+ self.assertEqual((doc["status"], doc["silent"]), ("completed", []))
+
def test_join_dry_run_plans_on_pending_segments(self):
"""Under --dry-run a segment that does not exist yet is an earlier step's output. Its
probe is the dry-run stub, whose (0x0) video stream used to decide the mode: one pending
diff --git a/tests/test_orchestration.py b/tests/test_orchestration.py
index 610609c..3aca927 100644
--- a/tests/test_orchestration.py
+++ b/tests/test_orchestration.py
@@ -848,6 +848,21 @@ class OrchestrationTests(MediaFixtures):
self.assertTrue(any("--aspect 16:9 --width 1920 --height 1080" in line for line in proc.stderr.splitlines()
if "fit.py" in line), proc.stderr[-2000:])
+ def test_render_check_content_forwards_the_content_rows(self):
+ """`"check": {"content": true}` forwards check.py --content; the default leaves the rows out."""
+ rows = {}
+ for flag in (True, None):
+ proj = OUT / "project_check_content.json"
+ chk = {"platform": "custom"}
+ if flag:
+ chk["content"] = True
+ proj.write_text(json.dumps({"output": "render_check_content.mp4",
+ "clips": [{"src": "source.mp4", "in": 0, "out": 2}], "check": chk}), encoding="utf-8")
+ data = json.loads(script("render.py", proj, "--fast", "--json").stdout)
+ rows[flag] = {r["check"] for r in data["check"]["checks"]}
+ self.assertTrue({"black", "frozen", "silence"} <= rows[True], rows)
+ self.assertFalse({"black", "frozen", "silence"} & rows[None], rows)
+
def test_render_exits_nonzero_when_the_check_stage_fails(self):
"""A render whose deliverable fails its own check stage must not report success."""
proj = OUT / "project_bad_check.json"
diff --git a/tests/test_picture.py b/tests/test_picture.py
index ef1a1ef..f89eed2 100644
--- a/tests/test_picture.py
+++ b/tests/test_picture.py
@@ -270,6 +270,67 @@ class PictureTests(MediaFixtures):
self.assertClose(m["duration"], 12.0, 0.15)
self.assertEqual(m["video"]["width"], 1280)
+ def test_caption_refuses_cues_that_are_never_visible(self):
+ """2.2.6: a burn that draws nothing is a refusal, not a verified output -- every cue after
+ the end of the video, every cue blank, or an ASS with no Dialogue lines."""
+ late = OUT / "cap_late.srt"
+ late.write_text("1\n00:00:20,000 --> 00:00:22,000\nToo late\n", encoding="utf-8")
+ blank = OUT / "cap_blank.srt"
+ blank.write_text("1\n00:00:01,000 --> 00:00:02,000\n \n\n2\n00:00:03,000 --> 00:00:04,000\n \n",
+ encoding="utf-8")
+ empty_ass = OUT / "cap_empty.ass"
+ empty_ass.write_text("[Script Info]\nScriptType: v4.00+\n\n[Events]\nFormat: Layer, Start, End, "
+ "Style, Name, MarginL, MarginR, MarginV, Effect, Text\n", encoding="utf-8")
+ late_ass = OUT / "cap_late.ass"
+ late_ass.write_text(empty_ass.read_text(encoding="utf-8")
+ + "Dialogue: 0,0:00:30.00,0:00:31.00,Default,,0,0,0,,Late\n", encoding="utf-8")
+ for flag, path, needle in (("--srt", late, "no cue falls inside the video"),
+ ("--srt", blank, "every cue is blank"),
+ ("--ass", empty_ass, "no Dialogue lines"),
+ ("--ass", late_ass, "no cue falls inside the video")):
+ for dry in ((), ("--dry-run",)):
+ out = OUT / "cap_invisible.mp4"
+ proc = script("caption.py", self.src, flag, path, *dry, "--json", "-o", out, expect_fail=True)
+ doc = json.loads(proc.stdout)
+ self.assertEqual(doc["error"]["kind"], "input", (path, dry))
+ self.assertIn(needle, doc["error"]["message"], (path, dry))
+ self.assertIn("first cue starts at 20.0 s", json.loads(script(
+ "caption.py", self.src, "--srt", late, "--json", "-o", OUT / "cap_invisible.mp4",
+ expect_fail=True).stdout)["error"]["message"])
+
+ def test_caption_reports_cues_burned_and_outside(self):
+ srt = OUT / "cap_partly.srt"
+ srt.write_text("1\n00:00:01,000 --> 00:00:03,000\nSeen\n\n2\n00:00:30,000 --> 00:00:32,000\nUnseen\n",
+ encoding="utf-8")
+ out = OUT / "cap_partly.mp4"
+ doc = json.loads(script("caption.py", self.src, "--srt", srt, "--preset", "ultrafast",
+ "--json", "-o", out).stdout)
+ self.assertEqual(doc["caption"]["cues_burned"], 1)
+ self.assertEqual(doc["caption"]["cues_outside"], 1)
+ self.assertTrue(any("outside the video" in n for n in doc.get("notes", [])))
+
+ def test_caption_real_run_does_not_say_rerun_without_dry_run(self):
+ srt = OUT / "cap_adj.srt"
+ srt.write_text("1\n00:00:01,000 --> 00:00:03,000\nHello\n", encoding="utf-8")
+ out = OUT / "cap_adj.mp4"
+ doc = json.loads(script("caption.py", self.src, "--srt", srt, "--offset", "0.5", "--preset",
+ "ultrafast", "--json", "-o", out).stdout)
+ notes = " ".join(doc.get("notes", []))
+ self.assertIn("_adjusted.srt", notes)
+ self.assertNotIn("without --dry-run", notes)
+ dry = json.loads(script("caption.py", self.src, "--srt", srt, "--offset", "0.5", "--dry-run",
+ "--json", "-o", out).stdout)
+ self.assertIn("without --dry-run", " ".join(dry.get("notes", [])))
+
+ def test_graphics_and_overlay_refuse_blank_text(self):
+ out = OUT / "blank_text.mp4"
+ for tool, args in (("graphics.py", ("--template", "title", "--title", " ")),
+ ("graphics.py", ("--template", "lower-third", "--name", " ", "--title", "x")),
+ ("graphics.py", ("--template", "sticker", "--text", "\t")),
+ ("overlay.py", ("--text", " "))):
+ proc = script(tool, self.src, *args, "--json", "-o", out, expect_fail=True)
+ self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input", (tool, args))
+
def test_caption_wraps_a_long_latin_cue_to_the_safe_area(self):
"""A 60-character cue at size 48 on a 640-wide frame does not fit one line; it is wrapped
to the safe area and, past --max-lines, split into consecutive cues rather than running
@@ -3380,5 +3441,89 @@ class FitSizeTests(unittest.TestCase):
self.assertEqual(self._fit([self.CW1])["size"], 16)
+class AsrNoSpeechTests(unittest.TestCase):
+ """An engine that ran and heard nothing is `<engine> found no speech in <input>` (kind input,
+ reason no_speech). Before 2.3.0 faster-whisper fell through to "no local speech-to-text engine
+ found" and whisper.cpp / openai-whisper said "no cues found in /tmp/ffskill_asr_*/audio.srt".
+ The engines are stubbed: none is installed where this suite runs."""
+
+ def _run(self, which, write_srt=None, faster=None):
+ import tempfile
+ import types
+ import _common
+ from _common import asr
+
+ class FakeShutil:
+ @staticmethod
+ def which(name):
+ return which.get(name)
+
+ class FakeSubprocess:
+ PIPE = subprocess.PIPE
+ TimeoutExpired = subprocess.TimeoutExpired
+
+ @staticmethod
+ def run(cmd, **kw):
+ write_srt(cmd)
+ return subprocess.CompletedProcess(cmd, 0, "", "")
+
+ calls = []
+
+ def fake_die(msg, code=1, kind="input", **extra):
+ calls.append((msg, kind, extra))
+ raise SystemExit(code)
+
+ mods = dict(sys.modules)
+ if faster is not None:
+ mods["faster_whisper"] = faster
+ else:
+ mods["faster_whisper"] = None # ImportError on import
+ tmp = tempfile.mkdtemp(prefix="asr_test_")
+ try:
+ with unittest.mock.patch.object(_common, "run_analysis", lambda *a, **k: None), \
+ unittest.mock.patch.object(asr, "die", fake_die), \
+ unittest.mock.patch.dict(sys.modules, mods), \
+ self.assertRaises(SystemExit):
+ asr._transcribe_in(tmp, "talk.wav", os.path.join(tmp, "out.srt"), None, "base", 0,
+ "ffmpeg", FakeShutil, FakeSubprocess)
+ finally:
+ shutil.rmtree(tmp, ignore_errors=True)
+ self.assertEqual(len(calls), 1, calls)
+ return calls[0]
+
+ def _assert_no_speech(self, call, engine):
+ msg, kind, extra = call
+ self.assertEqual(msg, f"{engine} found no speech in talk.wav")
+ self.assertEqual(kind, "input")
+ self.assertEqual(extra.get("reason"), "no_speech")
+ self.assertEqual(extra.get("engine"), engine)
+ self.assertNotIn("/tmp", msg)
+
+ def test_whisper_cpp_empty_srt_is_no_speech(self):
+ def write(cmd):
+ Path(cmd[cmd.index("-of") + 1] + ".srt").write_text("", encoding="utf-8")
+ self._assert_no_speech(self._run({"whisper-cli": "/opt/whisper/whisper-cli"}, write), "whisper.cpp")
+
+ def test_openai_whisper_empty_srt_is_no_speech(self):
+ def write(cmd):
+ Path(cmd[cmd.index("--output_dir") + 1], "audio.srt").write_text("\n", encoding="utf-8")
+ self._assert_no_speech(self._run({"whisper": "/usr/bin/whisper"}, write), "openai-whisper")
+
+ def test_faster_whisper_no_segments_is_no_speech_not_no_engine(self):
+ import types
+ mod = types.ModuleType("faster_whisper")
+
+ class WhisperModel:
+ def __init__(self, *a, **k):
+ pass
+
+ def transcribe(self, *a, **k):
+ return iter([]), None
+ mod.WhisperModel = WhisperModel
+ call = self._run({}, faster=mod)
+ self._assert_no_speech(call, "faster-whisper")
+ self.assertNotIn("no local speech-to-text engine", call[0])
+
+
if __name__ == "__main__":
unittest.main(verbosity=2)