Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
feat: 1.18.0 — measured analysis and multicam at scale (#246)
Shogo Kajiyama6058a4d84d98
diff --git a/docs/contract.md b/docs/contract.mdindex 3c1dc5f..87ad179 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -458,6 +458,18 @@ Per-tool keys added in 1.17.3, all additive:| `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |+Per-tool keys added in 1.18.0, all additive:++| key | tool | what it holds |+|---|---|---|+| `shots` | `scenes.py --shots` | `[{start, end, label, flow_magnitude}]` per detected scene, `label` one of `static`/`pan`/`motion` from a lightweight block-matching optical-flow proxy (frames decoded at 4 fps, 48x27, no external dependency). A shot too short to sample two frames is `static` with `flow_magnitude: 0` |+| `audio_peaks_db` | `scenes.py --audio-peaks` | `[{time, level}]`, measured dBFS loudness peaks. A **new** key: the pre-existing `audio_peaks` (always reported, unrelated unitless RMS figures used for `--highlights` scoring) keeps its 1.0 meaning unchanged |+| `speech` | `scenes.py --speech` | `[{time, speech_music_ratio}]`, a per-second zero-crossing-rate ratio against the file's own median — a measured proxy, not a speech/music classification |+| `motion_centre` | `cropdetect.py --motion-centre` | `[{time, x, y, x_frac, y_frac, motion}]` per second, sampled over the same windows as the crop-bar detection. `x`/`y` are source pixels, `x_frac`/`y_frac` a 0..1 fraction of `source_width`/`source_height`; a window with no measured motion reports `x`/`y`/`x_frac`/`y_frac: null`. Report only — this tool never picks a reframe |+| `speech_aware`, `speech_aware.breaths` | `silence.py --speech-aware` | `{min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}`. `breaths` are the sub-`--min-silence` gaps kept because they sit inside a sentence; the removal list (`silences`, `keep`, `removed_seconds`) already reflects the speech-aware classification. Composes with `--filler` through the same `keep_ranges()`/`merge_spans()` pipeline, so `--speech-aware --filler` produces one removal list |+| `sources` | `sync.py` | `[{path, offset_s, confidence, drift_ppm}]`, one entry per SOURCE. Present for every run, including the original single-SOURCE shape (where it mirrors the top-level `second`/`offset_seconds`/`confidence` additively). With 2+ SOURCEs it is the *only* per-source shape: there is no top-level `second`/`offset_seconds` because there is no single pair to put there |+| `switch_mode`, `min_shot` | `multicam.py --switch energy` | `"energy"` and the `--min-shot` value used (default 1.5s), alongside the existing `cuts` (`[[start, end, camera], ...]`) which already carries the camera index for `--edl`'s companion cut list |+`check.py` also gains an informational `subtitles` row on **every** platform:`PASS` when every soft subtitle stream carries a language tag, `WARN` when onedoes not (or when there are none). Like `channels` and `chapters` it is neverdiff --git a/docs/design-decisions.md b/docs/design-decisions.mdindex 549b23a..8332fc2 100644--- a/docs/design-decisions.md+++ b/docs/design-decisions.md@@ -445,3 +445,94 @@ folder" → `batch.py`, "a 60 s highlight" → `scenes.py`), `references/scriptsflags, and SKILL.md has single digits of headroom under its 30,000-byte budget. Only the captionsize got a row, because that one is a *different answer to a request the table already claims toroute*. If eval 18 shows agents missing `--filler` or `--snap beats`, 1.18.0 buys the rows.++**`--shots`' flow measurement is scoped to stay dependency-free and fast: 48x27 grayscale frames+at 4 fps, a 4x4 grid of block matches, +/-3 px search.** A real optical-flow library was never on+the table — the zero-dependency rule stdlib+ffmpeg covers every other script, and a Lucas-Kanade+or dense-flow implementation in pure Python would be both slower and no more honest for what+`--shots` actually needs, which is three labels, not a per-pixel field. 48x27 keeps a whole shot's+decode at a few KB (a feature-length input never risks memory the way `--beats`' 22050 Hz PCM+does), and 4 fps is enough to see whether the frame is panning, static or churning without+sampling every frame ffmpeg decodes. The zero-shift tie-break in `_block_match` (see the code+comment) exists because a textureless block — sky, an out-of-focus background, `--shots`' own+`test_scenes_shots_static_clip_is_labelled_static` fixture — ties every candidate offset on SAD,+and without an explicit bias towards "no motion" the scan reported the search window's first+corner as the measured displacement: a still frame read as steady motion in one direction, every+time. `agreement` (how consistently the 16 blocks agree on direction) is what tells a pan from+motion-inside-a-static-frame: a camera move shifts the whole picture one way, a subject moving in+front of a still background does not, and the two look identical in `magnitude` alone.++**`--audio-peaks` writes a new `audio_peaks_db` key rather than changing `audio_peaks`.** The+unconditional `audio_peaks` list scenes.py has always reported (`[{time, rms}]`) is a different+measurement in a different unit, used to score `--highlights`; the stability guarantee's "no key+given a different type" means an explicit request for dBFS-unit peaks needed its own name, not a+unit change to a key a caller has been reading since 1.0.++**`--speech` reports a ratio, not a label.** A zero-crossing-rate proxy (this window's ZCR over+the file's own median) says nothing about whether a stretch *is* speech or music — it says speech+transients cross zero faster than sustained tones, which is true often enough to be a usable+number and not true often enough to be a classifier. Naming the key `speech_music_ratio` rather+than `is_speech` keeps the tool on its side of the "analysis tools report numbers, they never+decide what's interesting" line: the calling agent reads the number and decides what a high or+low ratio means for its own request, the same way it already reads `--rank-by audio`'s RMS+figures without this skill calling any of them "the best scene".++**`--speech-aware` composes with `--filler` by feeding the same `keep_ranges()`/`merge_spans()`+pipeline `--filler` already uses, not a second removal pass.** The composition risk was two+independent cut lists disagreeing — a breath kept by `--speech-aware` getting cut anyway because+`--filler` found a word inside it, or the reverse. `speech_aware_silences()` returns two lists+(sentence-boundary silences to remove, breaths to keep) instead of a single filtered list, so a+breath is simply never added to the removal side: it is not "removed then added back", which+would leave a seam. The one designed interaction is a filler word *inside* a kept breath: the+breath itself is not cut, but `merge_spans()` still unions the filler span into the removal list+the same way it has always merged a filler word sitting inside a plain silencedetect gap (the+1.17.0 regression fix), so a real disfluency is still removed even from air `--speech-aware`+would otherwise keep whole. `BREATH_FLOOR` (0.12 s) exists because plain `detect()` only sees+gaps at or above `--min-silence`; without a second, shorter silencedetect pass the breaths inside+a sentence are invisible to begin with, not merely unclassified.++**sync.py's N-source shape adds a new optional `more_sources` positional rather than renaming+`second` to a list.** The stability guarantee treats an argparse dest as a CLI surface: renaming+`second` to `sources` (even as a 1+ `nargs`) would have been a rename of a positional the contract+promises never to rename, and the MCP `inputSchema` derives its property names directly from+argparse dests (see `_contract.input_schema`), so the rename would have propagated into a removed+MCP property too. `second` therefore keeps meaning exactly what it always has — the one other+recording a 2-source call aligns — and `more_sources` (`nargs="*"`, default `[]`) is purely+additive: omitted, the CLI and its JSON are unchanged from 1.17. `args.sources = [second] ++more_sources` is built once, right after parsing, so the rest of `main()` (and `measure_one()`,+factored out of the single-pair drift code that used to live inline) never has to know which+positional a given source came from.++**Why `sources` carries the single-source measurement too, additively, instead of only appearing+at N>1.** A caller that standardises on reading `sources` for a multi-camera job should not also+need a special case for the 2-source call — `sources[0]` is always the same measurement as the+top-level `offset_seconds`/`confidence` when there is exactly one. The reverse (a flat `second`+key materialising when there are three sources) is not offered, because there is no single "the"+second source to put there once there are two or three, and inventing one would be worse than+leaving the key out.++**`multicam.py --switch energy`'s window and merge rule.** Loudness is measured every 0.25 s (four+times finer than the default `--min-shot`, so a real cut point is not missed by more than 0.25 s)+and picking the winner is a plain argmax over cameras with a video stream — the same "biggest+number wins" rule `scenes.py --rank-by audio` already uses, not a smarter voice-activity model,+because a second measurement method would be a second thing to keep honest and this one is easy to+audit from the `cuts` list alone. Folding a run shorter than `--min-shot` into its neighbour+(rather than, say, discarding it or holding the previous camera) was chosen because it never+invents a cut that was not there and never drops the fact that a switch was measured at that+instant — it only refuses to *act* on a switch too brief to be a readable shot. The direction of+the fold (into the next run, or the previous one if it is the last) is arbitrary in the sense that+either choice is defensible; it is documented here rather than left to be rediscovered from the+code, and pinned by `test_multicam_switch_energy_respects_min_shot`.++**The multicam timeline needed no new render.py project stage.** A `--switch` cut list is already+exactly what `clips[]` expresses: one clip per cut, `src` the camera's own file, `in`/`out` on+that camera's own timeline (the reference cut, shifted by the camera's measured offset). Inventing+a `"multicam"` stage type would have meant a second way to say "play this camera from A to B" that+`render.py` would have to keep in sync with `clips[]` forever; reusing `clips[]` means a switch+list can be re-rendered with a different `--min-shot` by re-running `multicam.py --switch energy`+and rewriting the same project's `clips` array, with every other stage (captions, audio, export,+check) working on the multicam edit exactly as it would on any other project. `--edl` writes the+plain `cut.py --segments` file (`START-END` per line) rather than a `render.py` project directly,+because the camera index a project's clips need is already sitting in the JSON `cuts` field+(`[[start, end, camera], ...]`) — turning that into `clips[]` is a few lines in the calling agent,+not a new file format this tool would have to maintain.diff --git a/references/scripts.md b/references/scripts.mdindex 6f6552a..d7f1a60 100644--- a/references/scripts.md+++ b/references/scripts.md@@ -161,7 +161,7 @@ rounded, if they aren't.### cropdetect.py — measure black bars, report the crop rectangle```-cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N]+cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N] [--motion-centre]```Measurement only -- writes no file. Samples `--samples` windows spreadacross the file (default 5, totalling `--seconds` 10s of footage) and@@ -174,6 +174,14 @@ removing detected bars is wanted -- genuine letterboxed content (ascope-ratio film in a 16:9 frame) "detects" the same way as accidentalbars; look at the frame before cropping it away.+`--motion-centre` (1.18) reports the motion centroid once per second, sampled+across the same windows: `motion_centre: [{time, x, y, x_frac, y_frac, motion}]`,+`x`/`y` in source pixels and `x_frac`/`y_frac` as a 0..1 fraction of+`source_width`/`source_height`. A frame with no measured motion in a window+reports `x`/`y`/`x_frac`/`y_frac: null` and `motion: 0`. Report only, like the+crop rectangle above -- this hands the calling agent numbers to reframe a 9:16+crop with; it never picks the subject or the crop box itself.+### deinterlace.py — deinterlace interlaced footage```deinterlace.py INPUT [--mode frame|field] [--parity auto|tff|bff] [--only-interlaced] [-o OUT]@@ -465,7 +473,7 @@ each shorter clip's last frame (with silence) out to the longest.### silence.py — remove dead air / jump cuts```-silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]+silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it][--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02][--transcribe] [--filler-list] [--max-cuts 400]@@ -531,6 +539,17 @@ additive sibling covering everything that went. A filler word quiet enough tosit inside a detected silence is merged into it rather than counted twice, sothe two figures can be equal.+**`--speech-aware` (1.18)** keeps breaths shorter than `--min-silence` when+they sit inside a sentence, and only cuts at sentence-boundary pauses+(`--min-silence` or longer). It re-runs silence detection at a much shorter+floor to measure the short gaps at all, then classifies each one: shorter than+`--min-silence` is an in-sentence breath (kept, listed in+`speech_aware.breaths`); `--min-silence` or longer is a sentence boundary (cut,+same as the plain flag). It composes with `--filler` through the same+`keep_ranges()`/`merge_spans()` pipeline the filler-inside-a-silence fix+already uses, so `--speech-aware --filler` produces one removal list. `--edl`+writes the resulting cut list exactly as it does today.+### join.py — concatenate with transitions```join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]@@ -701,6 +720,7 @@ already has a picture.```scenes.py INPUT [--threshold 10] [--min-scene 1] [--highlights N [--target SECONDS] [--max-scene 15]] [--edl picks.txt] [--sheet scenes.png] [--json]scenes.py INPUT --beats [--beat-step 0.01] [--beat-range 60-200] [--min-confidence 0.5] [--json]+scenes.py INPUT [--shots] [--audio-peaks] [--speech] [--json]```Lists scenes with audio energy, the loudest moments, and (with`--highlights`) proposes N ranges that add up to `--target` seconds, biased to@@ -751,6 +771,29 @@ would change a file on the strength of it (`cut.py --snap beats`). `--edl` with`--beats` is unchanged: beats are never written as an EDL, because a beat isnot a cut. No audio stream is a `kind: input` refusal.+**`--shots`, `--audio-peaks`, `--speech` (1.18)** are three independent+measurements that combine with each other and with `--beats`/`--highlights`:++- `--shots` labels each already-detected scene `static`/`pan`/`motion` by a+ lightweight block-matching optical-flow proxy (frames decoded at 4 fps,+ 48x27, no external dependency), reporting `shots: [{start, end, label,+ flow_magnitude}]`. A shot too short to sample two frames is `static` with+ `flow_magnitude: 0` — there is nothing to measure motion between. A proxy,+ the same spirit as `--rank-by`: it reports what a coarse block match saw,+ not what is interesting about the shot.+- `--audio-peaks` reports loudness peaks as measured dBFS: `audio_peaks_db:+ [{time, level}]`. This is a new key, kept separate from the pre-existing+ (always-on) `audio_peaks` list, whose entries are an unrelated unitless RMS+ figure used for `--highlights` scoring — a different unit needed a+ different key so `audio_peaks`'s meaning does not change underneath a+ caller reading it since 1.0.+- `--speech` reports a per-second zero-crossing-rate ratio,+ `speech: [{time, speech_music_ratio}]` — speech's rapid consonant+ transients raise the zero-crossing rate; sustained tones (music, room+ tone) cross zero at a steadier rate. `1.0` means "typical for this file's+ own median," not an absolute threshold. This is a measured number, not a+ speech/music classification — nothing here decides which stretch is which.+### check.py — pre-delivery compliance```check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--json]@@ -905,7 +948,8 @@ hand the path to the user together with the numbers.### multicam.py — align several cameras and switch between them```-multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."] | [--auto N] [--audio IDX] [--fix-drift]+multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."|energy [--min-shot 1.5]] | [--auto N]+ [--audio IDX] [--fix-drift] [--edl cuts.txt][--offsets-only] [--width W --height H --fps N] [-o OUT]```All inputs are aligned to the first one by audio (same engine as `sync.py`,@@ -915,6 +959,23 @@ simply alternates every N seconds. Audio comes from the reference unless`--audio` picks another input, e.g. an external recorder that has no video.`--offsets-only` reports offsets and confidence without rendering.+**`--switch energy` (1.18)** auto-switches to whichever camera (of those with+a video stream) measures the loudest audio at each 0.25 s window on the+reference timeline, then folds any run shorter than `--min-shot` (default+1.5 s) into its neighbour so the cut never lingers on a shot too short to+read. A measured loudest-camera pick, the same spirit as `scenes.py+--rank-by audio`: a proxy for who is talking, not a judgement — a loud crowd+or a hot mic wins over a quiet subject exactly like the scene ranking does.+`--edl` (1.18, any switch mode) writes the resulting cut list as `cut.py+--segments`-format `START-END` lines; the camera index for each cut is+already in the JSON `cuts` field (`[[start, end, camera], ...]`), which is+what `--edl` leaves out on purpose — one file for an NLE's cut list, one+field for the camera it came from. The multicam timeline needs no dedicated+project format: each cut is a `render.py` clip (`{"src": ..., "in": ...,+"out": ...}` on that camera's own timeline, shifted by its measured offset),+so a switch list can be re-rendered with different `--min-shot` values by+editing `project.json`'s `clips`, not by inventing a second timeline schema.+### verify.py — real-footage verification kit```verify.py FILES_OR_FOLDERS [--quick] [--report verify.md] [--out DIR --keep] [--seconds 6] [--json]@@ -1185,18 +1246,18 @@ draws whatever glyph the text font has; `--emoji none` strips them).### sync.py — offset detection, alignment, drift correction```-sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]- [--replace-audio | --trim-second] [-o OUT]+sync.py REFERENCE SOURCE [SOURCE ...] [--json] [--max-offset 30] [--analyze-seconds 120]+ [--fix-drift [--drift-window 60]] [--replace-audio | --trim-second] [-o OUT]```Cross-correlates loudness envelopes: coarse FFT search (20 ms), then a direct1 ms refinement (pure Python, a 2-minute window takes ~1-3 s). Positive offset-= the second recording started later. `--replace-audio` writes the reference-video with the second file's audio aligned (video stream copied); the output-keeps the reference's full length -- a shorter or head-trimmed second file is+= the source recording started later. `--replace-audio` writes the reference+video with the source file's audio aligned (video stream copied); the output+keeps the reference's full length -- a shorter or head-trimmed source file ispadded with silence, never allowed to cut the picture.-`--trim-second` writes the second file shifted to the reference timeline.+`--trim-second` writes the source file shifted to the reference timeline.`--fix-drift` measures the offset again near the end of the overlap, reports-the clock difference in ppm, and resamples the second file so a 60-minute+the clock difference in ppm, and resamples the source file so a 60-minutetake stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).Use it whenever the recording is longer than ~10 minutes. Check `confidence`(0–1, normalised correlation with a runner-up penalty); below 0.3 the match is@@ -1206,6 +1267,20 @@ window 95 %, misses flagged below 0.3. Keep `--analyze-seconds` at least 4×`--max-offset` (default 120 s vs 30 s): lags with under 35 % overlap areignored, so an offset larger than ~60 % of the window cannot be found.+**One reference, 1+ SOURCE (1.18).** With exactly one SOURCE the CLI keeps its+original shape byte for byte -- `second`, `offset_seconds`, `confidence`,+`meaning`, `drift` at the top level, `--replace-audio`/`--trim-second`+available -- with the same measurement additively mirrored under a new+`sources: [{path, offset_s, confidence, drift_ppm}]` list. With 2+ SOURCEs the+result is one offsets JSON, `{reference, sources: [...]}`, and no flat+`second`/`offset_seconds` (there is no single pair to put there);+`--replace-audio`/`--trim-second` refuse with `kind: input`, since each writes+one synced output and there is more than one source to choose from. This is+the same offset/drift measurement `multicam.py` already loops over per camera+internally (it imports `sync.measure_offset`); `sync.py`'s own N-source CLI+now exposes that loop directly, for aligning cameras without cutting between+them.+### color.py — HDR to SDR, LUTs, colour tags, Dolby Vision```color.py INPUT --to-sdr [--tonemap hable|mobius|reinhard|bt2390] [--peak 1000] [--desat 0] [-o OUT]diff --git a/scripts/_common/__init__.py b/scripts/_common/__init__.pyindex 853ea65..ae77739 100644--- a/scripts/_common/__init__.py+++ b/scripts/_common/__init__.py@@ -68,12 +68,13 @@ from _common.emit import ()from _common.probe import (analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,- _fraction, keyframes_near, SCORE_RE, SIL_RE,+ _fraction, keyframes_near, SCORE_RE, SIL_RE, decode_gray_frames,measured_level_dbfs, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output)from _common.decision import (aac_args, add_pad_fill_args, audio_codec_for, AUDIO_CODECS, brand_caption_style, BRAND_DEFAULTS,description_block, _evidence_rank, fmt_chapter_time, propose_chapters,+ frame_flow, label_shot_flow, MOTION_GRID, MOTION_SEARCH, MOTION_STATIC_PX, MOTION_PAN_SPREAD,filler_spans, FILLER_WORDS, FILLER_AMBIGUOUS, FILLER_DISCOURSE_MARKERS, FILLER_MAX_WORD,FILLER_MIN_GAP, FILLER_PAD, normalise_filler_token,beat_grid, snap_points, BEAT_MIN_CONFIDENCE, BEAT_ONSET_K, BEAT_OCTAVE_MARGIN,@@ -205,6 +206,7 @@ __all__ = ["place_output", "_plan_at_exit", "_plan_inputs", "_PLAN_STRIP", "PLAN_VERSION", "PREFERRED_FAMILIES","print_json", "probe", "PROBE_TIMEOUT", "_progress_line", "read_text_or_die", "refuse_output_is_input","_remember_output", "require_tool", "resolve_emoji_assets", "_result_v2", "rms_envelope", "run", "run_analysis",+ "decode_gray_frames", "frame_flow", "label_shot_flow", "MOTION_GRID", "MOTION_SEARCH", "MOTION_STATIC_PX", "MOTION_PAN_SPREAD","_run_captured", "run_keeping_subtitles", "run_tool", "_run_with_progress", "_SCRIPT_FONT_CACHE","_script_font_entry", "script_font_for_text", "script_font_status", "_script_font_uncached", "_SCRIPT_RANGES","SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote",diff --git a/scripts/_common/decision.py b/scripts/_common/decision.pyindex 8afdbb4..e990ec2 100644--- a/scripts/_common/decision.py+++ b/scripts/_common/decision.py@@ -864,3 +864,89 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = Fm["start"] = round(m["start"], 4)m["end"] = round(m["end"], 4)return merged+++# --- 1.18.0: lightweight block-matching motion estimate (scenes.py --shots, cropdetect.py+# --motion-centre) ---------------------------------------------------------------------------++MOTION_GRID = 4 # NxN anchor blocks per frame+MOTION_SEARCH = 3 # +/- pixels searched per block, at the decoded (low) resolution+MOTION_STATIC_PX = 0.35 # average per-frame displacement below this, at decode resolution, is "static"+MOTION_PAN_SPREAD = 0.6 # block-to-block direction agreement above this (0..1) reads as a pan+++def _block_match(prev: bytes, cur: bytes, w: int, h: int, cx: int, cy: int, half: int, search: int) -> "Tuple[float, float]":+ """(dx, dy) that best aligns a `half*2` square centred at (cx, cy) in `prev` to `cur`,+ searched over +/- `search` px by sum-of-absolute-differences. Coordinates and the returned+ offset are in decoded-frame pixels (a handful of pixels a side at 1.18.0's sample size)."""+ x0, y0 = max(half, min(w - half - 1, cx)), max(half, min(h - half - 1, cy))+ ref = [prev[(y0 + dy) * w + (x0 + dx)] for dy in range(-half, half + 1) for dx in range(-half, half + 1)]++ def sad_at(xx: int, yy: int) -> "Optional[int]":+ if xx - half < 0 or xx + half >= w or yy - half < 0 or yy + half >= h:+ return None+ sad = 0+ for dy in range(-half, half + 1):+ row = (yy + dy) * w+ for dx in range(-half, half + 1):+ sad += abs(ref[(dy + half) * (2 * half + 1) + (dx + half)] - cur[row + xx + dx])+ return sad++ # Zero shift is the tie-break candidate, not the search order's first cell: on a textureless+ # block (a flat colour, sky, an out-of-focus background) every offset scores the same SAD, and+ # without an explicit tie towards "no motion" the scan used to report the search window's+ # first corner as the measured displacement -- a still frame with nothing to match against+ # read as steady motion in one direction, every time.+ best_sad, best = sad_at(x0, y0), (0.0, 0.0)+ if best_sad is None:+ best_sad = float("inf")+ for sy in range(-search, search + 1):+ for sx in range(-search, search + 1):+ if sx == 0 and sy == 0:+ continue+ sad = sad_at(x0 + sx, y0 + sy)+ if sad is not None and sad < best_sad:+ best_sad, best = sad, (float(sx), float(sy))+ return best+++def frame_flow(prev: bytes, cur: bytes, w: int, h: int, *, grid: int = MOTION_GRID,+ search: int = MOTION_SEARCH) -> "Dict[str, Any]":+ """One measurement between two consecutive decoded grayscale frames: the mean block+ displacement vector, its magnitude, and how consistently the blocks agree on direction+ (0 = every block moved a different way, 1 = every block agrees -- a pan or dolly moves the+ whole frame one way, on-screen motion inside a mostly-static frame does not)."""+ half = max(1, min(w, h) // (grid * 3))+ vecs: "List[Tuple[float, float]]" = []+ for gy in range(grid):+ for gx in range(grid):+ cx = int((gx + 0.5) * w / grid)+ cy = int((gy + 0.5) * h / grid)+ vecs.append(_block_match(prev, cur, w, h, cx, cy, half, search))+ mdx = sum(v[0] for v in vecs) / len(vecs)+ mdy = sum(v[1] for v in vecs) / len(vecs)+ magnitude = math.hypot(mdx, mdy)+ mean_len = sum(math.hypot(*v) for v in vecs) / len(vecs)+ agreement = (magnitude / mean_len) if mean_len > 1e-6 else 1.0 # 1.0 = every block agrees+ return {"dx": mdx, "dy": mdy, "magnitude": magnitude, "agreement": min(1.0, agreement)}+++def label_shot_flow(flows: "Sequence[Dict[str, Any]]") -> "Dict[str, Any]":+ """{label, flow_magnitude} for one shot from its per-frame-pair flow measurements.++ static: mean displacement below MOTION_STATIC_PX. pan: above it, and blocks agree on+ direction (a camera move shifts the whole frame). motion: above it, blocks disagree (motion+ inside an otherwise still frame -- handheld jitter, or a subject moving across a static+ background). This is a measured proxy, the same spirit as scenes.py --rank-by: it reports+ what a coarse block match saw, not what is interesting about the shot."""+ if not flows:+ return {"label": "static", "flow_magnitude": 0.0}+ magnitude = sum(f["magnitude"] for f in flows) / len(flows)+ agreement = sum(f["agreement"] for f in flows) / len(flows)+ if magnitude < MOTION_STATIC_PX:+ label = "static"+ elif agreement >= MOTION_PAN_SPREAD:+ label = "pan"+ else:+ label = "motion"+ return {"label": label, "flow_magnitude": round(magnitude, 3)}diff --git a/scripts/_common/probe.py b/scripts/_common/probe.pyindex aa1559f..3c58a3d 100644--- a/scripts/_common/probe.py+++ b/scripts/_common/probe.py@@ -77,6 +77,32 @@ MEDIA_EXT = {".mp4", ".mov", ".mkv", ".webm", ".m4v", ".avi", ".ts", ".mts", ".m".wav", ".flac", ".mp3", ".m4a", ".aac", ".ogg", ".opus", ".aif", ".aiff", ".caf", ".wma", ".png", ".jpg", ".jpeg", ".webp"}+def decode_gray_frames(path: str, fps: float, width: int, height: int, *, start: float = 0.0,+ seconds: "Optional[float]" = None, check: bool = True) -> "List[bytes]":+ """Decode `path` to raw 8-bit grayscale frames at a low `fps`/`width`x`height`, one ffmpeg+ pass to stdout. Used by scenes.py --shots (per-shot flow label) and cropdetect.py+ --motion-centre (motion centroid): both need pixel data, not a filter's own summary number,+ but at 1.18.0's resolutions (tens of pixels a side, a few fps) a whole shot is a few KB, so+ piping raw frames through Python stays cheap and needs no extra dependency."""+ ffmpeg = require_tool("ffmpeg")+ cmd = [ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin"]+ if start:+ cmd += ["-ss", f"{start:.3f}"]+ cmd += ["-i", path]+ if seconds is not None:+ cmd += ["-t", f"{max(0.0, seconds):.3f}"]+ cmd += ["-vf", f"fps={fps:g},scale={width}:{height}:flags=area,format=gray",+ "-f", "rawvideo", "-"]+ proc = run_analysis(cmd, check=False, text=False)+ if proc.returncode != 0 or not proc.stdout:+ if check:+ die(f"could not decode frames from {path}:\n{proc.stderr.decode(errors='replace').strip()}", kind="ffmpeg")+ return []+ frame_size = width * height+ data = proc.stdout+ return [data[i:i + frame_size] for i in range(0, len(data) - frame_size + 1, frame_size)]++def _output_failed(path: str, why: str) -> "None":"""An ffmpeg run reported success but the artifact is not usable: say so, and do not leave a0-byte file behind that a later step could mistake for a result."""diff --git a/scripts/_contract.py b/scripts/_contract.pyindex 42dea62..2a6bb52 100644--- a/scripts/_contract.py+++ b/scripts/_contract.py@@ -411,11 +411,16 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:extra = {"audiogram": {"type": "object", "description": "{style, background ('image' or 'color'), image, position, vis_height, platform, captions, title, stages, verified} -- present on every run, so a plain waveform answers background 'color' (1.16)"},"notes": {"type": "array", "items": {"type": "string"}}}elif name == "scenes":- extra = {"file": {"type": "string"}, "duration": {"type": "number"}, "scene_count": {"type": "integer"}, "scenes": {"type": "array"}, "audio_peaks": {"type": "array"}}+ extra = {"file": {"type": "string"}, "duration": {"type": "number"}, "scene_count": {"type": "integer"}, "scenes": {"type": "array"}, "audio_peaks": {"type": "array"},+ "shots": {"type": "array", "description": "--shots (1.18): [{start, end, label, flow_magnitude}]"},+ "audio_peaks_db": {"type": "array", "description": "--audio-peaks (1.18): [{time, level}] measured dBFS, distinct from audio_peaks above"},+ "speech": {"type": "array", "description": "--speech (1.18): [{time, speech_music_ratio}]"}}elif name == "silence":- extra = {"silences": {"type": "array"}, "keep": {"type": "array"}, "input_duration": {"type": "number"}, "kept_duration": {"type": "number"}, "removed_seconds": {"type": "number"}}+ extra = {"silences": {"type": "array"}, "keep": {"type": "array"}, "input_duration": {"type": "number"}, "kept_duration": {"type": "number"}, "removed_seconds": {"type": "number"},+ "speech_aware": {"type": "object", "description": "--speech-aware (1.18): {min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}"}}elif name == "sync":- extra = {"reference": {"type": "string"}, "second": {"type": "string"}, "offset_seconds": {"type": "number"}, "confidence": {"type": "number"}, "meaning": {"type": "string"}, "drift": {"type": "object"}}+ extra = {"reference": {"type": "string"}, "second": {"type": "string"}, "offset_seconds": {"type": "number"}, "confidence": {"type": "number"}, "meaning": {"type": "string"}, "drift": {"type": "object"},+ "sources": {"type": "array", "description": "1.18: [{path, offset_s, confidence, drift_ppm}], one per SOURCE; the only per-source shape once more than one SOURCE is given"}}elif name == "look":extra = {"outputs": {"type": "array", "items": {"type": "string"}}}elif name == "render":diff --git a/scripts/cropdetect.py b/scripts/cropdetect.pyindex 0cf8e2a..bd5ea26 100644--- a/scripts/cropdetect.py+++ b/scripts/cropdetect.py@@ -33,7 +33,10 @@ import sysfrom collections import Counterfrom typing import Dict, List, Tuple-from _common import add_common, apply_common, die, emit, info, print_json, probe, require_tool, run_analysis+from _common import add_common, apply_common, die, emit, info, print_json, probe, require_tool, run_analysis, decode_gray_frames, frame_flow++MOTION_CENTRE_FPS = 2.0+MOTION_CENTRE_W, MOTION_CENTRE_H = 64, 36CROP_RE = re.compile(r"crop=(\d+):(\d+):(\d+):(\d+)")@@ -61,6 +64,52 @@ def detect(path: str, seconds: float, samples: int, limit: float, round_to: int,return rects+def motion_centre(path: str, seconds: float, samples: int, duration: float, sw: int, sh: int) -> List[Dict]:+ """Per-second motion centroid: {time, x, y, x_frac, y_frac} -- a report-only measurement,+ the same sampled-window approach as detect() above. x/y are pixel coordinates in the SOURCE+ frame (consistent with the --round crop rectangle this tool already reports in source+ pixels); x_frac/y_frac are the same position as a 0..1 fraction of source_width/height, for a+ caller that wants to reframe without first knowing the source size. This never picks a+ subject -- it reports where in the frame the measured pixel motion was concentrated, which is+ not the same thing as where the interesting subject is (a moving background behind a still+ speaker centres the motion on the background)."""+ per_window = max(0.5, seconds / max(1, samples))+ cell = 8 # NxN diff grid at decode resolution+ cw, ch = MOTION_CENTRE_W / cell, MOTION_CENTRE_H / cell+ out: List[Dict] = []+ for i in range(samples):+ start = 0.0 if duration <= 0 else (duration - per_window) * i / max(1, samples - 1) if samples > 1 else 0.0+ start = max(0.0, start)+ frames = decode_gray_frames(path, MOTION_CENTRE_FPS, MOTION_CENTRE_W, MOTION_CENTRE_H,+ start=start, seconds=per_window, check=False)+ for k in range(len(frames) - 1):+ prev, cur = frames[k], frames[k + 1]+ wsum = wx = wy = 0.0+ for gy in range(cell):+ for gx in range(cell):+ x0, x1 = int(gx * cw), int((gx + 1) * cw)+ y0, y1 = int(gy * ch), int((gy + 1) * ch)+ diff = 0+ for y in range(y0, y1):+ row = y * MOTION_CENTRE_W+ for x in range(x0, x1):+ diff += abs(prev[row + x] - cur[row + x])+ cx = (gx + 0.5) / cell+ cy = (gy + 0.5) / cell+ wsum += diff+ wx += diff * cx+ wy += diff * cy+ t = start + (k + 1) / MOTION_CENTRE_FPS+ if wsum <= 0:+ out.append({"time": round(t, 2), "x": None, "y": None, "x_frac": None, "y_frac": None, "motion": 0.0})+ continue+ xf, yf = wx / wsum, wy / wsum+ out.append({"time": round(t, 2), "x": round(xf * sw), "y": round(yf * sh),+ "x_frac": round(xf, 3), "y_frac": round(yf, 3), "motion": round(wsum / (MOTION_CENTRE_W * MOTION_CENTRE_H), 2)})+ out.sort(key=lambda r: r["time"])+ return out++def main() -> int:ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)ap.add_argument("input")@@ -68,6 +117,10 @@ def main() -> int:ap.add_argument("--samples", type=int, default=5, help="number of windows spread across the file (default 5)")ap.add_argument("--limit", type=float, default=0.0941176, help="black-pixel threshold, 0..1 (default ~0.094, cropdetect's own default)")ap.add_argument("--round", type=int, default=16, dest="round_to", help="the reported width/height are rounded to a multiple of this (default 16)")+ ap.add_argument("--motion-centre", action="store_true",+ help="report the motion centroid per second, sampled the same way as the crop "+ "detection above (report only -- this tool never picks a reframe, it hands "+ "the calling agent numbers to reframe with)")add_common(ap)args = ap.parse_args()apply_common(args)@@ -102,6 +155,11 @@ def main() -> int:info(f"detected crop={w}:{h}:{x}:{y} (source {sw}x{sh}, confidence {result['confidence']:.0%}) -- "f"crop.py {args.input} --x {x} --y {y} --width {w} --height {h}")+ if args.motion_centre:+ centre = motion_centre(args.input, args.seconds, args.samples, duration, sw, sh)+ result["motion_centre"] = centre+ info(f"--motion-centre: {len(centre)} measured points")+if args.json:emit(None, **result)else:diff --git a/scripts/multicam.py b/scripts/multicam.pyindex 2104abe..317a813 100755--- a/scripts/multicam.py+++ b/scripts/multicam.py@@ -26,12 +26,16 @@ Examples:python3 multicam.py camA.mp4 camB.mp4 --auto 8 -o edit.mp4 # alternate cameras every 8 s"""import argparse+import mathimport sys-from typing import List, Tuple+from typing import Dict, List, Tuple-from _common import video_args, aac_args, add_common, apply_common, default_output, die, emit, ffmpeg_base, info, time_arg, probe, run, x264_args, X264_PRESETS, fmt_secs+from _common import video_args, aac_args, add_common, apply_common, default_output, die, emit, ffmpeg_base, info, time_arg, probe, run, x264_args, X264_PRESETS, fmt_secs, decode_pcm_mono, rms_envelope, STATEfrom sync import measure_offset+ENERGY_WINDOW_S = 0.25 # --switch energy: loudness measured over this window on the reference timeline+ENERGY_RATE = 8000 # decode rate for the per-camera loudness envelope+def parse_switch(spec: str, n: int) -> List[Tuple[float, float, int]]:out = []@@ -55,12 +59,75 @@ def parse_switch(spec: str, n: int) -> List[Tuple[float, float, int]]:return out+def energy_switch(inputs: List[str], metas: List["Dict"], offsets: List[float], ratios: List[float],+ ref_dur: float, min_shot: float, window_s: float = ENERGY_WINDOW_S) -> List[Tuple[float, float, int]]:+ """Auto-switch cuts: at every `window_s` step on the reference timeline, cut to whichever+ camera (of those WITH video) measures the loudest audio at that moment, then merge the+ winners into runs and fold any run shorter than `min_shot` into its neighbour. This is a+ measured loudest-camera pick, the same spirit as scenes.py --rank-by audio: it is a proxy for+ "who is talking", not a judgement -- a loud crowd or a camera with a hot mic wins over a+ quiet subject exactly like scenes.py's own audio ranking does."""+ candidates = [c for c, m in enumerate(metas) if m.get("video")]+ if not candidates:+ die("no input has video; --switch energy needs at least one camera with a picture")+ envs: "Dict[int, List[float]]" = {}+ step = max(1, int(ENERGY_RATE * window_s))+ for c in candidates:+ samples = decode_pcm_mono(inputs[c], ENERGY_RATE, check=False)+ envs[c] = rms_envelope(samples, step) if samples else []+ n_windows = max(1, int(math.ceil(ref_dur / window_s)))+ winners: List[int] = []+ for i in range(n_windows):+ t = i * window_s+ best_c, best_v = candidates[0], -1.0+ for c in candidates:+ src_t = (t - offsets[c]) * ratios[c]+ idx = int(src_t / window_s)+ env = envs[c]+ val = env[idx] if 0 <= idx < len(env) else -1.0+ if val > best_v:+ best_v, best_c = val, c+ winners.append(best_c)+ # collapse into runs+ runs: List[List] = []+ for i, c in enumerate(winners):+ t0, t1 = i * window_s, min(ref_dur, (i + 1) * window_s)+ if runs and runs[-1][2] == c:+ runs[-1][1] = t1+ else:+ runs.append([t0, t1, c])+ # fold a run shorter than min_shot into its neighbour: the next run if there is one, else the+ # previous one -- a single pass is enough because folding only ever lengthens a run, and a+ # run just extended is re-checked on the next iteration through the while loop.+ changed = True+ while changed and len(runs) > 1:+ changed = False+ for i, r in enumerate(runs):+ if r[1] - r[0] < min_shot:+ if i + 1 < len(runs):+ runs[i + 1][0] = r[0]+ else:+ runs[i - 1][1] = r[1]+ del runs[i]+ changed = True+ break+ return [(s, e, c) for s, e, c in runs]++def main() -> int:ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)ap.add_argument("inputs", nargs="+", help="reference camera first, then other cameras / recorders")ap.add_argument("-o", "--output", help="output file (default: <reference>_multicam.mp4)")- ap.add_argument("--switch", help="switch list START-END:CAM,... on the reference timeline")+ ap.add_argument("--switch", help="switch list START-END:CAM,... on the reference timeline, or "+ "the literal 'energy' to auto-switch to whichever camera is "+ "loudest at each moment (respecting --min-shot)")ap.add_argument("--auto", type=float, help="no switch list: alternate through the cameras every N seconds")+ ap.add_argument("--min-shot", type=float, default=1.5,+ help="--switch energy: never cut to a new camera for less than this many "+ "seconds (default 1.5)")+ ap.add_argument("--edl", help="write the resulting cut list to this file, one START-END per line "+ "(cut.py --segments format); the camera index for each cut is in "+ "the JSON `cuts` field alongside it")ap.add_argument("--audio", type=int, default=0, help="input index to take audio from (default 0 = reference)")ap.add_argument("--offsets-only", action="store_true", help="print the measured offsets and exit")ap.add_argument("--max-offset", type=float, default=30.0)@@ -137,8 +204,13 @@ def main() -> int:print(f"{p}: {o:+.3f}s (confidence {c:.2f})")return 0+ if args.min_shot <= 0:+ die(f"--min-shot must be positive, got {args.min_shot:g}")ref_dur = metas[0]["duration"] or 0.0- if args.switch:+ energy_mode = args.switch == "energy"+ if energy_mode:+ cuts = energy_switch(args.inputs, metas, offsets, ratios, ref_dur, args.min_shot)+ elif args.switch:cuts = parse_switch(args.switch, n)elif args.auto:if args.auto <= 0:@@ -167,6 +239,13 @@ def main() -> int:if not metas[c].get("video"):die(f"camera {c} ({args.inputs[c]}) has no video; it can only be used with --audio")+ if args.edl:+ if not STATE.dry_run:+ with open(args.edl, "w", encoding="utf-8") as fh:+ for s, e, _c in filled:+ fh.write(f"{s:.3f}-{e:.3f}\n")+ info(f"wrote {args.edl}")+v0 = metas[0]["video"]w, h = args.width or v0["width"], args.height or v0["height"]if v0.get("rotation") in (90, -90, 270, -270) and not (args.width or args.height):@@ -212,7 +291,8 @@ def main() -> int:run(cmd)r = probe(output, role="output")info(f"wrote {output} ({fmt_secs(r['duration'])}, {len(filled)} cuts, audio from input {a})")- emit(output, cuts=[[round(s, 3), round(e, 3), c] for s, e, c in filled], **report)+ extra = {"switch_mode": "energy", "min_shot": args.min_shot} if energy_mode else {}+ emit(output, cuts=[[round(s, 3), round(e, 3), c] for s, e, c in filled], **report, **extra)return 0diff --git a/scripts/scenes.py b/scripts/scenes.pyindex 91aff53..088664b 100755--- a/scripts/scenes.py+++ b/scripts/scenes.py@@ -25,7 +25,11 @@ from typing import Dict, List, Optional, Tuple# `detect_scenes` moved into _common/probe.py in 1.16.0 (see silence.py); the body is unchanged.from _common import (detect_scenes, STATE, add_common, apply_common, beat_grid, default_font_file, die, emit,escape_filter_path, ffmpeg_base, info, print_json, probe, run, decode_pcm_mono,- rms_envelope, BEAT_MIN_CONFIDENCE)+ rms_envelope, BEAT_MIN_CONFIDENCE, decode_gray_frames, frame_flow, label_shot_flow)++SHOT_FPS = 4.0 # frames/second sampled for --shots' flow estimate+SHOT_W, SHOT_H = 48, 27 # decode size for --shots (16:9-ish; enough blocks, still a few KB/shot)+SPEECH_STEP_S = 1.0 # --speech energy-ratio window@@ -54,6 +58,51 @@ def parse_beat_range(text: str) -> "tuple":return (lo, hi)+def shot_flow_label(path: str, start: float, end: float) -> Dict:+ """{start, end, label, flow_magnitude} for one shot: decode it at SHOT_FPS/SHOT_W x SHOT_H+ and block-match consecutive frames (see _common.decision.frame_flow / label_shot_flow). A+ shot under two sampled frames has nothing to compare and is reported static with+ flow_magnitude 0 -- there is no motion measurement to make on a single frame."""+ frames = decode_gray_frames(path, SHOT_FPS, SHOT_W, SHOT_H, start=start, seconds=max(0.0, end - start), check=False)+ flows = [frame_flow(frames[i], frames[i + 1], SHOT_W, SHOT_H) for i in range(len(frames) - 1)]+ label = label_shot_flow(flows)+ return {"start": round(start, 3), "end": round(end, 3), "label": label["label"], "flow_magnitude": label["flow_magnitude"]}+++def audio_peaks_db(samples: "List[float]", rate: int, step_s: float = 0.25) -> List[Dict]:+ """--audio-peaks: local maxima of the loudness envelope reported as measured dBFS, not the+ unitless RMS scenes.py has always put in the (unconditional) `audio_peaks` key -- a+ different unit needs a different key so the existing one keeps meaning what it always has."""+ env = rms_envelope(samples, int(rate * step_s))+ peaks = []+ for i, val in enumerate(env):+ if val <= 1e-6:+ continue+ if (i == 0 or env[i - 1] <= val) and (i == len(env) - 1 or env[i + 1] <= val):+ level = round(20 * math.log10(val), 1)+ peaks.append({"time": round(i * step_s, 2), "level": level})+ return peaks+++def speech_music_ratio(samples: "List[float]", rate: int, step_s: float = SPEECH_STEP_S) -> List[Dict]:+ """--speech: a per-second zero-crossing-rate ratio, reported as a measured number, not a+ speech/music label. Speech's rapid consonant transients drive the zero-crossing rate up and+ make it jump window to window; sustained tones (music, a held note, room tone) cross zero at+ a steadier rate. ratio = this window's ZCR / the file's median ZCR, so 1.0 is "typical for+ this file" regardless of its overall noisiness -- a proxy, in the same spirit as scenes.py's+ --rank-by, not a classifier: nothing here decides what is speech."""+ step = max(1, int(rate * step_s))+ zcrs: List[float] = []+ for i in range(0, len(samples) - step + 1, step):+ block = samples[i:i + step]+ crossings = sum(1 for a, b in zip(block, block[1:]) if (a >= 0) != (b >= 0))+ zcrs.append(crossings / max(1, len(block) - 1))+ if not zcrs:+ return []+ med = sorted(zcrs)[len(zcrs) // 2] or 1e-9+ return [{"time": round(i * step_s, 2), "speech_music_ratio": round(z / med, 3)} for i, z in enumerate(zcrs)]++def main() -> int:ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)ap.add_argument("input")@@ -76,6 +125,15 @@ def main() -> int:ap.add_argument("--min-confidence", type=float, default=BEAT_MIN_CONFIDENCE,help="with --beats: below this confidence the grid is still reported, marked "f"usable: false (default {BEAT_MIN_CONFIDENCE})")+ ap.add_argument("--shots", action="store_true",+ help="label each detected shot static / pan / motion by a measured optical-flow "+ "proxy (lightweight block matching over sampled frames); reports flow_magnitude too")+ ap.add_argument("--audio-peaks", action="store_true",+ help="report loudness peaks as {time, level} in measured dBFS (separate from the "+ "always-on `audio_peaks` RMS list used to pick --highlights)")+ ap.add_argument("--speech", action="store_true",+ help="report a per-second speech-vs-music energy ratio (zero-crossing-rate proxy, "+ "a measurement, not a speech/music classification)")ap.add_argument("--sheet", help="write a contact sheet PNG with the first frame of every scene")ap.add_argument("--no-timecode", action="store_true", help="--sheet without the burnt-in timecode stamp (a way out if drawtext itself is unusable, see doctor)")add_common(ap)@@ -100,7 +158,8 @@ def main() -> int:# With --beats the file is decoded once, at the finer rate, and both envelopes come from that# one pass: the 0.5 s scene blocks are an exact multiple of the 10 ms onset blocks.beat_rate = 22050- fine_samples = decode_pcm_mono(args.input, beat_rate, check=False) if (args.beats and meta.get("audio")) else None+ need_fine = args.beats or args.audio_peaks or args.speech+ fine_samples = decode_pcm_mono(args.input, beat_rate, check=False) if (need_fine and meta.get("audio")) else Noneif fine_samples is not None:env = audio_envelope(args.input, step_s, rate=beat_rate, samples=fine_samples)else:@@ -128,6 +187,32 @@ def main() -> int:result: Dict = {"file": args.input, "duration": round(dur, 3), "scene_count": len(scenes), "scenes": scenes, "audio_peaks": peaks}info(f"{len(scenes)} scenes, {len(peaks)} audio peaks over {dur:.1f}s")+ if args.shots:+ shots = [shot_flow_label(args.input, s, e) for s, e in zip(bounds[:-1], bounds[1:]) if e - s > 0.05]+ result["shots"] = shots+ counts = {}+ for sh in shots:+ counts[sh["label"]] = counts.get(sh["label"], 0) + 1+ info(f"--shots: {len(shots)} shots (" + ", ".join(f"{v} {k}" for k, v in sorted(counts.items())) + ")")++ if args.audio_peaks:+ if not meta.get("audio"):+ result["audio_peaks_db"] = []+ info("--audio-peaks: no audio stream, nothing to measure")+ else:+ db_samples = fine_samples if fine_samples is not None else decode_pcm_mono(args.input, 22050, check=False)+ result["audio_peaks_db"] = audio_peaks_db(db_samples, 22050 if fine_samples is not None else 22050)+ info(f"--audio-peaks: {len(result['audio_peaks_db'])} peaks")++ if args.speech:+ if not meta.get("audio"):+ result["speech"] = []+ info("--speech: no audio stream, nothing to measure")+ else:+ sp_samples = fine_samples if fine_samples is not None else decode_pcm_mono(args.input, 22050, check=False)+ result["speech"] = speech_music_ratio(sp_samples, 22050)+ info(f"--speech: {len(result['speech'])} one-second windows")+if args.beats:# A beat grid is a measurement of the music's periodicity, not a statement about where a# cut belongs. scenes.py reports what it measured, including a low confidence: reporting adiff --git a/scripts/silence.py b/scripts/silence.pyindex 4728fc7..c909c02 100755--- a/scripts/silence.py+++ b/scripts/silence.py@@ -42,6 +42,32 @@ def merge_spans(spans: List[Tuple[float, float]]) -> List[Tuple[float, float]]:return out+BREATH_FLOOR = 0.12 # shortest gap silencedetect is asked for under --speech-aware+++def speech_aware_silences(path: str, threshold: float, min_silence: float) -> "Tuple[List[Tuple[float, float]], List[Tuple[float, float]]]":+ """(sentence-boundary silences to remove, breaths kept) for --speech-aware.++ Re-runs the same silencedetect() already used everywhere else in this file, but with a much+ shorter minimum duration, so short in-sentence breaths are measured at all -- the plain+ `detect()` call above never sees them because its own --min-silence floor filters them out+ before they reach Python. Every gap silencedetect measured that is still shorter than+ --min-silence sits *inside* a sentence -- the two speech-flagged stretches on either side of+ it are close together in time because nothing longer separated them from the gaps around+ them -- and is kept rather than cut; --min-silence keeps the meaning it already has for+ --filler and the plain run: the shortest gap this tool will remove."""+ floor = min(BREATH_FLOOR, min_silence)+ fine = detect(path, threshold, floor)+ boundaries, breaths = [], []+ for s, e in fine:+ length = (e - s) if e != float("inf") else float("inf")+ if length >= min_silence:+ boundaries.append((s, e))+ else:+ breaths.append((s, e))+ return boundaries, breaths++def keep_ranges(silences: List[Tuple[float, float]], duration: float, margin: float, min_keep: float) -> List[Tuple[float, float]]:keeps: List[Tuple[float, float]] = []cursor = 0.0@@ -182,6 +208,11 @@ def main() -> int:ap.add_argument("--min-silence", type=float, default=0.6, help="only remove gaps at least this long in seconds (default 0.6)")ap.add_argument("--margin", type=float, default=0.15, help="seconds of silence to keep on each side of speech (default 0.15)")ap.add_argument("--min-keep", type=float, default=0.2, help="drop kept pieces shorter than this (default 0.2)")+ ap.add_argument("--speech-aware", action="store_true",+ help="keep breaths shorter than --min-silence when they sit inside a sentence "+ "(measured by re-running silence detection at a much shorter floor), and "+ "only cut at sentence-boundary pauses (--min-silence or longer). Composes "+ "with --filler through the same keep_ranges() removal list.")ap.add_argument("--list", action="store_true", help="only print silences and the kept ranges")ap.add_argument("--edl", help="write the kept ranges to this file, one START-END per line")fil = ap.add_argument_group("filler words (1.17)")@@ -229,7 +260,11 @@ def main() -> int:if not meta.get("audio"):die("input has no audio stream to analyse")duration = meta.get("duration") or 0.0- silences = detect(args.input, args.threshold, args.min_silence)+ breaths: List[Tuple[float, float]] = []+ if args.speech_aware:+ silences, breaths = speech_aware_silences(args.input, args.threshold, args.min_silence)+ else:+ silences = detect(args.input, args.threshold, args.min_silence)filler_info, filler_ranges = resolve_filler(args, meta) if args.filler else (None, [])# One sorted, merged removal list through the graph the tool already has: filler removal IS# time-range removal, so it reuses keep_ranges() and the same aselect/concat chain.@@ -253,6 +288,16 @@ def main() -> int:"kept_duration": round(kept, 3),"removed_seconds": round(silence_only, 3),}+ if args.speech_aware:+ summary["speech_aware"] = {+ "min_silence": args.min_silence, "floor": min(BREATH_FLOOR, args.min_silence),+ "breaths_kept": len(breaths),+ "breaths_kept_seconds": round(sum((e - s) for s, e in breaths if e != float("inf")), 3),+ "breaths": [[round(s, 3), None if e == float("inf") else round(e, 3)] for s, e in breaths],+ }+ info(f"--speech-aware: {len(breaths)} breath(s) kept "+ f"({summary['speech_aware']['breaths_kept_seconds']:.2f}s), "+ f"{len(silences)} sentence-boundary silence(s) cut")if filler_info is not None:# removed_seconds above is the silence-only figure, unchanged in meaning; the filler share# is reported inside `filler`, and removed_seconds_total is the additive sibling thatdiff --git a/scripts/sync.py b/scripts/sync.pyindex 6023e16..5057109 100755--- a/scripts/sync.py+++ b/scripts/sync.py@@ -30,7 +30,7 @@ import jsonimport mathimport osimport sys-from typing import List+from typing import Dict, Listfrom _common import video_args, add_common, apply_common, emit, aac_args, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, x264_args, decode_pcm_mono, rms_envelope@@ -195,84 +195,126 @@ def measure_offset(ref_path: str, oth_path: str, start: float, seconds: float, sreturn offset, max(0.0, min(1.0, score))-def main() -> int:- ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)- ap.add_argument("reference", help="reference recording (usually the camera video)")- ap.add_argument("second", help="recording to align (external audio or second camera)")- ap.add_argument("-o", "--output", help="output file when writing a synced result")- ap.add_argument("--max-offset", type=float, default=30.0, help="largest offset to search in seconds (default 30)")- ap.add_argument("--analyze-seconds", type=float, default=120.0, help="how much audio to analyse from each file (default 120)")- ap.add_argument("--step-ms", type=float, default=20.0, help="coarse envelope resolution in ms for the FFT search (default 20)")- ap.add_argument("--fine-ms", type=float, default=1.0, help="fine resolution in ms for the refinement pass, 0 to skip (default 1)")- ap.add_argument("--fix-drift", action="store_true", help="also measure the offset near the END and correct clock drift by resampling the second file")- ap.add_argument("--drift-window", type=float, default=60.0, help="seconds of audio analysed at each end for drift (default 60)")- mode = ap.add_mutually_exclusive_group()- mode.add_argument("--replace-audio", action="store_true", help="write reference video with the second file's audio, aligned")- mode.add_argument("--trim-second", action="store_true", help="write the second file shifted so it lines up with the reference")- ap.add_argument("--crf", type=int, default=18)- add_common(ap)- args = ap.parse_args()- apply_common(args)- if args.analyze_seconds > 900:- die(f"--analyze-seconds {args.analyze_seconds:g}: the window is decoded into memory; 900 s is the ceiling")-- for p in (args.reference, args.second):- if not probe(p).get("audio"):- die(f"{p} has no audio stream to correlate")-- offset, score = measure_offset(args.reference, args.second, 0.0, args.analyze_seconds, args.step_ms, args.max_offset, args.fine_ms)-+def measure_one(reference: str, path: str, args) -> Dict:+ """Everything sync.py measures for ONE other source against the reference: offset,+ confidence and (with --fix-drift) the same end-of-file drift measurement main() has always+ made for a single pair. Used for every source when N>1, and for the historical single-pair+ CLI shape when N==1 -- one measurement, so the two shapes cannot disagree."""+ offset, score = measure_offset(reference, path, 0.0, args.analyze_seconds, args.step_ms, args.max_offset, args.fine_ms)drift_ratio = 1.0drift_info = Noneif args.fix_drift:- ref_dur = probe(args.reference)["duration"] or 0.0- sec_dur = probe(args.second)["duration"] or 0.0- overlap_end = min(ref_dur, sec_dur + offset) # last reference time both files cover+ ref_dur = probe(reference)["duration"] or 0.0+ sec_dur = probe(path)["duration"] or 0.0+ overlap_end = min(ref_dur, sec_dur + offset)head_len = min(args.analyze_seconds, overlap_end)tail_start = overlap_end - args.drift_windowif tail_start <= head_len / 2 + 5:- info("warning: files too short to measure drift reliably; skipping drift correction")+ info(f"warning: {path} too short to measure drift reliably; skipping drift correction")else:ref_start = tail_startsec_start = tail_start - offsetif sec_start < 0:ref_start -= sec_startsec_start = 0.0- ref_s = decode_mono(args.reference, args.drift_window, ref_start)- oth_s = decode_mono(args.second, args.drift_window, sec_start)+ ref_s = decode_mono(reference, args.drift_window, ref_start)+ oth_s = decode_mono(path, args.drift_window, sec_start)step = max(1, int(SR * args.step_ms / 1000))lag, end_score = cross_correlate(envelope(ref_s, step), envelope(oth_s, step), int(2.0 * SR / step))residual = lag * step / SRif args.fine_ms:residual = refine(ref_s, oth_s, residual, max(1, int(SR * args.fine_ms / 1000)), args.step_ms / 1000 * 2)- # both measurements represent the offset at the centre of their windowshead_mid = head_len / 2tail_mid = ref_start + args.drift_window / 2elapsed = tail_mid - head_midif elapsed > 0 and end_score > 0.1:- # offset(T) = offset0 - (ratio - 1) * T, where ratio is how fast the second file's clock- # runs relative to the reference (ratio > 1 = the second file is too long / plays slow)drift_ratio = 1.0 - residual / elapsed- offset = offset + (drift_ratio - 1.0) * head_mid # extrapolate back to T = 0+ offset = offset + (drift_ratio - 1.0) * head_middrift_info = {"residual_at_end_seconds": round(residual, 4), "measured_over_seconds": round(elapsed, 2),"drift_ppm": round((drift_ratio - 1) * 1e6, 1),"meaning": "second file runs %.1f ppm %s (%.3fs over %.0fs); it will be resampled to match" % (abs(drift_ratio - 1) * 1e6, "long/slow" if drift_ratio > 1 else "short/fast", abs(residual), elapsed),"confidence": round(end_score, 3)}else:- info("warning: could not measure drift with confidence; skipping drift correction")-- result = {- "reference": args.reference,- "second": args.second,- "offset_seconds": round(offset, 4),- "confidence": round(max(0.0, min(1.0, score)), 3),- "meaning": ("second starts %.3fs %s than reference" % (abs(offset), "later" if offset > 0 else "earlier")),- }- if drift_info:- result["drift"] = drift_info- if result["confidence"] < 0.1:- info("warning: low correlation confidence; check that both files contain the same audio event")+ info(f"warning: could not measure drift with confidence for {path}; skipping drift correction")+ return {"path": path, "offset_seconds": round(offset, 4), "offset_s": round(offset, 4),+ "confidence": round(max(0.0, min(1.0, score)), 3), "drift_ratio": drift_ratio, "drift": drift_info,+ "drift_ppm": (drift_info["drift_ppm"] if drift_info else None)}+++def main() -> int:+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ ap.add_argument("reference", help="reference recording (usually the camera video)")+ ap.add_argument("second", help="recording to align (external audio or second camera)")+ ap.add_argument("more_sources", nargs="*", metavar="SOURCE3...", default=[],+ help="(1.18) additional recordings beyond `second` to align to the same "+ "reference -- a third camera, a second external recorder. With none of "+ "these, sync.py keeps its original 2-source shape (offset_seconds/"+ "confidence/drift at the top level, --replace-audio/--trim-second "+ "available). With 1+, all sources (second plus these) are measured and "+ "reported as one offsets JSON {reference, sources: [{path, offset_s, "+ "confidence, drift_ppm}]}, and --replace-audio/--trim-second refuse: "+ "each writes ONE synced output and there is more than one source")+ ap.add_argument("-o", "--output", help="output file when writing a synced result (one-source runs only)")+ ap.add_argument("--max-offset", type=float, default=30.0, help="largest offset to search in seconds (default 30)")+ ap.add_argument("--analyze-seconds", type=float, default=120.0, help="how much audio to analyse from each file (default 120)")+ ap.add_argument("--step-ms", type=float, default=20.0, help="coarse envelope resolution in ms for the FFT search (default 20)")+ ap.add_argument("--fine-ms", type=float, default=1.0, help="fine resolution in ms for the refinement pass, 0 to skip (default 1)")+ ap.add_argument("--fix-drift", action="store_true", help="also measure the offset near the END and correct clock drift by resampling the second file")+ ap.add_argument("--drift-window", type=float, default=60.0, help="seconds of audio analysed at each end for drift (default 60)")+ mode = ap.add_mutually_exclusive_group()+ mode.add_argument("--replace-audio", action="store_true", help="write reference video with the second file's audio, aligned")+ mode.add_argument("--trim-second", action="store_true", help="write the second file shifted so it lines up with the reference")+ ap.add_argument("--crf", type=int, default=18)+ add_common(ap)+ args = ap.parse_args()+ apply_common(args)+ if args.analyze_seconds > 900:+ die(f"--analyze-seconds {args.analyze_seconds:g}: the window is decoded into memory; 900 s is the ceiling")+ args.sources = [args.second] + list(args.more_sources)++ for p in (args.reference,) + tuple(args.sources):+ if not probe(p).get("audio"):+ die(f"{p} has no audio stream to correlate")++ n_sources = len(args.sources)+ if (args.replace_audio or args.trim_second) and n_sources > 1:+ die("--replace-audio/--trim-second write ONE synced output and need exactly one SOURCE; "+ f"{n_sources} were given. Run sync.py once per source, or drop the flag and read the "+ "offsets JSON.", kind="input")++ measured = [measure_one(args.reference, p, args) for p in args.sources]+ for p, m in zip(args.sources, measured):+ if m["confidence"] < 0.1:+ info(f"warning: {p}: low correlation confidence; check that it shares an audio event with the reference")++ sources_block = [{"path": p, "offset_s": m["offset_s"], "confidence": m["confidence"],+ **({"drift_ppm": m["drift_ppm"]} if args.fix_drift else {})}+ for p, m in zip(args.sources, measured)]++ if n_sources == 1:+ # The original 2-source shape, unchanged, plus (additively) the same measurement under+ # `sources` so a caller reading the new key gets the same numbers for a single source.+ args.second = args.sources[0]+ m = measured[0]+ offset, score, drift_ratio, drift_info = m["offset_seconds"], m["confidence"], m["drift_ratio"], m["drift"]+ result = {+ "reference": args.reference,+ "second": args.second,+ "offset_seconds": offset,+ "confidence": score,+ "meaning": ("second starts %.3fs %s than reference" % (abs(offset), "later" if offset > 0 else "earlier")),+ "sources": sources_block,+ }+ if drift_info:+ result["drift"] = drift_info+ if score < 0.1:+ info("warning: low correlation confidence; check that both files contain the same audio event")+ else:+ offset = drift_ratio = drift_info = None # not used below; output writing is 1-source only+ result = {"reference": args.reference, "sources": sources_block}+ info(f"measured {n_sources} sources against {args.reference}: " ++ ", ".join(f"{s['path']}: {s['offset_s']:+.3f}s (confidence {s['confidence']:.2f})" for s in sources_block))if args.replace_audio or args.trim_second:output = args.output or default_output(args.reference if args.replace_audio else args.second, "synced", "mp4")@@ -339,12 +381,18 @@ def main() -> int:if args.json:emit(result.get("output"), **{k: v for k, v in result.items() if k != "output"})- else:+ elif n_sources == 1:print(f"offset: {result['offset_seconds']:+.3f}s ({result['meaning']}), confidence {result['confidence']:.2f}")if drift_info:print(f"drift: {drift_info['drift_ppm']:+.1f} ppm ({drift_info['residual_at_end_seconds']:+.3f}s over {drift_info['measured_over_seconds']:.0f}s), confidence {drift_info['confidence']:.2f}")if "output" in result:print(result["output"])+ else:+ for s in sources_block:+ line = f"{s['path']}: {s['offset_s']:+.3f}s, confidence {s['confidence']:.2f}"+ if args.fix_drift:+ line += f", drift {s['drift_ppm']:+.1f} ppm" if s["drift_ppm"] is not None else ""+ print(line)return 0diff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.jsonindex 7fe836c..35966f5 100644--- a/tests/fixtures/mcp_tools.json+++ b/tests/fixtures/mcp_tools.json@@ -302,6 +302,7 @@"json","json_brief","limit",+ "motion_centre","overwrite","plan","progress",@@ -741,6 +742,7 @@"codec","crf","dry_run",+ "edl","fast","fix_drift","fps",@@ -749,6 +751,7 @@"json","json_brief","max_offset",+ "min_shot","offsets_only","output","overwrite",@@ -994,6 +997,7 @@"name": "scenes","properties": ["argv",+ "audio_peaks","beat_range","beat_step","beats",@@ -1014,6 +1018,8 @@"rank_by","ratio","sheet",+ "shots",+ "speech","target","threshold","timeout"@@ -1076,6 +1082,7 @@"preset","progress","quality",+ "speech_aware","threshold","timeout","transcribe",@@ -1205,6 +1212,7 @@"json","json_brief","max_offset",+ "more_sources","output","overwrite","plan",diff --git a/tests/test_analysis.py b/tests/test_analysis.pyindex 2b69848..b0bd0ab 100644--- a/tests/test_analysis.py+++ b/tests/test_analysis.py@@ -64,6 +64,30 @@ class AnalysisTests(MediaFixtures):def test_cropdetect_bad_seconds_refused(self):script("cropdetect.py", self.src, "--seconds", "0", expect_fail=True)+ # ---------------------------------------------------------------- 1.18.0: cropdetect.py --motion-centre+ def test_cropdetect_motion_centre_reports_per_second_points(self):+ data = json.loads(script("cropdetect.py", self.src, "--seconds", "4", "--samples", "2",+ "--motion-centre", "--json").stdout)+ self.assertIn("motion_centre", data)+ self.assertGreater(len(data["motion_centre"]), 0)+ for pt in data["motion_centre"]:+ self.assertIn("time", pt)+ if pt["x"] is not None:+ self.assertGreaterEqual(pt["x"], 0)+ self.assertLessEqual(pt["x"], data["source_width"])+ self.assertGreaterEqual(pt["x_frac"], 0.0)+ self.assertLessEqual(pt["x_frac"], 1.0)++ def test_cropdetect_motion_centre_writes_no_file(self):+ before = set(OUT.iterdir())+ script("cropdetect.py", self.src, "--seconds", "1", "--samples", "1", "--motion-centre")+ self.assertEqual(before, set(OUT.iterdir()))++ def test_cropdetect_motion_centre_is_off_by_default(self):+ data = json.loads(script("cropdetect.py", self.src, "--seconds", "1", "--samples", "1",+ "--json").stdout)+ self.assertNotIn("motion_centre", data)+def test_fonts_dir_does_not_bypass_the_coverage_check(self):"""--fonts-dir says "also look here", not "this exact face": a directory that does notcover the script must not silently switch the guarantee off."""@@ -119,6 +143,59 @@ class AnalysisTests(MediaFixtures):again = json.loads(script("sync.py", self.src, out, "--json").stdout)self.assertClose(again["offset_seconds"], 0.0, 0.05)+ # ---------------------------------------------------------------- 1.18.0: sync.py 3+ sources+ def _cam3(self):+ """A third source: the reference shifted 1.2 s later, so it has a real measurable offset+ distinct from self.mic's 2.5 s."""+ cam3 = OUT / "sync_cam3.wav"+ if not cam3.exists():+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",+ "-i", "aevalsrc=0:s=48000:d=1.2", "-i", self.src,+ "-filter_complex", "[0:a][1:a]concat=n=2:v=0:a=1", cam3)+ return cam3++ def test_sync_two_sources_keeps_the_original_shape(self):+ """The 2-source CLI shape is a special case of the N-source one, byte-for-byte: the same+ top-level offset_seconds/confidence/second keys tested since 1.0, plus (additively) the+ new `sources` list."""+ data = json.loads(script("sync.py", self.src, self.mic, "--json").stdout)+ self.assertIn("offset_seconds", data)+ self.assertIn("second", data)+ self.assertIn("confidence", data)+ self.assertIn("sources", data)+ self.assertEqual(len(data["sources"]), 1)+ self.assertEqual(data["sources"][0]["path"], str(self.mic))+ self.assertAlmostEqual(data["sources"][0]["offset_s"], data["offset_seconds"], delta=0.001)++ def test_sync_three_sources_writes_one_offsets_json(self):+ data = json.loads(script("sync.py", self.src, self.mic, self._cam3(), "--json").stdout)+ self.assertNotIn("second", data) # no single-pair shape for N>1+ self.assertNotIn("offset_seconds", data)+ self.assertEqual(data["reference"], str(self.src))+ self.assertEqual(len(data["sources"]), 2)+ by_path = {s["path"]: s for s in data["sources"]}+ self.assertClose(by_path[str(self.mic)]["offset_s"], 2.5, 0.05)+ self.assertClose(by_path[str(self._cam3())]["offset_s"], -1.2, 0.05)+ for s in data["sources"]:+ self.assertIn("confidence", s)++ def test_sync_three_sources_reports_drift_ppm_per_source(self):+ data = json.loads(script("sync.py", self.long_ref, self.long_drift, self.long_ref,+ "--fix-drift", "--json").stdout)+ drifts = {s["path"]: s["drift_ppm"] for s in data["sources"]}+ self.assertClose(drifts[str(self.long_drift)], 500.0, 60.0)+ self.assertClose(drifts[str(self.long_ref)], 0.0, 60.0)++ def test_sync_replace_audio_refuses_multiple_sources(self):+ r = script("sync.py", self.src, self.mic, self._cam3(), "--replace-audio",+ "-o", OUT / "nope.mp4", "--json", expect_fail=True)+ self.assertEqual(json.loads(r.stdout)["error"]["kind"], "input")++ def test_sync_three_sources_text_output(self):+ proc = script("sync.py", self.src, self.mic, self._cam3())+ self.assertIn(str(self.mic), proc.stdout)+ self.assertIn(str(self._cam3()), proc.stdout)+# ---------------------------------------------------------------- real-world materialdef test_probe_detects_vfr_rotation_surround_hdr(self):self.assertTrue(probe(str(self.vfr))["video"]["variable_frame_rate_suspected"])@@ -579,6 +656,53 @@ class AnalysisTests(MediaFixtures):"--json").stdout)self.assertNotIn("beat_grid", data)+ # --------------------------------------------------------------- 1.18.0: --shots / --audio-peaks / --speech+ def test_scenes_shots_labels_every_scene(self):+ data = json.loads(script("scenes.py", self.src, "--shots", "--json").stdout)+ self.assertIn("shots", data)+ self.assertEqual(len(data["shots"]), data["scene_count"])+ for sh in data["shots"]:+ self.assertIn(sh["label"], ("static", "pan", "motion"))+ self.assertGreaterEqual(sh["flow_magnitude"], 0.0)+ self.assertIn("start", sh)+ self.assertIn("end", sh)++ def test_scenes_shots_static_clip_is_labelled_static(self):+ """A single unmoving frame held for the whole clip has nothing to flow: static, magnitude 0."""+ still = OUT / "still.mp4"+ if not still.exists():+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",+ "-i", "color=c=gray:s=320x180:rate=10", "-f", "lavfi", "-i", "sine=f=440",+ "-t", "4", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p",+ "-c:a", "aac", still)+ data = json.loads(script("scenes.py", still, "--shots", "--json").stdout)+ self.assertTrue(all(sh["label"] == "static" for sh in data["shots"]))+ self.assertTrue(all(sh["flow_magnitude"] == 0.0 for sh in data["shots"]))++ def test_scenes_audio_peaks_db_is_measured_in_dbfs_and_separate_from_the_rms_list(self):+ data = json.loads(script("scenes.py", self.src, "--audio-peaks", "--json").stdout)+ self.assertIn("audio_peaks_db", data)+ self.assertIn("audio_peaks", data) # the pre-existing key, unchanged+ for p in data["audio_peaks_db"]:+ self.assertIn("time", p)+ self.assertIn("level", p)+ self.assertLess(p["level"], 0) # dBFS, not linear RMS++ def test_scenes_speech_reports_a_ratio_not_a_label(self):+ data = json.loads(script("scenes.py", self.src, "--speech", "--json").stdout)+ self.assertIn("speech", data)+ self.assertGreater(len(data["speech"]), 0)+ for w in data["speech"]:+ self.assertIn("time", w)+ self.assertIn("speech_music_ratio", w)+ self.assertGreaterEqual(w["speech_music_ratio"], 0.0)++ def test_scenes_shots_audio_peaks_speech_combine_with_beats_and_highlights(self):+ data = json.loads(script("scenes.py", self._beats(), "--shots", "--audio-peaks", "--speech",+ "--beats", "--highlights", "2", "--json").stdout)+ for key in ("shots", "audio_peaks_db", "speech", "beats", "highlights"):+ self.assertIn(key, data)+class ProposeChaptersTests(unittest.TestCase):"""1.16: the pure half of metadata.py --auto-chapters. No media, no subprocess."""diff --git a/tests/test_editing.py b/tests/test_editing.pyindex 89ffa2e..ac1104d 100644--- a/tests/test_editing.py+++ b/tests/test_editing.py@@ -1272,6 +1272,72 @@ class EditingTests(MediaFixtures):"--json").stdout)["filler"]self.assertIn("so", extra["removed_words"])+ # ------------------------------------------------ 1.18.0: silence.py --speech-aware+ def _breathy(self):+ """10 s: speech 0-3s, a 0.25 s breath (well under the 0.6 s default --min-silence),+ speech 3.25-6s, a 1 s sentence-boundary pause 6-7s, speech 7-10s."""+ clip = OUT / "breathy.mp4"+ if not clip.exists():+ expr = "0.5*sin(2*PI*440*t)*(lt(t\\,3)+between(t\\,3.25\\,6)+gt(t\\,7))"+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",+ "-i", f"aevalsrc='{expr}':s=48000", "-t", "10", clip)+ return clip++ def test_speech_aware_keeps_a_short_breath_and_cuts_the_sentence_boundary(self):+ data = json.loads(script("silence.py", self._breathy(), "--speech-aware", "--list",+ "--margin", "0.05", "--json").stdout)+ sa = data["speech_aware"]+ self.assertEqual(sa["breaths_kept"], 1)+ self.assertAlmostEqual(sa["breaths"][0][0], 3.0, delta=0.1)+ self.assertAlmostEqual(sa["breaths"][0][1], 3.25, delta=0.1)+ # the sentence-boundary pause (6-7s) is the only one removed+ self.assertEqual(len(data["silences"]), 1)+ self.assertAlmostEqual(data["silences"][0][0], 6.0, delta=0.1)++ def test_speech_aware_without_the_flag_would_have_cut_the_breath_too(self):+ """With a --min-silence short enough to catch the breath, the plain (non-speech-aware)+ run removes it; --speech-aware is what keeps it."""+ plain = json.loads(script("silence.py", self._breathy(), "--min-silence", "0.2",+ "--threshold", "-35", "--list", "--json").stdout)+ aware = json.loads(script("silence.py", self._breathy(), "--min-silence", "0.6",+ "--speech-aware", "--list", "--json").stdout)+ self.assertEqual(len(plain["silences"]), 2) # breath + sentence pause, both cut+ self.assertEqual(len(aware["silences"]), 1) # only the sentence pause+ self.assertEqual(aware["speech_aware"]["breaths_kept"], 1)++ def test_speech_aware_writes_an_edl_like_the_plain_flag(self):+ edl = OUT / "speech_aware.edl"+ script("silence.py", self._breathy(), "--speech-aware", "--edl", edl, "--list", "--json")+ lines = edl.read_text(encoding="utf-8").strip().splitlines()+ self.assertGreaterEqual(len(lines), 1)+ for line in lines:+ a, b = line.split("-")+ self.assertLess(float(a), float(b))++ def test_speech_aware_composes_with_filler_into_one_removal_list(self):+ """The trickiest part of 1.18.0: --speech-aware and --filler must flow through the same+ keep_ranges() and produce one removal list, not two independent cuts. A filler word+ placed inside the kept breath must still be removed -- and the breath must still not be+ cut just because --filler ran."""+ clip = self._breathy()+ words = OUT / "words_breath.json"+ # a filler word squarely inside the kept breath (3.0-3.25s)+ words.write_text(json.dumps({"language": "en", "segments": [+ {"words": [{"word": "um", "start": 3.05, "end": 3.15}]}]}), encoding="utf-8")+ out = OUT / "speech_aware_filler.mp4"+ data = json.loads(script("silence.py", clip, "--speech-aware", "--filler",+ "--words", words, "-o", out, "--margin", "0.02", "--json").stdout)+ self.assertIn("speech_aware", data)+ self.assertIn("filler", data)+ # the breath is still reported as kept by --speech-aware...+ self.assertEqual(data["speech_aware"]["breaths_kept"], 1)+ # ...but the filler word inside it is removed anyway: one composed list, not two cuts+ # that ignore each other. removed_seconds_total covers both the sentence pause and the+ # filler word.+ self.assertGreater(data["removed_seconds_total"], data["removed_seconds"])+ m = probe(str(out))+ self.assertLess(m["duration"], data["input_duration"])+def test_filler_warns_about_a_japanese_discourse_marker(self):words = [("\u305d\u308c\u306f", 0.2, 0.5), ("\u306a\u3093\u304b", 0.6, 0.85), ("\u3044\u3044", 1.0, 1.3)]data = json.loads(script("silence.py", self._gappy(), "--filler", "--filler-list",diff --git a/tests/test_orchestration.py b/tests/test_orchestration.pyindex 84cf0c8..e26e5ac 100644--- a/tests/test_orchestration.py+++ b/tests/test_orchestration.py@@ -253,6 +253,96 @@ class OrchestrationTests(MediaFixtures):self.assertClose(probe(str(auto))["duration"], 12.0, 0.2)script("multicam.py", self.src, camB, "--switch", "0-3:5", expect_fail=True)+ # ------------------------------------------------------------ 1.18.0: --switch energy / --edl+ def _loud_cams(self):+ """Two cameras on one reference timeline: camA loud 0-6s / quiet 6-12s, camB the reverse+ -- --switch energy should pick camA for the first half and camB for the second."""+ camA = OUT / "mc_energy_a.mp4"+ camB = OUT / "mc_energy_b.mp4"+ if not camA.exists():+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",+ "-i", "testsrc2=size=160x90:rate=30", "-f", "lavfi",+ "-i", "aevalsrc='0.8*sin(2*PI*440*t)*lt(t\\,6)+0.01*sin(2*PI*440*t)*gt(t\\,6)':s=48000",+ "-t", "12", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-c:a", "aac", camA)+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",+ "-i", "testsrc2=size=160x90:rate=30,hue=h=90", "-f", "lavfi",+ "-i", "aevalsrc='0.01*sin(2*PI*440*t)*lt(t\\,6)+0.8*sin(2*PI*440*t)*gt(t\\,6)':s=48000",+ "-t", "12", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-c:a", "aac", camB)+ return camA, camB++ def test_multicam_switch_energy_picks_the_loudest_camera(self):+ camA, camB = self._loud_cams()+ out = OUT / "mc_energy.mp4"+ data = json.loads(script("multicam.py", camA, camB, "--switch", "energy", "--min-shot", "1",+ "--fast", "-o", out, "--json").stdout)+ self.assertEqual(data["switch_mode"], "energy")+ cams_by_time = {}+ for s, e, c in data["cuts"]:+ cams_by_time[(s + e) / 2] = c+ early = [c for t, c in cams_by_time.items() if t < 5]+ late = [c for t, c in cams_by_time.items() if t > 7]+ self.assertTrue(early and all(c == 0 for c in early), f"expected camera 0 (louder) early: {data['cuts']}")+ self.assertTrue(late and all(c == 1 for c in late), f"expected camera 1 (louder) late: {data['cuts']}")++ def test_multicam_switch_energy_respects_min_shot(self):+ camA, camB = self._loud_cams()+ out = OUT / "mc_energy_minshot.mp4"+ data = json.loads(script("multicam.py", camA, camB, "--switch", "energy", "--min-shot", "3",+ "--fast", "-o", out, "--json").stdout)+ for s, e, _c in data["cuts"]:+ # the very first/last cut may be shorter (it borders the clip edge), interior cuts+ # must respect --min-shot+ if s > 0.01 and e < 11.99:+ self.assertGreaterEqual(round(e - s, 2), 3.0 - 0.05, data["cuts"])++ def test_multicam_edl_matches_the_reported_cuts(self):+ camA, camB = self._loud_cams()+ edl = OUT / "mc_energy.edl"+ out = OUT / "mc_energy_edl.mp4"+ data = json.loads(script("multicam.py", camA, camB, "--switch", "energy", "--min-shot", "1",+ "--edl", edl, "--fast", "-o", out, "--json").stdout)+ lines = edl.read_text(encoding="utf-8").strip().splitlines()+ self.assertEqual(len(lines), len(data["cuts"]))+ for line, (s, e, _c) in zip(lines, data["cuts"]):+ a, b = (float(x) for x in line.split("-"))+ self.assertAlmostEqual(a, s, delta=0.01)+ self.assertAlmostEqual(b, e, delta=0.01)+ self.assertLess(a, b)++ def test_multicam_switch_energy_needs_at_least_one_video_camera(self):+ wav_a = OUT / "mc_energy_audio_only_a.wav"+ wav_b = OUT / "mc_energy_audio_only_b.wav"+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", f"aevalsrc='{TONES}':s=48000", "-t", "4", wav_a)+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", f"aevalsrc='{TONES}':s=48000", "-t", "4", wav_b)+ script("multicam.py", wav_a, wav_b, "--switch", "energy", "-o", OUT / "mc_nope.mp4", expect_fail=True)++ def test_multicam_min_shot_must_be_positive(self):+ camA, camB = self._loud_cams()+ script("multicam.py", camA, camB, "--switch", "energy", "--min-shot", "0", "-o", OUT / "mc_nope2.mp4", expect_fail=True)++ def test_multicam_edl_is_renderable_as_a_render_project(self):+ """The multicam timeline is expressible as a render.py project without a new stage type:+ each cut becomes a clip that points at its own camera's file with in/out on that camera's+ OWN timeline (reference time shifted by the measured offset), using render.py's existing+ clips array -- see docs/contract.md and docs/design-decisions.md for why no new project+ schema was needed."""+ camA, camB = self._loud_cams()+ data = json.loads(script("multicam.py", camA, camB, "--switch", "energy", "--min-shot", "1",+ "--fast", "-o", OUT / "mc_energy_proj.mp4", "--json").stdout)+ offsets = data["offsets_seconds"]+ clips = [{"src": [str(camA), str(camB)][c], "in": round(s - offsets[c], 3), "out": round(e - offsets[c], 3)}+ for s, e, c in data["cuts"]]+ project = OUT / "mc_project.json"+ project.write_text(json.dumps({"output": "mc_rendered.mp4", "clips": clips}), encoding="utf-8")+ out = OUT / "mc_rendered.mp4"+ script("render.py", project, "--fast", "-o", out)+ self.assertTrue(out.exists())+ m = probe(str(out))+ # the measured per-camera offset carries its own small error, so this is a looser bound+ # than the multicam output's own duration check -- the point is that render.py accepts the+ # clip list at all, not a frame-accurate re-derivation of multicam's own timeline math.+ self.assertGreater(m["duration"], 8.0)+def test_multicam_fix_drift_trims_before_resample_not_after(self):"""--fix-drift's audio path computes a_start (an atrim start point) in the source's ownpre-correction time axis, but the filter chain used to apply asetrate/aresample (the drift