Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
multicam.py: warn on low-confidence cameras; document sync != lip sync (#28)
Shogo Kajiyama5a227bf29ef9
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f5c2e2c..46b0a6e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,8 @@## Unreleased — FFmpeg 8+ / Windows compatibility for caption and color+- **`multicam.py` low-confidence warning.** `sync.py` warns on stderr when its cross-correlation confidence is below 0.1 ("check that both files contain the same audio event"); `multicam.py` used the same measurement per camera but never warned, even though it applies the offset to a rendered cut rather than just reporting it. It now warns per camera below the same threshold. SKILL.md's existing "check `confidence` before trusting a sync" guidance now names `multicam.py`'s per-camera confidence explicitly.+- **`sync.py`/`multicam.py` documentation: audio sync is not lip sync.** Both align audio tracks to each other by cross-correlation; neither has ever done any face or mouth detection, and a high `confidence` only means the audio matched well, not that the final picture looks in sync. This was previously undocumented; `sync.py`'s docstring, `multicam.py`'s docstring and SKILL.md now say so explicitly. No behaviour change.- **`export.py --preset copy`.** Every existing preset re-encodes (even `prores`/`h265`, whichkeep the source resolution). A caller with nothing to change — the deliverable already matchesthe source, no platform target — had no way to get a real, delivered file out of `export.py`diff --git a/README.md b/README.mdindex 94fdf63..06d6c08 100644--- a/README.md+++ b/README.md@@ -144,7 +144,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth| Tool | What it does ||---|---|| `audio.py` | Voice clean-up chain, FFT denoise, typed compressor / limiter / gate, music bed with sidechain ducking, fades, 5.1 → stereo, track replacement, extraction (`-o out.wav`), `--audio-stream N` |-| `sync.py` | Offset between two recordings by audio cross-correlation (1 ms, pure Python), clock-drift correction; aligned video or audio out |+| `sync.py` | Offset between two recordings by audio cross-correlation (1 ms, pure Python), clock-drift correction; aligned video or audio out (audio-to-audio only — no lip-sync/face detection) || `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP or any target, video stream-copied; `--measure-only` |**Picture**diff --git a/SKILL.md b/SKILL.mdindex db735ef..b07921a 100644--- a/SKILL.md+++ b/SKILL.md@@ -176,7 +176,8 @@ Keep it to those five lines plus anything the user must decide. Attach the conta- Re-encoding an HDR (iPhone, HDR10) source through the SDR path: colours go flat. The scripts keep HDR; if you hand-write ffmpeg, do not tag BT.709 on BT.2020 pixels.- Lossless `-c copy` cuts on VFR or non-keyframe boundaries: the file "works" but starts on a frozen or wrong frame. `cut.py` re-encodes automatically when the snap exceeds 0.5 s; respect that.-- A sync with `confidence` under 0.3, or an offset larger than 60 % of the analysis window: probably wrong; enlarge `--analyze-seconds` or find a clap.+- A sync or multicam alignment with `confidence` under 0.3, or an offset larger than 60 % of the analysis window: probably wrong; enlarge `--analyze-seconds` or find a clap. `multicam.py` reports one `confidence` per camera — check all of them, not just that the command succeeded, before trusting the cut.+- `sync.py`/`multicam.py` align audio tracks to each other, never lip sync (mouth movement vs. audio) — there is no face or mouth detection anywhere in this skill. A high confidence means the audio matched well, not that the picture looks right; if the user asks whether lip sync is correct, that needs a look at the actual video, not just the reported offset.- "Normalised" audio that still clips: check true peak, not just LUFS (`check.py` does both).- Normalising ambience or near-silence to a speech target: a clip measured at-40 LUFS or below is room tone, wind or nothing; raising it 25 dB raises thediff --git a/scripts/multicam.py b/scripts/multicam.pyindex 0158bf9..c26f1b7 100755--- a/scripts/multicam.py+++ b/scripts/multicam.py@@ -11,6 +11,14 @@ Switch list format: "START-END:CAM,START-END:CAM,..." with times on thereference timeline (seconds or mm:ss) and CAM = input index (0 = reference).Gaps fall back to camera 0.+Each camera's `confidence` (in the report, and warned on stderr below 0.1)+is how well its audio matched the reference's, not a guarantee the cut lands+in sync: a source with no shared audio event (music-only vs. a silent room,+or two rooms recording different conversations) can score low and still get+an offset applied. Check it before trusting a low-confidence multicam edit.+This aligns audio tracks to each other, the same as sync.py, and does not+check lip sync (mouth movement vs. audio) at all -- see sync.py's docstring.+Examples:python3 multicam.py camA.mp4 camB.mp4 --offsets-only # just report the offsetspython3 multicam.py camA.mp4 camB.mp4 --switch "0-12:0,12-30:1,30-45:0" -o edit.mp4@@ -108,6 +116,8 @@ def main() -> int:ratios.append(ratio)conf.append(score)info(f"{p}: offset {off:+.3f}s (confidence {score:.2f})" + (f", drift {(ratio - 1) * 1e6:+.0f} ppm" if args.fix_drift else ""))+ if score < 0.1:+ info(f"warning: {p} has low correlation confidence ({score:.2f}); check that it shares an audio event with the reference before trusting this offset")report = {"inputs": args.inputs, "offsets_seconds": [round(o, 4) for o in offsets],"confidence": [round(c, 3) for c in conf]}diff --git a/scripts/sync.py b/scripts/sync.pyindex 10a3fc6..47a388e 100755--- a/scripts/sync.py+++ b/scripts/sync.py@@ -9,6 +9,14 @@ Python (coarse, 20 ms), then refined by direct correlation at 1 ms.Offset semantics: a positive offset means the SECOND input starts LATERthan the reference, i.e. `second` must be shifted earlier by that amount.+This aligns two AUDIO tracks to each other; it does not check or guarantee+lip sync (mouth movement matching the audio). It assumes each recording's+own audio is already correctly timed against its own picture, which holds+for ordinary cameras and phones (same device, same clock) but not for a+capture device with its own internal audio/video offset. There is no+face or mouth detection anywhere in this codebase to verify that; the only+way to confirm the final result actually looks in sync is to watch it.+Examples:python3 sync.py camera.mp4 lavmic.wav # print offset onlypython3 sync.py camera.mp4 lavmic.wav --replace-audio -o synced.mp4diff --git a/tests/test_all.py b/tests/test_all.pyindex b8cd59a..0caf6d8 100644--- a/tests/test_all.py+++ b/tests/test_all.py@@ -593,6 +593,18 @@ class FFmpegSkillTests(unittest.TestCase):self.assertClose(probe(str(auto))["duration"], 12.0, 0.2)script("multicam.py", self.src, camB, "--switch", "0-3:5", expect_fail=True)+ def test_multicam_warns_on_a_camera_with_no_shared_audio_event(self):+ """A camera whose audio has nothing in common with the reference must not align silently."""+ unrelated = OUT / "camC_unrelated.mp4"+ # a flat-envelope tone: nothing for the envelope-based cross-correlation to lock onto,+ # unlike the reference's gated tones -- unrelated in the way a different room's constant hum would be+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=640x360:rate=30",+ "-f", "lavfi", "-i", "sine=frequency=233:sample_rate=48000", "-t", "12", "-c:v", "libx264", "-preset", "veryfast", "-c:a", "aac", unrelated)+ proc = script("multicam.py", self.src, unrelated, "--offsets-only", "--json")+ data = json.loads(proc.stdout)+ self.assertLess(data["confidence"][1], 0.1)+ self.assertIn("low correlation confidence", proc.stderr)+def test_verify_kit_runs_on_real_world_fixtures(self):folder = OUT / "vfx"folder.mkdir(exist_ok=True)