Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

README: OSS landing page for 0.9.1, FFmpeg Skill brand assets, package metadata (#21)

Shogo Kajiyamad27c7762ab52
diff --git a/README.md b/README.md
index 6b967c7..119d4fe 100644
--- a/README.md
+++ b/README.md
@@ -1,177 +1,328 @@
-# ffmpeg-skill
+<p align="center">
+ <img src="assets/logo.png" alt="FFmpeg Skill: media processing for AI agents" width="760">
+</p>
-**Give your coding agent a video editor.** Local FFmpeg, Python standard library, nothing else.
+<h1 align="center">ffmpeg-skill</h1>
-![before / after demo](assets/demo.gif)
+<p align="center"><strong>Give your coding agent a video editor.</strong></p>
+
+<p align="center">
+ Local FFmpeg · No cloud · No API keys · Python standard library<br>
+ Claude Code · Cursor · Codex · MCP
+</p>
+
+<p align="center">
+ <a href="https://github.com/kajisho5/ffmpeg-skill/actions/workflows/ci.yml"><img src="https://github.com/kajisho5/ffmpeg-skill/actions/workflows/ci.yml/badge.svg" alt="tests"></a>
+ <a href="https://www.npmjs.com/package/ffmpeg-skill"><img src="https://img.shields.io/npm/v/ffmpeg-skill" alt="npm"></a>
+ <img src="https://img.shields.io/badge/python-3.9%2B-blue" alt="Python 3.9+">
+ <img src="https://img.shields.io/badge/ffmpeg-5.0%2B-orange" alt="FFmpeg 5.0+">
+ <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT"></a>
+</p>
```bash
npx ffmpeg-skill
```
-`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any other agent that reads `SKILL.md`. It teaches the agent a fixed editing workflow (probe → edit losslessly where possible → verify) and ships eight small CLI scripts that do the actual work with `ffmpeg`/`ffprobe`. Think of it as the fully local FFmpeg counterpart to cloud video-agent tools such as browser-use/video-use.
-
-**No API keys. No cloud. No dependencies.** If `ffmpeg` and `python3` are on your PATH, it works — offline, on any footage you'd rather not upload.
-
-## Features
-
-- **Probe first, verify last** — the skill forces the agent to read real duration/fps/resolution before editing and to check the result after, so you get "final.mp4: 59.98 s, 1080×1920, 30 fps" instead of guesses.
-- **Lossless when possible** — cuts and joins use stream copy by default; re-encoding only happens when it must (frame-accurate cuts, filters, format changes).
-- **Cut & join** segments with `mm:ss` / `hh:mm:ss.ms` times.
-- **Declarative edits** — describe the whole edit in a `project.json` (clips, transitions, captions, overlays, music, loudness, export, check) and re-render after every tweak.
-- **MCP server** — `mcp/server.py` exposes every script as an MCP tool over stdio (stdlib only) for Claude Desktop, Cursor or any MCP client; tool names, order and `inputSchema` are derived from the contract, so the MCP surface follows the scripts.
-- **Batch / watch folder** — one recipe over a whole shoot with a content-hash cache; re-runs only touch what changed.
-- **Optional local transcription** — `caption.py --transcribe` uses whisper.cpp / faster-whisper / openai-whisper when present; never required.
-- **Brand kit** — one `brand.json` (fonts, colours, logo, safe margins, caption style) applied by captions, overlays, graphics and projects.
-- **Motion graphics without assets** — lower-thirds, title cards, chapter chips, progress bars, countdowns and corner bugs drawn by FFmpeg.
-- **HTML delivery report** — before/after contact sheets, media facts, loudness, compliance and the commands run, in one file.
-- **Scene detection and highlight picks** — find cuts and loud moments, get a 60-second digest proposal as a cut list.
-- **Delivery checks** — PASS/FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure.
-- **Multicam** — align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list.
-- **Real-footage verification kit** — run the whole toolchain on your own device files and get a PASS/FAIL report.
-- **Silence removal / jump cuts** — detect dead air, keep a margin around speech, render frame-accurate in one pass; export the cut list for hand editing.
-- **Join with transitions** — crossfade, wipes, fade-to-black between mismatched clips (any size, fps, audio layout).
-- **Agent eyes** — contact sheets, single frames and before/after comparisons as PNG so the agent verifies caption placement, crops and colour visually.
-- **Plan before render** — every script has `--dry-run` (print the ffmpeg commands), `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress` (percent / ETA).
-- **Captions** — burn SRT/ASS with font, size, colour, outline and position control; generate SRT from a plain timed-text file; animated (fade/pop/slide) and word-by-word karaoke highlight timed to the speech energy in the audio.
-- **Fit** to an exact duration (pitch-preserving speed change or trim) and to 16:9 / 9:16 / 1:1 / 4:5 by padding or cropping; motion-interpolated or blended slow motion.
-- **Real-world footage handling** — variable-frame-rate phone clips are conformed to constant fps automatically, rotation metadata is honoured, 10-bit HEVC and 5.1 sources are handled.
-- **Multicam / external-audio sync** — offset detection by cross-correlation implemented in pure Python (no numpy), 1 ms resolution, plus clock-drift correction for long takes.
-- **Colour management** — HDR10 / HLG / Dolby Vision (iPhone) → SDR BT.709 tone mapping, Dolby Vision layer stripping, 3D LUT (.cube) for Log footage and looks, Log-footage detection, metadata-only retagging.
-- **Audio post** — voice clean-up chain (highpass, de-esser, FFT denoise, compressor), background music with sidechain ducking, fades, 5.1 → stereo downmix, track replacement.
-- **Loudness** — two-pass EBU R128 normalisation to −14 LUFS (or any target) with true-peak ceiling.
-- **Overlays** — logos, watermarks and titles with position, time range, opacity and fades.
-- **Export presets** — YouTube, Instagram Reels/Shorts/TikTok, X, ProRes 422 HQ master, H.265, GIF — all tagged BT.709.
-- **Agent-friendly CLI** — every script has `--help`, prints the output path on stdout, exits non-zero with a reason on stderr, and names outputs `<input>_<operation>.<ext>` by default.
+![before / after demo](assets/demo.gif)
-## Install
+`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **21 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
+
+If `ffmpeg` and `python3` are on your PATH, it works: offline, on footage you would rather not upload.
+
+---
+
+**Contents**
+[Why](#why) · [Quick start](#quick-start) · [How it works](#how-it-works) · [Design principles](#design-principles) · [Tools](#tools) · [Audio](#audio-is-a-first-class-input) · [Built for agents](#built-for-agents) · [FFmpeg compatibility](#ffmpeg-compatibility) · [Tested on real footage](#tested-on-real-footage) · [Install](#install) · [Requirements](#requirements) · [Development](#development) · [Docs](#docs)
+
+---
+
+## Why
+
+An agent that "knows FFmpeg" still guesses: it assumes a frame rate, picks a codec the container cannot hold, re-encodes a file that only needed a stream copy, and reports "done" without opening the result. ffmpeg-skill exists to take the guessing out:
+
+- **Real files first.** Every job starts with `probe.py`; the agent decides from the measured duration, fps, resolution, colour and audio layout, not from the file name.
+- **Structured tools, not shell strings.** Each operation is a script with typed arguments. Nothing runs through a shell; no filter graph is accepted from the caller.
+- **A contract the agent can read.** `contract --json` states, for every tool, what it takes, what it writes, which FFmpeg components it needs and how the result is verified. The MCP surface is derived from it.
+- **Verification after execution.** The result is probed, checked against the destination's spec and, when the picture changed, looked at as a contact sheet.
+- **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required.
+
+## Quick start
```bash
-# Claude Code (default) → ~/.claude/skills/ffmpeg-skill
+# 1. install the skill for Claude Code (Cursor: --cursor, Codex: --codex, all three: --all)
npx ffmpeg-skill
-# Cursor → ~/.cursor/skills/ffmpeg-skill
-npx ffmpeg-skill --cursor
+# 2. check the machine: ffmpeg, ffprobe and every FFmpeg component the tools need
+npx ffmpeg-skill doctor
-# Codex → ~/.codex/skills/ffmpeg-skill
-npx ffmpeg-skill --codex
+# 3. (for agent frameworks) read the machine-readable contract
+npx ffmpeg-skill contract --json | head -40
+```
-# everything, or a project-local copy, or a custom directory
-npx ffmpeg-skill --all
-npx ffmpeg-skill --project
-npx ffmpeg-skill --dir ./my-skills
+Then talk to your agent:
+
+> "Take `interview.mp4`, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."
+
+The agent runs `probe.py`, `cut.py --segments 0:45-3:10,5:00-6:30`, `fit.py --duration 60 --aspect 9:16 --fit crop`, `export.py --preset reels`, `check.py --platform reels` and `look.py`, then reports "final.mp4: 59.98 s, 1080×1920, 30 fps, AAC stereo" with the contact sheet it inspected.
+
+The tools also work on their own, from any shell:
+
+```bash
+S=~/.claude/skills/ffmpeg-skill/scripts
+python3 $S/probe.py input.mp4 --compact
+python3 $S/fit.py input.mp4 --duration 60 --aspect 9:16 --dry-run # print the plan, run nothing
+python3 $S/export.py input.mp4 --preset reels --json # structured result with a probe of the output
```
-Or without Node: clone this repo and copy `SKILL.md` and `scripts/` into your agent's skills directory.
+More requests and the commands behind them: [examples/README.md](examples/README.md). To see everything run end-to-end on generated footage: `npm run demo`.
+
+## How it works
+
+```mermaid
+flowchart TD
+ U[User request] --> A[AI agent<br/>Claude Code · Cursor · Codex]
+ A -->|reads| S[SKILL.md<br/>workflow, request → tool map, report format]
+ A -->|runs| T[Structured tool<br/>scripts/&lt;name&gt;.py, typed argparse flags]
+ T --> C[Contract<br/>input schema · role · capabilities · verification policy]
+ C --> D[Capability detection<br/>doctor: available / missing / unknown]
+ D --> F[FFmpeg execution<br/>no shell, stream copy when possible]
+ F --> V[Verification<br/>probe · check · look.py contact sheet]
+ V --> R[Structured result<br/>--json: status, output, commands, probe]
+ R --> A
+```
-You also need FFmpeg:
+Over MCP the same tools are reached through a transport that holds no tool table of its own:
-| OS | Command |
-|----|---------|
-| macOS | `brew install ffmpeg` |
-| Ubuntu / Debian | `sudo apt install ffmpeg` |
-| Windows | `winget install Gyan.FFmpeg` |
+```mermaid
+flowchart LR
+ M[MCP client<br/>Claude Desktop · Cursor · any client] --> P[mcp/server.py<br/>stdio JSON-RPC]
+ P -->|tools/list| C[Contract-derived ToolSpecs<br/>names · order · inputSchema]
+ P -->|tools/call| T[scripts/&lt;name&gt;.py]
+ C -.derived from.-> K[scripts/_contract.py]
+ T -.described by.-> K
+```
+
+Names, order and `inputSchema` in `tools/list` are translated from each tool's argparse parser at start-up, so a new flag or a new script appears in MCP with no edit to `mcp/`. A test copies the skill, adds, removes and edits a script, and reads `tools/list` again to prove it.
+
+## Design principles
+
+These are the rules the skill file gives the agent and the code enforces. Together they are what separates this from a list of FFmpeg one-liners.
+
+1. **Probe first.** No tool decides from the file name. `probe.py` measures duration, fps (with variable-frame-rate detection), resolution, rotation, bit depth, HDR format including Dolby Vision, colour tags and every audio stream before anything is cut.
+2. **Lossless when possible.** `cut.py`, `join.py` and `loudness.py` stream-copy what they do not need to touch. Re-encoding happens only when it must: frame-accurate cuts, filters, format changes, or a keyframe farther than the tolerance.
+3. **Plan before render.** Every tool takes `--dry-run` (print the ffmpeg command lines, write nothing), `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress` (percent and ETA). A test runs every tool under `--dry-run` behind a fake ffmpeg and asserts that no ffmpeg call happened and no file appeared.
+4. **Machine-readable contract.** `contract --json` describes all 21 tools: input schema generated from the parser, output schema, role, required and conditional FFmpeg capabilities, dry-run support, the verification tools to run afterwards, whether a visual check is required, `mutates_input: false`.
+5. **Contract-derived MCP.** `mcp/server.py` builds its `tools/list` from the contract. Tool names, order and `inputSchema` cannot drift from the scripts; a test keeps the two byte-identical.
+6. **Capability detection.** `doctor` reads `ffmpeg -encoders / -filters / -bsfs` and reports which of the components the tools need are present on this build (libx264, libass, zscale, loudnorm, xfade, …), before a job fails inside ffmpeg.
+7. **Unknown is not missing.** When a listing cannot be read (a layout the parser does not know, ffmpeg exiting non-zero) the affected capabilities are `unknown`: never `missing`, never silently `available`. An installed filter is not reported absent; a failed detection is not a pass.
+8. **Verify the result.** The output is probed, and when the picture changed (captions, overlays, crops, colour, transitions) the agent runs `look.py` and inspects the PNG. The report is not finished until its `Look:` line names that image; audio-only jobs say `Look: not needed`.
+9. **Keep originals.** No tool overwrites its input. Outputs are new files named `<input>_<operation>.<ext>` unless told otherwise, and a test hashes every input after the run.
+
+## Tools
+
+21 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, non-zero exit and a reason on stderr on failure.
+
+**Analysis and inspection**
+
+| Tool | What it does |
+|---|---|
+| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision, colour space, rotation, every audio stream; `--analyze` flags Log footage |
+| `scenes.py` | Scene changes, audio peaks, highlight proposals and a per-scene sheet; cut list for `cut.py --segments` |
+| `look.py` | Contact sheet, single frames, side-by-side comparison as PNG so the agent can see what it made |
+
+**Editing**
+
+| Tool | What it does |
+|---|---|
+| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision` |
+| `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio; audio-only inputs are joined as audio |
+| `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list |
+| `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop); force constant fps |
+
+**Audio**
+
+| Tool | What it does |
+|---|---|
+| `audio.py` | Voice clean-up chain, FFT denoise, typed compressor / limiter / gate, music bed with sidechain ducking, fades, 5.1 → stereo, track replacement, extraction (`-o out.wav`), `--audio-stream N` |
+| `sync.py` | Offset between two recordings by audio cross-correlation (1 ms, pure Python), clock-drift correction; aligned video or audio out |
+| `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP or any target, video stream-copied; `--measure-only` |
+
+**Picture**
+
+| Tool | What it does |
+|---|---|
+| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; animated and word-by-word karaoke timed to the speech energy; optional local transcription |
+| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades |
+| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs drawn by FFmpeg from a brand kit |
+| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting |
-## Usage
+**Delivery**
-Once installed, just talk to your agent. Five things you can say to Claude Code:
+| Tool | What it does |
+|---|---|
+| `export.py` | Presets `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif`, all tagged BT.709 |
+| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure and a `format` / `judgement` kind per row |
+| `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run |
+
+**Orchestration**
+
+| Tool | What it does |
+|---|---|
+| `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays, music, loudness, export, check); `--init`, `--dry-run`, `--stop-after` |
+| `batch.py` | Apply a step recipe or a project to a folder with a content-hash cache; `--watch` |
+| `multicam.py` | Align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list |
+| `verify.py` | Run the toolchain on real device files and report PASS / FAIL per step |
+
+Not tools, but part of the surface: `mcp/server.py` (the MCP transport) and `scripts/_contract.py` (`contract --json`, `doctor`). Per-flag reference for every tool: [references/scripts.md](references/scripts.md).
+
+## Audio is a first-class input
-1. **"Take `interview.mp4`, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."**
- → `probe.py` → `cut.py --segments 0:45-3:10,5:00-6:30` → `fit.py --duration 60 --aspect 9:16 --fit crop` → `export.py --preset reels` → `probe.py` to confirm 60.0 s at 1080×1920.
-2. **"Burn these captions in TikTok style, words popping in with a yellow highlight, in Japanese."**
- → `caption.py --text cues.txt --font "Noto Sans CJK JP" --animate pop --karaoke --highlight-color FFD200`.
-3. **"The lav mic recording is out of sync with the camera and drifts over the hour — fix it, clean up the hiss and normalise to −14 LUFS."**
- → `sync.py camera.mp4 lav.wav --fix-drift --replace-audio` → `audio.py --voice` → `loudness.py` → report the detected offset, drift ppm and final LUFS.
-4. **"Put our logo in the top-right corner for the whole video at 80% opacity, and a title card for the first 4 seconds."**
- → `overlay.py --image logo.png --position top-right --scale 220 --opacity 0.8` → `overlay.py --text "…" --start 0 --end 4 --fade 0.4`.
-5. **"This iPhone HDR clip looks washed out on YouTube — fix it and give me a ProRes master too."**
- → `probe.py` (shows `hdr: true`) → `color.py --to-sdr` → `export.py --preset youtube` and `export.py --preset prores`.
+WAV, FLAC, MP3, M4A/AAC, OGG and Opus go through `probe`, `cut`, `join`, `silence`, `loudness`, `audio`, `sync` and `check --platform podcast` with the same commands as video. The output extension picks the codec: `-o out.wav` writes PCM, `-o out.flac` FLAC, `-o out.mp3` MP3, `-o out.m4a` AAC.
-The scripts also work on their own:
+- **Extraction.** An audio extension on a video input drops the picture: `audio.py talk.mp4 -o talk.wav`, or `--voice -o talk.m4a` to clean it on the way. `--audio-stream N` picks a track; `probe` lists them under `audio_streams`.
+- **Join.** `join.py intro.wav episode.m4a outro.wav -o full.flac` resamples every clip to one rate and channel layout and crossfades them (`--transition none` for a butt join). Audio and video inputs cannot be mixed in one join.
+- **Sample-accurate trims.** `cut.py talk.wav --start 1.2345 --end 2.3456 --accurate` trims at the sample; the JSON reports `precision` (`packet` for a stream copy, `sample` for PCM / FLAC, `codec_frame` when a lossy encoder frames the audio again, `frame` for video) and the measured `duration_error_ms`. A `.wav` never receives compressed packets.
+- **Typed dynamics.** `audio.py --compress --comp-threshold -20 --comp-ratio 4`, `--limit --limit-ceiling -1`, `--gate --gate-threshold -45`. Each flag is one documented option of FFmpeg's `acompressor`, `alimiter` or `agate`, range-checked before ffmpeg runs; no filter string is accepted from the caller.
+- **Loudness.** `loudness.py talk.wav -I -16 --tp -1.5 -o talk.m4a` for podcast levels; `check.py talk.m4a --platform podcast` measures LUFS and true peak.
+
+Picture tools (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" instead of inventing a picture.
+
+## Built for agents
+
+### Machine-readable contract
```bash
-python3 ~/.claude/skills/ffmpeg-skill/scripts/probe.py input.mp4 --compact
-python3 ~/.claude/skills/ffmpeg-skill/scripts/fit.py input.mp4 --duration 60 --aspect 9:16
+npx ffmpeg-skill contract --json # or: python3 scripts/_contract.py --json
+npx ffmpeg-skill contract --json --static # without environment detection
```
-More examples: [examples/README.md](examples/README.md). To see everything run end-to-end on generated footage: `bash examples/make_demo.sh`.
-
-## Scripts
-
-| Script | What it does |
-|--------|--------------|
-| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision, colour space, rotation, audio channels as JSON; `--analyze` flags Log footage |
-| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact |
-| `render.py` | Render a whole edit from `project.json`; `--init`, `--dry-run`, `--stop-after` |
-| `batch.py` | Apply a step recipe or render project to a folder, cached, optional watch |
-| `mcp/server.py` | MCP server exposing all scripts as tools (stdio JSON-RPC) |
-| `graphics.py` | Lower-third, title, chapter, progress, countdown, bug templates (brand colours) |
-| `report.py` | Single-file HTML delivery report with sheets, facts, loudness, compliance, commands |
-| `scenes.py` | Scene changes, audio peaks, highlight proposals and per-scene sheet |
-| `check.py` | Pre-delivery compliance per platform (duration, aspect, codec, colour, loudness, size) |
-| `multicam.py` | Align cameras/recorders by audio and switch between them from a time list |
-| `verify.py` | Run the toolchain on real device files and report PASS/FAIL per step |
-| `silence.py` | Detect and remove silences (jump cuts), list or export the cut list |
-| `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio |
-| `look.py` | Contact sheet, single frames, side-by-side comparison as PNG for visual checks |
-| `caption.py` | Burn SRT/ASS (font, size, colour, outline, position); build SRT from timed plain text; animated + karaoke ASS |
-| `fit.py` | Fit to a duration (speed or trim, smooth slow-mo) and/or aspect ratio (pad or crop), force constant fps |
-| `sync.py` | Detect offset between two recordings by audio cross-correlation (1 ms), correct clock drift; output aligned video/audio |
-| `color.py` | HDR10/HLG/Dolby Vision → SDR tone mapping, DV layer stripping, 3D LUT application, colour-tag rewriting |
-| `audio.py` | Denoise / voice chain, music bed with auto-ducking, fades, downmix, replace track |
-| `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP (or custom), video stream-copied |
-| `overlay.py` | Composite image/logo or drawtext title with position, time range, opacity, fade |
-| `export.py` | Presets: `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif` |
-
-All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stderr message on failure.
-
-## Measured, not assumed
-
-`tests/corpus.py` downloads public real-device footage (GoPro, DJI, iPhone incl. Dolby Vision, Android screen recordings, HDR10, 24p, Tears of Steel) and runs the toolchain on it; `tests/bench_sync.py`, `bench_silence.py` and `bench_scenes.py` score the algorithms against known ground truth.
-
-| What | Result (0.8.0, local ffmpeg 6.1) |
+The contract is generated from the code that runs, not maintained beside it. For each of the 21 tools (`ffmpeg-skill/<name>`) it states:
+
+| Field | Meaning |
|---|---|
-| Real-device corpus, 10 files | 92 verify steps, all pass after fixes |
-| sync.py, ±30 s offsets, gain/noise/EQ, real dialogue+music | 120 s windows (the documented rule): 40/40 within 10 ms, max 1.1 ms. 60 s stress windows: 95 % within 10 ms, 4 of 5 misses flagged by confidence |
-| silence.py, 20 cases, known gaps | 0 missed gaps, ≤ 1 ms leftover silence |
-| scenes.py, 53 hard cuts between single takes (GoPro/DJI/iPhone/…) | precision 0.95, recall 1.00, F1 0.97 at the default threshold |
+| `input_schema` | generated from the tool's argparse parser: properties, types, enums, defaults, required, positional order, mutually exclusive groups |
+| `output_schema` | what `--json` prints: `status`, `output`, `commands`, `probe`, plus tool-specific fields (`precision`, `checks`, `offset_seconds`, …) |
+| `role` | `analysis`, `analysis_and_execution`, `execution` or `verification` |
+| `capabilities` | the FFmpeg encoders, filters and bitstream filters the tool always needs, and the ones needed only for a flag or input |
+| `supports_dry_run`, `supports_json` | measured by the tests, not declared |
+| `verification` | which tools to run on the output afterwards (`probe`, `check`, `look`) |
+| `requires_visual_verification` | the picture changed; inspect the contact sheet |
+| `audio_only`, `video_required` | whether an audio-only input is accepted or refused |
+| `mutates_input` | always `false` |
+| `idempotency_hint` | `bit_exact`, `content_equivalent`, `cached` or `environment_dependent` |
+
+`contract_version` (1.0) is separate from the skill version, so a consumer can pin the shape and read the version for provenance. The document also states the invocation mapping (structured arguments → argv), the JSON shapes for success and failure (`{"status": "failed", "error": {"kind": "input | ffmpeg | missing_tool", "message": …}}`), and that no tool runs a shell or executes anything other than the named script, `ffmpeg` and `ffprobe`. Field-by-field reference: [docs/contract.md](docs/contract.md).
+
+### MCP
+
+```json
+{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/Users/you/.claude/skills/ffmpeg-skill/mcp/server.py"]}}}
+```
+
+`mcp/server.py` is a stdio JSON-RPC transport with no tool table of its own. `tools/list` is derived from the contract at start-up: the same 21 names, the same order, and `inputSchema` translated from each tool's `input_schema`. `tools/call` maps structured arguments to argv and runs the named script; a raw `argv` form is accepted for compatibility and marked non-canonical. `python3 mcp/server.py --list` prints the tools; `--call probe '{"inputs": ["a.mp4"]}'` runs one from the shell.
+
+### Capability detection
+
+```bash
+npx ffmpeg-skill doctor # human-readable
+npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors
+```
+
+`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`.
+
+## FFmpeg compatibility
+
+The tools need FFmpeg 5.0 or later. The capability parser has been run against the listings of these builds:
+
+| FFmpeg | `-filters` row layout | Source |
+|---|---|---|
+| 6.1.1 | three flag characters: `..C acompressor A->A` | Ubuntu 24.04 apt, captured |
+| 7.x | same as 6.x | constructed fixture (no capture at hand) |
+| 8.1.2 | two flag characters: `TS aap AA->A`, three-character legend, `------` separator | Homebrew on the macOS CI runner, captured |
+| 9.0.1 | same as 8.x, CRLF | gyan.dev build on the Windows CI runner, captured |
+
+FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on the old width matches nothing on FFmpeg 8 and, if "nothing matched" is read as "nothing installed", reports every filter missing; that is what 0.9.0 did on macOS and Windows. Since 0.9.1 rows are recognised by their io-spec token (`A->A`, `AA->A`, `|->V`, `N->N`), so the flag width, the legend and the separator do not matter, and a listing that still cannot be read yields `unknown` rather than `missing`. The captured listings live in [tests/fixtures/](tests/fixtures/README.md) with their provenance; CI uploads each runner's listing and `doctor --json` as an artifact so a new layout is visible before it bites.
+
+## Tested on real footage
+
+| Result | Measurement |
+|---|---|
+| **92 / 92** | verification steps on a 10-file real-device corpus (GoPro, DJI, iPhone incl. Dolby Vision, Android screen recordings, HDR10, 24p, Tears of Steel), 0.8.0, local ffmpeg 6.1 |
+| **40 / 40 within 10 ms** | `sync.py` offset detection, ±30 s offsets with gain, noise and EQ changes on real dialogue and music, 120 s windows (max error 1.1 ms); 60 s stress windows 95 % within 10 ms, 4 of 5 misses flagged by confidence |
+| **0 missed gaps** | `silence.py`, 20 cases with known gaps, ≤ 1 ms leftover silence |
+| **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
+| **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
+| **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
+| **6 / 6** | 0.9.1 audio evals (audio join, extraction, track selection, sample-accurate trim, typed dynamics; 2 in Japanese): routing, report format and audio-as-audio handling 6/6 |
```bash
python3 tests/corpus.py --fetch --verify # ~1.4 GB download, then verify (slow on 4K)
python3 tests/bench_sync.py --cases 100
+python3 tests/bench_silence.py
+python3 tests/bench_scenes.py
```
-## Machine-readable contract
+Benchmarks live in `tests/bench_*.py`, agent evals in [evals/](evals/), results by iteration in `evals/results/`.
+
+## Install
```bash
-npx ffmpeg-skill contract --json # every tool as a ToolSpec: schema from argparse, role, capabilities, verification, dry-run
-npx ffmpeg-skill doctor # which required ffmpeg components this machine has
+npx ffmpeg-skill # Claude Code → ~/.claude/skills/ffmpeg-skill
+npx ffmpeg-skill --cursor # Cursor → ~/.cursor/skills/ffmpeg-skill
+npx ffmpeg-skill --codex # Codex → ~/.codex/skills/ffmpeg-skill
+npx ffmpeg-skill --all # all three
+npx ffmpeg-skill --project # this project → ./.claude/skills/ffmpeg-skill
+npx ffmpeg-skill --dir ./my-skills
+npx ffmpeg-skill --uninstall # remove from the selected targets
```
-For agent frameworks that treat ffmpeg-skill as an execution skill: `contract --json` lists the 21 tools as `ffmpeg-skill/<name>` with input/output schemas, `analysis` / `execution` / `verification` roles, the ffmpeg encoders and filters each one needs, whether the result must be probed, checked or looked at, and that no tool modifies its input or runs a shell. `contract_version` (1.0) is separate from the skill version. Details in [docs/contract.md](docs/contract.md).
+Without Node: clone this repository and copy `SKILL.md`, `scripts/`, `references/` and `mcp/` into your agent's skills directory.
-## MCP
+After installing:
-```json
-{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/Users/you/.claude/skills/ffmpeg-skill/mcp/server.py"]}}}
+```bash
+npx ffmpeg-skill doctor # every required FFmpeg component present?
+npx ffmpeg-skill contract --json # what the agent framework will see
```
-`python3 mcp/server.py --list` prints the tools (the same set, order and schemas as `contract --json`: the contract is the source of truth, MCP is the transport); `--call probe '{"inputs": ["a.mp4"]}'` runs one from the shell.
+FFmpeg itself:
+
+| OS | Command |
+|----|---------|
+| macOS | `brew install ffmpeg` |
+| Ubuntu / Debian | `sudo apt install ffmpeg` |
+| Windows | `winget install Gyan.FFmpeg` |
## Requirements
-- FFmpeg 5.0+ with `libx264`, `libx265`, `libass`, `prores_ks` and `libzimg` (for `color.py --to-sdr`); the default builds from Homebrew, apt and gyan.dev include all of them
-- Python 3.9+
+- FFmpeg 5.0+. Always required: `libx264`, `aac`, and the `drawtext`, `subtitles` (libass), `loudnorm`, `xfade`, `acrossfade`, `scdet`, `silencedetect` and `tile` filters. Needed only by the flags that use them: `libx265`, `prores_ks`, `libzimg` / `zscale`, `libmp3lame`, `libopus`, `libvorbis`, the `ass` filter. `doctor` tells you which are present. The apt and gyan.dev builds carry all of them; some Homebrew bottles lack `libass` / `libfreetype` / `libzimg`, which `doctor` reports as missing.
+- Python 3.9+, standard library only
- Node 16+ only for the `npx` installer
## Development
```bash
-bash examples/make_demo.sh # generates footage, runs every script, rebuilds assets/demo.gif
-python3 tests/test_all.py # end-to-end tests incl. VFR, rotated, 5.1, 10-bit HDR10 and drifting sources (needs ffmpeg)
-python3 evals/run.py --list # routing eval prompts (see evals/)
+npm test # tests/test_all.py (end-to-end incl. VFR, rotated, 5.1, HDR10, drifting sources) + tests/test_contract.py
+npm run release-check # pack, install, contract from the installed copy, MCP == contract, doctor, tests, contract evals
+npm run demo # generate footage, run every tool, rebuild assets/demo.gif
+python3 evals/run.py --list # agent eval prompts (see evals/)
node bin/install.js --dir /tmp/skills # try the installer without touching ~/.claude
```
+CI (`.github/workflows/ci.yml`) runs on every pull request and on pushes to `main`, on Ubuntu (FFmpeg 6.1), macOS (Homebrew FFmpeg 8.x) and Windows (gyan.dev FFmpeg 9.x), and uploads each runner's FFmpeg listings as an artifact.
+
+## Docs
+
+| | |
+|---|---|
+| [SKILL.md](SKILL.md) | what the agent reads: workflow, request → tool map, audio-only rules, report format, pitfalls |
+| [references/scripts.md](references/scripts.md) | per-flag reference for every tool |
+| [references/devices.md](references/devices.md) | real-device notes (iPhone HDR, GoPro, DJI, screen recordings) |
+| [docs/contract.md](docs/contract.md) | the execution contract field by field, MCP relationship, how a planner consumes it |
+| [examples/README.md](examples/README.md) | natural-language requests and the commands behind them, `brand.json`, `project.json`, batch recipes |
+| [tests/fixtures/README.md](tests/fixtures/README.md) | captured and constructed FFmpeg listings, which is which |
+| [CHANGELOG.md](CHANGELOG.md) | what changed in each release |
+
## Support
If this skill saves you time, you can help keep it maintained through [GitHub Sponsors](https://github.com/sponsors/kajisho5). Issues and pull requests are just as welcome.
diff --git a/assets/icon.png b/assets/icon.png
new file mode 100644
index 0000000..84a90ea
Binary files /dev/null and b/assets/icon.png differ
diff --git a/assets/logo.png b/assets/logo.png
new file mode 100644
index 0000000..3027bcd
Binary files /dev/null and b/assets/logo.png differ
diff --git a/package.json b/package.json
index b2fbc3c..d940bf8 100644
--- a/package.json
+++ b/package.json
@@ -1,8 +1,20 @@
{
"name": "ffmpeg-skill",
"version": "0.9.1",
- "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: MCP server, batch processing, declarative project rendering, brand kits, motion-graphics templates, HTML delivery reports, scene detection, delivery checks, cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
- "keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],
+ "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 21 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
+ "keywords": [
+ "ffmpeg",
+ "video",
+ "video-editing",
+ "video-processing",
+ "audio",
+ "agent-skill",
+ "claude-code",
+ "cursor",
+ "codex",
+ "mcp",
+ "skill"
+ ],
"license": "MIT",
"author": "kajisho5",
"repository": {