Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

test: expand trigger set with 5 symptom-only 1.18 prompts (#262)

Shogo Kajiyama33a0a3e30271
diff --git a/docs/roadmap.md b/docs/roadmap.md
index 964d955..37a4bbf 100644
--- a/docs/roadmap.md
+++ b/docs/roadmap.md
@@ -17,7 +17,7 @@ minor and `fix` PRs into a patch, so each block below is one or two `feat` PRs p
- **planned** — not released. Nothing below a *planned* heading exists in any published version;
the feature lines are the intent, not a description of the code.
-The released version today is **1.19.3** — `multicam.py --write-project FILE` writes a
+The released version today is **1.20.0** — `multicam.py --write-project FILE` writes a
`render.py` project that reproduces the tool's own cut decision (eval 21's `mc2` gap, no more
hand-translating cuts/offsets into `clips[]`), and `--filler` alone no longer jump-cuts unrelated
dead-air silence (eval 19's `fw1`/`fw3`: "remove the ums" silently removed ~5s of unasked-for
@@ -27,7 +27,8 @@ start the "1.19.0: observability, portability" theme further down this document,
planned. 1.19.2 is docs-only (SKILL.md's overlong-word and MCP core-12 rows, this same truth-up).
1.19.3 extends 1.18.4's column-edge slice to `graphics.py`'s own `wrapped()` helper (lower-third,
title, sticker, hook and meme labels), so an unbreakable overlong word no longer clips off-frame
-on those templates either.
+on those templates either. 1.20.0 is `look.py --ink` (#261), a `feat:` release, hence the minor
+bump — not the start of a new theme.
| version | state | evidence |
|---|---|---|
@@ -56,6 +57,7 @@ on those templates either.
| 1.19.1 | shipped, eval pending | eval 19's `fw1`/`fw3`: `silence.py --filler` alone (no `--speech-aware`) jump-cut unrelated dead-air silence gaps too, ~5s unasked. Fixed: generic silence detection is skipped unless `--speech-aware` is also given, so `--filler` alone removes only the timed filler-word spans; `--speech-aware` alone and `--filler --speech-aware` combined are unchanged |
| 1.19.2 | shipped, docs-only | SKILL.md rows for 1.18.3's MCP core-12 (previously undocumented) and 1.18.4's overlong-word column-edge slice, plus this document's own roadmap truth-up. No script changes |
| 1.19.3 | shipped, eval pending | `graphics.py`'s `wrapped()` helper (used by the lower-third, title, sticker, hook and meme templates) turns on the same `slice_overlong` escape hatch `caption.py` turned on in 1.18.4, so a single unbreakable overlong atom drawn through a template no longer renders past the frame's safe width. `broken_inside_word` reported the same way as `caption.py`. A fitting Thai phrase or katakana run is unchanged, guarded by the same font-availability skip as `ShapingTests`. Tool count still 42, no new CLI flag for this change specifically -- `look.py --ink` is a separate, not-yet-versioned addition tracked in its own PR |
+| 1.20.0 | shipped, eval pending | `look.py --ink` (#261): measures the count and fraction of non-background pixels in a PNG (a still frame, a burned caption/graphics overlay, or a `--compare` diff), pixel-scan only -- no new dependency. `feat:` release, hence the minor bump. Tool count still 42 |
| 1.21.0, 2.0.0 | planned | — |
## 1.8.0 — one-call delivery, quieter checks, encoder flags (shipped + evaluated, eval 8)
diff --git a/evals/README.md b/evals/README.md
index bf5e542..e96e38d 100644
--- a/evals/README.md
+++ b/evals/README.md
@@ -2,7 +2,7 @@
Two kinds of eval live here:
-- **Trigger tests** (`trigger/`) — does a model pick this skill for a request, and leave it alone for a near miss? 50 prompts (45 plus the five 1.17 features), see `trigger/README.md`.
+- **Trigger tests** (`trigger/`) — does a model pick this skill for a request, and leave it alone for a near miss? 55 prompts (45 plus the five 1.17 features, plus five for 1.18), see `trigger/README.md`.
- **Agent runs** (`agent_prompts_*.json` + `grade_runs_24.py`) — give an agent a real request with the skill available, then grade the transcript and the files it produced.
`tasks.json` + `run.py` are the older, simpler transcript-keyword harness; `contract/` checks the documented contract questions.
diff --git a/evals/trigger/README.md b/evals/trigger/README.md
index 2209f6a..cf0b285 100644
--- a/evals/trigger/README.md
+++ b/evals/trigger/README.md
@@ -1,6 +1,6 @@
# Trigger tests: should / should-not
-`prompts.json` holds 26 requests the skill must trigger on (English, Japanese, Chinese, Korean, Spanish, Arabic, Thai, Hindi, Hebrew, Russian, Vietnamese and Turkish, some without a file extension or the word "edit") and 19 near-miss requests it must not (still images, installing ffmpeg — in English, Japanese, French and Greek, downloading from YouTube, writing a script in Japanese, Portuguese and Italian, an editor shortcut, thumbnail design, CSV to JSON, a codec explainer, AI video generation in English, German and Indonesian, meeting transcription, an emoji sticker pack as PNGs — image work, not video, chapter titles for a blog outline with no media, and designing a podcast cover, which is image design rather than an edit). 1.16 added three that must trigger: a podcast episode to post on YouTube, a Japanese request for chapter marks with no idea where to cut, and a Spanish request for switchable English and Spanish subtitle tracks. 1.17 added five more that must trigger, one per feature and none naming a tool or a flag: captions coming out in three lines with half the sentence late (en), cutting to the BGM's beat (ja), stripping the ums and uhs out of a podcast recording (en), 200 videos in a folder taking an eternity (es), and "I only changed the export preset, do I have to re-render the whole edit?" (en) — so the set is 31 should-trigger and 19 near-miss, 50 in all.
+`prompts.json` holds 26 requests the skill must trigger on (English, Japanese, Chinese, Korean, Spanish, Arabic, Thai, Hindi, Hebrew, Russian, Vietnamese and Turkish, some without a file extension or the word "edit") and 19 near-miss requests it must not (still images, installing ffmpeg — in English, Japanese, French and Greek, downloading from YouTube, writing a script in Japanese, Portuguese and Italian, an editor shortcut, thumbnail design, CSV to JSON, a codec explainer, AI video generation in English, German and Indonesian, meeting transcription, an emoji sticker pack as PNGs — image work, not video, chapter titles for a blog outline with no media, and designing a podcast cover, which is image design rather than an edit). 1.16 added three that must trigger: a podcast episode to post on YouTube, a Japanese request for chapter marks with no idea where to cut, and a Spanish request for switchable English and Spanish subtitle tracks. 1.17 added five more that must trigger, one per feature and none naming a tool or a flag: captions coming out in three lines with half the sentence late (en), cutting to the BGM's beat (ja), stripping the ums and uhs out of a podcast recording (en), 200 videos in a folder taking an eternity (es), and "I only changed the export preset, do I have to re-render the whole edit?" (en) — so the set was 31 should-trigger and 19 near-miss, 50 in all. 1.18/1.19 added five more that must trigger, one per 1.18.0 analysis/multicam feature and again none naming a tool or a flag: is the camera locked off or drifting (en, `cropdetect.py`'s motion signal), keep the breaths but cut the odd long silence (ja, `silence.py --speech-aware`), where's the motion so a 9:16 crop doesn't cut off the action (en, `cropdetect.py --motion-centre`), three cameras at once, always cut to whoever's loudest (es, `multicam.py --switch energy`), and three unsynced phone recordings of the same panel (en, `sync.py` with more than one extra source) — so the set is now 36 should-trigger and 19 near-miss, 55 in all.
Method: show an independent model a skill catalog made of this skill's `description` plus four decoy skills that own the neighbouring territory (image-tools, web-research, copywriter, meeting-notes), give it the raw request, and ask which one skill it would load or `none`. A run is correct when ffmpeg-skill is chosen exactly for the should-trigger prompts.
diff --git a/evals/trigger/prompts.json b/evals/trigger/prompts.json
index 83619ea..a5c9e7a 100644
--- a/evals/trigger/prompts.json
+++ b/evals/trigger/prompts.json
@@ -298,5 +298,35 @@
"lang": "de",
"prompt": "entwirf ein Podcast-Cover für Spotify",
"should_trigger": false
+ },
+ {
+ "id": "t51",
+ "lang": "en",
+ "prompt": "not sure if the camera in this clip is locked off or drifting a little, can you tell me? handheld_take3.mov",
+ "should_trigger": true
+ },
+ {
+ "id": "t52",
+ "lang": "ja",
+ "prompt": "このインタビュー、息継ぎの間は残したいけど、変な長い沈黙だけ削りたい。interview.mp4",
+ "should_trigger": true
+ },
+ {
+ "id": "t53",
+ "lang": "en",
+ "prompt": "where's most of the movement happening in this shot? I want to crop it to 9:16 without cutting off the action. skate_run.mp4",
+ "should_trigger": true
+ },
+ {
+ "id": "t54",
+ "lang": "es",
+ "prompt": "grabamos la entrevista con tres cámaras a la vez, ¿puedes montarlo cortando siempre a quien está hablando más fuerte? cam1.mp4 cam2.mp4 cam3.mp4",
+ "should_trigger": true
+ },
+ {
+ "id": "t55",
+ "lang": "en",
+ "prompt": "I've got three separate recordings of the same panel from different phones, they're not lined up at all, can you get them in sync? phoneA.mp4 phoneB.mp4 phoneC.mp4",
+ "should_trigger": true
}
]