Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
{
"iteration": 15,
"date": "2026-09-13",
"skill_version": "1.14.0",
"agent_model": "claude-sonnet (1 run per prompt, 12-13 in parallel, six batches)",
"ffmpeg": "6.1.1 (Ubuntu 24.04 apt)",
"runs": 76,
"prompts": "evals/agent_prompts_24.json (65) + evals/agent_prompts_exec.json (11)",
"repeats": 1,
"harness_note": "Same 76-prompt set and boilerplate as iteration 14, run against the released 1.14.0. Tool calls counted from tool_use entries in the agent transcripts. Focused independent grader on the 13 delivery-shaped runs plus 13 picture/language runs (every PNG opened); language and labels on all 76.",
"purpose": "Measure what 1.14.0's delivery templates change on the delivery prompts (route taken, number of encodes, platform check outcome) and keep the stable set at iteration-14 levels.",
"regex_grader (evals/grade_runs_24.py)": {
"65-set": {
"routing_act_prompts": "100% over 43",
"mean_commands": "4.0",
"refusal_honesty": "22/22",
"language_by_prompt": "all 17 languages 100% (ja 12/12, zh 3/3, ko 2/2, es 3/3, pt 2/2, fr 2/2, de 2/2, ar 2/2, th 2/2, hi 2/2, he 2/2, ru 2/2, el 2/2, vi 2/2, id 2/2, tr 2/2, it 2/2)",
"report_format": "65/65",
"visual_check_when_picture_changed": "18/18",
"audio_only_handled_as_audio": "4/4"
},
"exec-set": {
"routing_act_prompts": "100% over 10",
"mean_commands": "2.4",
"real_execution": "6/6",
"honest_failure_reporting": "5/5, false success 0",
"audio_only_handled_as_audio": "3/3"
},
"grader_changes_this_iteration": "Delivery and reel prompts now accept render.py as the check (a template or project run embeds check.py); refusal keywords 'none of', 'forbid', 'не переводит', 'не выполн' and Indonesian/Turkish stopwords added after four honest reports were missed (r05, ru2, id1, tr1), then re-graded."
},
"trigger_set": {
"file": "evals/trigger/results-2026-09-13-it15.json",
"judge": "claude-sonnet, one batch of 38",
"should_trigger": "22/22",
"should_not_trigger": "16/16"
},
"independent_grader": {
"file": "evals/results/iteration-15-independent-grades.json",
"model": "claude-opus, focused pass (26 runs fully: 13 delivery-shaped + 13 picture/language; language and labels on all 76; every PNG opened; check.py re-run on every delivery output)",
"summary": {
"runs": 76,
"runs_graded_fully": 26,
"routing": "26/26 - correct script for every request (render.py --template for delivery, caption.py for burn-in, graphics.py for lower-thirds, color.py --to-sdr, audio.py --replace, loudness.py+metadata.py for the podcast). No raw ffmpeg/ffprobe anywhere in any of the 76 run.md files.",
"honest": "26/26 - every produced file re-probes as reported; no Done: is contradicted by disk. dl8's prose is honest about the upload being impossible (its LABEL is the defect, counted under format).",
"language": "75/76 - every final report is in the prompt's language except dl4 (es prompt), whose Done:/Steps:/Check: lines are in English and only Look:/Notes: in Spanish.",
"format": "74/76 - dl8 carries 'Done:' on a refuse-the-verb prompt where 'Failed: (upload impossible)' is the correct label; he2 emits BOTH 'Done: (nothing rendered)' and 'Failed:' (double label, same defect k02 had in iteration 14). All 76 carry the five labels.",
"look_when_picture_changed": "23/23 - every run that changed the picture wrote and read a still or contact sheet; dl7, e11 and x05-replace-audio correctly say 'Look: not needed' for audio-only changes.",
"visual_check_non_latin": "10 of 11 non-Latin text runs render real, correctly shaped glyphs (th1 Thai marks, he1 Hebrew RTL, a01 Arabic joined forms + RTL, c01 Chinese, k01 Hangul, dl3 Japanese, j03 Japanese, vi1 Vietnamese diacritics, id1/fr1/p01 Latin). hi1 Devanagari is genuinely garbled - confirmed from the PNG and disclosed by the agent.",
"platform_checked_delivery": "13/13 delivery-shaped outputs pass check.py --platform X with 0 FAIL and 0 WARN when the grader re-runs it (iteration 14: 7 of 8).",
"encodes_delivery": "12 of 13 delivery runs took exactly one encoding pass (e11 took zero video encodes); only j06-report took two. Iteration 14: 4 of 7 producing delivery runs took two.",
"quality_mean": 4.58,
"quality_min": 3
}
},
"delivery_prompts (second measurement, first at 1.14.0)": {
"route": "12 of 13 delivery-shaped runs went through render.py --template (or a template-written project); j06 used export.py directly",
"encodes": "12/13 one encode (iteration 14: 3/7); j06 two (export.py without --normalize, then with)",
"platform_check": "13/13 pass check.py --platform X with 0 FAIL on the grader's own re-run (iteration 14: 7/8; dl6 LinkedIn's loudness FAIL is gone)",
"dl8": "upload refused honestly and the TikTok-ready file produced via the template, real 9:16 crop instead of iteration 14's letterbox; still labelled Done: (label rule queued for 1.15.0)",
"tokens_dl": {
"runs": 8,
"mean": 80368,
"median": 76013,
"min": 73179,
"max": 90441
},
"tokens_dl_iteration_14": {
"runs": 8,
"mean": 80108,
"median": 78622,
"min": 73076,
"max": 88680
}
},
"tokens_per_run (subagent_tokens)": {
"all_76 at 1.14.0": {
"runs": 76,
"mean": 73356,
"median": 72043,
"min": 65066,
"max": 97099
},
"all_76 at 1.13.0 (iteration 14)": {
"runs": 76,
"mean": 72724,
"median": 71754,
"min": 57519,
"max": 88680
},
"tool_calls_mean (tool_use entries)": 7.4,
"doctor_runs": 6,
"json_brief_runs": 31,
"template_runs": 15,
"per_prompt": {
"dl7": 73179,
"e11-ready": 71899,
"dl2": 73743,
"dl5": 74209,
"dl6": 76095,
"j06-report": 80728,
"e01-reel": 97099,
"j01-reel": 88718,
"j08-project": 84998,
"dl4": 90441,
"dl1": 90044,
"dl8": 75931,
"dl3": 89304,
"c02": 67716,
"a02": 67983,
"d01": 70039,
"a01": 73798,
"d02": 74062,
"e05-lufs": 70856,
"c01": 78034,
"e02-cut-lossless": 70929,
"e06-silence": 72821,
"e08-surround": 73204,
"e03-logo": 78468,
"e07-hdr": 74585,
"e04-sync": 80583,
"f01-missing-input": 72187,
"el1": 67942,
"el2": 65066,
"f02-corrupt-input": 67313,
"f04-beyond-duration": 67140,
"e12-vfr": 72401,
"f03-ffmpeg-failure": 69920,
"f05-unsupported": 67192,
"fr2": 66931,
"e10-highlights": 72422,
"e09-join": 78235,
"fr1": 76174,
"he1": 75319,
"he2": 69299,
"hi2": 67576,
"it1": 67901,
"it2": 67270,
"id2": 69497,
"id1": 73487,
"hi1": 75058,
"j04-bgm": 71040,
"j07-probe": 69544,
"j03-lower": 79331,
"j05-ma": 73831,
"j02-60s": 74425,
"k02": 68056,
"k01": 76868,
"r02-tts": 67668,
"r03-download": 67273,
"p02": 70981,
"r04-translate": 72389,
"p01": 73513,
"r05-no-raw-ffmpeg": 70676,
"s01": 67479,
"r01-face": 71690,
"s02": 68360,
"ru1": 70291,
"ru2": 71712,
"th2": 67932,
"tr1": 71508,
"vi2": 68063,
"x03-audio-trim": 68354,
"th1": 75581,
"x01-lufs-wav": 70176,
"x02-wav-to-mp3": 71599,
"x06-reverse": 70613,
"vi1": 81454,
"x04-transcode": 72652,
"x05-replace-audio": 70794,
"tr2": 73398
}
},
"comparison_with_iteration_14": {
"routing": "76/76 (same)",
"honest": "76/76 regex, 26/26 Opus (was 75/76: dl8 label still Done: but no longer counted as dishonest by the grader; the label rule ships in 1.15.0)",
"format": "76/76 regex, 74/76 Opus (he2 double label, dl4 mixed labels; was 75/76)",
"language": "76/76 regex, 75/76 Opus (dl4: three of five labels in English on a Spanish request)",
"look": "18/18 regex, 23/23 Opus",
"tokens_mean_all_76": "73,356 (was 72,724)",
"trigger": "38/38 (same)",
"quality_mean": "4.58 over 26 (was 4.65 over 26 different runs)",
"delivery_one_encode": "12/13 (was 3/7)",
"delivery_platform_pass": "13/13 (was 7/8)"
},
"findings": [
"1.14.0 did what it was built for: delivery requests route to render.py --template, finish in one encode, and pass their platform check, including the LinkedIn case that failed loudness at 1.13.0.",
"The remaining two-encode run (j06, X export with an HTML report) named a platform but did not use a template; the SKILL.md delivery row should say that a named platform means --normalize in the same export.",
"Stable 50 unchanged: routing, honesty, format, language, tokens flat (73.4k vs 72.7k).",
"Devanagari through graphics.py is still wrong-shaped at 1.14.0 (expected; 1.15.0 routes shaping scripts through libass); the agent disclosed it again.",
"Caption wrap defects seen in iteration 14 recur (orphan single characters in th1 and dl3, mid-phrase breaks in dl1 and dl4); queued in 1.15.0.",
"Labels: dl8 Done: on a refused upload, he2 both Done: and Failed:; the one-sentence label rule ships in 1.15.0.",
"dl4 mixed English and Spanish across the five labels: the only language drift in 76 runs.",
"Four runs invoked scripts via ~/.claude/skills/ffmpeg-skill (symlink to the repo; executed HEAD)."
],
"harness_or_model_issues": [
"One pass per prompt; the delivery encode count is now measured on 13 runs and is deterministic given the route, so repeats are not needed for that claim."
],
"follow_ups": [
"1.15.0 (in progress): label rule, caption wrap, Devanagari shaping, emoji.",
"SKILL.md delivery row: named platform means --normalize in the same export (fits the 1.15.0 byte budget only if a trim is found; otherwise 1.16.0).",
"evals: keep the 76 set; iteration 16 adds emoji/shaping prompts from the 1.15.0 spec."
]
}