Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
{
"iteration": 14,
"date": "2026-09-13",
"skill_version": "1.13.0",
"agent_model": "claude-sonnet (1 run per prompt, 12-13 in parallel, six batches)",
"ffmpeg": "6.1.1 (Ubuntu 24.04 apt)",
"runs": 76,
"prompts": "evals/agent_prompts_24.json (65) + evals/agent_prompts_exec.json (11)",
"repeats": 1,
"harness_note": "First iteration on the 76-prompt set: the 50 of iteration 13 plus 18 language prompts (act + refusal in th, hi, he, ru, el, vi, id, tr, it) and 8 delivery prompts (dl1-dl8: TikTok/Reels/Shorts/LinkedIn/Douyin/podcast/'post it for me'). Fixtures written into OUTDIR by evals/write_fixtures.py. Same prompt boilerplate as iterations 12-13. Tool calls counted from tool_use entries in the agent transcripts. Focused independent grader on the 26 new runs (every PNG opened), language and labels on all 76.",
"purpose": "Verify 1.13.0 did not move the stable 50, measure the nine new languages end to end (font choice, glyph shaping, report language), and get a first baseline on delivery requests before 1.14.0 ships templates.",
"regex_grader (evals/grade_runs_24.py)": {
"65-set": {
"routing_act_prompts": "100% over 43",
"mean_commands": "4.5",
"refusal_honesty": "22/22",
"language_by_prompt": "ja 12/12, zh 3/3, ko 2/2, es 3/3, pt 2/2, fr 2/2, de 2/2, ar 2/2, th 2/2, hi 2/2, he 2/2, ru 2/2, el 2/2, vi 2/2, id 2/2, tr 2/2, it 2/2",
"report_format": "65/65",
"visual_check_when_picture_changed": "18/18",
"audio_only_handled_as_audio": "4/4"
},
"exec-set": {
"routing_act_prompts": "100% over 10",
"mean_commands": "2.7",
"real_execution": "6/6",
"honest_failure_reporting": "5/5, false success 0",
"audio_only_handled_as_audio": "3/3"
},
"grader_changes_this_iteration": "Three honest refusals (r04 es 'no se puede', ru2 'не выполняют / нет инструмента', tr2 'kapsamı dışında') and one terse Italian report (it1) were missed by the keyword and stopword lists; lists extended in this PR and the runs re-graded. dl7 expectation widened to render|loudness|export (a podcast has no picture to fit or export)."
},
"trigger_set": {
"file": "evals/trigger/results-2026-09-13-it14.json",
"judge": "claude-sonnet, one batch of 38 (22 + 16 near-miss)",
"should_trigger": "22/22",
"should_not_trigger": "16/16",
"note": "first run on the 38-query set; the nine new languages route on file names and the verbs the description lists"
},
"independent_grader": {
"file": "evals/results/iteration-14-independent-grades.json",
"model": "claude-opus, focused pass (26 runs fully: 18 language + 8 delivery; language and labels on all 76; every PNG opened)",
"summary": {
"runs": 76,
"runs_graded_fully": 26,
"routing": "26/26 (graded set) - right scripts for every request, no raw ffmpeg anywhere",
"honest": "25/26 (graded set) - every produced file re-probes exactly as reported; dl8 is the one blemish (honest prose, wrong label: 'Done:' on a refuse-prompt)",
"language": "76/76 (all runs: 9 new languages th/hi/he/ru/el/vi/id/tr/it plus ja/zh/ko/es/pt/fr/de/ar/en - every final report in the prompt's language, no drift)",
"format": "75/76 (all five labels present everywhere; k02 emits BOTH 'Done: (해당 없음)' and 'Failed:' - a double label)",
"look_when_picture_changed": "14/14 (graded set: th1 hi1 he1 vi1 id1 he2 dl1-dl6 dl8; ru1 tr1 el1 it1 dl7 audio-only or audio-only change, correctly 'not needed'; refusals correctly 'not needed')",
"visual_check_non_latin": "4/5 non-Latin text runs rendered real, correctly shaped glyphs (th1 Thai OK, he1 Hebrew RTL OK, vi1 Vietnamese diacritics OK, id1 Latin OK, dl3 Japanese OK) - hi1 Devanagari is genuinely garbled, confirmed from the image",
"platform_checked_delivery": "7/8 delivery runs reached a check.py PASS (dl6 left 2 loudness FAILs, honestly disclosed and defensible for LinkedIn)",
"quality_mean": 4.65,
"quality_min": 3
},
"glyphs": "th1 Thai, he1 Hebrew (RTL), vi1 Vietnamese diacritics, id1 Latin and every ja/zh/ko/ar run rendered real glyphs with the font the run.md names. hi1 (Hindi lower-third) is the one failure: drawtext does not shape Devanagari, so the frame reads 'प्रयाशर्म / नरि्देशक' for 'प्रिया शर्मा / निर्देशक' (i-matra not reordered, final aa-matra dropped). graphics.py reported 'FreeSans.ttf (covers hi)', so the coverage gate cannot see a shaping failure. The agent noticed and disclosed it. caption.py goes through libass and is unaffected."
},
"delivery_prompts (dl1-dl8, first baseline)": {
"reached_platform_pass": "7/8 (dl6 LinkedIn 1:1 left loudness/true peak FAIL, disclosed)",
"encodes": "dl2 and dl7 one encode; dl1, dl3, dl4, dl5 exported without --normalize, hit the loudness warning and re-encoded (two full encodes each)",
"refusal": "dl8 ('post this to TikTok') refused the upload correctly and produced the TikTok-ready file, but labelled the report Done: rather than Failed:; its file is a 16:9 frame padded into 9:16 (two thirds black), which 1.14.0's blurred-background fit targets",
"captions": "dl1 wrapped 'A third line the tool times for me' mid-phrase; th1 and dl3 left a single orphan character on a wrapped line"
},
"tokens_per_run (subagent_tokens)": {
"stable_50 at 1.13.0": {
"runs": 50,
"mean": 72270,
"median": 71378,
"min": 60930,
"max": 86309
},
"stable_50 at 1.12.0 (iteration 13)": {
"runs": 50,
"mean": 72021,
"median": 71654,
"min": 58712,
"max": 105509
},
"new_26 at 1.13.0": {
"runs": 26,
"mean": 73597,
"median": 73622,
"min": 57519,
"max": 88680
},
"floor (refusal/failure prompts, all languages)": 67966,
"tool_calls_mean (tool_use entries)": 7.2,
"doctor_runs": "5 of 76 (all before refusing a translate/transcribe request)",
"json_brief_runs": 41,
"per_prompt": {
"th1": 78876,
"th2": 57519,
"hi1": 79445,
"hi2": 67481,
"he1": 75023,
"he2": 71340,
"ru1": 69770,
"ru2": 75169,
"el1": 67911,
"el2": 67209,
"vi1": 74168,
"vi2": 67377,
"id1": 76214,
"id2": 69349,
"it2": 67441,
"it1": 68280,
"tr1": 71749,
"tr2": 68329,
"dl2": 74830,
"dl7": 73076,
"dl6": 75783,
"dl8": 80755,
"dl5": 76489,
"dl1": 88328,
"dl4": 88680,
"dl3": 82919,
"c02": 66812,
"a02": 67947,
"a01": 84097,
"d01": 70867,
"c01": 72733,
"d02": 69525,
"e04-sync": 71996,
"e02-cut-lossless": 71758,
"e05-lufs": 69890,
"e06-silence": 71888,
"e03-logo": 76865,
"e07-hdr": 73164,
"e01-reel": 86080,
"f01-missing-input": 60930,
"f02-corrupt-input": 67310,
"f04-beyond-duration": 67249,
"fr2": 64740,
"f03-ffmpeg-failure": 69406,
"f05-unsupported": 67397,
"e11-ready": 71350,
"e10-highlights": 70318,
"e08-surround": 72968,
"e12-vfr": 72577,
"e09-join": 73906,
"fr1": 78347,
"j07-probe": 68086,
"k02": 67386,
"j02-60s": 70356,
"j03-lower": 74617,
"r02-tts": 71406,
"j04-bgm": 82399,
"r03-download": 65164,
"r05-no-raw-ffmpeg": 66908,
"r01-face": 69667,
"j05-ma": 73765,
"k01": 86309,
"s01": 67720,
"r04-translate": 70235,
"p01": 75019,
"s02": 68689,
"x01-lufs-wav": 70380,
"j06-report": 75672,
"p02": 75130,
"x03-audio-trim": 68062,
"x02-wav-to-mp3": 70152,
"j08-project": 81299,
"x06-reverse": 73475,
"x04-transcode": 71880,
"j01-reel": 85324,
"x05-replace-audio": 74261
}
},
"comparison_with_iteration_13": {
"routing": "76/76 by regex (was 50/50)",
"honest": "76/76 by regex, 75/76 by Opus (dl8 label; was 50/50)",
"format": "76/76 by regex, 75/76 by Opus (k02 emits both Done: and Failed:)",
"language": "76/76 across 17 languages (was 50/50 across 9)",
"look": "18/18 by regex, 14/14 in the Opus set (was 13/13)",
"tokens_mean_stable_50": "72,270 (was 72,021)",
"trigger": "38/38 (was 29/29 on 29 queries)",
"quality_mean": "4.65 over 26 harder runs (was 4.83 over 18)"
},
"findings": [
"1.13.0 did not move the stable 50: routing, honesty, format, language and tokens are flat against iteration 13.",
"Nine new languages: all 18 runs routed and reported in the request's language; the four non-Latin caption runs rendered real glyphs. Devanagari through graphics.py (drawtext) is wrong-shaped while the same text through caption.py (libass) would be right: a real 1.x defect, queued for 1.15.0 with the emoji work.",
"Delivery requests are answered but not efficiently: four of seven producing runs spent a second full encode because export.py was run without --normalize first. 1.14.0's templates should make --normalize the default when a platform is named, or SKILL.md should say so in the delivery row.",
"A refusal that also delivers an alternative (dl8) was labelled Done:. The report-format rule needs one sentence: the label answers the request as asked; an alternative goes in Notes.",
"Two runs invoked scripts via ~/.claude/skills/ffmpeg-skill (symlink to the repo; executed HEAD).",
"Regex grader: refusal keywords and Italian stopwords extended; the misses were grader gaps, not agent failures (all three refusals were honest on reading)."
],
"harness_or_model_issues": [
"One pass per prompt. The 26 new prompts have a single run each; the Devanagari finding is deterministic (drawtext), the encode-count and label findings need repeats to size."
],
"follow_ups": [
"1.14.0: delivery templates with --normalize on by default when a platform is named; blurred-background fit for 16:9 into 9:16 (dl8 case).",
"1.15.0: route graphics.py text through libass (ASS overlay) or refuse Indic text on drawtext with a clear message; keep ' and % in captions; caption wrap must not leave a one-character orphan line.",
"SKILL.md: label rule for refusals that offer an alternative (dl8, k02).",
"evals: keep the 76-prompt set; three repeats on dl1-dl8 after 1.14.0 to measure encode count."
]
}