Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
caption.py: SMPTE non-drop-frame timecode cues (#54) (#61)
Shogo Kajiyama60320d3a1e4a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 595df1d..3756c58 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,8 +4,8 @@## Unreleased+- **`caption.py --text`: SMPTE non-drop-frame timecode cues.** Cues were positioned by decimal seconds only; broadcast-style deliverables often supply cue timing as `hh:mm:ss:ff` frame timecode instead. `--text` cue lines now also accept that format (e.g. `00:00:03:15 --> 00:00:06:00 ...`) — the frame count is converted to seconds with `--fps`, or the input video's own probed fps when `--input` is given and `--fps` is not. A cue that is shaped like a timecode but has no fps available (no `--fps`, no `--input`) is refused with a clear error naming the missing `--fps`, rather than silently misread as a plain text line the way an ordinary unparseable cue line already is. `_common.py` gains the reusable pieces other tools can build on later: `parse_time()` takes an optional `fps` argument for the `hh:mm:ss:ff` case (raising the new `MissingFpsError`, a `ValueError` subclass, when fps is needed but absent), and `fmt_smpte_time()` formats seconds back to `hh:mm:ss:ff` — used here to echo the interpreted cue range in the `wrote ... .srt` report line so a caller can confirm the timecode was read correctly. The written `.srt` itself stays decimal-millisecond SRT timing, since that is the only timing SRT/mux subtitle codecs actually carry; nothing claims frame-exact precision it can't hold. Drop-frame (29.97/59.94 fps) counting is out of scope. Closes [#54](https://github.com/kajisho5/ffmpeg-skill/issues/54).- **`references/process-pitfalls.md`: development-process mistakes already made once.** Distinct from `references/ci-platform-pitfalls.md` (ffmpeg/CI behaviour differences): this is about the process of making a change safely, not FFmpeg itself. Three entries to start: narrowing a `TOOL_META[...]["required"]` capability list without first grepping `tests/test_contract.py`'s `DoctorDetectionTests` for the fixture-pinned `usable` outcome it protects (nearly broken twice, in #51 and #52, caught before landing both times); retrying a git tag push or GitHub Release creation in this environment, where both are scoped out (branch pushes work, tag pushes 403 at the git-receive-pack level; no `create_release`/`create_tag` MCP tool exists; the outbound proxy itself blocks a raw REST API call to the releases endpoint) rather than accepting it after one confirmation; and redesigning a test fixture a third time instead of recognising, after two independently-redesigned fixtures failed differently on the same platform, that the platform's real behaviour (not the fixture) is the actual cause (`stabilize.py`'s macOS libvidstab test). A living document — add to it whenever one of these recurs. Docs-only; no behaviour changed.-- **`doctor --json`: `gpu_encoders`.** No tool here uses GPU-accelerated encoding — every tool assumes CPU x264/x265 — but `doctor` had no way to answer "is GPU encoding available on this machine" at all, unlike every other capability it already reports `yes`/`no`/`unknown` for. `gpu_encoders` reports GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) present in this ffmpeg *build*, read from the same `-encoders` listing `doctor` already parses — `{"status": "parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. Deliberately build-presence only: proving a real GPU/driver will accept a job would need an actual encode, which `doctor`'s introspection never runs (matching its existing 10s-timeout, listing-only philosophy). Purely informational — no tool declares or requires a GPU encoder, so this field never affects `ok` or any tool's `usable`. The human-readable `doctor` output gets one line naming what's present (or "none"). Closes [#52](https://github.com/kajisho5/ffmpeg-skill/issues/52).- **`caption.py --mode mux`: soft subtitle stream instead of burn-in.** Every caption call previously re-encoded both streams to render pixels (`reencodes_video`/`reencodes_audio`: `"always"`) even when the caller only wanted a subtitle track added, not the picture changed. `--mode mux` (new; `--mode burn` stays the default) copies video and audio untouched (`-c:v copy -c:a copy`) and adds the SRT as a separate, player-toggleable subtitle stream — `reencodes_video`/`reencodes_audio` are now `"conditional"` with a note explaining the split. Only takes a plain SRT (`--srt`/`--text`/`--transcribe`), not `--ass`, `--animate` or `--karaoke`: styling and animation render pixels, so they have no soft-subtitle equivalent and are refused with a pointer to `--mode burn`. The subtitle codec is picked from the output container (`mov_text` for `.mp4`/`.m4v`/`.mov`, `srt` for `.mkv`, `webvtt` for `.webm`); an unrecognized container is refused rather than guessed at. `contract --json` gains three new optional capabilities (`encoder:mov_text`/`encoder:webvtt`/`encoder:srt`, each `"when"`-gated to the matching output container) so `doctor` can report them honestly; the existing required capabilities (`encoder:libx264`, `encoder:aac`, `filter:subtitles`) are unchanged, since `--mode burn` is still the default and doctor's usability model doesn't vary by flag. Closes [#51](https://github.com/kajisho5/ffmpeg-skill/issues/51) — timecode-aware cue timing and explicit multi-audio-track selection were split out to [#54](https://github.com/kajisho5/ffmpeg-skill/issues/54) and [#55](https://github.com/kajisho5/ffmpeg-skill/issues/55) to keep this change reviewable.- **Test: the prose tool count in README/`docs/contract.md`/`package.json` is now checked against the real tool list.** 0.11.0 was cut to fix exactly this drift (README said 28 twice and 22 once; `package.json` said 21) by hand, with nothing to stop it recurring. `tests/test_contract.py`'s `test_docs_tool_count_matches_the_real_tool_list` scans all three for `"<N> tools"` wording and fails if any number doesn't match `scripts/`'s actual public-tool count, so the next tool added/removed without updating every mention fails CI instead of drifting silently. `contract_version`/`skill.version` were already split (0.9.0) and already documented as "additive keeps `contract_version`, breaking bumps it" — `docs/contract.md` now also repeats the tagged-version pin guidance next to that table, since `capability_map` makes this the most cross-repo-facing part of the contract. Closes [#50](https://github.com/kajisho5/ffmpeg-skill/issues/50).diff --git a/references/scripts.md b/references/scripts.mdindex 2397c92..2dae950 100644--- a/references/scripts.md+++ b/references/scripts.md@@ -283,13 +283,16 @@ frame like an editor would. Use `--compare` to show before/after to the user.### caption.py — subtitles (static, animated, karaoke)```caption.py INPUT --srt FILE | --ass FILE | --text CUES.txt [--write-srt OUT.srt]- [--mode burn|mux]+ [--mode burn|mux] [--fps N][--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB][--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N][--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass] [-o OUT]caption.py --text CUES.txt --write-srt OUT.srt # generate the SRT only```-Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`.+Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`,+or `00:00:03:15 --> 00:00:06:00 SMPTE non-drop-frame timecode` (`hh:mm:ss:ff`, frame count+converted with `--fps`, or the input video's own fps when `--input` is given and `--fps` is+not — a timecode-shaped cue with no fps available is refused rather than misread as plain text).Lines without a time run for `--auto-seconds` (3 s) after the previous cue. `|` is a line break.`--animate`/`--karaoke` generate a styled ASS (PlayRes = video size) from theSRT/text cues: `pop` is the short-form "bouncy" entrance, `--karaoke` fills eachdiff --git a/scripts/_common.py b/scripts/_common.pyindex 4346e98..e50d482 100755--- a/scripts/_common.py+++ b/scripts/_common.py@@ -392,12 +392,30 @@ def default_output(input_path: str, suffix: str, ext: Optional[str] = None) -> sreturn str(p.with_name(f"{p.stem}_{suffix}.{new_ext}"))-def parse_time(value: str) -> float:- """Accept seconds ('12.5'), mm:ss ('1:30'), hh:mm:ss(.ms) ('00:01:30.250') or SRT '00:01:30,250'."""+class MissingFpsError(ValueError):+ """parse_time() saw an hh:mm:ss:ff SMPTE timecode but no fps was given to convert it -- distinct+ from a plain ValueError so a caller that falls back to treating unparseable text as a literal+ line (e.g. caption.py's free-text cue format) can still fail loudly on this one, instead of+ silently swallowing a mistyped/missing --fps as an auto-timed line of digits."""+++def parse_time(value: str, fps: Optional[float] = None) -> float:+ """Accept seconds ('12.5'), mm:ss ('1:30'), hh:mm:ss(.ms) ('00:01:30.250'), SRT '00:01:30,250',+ or -- when `fps` is given -- SMPTE non-drop-frame timecode 'hh:mm:ss:ff' ('00:01:30:15')."""v = value.strip().replace(",", ".")if not v:raise ValueError("empty time")parts = v.split(":")+ if len(parts) == 4:+ if fps is None or fps <= 0:+ raise MissingFpsError(f"'{value}' looks like an hh:mm:ss:ff SMPTE timecode, but no fps was given to convert its frame count to seconds")+ h, m, s, f = parts+ if "." in f:+ raise ValueError(f"bad SMPTE timecode: {value}")+ frame, whole_fps = int(f), int(round(fps))+ if not (0 <= frame < whole_fps):+ raise ValueError(f"bad SMPTE timecode '{value}': frame {frame} is out of range for {fps:g} fps (0-{whole_fps - 1})")+ return int(h) * 3600 + int(m) * 60 + int(s) + frame / fpsif len(parts) > 3:raise ValueError(f"bad time: {value}")total = 0.0@@ -416,6 +434,20 @@ def fmt_srt_time(seconds: float) -> str:return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"+def fmt_smpte_time(seconds: float, fps: float) -> str:+ """SMPTE non-drop-frame timecode 'hh:mm:ss:ff' for a real fps (not the fractional NTSC rates+ -- 29.97/59.94 need drop-frame counting to stay wall-clock accurate, which this does not do)."""+ if seconds < 0:+ seconds = 0.0+ whole_fps = int(round(fps))+ total_frames = int(round(seconds * fps))+ frame = total_frames % whole_fps+ secs_total = total_frames // whole_fps+ h, rem = divmod(secs_total, 3600)+ m, s = divmod(rem, 60)+ return f"{h:02d}:{m:02d}:{s:02d}:{frame:02d}"++def escape_filter_path(path: str) -> str:"""Escape a file path for use as a filter option value (subtitles=, ass=, lut3d=file=, fontfile=, fontsdir=).diff --git a/scripts/caption.py b/scripts/caption.pyindex 3ff7756..fb51c58 100755--- a/scripts/caption.py+++ b/scripts/caption.py@@ -19,6 +19,7 @@ The subtitle codec is picked from the output container: mov_text forText-to-SRT input format (one cue per line, blank lines ignored):0:00-0:03 Hello and welcome00:00:03.500 --> 00:00:06 Second line | with a manual line break+ 00:00:03:15 --> 00:00:06:00 SMPTE non-drop-frame timecode (hh:mm:ss:ff, needs --fps)Text without a time is auto-timed after the previous cue (--auto-seconds)Examples:@@ -35,7 +36,7 @@ import sysfrom pathlib import Pathfrom typing import List, Optional, Tuple-from _common import STATE, color_hex, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, fmt_srt_time, info, parse_time, probe, run, x264_args+from _common import STATE, color_hex, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, fmt_srt_time, fmt_smpte_time, info, MissingFpsError, parse_time, probe, run, x264_argsALIGN = {"bottom": 2, "top": 8, "center": 5, "bottom-left": 1, "bottom-right": 3, "top-left": 7, "top-right": 9}@@ -44,7 +45,7 @@ TIME_RE = re.compile()-def parse_text_cues(path: str, auto_seconds: float, gap: float) -> List[Tuple[float, float, str]]:+def parse_text_cues(path: str, auto_seconds: float, gap: float, fps: Optional[float] = None) -> List[Tuple[float, float, str]]:cues: List[Tuple[float, float, str]] = []cursor = 0.0with open(path, encoding="utf-8") as fh:@@ -55,7 +56,9 @@ def parse_text_cues(path: str, auto_seconds: float, gap: float) -> List[Tuple[flm = TIME_RE.match(line)if m:try:- start, end = parse_time(m.group("a")), parse_time(m.group("b"))+ start, end = parse_time(m.group("a"), fps), parse_time(m.group("b"), fps)+ except MissingFpsError as e:+ die(f"cue '{line}': {e} -- pass --fps, or --input's own fps is used automatically when given")except ValueError:start, end, text = cursor, cursor + auto_seconds, line.strip()else:@@ -315,6 +318,9 @@ def main() -> int:src.add_argument("--write-srt", help="where to save the generated SRT (default: <text>.srt)")src.add_argument("--auto-seconds", type=float, default=3.0, help="duration for cues without timing (default 3)")src.add_argument("--gap", type=float, default=0.0, help="gap after auto-timed cues in seconds")+ src.add_argument("--fps", type=float, default=None,+ help="frame rate for interpreting hh:mm:ss:ff SMPTE timecode cues in --text (non-drop-frame); "+ "defaults to the input video's own fps when --input is given, required otherwise")sty = ap.add_argument_group("style (SRT only)")sty.add_argument("--brand", help="brand.json: font, colours, caption size/position/animation defaults")sty.add_argument("--font", default=None, help="font family, e.g. 'Noto Sans CJK JP' for Japanese (default DejaVu Sans or brand font)")@@ -367,6 +373,10 @@ def main() -> int:if args.animate != "none" or args.karaoke:die("--animate/--karaoke render pixels into the picture and require --mode burn")+ fps_for_tc = args.fps+ if fps_for_tc is None and args.input:+ fps_for_tc = probe(args.input).get("video", {}).get("fps")+srt_path = args.srtif args.transcribe:if not args.input:@@ -376,7 +386,7 @@ def main() -> int:info(f"wrote {srt_path} ({len(cues)} cues)")args.text = Noneif args.text:- cues = parse_text_cues(args.text, args.auto_seconds, args.gap)+ cues = parse_text_cues(args.text, args.auto_seconds, args.gap, fps_for_tc)if args.write_srt:srt_path = args.write_srtelif args.input:@@ -387,7 +397,8 @@ def main() -> int:srt_path = os.path.splitext(args.text)[0] + ".srt"if not STATE.dry_run:write_srt(cues, srt_path)- info(f"wrote {srt_path} ({len(cues)} cues)")+ tc_range = f", {fmt_smpte_time(cues[0][0], fps_for_tc)}-{fmt_smpte_time(cues[-1][1], fps_for_tc)} @ {fps_for_tc:g}fps" if fps_for_tc else ""+ info(f"wrote {srt_path} ({len(cues)} cues{tc_range})")if not args.input:print(srt_path)return 0diff --git a/tests/test_all.py b/tests/test_all.pyindex b066812..e303aac 100644--- a/tests/test_all.py+++ b/tests/test_all.py@@ -510,6 +510,33 @@ class FFmpegSkillTests(unittest.TestCase):self.assertTrue((sub / "cap_side.srt").exists(), "SRT written beside the output")self.assertFalse((OUT / "source.srt").exists(), "no SRT dropped next to the source")+ def test_caption_smpte_timecode_cues_convert_frames_to_seconds(self):+ tc = OUT / "tc_cues.txt"+ tc.write_text("00:00:00:12 --> 00:00:02:00 Hello\n00:00:02:00 --> 00:00:04:00 World\n", encoding="utf-8")+ srt = OUT / "tc.srt"+ script("caption.py", "--text", tc, "--write-srt", srt, "--fps", "24")+ text = srt.read_text(encoding="utf-8")+ self.assertIn("00:00:00,500 --> 00:00:02,000", text, "frame 12 at 24fps is exactly 0.5s")++ def test_caption_smpte_timecode_without_fps_fails_loudly(self):+ """A cue that looks like hh:mm:ss:ff but has no --fps must not be silently swallowed as auto-timed text."""+ tc = OUT / "tc_no_fps.txt"+ tc.write_text("00:00:00:12 --> 00:00:02:00 Hello\n", encoding="utf-8")+ proc = script("caption.py", "--text", tc, "--write-srt", OUT / "tc_no_fps.srt", "--json", expect_fail=True)+ doc = json.loads(proc.stdout)+ self.assertEqual(doc["error"]["kind"], "input")+ self.assertIn("fps", doc["error"]["message"])+ self.assertFalse((OUT / "tc_no_fps.srt").exists())++ def test_caption_fps_defaults_to_the_input_videos_own_fps(self):+ tc = OUT / "tc_auto_fps.txt"+ tc.write_text("00:00:00:15 --> 00:00:02:00 Hello\n", encoding="utf-8")+ out = OUT / "tc_auto.mp4"+ script("caption.py", self.src, "--text", tc, "-o", out, "--preset", "veryfast")+ srt_text = (OUT / "tc_auto.srt").read_text(encoding="utf-8")+ # self.src is 30fps: frame 15 is exactly 0.5s+ self.assertIn("00:00:00,500 --> 00:00:02,000", srt_text)+def test_caption_mux_copies_streams_and_adds_a_subtitle_track(self):srt = OUT / "mux_cues.srt"out = OUT / "cap_mux.mp4"