Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

fix: join dry-run plans pending inputs; --export-timeline times, frame and FCPXML order (#290)

Shogo Kajiyama9ef1e4292f9f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8d314f3..7997ff3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,82 @@
## Unreleased
-(nothing yet)
+- `join.py --dry-run` no longer fails a plan because one of its inputs is not written yet. The
+ pending file's placeholder probe always claimed a video stream, so a TTS list with one line
+ still to come was refused as an audio/video mix ("tts_01.wav has no video stream while
+ tts_03.wav has one"), a list of nothing but pending `.wav` lines planned a libx264 join into
+ `voice.wav`, and a `render.py --dry-run` of an audio project with one trimmed and one untrimmed
+ clip failed at the join. A pending input's extension now stands in for its streams: any
+ audio extension the skill reads (`.wav`, `.m4a`, `.aiff`, `.caf`, ...) means no picture. A
+ join's audio rate and layout come from the inputs that exist, and each pending input is
+ named -- a stderr note, the new `pending` key and `notes` -- instead of being passed over in
+ silence. A pending `.mp4` next to a measured `.wav` is still refused as the mix a real run
+ refuses, now naming the `.wav` and the file expected to hold a picture. With either
+ `--on-missing` a dry run plans on a pending input and never lists it under `skipped`; a real
+ run still refuses or skips a file that is missing when it runs, and under `skip` the note
+ says when skipping would leave fewer than two inputs, which a real run refuses.
+ `expected_duration` is `null` while an input is pending: the placeholder's 0 s made it negative,
+ or shorter than the one clip that exists. In a video join, `notes` also says when the planned
+ frame and rate (a pending first input) or xfade offsets (after a pending input) are the
+ placeholder's unmeasured values. Real runs are unchanged.
+- `render.py --export-timeline` writes FCPXML that validates against Apple's FCPXML 1.10 DTD:
+ the music bed now comes before the chapter markers inside the first clip (the DTD's order is
+ `timeMap`, connected clips, markers). 2.1.0-2.2.1 wrote them the other way round, which does
+ not validate -- and Final Cut validates what it imports against that DTD.
+- `--export-timeline` reads clip `in`/`out` and chapter times the way the render does (`"0:05"`,
+ `hh:mm:ss:ff` at the source's fps, `@fps`). Before, any string time was a Python traceback
+ with nothing on stdout under `--json` -- including the `"in": "0:00"` of `render.py --init`'s
+ own starter project. A time that does not parse is now a `kind: input` failure naming the
+ field (`clips[0].in`). A clip with no picture has no fps of its own, and `frame.fps` does not
+ stand in for one in either path: its `hh:mm:ss:ff` needs `@fps`, as `cut.py` asks.
+- `render.py` refuses, as `kind: input` naming the key, a value that is not an object where
+ the run reads it. 2.2.1 died there with a traceback and nothing on stdout under `--json`. In
+ the render and `--export-timeline` alike, that covers a string `frame`, a clip written as a
+ bare path, a clip with no `src`, a string `audio` and a string `transition` between two or
+ more clips. In the render it also covers every stage section (`"export": "reels"`,
+ `"captions": "subs.srt"`) and the `snap` of a clip it cuts, up to the `--stop-after`
+ stage (a `--stop-after fit` preview never reads `captions`). Nothing a full run never reads is
+ refused. A single clip's `"transition": "none"` and an uncut clip's `snap` still render and
+ export. `--export-timeline` still writes a timeline for a stage section that is not an
+ object and lists it in `not_exported`, as 2.2.1 did.
+- The exported sequence is the frame the project asks for, sized by fit.py's own rule:
+ `frame: {aspect: "16:9", width: 1920}` is 1920x1080 over any source (it was 1920x2160 over a
+ 4K one, the height taken from the source). An aspect-only frame takes the export preset's
+ size when the preset has that aspect, as the render's does. fit.py now sizes its output with
+ the same shared function (`frame_size()`) and reads `--aspect` with the same parser
+ (`aspect_ratio()`), which reads each side with `int()` as fit.py did, so fit.py accepts and
+ sizes every `--aspect` exactly as 2.2.1 did. The export now refuses a `frame.aspect` of `16/9`
+ or `2.39:1` when the render would hand it to fit.py, which refuses it too. The render's own preset match
+ (`frame_from_preset()`) keeps 2.2.1's looser reading, so a `16/9` frame replaced by the
+ project's `fit.aspect` renders exactly as before, and the export writes its frame.
+- A render of several clips delivers that frame too. When the frame gives an aspect and one
+ side, and the project's `fit` object gives no width, height or other aspect, `render.py`
+ now gives fit.py that side. Before, join.py sized the join at the first clip's aspect and
+ fit.py fitted the new aspect inside it: `{aspect: "9:16", width: 1080}` (`render.py`'s own
+ docstring frame) over 16:9 clips came out 342x608 -- the delivered size without an export
+ preset, and the size captions were burned at with one. Every other multi-clip project renders
+ at the size 2.2.1 rendered.
+- FCPXML describes each video asset as its own source (size and frame rate) and the sequence
+ as the frame; one shared format made a 320x180 file under a 9:16 frame claim to be
+ 1080x1920. A clip of another aspect states its spatial conform, `adjust-conform` `fill` for
+ `frame.fit: crop` and `fit` for pad; the DTD reads a missing one as fit, so a crop project
+ was exported as a fit. EDL and OTIO cannot state a conform, so there the reframe is listed in
+ `not_exported` with the project's `frame.fit`, as is `blur`'s blurred background in all three.
+
+- Known, carried into a minor release (`docs/design-decisions.md` records each one):
+ - A multi-clip render with `frame.fit: crop` keeps bars on every clip of another aspect than
+ the joined frame, even when all the clips share one aspect: join.py pads the clips into
+ the frame before fit.py crops. The exported timeline states `fill`, the project's choice.
+ - A project `fit` object with its own width, height or aspect is still fitted inside a
+ multi-clip join's picture, as in 2.2.1 (`fit {aspect 4:5}` under `{aspect 9:16, width 72}`
+ renders 32x40 against a 72x128 sequence). Which of `frame` and `fit` wins waits for that
+ release.
+ - For a value that is not an object in a section a path never reads, the export and the
+ render can still disagree, as in 2.2.1. Refusing it in both paths would refuse projects
+ 2.2.1 rendered or exported.
+ - `batch.py` names every intermediate of a `.ogg`, `.opus`, `.aac`, `.aif`/`.aiff`, `.caf`
+ or `.wma` source `.mp4`, so a `--dry-run` of a join step over those sources is refused as
+ an audio/video mix that the real batch completes (as in 2.2.1).
## 2.2.1
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index 0225004..ba8e466 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -31,6 +31,33 @@ exists. When a decision changes, edit the entry in the same PR.
- **When a measured input is an intermediate an earlier dry-run stage would have written**
(render/batch plans), the measurement is skipped with a note rather than failing the plan.
Code: `_common.dry_run_input_pending()`.
+- **`join.py --dry-run` plans on an input that does not exist yet, whatever `--on-missing`
+ says.** The file is taken for an earlier step's output (render's cut clips, a TTS line): it
+ stays in the planned command and is named under `pending` and in `notes`, never under
+ `skipped`, which in a dry run as in a real run holds only what the join left out. A real run
+ decides on the file as it then is: refused under `fail`, skipped under `skip` -- and refused
+ under `skip` too when skipping leaves fewer than two inputs, which the note then says. Its
+ probe is the dry-run stub, whose video stream means nothing, so its extension stands in: an
+ audio extension -- any this skill reads (`_common.AUDIO_MEDIA_EXT`: `.wav`, `.m4a`, `.aiff`,
+ `.caf`, ...), not only the ones it writes -- is taken for a file with no picture, any other
+ for one with a picture, and the measured and pending inputs meet the rules a real run applies
+ (all without a picture: an audio join; a mix: refused, naming a measured picture before a
+ pending one). The guess can be wrong -- an `.mp4` holding only audio is refused as a mix a
+ real run would have joined as audio, and batch.py names every intermediate of a `.ogg`,
+ `.opus`, `.aac`, `.aif`/`.aiff`, `.caf` or `.wma` source `.mp4`, so a dry run of a batch join
+ step over those sources is refused where the real batch completes (batch.py keeps the
+ source's extension only for `.wav`, `.mp3`, `.m4a` and `.flac`; widening that changes the
+ container a real run writes, and waits for a minor release) -- but a plan that completes and
+ a real run that then refuses the mix, after the earlier steps ran, is the worse error. A
+ join's audio rate and layout come from the inputs that exist, and `expected_duration` is null
+ while any input is pending: the stub's 0 s would count the clip as nothing. The video join's
+ planned frame and rate still come from the first input and its xfade offsets from every
+ input's length: a pending first input's frame and rate, and the offsets after a pending
+ input, are the stub's unmeasured 0x0 / 0 s, and the plan's `notes` say they are placeholders
+ rather than borrowing another clip's (#77 made the stub honest instead of plausible). Code:
+ `join.preflight()`, `join.main()`, `join.unpending_length()`.
+ Tests: `test_join_dry_run_plans_on_pending_segments`,
+ `test_render_dry_run_joins_a_trimmed_and_an_untrimmed_audio_clip`.
- **`verify` accepts `--dry-run` and ignores it.** Its job is to run the tools for real.
Contract: `_contract.DRY_RUN_NOTES["verify"]`.
@@ -579,6 +606,78 @@ not a new file format this tool would have to maintain.
`timeline.not_exported` and stderr. A caption track or a title in FCPXML would be a second
implementation of `caption.py`/`graphics.py` whose output no one here can see.
Test: `test_build_refuses_what_a_timeline_cannot_hold`.
+- **The sequence is the project's frame, sized by fit.py's own rule, and the render delivers
+ that size.** The project's `frame` goes through `frame_from_preset()` (an aspect-only frame
+ takes the export preset's size when the preset has that aspect) and then `frame_size()`, the
+ function fit.py sizes its output with: one side and an aspect give the other, so
+ `{aspect 16:9, width 1920}` is 1920x1080 over any source (2.1.0-2.2.1 took the height from
+ the source: 1920x2160 over 4K). The aspect is read with `aspect_ratio()`, which fit.py now
+ reads `--aspect` with too: two sides split at `:`, each read with `int()` as fit.py read them
+ in 2.2.1 (`+16:9` and `1_6:9` parse, `0:9` is a zero ratio that sizes as no aspect). So the
+ export refuses `16/9` where the render hands it to fit.py, which refuses it too, and fit.py
+ refuses nothing it took in 2.2.1. When the project's own `fit.aspect` replaces it, the render
+ never gives fit.py the frame's aspect and completes, so the export sizes the sequence
+ without it. `frame_from_preset()` keeps 2.2.1's looser match (`16/9`, `1.78:1`) for the
+ preset size: it drops nothing a 2.2.1 render was sized by.
+ A render of several clips gives fit.py the frame's side when the frame gives an aspect and
+ one side, and the project's `fit` object gives no width, height or other aspect. Before,
+ join.py sized the join at the first clip's aspect and fit.py fitted the aspect inside it
+ (`{aspect 9:16, width 1080}` over 16:9 clips rendered 342x608). Every other multi-clip
+ project renders as 2.2.1 rendered it: a frame with both sides is already the join's size,
+ and a `fit` object with its own size or aspect is fitted inside the joined picture
+ (`fit {aspect 4:5}` under `{aspect 9:16, width 72}` over 16:9 clips renders 32x40 against a
+ 72x128 sequence, where one clip renders 72x90). Filling the frame's sides in there too
+ overrode the `fit` object (`fit {width 540}` under a 1080x1920 crop frame rendered 540x1920,
+ half the picture cut away). Which of `frame` and `fit` wins is deferred to a minor release.
+ Three render gaps remain, all from join.py sizing the join before fit.py reframes it. (1)
+ With several clips, one side and no aspect, join.py rounds an odd side down where fit.py
+ rounds up (`{width 100}` over 4:3 renders 100x74, the sequence is 100x76). (2) join.py pads
+ every clip into the joined frame before fit.py crops: the frame's own W x H when it gives
+ both sides (as every template frame does after `frame_from_preset()`), else the first clip's
+ aspect. So a multi-clip `frame.fit: crop` render keeps bars on every clip of another aspect
+ than that frame, even when all the clips share one aspect, while the timeline states
+ `fill`, the project's choice (the timeline follows the project, not the render's drift).
+ Passing `frame.fit` to join.py would change rendered output and waits for a minor release.
+ (3) The `fit`-object case above.
+ Tests: `test_sequence_frame_is_the_frame_the_render_delivers`,
+ `test_a_multi_clip_render_delivers_the_sequence_size`,
+ `test_the_export_and_fit_py_read_one_aspect_grammar`,
+ `test_render_frame_aspect_takes_the_export_presets_size`.
+- **A clip of another aspect is a reframe FCPXML states and EDL and OTIO cannot.** FCPXML's
+ `adjust-conform` is written on each such clip, `fill` for `frame.fit: crop` and `fit` for pad
+ (the DTD reads a missing one as fit, so leaving it out exported every crop as a fit). blur
+ has no conform: the clip is fitted, and the blurred background is named in `not_exported`.
+ EDL and OTIO have no field for a conform, so there the reframe is named in `not_exported`
+ with the project's `frame.fit`, and the editor's default decides. Each FCPXML video asset
+ carries its own source's format (size and frame rate) and the sequence the frame's: with one
+ format for both, a 320x180 asset under a 9:16 frame claimed to be 1080x1920.
+ Test: `test_fcpxml_describes_each_source_and_states_the_conform`.
+- **Times are read the way render.py's own stages read them.** `clips[].in`/`out` take
+ `parse_time()`'s grammar at the source's fps, as cut.py reads them. `frame.fps` does not stand
+ in: cut.py has no other fps to use, so a clip with no picture needs `@fps` on an
+ `hh:mm:ss:ff` time in both paths. A chapter's `at` takes no fps, as metadata.py reads it, so
+ `hh:mm:ss:ff` there needs its `@fps` in both paths too. Neither path accepts a time the
+ other refuses. A time that does not parse is `kind: input` (2.1.0-2.2.1 called `float()`,
+ and `--init`'s own `"in": "0:00"` was a traceback).
+ Tests: `test_times_are_read_with_the_render_grammar`,
+ `test_a_bad_time_is_an_input_failure_not_a_traceback`.
+- **A value that is not an object is refused only where the run reads it.**
+ `validate_project()` refuses it as `kind: input`, naming the key, before the first stage.
+ In both paths that covers a string `frame`, a clip written as a bare path, a clip with no
+ `src`, a string `audio` and a string `transition` between two or more clips. In the render
+ it also covers every stage section (`"export": "reels"`, `"captions": "subs.srt"`) and the
+ `snap` of a clip it cuts, up to the `--stop-after` stage: a section read only after it
+ (`"captions": "subs.srt"` under `--stop-after fit`, a string `transition` under
+ `--stop-after clips`) is not refused, as 2.2.1 completed those previews. A full run of each
+ of these died in 2.2.1 with a traceback and nothing on stdout under `--json`. A value
+ the run never reads is left as 2.2.1 left it. That is a single
+ clip's `transition` (a batch.py project recipe that carries join.py's `--transition none`),
+ an uncut clip's `snap`, and, in `--export-timeline`, every section it only lists in
+ `not_exported`. Those projects rendered or exported in 2.2.1, and a patch release does not
+ refuse them. So for these shapes the export can accept a project the render refuses, as in
+ 2.2.1. Refusing them in both paths is deferred to a minor release. Code:
+ `render.validate_project()`, `render.require_object()`.
+ Test: `test_a_bad_time_is_an_input_failure_not_a_traceback`.
- **OTIO's `source_range` is timeline length, with the speed in a `LinearTimeWarp`.** Core
OTIO does not rescale a clip's duration by its effects; the first version stored the
speed-scaled source length and the reference `opentimelineio` library (0.18) measured the
@@ -588,5 +687,14 @@ not a new file format this tool would have to maintain.
All three formats were read back by the reference OTIO library and its FCPXML / CMX 3600
adapters at the right length, but none of Final Cut, Resolve or Premiere is available in this
environment, and OTIO's own FCPXML adapter ignores dissolves, markers and retiming, so those
- parts of the FCPXML are checked only against Apple's schema by hand. The editor round trip is
- on #143's real-hands checklist.
+ parts of the FCPXML are checked against Apple's FCPXML 1.10 DTD with `xmllint --dtdvalid`, by
+ hand: the DTD ships inside Final Cut, not in this repository, so no test reads it. 2.1.0 made
+ the same claim and its files did not validate: the music bed came after the chapter markers
+ inside the first clip, where the DTD's `asset-clip` holds `timeMap`, then the `adjust-*`
+ elements (`adjust-conform`), then anchored items (the connected music clip), then markers.
+ That order is pinned by `test_fcpxml_spine_matches_the_cut` and
+ `test_fcpxml_keeps_the_dtd_child_order_on_a_retimed_first_clip`. OTIO's FCPXML adapter
+ reads each clip's offset at its asset's frame rate and compares the frame counts across
+ rates, so a project mixing source rates reads back there with gaps the file does not have
+ (the file's offsets are rational seconds). The editor round trip is on #143's real-hands
+ checklist.
diff --git a/references/scripts.md b/references/scripts.md
index 1b4202e..318da87 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -563,7 +563,13 @@ before anything runs: missing, empty (0 bytes) or unreadable as media. By defaul
are named in one refusal (`kind: input`, `problems: [{index, path, reason}]`, nothing encoded);
`--on-missing skip` joins the usable ones and reports the rest under `skipped` (always present,
`[]` when nothing was). A list with no entries is refused as such, and a skip that leaves fewer
-than two inputs is refused rather than "joining" one file.
+than two inputs is refused rather than "joining" one file. Under `--dry-run` an input that does
+not exist yet is an earlier step's output, with either `--on-missing`: it stays in the plan and
+is named under `pending: [{index, path}]` and in `notes` (never `skipped`). Its extension stands
+in for its streams (any audio extension the skill reads, `.aiff` and `.caf` included: no
+picture), so a pending `.mp4` next to measured audio is refused as the mix a real run would
+refuse; `expected_duration` is null while any input is pending, and `notes` names the planned
+frame, rate or xfade offsets that rest on a pending input's unmeasured placeholder.
Normalises every clip to one frame size, fps, `yuv420p`, 48 kHz and one
channel layout (the widest clip's -- a 5.1 clip keeps 5.1 -- or `--channels`;
silent track generated for clips without audio), then chains `xfade` +
@@ -586,13 +592,20 @@ Premiere, Resolve, Avid) or `.otio` (OpenTimelineIO JSON: Resolve natively, othe
OTIO's adapters). Clips keep their in/out and speed (an M2 line, a `timeMap`, a
`LinearTimeWarp`); `transition` becomes a cross dissolve centred on each cut, trimmed so the
timeline is exactly as long as `join.py`'s crossfade would render it; `audio.music` is its own
-track; inline `chapters` are markers. Everything else in the project -- captions, graphics,
-overlays, the silence cut, fit, audio processing, loudness, export -- is listed in the result's
-`timeline.not_exported` and on stderr, never dropped silently. Media paths are absolute file
-URLs to the sources, nothing is copied or encoded, and the existing-output rule applies
-(`--overwrite` to replace). The timeline follows the project's numbers exactly; a render of
-the same project can come out a few frames longer, because `join.py` offsets its crossfades
-by each part's container duration, audio included.
+track; inline `chapters` are markers. Times take the render's own grammar (`"0:05"`,
+`hh:mm:ss:ff` at the source's fps), and the sequence is the project's `frame` sized by
+fit.py's rule (`{aspect 16:9, width 1920}` is 1920x1080; an aspect alone takes the export
+preset's size when the preset has that aspect; an aspect fit.py refuses, such as `16/9`, is
+refused unless the project's own `fit.aspect` replaces it in the render). A clip of another
+aspect gets its spatial conform in FCPXML (`adjust-conform`: `fill` for `frame.fit: crop`,
+`fit` for pad); EDL and OTIO cannot state one. Everything else in the project -- captions,
+graphics, overlays, the silence cut, fit, a reframe the format cannot state (and `blur`'s
+background), audio processing, loudness, export -- is listed in the result's
+`timeline.not_exported` and on stderr, never dropped silently. Media paths are absolute file URLs to the sources, nothing is
+copied or encoded, and the existing-output rule applies (`--overwrite` to replace). The
+timeline follows the project's numbers exactly; a render of the same project can come out a
+few frames longer, because `join.py` offsets its crossfades by each part's container duration,
+audio included.
A plan is a single tool's dry run as an artifact: `cut.py in.mp4 --start 2 --end 8
--plan cut.json` writes `{plan_version, tool, argv, inputs (path, size, sha256 of
head+tail), commands, output, verify}` and runs nothing. `render.py cut.json`
diff --git a/scripts/_common/__init__.py b/scripts/_common/__init__.py
index 189ea5e..a5a7053 100644
--- a/scripts/_common/__init__.py
+++ b/scripts/_common/__init__.py
@@ -69,10 +69,10 @@ from _common.emit import (
from _common.probe import (
analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
_fraction, keyframes_near, SCORE_RE, SIL_RE, decode_gray_frames,
- measured_level_dbfs, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output
+ measured_level_dbfs, AUDIO_MEDIA_EXT, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output
)
from _common.decision import (
- aac_args, add_pad_fill_args, audio_codec_for, AUDIO_CODECS, brand_caption_style, BRAND_DEFAULTS,
+ aac_args, add_pad_fill_args, aspect_ratio, audio_codec_for, AUDIO_CODECS, brand_caption_style, BRAND_DEFAULTS,
description_block, _evidence_rank, fmt_chapter_time, propose_chapters,
frame_flow, label_shot_flow, MOTION_GRID, MOTION_SEARCH, MOTION_STATIC_PX, MOTION_PAN_SPREAD,
filler_spans, FILLER_WORDS, FILLER_AMBIGUOUS, FILLER_DISCOURSE_MARKERS, FILLER_MAX_WORD,
@@ -82,7 +82,7 @@ from _common.decision import (
BEAT_Z_FLOOR, BEAT_Z_SPAN,
_onset_strength, _pick_onsets, _autocorrelation_peak, _grid_score,
brand_states_font, cfr_args, concat_list_line, db_to_linear, default_output, encoder_args, escape_filter_path,
- fmt_secs, fmt_smpte_time, fmt_srt_time, is_audio_output, load_brand, MissingFpsError, pad_filters, parse_time,
+ fmt_secs, fmt_smpte_time, fmt_srt_time, frame_size, is_audio_output, load_brand, MissingFpsError, pad_filters, parse_time,
signed_time_arg, SVT_PRESET, time_arg, video_args, x264_args, _x264_raw
)
from _common.asr import (
@@ -201,7 +201,7 @@ __all__ = [
"fonts_dir_covers_script", "_fraction", "has_emoji", "info", "INSTALL_HINTS", "install_signal_handlers",
"is_audio_output", "_is_emoji_base", "_is_emoji_char", "_is_ffmpeg", "_is_mark", "_KEYCAP", "_KEYCAP_BASES",
"keyframes_near", "LANGUAGE_NAMES", "LATIN_EM", "LEADING_VOWELS", "_libass_color_probe", "_limit_for",
- "load_brand", "measured_level_dbfs", "MEDIA_EXT", "MissingFpsError", "needs_shaping", "NO_SPACE_SCRIPTS",
+ "load_brand", "measured_level_dbfs", "AUDIO_MEDIA_EXT", "MEDIA_EXT", "MissingFpsError", "needs_shaping", "NO_SPACE_SCRIPTS",
"_odd_dimension_retry", "_on_signal", "_output_failed", "_OutputLock", "pad_filters", "parse_time", "_pid_dead",
"place_output", "_plan_at_exit", "_plan_inputs", "_PLAN_STRIP", "PLAN_VERSION", "PREFERRED_FAMILIES",
"print_json", "probe", "PROBE_TIMEOUT", "_progress_line", "read_text_or_die", "refuse_output_is_input",
diff --git a/scripts/_common/decision.py b/scripts/_common/decision.py
index d613f3e..349c9e5 100644
--- a/scripts/_common/decision.py
+++ b/scripts/_common/decision.py
@@ -9,9 +9,11 @@ from __future__ import annotations
import json
import math
import os
+import re
import argparse
+from fractions import Fraction
from pathlib import Path
-from typing import Any, Dict, List, Optional, Sequence
+from typing import Any, Dict, List, Optional, Sequence, Tuple
from _common.color import bt709_tag_args, _sdr_bt709
from _common.emit import die
from _common.runner import CODECS, STATE, ffmpeg_encoders
@@ -47,6 +49,60 @@ def add_pad_fill_args(parser: "argparse.ArgumentParser") -> None:
parser.add_argument("--pad-blur", type=int, default=20, help="blur radius in pixels for --pad-fill blur (default 20)")
+def aspect_ratio(value: Any) -> Optional[Fraction]:
+ """An aspect written W:H as an exact ratio, or None. fit.py --aspect and --export-timeline's
+ sequence frame read an aspect through this one function, so the export refuses `16/9` and
+ `2.39:1` where the render hands them to fit.py, which refuses them too (the timeline once
+ took them and wrote a sequence for a project whose render then failed in fit.py). Each side
+ is read with int(), as fit.py read it in 2.2.1, so a patch release refuses nothing fit.py
+ took: `+16:9` and `1_6:9` still parse, and `0:9` is a zero ratio, which frame_size() treats
+ as no aspect (2.2.1 kept the source frame). render.py's frame_from_preset() keeps 2.2.1's
+ looser match: it only picks the export preset's size."""
+ parts = str(value).split(":")
+ if len(parts) != 2:
+ return None
+ try:
+ return Fraction(int(parts[0]), int(parts[1]))
+ except (ValueError, ZeroDivisionError):
+ return None
+
+
+def _even(n: float) -> int:
+ v = int(round(n))
+ return v if v % 2 == 0 else v + 1
+
+
+def frame_size(ratio: Optional[Fraction], width: Optional[int], height: Optional[int],
+ src_w: int, src_h: int) -> Tuple[int, int]:
+ """The frame fit.py sizes for --aspect `ratio` (None: the source's own) and --width/--height,
+ from a src_w x src_h picture -- and so the sequence frame render.py --export-timeline writes
+ for the same project frame. Both sides given are used directly; one side takes the other from
+ the aspect. With neither, the canvas takes the new aspect without exceeding the source's
+ resolution in either dimension: a narrower/taller target than the source (e.g. 9:16 from a
+ 16:9 source) must be bounded by the source's HEIGHT, not its width -- bounding by width there
+ multiplies the height by src_ratio/ratio (a 1920x1080 source asked for 9:16 used to come out
+ 1920x3414, a ~3.16x upscale in both fit=pad and fit=crop, entirely unrequested). Every side is
+ rounded to even."""
+ src_ratio = Fraction(src_w, src_h) if src_h else None
+ if ratio is None:
+ ratio = src_ratio
+ if width and height:
+ return _even(width), _even(height)
+ if width:
+ out_w = _even(width)
+ return out_w, (_even(out_w / ratio) if ratio else width)
+ if height:
+ out_h = _even(height)
+ return (_even(out_h * ratio) if ratio else height), out_h
+ if ratio and src_ratio:
+ if ratio <= src_ratio:
+ out_h = _even(src_h)
+ return _even(out_h * ratio), out_h
+ out_w = _even(src_w)
+ return out_w, _even(out_w / ratio)
+ return _even(src_w), _even(src_h)
+
+
# x264 preset names mapped onto SVT-AV1's 0-13 speed scale (lower = slower / better)
SVT_PRESET = {"ultrafast": 12, "superfast": 11, "veryfast": 10, "faster": 9, "fast": 8, "medium": 6, "slow": 4, "slower": 3, "veryslow": 2, "placebo": 1}
diff --git a/scripts/_common/probe.py b/scripts/_common/probe.py
index 4fd5420..7313cea 100644
--- a/scripts/_common/probe.py
+++ b/scripts/_common/probe.py
@@ -73,8 +73,11 @@ def rms_envelope(samples: Sequence[float], step: int, *, full_blocks_only: bool
return env
+# the audio members of MEDIA_EXT: every audio file this skill reads, a wider set than the audio
+# outputs it writes (AUDIO_CODECS / is_audio_output() -- no .aif, .aiff, .caf or .wma there)
+AUDIO_MEDIA_EXT = {".wav", ".flac", ".mp3", ".m4a", ".aac", ".ogg", ".opus", ".aif", ".aiff", ".caf", ".wma"}
MEDIA_EXT = {".mp4", ".mov", ".mkv", ".webm", ".m4v", ".avi", ".ts", ".mts", ".m2ts", ".mxf", ".3gp", ".wmv", ".gif",
- ".wav", ".flac", ".mp3", ".m4a", ".aac", ".ogg", ".opus", ".aif", ".aiff", ".caf", ".wma", ".png", ".jpg", ".jpeg", ".webp"}
+ ".png", ".jpg", ".jpeg", ".webp"} | AUDIO_MEDIA_EXT
def decode_gray_frames(path: str, fps: float, width: int, height: int, *, start: float = 0.0,
diff --git a/scripts/_common/timeline.py b/scripts/_common/timeline.py
index 5c49103..ad339fe 100644
--- a/scripts/_common/timeline.py
+++ b/scripts/_common/timeline.py
@@ -13,18 +13,23 @@ either side of the new cut: the same frames blend over the same span, and the to
the rendered one.
Only what an editor timeline can carry is exported: clips (with in/out and speed), the
-transition between them, a music bed on its own audio track, and chapters as markers.
-Everything else in the project -- captions, graphics, overlays, the silence cut, fit/crop,
-audio processing, loudness, the export preset -- is listed under `not_exported`, never silently
-dropped.
+transition between them, a music bed on its own audio track, and chapters as markers, on a
+sequence of the project's frame (sized by the rule fit.py renders it with). FCPXML also states
+how a clip of another aspect fills that frame (`adjust-conform`); EDL and OTIO cannot.
+Everything else in the project -- captions, graphics, overlays, the silence cut, fit/crop, a
+reframe the format cannot state, audio processing, loudness, the export preset -- is listed
+under `not_exported`, never silently dropped.
"""
import json
+import math
import os
from fractions import Fraction
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
from xml.sax.saxutils import escape as _xml_escape
+from _common.decision import MissingFpsError, aspect_ratio, frame_size, parse_time
+
FORMATS = {".edl": "edl", ".fcpxml": "fcpxml", ".otio": "otio"}
# Common NTSC rates are stored as the exact ratio an editor expects, not as 29.97.
_NTSC = {23.976: Fraction(24000, 1001), 29.97: Fraction(30000, 1001), 59.94: Fraction(60000, 1001),
@@ -43,6 +48,9 @@ _NOT_EXPORTED = {
"brand": "brand styles",
"snap": "beat snapping (the timeline uses the project's in/out as written, before any snap)",
}
+# frame.fit -> the FCPXML adjust-conform type that fills the sequence the same way. blur has no
+# conform of its own: the picture is fitted whole, and its blurred background is not exported.
+_CONFORM = {"pad": "fit", "crop": "fill", "blur": "fit"}
_AUDIO_NOT_EXPORTED = ("replace", "voice", "denoise", "duck", "duck_amount", "duck_threshold", "duck_attack",
"duck_release", "gain", "stereo", "mono", "downmix", "stereo_widen", "effects",
"effects_volume", "stems", "music_volume", "fade_in", "fade_out", "music_fade_out",
@@ -63,6 +71,43 @@ def exact_rate(fps: float) -> Fraction:
return frac
+def _number(value: Any, what: str) -> float:
+ """A numeric project value (a rate, a size, a speed, a transition length) as a float, or a
+ TimelineError naming it -- never float()'s traceback."""
+ try:
+ number = float(value)
+ except (TypeError, ValueError):
+ number = math.nan
+ if not math.isfinite(number):
+ raise TimelineError(f"{what}: {value!r} is not a number")
+ return number
+
+
+def _seconds(value: Any, what: str, fps: Optional[float]) -> float:
+ """A project time read with the grammar render.py's own stages read it (time_arg()): seconds,
+ mm:ss, hh:mm:ss.ms, or hh:mm:ss:ff at `fps` or its own @fps. A value it refuses is a
+ TimelineError, so `--json` gets a `kind: input` document, not a traceback and empty stdout."""
+ try:
+ seconds = parse_time(str(value), fps)
+ except MissingFpsError as exc:
+ raise TimelineError(f"{what}: {exc}")
+ except ValueError as exc:
+ raise TimelineError(f"{what}: {exc} (use seconds, mm:ss, hh:mm:ss.ms, or hh:mm:ss:ff at the "
+ "source's fps or with an explicit @fps suffix)")
+ if not math.isfinite(seconds):
+ raise TimelineError(f"{what}: {value!r} is not a time")
+ return seconds
+
+
+def _picture(meta: Dict[str, Any]) -> Optional[Tuple[int, int]]:
+ """A source's displayed width x height (rotation metadata swaps them, as fit.py reads it)."""
+ video = meta.get("video") or {}
+ w, h = int(video.get("width") or 0), int(video.get("height") or 0)
+ if not (w and h):
+ return None
+ return (h, w) if video.get("rotation") in (90, -90, 270, -270) else (w, h)
+
+
def build(proj: Dict[str, Any], probes: Dict[str, Dict[str, Any]], rel) -> Dict[str, Any]:
"""The editor-neutral timeline: every time in frames at `rate`.
@@ -71,44 +116,76 @@ def build(proj: Dict[str, Any], probes: Dict[str, Dict[str, Any]], rel) -> Dict[
clips_in = proj.get("clips") or []
if not clips_in:
raise TimelineError("project.clips is empty")
+ for i, c in enumerate(clips_in):
+ if not isinstance(c, dict) or not c.get("src"):
+ raise TimelineError(f"clips[{i}]: no src")
frame = proj.get("frame") or {}
first = probes[rel(clips_in[0]["src"])]
- fps = frame.get("fps") or (first.get("video") or {}).get("fps")
+ frame_fps = _number(frame["fps"], "frame.fps") if frame.get("fps") else None
+ fps = frame_fps or (first.get("video") or {}).get("fps")
if not fps:
raise TimelineError("no frame rate: the first clip has no video stream and project.frame.fps is not set")
rate = exact_rate(float(fps))
- width = int(frame.get("width") or (first.get("video") or {}).get("width") or 1920)
- height = int(frame.get("height") or (first.get("video") or {}).get("height") or 1080)
+ # The sequence is the project's frame sized by fit.py's rule (render.py has already filled an
+ # aspect-only frame from the export preset), so {aspect 16:9, width 1920} is 1920x1080
+ # whatever the source's height -- the size the render's fit stage delivers. No frame at all
+ # is the first clip's own picture, which is what the render keeps.
+ aspect = frame.get("aspect")
+ ratio = aspect_ratio(aspect) if aspect else None
+ if aspect and ratio is None:
+ raise TimelineError(f"frame.aspect {aspect!r}: use W:H with whole numbers, like 16:9 (fit.py refuses it too)")
+ want_w, want_h = (int(_number(frame[k], f"frame.{k}")) if frame.get(k) else None for k in ("width", "height"))
+ if (want_w or 0) < 0 or (want_h or 0) < 0:
+ raise TimelineError(f"frame width/height must be positive, got {frame.get('width')!r} x {frame.get('height')!r}")
+ src_w, src_h = _picture(first) or (1920, 1080)
+ if ratio is None and not want_w and not want_h:
+ width, height = src_w, src_h
+ else:
+ width, height = frame_size(ratio, want_w, want_h, src_w, src_h)
+ fit_mode = str(frame.get("fit") or "pad")
+ if fit_mode not in _CONFORM:
+ raise TimelineError(f"frame.fit {frame.get('fit')!r}: use pad, crop or blur (fit.py's --fit)")
def fr(seconds: float) -> int:
return int(round(Fraction(seconds).limit_denominator(1000000) * rate))
trans = proj.get("transition") or {}
t_type = str(trans.get("type", "fade")) if len(clips_in) > 1 else "none"
- d_frames = 0 if t_type == "none" else fr(float(trans.get("duration", 0.5)))
+ d_frames = 0 if t_type == "none" else fr(_number(trans.get("duration", 0.5), "transition.duration"))
notes: List[str] = []
if d_frames and t_type != "fade":
notes.append(f'transition "{t_type}" is exported as a cross dissolve: editors do not share xfade\'s other patterns')
clips: List[Dict[str, Any]] = []
+ reframed: List[str] = []
for i, c in enumerate(clips_in):
src = rel(c["src"])
meta = probes[src]
dur = float(meta.get("duration") or 0.0)
- start = float(c.get("in") or 0.0)
- end = float(c["out"]) if c.get("out") is not None else dur
+ # in/out are on the source's own clock, as cut.py reads them: hh:mm:ss:ff at the
+ # source's fps, never frame.fps. A clip with no picture has no fps, and cut.py takes no
+ # other, so its hh:mm:ss:ff needs an @fps suffix here as in the render.
+ clip_fps = (meta.get("video") or {}).get("fps")
+ start = _seconds(c["in"], f"clips[{i}].in", clip_fps) if c.get("in") is not None else 0.0
+ if start < 0:
+ raise TimelineError(f"clips[{i}].in must not be negative, got {c['in']!r}")
+ end = _seconds(c["out"], f"clips[{i}].out", clip_fps) if c.get("out") is not None else dur
if end <= start:
raise TimelineError(f"clip {i}: out ({end:g}) is not after in ({start:g})")
- speed = float(c.get("speed") or 1.0)
+ speed = _number(c.get("speed") or 1.0, f"clips[{i}].speed")
if not speed > 0:
raise TimelineError(f"clip {i}: speed must be positive, got {c.get('speed')!r}")
+ picture = _picture(meta)
+ reframe = bool(picture and abs(Fraction(*picture) - Fraction(width, height)) > Fraction(1, 100))
+ if reframe:
+ reframed.append(os.path.basename(src))
src_in, src_out = fr(start), fr(end)
clips.append({"index": i, "src": os.path.abspath(src), "name": os.path.basename(src),
"src_in": src_in, "src_out": src_out, "speed": speed,
"length": int(round((src_out - src_in) / speed)),
"media_frames": fr(dur), "has_video": bool(meta.get("video")),
"has_audio": bool(meta.get("audio")),
- "width": (meta.get("video") or {}).get("width"), "height": (meta.get("video") or {}).get("height")})
+ "picture": picture, "fps": clip_fps, "reframed": reframe})
# Centre each dissolve on the cut: trim d/2 off the outgoing tail and d/2 off the incoming head.
half_a, half_b = d_frames // 2, d_frames - d_frames // 2
@@ -140,22 +217,44 @@ def build(proj: Dict[str, Any], probes: Dict[str, Dict[str, Any]], rel) -> Dict[
markers: List[Dict[str, Any]] = []
chapters = proj.get("chapters")
if isinstance(chapters, list):
- for ch in chapters:
- at = fr(float(ch.get("at", 0)))
+ for j, ch in enumerate(chapters):
+ # read as render.py's chapters stage reads it (metadata.py): a chapter list carries
+ # no fps, so hh:mm:ss:ff needs its @fps there and here alike
+ at = fr(_seconds(ch.get("at", 0), f"chapters[{j}].at", None))
if 0 <= at < total:
markers.append({"at": at, "title": str(ch.get("title") or "")})
elif chapters:
notes.append("chapters given as a file are not exported as markers; write them inline as [{at, title}]")
+ # A clip of another aspect than the sequence is a reframe: whether the file can state it
+ # depends on the format, so summary() adds it to not_exported per format (_reframe_note).
not_exported = [text for key, text in _NOT_EXPORTED.items() if proj.get(key)]
dropped_audio = sorted(k for k in _AUDIO_NOT_EXPORTED if audio.get(k) not in (None, False, "", 0) and k != "music")
if dropped_audio:
not_exported.append("audio processing: " + ", ".join(dropped_audio))
return {"name": Path(str(proj.get("output") or "timeline")).stem, "rate": rate, "width": width, "height": height,
+ "fit": fit_mode, "reframed": list(dict.fromkeys(reframed)),
"clips": clips, "transition": t_type if d_frames else "none", "transition_frames": d_frames,
"music": music, "markers": markers, "total": total, "notes": notes, "not_exported": not_exported}
+def _reframe_note(tl: Dict[str, Any], fmt: str) -> List[str]:
+ """What `not_exported` says about the clips whose picture has another aspect than the
+ sequence. FCPXML states each one's spatial conform (adjust-conform: fit for frame.fit pad,
+ fill for crop), so there only blur's blurred background is missing; EDL and OTIO have no
+ field for a conform at all, so the editor's own default decides and the caller is told."""
+ if not tl["reframed"]:
+ return []
+ names = ", ".join(tl["reframed"])
+ if fmt == "fcpxml":
+ if tl["fit"] != "blur":
+ return []
+ return [f"frame.fit blur's blurred background behind {names}: the FCPXML fits the picture into "
+ f"{tl['width']}x{tl['height']} (adjust-conform fit), leaving bars where the render blurs"]
+ return [f"the reframe to {tl['width']}x{tl['height']} (frame.fit {tl['fit']}) of {names}: "
+ "set the clips' spatial conform in the editor (fit = pad, fill = crop)"]
+
+
# ---------------------------------------------------------------------------------------- EDL
def _tc(frames: int, fps_int: int) -> str:
h, rem = divmod(frames, fps_int * 3600)
@@ -234,21 +333,40 @@ def _url(path: str) -> str:
def to_fcpxml(tl: Dict[str, Any]) -> str:
"""FCPXML 1.10 (Final Cut Pro 10.6+, DaVinci Resolve): one library/event/project, a spine of
asset-clips with cross dissolves between them, the music bed as a connected clip on lane -1
- of the first clip, chapters as chapter-markers. Speed is a linear timeMap on the clip."""
+ of the first clip, chapters as chapter-markers. Speed is a linear timeMap on the clip. The
+ sequence has its own format (r1); each video asset has its source's, and a clip of another
+ aspect states its spatial conform (adjust-conform) for frame.fit."""
rate = tl["rate"]
q = lambda s: _xml_escape(str(s), {'"': "&quot;"}) # noqa: E731
fd = _t(1, rate)
- res = [f' <format id="r1" name="FFVideoFormat{tl["height"]}p" frameDuration="{fd}" width="{tl["width"]}" height="{tl["height"]}"/>']
- ids: Dict[str, str] = {}
-
- def asset(path: str, name: str, frames: int, has_video: bool, has_audio: bool) -> str:
+ res: List[str] = []
+ ids: Dict[Any, str] = {} # a format's (width, height, frameDuration) or an asset's path -> its id
+
+ def fmt(width: int, height: int, frame_duration: str) -> str:
+ key = (width, height, frame_duration)
+ if key not in ids:
+ ids[key] = f"r{len(ids) + 1}"
+ res.append(f' <format id="{ids[key]}" name="FFVideoFormat{height}p" frameDuration="{frame_duration}" '
+ f'width="{width}" height="{height}"/>')
+ return ids[key]
+
+ seq_fmt = fmt(tl["width"], tl["height"], fd) # r1: the sequence's frame
+
+ def clip_fmt(c: Dict[str, Any]) -> str:
+ # An asset describes its own media, not the sequence: with every asset on r1, a 320x180
+ # file under a 1080x1920 frame claimed to be 1080x1920, and its conform had nothing to do.
+ if not (c["has_video"] and c["picture"]):
+ return seq_fmt
+ return fmt(c["picture"][0], c["picture"][1], _t(1, exact_rate(float(c["fps"]))) if c["fps"] else fd)
+
+ def asset(path: str, name: str, frames: int, has_video: bool, has_audio: bool, format_id: str = "") -> str:
if path in ids:
return ids[path]
- rid = f"r{len(ids) + 2}"
+ rid = f"r{len(ids) + 1}"
ids[path] = rid
res.append(f' <asset id="{rid}" name="{q(name)}" start="0s" duration="{_t(frames, rate)}" '
f'hasVideo="{int(has_video)}" hasAudio="{int(has_audio)}"'
- + (' format="r1"' if has_video else "") + ' audioSources="1" audioChannels="2">\n'
+ + (f' format="{format_id}"' if has_video else "") + ' audioSources="1" audioChannels="2">\n'
f' <media-rep kind="original-media" src="{q(_url(path))}"/>\n </asset>')
return rid
@@ -256,7 +374,8 @@ def to_fcpxml(tl: Dict[str, Any]) -> str:
d = tl["transition_frames"]
markers = list(tl["markers"])
for i, c in enumerate(tl["clips"]):
- rid = asset(c["src"], c["name"], c["media_frames"], c["has_video"], c["has_audio"])
+ cid = clip_fmt(c)
+ rid = asset(c["src"], c["name"], c["media_frames"], c["has_video"], c["has_audio"], cid)
if i > 0 and d:
spine.append(f' <transition name="Cross Dissolve" offset="{_t(c["record_in"] - c["trim_head"], rate)}" '
f'duration="{_t(d, rate)}"/>')
@@ -273,21 +392,28 @@ def to_fcpxml(tl: Dict[str, Any]) -> str:
else:
start_attr = _t(src_start, rate)
local0 = src_start if abs(c["speed"] - 1.0) <= 1e-6 else 0
- for mk in [m for m in markers if c["record_in"] <= m["at"] < c["record_in"] + c["visible"]]:
- inner.append(f' <chapter-marker start="{_t(local0 + mk["at"] - c["record_in"], rate)}" '
- f'duration="{fd}" value="{q(mk["title"])}"/>')
+ # the FCPXML 1.10 DTD's order inside an asset-clip: timeMap, then the adjust-* elements
+ # (adjust-conform), then anchored items (the music bed), then markers. Anything out of
+ # that order does not validate, and Final Cut validates what it imports against that DTD.
+ if c["reframed"]:
+ # the DTD reads a missing adjust-conform as fit, so crop has to say fill; fit is
+ # written too, so the file states the project's choice rather than a default
+ inner.append(f' <adjust-conform type="{_CONFORM[tl["fit"]]}"/>')
if i == 0 and tl["music"]:
m = tl["music"]
mid = asset(m["src"], m["name"], m["media_frames"], False, True)
inner.append(f' <asset-clip ref="{mid}" lane="-1" offset="{_t(local0, rate)}" name="{q(m["name"])}" '
f'start="0s" duration="{_t(m["length"], rate)}" audioRole="music"/>')
+ for mk in [m for m in markers if c["record_in"] <= m["at"] < c["record_in"] + c["visible"]]:
+ inner.append(f' <chapter-marker start="{_t(local0 + mk["at"] - c["record_in"], rate)}" '
+ f'duration="{fd}" value="{q(mk["title"])}"/>')
head = (f' <asset-clip ref="{rid}" offset="{_t(c["record_in"], rate)}" name="{q(c["name"])}" '
- f'start="{start_attr}" duration="{_t(c["visible"], rate)}" format="r1" tcFormat="NDF"')
+ f'start="{start_attr}" duration="{_t(c["visible"], rate)}" format="{cid}" tcFormat="NDF"')
spine.append(head + (">\n" + "\n".join(inner) + "\n </asset-clip>" if inner else "/>"))
out = ['<?xml version="1.0" encoding="UTF-8"?>', "<!DOCTYPE fcpxml>", '<fcpxml version="1.10">',
" <resources>", *res, " </resources>", " <library>", f' <event name="{q(tl["name"])}">',
f' <project name="{q(tl["name"])}">',
- f' <sequence format="r1" duration="{_t(tl["total"], rate)}" tcStart="0s" tcFormat="NDF">',
+ f' <sequence format="{seq_fmt}" duration="{_t(tl["total"], rate)}" tcStart="0s" tcFormat="NDF">',
" <spine>", *spine, " </spine>", " </sequence>", " </project>", " </event>",
" </library>", "</fcpxml>"]
return "\n".join(out) + "\n"
@@ -376,4 +502,5 @@ def summary(tl: Dict[str, Any], fmt: str) -> Dict[str, Any]:
"duration": round(float(Fraction(tl["total"]) / rate), 3), "frames": tl["total"],
"clips": len(tl["clips"]), "transition": tl["transition"],
"transition_frames": tl["transition_frames"], "music": bool(tl["music"]),
- "markers": len(tl["markers"]), "notes": tl["notes"], "not_exported": tl["not_exported"]}
+ "markers": len(tl["markers"]), "notes": tl["notes"],
+ "not_exported": tl["not_exported"] + _reframe_note(tl, fmt)}
diff --git a/scripts/_contract.py b/scripts/_contract.py
index 61b1300..5d91432 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -451,10 +451,13 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
"reencoded": {"type": "boolean"},
"lossless_alternative": {"type": ["string", "null"], "description": "when a lossless cut re-encoded because of the keyframe snap: the --start that would stream-copy instead, and how far it is from the request"}}
elif name == "join":
- extra = {"mode": {"enum": ["video", "audio"]}, "clips": {"type": "integer"}, "transition": {"type": "string"}, "expected_duration": {"type": "number"},
+ extra = {"mode": {"enum": ["video", "audio"]}, "clips": {"type": "integer"}, "transition": {"type": "string"},
+ "expected_duration": {"type": ["number", "null"], "description": "clip lengths minus the transitions; null under --dry-run while an input is pending (its length is unknown)"},
"sample_rate": {"type": "integer", "description": "audio mode only"}, "channels": {"type": "integer", "description": "audio mode only"},
"video": {"type": "boolean", "description": "false in audio mode: the output has no video stream"},
- "skipped": {"type": "array", "description": "inputs --on-missing skip left out: [{index, path, reason}] ([] when none were)"}}
+ "skipped": {"type": "array", "description": "inputs --on-missing skip left out: [{index, path, reason}] ([] when none were)"},
+ "pending": {"type": "array", "description": "--dry-run: inputs that do not exist yet, planned on as an earlier step's output and never also under skipped: [{index, path}] ([] otherwise)"},
+ "notes": {"type": "array", "items": {"type": "string"}, "description": "--dry-run with pending inputs: which ones, what a real run does if one is still missing (including a skip that leaves fewer than two inputs), whether the mode came from the extensions, and which planned numbers are the unmeasured stub's placeholders (a pending first clip's frame and rate, xfade offsets after a pending clip)"}}
elif name == "audio":
extra = {"video": {"type": "boolean", "description": "true when the input's video stream was copied; false for an audio output extension (extraction)"},
"audio_stream": {"type": "integer", "description": "which input audio stream was processed (--audio-stream)"},
diff --git a/scripts/fit.py b/scripts/fit.py
index dcccd21..e6114ea 100755
--- a/scripts/fit.py
+++ b/scripts/fit.py
@@ -39,20 +39,16 @@ import sys
from fractions import Fraction
from typing import List
-from _common import video_args, STATE, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, validate_color, x264_args, pad_filters, add_pad_fill_args, X264_PRESETS, time_arg, fmt_secs
+from _common import video_args, STATE, add_common, apply_common, emit, aac_args, aspect_ratio, cfr_args, default_output, die, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, validate_color, x264_args, pad_filters, add_pad_fill_args, X264_PRESETS, time_arg, fmt_secs, frame_size
BLUR_DARKEN = 0.15 # how much --fit blur dims the blurred background copy (eq brightness)
-ASPECT_PRESETS = {"16:9": Fraction(16, 9), "9:16": Fraction(9, 16), "1:1": Fraction(1, 1), "4:5": Fraction(4, 5), "4:3": Fraction(4, 3), "21:9": Fraction(21, 9)}
def parse_aspect(value: str) -> Fraction:
- if value in ASPECT_PRESETS:
- return ASPECT_PRESETS[value]
- try:
- w, h = value.split(":")
- return Fraction(int(w), int(h))
- except (ValueError, ZeroDivisionError):
+ # the one aspect grammar, shared with render.py's frame and --export-timeline's sequence
+ ratio = aspect_ratio(value)
+ if ratio is None:
die(f"bad aspect '{value}', use W:H like 16:9")
- return Fraction(1) # unreachable
+ return ratio
def atempo_chain(factor: float) -> str:
@@ -69,11 +65,6 @@ def atempo_chain(factor: float) -> str:
return ",".join(parts)
-def even(n: float) -> int:
- v = int(round(n))
- return v if v % 2 == 0 else v + 1
-
-
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("input")
@@ -192,31 +183,8 @@ def main() -> int:
# ---- aspect / size
if args.aspect or args.width or args.height:
- src_ratio = Fraction(sw, sh) if sh else None
- ratio = parse_aspect(args.aspect) if args.aspect else src_ratio
- if args.width and args.height:
- out_w, out_h = even(args.width), even(args.height)
- elif args.width:
- out_w = even(args.width)
- out_h = even(out_w / ratio) if ratio else args.width
- elif args.height:
- out_h = even(args.height)
- out_w = even(out_h * ratio) if ratio else args.height
- elif ratio and src_ratio:
- # No explicit --width/--height: size the canvas to the new aspect without exceeding
- # the source's own resolution in either dimension. A narrower/taller target than the
- # source (e.g. 9:16 from a 16:9 source) must be bounded by the source's HEIGHT, not
- # its width -- bounding by width there multiplies the height by src_ratio/ratio (a
- # 1920x1080 source asked for 9:16 used to come out 1920x3414, a ~3.16x upscale in
- # both fit=pad and fit=crop, entirely unrequested).
- if ratio <= src_ratio:
- out_h = even(sh)
- out_w = even(out_h * ratio)
- else:
- out_w = even(sw)
- out_h = even(out_w / ratio)
- else:
- out_w, out_h = even(sw), even(sh)
+ # the one sizing rule, shared with render.py --export-timeline's sequence frame
+ out_w, out_h = frame_size(parse_aspect(args.aspect) if args.aspect else None, args.width, args.height, sw, sh)
if args.fit == "crop":
vf.append(f"scale={out_w}:{out_h}:force_original_aspect_ratio=increase")
vf.append(f"crop={out_w}:{out_h}:(in_w-out_w)*{args.crop_x:g}:(in_h-out_h)*{args.crop_y:g}")
diff --git a/scripts/join.py b/scripts/join.py
index 9bc361f..cd93fbd 100755
--- a/scripts/join.py
+++ b/scripts/join.py
@@ -24,19 +24,48 @@ Examples:
Every input is checked before anything is joined: a missing, empty or unreadable segment is
refused with every problem named at once (kind input), never discovered one run at a time.
--on-missing skip joins the usable segments instead and lists the others under `skipped`.
+Under --dry-run a segment that does not exist yet is an earlier step's output: it is planned
+on and named under `pending` (never `skipped`), whichever --on-missing was given, and its
+extension stands in for what it will hold (an audio extension: no picture).
"""
import argparse
import os
import sys
-from typing import Any, Dict, List
+from typing import Any, Dict, List, Optional, Tuple
-from _common import STATE, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
+from _common import STATE, AUDIO_MEDIA_EXT, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, dry_run_input_pending, emit, ffmpeg_base, info, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
TRANSITIONS = ["fade", "dissolve", "wipeleft", "wiperight", "wipeup", "wipedown", "slideleft", "slideright",
"circleopen", "circleclose", "fadeblack", "fadewhite", "smoothleft", "smoothright", "radial", "none"]
LAYOUTS = {1: "mono", 2: "stereo", 6: "5.1", 8: "7.1"}
# inputs --on-missing skip dropped, reported in the success document (empty when nothing was)
STATE_SKIPPED: List[Dict[str, Any]] = []
+# --dry-run: inputs that do not exist yet, planned on as an earlier step's output
+STATE_PENDING: List[Dict[str, Any]] = []
+STATE_NOTES: List[str] = []
+
+
+def reported() -> Dict[str, Any]:
+ """The input bookkeeping both success documents carry: `skipped` is what the join left out,
+ `pending` what a dry run planned on without it existing; `notes` says the same in words."""
+ extra: Dict[str, Any] = {"skipped": list(STATE_SKIPPED), "pending": list(STATE_PENDING)}
+ if STATE_NOTES:
+ extra["notes"] = list(STATE_NOTES)
+ return extra
+
+
+def unpending_length(durs: List[float], d: float) -> Optional[float]:
+ """The joined length the clip durations add up to, or None while an input is pending: its
+ dry-run stub says 0 s, which would count the clip as nothing and still subtract a transition
+ for it (a negative length when nothing exists yet)."""
+ if STATE_PENDING:
+ return None
+ return round(sum(durs) - d * (len(durs) - 1), 3)
+
+
+def expected_text(expected: Optional[float]) -> str:
+ """unpending_length() as the stderr summary line puts it."""
+ return f"expected ~{expected:.3f}s" if expected is not None else "expected length unknown until the pending inputs exist"
def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
@@ -47,8 +76,11 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
for p, dur in zip(args.inputs, durs):
if d and dur <= d * 2 and not STATE.dry_run:
die(f"{p} is only {dur:.2f}s, too short for a {d:.2f}s crossfade; shorten --duration")
- rates = [m["audio"].get("sample_rate") or 48000 for m in metas]
- chans = [m["audio"].get("channels") or 2 for m in metas]
+ # a pending input's probe is the dry-run stub (rate and layout unmeasured): plan from the
+ # clips that exist, and fall back to 48 kHz stereo only when none does
+ known = [m for m in metas if not m.get("dry_run")] or metas
+ rates = [m["audio"].get("sample_rate") or 48000 for m in known]
+ chans = [m["audio"].get("channels") or 2 for m in known]
rate = args.sample_rate or rates[0]
channels = args.channels or max(chans)
layout = LAYOUTS.get(channels)
@@ -76,7 +108,7 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
prev = out
cmd += ["-filter_complex", ";".join(parts), "-map", "[aout]", "-vn"] + audio_codec_for(output) + [output]
run(cmd)
- expected = sum(durs) - d * (n - 1)
+ expected = unpending_length(durs, d)
r = probe(output, role="output")
a = r.get("audio") or {}
if not STATE.dry_run:
@@ -84,9 +116,9 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
die(f"{output} unexpectedly contains a video stream")
if a.get("sample_rate") != rate or a.get("channels") != channels:
die(f"{output} is {a.get('sample_rate')} Hz {a.get('channels')} ch, expected {rate} Hz {channels} ch")
- info(f"wrote {output} ({fmt_secs(r['duration'])}, expected ~{expected:.3f}s, audio {a.get('codec')} {channels}ch {rate}Hz, {n} clips, "
+ info(f"wrote {output} ({fmt_secs(r['duration'])}, {expected_text(expected)}, audio {a.get('codec')} {channels}ch {rate}Hz, {n} clips, "
+ ("crossfade" if d else "butt join") + ")")
- emit(output, mode="audio", skipped=list(STATE_SKIPPED), clips=n, transition=args.transition if d else "none", expected_duration=round(expected, 3),
+ emit(output, mode="audio", **reported(), clips=n, transition=args.transition if d else "none", expected_duration=expected,
sample_rate=rate, channels=channels, video=False)
return 0
@@ -118,17 +150,20 @@ def read_list(path: str) -> List[str]:
return entries
-def preflight(paths: List[str]) -> List[Dict[str, Any]]:
+def preflight(paths: List[str]) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]:
"""Every problem with every input, found before anything runs: missing, empty (0 bytes,
the usual trace of a TTS or download step that failed after creating its file), or not
readable as media. One ffprobe per file; nothing is decoded. Under --dry-run a file that
- does not exist yet is not a problem: in a planned pipeline it is an earlier step's output."""
+ does not exist yet is not a problem but pending (the second list, with a note): in a
+ planned pipeline it is an earlier step's output, and the plan keeps it."""
ffprobe = require_tool("ffprobe")
problems: List[Dict[str, Any]] = []
+ pending: List[Dict[str, Any]] = []
for i, p in enumerate(paths):
+ if dry_run_input_pending(p):
+ pending.append({"index": i, "path": p})
+ continue
if not os.path.exists(p):
- if STATE.dry_run:
- continue # a dry-run pipeline plans on outputs earlier steps have not written yet
problems.append({"index": i, "path": p, "reason": "missing"})
continue
if os.path.isdir(p):
@@ -143,7 +178,7 @@ def preflight(paths: List[str]) -> List[Dict[str, Any]]:
if proc.returncode != 0 or not kinds & {"audio", "video"}:
first = ((proc.stderr or "").strip().splitlines() or ["no audio or video stream"])[-1]
problems.append({"index": i, "path": p, "reason": f"unreadable: {first}"})
- return problems
+ return problems, pending
def main() -> int:
@@ -180,7 +215,7 @@ def main() -> int:
if not args.inputs:
die("give at least two clips (or --list FILE)")
skipped: List[Dict[str, Any]] = []
- problems = preflight(args.inputs)
+ problems, pending = preflight(args.inputs)
if problems and args.on_missing == "fail":
listing = "; ".join(f"#{p['index'] + 1} {p['path']}: {p['reason']}" for p in problems)
die(f"{len(problems)} of {len(args.inputs)} inputs cannot be joined: {listing}",
@@ -192,21 +227,53 @@ def main() -> int:
for p in problems:
info(f"skipping #{p['index'] + 1} {p['path']}: {p['reason']}")
STATE_SKIPPED[:] = skipped
+ STATE_PENDING[:] = pending
+ if pending:
+ # a dry run cannot know whether the file will be there: it plans on it, and says what a
+ # real run does if it still is not (the same --on-missing rule as any missing input --
+ # including the refusal when skipping would leave fewer than two inputs)
+ left = len(args.inputs) - len(pending)
+ if args.on_missing == "fail":
+ then = "refuses the join if one is still missing"
+ elif left >= 2:
+ then = "skips one that is still missing"
+ else:
+ then = (f"skips one that is still missing and refuses the join if that leaves fewer than two inputs "
+ f"(only {left} exist{'s' if left == 1 else ''} now)")
+ STATE_NOTES.append(f"{len(pending)} of {len(args.inputs) + len(skipped)} inputs do not exist yet and were planned on "
+ "as an earlier step's output: " + ", ".join(f"#{p['index'] + 1} {p['path']}" for p in pending)
+ + f"; a real run {then}")
if len(args.inputs) < 2:
die("give at least two clips" if not skipped else
f"only {len(args.inputs)} usable input(s) left after skipping {len(skipped)}; nothing to join",
skipped=skipped)
validate_color(args.pad_color, "--pad-color")
metas = [probe(p) for p in args.inputs]
- if all(not m.get("video") for m in metas):
+ # A pending input's probe is the dry-run stub, whose (0x0) video stream says nothing about the
+ # file an earlier step will write. Its extension stands in: a step names its output for what
+ # it holds, so an audio extension (.wav, .m4a, .aiff, ... -- every audio file this skill
+ # reads, not only the ones it writes) is taken for a file without a picture and any other for
+ # one with a picture. Measured and pending inputs then meet the same rules a real run
+ # applies: all without a picture is an audio join, a mix is refused.
+ pictured = [(os.path.splitext(p)[1].lower() not in AUDIO_MEDIA_EXT) if m.get("dry_run") else bool(m.get("video"))
+ for p, m in zip(args.inputs, metas)]
+ if all(m.get("dry_run") for m in metas):
+ STATE_NOTES.append(f"no input exists yet: {'a video' if any(pictured) else 'an audio'} join was planned from the file extensions")
+ if not any(pictured):
for p, m in zip(args.inputs, metas):
if not m.get("audio"):
die(f"{p} has neither a video nor an audio stream")
return join_audio(args, metas)
- for p, m in zip(args.inputs, metas):
- if not m.get("video"):
- others = [q for q, mm in zip(args.inputs, metas) if mm.get("video")]
- die(f"{p} has no video stream" + (f" while {others[0]} has one; join audio with audio or give every clip a picture" if others else ""))
+ for p, m, pic in zip(args.inputs, metas, pictured):
+ if not pic:
+ # name an input measured to have a picture first; a pending one only as expected to
+ measured = [q for q, mm in zip(args.inputs, metas) if mm.get("video") and not mm.get("dry_run")]
+ planned = [q for q, mm, pc in zip(args.inputs, metas, pictured) if pc and mm.get("dry_run")]
+ other = (f"{measured[0]} has one" if measured else
+ f"{planned[0]}, which does not exist yet, is expected to have one (its extension is not an audio one)" if planned else "")
+ die((f"{p} does not exist yet, and its audio extension says it will have no video stream" if m.get("dry_run")
+ else f"{p} has no video stream")
+ + (f" while {other}; join audio with audio or give every clip a picture" if other else ""))
first = metas[0]["video"]
fw, fh = first["width"], first["height"]
if first.get("rotation") in (90, -90, 270, -270):
@@ -263,8 +330,9 @@ def main() -> int:
hdr_meta = next((m for m in metas if (m.get("video") or {}).get("bt2020_or_hdr")), None)
pixfmt = "yuv420p10le" if hdr_meta else "yuv420p"
# the audio-only join keeps the widest layout; the video join used to force stereo and
- # silently dropped the centre/LFE of 5.1 material
- chans = [(m.get("audio") or {}).get("channels") or 2 for m in metas]
+ # silently dropped the centre/LFE of 5.1 material. A pending input's stub has no measured
+ # layout: plan from the inputs that exist, as join_audio() does.
+ chans = [(m.get("audio") or {}).get("channels") or 2 for m in ([m for m in metas if not m.get("dry_run")] or metas)]
channels = args.channels or max(chans)
layout = LAYOUTS.get(channels)
if layout is None:
@@ -289,14 +357,22 @@ def main() -> int:
parts.append(f"[{aprev}][a{i}]acrossfade=d={d:g}:c1=tri:c2=tri[{aout}]")
vprev, aprev = vout, aout
+ # A pending input's stub is unmeasured (0x0, 0 fps, 0 s -- #77 keeps it honest rather than
+ # plausible), and the planned command is built on it: say which numbers are placeholders.
+ if metas[0].get("dry_run") and not (args.width and args.height and args.fps):
+ STATE_NOTES.append(f"the first input is pending, so the planned frame and rate ({w}x{h} @ {fps:g}fps) are "
+ "placeholders: a real run takes them from that file")
+ if d and any(m.get("dry_run") for m in metas[:-1]):
+ STATE_NOTES.append("the planned xfade offsets count each pending input as 0 s long: a real run offsets "
+ "by its measured length")
output = args.output or default_output(args.inputs[0], "joined", "mp4")
cmd += ["-filter_complex", ";".join(parts), "-map", "[vout]", "-map", "[aout]"]
cmd += video_args(hdr_meta or metas[0], args.crf, args.preset) + aac_args() + [output]
run(cmd)
- expected = sum(durs) - d * (n - 1)
+ expected = unpending_length(durs, d)
r = probe(output, role="output")
- info(f"wrote {output} ({fmt_secs(r['duration'])}, expected ~{expected:.3f}s, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
- emit(output, mode="video", skipped=list(STATE_SKIPPED), clips=n, transition=args.transition, expected_duration=round(expected, 3),
+ info(f"wrote {output} ({fmt_secs(r['duration'])}, {expected_text(expected)}, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
+ emit(output, mode="video", **reported(), clips=n, transition=args.transition, expected_duration=expected,
dropped_non_av_streams=any(m.get("subtitle_streams") or m.get("data_streams") for m in metas))
return 0
diff --git a/scripts/render.py b/scripts/render.py
index cabc336..41e2a0b 100755
--- a/scripts/render.py
+++ b/scripts/render.py
@@ -64,7 +64,7 @@ from typing import Any, Dict, List, Optional, Sequence
from export import PRESETS, PLATFORM_OF
from _platforms import PLATFORMS, caption_defaults, resolve as resolve_platform
-from _common import STATE, add_common, brand_caption_style, load_brand, apply_common, child_args, die, emit, info, probe, run_tool, place_output, refuse_output_is_input, _check_existing_output, _check_output_path, fingerprint, PLAN_VERSION, ffmpeg_version
+from _common import STATE, add_common, aspect_ratio, brand_caption_style, load_brand, apply_common, child_args, die, emit, info, probe, run_tool, place_output, refuse_output_is_input, _check_existing_output, _check_output_path, fingerprint, PLAN_VERSION, ffmpeg_version
import subprocess
from _contract import CONTRACT_VERSION
from batch import file_key
@@ -394,7 +394,8 @@ def render_pack(names: List[str], args) -> int:
def check_keys(obj: Any, schema: str, label: str) -> None:
- """Refuse an unrecognised key, naming the object, the key and the nearest valid one."""
+ """Refuse an unrecognised key, naming the object, the key and the nearest valid one. A value
+ that is not an object is require_object()'s to refuse, and only where the run reads it."""
if not isinstance(obj, dict):
return
valid = OBJECT_KEYS[schema]
@@ -406,25 +407,76 @@ def check_keys(obj: Any, schema: str, label: str) -> None:
else f" (valid keys: {', '.join(sorted(valid))})"))
-def validate_project(proj: Dict[str, Any]) -> None:
+def require_object(obj: Any, label: str) -> None:
+ """Refuse a value that is not an object where the run reads it with .get(): "export": "reels"
+ in a render, a string frame, a clip written as a bare path. 2.2.1 died there with a traceback
+ and nothing on stdout under --json. null, false and {} still ask for nothing."""
+ if obj and not isinstance(obj, dict):
+ die(f"{label}: must be an object {{...}}, got {type(obj).__name__} {obj!r:.60}")
+
+
+# the stage that first reads each section; --stop-after before it means the run never reads it
+READ_AT = {"transition": "join", "silence": "silence", "fit": "fit", "captions": "captions",
+ "graphics": "graphics", "overlays": "overlays", "loudness": "loudness", "export": "export",
+ "check": "check"}
+_READ_ORDER = ["clips", "join", "silence", "fit", "captions", "graphics", "overlays", "audio", "loudness", "export", "check"]
+
+
+def _reaches(stage: str, stop_after: Optional[str]) -> bool:
+ return not stop_after or _READ_ORDER.index(stage) <= _READ_ORDER.index(stop_after)
+
+
+def validate_project(proj: Dict[str, Any], timeline: bool = False, stop_after: Optional[str] = None) -> None:
+ """Every project error the run would hit, before the first ffmpeg call. A value that is not
+ an object is refused only where this run reads it (require_object()): a full render reads
+ every stage's section, a transition only between two or more clips and a clip's snap only
+ on a clip it cuts; --export-timeline (`timeline`) reads the clips, frame, audio and a
+ transition between clips, and only names the other sections in not_exported. 2.2.1 rendered
+ a single clip with "transition": "none" and exported "captions": "subs.srt", reading neither,
+ and a patch release does not refuse them."""
check_keys(proj, "project", "project")
+ clips = proj.get("clips")
+ # the join (or the timeline's dissolves) reads the transition; an audiogram render joins nothing
+ several = isinstance(clips, list) and len(clips) > 1 and (timeline or not proj.get("audiogram"))
+ read = {"frame", "audio"} | ({"transition"} if several else set())
+ if not timeline:
+ read |= {"silence", "audiogram", "captions", "loudness", "fit", "export", "check", "snap"}
+ # --stop-after ends the run before later stages read their sections: 2.2.1 completed a
+ # `--stop-after fit` preview of a project with "captions": "subs.srt", never reading it
+ read = {n for n in read if n not in READ_AT or _reaches(READ_AT[n], stop_after)}
for name in ("frame", "transition", "silence", "audiogram", "captions", "audio", "loudness", "fit", "export", "check", "snap"):
+ if name in read:
+ require_object(proj.get(name), name)
check_keys(proj.get(name), name, name)
check_keys((proj.get("audio") or {}).get("stems"), "audio.stems", "audio.stems")
if isinstance(proj.get("chapters"), list):
# Every other project error is raised here, before the first ffmpeg call; a chapter typo
# found inside the last stage costs a whole render and leaves an unchaptered file behind.
for i, item in enumerate(proj["chapters"]):
- check_keys(item, "chapters[]", f"chapters[{i}]")
+ if isinstance(item, dict):
+ check_keys(item, "chapters[]", f"chapters[{i}]")
if not isinstance(item, dict) or item.get("at") is None or not str(item.get("title") or "").strip():
die(f'chapters[{i}]: needs {{"at": TIME, "title": STR}}')
for name in ("clips", "graphics", "overlays"):
items = proj.get(name)
- if isinstance(items, list):
- for i, item in enumerate(items):
- check_keys(item, f"{name}[]", f"{name}[{i}]")
- if name == "clips" and isinstance(item, dict) and item.get("snap") is not None:
- check_keys(item["snap"], "snap", f"clips[{i}].snap")
+ if name == "clips" or (not timeline and _reaches(READ_AT[name], stop_after)): # the export reads only the clips
+ if items and not isinstance(items, list):
+ die(f"{name}: must be a list of objects [{{...}}], got {type(items).__name__} {items!r:.60}")
+ for i, item in enumerate(items or []):
+ if not isinstance(item, dict):
+ die(f"{name}[{i}]: must be an object {{...}}, got {type(item).__name__} {item!r:.60}")
+ for i, item in enumerate(items if isinstance(items, list) else []):
+ check_keys(item, f"{name}[]", f"{name}[{i}]")
+ if name != "clips":
+ continue
+ # every clip stage starts from rel(c["src"]): without one it was a KeyError traceback
+ if not item.get("src"):
+ die(f"clips[{i}]: no src")
+ if item.get("snap") is not None:
+ # the render reads a clip's snap only on a clip it cuts (in/out), the export never
+ if not timeline and (item.get("in") is not None or item.get("out") is not None):
+ require_object(item["snap"], f"clips[{i}].snap")
+ check_keys(item["snap"], "snap", f"clips[{i}].snap")
_LAST_DOC: Dict[str, Any] = {} # the JSON document the most recent sh() child printed
@@ -703,7 +755,11 @@ def frame_from_preset(frame: Dict[str, Any], export: Dict[str, Any]) -> None:
"""Fill frame.width/height from the export preset when the project gave only an aspect.
Eval 7 (j08 twice, e01 by hand): "frame": {"aspect": "9:16"} with a reels export fitted a
1280x720 source to 406x720, captions were burned at that size, and export.py upscaled them
- soft. A preset that names a delivery frame of the same aspect is that frame."""
+ soft. A preset that names a delivery frame of the same aspect is that frame.
+ The match is 2.2.1's, looser than aspect_ratio(): `16/9` and `1.78:1` still take a 16:9
+ preset's size. A render that hands such a frame.aspect to fit.py fails there, as in 2.2.1;
+ one whose own fit.aspect replaces it, or that stops before fit, is sized by this match, and
+ a patch release does not change a render 2.2.1 completed."""
if not frame.get("aspect") or frame.get("width") or frame.get("height"):
return
preset = PRESETS.get(str(export.get("preset") or ""), {})
@@ -733,8 +789,20 @@ def export_timeline(proj: Dict[str, Any], rel, dest: str) -> int:
if not os.path.exists(path):
die(f"timeline source not found: {path}")
probes[path] = probe(path) # a timeline needs real durations and rates, dry run or not
+ # the sequence frame is the one the render would deliver: an aspect-only frame takes its size
+ # from the export preset exactly as the render's own frame does. An export that is not an
+ # object names no preset here: 2.2.1's export never read it, and validate_project() refuses
+ # it only in the render, which reads it.
+ frame = dict(proj.get("frame") or {})
+ export = proj.get("export")
+ frame_from_preset(frame, export if isinstance(export, dict) else {})
+ fit = proj.get("fit")
+ if isinstance(fit, dict) and fit.get("aspect") and frame.get("aspect") and aspect_ratio(frame["aspect"]) is None:
+ # the render hands fit.py the fit object's own aspect, never this one (only the preset
+ # match above reads it), and completes: the sequence is the frame's size without it
+ del frame["aspect"]
try:
- tl = tlmod.build(proj, probes, rel)
+ tl = tlmod.build(dict(proj, frame=frame), probes, rel)
except tlmod.TimelineError as exc:
die(f"--export-timeline: {exc}", kind="input")
text = tlmod.WRITERS[fmt](tl)
@@ -834,7 +902,7 @@ def main() -> int:
if "plan_version" in proj:
return execute_plan(proj, os.path.abspath(args.project))
base = Path(args.project).resolve().parent
- validate_project(proj)
+ validate_project(proj, timeline=bool(args.export_timeline), stop_after=args.stop_after)
if args.write_project:
# Only the filled project: the point is to edit it before rendering, so nothing runs.
try:
@@ -1059,9 +1127,21 @@ def main() -> int:
fit.setdefault("aspect", frame["aspect"])
if frame.get("fit"):
fit.setdefault("fit", frame["fit"])
- if frame.get("width") and len(parts) == 1:
+ # Several clips were joined by join.py: into frame.width x frame.height when the frame gives
+ # both, else at the FIRST clip's aspect. A frame of an aspect and ONE side then reached fit.py
+ # as the aspect alone, which fit.py bounds by the joined picture: {aspect 9:16, width 1080}
+ # over two 16:9 clips rendered 342x608, where one clip (and --export-timeline's sequence) is
+ # 1080x1920. Only that case gets the frame's side. A frame with both sides is already the
+ # join's size, and a project fit object with its own width, height or another aspect renders
+ # as 2.2.1 rendered it: filling the frame's sides in there overrode the fit object's aspect
+ # (fit {width 540} under a 1080x1920 frame came out 540x1920).
+ own = proj.get("fit") or {}
+ one_side = bool(frame.get("width")) != bool(frame.get("height"))
+ sized = len(parts) == 1 or bool(frame.get("aspect") and one_side and not own.get("width") and not own.get("height")
+ and own.get("aspect") in (None, frame.get("aspect")))
+ if frame.get("width") and sized:
fit.setdefault("width", frame["width"])
- if frame.get("height") and len(parts) == 1:
+ if frame.get("height") and sized:
fit.setdefault("height", frame["height"])
if frame.get("fps") and len(parts) == 1:
fit.setdefault("fps", frame["fps"])
diff --git a/tests/test_editing.py b/tests/test_editing.py
index bc54a1b..44e65e8 100644
--- a/tests/test_editing.py
+++ b/tests/test_editing.py
@@ -252,6 +252,28 @@ class EditingTests(MediaFixtures):
self.assertClose(m["duration"], 4.0, 0.15)
self.assertEqual(m["video"]["width"], m["video"]["height"])
+ def test_fit_aspect_reads_each_side_as_2_2_1_did(self):
+ """fit.py --aspect reads each side with int(), as 2.2.1 did: 2.2.2's shared parser briefly
+ refused `+9:16` and `1_6:9`, which int() takes, and `0:9`, which 2.2.1 ran as no aspect
+ (the source frame). A patch release refuses nothing fit.py took."""
+ from fractions import Fraction
+ from _common import aspect_ratio
+ self.assertEqual(aspect_ratio("+9:16"), Fraction(9, 16))
+ self.assertEqual(aspect_ratio("1_6:9"), Fraction(16, 9))
+ self.assertEqual(aspect_ratio(" 16 : 9 "), Fraction(16, 9))
+ self.assertEqual(aspect_ratio("0:9"), Fraction(0))
+ for bad in ("16/9", "2.39:1", "16:0", "wide", "1:2:3"):
+ self.assertIsNone(aspect_ratio(bad), bad)
+ # self.src is 1280x720: 9:16 bounded by its height is 406x720. A zero ratio is no aspect,
+ # so --width alone is a square, as 2.2.1 (whose else-branch took the width) made it
+ for aspect, extra, size in (("+9:16", (), (406, 720)), ("0:9", ("--width", "640"), (640, 640))):
+ with self.subTest(aspect):
+ out = OUT / f"fit_asp_{aspect.strip('+').replace(':', '_')}.mp4"
+ doc = json.loads(script("fit.py", self.src, "--aspect", aspect, *extra, "-o", out, "--fast", "--json", "--overwrite").stdout)
+ self.assertEqual(doc["status"], "completed")
+ m = probe(str(out))
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), size)
+
def test_fit_aspect_only_never_upscales_past_source_resolution(self):
"""With only --aspect given (no --width/--height), the "elif ratio and src_ratio" branch
used to bound a narrower/taller target by the source's WIDTH, not its height -- so a
@@ -983,6 +1005,105 @@ class EditingTests(MediaFixtures):
"--json", "-o", d / "clean.wav").stdout)
self.assertEqual(doc["skipped"], [])
+ def test_join_dry_run_plans_on_pending_segments(self):
+ """Under --dry-run a segment that does not exist yet is an earlier step's output. Its
+ probe is the dry-run stub, whose (0x0) video stream used to decide the mode: one pending
+ TTS line failed the plan with "tts_01.wav has no video stream while tts_03.wav has one",
+ and a list of nothing but pending .wav lines planned a libx264 join into voice.wav. A
+ pending segment's extension now stands in for its streams, rate and layout come from the
+ segments that exist, and it is planned on and named under `pending` -- never `skipped`
+ -- in both --on-missing modes; a real run still decides on the file as it then is."""
+ d = OUT / "join_pending"
+ d.mkdir(exist_ok=True)
+ for i in (1, 2):
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", f"sine=frequency={300 * i}:sample_rate=24000", "-t", "1.5", d / f"tts_0{i}.wav")
+ for gone in ("tts_03.wav", "intro.mp4", "later.wav"):
+ (d / gone).unlink(missing_ok=True) # what the earlier steps have not written yet
+ (d / "parts.txt").write_text("tts_01.wav\ntts_02.wav\ntts_03.wav\n", encoding="utf-8")
+ # the pending line first as well: the stub's (unmeasured) rate and layout must not become the plan's
+ (d / "first.txt").write_text("tts_03.wav\ntts_01.wav\ntts_02.wav\n", encoding="utf-8")
+ for lst, at in (("parts.txt", 2), ("first.txt", 0)):
+ for mode, then in (("fail", "refuses the join if one is still missing"), ("skip", "skips one that is still missing")):
+ proc = script("join.py", "--list", d / lst, "--transition", "none", "--on-missing", mode,
+ "--dry-run", "--json", "-o", d / "voice_plan.wav")
+ doc = json.loads(proc.stdout)
+ case = f"{lst} --on-missing {mode}"
+ self.assertEqual((doc["status"], doc["mode"], doc["clips"]), ("completed", "audio", 3), case)
+ self.assertEqual((doc["sample_rate"], doc["channels"]), (24000, 1), f"{case}: planned from the segments that exist, not the stub")
+ self.assertEqual([(p["index"], Path(p["path"]).name) for p in doc["pending"]], [(at, "tts_03.wav")], case)
+ self.assertEqual(doc["skipped"], [], f"{case}: a pending segment is planned on, not left out")
+ self.assertIsNone(doc["expected_duration"], f"{case}: the pending line's length is unknown, not 0 s")
+ self.assertIn("tts_03.wav", doc["commands"][0])
+ self.assertNotIn("libx264", doc["commands"][0])
+ self.assertTrue(any("tts_03.wav" in n and f"a real run {then}" in n for n in doc["notes"]), doc["notes"])
+ self.assertIn("tts_03.wav does not exist yet", proc.stderr)
+ self.assertFalse((d / "voice_plan.wav").exists())
+ # skipping the pending line would leave one input: the note predicts the refusal the real run gives
+ (d / "two.txt").write_text("tts_01.wav\ntts_03.wav\n", encoding="utf-8")
+ doc = json.loads(script("join.py", "--list", d / "two.txt", "--transition", "none", "--on-missing", "skip",
+ "--dry-run", "--json", "-o", d / "two.wav").stdout)
+ self.assertEqual(doc["status"], "completed")
+ self.assertTrue(any("refuses the join if that leaves fewer than two inputs" in n for n in doc["notes"]), doc["notes"])
+ proc = script("join.py", "--list", d / "two.txt", "--transition", "none", "--on-missing", "skip",
+ "--json", "-o", d / "two.wav", expect_fail=True)
+ self.assertIn("nothing to join", json.loads(proc.stdout)["error"]["message"])
+ # a pending .mp4 is expected to hold a picture: next to measured audio it is the mix a real
+ # run refuses, and a pending .wav next to measured video likewise
+ for inputs, out, says in (((d / "tts_01.wav", d / "intro.mp4"), "mix.wav",
+ "tts_01.wav has no video stream while " + str(d / "intro.mp4") + ", which does not exist yet, is expected to have one"),
+ ((self.src, d / "later.wav"), "mix.mp4",
+ str(d / "later.wav") + " does not exist yet, and its audio extension says it will have no video stream while " + str(self.src) + " has one")):
+ proc = script("join.py", *inputs, "--transition", "none", "--dry-run", "--json", "-o", d / out, expect_fail=True)
+ err = json.loads(proc.stdout)["error"]
+ self.assertEqual(err["kind"], "input")
+ self.assertIn(says, err["message"])
+ # a mix names the input measured to have a picture, never a pending file as if it had one
+ (d / "mixed.txt").write_text(f"tts_01.wav\nnot_yet.mp4\n{self.src}\n", encoding="utf-8")
+ proc = script("join.py", "--list", d / "mixed.txt", "--dry-run", "--json", "-o", d / "mixed.mp4", expect_fail=True)
+ err = json.loads(proc.stdout)["error"]
+ self.assertEqual(err["kind"], "input")
+ self.assertIn(f"while {self.src} has one", err["message"])
+ self.assertNotIn("not_yet.mp4", err["message"])
+ # nothing written yet at all: the extensions decide, and .wav plans an audio join
+ (d / "later.txt").write_text("gen_01.wav\ngen_02.wav\n", encoding="utf-8")
+ doc = json.loads(script("join.py", "--list", d / "later.txt", "--dry-run", "--json", "-o", d / "later.wav").stdout)
+ self.assertEqual((doc["mode"], len(doc["pending"]), doc["expected_duration"]), ("audio", 2, None))
+ self.assertFalse(any("libx264" in c for c in doc["commands"]), doc["commands"])
+ self.assertIn("acrossfade", doc["commands"][0])
+ self.assertTrue(any("file extensions" in n for n in doc["notes"]), doc["notes"])
+ # every audio extension this skill reads counts, not only the ones it writes: a pending
+ # .aiff (macOS `say -o`) was taken for a file with a picture and the plan refused as a mix
+ for i in (1, 2):
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi",
+ "-i", f"sine=frequency={300 * i}:sample_rate=24000", "-t", "1.5", d / f"say_0{i}.aiff")
+ (d / "say_03.aiff").unlink(missing_ok=True)
+ (d / "say.txt").write_text("say_01.aiff\nsay_02.aiff\nsay_03.aiff\n", encoding="utf-8")
+ doc = json.loads(script("join.py", "--list", d / "say.txt", "--dry-run", "--json", "-o", d / "say.wav").stdout)
+ self.assertEqual((doc["status"], doc["mode"], len(doc["pending"])), ("completed", "audio", 1))
+ (d / "say_new.txt").write_text("new_01.aiff\nnew_02.caf\n", encoding="utf-8")
+ doc = json.loads(script("join.py", "--list", d / "say_new.txt", "--dry-run", "--json", "-o", d / "say.wav").stdout)
+ self.assertEqual(doc["mode"], "audio", "nothing written yet: .aiff and .caf plan an audio join")
+ # a pending FIRST clip of a video join: its stub is unmeasured, so the planned frame, rate
+ # and xfade offsets built on it are named as placeholders, and the audio layout comes
+ # from the clips that exist (the stub's 0 channels made a mono join stereo)
+ (d / "cam_later.mp4").unlink(missing_ok=True)
+ proc = script("join.py", d / "cam_later.mp4", self.src, self.src, "--dry-run", "--json", "-o", d / "cams.mp4")
+ doc = json.loads(proc.stdout)
+ self.assertEqual((doc["status"], doc["mode"]), ("completed", "video"))
+ self.assertTrue(any("first input is pending" in n and "placeholders" in n for n in doc["notes"]), doc["notes"])
+ self.assertTrue(any("xfade offsets" in n for n in doc["notes"]), doc["notes"])
+ self.assertIn("channel_layouts=mono", doc["commands"][0])
+ self.assertNotIn("channel counts differ", proc.stderr)
+ doc = json.loads(script("join.py", self.src, self.src, d / "cam_later.mp4", "--dry-run", "--json", "-o", d / "cams.mp4").stdout)
+ self.assertFalse(any("placeholders" in n or "xfade offsets" in n for n in doc["notes"]),
+ "a pending LAST clip changes neither the frame nor an offset")
+ # the real run is unchanged: the missing line is skipped (or refused), never pending
+ doc = json.loads(script("join.py", "--list", d / "parts.txt", "--transition", "none", "--on-missing", "skip",
+ "--json", "-o", d / "voice.wav").stdout)
+ self.assertEqual((doc["clips"], doc["pending"], doc["expected_duration"]), (2, [], 3.0))
+ self.assertEqual([Path(p["path"]).name for p in doc["skipped"]], ["tts_03.wav"])
+
def test_join_audio_only_inputs(self):
"""WAV + M4A + MP3 of different rates and channel counts join as audio; video containers and mixed inputs are refused."""
st = OUT / "j_stereo44.m4a"
diff --git a/tests/test_orchestration.py b/tests/test_orchestration.py
index a3bcf23..bfffc47 100644
--- a/tests/test_orchestration.py
+++ b/tests/test_orchestration.py
@@ -69,6 +69,36 @@ class OrchestrationTests(MediaFixtures):
self.assertIsNone(data.get("snap"))
self.assertFalse(any("--snap" in c for c in data["commands"]))
+ def test_render_dry_run_joins_a_trimmed_and_an_untrimmed_audio_clip(self):
+ """A trimmed clip is a cut the dry run never wrote, so join.py saw its probe stub (which
+ always has a video stream) next to the untrimmed, audio-only clip and refused the plan as
+ an audio/video mix. The join is planned as audio, from the clip that exists."""
+ proj = OUT / "render_pending_audio.json"
+ proj.write_text(json.dumps({
+ "output": str(OUT / "render_pending_audio.wav"),
+ "transition": {"type": "none"},
+ "clips": [{"src": str(self.mic), "in": "0.5", "out": "2"}, {"src": str(self.mic)}],
+ }), encoding="utf-8")
+ data = json.loads(script("render.py", proj, "--dry-run", "--json").stdout)
+ self.assertEqual((data["status"], data["dry_run"], data["stages"]), ("completed", True, ["clips", "join"]))
+ join_cmd = [c for c in data["commands"] if "concat=n=2:v=0:a=1" in c]
+ self.assertEqual(len(join_cmd), 1, data["commands"])
+ self.assertNotIn("libx264", join_cmd[0])
+ self.assertTrue(any("clip00.wav does not exist yet" in c for c in data["commands"]), data["commands"])
+ # a trimmed video clip (clip00.mp4, pending) next to an untrimmed .wav is the audio/video
+ # mix the real run refuses: the plan refuses it too, for that reason, not as an audio join
+ # that cannot fill render's joined.mp4
+ proj = OUT / "render_pending_mix.json"
+ proj.write_text(json.dumps({
+ "output": str(OUT / "render_pending_mix.mp4"),
+ "transition": {"type": "none"},
+ "clips": [{"src": str(self.src), "in": "0.5", "out": "2"}, {"src": str(self.mic)}],
+ }), encoding="utf-8")
+ err = json.loads(script("render.py", proj, "--dry-run", "--json", expect_fail=True).stdout)["error"]
+ self.assertIn(f"{self.mic} has no video stream while ", err["message"])
+ self.assertIn("clip00.mp4, which does not exist yet, is expected to have one", err["message"])
+ self.assertNotIn("give -o an audio extension", err["message"])
+
def test_zero_or_negative_fps_refused_across_every_cfr_script(self):
"""--fps flows straight into cfr_args(meta, args.fps) / a `fps or source_fps or 30.0`
@@ -773,6 +803,25 @@ class OrchestrationTests(MediaFixtures):
frame = {"aspect": "9:16", "width": 540}
render.frame_from_preset(frame, {"preset": "reels"})
self.assertEqual(frame, {"aspect": "9:16", "width": 540})
+ # 2.2.1's match, kept: fit.py refuses "16/9" and "1.78:1", but a render whose own
+ # fit.aspect replaces them (or that stops before fit) was sized for delivery by it, and
+ # 2.2.2's first draft dropped that frame without a word (captions burned at source size)
+ for aspect in ("16/9", "1.78:1", "16:9"):
+ with self.subTest(aspect=aspect):
+ frame = {"aspect": aspect}
+ render.frame_from_preset(frame, {"preset": "youtube"})
+ self.assertEqual((frame.get("width"), frame.get("height")), (1920, 1080))
+ proj = OUT / "project_aspect_slash.json"
+ proj.write_text(json.dumps({
+ "output": "render_aspect_slash.mp4",
+ "clips": [{"src": "source.mp4", "in": "0:01", "out": "0:03"}],
+ "frame": {"aspect": "16/9"}, "fit": {"aspect": "16:9"},
+ "export": {"preset": "youtube"},
+ }), encoding="utf-8")
+ proc = script("render.py", proj, "--dry-run", "--json")
+ self.assertIn("frame: 1920x1080 from the youtube export preset", proc.stderr)
+ self.assertTrue(any("--aspect 16:9 --width 1920 --height 1080" in line for line in proc.stderr.splitlines()
+ if "fit.py" in line), proc.stderr[-2000:])
def test_render_exits_nonzero_when_the_check_stage_fails(self):
"""A render whose deliverable fails its own check stage must not report success."""
@@ -1692,6 +1741,13 @@ class TimelineExportTests(unittest.TestCase):
for word in ("captions", "export preset", "voice"):
self.assertIn(word, joined)
self.assertNotIn("silence", joined, "an empty silence block asked for nothing")
+ # what the render refuses too: a clip with no source, a fit.py --fit that does not exist
+ for proj, word in (({"clips": [{"src": "a.mp4"}, {"in": 0}]}, "clips[1]: no src"),
+ ({"clips": [{"src": "a.mp4"}], "frame": {"aspect": "9:16", "fit": "stretch"}}, "frame.fit")):
+ with self.subTest(refused=word):
+ with self.assertRaises(tl_mod.TimelineError) as ctx:
+ tl_mod.build(proj, probes, rel)
+ self.assertIn(word, str(ctx.exception))
def test_ntsc_rates_are_exact(self):
tl_mod = self._tl()
@@ -1700,6 +1756,128 @@ class TimelineExportTests(unittest.TestCase):
self.assertEqual(tl_mod.exact_rate(23.976), Fraction(24000, 1001))
self.assertEqual(tl_mod.exact_rate(25.0), Fraction(25))
+ def test_times_are_read_with_the_render_grammar(self):
+ """Clip in/out and chapter times in the forms render.py's own stages take ("0:01",
+ hh:mm:ss:ff at the source's fps, @fps) count the frames the numeric form counts. 2.2.1
+ called float() on them: the --init starter's own "in": "0:00" was a traceback."""
+ tl_mod = self._tl()
+ probes = self._probes(**{"a.mp4": 8})
+ probes["/m/v.m4a"] = {"duration": 8, "audio": {"channels": 2}}
+ rel = lambda p: f"/m/{p}" # noqa: E731
+
+ def frames(clip, at, **proj):
+ tl = tl_mod.build(dict(proj, clips=[dict({"src": "a.mp4"}, **clip)], chapters=[{"at": at, "title": "x"}]),
+ probes, rel)
+ c = tl["clips"][0]
+ return c["src_in"], c["src_out"], tl["total"], [m["at"] for m in tl["markers"]]
+
+ numeric = frames({"in": 1.5, "out": 5}, 2)
+ self.assertEqual(numeric, (45, 150, 105, [60]))
+ for clip, at in (({"in": "0:01.5", "out": "0:05"}, "0:02"),
+ ({"in": "00:00:01:15", "out": "00:00:05:00"}, "00:00:02:00@30"),
+ ({"in": "00:00:01:15@30", "out": "0:00:05,000"}, "2")):
+ with self.subTest(clip=clip, at=at):
+ self.assertEqual(frames(clip, at), numeric)
+ self.assertEqual(frames({"in": "0:00", "out": 5}, "0:00"), frames({"in": 0, "out": 5}, 0))
+ self.assertEqual(frames({"in": "0:00", "out": 5}, "0:00")[3], [0], 'a chapter at "0:00" is the first frame')
+ # hh:mm:ss:ff is on the SOURCE's clock, as cut.py reads it: 00:00:01:15 of a 30 fps source
+ # is 1.5 s, 38 frames of a 25 fps sequence (at the sequence's 25 fps it would be 1.6 s, 40)
+ self.assertEqual(frames({"in": "00:00:01:15", "out": 5}, 0, frame={"fps": 25})[:2], (38, 125))
+ # a clip with no picture has no fps, and cut.py takes no other: frame.fps does not stand in
+ # for it (the render refuses 00:00:01:15 there), so it needs its @fps in both paths
+ with self.assertRaisesRegex(tl_mod.TimelineError, "@fps"):
+ frames({"src": "v.m4a", "in": "00:00:01:15", "out": 5}, 0, frame={"fps": 30})
+ self.assertEqual(frames({"src": "v.m4a", "in": "00:00:01:15@30", "out": 5}, 0, frame={"fps": 30})[:2], (45, 150))
+ # a chapter list carries no fps: render.py's chapters stage needs @fps there too
+ for clip, at, word in (({"in": "1:xx"}, 0, "not a time"), ({"in": "nan"}, 0, "not a time"),
+ ({"in": -1}, 0, "negative"), ({"speed": "fast"}, 0, "not a number"),
+ ({}, "00:00:02:00", "@fps")):
+ with self.subTest(refused=clip or at):
+ with self.assertRaises(tl_mod.TimelineError) as ctx:
+ frames(clip, at)
+ self.assertIn(word, str(ctx.exception))
+ with self.assertRaisesRegex(tl_mod.TimelineError, "@fps"): # no picture and no frame.fps: no rate to read
+ tl_mod.build({"clips": [{"src": "a.mp4"}, {"src": "v.m4a", "in": "00:00:01:15"}]}, probes, rel)
+
+ def test_sequence_frame_is_the_frame_the_render_delivers(self):
+ """The sequence is sized by fit.py's own rule on the project frame. 2.2.1 took the width
+ from the project and the height from the source: {aspect 16:9, width 1920} over a 4K
+ source was a 1920x2160 sequence. A picture of another aspect is a reframe: EDL and OTIO
+ cannot state it, so they name it in not_exported with the project's frame.fit; FCPXML
+ states it (test_fcpxml_describes_each_source_and_states_the_conform)."""
+ tl_mod = self._tl()
+
+ def seq(frame, w, h, rotation=0, fmt="edl"):
+ probes = {"/m/a.mp4": {"duration": 8, "video": {"fps": 30.0, "width": w, "height": h, "rotation": rotation},
+ "audio": {"channels": 2}}}
+ tl = tl_mod.build({"clips": [{"src": "a.mp4"}], "frame": frame}, probes, lambda p: f"/m/{p}")
+ return (tl["width"], tl["height"]), [x for x in tl_mod.summary(tl, fmt)["not_exported"] if "reframe" in x or "blur" in x]
+
+ self.assertEqual(seq({"aspect": "16:9", "width": 1920}, 3840, 2160), ((1920, 1080), []))
+ for fit in ("crop", "pad", "blur"):
+ with self.subTest(fit=fit):
+ size, reframe = seq({"aspect": "16:9", "width": 1920, "fit": fit}, 640, 480)
+ self.assertEqual(size, (1920, 1080))
+ self.assertEqual(len(reframe), 1)
+ self.assertIn("a.mp4", reframe[0])
+ self.assertIn(f"(frame.fit {fit})", reframe[0], "the note reports the project's own frame.fit")
+ self.assertEqual(seq({"aspect": "16:9", "width": 1920, "fit": fit}, 640, 480, fmt="otio")[1], reframe)
+ # FCPXML states pad and crop as the clip's adjust-conform; only blur's background is missing
+ self.assertEqual(len(seq({"aspect": "16:9", "width": 1920, "fit": fit}, 640, 480, fmt="fcpxml")[1]),
+ 1 if fit == "blur" else 0)
+ self.assertEqual(seq({"width": 1920}, 640, 360), ((1920, 1080), []), "one side follows the source's aspect")
+ self.assertEqual(seq({"aspect": "9:16", "height": 1920}, 1920, 1080)[0], (1080, 1920))
+ self.assertEqual(seq({"aspect": "9:16"}, 1920, 1080)[0], (608, 1080), "fit.py's bound: no upscale")
+ self.assertEqual(seq({}, 1920, 1080, rotation=90), ((1080, 1920), []), "the displayed picture")
+ with self.assertRaises(tl_mod.TimelineError):
+ seq({"aspect": "wide"}, 1920, 1080)
+
+ def test_fcpxml_describes_each_source_and_states_the_conform(self):
+ """Each video asset carries its own source's format and the sequence its frame: with one
+ format for both, a 1280x720 asset under a 1080x1920 frame claimed to be 1080x1920. A clip
+ of another aspect states its conform, which the DTD reads as fit when it is absent: fill
+ for frame.fit crop, fit for pad."""
+ import xml.etree.ElementTree as ET
+ tl_mod = self._tl()
+ probes = self._probes(**{"a.mp4": 8})
+ probes["/m/tall.mp4"] = {"duration": 8, "video": {"fps": 30.0, "width": 1080, "height": 1920}, "audio": {"channels": 2}}
+ probes["/m/sq25.mp4"] = {"duration": 8, "video": {"fps": 25.0, "width": 96, "height": 96}, "audio": {"channels": 2}}
+ for fit, conform in (("crop", "fill"), ("pad", "fit")):
+ with self.subTest(fit=fit):
+ tl = tl_mod.build({"clips": [{"src": "a.mp4"}, {"src": "tall.mp4"}, {"src": "sq25.mp4"}],
+ "frame": {"aspect": "9:16", "width": 1080, "fit": fit}}, probes, lambda p: f"/m/{p}")
+ root = ET.fromstring(tl_mod.to_fcpxml(tl).encode("utf-8"))
+ formats = {f.get("id"): (f.get("width"), f.get("height"), f.get("frameDuration"))
+ for f in root.findall("./resources/format")}
+ assets = {a.get("name"): a.get("format") for a in root.findall("./resources/asset")}
+ seq = root.find("./library/event/project/sequence")
+ self.assertEqual(formats[seq.get("format")], ("1080", "1920", "1/30s"))
+ self.assertEqual(formats[assets["a.mp4"]], ("1280", "720", "1/30s"))
+ self.assertEqual(assets["tall.mp4"], seq.get("format"), "a source that is the frame shares its format")
+ self.assertEqual(formats[assets["sq25.mp4"]], ("96", "96", "1/25s"))
+ spine = seq.find("spine").findall("asset-clip")
+ self.assertEqual([c.get("format") for c in spine], [assets["a.mp4"], assets["tall.mp4"], assets["sq25.mp4"]])
+ self.assertEqual([[e.get("type") for e in c.findall("adjust-conform")] for c in spine],
+ [[conform], [], [conform]])
+
+ def test_fcpxml_keeps_the_dtd_child_order_on_a_retimed_first_clip(self):
+ """FCPXML 1.10's asset-clip content model is timeMap, the adjust-* elements, anchored
+ items, then markers."""
+ import xml.etree.ElementTree as ET
+ tl_mod = self._tl()
+ proj = {"clips": [{"src": "a.mp4", "in": 1, "out": 5, "speed": 1.25}, {"src": "b.mp4"}],
+ "audio": {"music": "bed.m4a"}, "chapters": [{"at": "0:00", "title": "Intro"}]}
+ tl = tl_mod.build(proj, self._probes(**{"a.mp4": 8, "b.mp4": 8, "bed.m4a": 30}), lambda p: f"/m/{p}")
+ first = ET.fromstring(tl_mod.to_fcpxml(tl).encode("utf-8")).find("./library/event/project/sequence/spine")[0]
+ self.assertEqual([e.tag for e in first], ["timeMap", "asset-clip", "chapter-marker"])
+ self.assertEqual((first[1].get("lane"), first[2].get("start")), ("-1", "0s"))
+ # a reframed clip's adjust-conform is one of the intrinsic adjustments: after timeMap,
+ # before the anchored music bed
+ tl = tl_mod.build(dict(proj, frame={"aspect": "1:1", "fit": "crop"}),
+ self._probes(**{"a.mp4": 8, "b.mp4": 8, "bed.m4a": 30}), lambda p: f"/m/{p}")
+ first = ET.fromstring(tl_mod.to_fcpxml(tl).encode("utf-8")).find("./library/event/project/sequence/spine")[0]
+ self.assertEqual([e.tag for e in first], ["timeMap", "adjust-conform", "asset-clip", "chapter-marker"])
+
@classmethod
def setUpClass(cls):
if not shutil.which("ffmpeg"):
@@ -1785,6 +1963,11 @@ class TimelineExportTests(unittest.TestCase):
self.assertIsNotNone(spine[2].find("timeMap"), "the 1.25x clip is retimed")
self.assertEqual(len(spine[0].findall("chapter-marker")), 2)
self.assertEqual(spine[0].find("asset-clip").get("lane"), "-1", "the music bed is a connected clip below")
+ # FCPXML 1.10's asset-clip holds anchored items before markers; 2.1.0-2.2.1 wrote the
+ # music bed after the chapter markers, a file that does not validate against the DTD
+ self.assertEqual([e.tag for e in spine[0]], ["asset-clip", "chapter-marker", "chapter-marker"])
+ fmt = root.find("./resources/format")
+ self.assertEqual((fmt.get("width"), fmt.get("height")), ("640", "360"), "no frame: the source's own picture")
for asset in root.findall("./resources/asset"):
self.assertTrue(asset.find("media-rep").get("src").startswith("file://"))
@@ -1802,3 +1985,212 @@ class TimelineExportTests(unittest.TestCase):
self.assertFalse(dry.exists(), "a dry run writes nothing")
proc = sh(sys.executable, SCRIPTS / "render.py", self.project, "--export-timeline", self.dir / "x.aaf", "--json", expect_fail=True)
self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input")
+
+ def _sequence(self, project, name):
+ """Export `project` as FCPXML: the result document, the sequence's width x height, the root."""
+ import xml.etree.ElementTree as ET
+ out = self.dir / name
+ if out.exists():
+ out.unlink()
+ doc = json.loads(script("render.py", project, "--export-timeline", out, "--json").stdout)
+ root = ET.parse(str(out)).getroot()
+ seq = root.find("./library/event/project/sequence")
+ fmt = root.find(f"./resources/format[@id='{seq.get('format')}']")
+ return doc, (int(fmt.get("width")), int(fmt.get("height"))), root
+
+ def test_the_init_starter_exports_as_written(self):
+ """--init's project with only its src replaced: "in": "0:00" / "out": "0:30" count the frames
+ 0 / 30 count, and frame {aspect 16:9, width 1920} over a 4:3 source is a 1920x1080 sequence
+ with the reframe stated (FCPXML) or named (EDL). 2.2.1 printed a traceback (and, given
+ numbers, a 1920x48 sequence)."""
+ src = self.dir / "long43.mp4"
+ if not src.exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=64x48:rate=30",
+ "-t", "30", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", src)
+ init = self.dir / "init.json"
+ script("render.py", "--init", init)
+ proj = json.loads(init.read_text(encoding="utf-8"))
+ self.assertEqual(proj["clips"][0]["in"], "0:00", "the starter is what this test is about")
+ proj["clips"][0]["src"] = src.name
+ init.write_text(json.dumps(proj), encoding="utf-8")
+ proj["clips"][0].update({"in": 0, "out": 30})
+ numeric = self.dir / "init_numeric.json"
+ numeric.write_text(json.dumps(proj), encoding="utf-8")
+ doc, size, root = self._sequence(init, "init.fcpxml")
+ numeric_doc, numeric_size, _ = self._sequence(numeric, "init_numeric.fcpxml")
+ self.assertEqual((doc["status"], doc["timeline"]["frames"]), ("completed", 900))
+ self.assertEqual(doc["timeline"], numeric_doc["timeline"])
+ self.assertEqual(size, (1920, 1080))
+ self.assertEqual(numeric_size, (1920, 1080))
+ conform = root.find("./library/event/project/sequence/spine/asset-clip/adjust-conform")
+ self.assertEqual(conform.get("type"), "fit", "no frame.fit is pad: the 4:3 picture is fitted into 16:9")
+ self.assertFalse(any("reframe" in x for x in doc["timeline"]["not_exported"]), "the FCPXML states it")
+ edl = self.dir / "init.edl"
+ if edl.exists():
+ edl.unlink()
+ edl_doc = json.loads(script("render.py", init, "--export-timeline", edl, "--json").stdout)
+ self.assertTrue(any("reframe" in x and "long43.mp4" in x and "(frame.fit pad)" in x
+ for x in edl_doc["timeline"]["not_exported"]), "an EDL cannot state it")
+
+ def test_an_aspect_only_frame_takes_the_export_preset_size(self):
+ """frame_from_preset() is the render's reading of an aspect-only frame; the export reads it too."""
+ proj = self.dir / "reels.json"
+ proj.write_text(json.dumps({"frame": {"aspect": "9:16"}, "export": {"preset": "reels"},
+ "clips": [{"src": "a.mp4", "in": 1, "out": 3}]}), encoding="utf-8")
+ doc, size, root = self._sequence(proj, "reels.fcpxml")
+ self.assertEqual(size, (1080, 1920))
+ asset = root.find("./resources/asset[@name='a.mp4']")
+ fmt = root.find(f"./resources/format[@id='{asset.get('format')}']")
+ self.assertEqual((fmt.get("width"), fmt.get("height")), ("640", "360"), "the asset is its source, not the frame")
+ clip = root.find("./library/event/project/sequence/spine/asset-clip")
+ self.assertEqual((clip.get("format"), clip.find("adjust-conform").get("type")), (asset.get("format"), "fit"))
+
+ def test_a_multi_clip_render_delivers_the_sequence_size(self):
+ """Several clips under a frame of an aspect and one side: join.py scales them to that side
+ at the FIRST clip's aspect, then fit.py reframes the join. fit.py used to get only the
+ aspect and bound it by the joined picture, so {aspect 9:16, width 72} over two 16:9 clips
+ rendered 22x40 against a 72x128 sequence (render.py's own docstring project, at 1080 wide:
+ 342x608 against 1080x1920)."""
+ for name, size, rate in (("w128.mp4", "128x72", 30), ("sq96.mp4", "96x96", 25)):
+ if not (self.dir / name).exists():
+ sh("ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i",
+ f"testsrc2=size={size}:rate={rate}", "-t", "2", "-c:v", "libx264", "-preset", "ultrafast",
+ "-pix_fmt", "yuv420p", self.dir / name)
+ for i, (frame, srcs) in enumerate((({"aspect": "9:16", "width": 72}, ("w128.mp4", "w128.mp4")),
+ ({"aspect": "16:9", "height": 72}, ("sq96.mp4", "w128.mp4")))):
+ with self.subTest(frame=frame):
+ proj = self.dir / f"multi{i}.json"
+ proj.write_text(json.dumps({"output": f"multi{i}.mp4", "frame": frame, "transition": {"type": "none"},
+ "clips": [{"src": src, "in": 0, "out": 1} for src in srcs]}), encoding="utf-8")
+ _, seq_size, _ = self._sequence(proj, f"multi{i}.fcpxml")
+ work = self.dir / f"multi{i}_work"
+ shutil.rmtree(work, ignore_errors=True)
+ doc = json.loads(script("render.py", proj, "--fast", "--stop-after", "fit", "--work", work, "--json").stdout)
+ m = probe(doc["output"])
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), seq_size)
+ # Only that case changed. A frame with both sides is already the join's size, and a
+ # project fit object with its own width or aspect renders the size 2.2.1 rendered: the
+ # first 2.2.2 draft filled the frame's sides in there too, and fit {width 36} under a
+ # 72x128 crop frame came out 36x128 (half the picture cut away and barred by the export).
+ for i, (frame, fit, size) in enumerate((
+ ({"aspect": "9:16", "width": 72, "height": 128, "fit": "crop"}, {"width": 36}, (36, 64)),
+ ({"width": 128, "height": 72}, {"aspect": "9:16", "fit": "crop"}, (40, 72)),
+ ({"aspect": "9:16", "width": 72}, {"aspect": "4:5"}, (32, 40)))):
+ with self.subTest(frame=frame, fit=fit):
+ proj = self.dir / f"multi_own{i}.json"
+ proj.write_text(json.dumps({"output": f"multi_own{i}.mp4", "frame": frame, "fit": fit,
+ "transition": {"type": "none"},
+ "clips": [{"src": "w128.mp4", "in": 0, "out": 1}] * 2}), encoding="utf-8")
+ work = self.dir / f"multi_own{i}_work"
+ shutil.rmtree(work, ignore_errors=True)
+ doc = json.loads(script("render.py", proj, "--fast", "--stop-after", "fit", "--work", work, "--json").stdout)
+ m = probe(doc["output"])
+ self.assertEqual((m["video"]["width"], m["video"]["height"]), size)
+
+ def test_the_export_and_fit_py_read_one_aspect_grammar(self):
+ """The render hands frame.aspect to fit.py --aspect, so the export refuses what fit.py
+ refuses: it once read "16/9" and "2.39:1" and wrote a sequence for a project whose render
+ then failed with fit.py's "bad aspect"."""
+ tl_mod = self._tl()
+ for aspect in ("16:9", "4:5", "16/9", "2.39:1", "0:9", "wide"):
+ with self.subTest(aspect=aspect):
+ fit = subprocess.run([sys.executable, str(SCRIPTS / "fit.py"), str(self.dir / "a.mp4"), "--aspect", aspect,
+ "--dry-run", "-o", str(self.dir / "aspect.mp4")],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
+ try:
+ tl_mod.build({"clips": [{"src": "a.mp4"}], "frame": {"aspect": aspect}},
+ self._probes(**{"a.mp4": 8}), lambda p: f"/m/{p}")
+ exported = True
+ except tl_mod.TimelineError:
+ exported = False
+ self.assertEqual(exported, fit.returncode == 0, fit.stderr[-300:])
+ # 0:9 is a zero ratio -- no aspect, as fit.py ran it in 2.2.1 -- in both
+ self.assertEqual(exported, aspect in ("16:9", "4:5", "0:9"))
+ # ...but only an aspect the render hands to fit.py: when the project's own fit.aspect
+ # replaces it, the render completes (sized by frame_from_preset()'s 2.2.1 match) and the
+ # export writes that frame instead of refusing a project 2.2.1 exported
+ proj = self.dir / "aspect_replaced.json"
+ proj.write_text(json.dumps({"frame": {"aspect": "16/9"}, "fit": {"aspect": "16:9"}, "export": {"preset": "youtube"},
+ "clips": [{"src": "a.mp4", "in": 0, "out": 2}]}), encoding="utf-8")
+ doc, size, _ = self._sequence(proj, "aspect_replaced.fcpxml")
+ self.assertEqual((doc["status"], size), ("completed", (1920, 1080)))
+ proj.write_text(json.dumps({"frame": {"aspect": "16/9"}, "export": {"preset": "youtube"},
+ "clips": [{"src": "a.mp4", "in": 0, "out": 2}]}), encoding="utf-8")
+ proc = script("render.py", proj, "--export-timeline", self.dir / "aspect_refused.fcpxml", "--json", expect_fail=True)
+ self.assertIn("frame.aspect '16/9'", json.loads(proc.stdout)["error"]["message"])
+
+ def test_a_bad_time_is_an_input_failure_not_a_traceback(self):
+ """A time that does not parse, and a project object that is not an object where the run
+ reads it, are `kind: input` with the field named. A string frame, a string clip, a clip
+ with no src or a transition between two clips was an AttributeError / KeyError traceback
+ with nothing on stdout under --json in the render and the export alike; "export": "reels"
+ and the other stage sections were one in the render only. A section the run never reads
+ is not refused: 2.2.1 rendered a single clip with "transition": "none" (a batch.py
+ project recipe carrying join.py's --transition none), and exported a timeline for
+ "captions": "subs.srt", listing it as not exported."""
+ both, render_only = "both", "render"
+ cases = {
+ "bad_time": ({"clips": [{"src": "a.mp4", "in": "1:xx", "out": 5}]}, "clips[0].in", both),
+ "frame_str": ({"frame": "16:9", "clips": [{"src": "a.mp4"}]}, "frame: must be an object", both),
+ "clip_str": ({"clips": ["a.mp4"]}, "clips[0]: must be an object", both),
+ "clips_obj": ({"clips": {"src": "a.mp4"}}, "clips: must be a list", both),
+ "no_src": ({"clips": [{"src": "a.mp4"}, {"in": 0}]}, "clips[1]: no src", both),
+ "audio_str": ({"audio": "bed.m4a", "clips": [{"src": "a.mp4"}]}, "audio: must be an object", both),
+ "joined_transition_str": ({"clips": [{"src": "a.mp4"}, {"src": "b.mp4"}], "transition": "fade"},
+ "transition: must be an object", both),
+ "export_str": ({"frame": {"aspect": "9:16"}, "export": "reels", "clips": [{"src": "a.mp4"}]},
+ "export: must be an object", render_only),
+ "captions_str": ({"captions": "subs.srt", "loudness": True, "check": "reels", "clips": [{"src": "a.mp4"}]},
+ "captions: must be an object", render_only),
+ "cut_snap_str": ({"clips": [{"src": "a.mp4", "in": 0, "out": 2, "snap": "beats"}]},
+ "clips[0].snap: must be an object", render_only),
+ }
+ for name, (body, message, where) in cases.items():
+ with self.subTest(name):
+ proj = self.dir / f"{name}.json"
+ proj.write_text(json.dumps(body), encoding="utf-8")
+ out = self.dir / f"{name}.edl"
+ if out.exists():
+ out.unlink()
+ # the render path refuses before its first stage (a bad time is cut.py's to refuse
+ # there, with its own message)
+ refused = ((("--dry-run",),) if name != "bad_time" else ()) + ((("--export-timeline", out),) if where == both else ())
+ for argv in refused:
+ proc = sh(sys.executable, SCRIPTS / "render.py", proj, *argv, "--json", expect_fail=True)
+ self.assertNotIn("Traceback", proc.stderr)
+ err = json.loads(proc.stdout)["error"]
+ self.assertEqual(err["kind"], "input")
+ self.assertIn(message, err["message"])
+ if where == both:
+ self.assertFalse(out.exists())
+ else:
+ # the export never reads the section: it writes the timeline and names it
+ doc = json.loads(script("render.py", proj, "--export-timeline", out, "--json").stdout)
+ self.assertEqual(doc["status"], "completed")
+ self.assertTrue(out.exists())
+ # nothing reads these, in either path: they render and export as in 2.2.1
+ for name, body in (("single_transition_str", {"clips": [{"src": "a.mp4"}], "transition": "none"}),
+ ("uncut_snap_str", {"clips": [{"src": "a.mp4", "snap": "beats"}]})):
+ with self.subTest(name):
+ proj = self.dir / f"{name}.json"
+ proj.write_text(json.dumps(dict(body, output=f"{name}.mp4")), encoding="utf-8")
+ out = self.dir / f"{name}.edl"
+ if out.exists():
+ out.unlink()
+ self.assertEqual(json.loads(script("render.py", proj, "--dry-run", "--json").stdout)["status"], "completed")
+ self.assertEqual(json.loads(script("render.py", proj, "--export-timeline", out, "--json").stdout)["status"],
+ "completed")
+ # --stop-after ends the run before a later stage reads its section: a preview of a
+ # project with shorthand sections completed in 2.2.1 and still does
+ for name, body, stop in (("stop_fit_captions", {"captions": "subs.srt", "export": "reels", "check": "youtube",
+ "graphics": "title", "clips": [{"src": "a.mp4", "in": 0, "out": 1}]}, "fit"),
+ ("stop_clips_transition", {"transition": "fade",
+ "clips": [{"src": "a.mp4", "in": 0, "out": 1}, {"src": "b.mp4"}]}, "clips")):
+ with self.subTest(name):
+ proj = self.dir / f"{name}.json"
+ proj.write_text(json.dumps(dict(body, output=f"{name}.mp4")), encoding="utf-8")
+ doc = json.loads(script("render.py", proj, "--stop-after", stop, "--dry-run", "--json").stdout)
+ self.assertEqual(doc["status"], "completed")
+ # without --stop-after the full run reads them, and refuses up front
+ proc = sh(sys.executable, SCRIPTS / "render.py", proj, "--dry-run", "--json", expect_fail=True)
+ self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input")