Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.
Python agent-skillsai-agentsaudio-processingcaptionsclaude-codecodexcoding-agentcursorffmpegloudness-normalizationmcpmcp-servermodel-context-protocolpythonsubtitlestranscodingvideovideo-editingvideo-processingwhisper
fix(color): convert RGB pictures, JPEG stills and grid cells to BT.709 (#333)
Shogo Kajiyama7816c4c99ff0
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7d16a97..4358748 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,9 @@## Unreleased-(nothing yet)+### Fixed++- **An RGB picture, a JPEG still and every `grid.py` cell are converted to the BT.709 their output is tagged with.** 2.10.1 converted a BT.601-tagged video source; everything else was still made YUV by FFmpeg's own default, BT.601, under a BT.709 tag, so pure red came out (255,24,0) and pure green (0,216,0) to any player that honours the tag (mean ΔRGB 8.8, max 38.9 on FFmpeg 5.1.1, 6.1, 7.1 and 8.1). Now converted explicitly (`scale=out_color_matrix=bt709`): a PNG/JPEG still (`insert.py`), a PNG sequence (`sequence.py`), an RGB clip (qtrle, PNG-in-MOV) through any re-encode, the logo or image of `overlay.py --image` (and `render.py`'s image/logo layers: there the RGBA logo also made `overlay=format=auto` composite in RGB, so the *whole video* came back BT.601, 9.04 mean / 40.6 max ΔRGB on a BT.709 source with 6.1; it now composites in YUV and leaves the video's pixels alone), `waveform.py --image` (the audiogram template) and the emoji PNGs of `graphics.py`/`caption.py`; a palette PNG is expanded first (swscale converts a palette with fixed BT.601). `grid.py` converts each cell by its own tags: it relabelled them on 5.1/6.1, and on 7.1+ `xstack` converted the cells to one another's matrix. Where a converted picture meets an untagged one (an untagged main under a logo, an untagged grid cell beside a converted one), the untagged one is labelled BT.709 (`setparams`, tags only), because FFmpeg 7.1+ otherwise converts one to the other taking "unknown" as BT.601 (the untagged main shifted by 7.08 mean / 25 max ΔRGB on 7.1.1 and 8.1.3). After the fix every measured patch is within 1.6 of its colour (2.6 for a dithered palette PNG). An untagged or BT.709 video source re-encoded alone, and a grid of untagged/BT.709 clips, plan the commands they did.## 2.10.1diff --git a/README.md b/README.mdindex 944b5aa..cb8c549 100644--- a/README.md+++ b/README.md@@ -248,7 +248,7 @@ These are the rules the skill file gives the agent and the code enforces.| Tool | What it does ||---|---|-| `export.py` | Presets `youtube`, `youtube4k`, `reels`, `tiktok`, `shorts`, `linkedin`, `facebook`, `x`, `youtube-hdr` (HEVC Main10, source HDR tags kept), `youtube-av1`, `prores`, `h265`, `gif`, `copy`, all tagged BT.709 unless they carry HDR (a BT.601-matrix source is converted to BT.709, not relabelled); `--normalize` meets the platform's loudness in the same call (`render.py` turns it on by default for platform presets) |+| `export.py` | Presets `youtube`, `youtube4k`, `reels`, `tiktok`, `shorts`, `linkedin`, `facebook`, `x`, `youtube-hdr` (HEVC Main10, source HDR tags kept), `youtube-av1`, `prores`, `h265`, `gif`, `copy`, all tagged BT.709 unless they carry HDR (a BT.601-matrix or RGB source is converted to BT.709, not relabelled); `--normalize` meets the platform's loudness in the same call (`render.py` turns it on by default for platform presets) || `proxy.py` | Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by `--width`/`--scale`, proxy-grade `--quality` (default 30), `--fps`, `--no-audio`; not a delivery preset || `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, Facebook, broadcast and podcast specs, from the same delivery table the export presets and templates read (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row || `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run; `--pack` renders a social pack table |diff --git a/docs/contract.md b/docs/contract.mdindex a977919..88f316e 100644--- a/docs/contract.md+++ b/docs/contract.md@@ -220,7 +220,12 @@ Main10 for HDR), so the flags add no behaviour to a caller that does not pass thEvery SDR line except prores is tagged BT.709; a source whose matrix is BT.601 (`smpte170m`,`bt470bg`, or `fcc`/`smpte240m`) is converted to it in the filter graph (`colorspace`, matrixonly; `_common.bt709_matrix_filter()`), so the label and the pixels agree. An untagged or BT.709-source adds no filter, and the tags themselves never convert (`bt709_tag_args()`).+source adds no filter, and the tags themselves never convert (`bt709_tag_args()`). An RGB picture+(a PNG or JPEG still, a PNG sequence, a qtrle clip, an `overlay --image` logo, an emoji PNG,+`waveform --image`) is converted to BT.709 YUV explicitly (`scale=out_color_matrix=bt709`; a JPEG is+a `bt470bg` source); `grid.py` converts each cell by its own tags. Where a converted picture meets an+untagged one in a mixing filter, the untagged one is labelled BT.709 (`setparams`, tags only), so+FFmpeg 7.1+'s colourspace negotiation converts neither.`cut.py --keep-hevc` (opt-in) re-encodes an SDR HEVC source as x265 8-bit BT.709 instead of x264; an HDR orBT.2020 source keeps the Main10 line with its own colour tags, with or without the flag. Theencoder each value needs is listed under the tool's optional capabilities (`--codec hevc` anddiff --git a/docs/design-decisions.md b/docs/design-decisions.mdindex 1f527aa..b7fd877 100644--- a/docs/design-decisions.md+++ b/docs/design-decisions.md@@ -141,9 +141,8 @@ exists. When a decision changes, edit the entry in the same PR.tool's own graph as its first step (`fold_bt709_matrix()`), so the tools that re-encode onesource needed no edit of their own; `join`, `multicam`, `broll` and `overlay --video` convert eachinput by its own tags in its own chain (`matrix_chain()`), since one source's tags say nothing- about another's. Not converted: a stream copy, a prores master, an HDR/BT.2020 source, `grid.py`- (an inspection sheet), stills, and the second input of a tool that composites a still over the- source. Measured on 5.1.1, 6.1 and a 7.1 development snapshot that predates 7.1's colour+ about another's. Not converted: a stream copy, a prores master, an HDR/BT.2020 source.+ Measured on 5.1.1, 6.1 and a 7.1 development snapshot that predates 7.1's colournegotiation (CI runs 8.x): 42.2 dB RGB PSNR between the source decodedas BT.601 and the output decoded as BT.709 (24.8 dB for the old relabel), untagged and BT.709sources unchanged. Code: `_common/color.py`. Tests:@@ -151,6 +150,43 @@ exists. When a decision changes, edit the entry in the same PR.`test_an_untagged_or_bt709_source_is_not_converted`,`test_every_reencoding_tool_plans_the_bt601_conversion`,`test_fold_bt709_matrix_into_the_tools_graph`.+- **An RGB picture is converted to BT.709 YUV by an explicit `scale`, where it enters; an untagged+ picture that meets a converted one is labelled, not converted.** An RGB picture (PNG/JPEG still,+ logo, emoji PNG, PNG sequence, qtrle clip) has no matrix of its own: FFmpeg gives it one wherever+ the graph first needs YUV, and its default there is BT.601 (5.1.1, 6.1; 7.1+ follows the link's+ tag, "unknown" = BT.601), so through 2.10.1 every such output was BT.601 under a BT.709 tag: pure+ red decoded (255,24,0), green (0,216,0), mean ΔRGB 8.8 / max 38.9, on 5.1.1, 6.1, 7.1.1 and 8.1.3.+ `scale=out_color_matrix=bt709:out_range=tv,format=…` pins the conversion, so no build's default or+ negotiation decides it: a source gets it as the encoder line's `-vf` (`bt709_matrix_filter()`,+ folded like the BT.601 one, `format=yuv444p` so the tool's own scaling still sees full chroma), a+ still a tool made RGBA (`overlay --image`, emoji, `waveform --image`'s composite) at the end of its+ own chain (`rgb_chain_to_bt709()`, `yuva420p` for overlay). Converting the logo there, not letting+ the overlay filter do it, matters for the video too: with an RGBA input `overlay=format=auto`+ composites in RGB, so the whole main picture went to RGB and came back BT.601 (9.04 mean / 40.6+ max on a BT.709 main with 5.1.1/6.1, 7.93 / 37.4 untagged on every build); with a `yuva420p` logo+ it composites in YUV and the main's pixels are not touched. A palette PNG is expanded to RGB first:+ swscale converts a palette with fixed BT.601 whatever `out_color_matrix` says (9.1 → 0.9 mean on+ 5.1.1/6.1). A JPEG probes as `yuvj420p`/`bt470bg` and is a BT.601 source like any other.+ `grid.py` converts each cell by its own tags (`matrix_chain()`): it is an inspection sheet, but one+ whose cells are compared with each other, and on 7.1+ `xstack` converted the cells to one+ another's matrix, so a BT.601 cell first took the BT.709 cell with it. FFmpeg 7.1+ negotiates one+ colourspace for all inputs of a mixing filter and converts the rest to it, taking "unknown" as+ BT.601: with a converted logo the *untagged main* shifted (7.08 mean / 25 max on 7.1.1 and 8.1.3,+ already 7.93 / 37.4 before this fix on 8.1), or the converted picture was converted back. So where+ a converted picture meets an untagged one, the untagged one is labelled BT.709 (`setparams`, tags+ only; `label_chain()`), which is what the output's tag says it is: no conversion is left to+ negotiate, and on 5.1/6.1 the label changes nothing. It is applied only there (a still over the+ video, a grid with a converted cell), so a one-input re-encode of an untagged source, and a grid of+ untagged/BT.709 clips, keep their commands; `join`/`multicam`/`broll`/`overlay --video` do not+ label yet, and on 7.1+ mixing an untagged and a BT.601 clip there still shifts one of them (a+ separate change). Known on FFmpeg 7.0.x only (not in CI): its `colorspace` filter does not declare+ the converted link, so a BT.601 clip *first* in `grid`/`join` still takes the others with it.+ Code: `_common/color.py`, `_common/decision.py`. Tests:+ `test_an_rgb_or_jpeg_source_is_converted_to_the_bt709_it_is_tagged_with`,+ `test_a_still_composited_over_the_video_is_converted_to_bt709`,+ `test_grid_converts_each_cell_by_its_own_tags`,+ `test_an_rgb_source_or_a_still_is_converted_to_the_bt709_it_is_tagged_with`,+ `test_still_and_rgb_tools_plan_the_bt709_conversion`.- **`--correct` converts through bt601 on both legs** (not bt709), because on every testedFFmpeg the 709 round trip through 8-bit 4:2:0 loses ~5 dB more than 601. Code:`color.py _rgb_stage()`. Test: `test_color_correct_identity_holds_on_a_bt709_tagged_source`.diff --git a/references/gotchas.md b/references/gotchas.mdindex 9d5677a..2e3a0d0 100644--- a/references/gotchas.md+++ b/references/gotchas.md@@ -11,7 +11,10 @@ detail, not an aside.SDR outputs are H.264 tagged BT.709 `yuv420p`. A source tagged with a BT.601 matrix(`smpte170m`, `bt470bg`; also `fcc`, `smpte240m`) has its pixels converted to that BT.709 matrixby every re-encode (through 2.10.0 only the label changed); `color.py --retag bt709` is a-stream copy and still only relabels. Untagged sources are taken as BT.709 and left alone. When `probe.py` reports+stream copy and still only relabels. Untagged sources are taken as BT.709 and left alone. An RGB+picture (PNG/JPEG still, logo, emoji PNG, PNG sequence, qtrle clip) is converted to BT.709 YUV+explicitly (through 2.10.1 FFmpeg's BT.601 default did it), and `grid.py` converts each cell by its+own tags. When `probe.py` reports`bt2020_or_hdr: true` (HDR10/PQ, HLG, Dolby Vision, or BT.2020 SDR), every editing script keepsthe output on the 10-bit path (HEVC Main10, source colour tags) so nothing is silentlyflattened. Decide with the user: keep HDR (fine for YouTube/phones) or rundiff --git a/references/scripts.md b/references/scripts.mdindex f726563..eaa7b55 100644--- a/references/scripts.md+++ b/references/scripts.md@@ -1615,7 +1615,7 @@ export.py INPUT --preset youtube|youtube4k|reels|tiktok|shorts|linkedin|facebook[--fit pad|crop] [--no-scale] [--allow-long] [--crf N] [--normalize] [-o OUT]export.py --list```-Scales into the preset frame (pad by default), tags BT.709 (a BT.601-matrix source is converted to it, as by every re-encode), sets `+faststart`,+Scales into the preset frame (pad by default), tags BT.709 (a BT.601-matrix or RGB source is converted to it, as by every re-encode), sets `+faststart`,trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long`. Itdoes not touch levels: for youtube / youtube4k / reels / x the written file ismeasured and the result's `loudness` (and a `notes` line) says when it isdiff --git a/scripts/_common/__init__.py b/scripts/_common/__init__.pyindex ce1f9bf..c6eed95 100644--- a/scripts/_common/__init__.py+++ b/scripts/_common/__init__.py@@ -83,7 +83,7 @@ from _common.decision import (_onset_strength, _pick_onsets, _autocorrelation_peak, _grid_score,brand_states_font, cfr_args, concat_list_line, db_to_linear, default_output, encoder_args, escape_filter_path,fmt_secs, fmt_smpte_time, fmt_srt_time, frame_size, is_audio_output, load_brand, MissingFpsError, pad_filters, parse_time,- signed_time_arg, source_codec_video_args, matrix_chain, SVT_PRESET, time_arg, video_args, x264_args, _x264_raw+ signed_time_arg, source_codec_video_args, label_chain, matrix_chain, rgb_chain_to_bt709, SVT_PRESET, time_arg, video_args, x264_args, _x264_raw)from _common.asr import (ASR_ENGINES, ASR_INSTALL_HINT, _asr_run, die_no_engine, parse_srt, transcribe, _transcribe_in,@@ -91,8 +91,8 @@ from _common.asr import (whisper_word_timings, write_srt)from _common.color import (- bt709_matrix_args, bt709_matrix_filter, bt709_tag_args, color_hex, _COLOR_TOKEN_RE, fold_bt709_matrix,- NON_BT709_MATRICES, _sdr_bt709, validate_color+ bt709_label_filter, bt709_matrix_args, bt709_matrix_filter, bt709_tag_args, color_hex, _COLOR_TOKEN_RE, fold_bt709_matrix,+ is_rgb, NON_BT709_MATRICES, rgb_to_bt709_filter, _sdr_bt709, validate_color)from _common.text import (ADVANCE_EM, BIDI_SCRIPTS, _char_em, char_script, default_font_file, detect_script, drawtext_boxborderw, drawtext_center_align,@@ -213,7 +213,7 @@ __all__ = ["SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote", "sibling_temp","_SIGNALS_INSTALLED", "signed_time_arg", "_stage_existing_output", "STATE", "SVT_PRESET", "text_width_em","time_arg", "_timed_out", "_to_float", "_to_int", "_unwatch", "validate_color", "verify_output",- "source_codec_video_args", "matrix_chain", "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS",+ "source_codec_video_args", "label_chain", "matrix_chain", "rgb_chain_to_bt709", "bt709_label_filter", "rgb_to_bt709_filter", "is_rgb", "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS","_x264_raw", "_ZWJ","ASR_ENGINES", "ASR_INSTALL_HINT", "_asr_run", "die_no_engine", "parse_srt", "transcribe","_transcribe_in", "transcribe_words", "_words_from_openai_whisper_json",diff --git a/scripts/_common/color.py b/scripts/_common/color.pyindex 8b1c42e..022f7ed 100644--- a/scripts/_common/color.py+++ b/scripts/_common/color.py@@ -51,8 +51,26 @@ def _sdr_bt709(encoder: str) -> "Tuple[str, List[str]]":# pre-BT.709 ones the colorspace filter also reads. bt2020nc/c sources take the HDR/BT.2020 path.NON_BT709_MATRICES = ("smpte170m", "bt470bg", "fcc", "smpte240m")-_MATRIX_FILTER_RE = re.compile(r"^colorspace=ispace=(?:%s):iprimaries=bt709:itrc=bt709:space=bt709:primaries=bt709:trc=bt709$"- % "|".join(NON_BT709_MATRICES))+# RGB pictures (ffprobe's pix_fmt names: PNG rgb24/rgba/pal8/rgb48be, qtrle, gbrp...). Their+# conversion to YUV is wherever the graph first needs YUV, and FFmpeg's default for it is BT.601+# (5.1, 6.1, 7.0; 7.1+ follows the link's tag, unspecified = BT.601). Gray has no matrix.+_RGB_PIX_FMT_RE = re.compile(r"^(?:rgb|bgr|argb|abgr|0rgb|0bgr|gbr|x2rgb|x2bgr|pal8)")+++def rgb_to_bt709_filter(pix_fmt: str = "yuv444p") -> str:+ """An RGB picture converted to BT.709 limited-range YUV right here, in `pix_fmt`, so no later+ conversion (an auto-inserted one, with FFmpeg's BT.601 default) decides the matrix. 4:4:4 by+ default: the tool's own scaling and the encoder's 4:2:0 come after, as they did on the RGB."""+ return f"scale=out_color_matrix=bt709:out_range=tv,format={pix_fmt}"+++def is_rgb(meta: Optional[Dict[str, Any]]) -> bool:+ """True when the probed picture is RGB (a PNG/still, a qtrle/png-in-mov clip, an image sequence)."""+ return bool(_RGB_PIX_FMT_RE.match(str(((meta or {}).get("video") or {}).get("pix_fmt") or "")))+++_MATRIX_FILTER_RE = re.compile(r"^(?:colorspace=ispace=(?:%s):iprimaries=bt709:itrc=bt709:space=bt709:primaries=bt709:trc=bt709|(?:format=rgb24,)?%s)$"+ % ("|".join(NON_BT709_MATRICES), re.escape(rgb_to_bt709_filter())))_INPUT_VIDEO_REF = re.compile(r"\[(\d+):v(?::0)?\]")@@ -67,14 +85,44 @@ def bt709_matrix_filter(meta: Optional[Dict[str, Any]]) -> Optional[str]:(iprimaries/itrc = the output's), so the picture a player that decodes with the source'smatrix showed is the picture the BT.709 output shows. The `colorspace` filter is in everyFFmpeg build (no zscale needed) and takes explicit input properties, so the conversion does- not depend on how a build reads or negotiates the frames' own tags. The range is left as it is."""+ not depend on how a build reads or negotiates the frames' own tags. The range is left as it is.++ An RGB source (is_rgb: a PNG/still, an image sequence, a qtrle clip) has no matrix of its own+ but gets one where the graph first needs YUV, and FFmpeg's default there is BT.601: through+ 2.10.1 such a source came out labelled BT.709 over BT.601-encoded pixels (pure red decoded as+ (255,24,0), pure green as (0,216,0), on 5.1.1 to 8.1). It is converted to BT.709 YUV here+ (rgb_to_bt709_filter). A JPEG (yuvj, tagged bt470bg) is a BT.601 source like any other."""v = (meta or {}).get("video") or {}matrix = v.get("color_space")- if v.get("bt2020_or_hdr") or matrix not in NON_BT709_MATRICES:+ if v.get("bt2020_or_hdr"):+ return None+ if is_rgb(meta):+ # a palette is turned into YUV with fixed BT.601 coefficients whatever out_color_matrix+ # says (5.1.1, 6.1): expand it to RGB first+ return ("format=rgb24," if v.get("pix_fmt") == "pal8" else "") + rgb_to_bt709_filter()+ if matrix not in NON_BT709_MATRICES:return Nonereturn f"colorspace=ispace={matrix}:iprimaries=bt709:itrc=bt709:space=bt709:primaries=bt709:trc=bt709"+def bt709_label_filter(meta: Optional[Dict[str, Any]]) -> Optional[str]:+ """For an UNTAGGED SDR YUV picture that meets a converted (BT.709-tagged) one in a filter that+ mixes pictures (overlay, xstack): its BT.709 label, tags only (setparams), or None for any+ other picture (tagged, RGB, HDR, no probe).++ FFmpeg 7.1+ negotiates one colourspace for all of a mixing filter's inputs and converts the+ others to it, taking "unknown" as BT.601: the untagged main under a converted logo, or an+ untagged grid cell beside a converted one, was shifted (7.08 mean / 25 max RGB on 7.1.1 and+ 8.1.3), or the converted picture converted back. Labelled as what the output says it is, the+ inputs agree and nothing is converted; on 5.1-7.0 the label changes nothing. The pixels are not+ touched, as for the untagged source of a one-input re-encode (bt709_tag_args)."""+ v = (meta or {}).get("video") or {}+ if (not v.get("pix_fmt") or v.get("bt2020_or_hdr") or is_rgb(meta)+ or (v.get("color_space") or "unknown") not in ("unknown", "unspecified")):+ return None+ return "setparams=colorspace=bt709" + ("" if v.get("color_range") == "pc" else ":range=tv")++def bt709_matrix_args(meta: Optional[Dict[str, Any]]) -> List[str]:"""["-vf", bt709_matrix_filter(meta)] for an encoder line that tags its output BT.709, or [].diff --git a/scripts/_common/decision.py b/scripts/_common/decision.pyindex 20bbc96..3024d6f 100644--- a/scripts/_common/decision.py+++ b/scripts/_common/decision.py@@ -14,7 +14,7 @@ import argparsefrom fractions import Fractionfrom pathlib import Pathfrom typing import Any, Dict, List, Optional, Sequence, Tuple-from _common.color import bt709_matrix_args, bt709_matrix_filter, bt709_tag_args, _sdr_bt709+from _common.color import bt709_label_filter, bt709_matrix_args, bt709_matrix_filter, bt709_tag_args, rgb_to_bt709_filter, _sdr_bt709from _common.emit import diefrom _common.runner import CODECS, STATE, ffmpeg_encoders, ffmpeg_version, hw_platform_reason@@ -512,7 +512,8 @@ def x264_args(crf: int = 18, preset: str = "medium", keep_bt709: bool = True,return args + (bt709_matrix_args(source) if keep_bt709 and STATE.codec != "prores" else [])-def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium", convert_matrix: bool = True) -> List[str]:+def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium", convert_matrix: bool = True,+ source: Optional[Dict[str, Any]] = None) -> List[str]:"""Encoder args that preserve what the source is.SDR sources -> H.264 8-bit tagged BT.709 (x264_args). HDR sources (HDR10/PQ, HLG,@@ -525,9 +526,14 @@ def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medfolds into the tool's own graph (color.fold_bt709_matrix). `convert_matrix=False` is for atool that mixes several sources and converts each one in its own graph (join, multicam,broll, overlay --video). --codec prores keeps the source's tags, so it converts nothing.+ An RGB source (a PNG, an image sequence, a qtrle clip) is converted the same way.++ `source`: the probe of the picture the encoder gets when the line is chosen without it+ (insert.py and sequence.py make a still or a PNG sequence into video on the SDR line,+ video_args(None)); its RGB or BT.601 pixels are converted to the BT.709 that line is tagged with."""v = (meta or {}).get("video") or {}- fix = bt709_matrix_args(meta) if convert_matrix and STATE.codec != "prores" else []+ fix = bt709_matrix_args(meta if source is None else source) if convert_matrix and STATE.codec != "prores" else []if STATE.codec:return encoder_args(STATE.codec, crf, preset, meta) + fixif not v.get("bt2020_or_hdr"):@@ -546,6 +552,28 @@ def matrix_chain(meta: Optional[Dict[str, Any]], line_meta: Optional[Dict[str, Areturn f + "," if f else ""+def label_chain(meta: Optional[Dict[str, Any]], line_meta: Optional[Dict[str, Any]]) -> str:+ """For a picture that meets a converted one in a mixing filter (a still over the video, a grid+ cell beside a converted cell), encoded through video_args(line_meta): the BT.709 label an+ untagged one's chain starts with ("<setparams>," -- bt709_label_filter), or "" (a tagged, RGB or+ HDR picture, or a line that does not tag BT.709)."""+ if STATE.codec == "prores" or ((line_meta or {}).get("video") or {}).get("bt2020_or_hdr"):+ return ""+ f = bt709_label_filter(meta)+ return f + "," if f else ""+++def rgb_chain_to_bt709(line_meta: Optional[Dict[str, Any]], pix_fmt: str = "yuva420p") -> str:+ """For a picture the tool's own graph has made RGB(A) -- a still's `format=rgba` chain+ (overlay.py --image, an emoji PNG) or an RGBA composite (waveform.py's --image plate) -- that+ is encoded through video_args(line_meta): "," + the conversion to BT.709 YUV to end that chain+ with (rgb_to_bt709_filter), or "" when the line does not tag BT.709 (the lines matrix_chain()+ leaves alone). Without it the graph's own RGB-to-YUV step used FFmpeg's BT.601 default."""+ if STATE.codec == "prores" or ((line_meta or {}).get("video") or {}).get("bt2020_or_hdr"):+ return ""+ return "," + rgb_to_bt709_filter(pix_fmt)++def source_codec_video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium",keep_hevc: bool = False) -> List[str]:"""video_args(), except that with `keep_hevc` (cut.py --keep-hevc) an SDR HEVC source re-encodesdiff --git a/scripts/_common/emoji.py b/scripts/_common/emoji.pyindex b555054..aef9d7e 100644--- a/scripts/_common/emoji.py+++ b/scripts/_common/emoji.py@@ -286,12 +286,16 @@ def resolve_emoji_assets(flag: "Optional[str]" = None, project: "Optional[str]"-def emoji_filter_chain(plan, base_label, out_label, first_input=1):+def emoji_filter_chain(plan, base_label, out_label, first_input=1, to_yuv="", base_chain=""):"""(chains, inputs) that composite the planned PNGs on top of `base_label`.`inputs` is a list of argv fragments, each ending in the asset path, to be appended to theffmpeg command in order (an overlay that fades needs `-loop 1` on its input so the still hasa timeline the fade filter can move along; one that does not is a plain `-i`).+ `to_yuv` ends each PNG's chain: the caller's rgb_chain_to_bt709(meta), so the RGBA asset is+ converted to the BT.709 the output is tagged with instead of by overlay's BT.601 default;+ `base_chain` (label_chain(meta, meta): an untagged source's BT.709 label) goes in front of+ the first overlay, so 7.1+ does not convert the two pictures to one another."""overlays = plan.get("overlays") or []if not overlays:@@ -329,11 +333,15 @@ def emoji_filter_chain(plan, base_label, out_label, first_input=1):chain += f",fade=t=out:st={max(gstart, gend - fout):.3f}:d={fout:.3f}:alpha=1"else:inputs.append(["-i", asset])+ chain += to_yuvif len(labels) > 1:chain += f",split={len(labels)}"chains.append(chain + "".join(f"[{l}]" for l in labels))pads[key] = labelscur = base_label+ if base_chain:+ chains.append(f"[{cur}]{base_chain}null[ebase]")+ cur = "ebase"remaining = {key: list(v) for key, v in pads.items()}for j, o in enumerate(overlays):label = remaining[_key(o)].pop(0)diff --git a/scripts/caption.py b/scripts/caption.pyindex 8ec26cf..84c5ec2 100755--- a/scripts/caption.py+++ b/scripts/caption.py@@ -44,7 +44,7 @@ from typing import Any, Dict, List, Optional, Tuplefrom _platforms import (PLATFORMS, PLATFORM_CHOICES, ASS_SCRIPT_HEIGHT, ass_units,resolve as resolve_platform)from _ass_overlay import EMOJI_SENTINEL, emoji_placeholder, ass_escape-from _common import emoji_filter_chain, EMOJI_ASSET_HINT, emoji_asset_for, emoji_codepoint_name, emoji_support, resolve_emoji_assets, ADVANCE_EM, LATIN_EM, NO_SPACE_SCRIPTS, _char_em, char_script, text_width_em, emoji_clusters, has_emoji, detect_script, BIDI_SCRIPTS, STATE, brand_states_font, script_font_for_text, signed_time_arg, brand_caption_style, color_hex, load_brand, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, fmt_srt_time, fmt_smpte_time, info, MissingFpsError, parse_time, probe, run, x264_args, X264_PRESETS, read_text_or_die, fmt_secs, refuse_existing_outputs+from _common import emoji_filter_chain, EMOJI_ASSET_HINT, emoji_asset_for, emoji_codepoint_name, emoji_support, resolve_emoji_assets, ADVANCE_EM, LATIN_EM, NO_SPACE_SCRIPTS, _char_em, char_script, text_width_em, emoji_clusters, has_emoji, detect_script, BIDI_SCRIPTS, STATE, brand_states_font, script_font_for_text, signed_time_arg, brand_caption_style, color_hex, label_chain, load_brand, rgb_chain_to_bt709, video_args, add_common, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, fmt_srt_time, fmt_smpte_time, info, MissingFpsError, parse_time, probe, run, x264_args, X264_PRESETS, read_text_or_die, fmt_secs, refuse_existing_outputs# The line breaker, lifted into _common/text.py in 1.16.0 so graphics.py can use the same rules.# The ASR bridge and the SRT reader/writer live in _common.asr since 1.17 (silence.py --filler# shares them). They stay caption.py's public names -- every caller and test that reached for@@ -1562,7 +1562,9 @@ def main() -> int:vf += f":fontsdir={escape_filter_path(args.fonts_dir)}"cmd = ffmpeg_base() + ["-i", args.input]- chains, emoji_inputs = emoji_filter_chain(emoji_plan or {}, "vsub", "vout") if emoji_plan else ([], [])+ chains, emoji_inputs = (emoji_filter_chain(emoji_plan or {}, "vsub", "vout", to_yuv=rgb_chain_to_bt709(meta),+ base_chain=label_chain(meta, meta) if rgb_chain_to_bt709(meta) else "")+ if emoji_plan else ([], []))if chains:for spec in emoji_inputs:cmd += specdiff --git a/scripts/graphics.py b/scripts/graphics.pyindex ae4bdfe..546d439 100755--- a/scripts/graphics.py+++ b/scripts/graphics.py@@ -34,7 +34,7 @@ from typing import List, Optionalfrom _platforms import PLATFORMS, PLATFORM_CHOICES, safe_margins_px, resolve as resolve_platformfrom _common import (aac_args, add_common, brand_caption_style, script_font_for_text, apply_common, cfr_args,color_hex, default_font_file, default_output, die, emit, escape_drawtext, escape_filter_path,- ffmpeg_base, info, load_brand, parse_time, probe, run, run_keeping_subtitles, video_args,+ ffmpeg_base, info, label_chain, load_brand, parse_time, probe, rgb_chain_to_bt709, run, run_keeping_subtitles, video_args,drawtext_boxborderw, drawtext_center_align, X264_PRESETS, time_arg, fmt_secs, STATE, drawtext_text_opts,LANGUAGE_NAMES, needs_shaping, detect_script, BIDI_SCRIPTS, font_family_of_file, font_family_for_script, has_emoji,emoji_clusters, emoji_codepoint_name, char_script, emoji_filter_chain, emoji_asset_for, emoji_support, resolve_emoji_assets,@@ -599,7 +599,9 @@ def main() -> int:vf += f":fontsdir={escape_filter_path(fdir)}"chains.append(f"[{last}]{vf}[vtxt]")last = "vtxt"- eo, emoji_inputs = emoji_filter_chain({"overlays": emoji_overlays}, last, "vfinal", first_input=1)+ eo, emoji_inputs = emoji_filter_chain({"overlays": emoji_overlays}, last, "vfinal", first_input=1,+ to_yuv=rgb_chain_to_bt709(meta),+ base_chain=label_chain(meta, meta) if rgb_chain_to_bt709(meta) else "")for spec in emoji_inputs:cmd += specasset = spec[-1]diff --git a/scripts/grid.py b/scripts/grid.pyindex b1f21d6..d278aab 100644--- a/scripts/grid.py+++ b/scripts/grid.py@@ -26,7 +26,7 @@ import osimport sysfrom _common import add_common, apply_common, aac_args, cfr_args, default_font_file, default_output, die, emit, X264_PRESETS, fmt_secs, \- escape_drawtext, escape_filter_path, ffmpeg_base, info, probe, run, validate_color, video_args+ escape_drawtext, escape_filter_path, ffmpeg_base, info, label_chain, matrix_chain, probe, run, validate_color, video_argsLABEL_MARGIN = 10@@ -91,6 +91,13 @@ def main() -> int:for p in args.inputs:cmd += ["-i", p]+ # each cell converted by its own tags to the BT.709 the grid is tagged with (a BT.601 or an RGB+ # input): without it 5.1/6.1 relabelled them, and 7.1+ converted cells to one another's matrix.+ # Beside a converted cell an untagged one gets a BT.709 label, so 7.1+'s xstack has nothing to+ # convert; a grid of untagged/BT.709 clips is what it was.+ convert = [matrix_chain(m, None) for m in metas]+ if any(convert):+ convert = [c or label_chain(m, None) for c, m in zip(convert, metas)]parts = []for i, p in enumerate(args.inputs):chain = [@@ -109,7 +116,7 @@ def main() -> int:chain.append(f"drawtext=text='{escape_drawtext(stem)}':{font_opt}:fontsize={args.font_size}:"f"fontcolor={args.font_color}:x=w-tw-{LABEL_MARGIN}:y=h-th-{LABEL_MARGIN}:"f"box=1:[email protected]:boxborderw=4")- parts.append(f"[{i}:v]{','.join(chain)}[v{i}]")+ parts.append(f"[{i}:v]{convert[i]}{','.join(chain)}[v{i}]")cw, ch = args.cell_width + args.gap, args.cell_height + args.gaplayout = "|".join(f"{c * cw}_{r * ch}" for r in range(args.rows) for c in range(args.cols))diff --git a/scripts/insert.py b/scripts/insert.pyindex 3e6677d..ef78371 100644--- a/scripts/insert.py+++ b/scripts/insert.py@@ -116,7 +116,9 @@ def main() -> int:output = args.output or default_output(args.input, "insert", "mp4")cmd = ffmpeg_base() + ["-loop", "1", "-i", args.input, "-t", f"{target:.3f}", "-vf", ",".join(vf)]- cmd += video_args(None, args.crf, args.preset)+ # the SDR line, whatever the image is; its RGB (PNG) or BT.601 (JPEG) pixels are converted to+ # the BT.709 that line is tagged with+ cmd += video_args(None, args.crf, args.preset, source=meta)cmd += ["-an", output]run(cmd)diff --git a/scripts/overlay.py b/scripts/overlay.pyindex 7222ac1..2951c63 100755--- a/scripts/overlay.py+++ b/scripts/overlay.py@@ -25,7 +25,7 @@ import sysfrom typing import List, Optionalfrom _platforms import PLATFORMS, PLATFORM_CHOICES, safe_margins_px, resolve as resolve_platform-from _common import STATE, script_font_for_text, drawtext_text_opts, needs_shaping, LANGUAGE_NAMES, has_emoji, detect_script, emoji_clusters, load_brand, matrix_chain, video_args, add_common, apply_common, default_font_file, emit, aac_args, cfr_args, default_output, die, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, validate_color, x264_args, X264_PRESETS, time_arg, fmt_secs+from _common import STATE, script_font_for_text, drawtext_text_opts, needs_shaping, LANGUAGE_NAMES, has_emoji, detect_script, emoji_clusters, load_brand, label_chain, matrix_chain, rgb_chain_to_bt709, video_args, add_common, apply_common, default_font_file, emit, aac_args, cfr_args, default_output, die, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, run_keeping_subtitles, validate_color, x264_args, X264_PRESETS, time_arg, fmt_secs# Per-edge margins: a platform's UI does not cover the same fraction of every edge (TikTok's# like column is 14 % of the width, its description block 22 % of the height), so a position@@ -266,7 +266,12 @@ def main() -> int:ov += f":enable='{enable}'"# -loop 1 turns the still into a timed stream so fade/enable expressions see real timestampscmd = ffmpeg_base() + ["-i", args.input, "-loop", "1", "-i", args.image]- fc = f"[1:v]{','.join(chain)},setpts=PTS-STARTPTS[ov];[0:v][ov]{ov}[out]"+ # the RGBA still is converted to the BT.709 the output is tagged with (overlay's own+ # RGB-to-YUV step used BT.601), and an untagged main is labelled BT.709 (tags only) so+ # 7.1+'s negotiation does not convert one picture to the other's "unknown" matrix+ base = label_chain(meta, meta) if rgb_chain_to_bt709(meta) else ""+ fc = (f"[1:v]{','.join(chain)},setpts=PTS-STARTPTS{rgb_chain_to_bt709(meta)}[ov];"+ + (f"[0:v]{base}null[base];[base]" if base else "[0:v]") + f"[ov]{ov}[out]")cmd += ["-filter_complex", fc, "-map", "[out]", "-map", f"0:a:{args.audio_stream}?"]if meta.get("duration"):# An explicit -t is exact and, unlike -shortest, only bounds the *main* input'sdiff --git a/scripts/sequence.py b/scripts/sequence.pyindex 69ecb79..0c10952 100644--- a/scripts/sequence.py+++ b/scripts/sequence.py@@ -131,7 +131,8 @@ def main() -> int:cmd = ffmpeg_base() + ["-f", "concat", "-safe", "0", "-i", str(list_path)]vf = [f"scale={out_w}:{out_h}", "setsar=1", f"fps={args.fps:g}"]cmd += ["-vf", ",".join(vf)]- cmd += video_args(None, args.crf, args.preset)+ # RGB (PNG) frames are converted to the BT.709 the line is tagged with (the first frame's probe)+ cmd += video_args(None, args.crf, args.preset, source=frame_meta)# The concat demuxer's trailing repeated-last-file trick (needed so the last real file's# duration line takes effect) has been observed to produce an extra frame's worth of# duration on some ffmpeg builds -- force the exact intended length rather than trust it.diff --git a/scripts/waveform.py b/scripts/waveform.pyindex 7b62011..8e88c63 100644--- a/scripts/waveform.py+++ b/scripts/waveform.py@@ -36,7 +36,7 @@ import sysfrom _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platformfrom _common import (add_common, apply_common, aac_args, default_output, die, emit, ffmpeg_base,- info, load_brand, pad_filters, probe, run, STATE, validate_color, video_args, X264_PRESETS,+ info, load_brand, pad_filters, probe, rgb_chain_to_bt709, run, STATE, validate_color, video_args, X264_PRESETS,fmt_secs, child_args, run_tool, measured_level_dbfs)from _common.emit import absorb_stage_hw@@ -185,7 +185,9 @@ def main() -> int:graph = f"[1:v]{plate},format=rgba[plate];[0:a:{args.audio_stream}]{vf},format=rgba"if args.opacity < 1.0:graph += f",colorchannelmixer=aa={args.opacity:g}"- graph += f"[vis];[plate][vis]overlay=x=(W-w)/2:y={vis_y}:format=auto[v]"+ # the composite is RGBA (the plate is): converted to the BT.709 the output is tagged with,+ # not by the encoder's BT.601 default+ graph += f"[vis];[plate][vis]overlay=x=(W-w)/2:y={vis_y}:format=auto{rgb_chain_to_bt709(None, 'yuv420p')}[v]"vf = graphelse:# showwaves/showspectrum paint the visualization on a transparent-black canvas; compositediff --git a/tests/test_contract.py b/tests/test_contract.pyindex edc3ca0..8f98fbe 100644--- a/tests/test_contract.py+++ b/tests/test_contract.py@@ -1094,6 +1094,60 @@ class ContractTests(unittest.TestCase):with self.assertRaises(SystemExit):fold(["ffmpeg", "-i", "a.mp4", "-filter_complex", "color=red[v]", "-map", "[v]"] + enc)+ def test_an_rgb_source_or_a_still_is_converted_to_the_bt709_it_is_tagged_with(self):+ """#332's follow-ups. An RGB picture (PNG, image sequence, qtrle) has no matrix; FFmpeg gave it+ BT.601 wherever the graph first needed YUV, under a BT.709 tag. It is converted where it enters:+ a source by the encoder line's -vf (folded like the BT.601 one; a palette is expanded first,+ swscale converts a palette with fixed BT.601), a still the tool made RGBA at the end of its own+ chain (rgb_chain_to_bt709). An untagged picture that meets a converted one in a mixing filter+ gets a BT.709 label, tags only, so 7.1+'s negotiation converts neither (label_chain). Gray,+ YUV, HDR, prores lines and a missing probe get nothing."""+ def meta(**v):+ return {"file": "src", "video": dict({"bt2020_or_hdr": False}, **v)}+ rgb = "scale=out_color_matrix=bt709:out_range=tv,format=yuv444p"+ for pf in ("rgb24", "rgba", "bgra", "argb", "gbrp", "rgb48be", "rgba64be", "bgr0"):+ self.assertEqual(_common.bt709_matrix_filter(meta(pix_fmt=pf, color_space="gbr")), rgb, pf)+ self.assertEqual(_common.bt709_matrix_filter(meta(pix_fmt=pf)), rgb, pf) # qtrle: no matrix tag at all+ self.assertEqual(_common.bt709_matrix_filter(meta(pix_fmt="pal8", color_space="gbr")), "format=rgb24," + rgb)+ for m in (meta(pix_fmt="gray"), meta(pix_fmt="ya8"), meta(pix_fmt="yuv420p"), meta(pix_fmt="yuvj420p"),+ meta(pix_fmt="rgb48be", bt2020_or_hdr=True), meta(color_space="gbr")):+ self.assertIsNone(_common.bt709_matrix_filter(m), m)+ # a JPEG probes as yuvj420p/bt470bg: a BT.601 source like any other+ self.assertTrue(_common.bt709_matrix_filter(meta(pix_fmt="yuvj420p", color_space="bt470bg", color_range="pc")).startswith("colorspace=ispace=bt470bg:"))+ png = meta(pix_fmt="rgb24", color_space="gbr")+ saved = (_common.STATE.codec, _common.STATE.hw)+ try:+ _common.STATE.hw = False+ for codec in (None, "h264", "hevc"):+ _common.STATE.codec = codec+ self.assertEqual(_common.video_args(png)[-2:], ["-vf", rgb], codec)+ self.assertEqual(_common.video_args(None, source=png)[-2:], ["-vf", rgb], codec) # insert.py / sequence.py+ self.assertNotIn("-vf", _common.video_args(None), codec)+ self.assertNotIn("-vf", _common.video_args(None, source=meta(pix_fmt="yuv420p")), codec)+ self.assertEqual(_common.matrix_chain(png, None), rgb + ",")+ self.assertEqual(_common.rgb_chain_to_bt709(None), ",scale=out_color_matrix=bt709:out_range=tv,format=yuva420p")+ self.assertEqual(_common.rgb_chain_to_bt709(meta(pix_fmt="yuv420p"), "yuv420p"),+ ",scale=out_color_matrix=bt709:out_range=tv,format=yuv420p")+ self.assertEqual(_common.label_chain(meta(pix_fmt="yuv420p"), None), "setparams=colorspace=bt709:range=tv,")+ self.assertEqual(_common.label_chain(meta(pix_fmt="yuvj420p", color_range="pc"), None), "setparams=colorspace=bt709,")+ for tagged in (meta(pix_fmt="yuv420p", color_space="bt709"), meta(pix_fmt="yuv420p", color_space="smpte170m"),+ png, meta(pix_fmt="yuv420p10le", bt2020_or_hdr=True), meta(), None):+ self.assertEqual(_common.label_chain(tagged, None), "", tagged)+ # an HDR line keeps its own tags, prores the source's: neither converts nor relabels a still+ hdr = meta(pix_fmt="yuv420p10le", bt2020_or_hdr=True)+ self.assertEqual(_common.rgb_chain_to_bt709(hdr), "")+ self.assertEqual(_common.label_chain(meta(pix_fmt="yuv420p"), hdr), "")+ _common.STATE.codec = "prores"+ self.assertNotIn("-vf", _common.video_args(None, source=png))+ self.assertEqual((_common.rgb_chain_to_bt709(None), _common.label_chain(meta(pix_fmt="yuv420p"), None)), ("", ""))+ finally:+ _common.STATE.codec, _common.STATE.hw = saved+ fold = _common.fold_bt709_matrix+ for conv in (rgb, "format=rgb24," + rgb):+ enc = ["-c:v", "libx264", "-vf", conv, "out.mp4"]+ self.assertEqual(fold(["ffmpeg", "-loop", "1", "-i", "a.png", "-vf", "scale=2:2"] + enc),+ ["ffmpeg", "-loop", "1", "-i", "a.png", "-vf", f"{conv},scale=2:2", "-c:v", "libx264", "out.mp4"])+# ------------------------------------------------------------------ consistency: MCP and installerdef test_mcp_tools_match_contract(self):"""FFMPEG_SKILL_MCP_FULL=1 is the escape hatch back to the pre-1.18.3 behaviour: every@@ -1812,7 +1866,10 @@ class ContractTests(unittest.TestCase):commands = " ".join(json.loads(proc.stdout)["commands"])self.assertEqual(commands.count(conv), expected, commands)if src == self.src:- self.assertNotIn("colorspace=", commands)+ # the source is not converted (overlay --image converts its logo, and labels+ # the untagged main BT.709 for the logo to meet: test_still_and_rgb_tools_plan...)+ self.assertNotIn("colorspace=ispace", commands)+ self.assertEqual(commands.count("setparams=colorspace=bt709"), 1 if "--image" in args else 0, commands)# the other way round: the BT.601 file as the second input is converted in its own chainfor name, args in (("join", [self.camb, s601]), ("broll", [self.camb, "--insert", s601, "--at", "1", "--duration", "1"]),("overlay", [self.camb, "--video", s601])):@@ -1820,6 +1877,55 @@ class ContractTests(unittest.TestCase):commands = " ".join(json.loads(tool(name, *args, "-o", self.out(f"bt601_2nd_{name}.mp4"), "--dry-run", "--json").stdout)["commands"])self.assertEqual(commands.count(conv), 1, commands)+ def test_still_and_rgb_tools_plan_the_bt709_conversion(self):+ """#332's follow-ups, planned: an RGB source (a PNG still, a PNG sequence, a qtrle clip) and a+ JPEG are converted to BT.709 by every tool that re-encodes them; a still composited over the+ video (overlay --image) is converted at the end of its own chain, and an untagged main labelled+ BT.709 for it; grid.py converts each cell by its own tags and labels an untagged cell only+ beside a converted one. A grid of untagged/BT.709 clips plans exactly what it did."""+ s601 = OUT / "c_source601.mp4"+ if not s601.exists():+ ffmpeg("-i", self.src, "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-x264-params",+ "colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m", "-c:a", "copy", s601)+ s709 = OUT / "c_source709.mp4"+ if not s709.exists():+ ffmpeg("-i", self.src, "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-x264-params",+ "colorprim=bt709:transfer=bt709:colormatrix=bt709", "-c:a", "copy", s709)+ png, jpg, qtrle, seq = OUT / "c_still.png", OUT / "c_still.jpg", OUT / "c_rgb.mov", OUT / "c_seq"+ ffmpeg("-f", "lavfi", "-i", "testsrc2=size=160x90", "-frames:v", "1", "-pix_fmt", "rgb24", png)+ ffmpeg("-i", png, "-pix_fmt", "yuvj420p", jpg)+ ffmpeg("-f", "lavfi", "-i", "testsrc2=size=160x90:rate=10", "-t", "1", "-c:v", "qtrle", "-pix_fmt", "rgb24", qtrle)+ seq.mkdir(exist_ok=True)+ for i in range(3):+ shutil.copyfile(png, seq / f"f_{i:04d}.png")+ rgb, bt601 = "scale=out_color_matrix=bt709:out_range=tv,format=yuv444p", "colorspace=ispace=bt470bg:"+ still = "scale=out_color_matrix=bt709:out_range=tv,format=yuva420p"+ label = "setparams=colorspace=bt709:range=tv"+ cases = [ # (tool, args, the strings the plan holds, and how many times)+ ("insert", [png, "--duration", "1"], {rgb: 1, bt601: 0, label: 0}),+ ("insert", [jpg, "--duration", "1"], {rgb: 0, bt601: 1, label: 0}),+ ("sequence", ["--dir", seq, "--pattern", "f_%04d.png"], {rgb: 1, label: 0}),+ ("fit", [qtrle, "--width", "320", "--height", "180"], {rgb: 1, label: 0}),+ ("export", [qtrle, "--preset", "x"], {rgb: 1, label: 0}),+ ("overlay", [self.src, "--image", png], {still: 1, label: 1, rgb: 0}),+ ("overlay", [self.src, "--image", jpg], {still: 1, label: 1, bt601: 0}),+ ("overlay", [s709, "--image", self.logo], {still: 1, label: 0}),+ ("overlay", [s601, "--image", self.logo], {still: 1, label: 0, "colorspace=ispace=smpte170m:": 1}),+ ("overlay", [self.src, "--image", self.logo, "--codec", "prores"], {still: 0, label: 0}),+ ("waveform", [self.src, "--image", png], {"scale=out_color_matrix=bt709:out_range=tv,format=yuv420p": 1}),+ ("grid", [self.src, s601, "--cols", "2", "--rows", "1"], {"colorspace=ispace=smpte170m:": 1, label: 1}),+ ("grid", [s601, self.src, png, s709, "--cols", "2", "--rows", "2"], {"colorspace=ispace=smpte170m:": 1, label: 1, rgb: 1}),+ ("grid", [self.src, s709, "--cols", "2", "--rows", "1"], {"colorspace=": 0, "out_color_matrix": 0, "setparams": 0}),+ ]+ for n, (name, args, want) in enumerate(cases):+ with self.subTest(tool=name, args=[Path(str(a)).name for a in args]):+ ext = ".mov" if "prores" in args else ".mp4"+ proc = tool(name, *args, "-o", self.out(f"rgbstill_{n}{ext}"), "--dry-run", "--json", check=False)+ self.assertEqual(proc.returncode, 0, proc.stderr[-600:])+ commands = " ".join(json.loads(proc.stdout)["commands"])+ for s, count in want.items():+ self.assertEqual(commands.count(s), count, f"{s!r} in {commands}")+def test_json_success_and_failure_shapes_and_exit_codes(self):doc = json.loads(tool("cut", self.wav, "--start", "1", "--end", "3", "-o", self.out("shape.wav"), "--json").stdout)self.assertEqual(doc["status"], "completed")diff --git a/tests/test_picture.py b/tests/test_picture.pyindex 5298817..dba62ce 100644--- a/tests/test_picture.py+++ b/tests/test_picture.py@@ -2310,6 +2310,170 @@ class Bt601ConversionTests(unittest.TestCase):self.assertGreater(self._rgb_psnr(self.clips[name], "bt709", out, "bt709", seek), 45.0, tool)+class RgbAndStillConversionTests(unittest.TestCase):+ """2.10.1 converted a BT.601-tagged video source to the BT.709 its re-encode is tagged with; an+ RGB picture (a PNG still or logo, an emoji PNG, a PNG sequence, a qtrle clip), a JPEG still and+ grid.py's cells were still made YUV by FFmpeg's own default, BT.601, and tagged BT.709: pure+ red decoded as (255,24,0) and pure green as (0,216,0) on 5.1.1, 6.1, 7.1 and 8.1 (mean+ ΔRGB 8.8, max 38.9 over the three patches below).++ Each output is decoded here, in Python, with the matrix and range its own tags name, and the+ solid patches are compared with the colours the source holds (the PNG's RGB; the JPEG decoded+ as BT.601 full range, what JPEG is). Converted, every patch is within ~1.6 of its colour."""++ COLOURS = [(255, 0, 0), (0, 255, 0), (40, 120, 200)]+ BAR = 64 # the still is three BAR x BAR bars side by side++ @classmethod+ def setUpClass(cls):+ encoders = subprocess.run(["ffmpeg", "-hide_banner", "-encoders"], stdout=subprocess.PIPE, text=True).stdout+ if "libx264" not in encoders:+ raise unittest.SkipTest("this ffmpeg build has no libx264")+ cls.dir = OUT / "rgbstill"+ cls.dir.mkdir(parents=True, exist_ok=True)+ d, bar = cls.dir, cls.BAR+ row = b"".join(bytes(cls.COLOURS[x // bar]) for x in range(3 * bar))+ (d / "bars.ppm").write_bytes(b"P6\n%d %d\n255\n" % (3 * bar, bar) + row * bar)+ q = ("ffmpeg", "-y", "-hide_banner", "-loglevel", "error")+ sh(*q, "-i", d / "bars.ppm", "-pix_fmt", "rgb24", d / "bars.png")+ sh(*q, "-i", d / "bars.ppm", "-pix_fmt", "pal8", d / "bars_pal8.png")+ sh(*q, "-i", d / "bars.ppm", "-q:v", "1", "-pix_fmt", "yuvj420p", d / "bars.jpg")+ (d / "seq").mkdir(exist_ok=True)+ for i in range(1, 11):+ shutil.copyfile(d / "bars.png", d / "seq" / f"f_{i:04d}.png")+ cls.emoji = d / "emoji"+ cls.emoji.mkdir(exist_ok=True)+ # a solid red 'emoji' (the asset is named by code point, like _emoji_assets())+ (d / "red.ppm").write_bytes(b"P6\n72 72\n255\n" + bytes((255, 0, 0)) * (72 * 72))+ sh(*q, "-i", d / "red.ppm", "-pix_fmt", "rgba", cls.emoji / "1f389.png")+ x264 = ("-c:v", "libx264", "-preset", "ultrafast", "-crf", "1", "-pix_fmt", "yuv420p")+ tags = {"bt709": ("-x264-params", "colorprim=bt709:transfer=bt709:colormatrix=bt709"), "untagged": (),+ "smpte170m": ("-x264-params", "colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m")}+ loop = (*q, "-loop", "1", "-framerate", "25", "-i", d / "bars.png", "-t", "1")+ # the bars as a base to composite on, 768x256 with audio, BT.709-tagged and untagged (encoded+ # with BT.709, what players take untagged video as): a shifted main picture shows as well+ for name, tag in (("base709", "bt709"), ("baseun", "untagged")):+ sh(*loop, "-f", "lavfi", "-i", "sine=d=1", "-vf", "scale=768:256:flags=neighbor:out_color_matrix=bt709:out_range=tv,format=yuv420p",+ *x264, *tags[tag], "-c:a", "aac", "-shortest", d / f"{name}.mp4")+ # the bars as video: an RGB qtrle clip, and BT.601- / BT.709-tagged and untagged clips encoded with their own matrix+ sh(*loop, "-vf", "scale=384:128:flags=neighbor", "-c:v", "qtrle", "-pix_fmt", "rgb24", d / "qtrle.mov")+ for name, m, tag in (("v601", "bt601", "smpte170m"), ("v709", "bt709", "bt709"), ("vun", "bt709", "untagged")):+ sh(*loop, "-vf", f"scale=384:128:flags=neighbor:out_color_matrix={m}:out_range=tv,format=yuv420p",+ *x264, *tags[tag], d / f"{name}.mp4")+ cls.jpeg = cls._patches(d / "bars.jpg", cls._bars(0, 0), t=0) # what the JPEG holds, decoded as JPEG++ @classmethod+ def _bars(cls, x, y, scale=1, inset=12):+ b = cls.BAR * scale+ return [(x + k * b + inset, y + inset, x + (k + 1) * b - inset, y + b - inset) for k in range(3)]++ @staticmethod+ def _patches(path, rects, t=0.5):+ """Mean RGB of each rect of the frame at `t`, decoded with the matrix and range the stream is+ tagged with (BT.601 for a smpte170m/bt470bg tag or an untagged JPEG, else BT.709)."""+ s = json.loads(subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0", "-show_entries",+ "stream=width,height,pix_fmt,color_space,color_range", "-of", "json", str(path)],+ stdout=subprocess.PIPE, text=True, check=True).stdout)["streams"][0]+ w, h, pf, cs = s["width"], s["height"], s["pix_fmt"], s.get("color_space") or "unknown"+ assert pf in ("yuv420p", "yuvj420p"), pf+ raw = subprocess.run(["ffmpeg", "-v", "error", *(["-ss", str(t)] if t else []), "-i", str(path), "-frames:v", "1",+ "-f", "rawvideo", "-pix_fmt", pf, "-"], stdout=subprocess.PIPE, check=True).stdout+ bt601 = cs in ("smpte170m", "bt470bg") or (cs == "unknown" and pf == "yuvj420p")+ kr, kb = (0.299, 0.114) if bt601 else (0.2126, 0.0722)+ full = pf == "yuvj420p" or s.get("color_range") == "pc"+ cw, ch = w // 2, h // 2+ out = []+ for x0, y0, x1, y1 in rects:+ mean = lambda off, stride, xs, ys: sum(raw[off + yy * stride + xx] for yy in ys for xx in xs) / (len(xs) * len(ys))+ y_ = mean(0, w, range(x0, x1), range(y0, y1))+ u = mean(w * h, cw, range(x0 // 2, x1 // 2), range(y0 // 2, y1 // 2)) - 128+ v = mean(w * h + cw * ch, cw, range(x0 // 2, x1 // 2), range(y0 // 2, y1 // 2)) - 128+ ys, cs_ = (1.0, 1.0) if full else (255 / 219, 255 / 224)+ Y = (y_ - (0 if full else 16)) * ys+ r, b = Y + 2 * (1 - kr) * v * cs_, Y + 2 * (1 - kb) * u * cs_+ g = (Y - kr * r - kb * b) / (1 - kr - kb)+ out.append(tuple(max(0.0, min(255.0, c)) for c in (r, g, b)))+ return out++ def _assert_colours(self, out, rects, want, t=0.5):+ v = probe(str(out))["video"]+ self.assertEqual(v["color_space"], "bt709", f"{out.name}: the output is not tagged BT.709")+ got = self._patches(out, rects, t)+ diffs = [abs(a - b) for g, w in zip(got, want) for a, b in zip(g, w)]+ # measured after the fix: mean <= 0.65, max <= 1.6 (pal8: 0.94 / 2.6, the PNG is dithered);+ # before it: mean >= 8.0, max >= 24 (BT.601 pixels under a BT.709 tag)+ self.assertLess(sum(diffs) / len(diffs), 3.0, f"{out.name}: {[tuple(round(c) for c in g) for g in got]} != {want}")+ self.assertLess(max(diffs), 8.0, f"{out.name}: {[tuple(round(c) for c in g) for g in got]} != {want}")++ def test_an_rgb_or_jpeg_source_is_converted_to_the_bt709_it_is_tagged_with(self):+ """Follow-up 1 of #332: RGB sources and JPEG stills, through every tool that takes one."""+ d = self.dir+ runs = [+ ("insert.py", [d / "bars.png", "--duration", "1", "--width", "384", "--height", "128"], self.COLOURS, 0.5),+ ("insert.py", [d / "bars_pal8.png", "--duration", "1", "--width", "384", "--height", "128"], self.COLOURS, 0.5),+ ("insert.py", [d / "bars.jpg", "--duration", "1", "--width", "384", "--height", "128"], self.jpeg, 0.5),+ ("sequence.py", ["--dir", d / "seq", "--pattern", "f_%04d.png", "--start-number", "1", "--fps", "10",+ "--width", "384", "--height", "128"], self.COLOURS, 0.3),+ ("fit.py", [d / "qtrle.mov", "--width", "384", "--height", "128"], self.COLOURS, 0.5),+ ("export.py", [d / "qtrle.mov", "--preset", "youtube", "--no-scale"], self.COLOURS, 0.5),+ ]+ for n, (tool, args, want, t) in enumerate(runs):+ with self.subTest(tool=tool, source=Path(str(args[0])).name):+ out = d / f"rgb_{n}_{tool[:-3]}.mp4"+ script(tool, *args, "--fast", "-o", out)+ self._assert_colours(out, self._bars(0, 0, 2), want, t)++ def test_a_still_composited_over_the_video_is_converted_to_bt709(self):+ """Follow-up 2 of #332: overlay.py --image (render.py's image/logo layers), waveform.py --image+ (the audiogram template) and graphics/caption emoji PNGs. 7.1+ already converted an RGBA layer+ by the main input's tag, so over a BT.709-tagged base only 5.1/6.1 were wrong; over an+ untagged one 7.1+ negotiates one colourspace for both pictures and took "unknown" as BT.601.+ The main's own bars are checked too: overlay's format=auto composited an RGBA logo in RGB, so+ the WHOLE picture went through RGB and came back BT.601 (9.04 mean / 40.6 max on a BT.709 main,+ 7.93 / 37.4 on an untagged one, 5.1.1 and 6.1; the untagged one on 7.1.1 and 8.1.3 too)."""+ d = self.dir+ main = [(k * 256 + 20, 20, k * 256 + 236, 100) for k in range(3)] # the base's bars, clear of the logo+ for img, base, want in (("bars.png", "baseun", self.COLOURS), ("bars.png", "base709", self.COLOURS),+ ("bars.jpg", "base709", self.jpeg), ("bars.jpg", "baseun", self.jpeg)):+ out = d / f"ov_{img.replace('.', '_')}_{base}.mp4"+ script("overlay.py", d / f"{base}.mp4", "--image", d / img, "--position", "bottom-right", "--margin", "0", "--fast", "-o", out)+ with self.subTest(tool="overlay.py", image=img, base=base, part="the image"):+ self._assert_colours(out, self._bars(576, 192), want)+ with self.subTest(tool="overlay.py", image=img, base=base, part="the main picture"):+ self._assert_colours(out, main, self.COLOURS)+ with self.subTest(tool="waveform.py"):+ out = d / "wave_png.mp4"+ script("waveform.py", d / "baseun.mp4", "--image", d / "bars.png", "--width", "384", "--height", "128",+ "--vis-height", "0.1", "--opacity", "0", "--fast", "-o", out)+ self._assert_colours(out, self._bars(0, 0, 2), self.COLOURS)+ filters = subprocess.run(["ffmpeg", "-hide_banner", "-filters"], stdout=subprocess.PIPE, text=True).stdout+ with self.subTest(tool="graphics.py --emoji png"):+ if not re.search(r"^\s*\S+\s+ass\s", filters, re.M):+ self.skipTest("this ffmpeg build has no ass filter")+ out = d / "emoji_png.mp4"+ doc = json.loads(script("graphics.py", d / "baseun.mp4", "--template", "sticker", "--text", "\U0001F389",+ "--position", "bottom-right", "--emoji", "png", "--emoji-assets", self.emoji,+ "--emoji-scale", "2", "--fast", "--json", "-o", out).stdout)+ m = re.search(r"scale=(\d+):\d+.*overlay=x=(\d+):y=(\d+)", " ".join(doc["commands"]))+ box, x, y = (int(g) for g in m.groups())+ self._assert_colours(out, main, self.COLOURS)+ self._assert_colours(out, [(x + 8, y + 8, x + box - 8, y + box - 8)], [(255, 0, 0)])++ def test_grid_converts_each_cell_by_its_own_tags(self):+ """Follow-up 3 of #332: grid.py relabelled every cell BT.709 (5.1.1, 6.1); on 7.1+ xstack+ negotiated one colourspace for every cell and converted the others to it, so a BT.601 cell+ took a BT.709 one with it, and an untagged cell beside a converted one was converted as+ BT.601. Every order, every cell."""+ d = self.dir+ for order in (("v601", "v709"), ("v709", "v601"), ("v601", "vun"), ("vun", "v601")):+ out = d / f"grid_{order[0]}_{order[1]}.mp4"+ script("grid.py", *[d / f"{o}.mp4" for o in order], "--cols", "2", "--rows", "1", "--cell-width", "384",+ "--cell-height", "128", "--label", "none", "--fast", "-o", out)+ for c, name in enumerate(order):+ with self.subTest(order=order, cell=name):+ self._assert_colours(out, self._bars(384 * c, 0, 2), self.COLOURS)++class ScriptFontTests(unittest.TestCase):"""1.12: which script text is written in, and a font file that actually covers it."""