Skip to content

Local video editor for coding agents: 42 FFmpeg tools with a machine-readable contract and an MCP server, for Claude Code, Cursor and Codex. Cut, captions, reframe, loudness, platform exports. No cloud, no API keys.

feat: structured refusals, diagnostics and an audit log (silent paths, part 1 of 3) (#349)

Shogo Kajiyama658f854d75d7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c90695a..fe598c6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,16 @@
## Unreleased
-(nothing yet)
+### Added
+
+- **Refusals say why in fields, not only in prose.** A refusal's `error` now carries `reason`, a stable code (`output_exists`, `output_is_input`, `output_is_directory`, `output_dir_missing`, `output_dir_not_writable`, `output_locked`, and `render.py`'s `plan_inputs_changed`, `plan_version_unsupported`, `plan_unknown_tool`, `plan_cwd_missing`, `plan_argv_invalid`), and `details`, the facts the message is written from. `contract --json` lists the codes under `json_output.error_reasons`. `render.py` forwards a stage's or a plan's tool's `reason` and `details` in its own `error`. Additive: `kind`, `message`, `code` and `hint` are unchanged.
+- **`diagnostics`: what a run decided on its own.** Success and failure documents carry `diagnostics: {warnings, fallbacks, skipped}` (each entry `{code, message, ...facts}`) when the run recorded any, beside the stderr line it always printed. `render.py` carries its stages' entries, tagged with `stage`.
+- **A refused plan is explained and logged.** `render.py PLAN` refusing a plan whose inputs changed lists every input as `{path, status, expected: {size, sha256_head_tail}, actual}` (`actual: "missing"` for a file that is gone) with `plan_sha256` and `checked_at` in `error.details`, and appends one JSON line to `<plan>.audit.jsonl` next to the plan, or to the new `render.py --audit-log FILE`. A log that cannot be written is a `diagnostics.warnings` entry, never the failure; a `--dry-run` writes none.
+- **Plans name the inputs they could not bind.** `--plan` writes `unbound_inputs`, the inputs that did not exist yet (an earlier step's output) and so carry no fingerprint; executing the plan reports them as `unbound_inputs` and a `plan_unbound_inputs` warning. `plan_version` stays 1.
+
+### Changed
+
+- **A live run keeps its output lock until its own time limit has passed.** A lock older than an hour was taken over even while its pid was running, so an encode longer than an hour could lose its lock to a second writer. The writer now dates its lock to its deadline (4x `--timeout` + 60 s; never with `--timeout 0`), and a live holder's lock is taken over only after that; a dead pid's lock is still taken over at once. A lock written by an earlier version is read as before (an hour). A takeover is recorded (`diagnostics.fallbacks`, `stale_lock_taken_over`, and a `lock_takeover` line in the hidden `.<output>.ffskill-audit.jsonl` or `--audit-log`), and so is running without a lock where none can be created (`output_unlocked`). A lock refusal names the holder's pid and the lock file in `error.details`.
## 2.10.11
diff --git a/README.md b/README.md
index 425fc5e..2c403f3 100644
--- a/README.md
+++ b/README.md
@@ -338,7 +338,7 @@ The contract is generated from the code that runs, not maintained beside it. For
Next to the tool list the document carries a top-level `deprecated` list (1.10): what 2.0.0 removes, since when, the replacement and the surface it lives on. `docs/contract.md` "What 2.0 changes" is written from it.
-`contract_version` (1.0) is separate from the skill version, so a consumer can pin the shape and read the version for provenance. The document also states the invocation mapping (structured arguments → argv), the JSON shapes for success and failure (`{"status": "failed", "error": {"kind": "input | ffmpeg | output | missing_tool | timeout | verification | interrupted", "message": …}}`), and that no tool runs a shell or executes anything other than the named script, `ffmpeg` and `ffprobe`. Field-by-field reference: [docs/contract.md](docs/contract.md).
+`contract_version` (1.0) is separate from the skill version, so a consumer can pin the shape and read the version for provenance. The document also states the invocation mapping (structured arguments → argv), the JSON shapes for success and failure (`{"status": "failed", "error": {"kind": "input | ffmpeg | output | missing_tool | timeout | verification | interrupted", "message": …}}`; a refusal adds `error.reason`, a stable code such as `output_exists` or `plan_inputs_changed`, and `error.details`, the facts behind it; warnings, fallbacks and skipped steps a run decided on its own are in `diagnostics`), and that no tool runs a shell or executes anything other than the named script, `ffmpeg` and `ffprobe`. Field-by-field reference: [docs/contract.md](docs/contract.md).
### MCP
@@ -388,7 +388,8 @@ The short list for humans. The agent-facing version, with the reasoning, is the
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
-- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, before anything runs); pass `--overwrite` where a replacement is intended.
+- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, `error.reason: output_exists`, before anything runs); pass `--overwrite` where a replacement is intended.
+- **Nothing happens silently.** A refusal names its `error.reason` and `error.details`; a stale output lock taken over, a run without a lock and a plan input nobody could fingerprint are recorded in `diagnostics` and on stderr. A refused plan and a lock takeover also append a line to an audit log (`<plan>.audit.jsonl`, or `render.py --audit-log FILE`).
- **Long chains belong in a plan.** Three hand-chained re-encodes lose quality and are hard to change; `render.py` runs the whole edit from one JSON file, and `--dry-run` shows every ffmpeg command before anything is written.
## FAQ
@@ -403,7 +404,7 @@ ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools
### Does it validate the filter graph before running?
-No. `--dry-run` prints the ffmpeg command a writing tool would run and does not start ffmpeg, so ffmpeg never parses that filter graph. ffprobe still runs when the tool probes an input that exists. Unless `--overwrite` is passed, an existing output is refused (`kind: input`) before ffmpeg. An output that is the same file as an input is refused either way. No media file is written, and `verified` is false; `--plan FILE` does write the plan JSON. A path that is not on disk yet is not measured: the probe is a stub (0×0, duration 0), so a number taken from it is not a measurement. The measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) still run their measurements; only the artifact is skipped, and `silence` with `--filler --transcribe` also runs the speech engine, which may download a model. `overlay` measures the lines of a multi-line `--text` with one drawtext on a generated frame on FFmpeg 8.0, 8.0.1 and 8.1.0, where it draws them one by one. `verify` accepts `--dry-run` and ignores it: its steps run. `--plan FILE` is that dry run written down — the planned argv, the command lines, and a fingerprint of each existing input (size, plus a sha256 of the first and last 8 MiB). `render.py` and `verify.py` refuse `--plan`. `render.py FILE` runs a plan only when those fingerprints still match, by running the tool again with the planned argv, not by executing the recorded ffmpeg lines; `--dry-run` on that run only prints them. `doctor` reports whether the filters, encoders and bitstream filters the contract names are in this build (`ffmpeg -filters`, `-encoders`, `-bsfs`) and which fonts are installed. A name in that listing is not a check of the graph this command would build, and a listed GPU encoder is not a successful encode.
+No. `--dry-run` prints the ffmpeg command a writing tool would run and does not start ffmpeg, so ffmpeg never parses that filter graph. ffprobe still runs when the tool probes an input that exists. Unless `--overwrite` is passed, an existing output is refused (`kind: input`) before ffmpeg. An output that is the same file as an input is refused either way. No media file is written, and `verified` is false; `--plan FILE` does write the plan JSON. A path that is not on disk yet is not measured: the probe is a stub (0×0, duration 0), so a number taken from it is not a measurement. The measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) still run their measurements; only the artifact is skipped, and `silence` with `--filler --transcribe` also runs the speech engine, which may download a model. `overlay` measures the lines of a multi-line `--text` with one drawtext on a generated frame on FFmpeg 8.0, 8.0.1 and 8.1.0, where it draws them one by one. `verify` accepts `--dry-run` and ignores it: its steps run. `--plan FILE` is that dry run written down — the planned argv, the command lines, and a fingerprint of each existing input (size, plus a sha256 of the first and last 8 MiB). `render.py` and `verify.py` refuse `--plan`. `render.py FILE` runs a plan only when those fingerprints still match (a mismatch is refused with each input's expected and actual fingerprint in `error.details`, and logged to `<plan>.audit.jsonl`), by running the tool again with the planned argv, not by executing the recorded ffmpeg lines; `--dry-run` on that run only prints them. `doctor` reports whether the filters, encoders and bitstream filters the contract names are in this build (`ffmpeg -filters`, `-encoders`, `-bsfs`) and which fonts are installed. A name in that listing is not a check of the graph this command would build, and a listed GPU encoder is not a successful encode.
### Can it remove burned-in subtitles or logos?
diff --git a/docs/contract.md b/docs/contract.md
index 761299f..6bdc246 100644
--- a/docs/contract.md
+++ b/docs/contract.md
@@ -259,16 +259,23 @@ asks for the CPU and gets what ffmpeg says about it. What ran is in the result's
`--plan FILE` (1.6) is a dry run that also writes a plan document: `{"plan_version": 1,
"tool", "argv", "cwd", "inputs": [{"path", "size", "sha256_head_tail"}], "commands",
-"output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
+"unbound_inputs", "output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
implies `--dry-run`, so the same execution rules apply; `inputs` covers the `-i` files of the
planned commands, every existing file named in argv (a recipe, a still, an SRT) and the
-subtitle/LUT/font files a filter reads. Tools that print their document without `--json`
+subtitle/LUT/font files a filter reads; `unbound_inputs` (absolute paths, additive to version 1)
+names the inputs that did not exist yet when the plan was made (an earlier step's output), which
+therefore carry no fingerprint. Tools that print their document without `--json`
(`probe`, the analysis tools) still write the plan at exit; `verify.py` and `render.py`
refuse `--plan` (their steps run for real; a project file is already a plan). `render.py FILE` executes a plan:
-it refuses (`kind: input`) when an input's size or head/tail hash differs from the plan,
+it refuses (`kind: input`, `error.reason: plan_inputs_changed`) when an input's size or head/tail
+hash differs from the plan, with every input's `{path, status, expected, actual}` in
+`error.details.inputs` (`actual` is `"missing"` for a file that is gone) beside `plan_sha256` and
+`checked_at`, and appends one record to the audit log (below); a wrong `plan_version`, an unknown
+tool, a vanished `cwd` or a malformed `argv` is refused the same way under its own reason. It then
runs the tool with the planned `argv`, then the verify steps (a `check` step may also carry
-`"content": true`, added by hand, for `check.py --content`), and reports `plan`, `tool`,
-`tool_result` and `check`. `plan_version` is bumped when the document's shape changes.
+`"content": true`, added by hand, for `check.py --content`), and reports `plan`, `plan_sha256`,
+`tool`, `tool_result`, `check` and `unbound_inputs` (each one also a `diagnostics.warnings` entry:
+nothing checked that file). `plan_version` is bumped when the document's shape changes.
### Repeatability
@@ -732,6 +739,39 @@ strings. `retryable` is currently always `false`: none of the kinds are distingu
today from a deterministic failure that would fail identically on a blind retry, so nothing
here claims otherwise until real exit-code/stderr sniffing exists to back that up.
+A refusal also carries `error.reason`, a stable snake_case code for *why*
+(`contract --json` → `json_output.error_reasons` lists them: `output_exists`, `output_is_input`,
+`output_is_directory`, `output_dir_missing`, `output_dir_not_writable`, `output_locked`, and
+`render.py`'s `plan_*` reasons), and `error.details`, the facts the message is written from, so a
+caller never parses the prose:
+
+```
+"error": {"kind": "input", "message": "refusing to overwrite existing output 'out.mp4': ...",
+ "code": "INPUT_INVALID", "retryable": false, "hint": "pass --overwrite, or choose another -o path",
+ "reason": "output_exists",
+ "details": {"outputs": [{"path": "out.mp4", "resolved": "/abs/out.mp4", "size": 48213, "mtime_ns": 1729660000000000000}]}}
+```
+
+A tool that runs another (`render.py` stages and plans) forwards the child's `reason` and
+`details` in its own `error`. Both are absent on a failure that is not a refusal, like `hint`.
+
+**Diagnostics.** What a run decided on its own -- a warning, a fallback it took, a step it
+skipped -- is recorded as it happens (a stderr line too) and lands in the document as
+`diagnostics: {warnings, fallbacks, skipped}`, each entry `{code, message, ...facts}`; it is
+present on success and failure documents alike, only when one list is not empty, and an entry a
+`render.py` stage reported carries `stage`. Codes today: `stale_lock_taken_over` (fallbacks),
+`output_unlocked`, `lock_deadline_unset`, `audit_log_unwritable` and `plan_unbound_inputs`
+(warnings).
+
+**Audit log.** Events a later reader must be able to reconstruct are appended as JSON lines,
+`{ts, tool, event, pid, plan_sha256, details}` (`plan_sha256` on plan events): `plan_rejected`,
+`lock_takeover`, `output_unlocked`, and `output_is_input_refused` when a log was named. The file
+is `render.py --audit-log FILE` when given; otherwise a refused plan's `<plan>.audit.jsonl` next
+to the plan, and a lock event's hidden `.<output>.ffskill-audit.jsonl` beside the output. Nothing
+is appended under `--dry-run`. A record that cannot be written is a `diagnostics.warnings` entry
+(`audit_log_unwritable`) and never fails or blocks the run; the refusal's `details.audit_log`
+says `{path, written}` either way.
+
## MCP relationship
`mcp/server.py` is a transport. It holds no tool table and no schema of its own:
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index 715c515..594769f 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -1756,3 +1756,68 @@ not a new file format this tool would have to maintain.
`doctor --json` describes, so it has a key (`ffmpeg_security.warnings`). The 3.9 floor and CI's
3.9 job stay until 3.0. Tests: `test_python_3_9_is_deprecated_on_every_surface`,
`test_doctor_prints_the_python_3_9_warning_on_stderr_only`, `test_installer_warns_once_on_python_3_9`.
+
+## Unreleased — nothing happens silently
+
+The principle: every refusal, rejection, skip, fallback or swallowed error leaves a structured
+record a caller can read -- fields in the `--json` document, not only prose in `error.message` --
+plus its stderr line. The entries below are the infrastructure and the first decisions taken
+under it; the rest of the silent paths are converted onto the same infrastructure in later PRs.
+
+- **A refusal is `refuse(kind, reason, message, **details)`: `error.reason` is a stable code and
+ `error.details` the facts.** `kind` keeps its meaning (the failure class, mapped 1:1 to `code`);
+ `reason` says *why* within it, from a closed list the contract publishes
+ (`json_output.error_reasons`), and `details` carries the values the message is written from
+ (paths, expected and actual). Both are absent on a failure that is not a refusal, the way `hint`
+ is, so the failure example in `docs/contract.md` keeps its four keys. `reason` lives inside
+ `error` because the top-level `reason` is already taken by `caption.py`'s `no_speech` refusal.
+ Code: `_common.emit.refuse()`, the runner's output guards. Tests:
+ `test_refusals_at_the_runner_choke_points_carry_a_reason_and_details`,
+ `test_every_refusal_reason_is_in_the_contract_and_the_runner_guards_use_refuse` (a source
+ check: every `refuse()` reason literal is in the contract and the guards never `die()` in prose).
+- **The collector's records are under `diagnostics`, not top-level `warnings`/`skipped`.**
+ `check.py`'s `warnings` is a count and `join.py`'s `skipped` its left-out inputs, so top-level
+ keys of those names would mean two things. `diagnostics: {warnings, fallbacks, skipped}` holds
+ `{code, message, ...facts}` entries recorded by `warn()`/`record_fallback()`/`record_skip()`
+ (each also prints its line), merged by `emit()` and `die()` as `hw_notes` becomes `hw.notes`,
+ present only when an entry exists. `render.py` takes a stage's entries with `absorb_stage()`,
+ adding `stage`. Code: `_common.emit._merge_collector()`, `absorb_stage()`. Test:
+ `test_a_stale_lock_taken_over_is_recorded_in_the_document_and_the_audit_log`.
+- **The audit log is appended only where an event names a place, and never fails the run.**
+ JSON lines `{ts, tool, event, pid, plan_sha256, details}`. A refused plan goes to
+ `<plan>.audit.jsonl` (the plan is what the reader will look at); a lock event to the hidden
+ `.<output>.ffskill-audit.jsonl` beside the lock it concerns; `render.py --audit-log FILE`
+ replaces both. An event with neither (an output-is-input refusal without `--audit-log`) is in
+ the document only: a log appearing next to arbitrary outputs unasked would contradict "a plan
+ leaves nothing behind". A failed write is a `diagnostics.warnings` entry
+ (`audit_log_unwritable`), and the refusal's `details.audit_log` says `{path, written}`. Nothing
+ is appended under `--dry-run`, even for a refusal the dry run predicts: a dry run writes
+ nothing, and `details.audit_log` then says `reason: "dry_run"`. Code: `_common.emit.audit()`,
+ `render._reject_plan()`. Tests: `test_a_refused_plan_says_what_changed_and_leaves_an_audit_record`,
+ `test_a_failed_audit_write_is_a_warning_not_a_failure`.
+- **A refused plan lists every input, not only the bad ones.** `details.inputs` is
+ `[{path, status: unchanged|changed|missing, expected, actual}]` so a caller sees the whole
+ binding at once; `actual` is the string `"missing"` for a file that is gone. The message keeps
+ the per-file prose and the word "changed". Inputs that did not exist when the plan was made are
+ recorded in the plan as `unbound_inputs` (additive: `plan_version` stays 1) and reported on
+ execution as a warning, not refused: a plan over an earlier step's output is the use case
+ `join.py`'s pending inputs exist for. Code: `render.execute_plan()`, `_common.emit._plan_unbound_inputs()`.
+ Test: `test_a_plan_names_the_inputs_it_could_not_bind_and_render_reports_them`.
+- **A live holder's output lock is taken over only after its own time limit.** Through 2.10 a lock
+ older than an hour was taken over even while its pid ran, so a legitimate encode longer than an
+ hour (an `--timeout 0` master, a slow AV1 pass) lost its lock to a second writer: the F1 bug the
+ lock exists for. A live pid is not proof of a live writer (a pid can be reused), so the takeover
+ is kept, but bounded by the holder's own deadline: `child_limit()` from the lock's creation
+ (4x `--timeout` + 60 s, which covers every retry `run()` makes under one lock); with
+ `--timeout 0`, never. The writer stores the deadline as the lock's mtime minus an hour, so the
+ rule every version applies ("older than an hour") means "past the deadline" in both directions
+ and the lock file still holds only the pid. A dead pid's lock is taken over at once, as before.
+ The cost: a pid reused by an unrelated long-lived process blocks the output until the
+ deadline (about two hours by default, forever under `--timeout 0`); the refusal names the pid
+ and the lock file to delete. A takeover is a `stale_lock_taken_over` fallback plus a
+ `lock_takeover` audit line; running without a lock where none can be created is an
+ `output_unlocked` warning plus an audit line. Code: `_common.runner._OutputLock`. Tests:
+ `test_a_live_holder_keeps_its_lock_past_an_hour_while_inside_its_own_limit`,
+ `test_with_no_time_limit_a_live_holders_lock_is_never_taken_over`,
+ `test_a_lock_an_older_version_wrote_is_taken_over_after_an_hour_as_before`,
+ `test_running_without_a_lock_is_a_recorded_warning`.
diff --git a/references/scripts.md b/references/scripts.md
index 02bbce7..51683c5 100644
--- a/references/scripts.md
+++ b/references/scripts.md
@@ -1,6 +1,6 @@
# Script reference
-Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
+Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, `error.reason: output_exists`, before anything runs) to replace an existing output unless `--overwrite` is given. Refusals carry `error.reason` and `error.details` (the facts behind the message; `contract --json` → `json_output.error_reasons`), and what a run decided on its own -- a stale output lock taken over, no lock possible, a plan input nobody fingerprinted -- is in `diagnostics: {warnings, fallbacks, skipped}` as well as on stderr. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
## Time grammar (every time-taking flag, 1.9)
@@ -709,7 +709,7 @@ output; the result says so with `dropped_non_av_streams: true`.
render.py --init project.json # starter file
render.py project.json [--fast] [--dry-run] [--stop-after STAGE] [--work DIR --keep]
render.py project.json --cache DIR [--from STAGE] # reuse identical stages from a previous run
-render.py plan.json # execute a plan written by <tool> --plan plan.json
+render.py plan.json [--audit-log FILE] # execute a plan written by <tool> --plan plan.json
render.py project.json --export-timeline edit.fcpxml|edit.edl|edit.otio # the cut for an editor, nothing rendered
```
`--export-timeline FILE` (2.1) writes the project's cut as an editor timeline instead of
@@ -734,12 +734,19 @@ few frames longer, because `join.py` offsets its crossfades by each part's conta
audio included.
A plan is a single tool's dry run as an artifact: `cut.py in.mp4 --start 2 --end 8
--plan cut.json` writes `{plan_version, tool, argv, inputs (path, size, sha256 of
-head+tail), commands, output, verify}` and runs nothing. `render.py cut.json`
-re-fingerprints the inputs (refusing, `kind: input`, if any changed since the
-plan), runs the tool with the planned argv, then the verify steps (probe; `check`
+head+tail), unbound_inputs, commands, output, verify}` and runs nothing
+(`unbound_inputs`: inputs that did not exist yet, so carry no fingerprint).
+`render.py cut.json` re-fingerprints the inputs (refusing, `kind: input`,
+`error.reason: plan_inputs_changed`, if any changed since the plan: every input's
+`{path, status, expected, actual}` -- `actual` `"missing"` for a file that is gone --
+plus `plan_sha256` and `checked_at` are in `error.details`, and one JSON line goes
+to `<plan>.audit.jsonl`, or to `--audit-log FILE`; a log that cannot be written is
+a `diagnostics.warnings` entry, never the failure, and a `--dry-run` writes none),
+runs the tool with the planned argv, then the verify steps (probe; `check`
for a `--platform` or a platform export preset; add `"content": true` to a check
-step for the black/frozen/silence rows), and reports `plan`, `tool`,
-`tool_result` and `check`. Show the plan to the user, get the yes, execute:
+step for the black/frozen/silence rows), and reports `plan`, `plan_sha256`, `tool`,
+`tool_result`, `check` and `unbound_inputs` (each also a `diagnostics.warnings`
+entry: nothing checked those files). Show the plan to the user, get the yes, execute:
one round trip instead of re-deriving the command.
`"export": {"preset": "reels", "normalize": true}` forwards `export.py --normalize`
so the rendered file meets the platform's loudness without a separate pass. Since
diff --git a/scripts/_common/__init__.py b/scripts/_common/__init__.py
index 8974cf9..9f0a3bd 100644
--- a/scripts/_common/__init__.py
+++ b/scripts/_common/__init__.py
@@ -63,8 +63,9 @@ from _common.runner import (
STATE, _timed_out, _unwatch, _watch, X264_PRESETS
)
from _common.emit import (
- _brief, _BRIEF_DROP, _brief_summary, _CURRENT_CTX, die, emit, info, _plan_at_exit, _plan_inputs, _PLAN_STRIP,
- PLAN_VERSION, print_json, _set_current_ctx, write_plan
+ absorb_stage, audit, _brief, _BRIEF_DROP, _brief_summary, _CURRENT_CTX, die, emit, info, note, _plan_at_exit, _plan_inputs,
+ _plan_unbound_inputs, _PLAN_STRIP, PLAN_VERSION, print_json, record_fallback, record_skip, refuse, _set_current_ctx, warn,
+ write_plan
)
from _common.probe import (
analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
@@ -216,6 +217,7 @@ __all__ = [
"time_arg", "_timed_out", "_to_float", "_to_int", "_unwatch", "validate_color", "verify_output",
"source_codec_video_args", "drawn_chain", "draws_chroma", "drawn_stream_format", "is_neutral_color", "rgb_island", "label_chain", "matrix_chain", "rgb_chain_to_bt709", "bt709_label_filter", "rgb_to_bt709_filter", "is_rgb", "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS",
"_x264_raw", "_ZWJ",
+ "absorb_stage", "audit", "note", "_plan_unbound_inputs", "record_fallback", "record_skip", "refuse", "warn",
"ASR_ENGINES", "ASR_INSTALL_HINT", "_asr_run", "die_no_engine", "parse_srt", "transcribe",
"_transcribe_in", "transcribe_words", "_words_from_openai_whisper_json",
"_words_from_whisper_cpp_json", "whisper_word_timings", "write_srt",
diff --git a/scripts/_common/emit.py b/scripts/_common/emit.py
index e67abe7..d8d4d59 100644
--- a/scripts/_common/emit.py
+++ b/scripts/_common/emit.py
@@ -13,7 +13,8 @@ from typing import Any, Dict, List, Optional, Sequence, Tuple
from _common.runner import Context, ERROR_CODE, ERROR_RETRYABLE, STATE
-def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]" = None, **extra: Any) -> "None":
+def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]" = None,
+ error_extra: "Optional[Dict[str, Any]]" = None, **extra: Any) -> "None":
"""Exit with a message. Under --json also print a machine-readable failure document
(status: failed) on stdout so callers get the same shape as a success; exit codes are unchanged.
@@ -21,7 +22,10 @@ def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]
platform rows, render.py's check stage, batch.py's per-item results, verify.py's steps) keeps
reporting that detail while the top-level status says failed. Before 1.4.3 those four printed
`status: "completed"` next to a non-zero exit code, so a caller keying on the status alone
- read a failed delivery as a success."""
+ read a failed delivery as a success.
+
+ `error_extra` goes inside `error` (refuse() passes `reason` and `details` there); the
+ collector's records (warn() and friends) are merged into the document as emit() merges them."""
hint = extra.pop("hint", None)
ctx = ctx or STATE # 1.10: the optional per-request Context (2.0 makes it required); STATE is the default instance
_set_current_ctx(ctx) # the atexit hook has no argument: it reads the ctx emit()/die() last used
@@ -48,11 +52,103 @@ def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]
}
if hint:
doc["error"]["hint"] = hint
+ for key, value in (error_extra or {}).items():
+ if value is not None:
+ doc["error"][key] = value
doc.update(extra)
+ _merge_collector(doc, ctx)
print_json(doc)
sys.exit(code)
+def refuse(kind: str, reason: str, msg: str, *, hint: Optional[str] = None, code: int = 1,
+ ctx: "Optional[Context]" = None, extra: "Optional[Dict[str, Any]]" = None, **details: Any) -> "None":
+ """die() for a refusal a caller has to act on: `error.reason` is a stable snake_case code for
+ *why* (output_exists, output_is_input, output_locked, plan_inputs_changed, ...), and
+ `error.details` holds the facts the message is written from (paths, expected and actual
+ values), so a caller never parses the prose. `extra` adds top-level keys as die()'s do."""
+ die(msg, code=code, kind=kind, ctx=ctx, hint=hint, error_extra={"reason": reason, "details": json_safe(details)}, **(extra or {}))
+
+
+def _record(bucket: str, msg: str, code: str, ctx: "Optional[Context]", data: Dict[str, Any], prefix: str) -> Dict[str, Any]:
+ ctx = ctx or STATE
+ entry: Dict[str, Any] = {"code": code, "message": msg}
+ entry.update(data)
+ getattr(ctx, bucket).append(entry)
+ sys.stderr.write(f"{prefix}{msg}\n")
+ return entry
+
+
+def warn(msg: str, code: str, *, ctx: "Optional[Context]" = None, **data: Any) -> Dict[str, Any]:
+ """Say something went other than asked, on stderr AND in the document
+ (`diagnostics.warnings`: {code, message, ...data}). Use it where a bare info("warning: ...")
+ would leave the caller of --json nothing to read."""
+ return _record("warnings", msg, code, ctx, data, "warning: ")
+
+
+def record_fallback(msg: str, code: str, *, ctx: "Optional[Context]" = None, **data: Any) -> Dict[str, Any]:
+ """The run took another route than the asked one and went on (`diagnostics.fallbacks`)."""
+ return _record("fallbacks", msg, code, ctx, data, "")
+
+
+def record_skip(msg: str, code: str, *, ctx: "Optional[Context]" = None, **data: Any) -> Dict[str, Any]:
+ """A step that was asked for, or would normally run, did not (`diagnostics.skipped`)."""
+ return _record("skipped", msg, code, ctx, data, "")
+
+
+def note(msg: str, ctx: "Optional[Context]" = None) -> None:
+ """A plain `notes` line that also goes to stderr."""
+ ctx = ctx or STATE
+ if msg not in ctx.notes:
+ ctx.notes.append(msg)
+ sys.stderr.write(f"{msg}\n")
+
+
+def _merge_collector(doc: Dict[str, Any], ctx: "Context") -> None:
+ """The collector into a result or failure document: ctx.notes join `notes` (once each), and
+ the three record lists become `diagnostics`, present only when one of them is not empty
+ (`warnings` and `skipped` are not top-level keys: check.py's `warnings` is a count and join.py's
+ `skipped` its left-out inputs)."""
+ extra_notes = [n for n in ctx.notes if n not in (doc.get("notes") or [])]
+ if extra_notes:
+ doc["notes"] = list(doc.get("notes") or []) + extra_notes
+ diag = {k: list(getattr(ctx, k)) for k in ("warnings", "fallbacks", "skipped") if getattr(ctx, k)}
+ if diag:
+ merged = dict(doc.get("diagnostics") or {})
+ for k, v in diag.items():
+ merged[k] = list(merged.get(k) or []) + [e for e in v if e not in (merged.get(k) or [])]
+ doc["diagnostics"] = merged
+
+
+def audit(event: str, details: Dict[str, Any], *, default_path: Optional[str] = None, plan_sha256: Optional[str] = None,
+ ctx: "Optional[Context]" = None) -> Dict[str, Any]:
+ """Append one JSON line to the audit log: {ts, tool, event, pid, plan_sha256?, details}.
+ The log is render.py's --audit-log FILE when given, else the event's own `default_path` (a
+ plan's `<plan>.audit.jsonl`, a lock's hidden sibling); neither = nothing to write. Never under
+ --dry-run (a dry run writes nothing). A write that fails is a recorded warning, never a
+ failure of the run. Returns what happened, for the caller's `details.audit_log`."""
+ import datetime
+ ctx = ctx or STATE
+ path = ctx.audit_log or default_path
+ if not path:
+ return {"path": None, "written": False, "reason": "no_audit_log"}
+ if ctx.dry_run:
+ return {"path": path, "written": False, "reason": "dry_run"}
+ rec: Dict[str, Any] = {"ts": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ"),
+ "tool": os.path.splitext(os.path.basename(sys.argv[0] or "?"))[0], "event": event, "pid": os.getpid()}
+ if plan_sha256:
+ rec["plan_sha256"] = plan_sha256
+ rec["details"] = details
+ try:
+ with open(path, "a", encoding="utf-8") as f:
+ f.write(json.dumps(json_safe(rec), ensure_ascii=False, sort_keys=True) + "\n")
+ except OSError as exc:
+ warn(f"could not append the {event} record to the audit log {path} ({exc}); the run is unaffected",
+ "audit_log_unwritable", ctx=ctx, path=path, event=event, error=str(exc))
+ return {"path": path, "written": False, "reason": "unwritable", "error": str(exc)}
+ return {"path": path, "written": True}
+
+
def info(msg: str, ctx: "Optional[Context]" = None) -> None:
# under --dry-run nothing is written; do not let scripts claim otherwise
ctx = ctx or STATE
@@ -195,6 +291,24 @@ def absorb_stage_hw(stage: str, doc: Any, ctx: "Optional[Context]" = None) -> No
"used": hw.get("used"), "fallback": bool(hw.get("fallback")), "quality_note": quality})
+def absorb_stage(stage: str, doc: Any, ctx: "Optional[Context]" = None) -> Dict[str, Any]:
+ """A child tool's --json document into this result: its `hw` facts (absorb_stage_hw) and its
+ `diagnostics` records, each tagged with the stage it came from. Returns the fields a re-raise
+ of the stage's failure carries in its own `error`: the child's `reason` and `details`."""
+ ctx = ctx or STATE
+ absorb_stage_hw(stage, doc, ctx)
+ if not isinstance(doc, dict):
+ return {}
+ for bucket in ("warnings", "fallbacks", "skipped"):
+ for entry in (doc.get("diagnostics") or {}).get(bucket) or []:
+ if isinstance(entry, dict):
+ tagged = dict(entry, stage=entry.get("stage") or stage)
+ if tagged not in getattr(ctx, bucket):
+ getattr(ctx, bucket).append(tagged)
+ err = doc.get("error") if isinstance(doc.get("error"), dict) else {}
+ return {k: err[k] for k in ("reason", "details") if err.get(k) is not None}
+
+
def steps_encoder_report(steps: Sequence["Tuple[str, Any]"]) -> Dict[str, Any]:
"""batch.py's per-item `encoder`, `hw` and `notes`, from the --json documents of the steps
(child processes) that made the item: the last encoder a step ran (a later stream copy keeps
@@ -246,6 +360,7 @@ def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any
hw_notes = [n for n in _hw_notes_for(ctx) if n not in (doc.get("notes") or [])]
if hw_notes:
doc["notes"] = list(doc.get("notes") or []) + hw_notes
+ _merge_collector(doc, ctx)
if ctx.plan:
doc["plan"] = write_plan(ctx.plan, output, extra, ctx=ctx)
print_json(_brief(doc, meta) if ctx.json_brief else doc)
@@ -345,6 +460,41 @@ def _plan_inputs(commands: Sequence[str], argv: Sequence[str] = (), ctx: "Option
return seen
+def _plan_unbound_inputs(commands: Sequence[str], ctx: "Optional[Context]" = None) -> List[str]:
+ """The inputs _plan_inputs() could not fingerprint because they did not exist when the plan
+ was made: a planned command's `-i` file (or a side input registered through
+ escape_filter_path) that is not on disk and is not written by an earlier planned command (the
+ tool's own intermediates are bound by the command that makes them). render.py reports them on
+ execution: nothing checks that their content is what the plan was made for."""
+ import shlex
+ produced: List[str] = []
+ out: List[str] = []
+
+ def consider(p: str) -> None:
+ if (not p or p == "-" or p.startswith("pipe:") or os.path.exists(p)
+ or os.path.abspath(p) in produced or os.path.abspath(p) in out):
+ return
+ out.append(os.path.abspath(p))
+ for p in (ctx or STATE).plan_inputs:
+ consider(p)
+ for line in commands:
+ try:
+ toks = shlex.split(line.split("] ", 1)[1] if line.startswith("[dry-run] ") else line)
+ except ValueError:
+ continue
+ fmt = None # an input's -f (lavfi's "inputs" are filter graphs, not files)
+ for i, tok in enumerate(toks[:-1]):
+ if tok == "-f":
+ fmt = toks[i + 1]
+ elif tok == "-i":
+ if fmt != "lavfi":
+ consider(toks[i + 1])
+ fmt = None
+ if toks:
+ produced.append(os.path.abspath(toks[-1]))
+ return out
+
+
def write_plan(path: str, output: Optional[str], extra: Dict[str, Any], ctx: "Optional[Context]" = None) -> str:
"""The dry run as an artifact: what will run, on which exact inputs, producing what, checked
how. `render.py PLAN` executes it after re-fingerprinting the inputs (issue #189 C).
@@ -382,6 +532,8 @@ def write_plan(path: str, output: Optional[str], extra: Dict[str, Any], ctx: "Op
"argv": cleaned,
"cwd": os.getcwd(),
"inputs": [fingerprint(p) for p in _plan_inputs(ctx.commands, cleaned, ctx)],
+ # inputs that did not exist yet, so carry no fingerprint (render.py reports them on execution)
+ "unbound_inputs": _plan_unbound_inputs(ctx.commands, ctx),
"commands": list(ctx.commands),
"output": os.path.abspath(output) if output else None,
"verify": verify,
diff --git a/scripts/_common/runner.py b/scripts/_common/runner.py
index 7274611..21e7367 100644
--- a/scripts/_common/runner.py
+++ b/scripts/_common/runner.py
@@ -217,7 +217,8 @@ class Context:
"""
__slots__ = ("dry_run", "json", "json_brief", "progress", "fast", "duration_hint", "commands", "timeout", "overwrite", "written", "preexisting", "plan", "plan_written", "plan_inputs", "codec",
- "hw", "hw_source", "hw_notes", "hw_swaps", "hw_fallback", "hw_env_ignored", "hw_stages", "hw_auto")
+ "hw", "hw_source", "hw_notes", "hw_swaps", "hw_fallback", "hw_env_ignored", "hw_stages", "hw_auto",
+ "notes", "warnings", "fallbacks", "skipped", "audit_log")
def __init__(self) -> None:
self.reset()
@@ -246,6 +247,15 @@ class Context:
self.hw_env_ignored = False # $FFMPEG_SKILL_HW=1 was set but this tool's delivery preset does not take it (export.py)
self.hw_stages: List[Dict[str, Any]] = [] # render.py: the `hw` facts each stage (a child process) reported
self.hw_auto: List[str] = [] # CPU encoders (libx264/libx265) this ffmpeg build lacks, whose encodes ran on VideoToolbox without --hw
+ # The collector (docs/design-decisions.md, "Nothing happens silently"): what this run
+ # decided on its own -- a warning, a fallback, a step it skipped -- recorded as it happens
+ # by warn()/record_fallback()/record_skip() (stderr too) and merged by emit()/die() into
+ # the document: `notes` (strings) and `diagnostics: {warnings, fallbacks, skipped}`.
+ self.notes: List[str] = []
+ self.warnings: List[Dict[str, Any]] = []
+ self.fallbacks: List[Dict[str, Any]] = []
+ self.skipped: List[Dict[str, Any]] = []
+ self.audit_log: Optional[str] = None # render.py --audit-log FILE: where audit() appends; None = the event's own default
STATE = Context()
@@ -531,12 +541,21 @@ def _check_no_overwrite_input(cmd: Sequence[str]) -> None:
if a == "-i" and i + 1 < len(cmd):
inp = cmd[i + 1]
try:
- if os.path.realpath(inp) == out_real:
- die(f"refusing to run: output {output!r} is the same file as input {inp!r} "
- f"(would overwrite it while ffmpeg is still reading it) -- choose a different --output/-o path",
- kind="input")
+ same = os.path.realpath(inp) == out_real
except OSError:
continue
+ if same:
+ _refuse_output_is_input(output, inp, out_real, "would overwrite it while ffmpeg is still reading it")
+
+
+def _refuse_output_is_input(output: str, inp: str, real: str, why: str) -> None:
+ """The one refusal both output-is-input guards end in: structured, and in the audit log when
+ one is configured (render.py --audit-log)."""
+ details = {"output": output, "input": inp, "resolved": real}
+ audit("output_is_input_refused", details)
+ refuse("input", "output_is_input",
+ f"refusing to run: output {output!r} is the same file as input {inp!r} ({why}) -- choose a different --output/-o path",
+ **details)
def refuse_output_is_input(output: str, *inputs: str) -> None:
@@ -555,8 +574,7 @@ def refuse_output_is_input(output: str, *inputs: str) -> None:
except OSError:
continue
if same:
- die(f"refusing to run: output {output!r} is the same file as input {inp!r} "
- f"(the result would replace the source) -- choose a different --output/-o path", kind="input")
+ _refuse_output_is_input(output, inp, out_real, "the result would replace the source")
def _check_output_path(cmd: Sequence[str]) -> None:
@@ -567,12 +585,15 @@ def _check_output_path(cmd: Sequence[str]) -> None:
if output == "-" or output.startswith("pipe:") or output.startswith("-"):
return
if os.path.isdir(output):
- die(f"output {output!r} is a directory; pass a file path (e.g. {os.path.join(output, 'result.mp4')!r})")
+ refuse("input", "output_is_directory",
+ f"output {output!r} is a directory; pass a file path (e.g. {os.path.join(output, 'result.mp4')!r})", output=output)
parent = os.path.dirname(os.path.abspath(output))
if not os.path.isdir(parent):
- die(f"output directory {parent!r} does not exist; create it first (this tool never creates directories)")
+ refuse("input", "output_dir_missing",
+ f"output directory {parent!r} does not exist; create it first (this tool never creates directories)",
+ output=output, directory=parent)
if not os.access(parent, os.W_OK):
- die(f"output directory {parent!r} is not writable")
+ refuse("input", "output_dir_not_writable", f"output directory {parent!r} is not writable", output=output, directory=parent)
EVEN_SCALE = "scale=trunc(iw/2)*2:trunc(ih/2)*2"
@@ -602,14 +623,29 @@ def _pid_dead(pid: int) -> bool:
return False
+LOCK_HOUR = 3600.0
+LOCK_UNLIMITED = 10 * 365 * 86400.0 # --timeout 0: the deadline is pushed past any run (and kept short of 2038)
+
+
class _OutputLock:
"""Two runs writing the same output at once used to both report `completed` while one of
them described the other's file (sweep F1). A lock file next to the output, created with
- O_EXCL and holding the writer's pid, makes the second run refuse as `kind: input`. A lock
- whose pid is dead (POSIX) or older than an hour is stale and taken over."""
+ O_EXCL and holding the writer's pid, makes the second run refuse (`kind: input`,
+ `reason: output_locked`, the holder in `details`).
+
+ Stale locks: a lock whose pid is dead is taken over. A lock whose pid is alive is taken over
+ only once its holder's own time limit has passed: the writer sets the lock's mtime to that
+ deadline minus an hour, so "older than an hour" -- the rule every version since 1.x applies,
+ which this keeps compatible in both directions -- means "past the deadline". The deadline is
+ child_limit() from now (4x --timeout + 60 s, the ceiling on everything one run() does under
+ the lock); with --timeout 0 there is none and a live holder is never taken over. Through 2.10
+ the hour ran from the lock's creation, so a legitimate run longer than an hour lost its lock
+ to a second writer. A takeover is recorded (`diagnostics.fallbacks`, stderr, and the audit
+ log); so is running without a lock where none can be created (`diagnostics.warnings`)."""
def __init__(self, output: str) -> None:
self.path: Optional[str] = None
self.fd: Optional[int] = None
+ self.output = output
if output == "-" or output.startswith("pipe:") or output.startswith("-"):
return
d, base = os.path.split(os.path.abspath(output))
@@ -618,33 +654,95 @@ class _OutputLock:
def __enter__(self) -> "_OutputLock":
if not self.path:
return self
+ taken: Optional[Dict[str, Any]] = None
for attempt in (0, 1):
try:
self.fd = os.open(self.path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
os.write(self.fd, str(os.getpid()).encode())
+ self._set_deadline()
+ if taken is not None:
+ self._record_takeover(taken)
return self
except FileExistsError:
- if attempt == 0 and self._stale():
+ holder = self._holder()
+ if attempt == 0 and holder["stale"]:
try:
os.remove(self.path)
- except OSError:
- pass
- continue
- die(f"another run is writing {os.path.basename(self.path)[1:-len('.ffskill-lock')]!r} right now "
- f"(lock {self.path}); wait for it or choose a different --output/-o path")
- except OSError:
- return self # unlockable location (read-only dir surfaces elsewhere): proceed without a lock
+ except OSError as exc:
+ # Windows will not delete a file another process holds open, and a live
+ # holder keeps its lock open until it finishes: the takeover fails and
+ # the run is refused below, with the reason, instead of passing silently
+ holder["takeover_error"] = str(exc)
+ holder["stale"] = False
+ else:
+ taken = holder
+ continue
+ name = os.path.basename(self.path)[1:-len(".ffskill-lock")]
+ refuse("input", "output_locked",
+ f"another run is writing {name!r} right now (lock {self.path}"
+ + (f", pid {holder['pid']}" if holder.get("pid") else "") + "); wait for it or choose a different --output/-o path",
+ hint=("wait for that run, or choose another -o path; if pid "
+ f"{holder.get('pid')} is not an ffmpeg-skill run, delete {self.path}"),
+ lock=self.path, output=os.path.abspath(self.output), **{k: v for k, v in holder.items() if k != "stale"})
+ except OSError as exc:
+ # an unlockable location (a read-only directory surfaces elsewhere): the run goes
+ # on without a lock, and says so -- two runs could now both write this output
+ details = {"lock": self.path, "output": os.path.abspath(self.output), "error": str(exc)}
+ warn(f"could not create the output lock {self.path} ({exc}); running without one, so a second run "
+ "writing the same output at the same time would not be refused", "output_unlocked", **details)
+ audit("output_unlocked", details, default_path=self._audit_default())
+ return self
return self
- def _stale(self) -> bool:
+ def _audit_default(self) -> Optional[str]:
+ if not self.path:
+ return None
+ d, base = os.path.split(os.path.abspath(self.output))
+ return os.path.join(d, f".{base}.ffskill-audit.jsonl")
+
+ def _set_deadline(self) -> None:
+ import time
+ limit = child_limit()
+ deadline = time.time() + (limit if limit else LOCK_UNLIMITED)
try:
- pid = int(open(self.path).read().strip() or "0")
- if pid > 0 and _pid_dead(pid):
- return True
- import time
- return time.time() - os.path.getmtime(self.path) > 3600
+ os.utime(self.path, (deadline - LOCK_HOUR, deadline - LOCK_HOUR))
+ except (OSError, OverflowError, ValueError) as exc:
+ warn(f"could not date the output lock {self.path} ({exc}); another run may take it over an hour from now",
+ "lock_deadline_unset", lock=self.path, error=str(exc))
+
+ def _holder(self) -> Dict[str, Any]:
+ """Who holds the lock and whether it is stale: {pid, pid_alive, deadline (UTC, the
+ mtime plus an hour), why: pid_dead | past_deadline | unreadable | held, stale}."""
+ import datetime
+ import time
+ out: Dict[str, Any] = {"pid": None, "pid_alive": None, "deadline": None, "why": "unreadable", "stale": True}
+ try:
+ with open(self.path, encoding="utf-8", errors="replace") as f:
+ text = f.read().strip()
+ deadline = os.path.getmtime(self.path) + LOCK_HOUR
+ pid = int(text or "0")
except (OSError, ValueError):
- return True
+ return out
+ out["deadline"] = datetime.datetime.fromtimestamp(deadline, datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+ out["pid"] = pid if pid > 0 else None
+ if pid > 0 and _pid_dead(pid):
+ out.update(pid_alive=False, why="pid_dead")
+ return out
+ out["pid_alive"] = pid > 0 or None
+ if time.time() > deadline:
+ out["why"] = "past_deadline"
+ return out
+ out.update(why="held", stale=False)
+ return out
+
+ def _record_takeover(self, holder: Dict[str, Any]) -> None:
+ details = {"lock": self.path, "output": os.path.abspath(self.output),
+ **{k: v for k, v in holder.items() if k != "stale"}}
+ why = {"pid_dead": f"its pid {holder.get('pid')} no longer runs",
+ "past_deadline": f"its holder (pid {holder.get('pid')}, still running) is past its own time limit",
+ }.get(holder.get("why") or "", "it could not be read")
+ details["audit_log"] = audit("lock_takeover", details, default_path=self._audit_default())
+ record_fallback(f"took over the stale output lock {self.path}: {why}", "stale_lock_taken_over", **details)
def __exit__(self, *exc: Any) -> None:
if self.fd is not None:
@@ -698,6 +796,7 @@ def refuse_existing_outputs(paths: Sequence[str]) -> None:
.srt/.ass sidecars) calls this up front so a hand-edited sidecar is refused like the video
is, and a dry run predicts the same refusal."""
existing: List[str] = []
+ facts: List[Dict[str, Any]] = []
for output in paths:
if not output:
continue
@@ -708,21 +807,25 @@ def refuse_existing_outputs(paths: Sequence[str]) -> None:
continue
if not exists or real in STATE.written:
continue
+ size = mtime_ns = None
try:
st = os.stat(output)
STATE.preexisting[real] = (st.st_size, st.st_mtime_ns)
+ size, mtime_ns = st.st_size, st.st_mtime_ns
except OSError:
pass
if output not in existing:
existing.append(output)
+ facts.append({"path": output, "resolved": real, "size": size, "mtime_ns": mtime_ns})
if not existing or STATE.overwrite:
return
+ hint = "pass --overwrite, or choose another -o path"
if len(existing) == 1:
- die(f"refusing to overwrite existing output {existing[0]!r}: pass --overwrite to replace it, or choose another -o path",
- kind="input", hint="pass --overwrite, or choose another -o path")
- die("refusing to overwrite existing outputs " + ", ".join(repr(p) for p in existing)
- + ": pass --overwrite to replace them, or choose another -o path",
- kind="input", hint="pass --overwrite, or choose another -o path")
+ refuse("input", "output_exists",
+ f"refusing to overwrite existing output {existing[0]!r}: pass --overwrite to replace it, or choose another -o path",
+ hint=hint, outputs=facts)
+ refuse("input", "output_exists", "refusing to overwrite existing outputs " + ", ".join(repr(p) for p in existing)
+ + ": pass --overwrite to replace them, or choose another -o path", hint=hint, outputs=facts)
def _remember_output(cmd: Sequence[str]) -> None:
@@ -1415,4 +1518,4 @@ def read_text_or_die(path: str, flag: str) -> str:
# emit needs runner's Context/STATE/ERROR_CODE, so the two form a cycle that has to be cut
# somewhere: by the time this line runs every name emit reads from runner above is defined, and
# every name runner reads from emit is only ever read inside a function body, never at import.
-from _common.emit import _plan_at_exit, die, hw_report, info # noqa: E402
+from _common.emit import _plan_at_exit, audit, die, hw_report, info, record_fallback, refuse, warn # noqa: E402
diff --git a/scripts/_contract.py b/scripts/_contract.py
index 732b059..19c7304 100644
--- a/scripts/_contract.py
+++ b/scripts/_contract.py
@@ -397,6 +397,37 @@ def input_schema(parser: argparse.ArgumentParser) -> Dict[str, Any]:
return schema
+# The collector's records (docs/design-decisions.md, "Nothing happens silently"): on success and
+# failure documents alike, present only when one list is not empty.
+DIAGNOSTICS_SCHEMA = {
+ "type": "object",
+ "description": "what the run decided on its own, recorded as it happened (each also a stderr line): warnings (something went "
+ "other than asked), fallbacks (another route was taken and the run went on), skipped (a step that would "
+ "normally run did not). Each entry is {code, message, ...facts}; an entry a render.py stage reported carries "
+ "`stage`. Absent when all three are empty",
+ "properties": {k: {"type": "array", "items": {"type": "object", "properties": {"code": {"type": "string"}, "message": {"type": "string"}}}}
+ for k in ("warnings", "fallbacks", "skipped")},
+}
+
+# error.reason: a stable code for why a refusal happened, beside `kind` (which says what class of
+# failure it is). Every refuse() call in scripts/ names one of these (a test reads the sources).
+ERROR_REASONS = {
+ "output_exists": "an output already exists and --overwrite was not given; details.outputs: [{path, resolved, size, mtime_ns}]",
+ "output_is_input": "the output resolves to the same file as an input; details: {output, input, resolved}",
+ "output_is_directory": "the output path is a directory; details: {output}",
+ "output_dir_missing": "the output's directory does not exist; details: {output, directory}",
+ "output_dir_not_writable": "the output's directory is not writable; details: {output, directory}",
+ "output_locked": "another run holds the output's lock; details: {lock, output, pid, pid_alive, deadline, why}",
+ "plan_inputs_changed": "render.py PLAN: a fingerprinted input is missing or changed; details: {plan, plan_sha256, checked_at, "
+ "inputs: [{path, status: unchanged|changed|missing, expected: {size, sha256_head_tail}, actual: {size, "
+ "sha256_head_tail} | \"missing\"}], unbound_inputs, audit_log}",
+ "plan_version_unsupported": "render.py PLAN: plan_version is not one this version executes; details: {plan, plan_sha256, checked_at, plan_version, supported, audit_log}",
+ "plan_unknown_tool": "render.py PLAN: the plan names no tool render.py can run; details: {plan, plan_sha256, checked_at, tool, audit_log}",
+ "plan_cwd_missing": "render.py PLAN: the directory the plan was made in is gone; details: {plan, plan_sha256, checked_at, cwd, audit_log}",
+ "plan_argv_invalid": "render.py PLAN: argv is not a list; details: {plan, plan_sha256, checked_at, argv_type, audit_log}",
+}
+
+
def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str, Any]:
"""What the tool prints on stdout with --json (keys observed in the implementation). `hw`: the
tool takes --hw, so its `notes` can also carry a VideoToolbox encode's quality line."""
@@ -411,7 +442,8 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
"hw": {"type": "object", "description": "present when VideoToolbox was asked for (--hw, $FFMPEG_SKILL_HW, a render.py stage, a batch.py item), on export.py when $FFMPEG_SKILL_HW=1 is set but its delivery preset does not take it, and when this ffmpeg build has no libx264/libx265 and an encode ran on VideoToolbox instead: {requested: whether this run's encodes were asked onto VideoToolbox (false for that export, and for the build without libx264/libx265), source: flag|env|auto (auto: nobody asked, the build has no libx264/libx265 and this Apple Silicon machine ran VideoToolbox in its place), used: true|false whether the encoder that ran last is VideoToolbox, null under --dry-run (nothing ran) and when this process ran no encode itself (batch.py: its stages are child processes; its top-level hw gathers the items' requested/source/fallback/notes and each results row carries its item's encoder and hw), fallback: a job VideoToolbox refused (encoder open or compression session failure; nothing else is retried) was re-encoded on the CPU, notes: why an encode stayed on or fell back to the CPU, and, when $FFMPEG_SKILL_HW chose the GPU and it ran, what that costs and that --no-hw opts out}. A VideoToolbox encode also adds a top-level notes line: its quality is not CRF-equivalent. A kind: ffmpeg failure under --hw / $FFMPEG_SKILL_HW carries encoder and hw too"},
"plan": {"type": "string", "description": "with --plan FILE: the plan document written (the run itself is a dry run)"},
"verified": {"type": "boolean", "description": "true only when the artifact was written, probed, and every self-check the tool ran (verification) met its target; false under --dry-run"},
- "verification": {"type": "array", "items": {"type": "object", "properties": {"step": {"type": "string"}, "ok": {"type": "boolean"}}}, "description": "what the tool itself verified: probe, plus loudness (loudness.py, export platform presets) or check (render)"}}
+ "verification": {"type": "array", "items": {"type": "object", "properties": {"step": {"type": "string"}, "ok": {"type": "boolean"}}}, "description": "what the tool itself verified: probe, plus loudness (loudness.py, export platform presets) or check (render)"},
+ "diagnostics": DIAGNOSTICS_SCHEMA}
extra: Dict[str, Any] = {}
if name == "check":
extra = {"platform": {"type": "string"}, "ok": {"type": "boolean"}, "failed": {"type": "integer"}, "warnings": {"type": "integer"},
@@ -451,6 +483,8 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
elif name == "render":
extra = {"stages": {"type": "array", "items": {"type": "string"}}, "check": {"type": ["object", "null"]},
"plan": {"type": "string", "description": "when the argument was a plan.json (written by <tool> --plan): its path"},
+ "plan_sha256": {"type": ["string", "null"], "description": "plan execution: sha256 of the plan file as executed"},
+ "unbound_inputs": {"type": "array", "items": {"type": "string"}, "description": "plan execution: inputs that did not exist when the plan was made, so carry no fingerprint and were not checked (also a diagnostics.warnings entry, code plan_unbound_inputs)"},
"tool": {"type": "string", "description": "plan execution: the tool the plan ran"},
"tool_result": {"type": "object", "description": "plan execution: the tool's own --json document"},
"timeline": {"type": "object", "description": "--export-timeline: format, rate, duration, frames, clips, transition, "
@@ -1692,7 +1726,21 @@ def build(detect: bool = True) -> Dict[str, Any]:
},
"json_output": {
"success": {"status": "completed", "exit_code": 0, "stdout": "one JSON document (output_schema)"},
- "failure": {"status": "failed", "exit_code": "non-zero (127 when ffmpeg/ffprobe is missing)", "stdout": "{\"status\": \"failed\", \"exit_code\": N, \"error\": {\"kind\": ..., \"message\": ...}, \"commands\": [...]} when --json was given", "stderr": "human-readable message"},
+ "failure": {"status": "failed", "exit_code": "non-zero (127 when ffmpeg/ffprobe is missing)", "stdout": "{\"status\": \"failed\", \"exit_code\": N, \"error\": {\"kind\": ..., \"message\": ..., \"code\": ..., \"retryable\": false}, \"commands\": [...]} when --json was given; a refusal adds error.reason (error_reasons) and error.details (the facts the message is written from), and a run that recorded warnings, fallbacks or skipped steps before failing carries diagnostics", "stderr": "human-readable message",
+ "error": {"kind": "error_kinds", "message": "the reason in prose", "code": "a static relabelling of kind", "retryable": "always false",
+ "hint": "optional: the flag change that would make a retry meaningful",
+ "reason": "optional, on refusals: a stable code from error_reasons",
+ "details": "optional, with reason: an object of the facts behind the refusal (paths, expected and actual values)"},
+ "diagnostics": DIAGNOSTICS_SCHEMA["description"]},
+ "error_reasons": ERROR_REASONS,
+ "audit_log": {"format": "JSON lines, appended: {ts, tool, event, pid, plan_sha256 (plan events), details}",
+ "events": {"plan_rejected": "render.py refused a plan (details: the refusal's error.details plus reason)",
+ "lock_takeover": "a stale output lock was taken over (details: lock, output, pid, pid_alive, deadline, why)",
+ "output_unlocked": "no output lock could be created and the run went on without one",
+ "output_is_input_refused": "the output-is-input refusal (only when an audit log was named)"},
+ "location": "render.py --audit-log FILE when given; otherwise a refused plan's <plan>.audit.jsonl, and a lock event's "
+ "hidden .<output>.ffskill-audit.jsonl beside the output. Never written under --dry-run; a write that "
+ "fails is a diagnostics.warnings entry (audit_log_unwritable), never a failure"},
"error_kinds": {"input": "missing or unsuitable input, bad arguments", "ffmpeg": "ffmpeg/ffprobe returned an error (message carries the last stderr lines)", "output": "ffmpeg exited 0 but the artifact is missing, empty or unreadable (an empty file is removed)", "missing_tool": "ffmpeg or ffprobe not on PATH", "timeout": "one ffmpeg/ffprobe run exceeded --timeout (default 1800 s, FFMPEG_SKILL_TIMEOUT) and was killed; partial output removed; exit 124", "verification": "the tool ran but its result failed the requested check: check.py platform rows (checks attached), render.py's check stage (output written, check attached), batch.py items (results attached), verify.py steps (files attached), color.py --retag (output written, but ffprobe does not read the requested colour tags on it); exit 1", "interrupted": "the tool received SIGINT or SIGTERM: the running ffmpeg (or sibling script) was stopped and its partial output removed; exit 130 or 143"},
"success_criterion": "exit 0 AND the output exists AND is non-empty AND ffprobe reads a stream from it; only then is status completed printed and the output probe attached",
},
diff --git a/scripts/render.py b/scripts/render.py
index ea9733a..ae2b1b4 100755
--- a/scripts/render.py
+++ b/scripts/render.py
@@ -72,7 +72,7 @@ from export import PRESETS, PLATFORM_OF
from _platforms import PLATFORMS, caption_defaults, resolve as resolve_platform
from _common import STATE, add_common, aspect_ratio, brand_caption_style, load_brand, apply_common, child_args, die, emit, info, probe, require_tool, run, run_tool, place_output, refuse_output_is_input, _check_existing_output, _check_output_path, fingerprint, PLAN_VERSION, ffmpeg_version, ffmpeg_exe
from _common.runner import HW_ENV, HW_FORCED_ENV, add_hw_orchestrator_args
-from _common.emit import absorb_stage_hw
+from _common.emit import absorb_stage, audit, refuse, warn
import subprocess
from _contract import CONTRACT_VERSION
from batch import file_key
@@ -364,7 +364,7 @@ def render_pack(names: List[str], args) -> int:
if line.startswith("$ ") or line.startswith("[dry-run]"):
STATE.commands.append(line[2:] if line.startswith("$ ") else line)
STATE.commands.extend(str(c) for c in (doc.get("commands") or []))
- absorb_stage_hw(name, doc)
+ absorb_stage(name, doc)
out = doc.get("output") or dest_out
chk = doc.get("check") or {}
ok = proc.returncode == 0 and doc.get("status") == "completed"
@@ -580,26 +580,26 @@ def sh(script: str, *argv: Any, extra: List[str] = None, stage: str = None) -> s
# needs (a timeout inside audio.py is a timeout, not an "input" error of render.py).
err = doc.get("error") or {}
extra_fields = {"hint": err["hint"]} if err.get("hint") else {}
- extra_fields.update(_stage_failure_hw(Path(script).stem, doc))
+ error_extra = absorb_stage(Path(script).stem, doc) # the stage's reason/details and records ride along
+ extra_fields.update(_stage_failure_hw(doc))
die(f"{script} failed: {err.get('message') or (proc.stderr.strip().splitlines() or ['?'])[-1][:300]}",
- code=int(doc.get("exit_code") or 1), kind=err.get("kind") or "input", stage=script, **extra_fields)
+ code=int(doc.get("exit_code") or 1), kind=err.get("kind") or "input", stage=script, error_extra=error_extra, **extra_fields)
_LAST_DOC.clear()
_LAST_DOC.update(doc if isinstance(doc, dict) else {})
- absorb_stage_hw(Path(script).stem, doc) # a stage's GPU facts (ran on VideoToolbox, fell back and why) reach this result
+ absorb_stage(Path(script).stem, doc) # a stage's GPU facts (ran on VideoToolbox, fell back and why) and its diagnostics reach this result
out_path = str(doc.get("output") or "")
if key:
cache_store(stage, key, out_path or (dest or ""), time.time() - started)
return out_path
-def _stage_failure_hw(stage: str, doc: Any) -> Dict[str, Any]:
- """A failed stage's GPU facts, for render's own failure document: the stage's `hw` joins the
- earlier stages' (a fallback before the failure is not lost), and its `encoder` -- the command
- that failed -- is forwarded. die() then adds `hw` for a kind: ffmpeg failure, as a tool's own
+def _stage_failure_hw(doc: Any) -> Dict[str, Any]:
+ """A failed stage's `encoder` -- the command that failed -- for render's own failure document
+ (absorb_stage() has already joined the stage's `hw` to the earlier stages'; a fallback before
+ the failure is not lost). die() then adds `hw` for a kind: ffmpeg failure, as a tool's own
run() failure carries it."""
if not isinstance(doc, dict):
return {}
- absorb_stage_hw(stage, doc)
return {"encoder": doc["encoder"]} if doc.get("encoder") else {}
@@ -731,40 +731,94 @@ def cache_store(stage: str, key: str, produced: str, seconds: float) -> None:
info(f"cache: could not store the {stage} artifact ({exc}); the run is unaffected")
+def _utc_now() -> str:
+ import datetime
+ return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+
+
+def _plan_sha256(path: str) -> Optional[str]:
+ try:
+ return hashlib.sha256(Path(path).read_bytes()).hexdigest()
+ except OSError:
+ return None
+
+
+def _reject_plan(path: str, plan_sha256: Optional[str], reason: str, msg: str, *, hint: Optional[str] = None,
+ **details: Any) -> None:
+ """Refuse a plan: one audit record (the plan's `<plan>.audit.jsonl`, or --audit-log), then
+ the failure document with `error.reason` and `error.details` -- the plan, its sha256, when it
+ was checked, the facts below, and where the audit record went."""
+ details = dict({"plan": path, "plan_sha256": plan_sha256, "checked_at": _utc_now()}, **details)
+ details["audit_log"] = audit("plan_rejected", dict(details, reason=reason), default_path=path + ".audit.jsonl",
+ plan_sha256=plan_sha256)
+ refuse("input", reason, msg, hint=hint, extra={"plan": path}, **details)
+
+
+def plan_input_status(inputs: Sequence[Any]) -> List[Dict[str, Any]]:
+ """Each fingerprinted plan input against the file as it is now: {path, status: unchanged |
+ changed | missing, expected: {size, sha256_head_tail}, actual: {size, sha256_head_tail} or
+ "missing"}."""
+ rows: List[Dict[str, Any]] = []
+ for inp in inputs:
+ inp = inp if isinstance(inp, dict) else {"path": inp}
+ p = inp.get("path")
+ expected = {"size": inp.get("size"), "sha256_head_tail": inp.get("sha256_head_tail")}
+ if not p or not os.path.isfile(str(p)):
+ rows.append({"path": p, "status": "missing", "expected": expected, "actual": "missing"})
+ continue
+ try:
+ now = fingerprint(str(p))
+ except OSError:
+ rows.append({"path": p, "status": "missing", "expected": expected, "actual": "missing"})
+ continue
+ actual = {"size": now["size"], "sha256_head_tail": now["sha256_head_tail"]}
+ rows.append({"path": p, "status": "unchanged" if actual == expected else "changed", "expected": expected, "actual": actual})
+ return rows
+
+
def execute_plan(plan: Dict[str, Any], path: str) -> int:
"""Run a plan written by `<tool> --plan FILE`: refuse if any fingerprinted input changed since
the plan was made (the plan's commands would then describe a different edit), run the tool
- with the planned argv, then the plan's verify steps. --dry-run prints the planned commands."""
+ with the planned argv, then the plan's verify steps. --dry-run prints the planned commands.
+
+ Every refusal is `kind: input` with `error.reason` and `error.details` and appends one audit
+ record (docs/design-decisions.md, "Nothing happens silently")."""
+ digest = _plan_sha256(path)
if plan.get("plan_version") != PLAN_VERSION:
- die(f"{path}: plan_version {plan.get('plan_version')!r} is not {PLAN_VERSION}")
+ _reject_plan(path, digest, "plan_version_unsupported",
+ f"{path}: plan_version {plan.get('plan_version')!r} is not {PLAN_VERSION}",
+ plan_version=plan.get("plan_version"), supported=[PLAN_VERSION])
tool = str(plan.get("tool") or "")
script = HERE / f"{tool}.py"
if not re.fullmatch(r"[a-z][a-z0-9_]*", tool) or not script.exists() or tool == "render":
- die(f"{path}: unknown tool {tool!r}")
- changed = []
- for inp in plan.get("inputs") or []:
- p = inp.get("path")
- if not p or not os.path.isfile(p):
- changed.append(f"{p}: missing")
- continue
- now = fingerprint(p)
- if now["size"] != inp.get("size") or now["sha256_head_tail"] != inp.get("sha256_head_tail"):
- changed.append(f"{p}: content changed since the plan was made")
- if changed:
- die("plan inputs differ from what was planned; re-run the tool with --plan to make a new plan:\n " + "\n ".join(changed),
- hint="plans are bound to the exact input files they were made from")
+ _reject_plan(path, digest, "plan_unknown_tool", f"{path}: unknown tool {tool!r}", tool=tool)
+ unbound = [str(u) for u in (plan.get("unbound_inputs") or [])]
+ rows = plan_input_status(plan.get("inputs") or [])
+ bad = [r for r in rows if r["status"] != "unchanged"]
+ if bad:
+ lines = [f"{r['path']}: missing" if r["status"] == "missing" else f"{r['path']}: content changed since the plan was made"
+ for r in bad]
+ _reject_plan(path, digest, "plan_inputs_changed",
+ "plan inputs differ from what was planned; re-run the tool with --plan to make a new plan:\n " + "\n ".join(lines),
+ hint="plans are bound to the exact input files they were made from",
+ inputs=rows, unbound_inputs=unbound)
+ if unbound:
+ warn(f"{len(unbound)} plan input(s) did not exist when the plan was made, so nothing checks that they are the files "
+ "it was made for: " + ", ".join(unbound), "plan_unbound_inputs", paths=unbound)
if plan.get("cwd"):
if not os.path.isdir(plan["cwd"]):
- die(f"plan cwd {plan['cwd']} no longer exists; relative paths in the plan would resolve elsewhere -- re-plan")
+ _reject_plan(path, digest, "plan_cwd_missing",
+ f"plan cwd {plan['cwd']} no longer exists; relative paths in the plan would resolve elsewhere -- re-plan",
+ cwd=plan["cwd"])
os.chdir(plan["cwd"])
if not isinstance(plan.get("argv"), list):
- die(f"{path}: argv must be a list")
+ _reject_plan(path, digest, "plan_argv_invalid", f"{path}: argv must be a list", argv_type=type(plan.get("argv")).__name__)
argv = [str(a) for a in plan["argv"]]
if STATE.dry_run:
for c in plan.get("commands") or []:
STATE.commands.append(c)
info("[dry-run] " + c)
- emit(plan.get("output"), plan=path, tool=tool, stages=[tool], check=None)
+ emit(plan.get("output"), plan=path, plan_sha256=digest, tool=tool, stages=[tool], check=None, unbound_inputs=unbound)
return 0
info(f"executing plan {path}: {tool} " + " ".join(argv))
proc = run_tool([str(script)] + argv + child_args() + ["--json"])
@@ -779,9 +833,10 @@ def execute_plan(plan: Dict[str, Any], path: str) -> int:
doc = {}
if proc.returncode != 0 or doc.get("status") != "completed":
err = doc.get("error") or {}
+ error_extra = absorb_stage(tool, doc) # the tool's reason/details and records ride along
die(f"{tool} failed while executing the plan: {err.get('message') or proc.stderr.strip()[-300:]}",
- kind=err.get("kind", "ffmpeg"), plan=path, tool=tool, **_stage_failure_hw(tool, doc))
- absorb_stage_hw(tool, doc) # the tool's GPU facts (fell back and why, ran on VideoToolbox) reach this result
+ kind=err.get("kind", "ffmpeg"), plan=path, tool=tool, error_extra=error_extra, **_stage_failure_hw(doc))
+ absorb_stage(tool, doc) # the tool's GPU facts (fell back and why, ran on VideoToolbox) and diagnostics reach this result
output = doc.get("output") or plan.get("output")
check_result = None
exit_code = 0
@@ -805,7 +860,7 @@ def execute_plan(plan: Dict[str, Any], path: str) -> int:
info(f"plan done: {output}, but the {check_result.get('platform') or 'content'} check failed" + (f": {', '.join(failed_rows)}" if failed_rows else ""))
else:
info(f"plan done: {output}")
- emit(output, plan=path, tool=tool, stages=[tool] + (["check"] if check_result else []), check=check_result, tool_result=doc,
+ emit(output, plan=path, plan_sha256=digest, unbound_inputs=unbound, tool=tool, stages=[tool] + (["check"] if check_result else []), check=check_result, tool_result=doc,
verification=([{"step": "check", "ok": not exit_code, "platform": check_result.get("platform")}] if check_result else [])
+ [s for s in (doc.get("verification") or []) if s.get("step") != "probe"])
return 0
@@ -902,6 +957,10 @@ def main() -> int:
help="write the project's cut as an editor timeline instead of rendering it: .fcpxml (Final Cut, "
"Resolve), .edl (CMX 3600: Premiere, Resolve, Avid) or .otio (OpenTimelineIO); clips, speed, "
"transitions, the music bed and chapter markers carry over, everything else is listed as not exported")
+ ap.add_argument("--audit-log", metavar="FILE",
+ help="append the audit records of this run (a refused plan, a lock taken over, ...) to FILE as JSON lines; "
+ "default for a refused plan: <plan>.audit.jsonl next to it. A record that cannot be written is a "
+ "warning, never a failure")
ap.add_argument("--stop-after", choices=["clips", "join", "silence", "fit", "captions", "graphics", "overlays", "audio", "loudness", "export"], help="stop after this stage (for iterating)")
tpl = ap.add_argument_group("delivery templates (one command per destination)")
tpl.add_argument("--template", metavar="NAME", help="render INPUT with a shipped template: " + ", ".join(template_names())
@@ -923,6 +982,7 @@ def main() -> int:
add_hw_orchestrator_args(ap)
args = ap.parse_args()
apply_common(args)
+ STATE.audit_log = os.path.abspath(args.audit_log) if args.audit_log else None
if args.list_templates:
list_templates()
diff --git a/tests/fixtures/mcp_tools.json b/tests/fixtures/mcp_tools.json
index 117374b..548037c 100644
--- a/tests/fixtures/mcp_tools.json
+++ b/tests/fixtures/mcp_tools.json
@@ -901,6 +901,7 @@
"name": "render",
"properties": [
"argv",
+ "audit_log",
"brand",
"cache",
"chapters",
diff --git a/tests/test_contract.py b/tests/test_contract.py
index 3ded738..cc76e14 100644
--- a/tests/test_contract.py
+++ b/tests/test_contract.py
@@ -2984,6 +2984,129 @@ class ContractTests(unittest.TestCase):
self.assertEqual(json.loads(proc.stdout)["error"]["kind"], "input")
self.assertIn("changed", json.loads(proc.stdout)["error"]["message"])
+ def test_a_refused_plan_says_what_changed_and_leaves_an_audit_record(self):
+ """docs/design-decisions.md, "Nothing happens silently": a refused plan named the changed
+ input only in prose. The refusal now carries error.reason plan_inputs_changed and
+ error.details (each input's expected and actual size + head/tail hash, or "missing"; the
+ plan's sha256; when it was checked) and appends one JSON line to <plan>.audit.jsonl, or to
+ --audit-log FILE. A log that cannot be written is a warning, never the failure."""
+ import hashlib
+ src = self.out("aud_src.mp4"); shutil.copy(self.src, src)
+ plan = self.out("aud_plan.json"); out = self.out("aud_out.mp4")
+ tool("cut", src, "--start", "0", "--end", "1", "-o", out, "--plan", plan)
+ p = json.loads(plan.read_text())
+ self.assertEqual(p["unbound_inputs"], [])
+ ffmpeg("-i", self.src, "-t", "0.5", "-c", "copy", "-y", src)
+ log = Path(str(plan) + ".audit.jsonl")
+ # a dry run predicts the refusal and writes nothing, the audit log included
+ doc = json.loads(tool("render", plan, "--dry-run", "--json", check=False).stdout)
+ self.assertEqual(doc["error"]["reason"], "plan_inputs_changed")
+ self.assertEqual(doc["error"]["details"]["audit_log"], {"path": str(log), "written": False, "reason": "dry_run"})
+ self.assertFalse(log.exists())
+ proc = tool("render", plan, "--json", check=False)
+ err = json.loads(proc.stdout)["error"]
+ self.assertEqual((err["kind"], err["reason"]), ("input", "plan_inputs_changed"))
+ self.assertIn("changed", err["message"])
+ d = err["details"]
+ self.assertEqual(d["plan_sha256"], hashlib.sha256(plan.read_bytes()).hexdigest())
+ self.assertRegex(d["checked_at"], r"^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ$")
+ row = d["inputs"][0]
+ self.assertEqual((row["path"], row["status"]), (str(src), "changed"))
+ self.assertEqual(row["expected"], {"size": p["inputs"][0]["size"], "sha256_head_tail": p["inputs"][0]["sha256_head_tail"]})
+ self.assertEqual(row["actual"]["size"], src.stat().st_size)
+ self.assertNotEqual(row["actual"]["sha256_head_tail"], row["expected"]["sha256_head_tail"])
+ self.assertEqual(d["audit_log"], {"path": str(log), "written": True})
+ rec = json.loads(log.read_text(encoding="utf-8").splitlines()[-1])
+ self.assertEqual((rec["event"], rec["tool"], rec["plan_sha256"]), ("plan_rejected", "render", d["plan_sha256"]))
+ self.assertEqual(rec["details"]["inputs"], d["inputs"])
+ self.assertTrue(rec["ts"].endswith("Z"))
+ # a missing input: actual "missing"; --audit-log names another file
+ src.unlink()
+ other = self.out("aud_other.jsonl")
+ err = json.loads(tool("render", plan, "--json", "--audit-log", other, check=False).stdout)["error"]
+ self.assertEqual((err["details"]["inputs"][0]["status"], err["details"]["inputs"][0]["actual"]), ("missing", "missing"))
+ self.assertEqual(json.loads(other.read_text(encoding="utf-8"))["event"], "plan_rejected")
+ self.assertEqual(len(log.read_text(encoding="utf-8").splitlines()), 1, "--audit-log replaced the default")
+ # an audit log that cannot be written: recorded, and the refusal is still the refusal
+ doc = json.loads(tool("render", plan, "--json", "--audit-log", self.work / "no_such_dir" / "a.jsonl", check=False).stdout)
+ self.assertEqual(doc["error"]["reason"], "plan_inputs_changed")
+ self.assertFalse(doc["error"]["details"]["audit_log"]["written"])
+ self.assertEqual([w["code"] for w in doc["diagnostics"]["warnings"]], ["audit_log_unwritable"])
+ # the neighbours are structured too
+ bad = self.out("aud_v2.json"); bad.write_text(json.dumps(dict(p, plan_version=99)))
+ err = json.loads(tool("render", bad, "--json", check=False).stdout)["error"]
+ self.assertEqual((err["reason"], err["details"]["plan_version"]), ("plan_version_unsupported", 99))
+ bad.write_text(json.dumps(dict(p, tool="nope")))
+ self.assertEqual(json.loads(tool("render", bad, "--json", check=False).stdout)["error"]["reason"], "plan_unknown_tool")
+
+ def test_a_plan_names_the_inputs_it_could_not_bind_and_render_reports_them(self):
+ """_plan_inputs() skips an input that does not exist yet (an earlier step's output), so
+ nothing fingerprinted it and nothing said so. The plan now lists it under unbound_inputs
+ and render.py reports it on execution: a top-level unbound_inputs and a
+ diagnostics.warnings entry."""
+ a = self.out("ub_a.mp4"); shutil.copy(self.src, a)
+ pending = self.out("ub_pending.mp4")
+ if pending.exists():
+ pending.unlink()
+ plan = self.out("ub_plan.json"); out = self.out("ub_out.mp4")
+ tool("join", a, pending, "--transition", "none", "-o", out, "--plan", plan)
+ p = json.loads(plan.read_text())
+ self.assertEqual(p["unbound_inputs"], [str(pending)])
+ self.assertEqual([i["path"] for i in p["inputs"]], [str(a)])
+ shutil.copy(self.src, pending)
+ doc = json.loads(tool("render", plan, "--json").stdout)
+ self.assertEqual(doc["status"], "completed")
+ self.assertEqual(doc["unbound_inputs"], [str(pending)])
+ warning = doc["diagnostics"]["warnings"][0]
+ self.assertEqual((warning["code"], warning["paths"]), ("plan_unbound_inputs", [str(pending)]))
+
+ def test_refusals_at_the_runner_choke_points_carry_a_reason_and_details(self):
+ """The runner's guards refused in prose only. Each now names error.reason and the facts in
+ error.details; render.py executing a plan forwards its tool's reason and details."""
+ out = self.out("rf_exists.mp4"); out.write_bytes(b"theirs")
+ err = json.loads(tool("cut", self.src, "--start", "0", "--end", "1", "-o", out, "--json", check=False).stdout)["error"]
+ self.assertEqual(err["reason"], "output_exists")
+ self.assertEqual(err["details"]["outputs"][0]["path"], str(out))
+ self.assertEqual(err["details"]["outputs"][0]["size"], 6)
+ clone = self.out("rf_same.mp4"); shutil.copy(self.src, clone)
+ err = json.loads(tool("crop", clone, "--x", "0", "--y", "0", "--width", "32", "--height", "32", "--json",
+ "-o", self.work / ("." + os.sep + clone.name), check=False).stdout)["error"]
+ self.assertEqual((err["reason"], err["details"]["resolved"]), ("output_is_input", os.path.realpath(clone)))
+ err = json.loads(tool("cut", self.src, "--start", "0", "--end", "1", "-o", self.work / "nodir" / "x.mp4", "--json", check=False).stdout)["error"]
+ self.assertEqual((err["reason"], err["details"]["directory"]), ("output_dir_missing", str(self.work / "nodir")))
+ err = json.loads(tool("cut", self.src, "--start", "0", "--end", "1", "-o", self.work, "--json", check=False).stdout)["error"]
+ self.assertEqual(err["reason"], "output_is_directory")
+ # a plan whose output now exists: render re-raises the tool's own structured refusal
+ plan = self.out("rf_plan.json"); pout = self.out("rf_plan_out.mp4")
+ tool("cut", self.src, "--start", "0", "--end", "1", "-o", pout, "--plan", plan)
+ pout.write_bytes(b"theirs")
+ err = json.loads(tool("render", plan, "--json", check=False).stdout)["error"]
+ self.assertEqual((err["kind"], err["reason"]), ("input", "output_exists"))
+ self.assertEqual(err["details"]["outputs"][0]["path"], str(pout))
+
+ def test_a_stale_lock_taken_over_is_recorded_in_the_document_and_the_audit_log(self):
+ """A dead writer's lock was taken over without a word. The run now records it
+ (diagnostics.fallbacks, stderr) and appends a lock_takeover line to the hidden
+ .<output>.ffskill-audit.jsonl; render.py carries a stage's record with its stage name."""
+ out = self.out("lk_out.mp4")
+ lock = self.out(".lk_out.mp4.ffskill-lock"); lock.write_text("999999")
+ proc = tool("cut", self.src, "--start", "0", "--end", "1", "-o", out, "--json")
+ doc = json.loads(proc.stdout)
+ fb = doc["diagnostics"]["fallbacks"][0]
+ self.assertEqual((fb["code"], fb["why"], fb["pid"], fb["pid_alive"]), ("stale_lock_taken_over", "pid_dead", 999999, False))
+ self.assertIn("took over the stale output lock", proc.stderr)
+ rec = json.loads(self.out(".lk_out.mp4.ffskill-audit.jsonl").read_text(encoding="utf-8").splitlines()[-1])
+ self.assertEqual((rec["event"], rec["tool"], rec["details"]["pid"]), ("lock_takeover", "cut", 999999))
+ # through render executing a plan: the stage's record comes back tagged
+ plan = self.out("lk_plan.json"); pout = self.out("lk_plan_out.mp4")
+ tool("cut", self.src, "--start", "0", "--end", "1", "-o", pout, "--plan", plan)
+ self.out(".lk_plan_out.mp4.ffskill-lock").write_text("999999")
+ doc = json.loads(tool("render", plan, "--json").stdout)
+ fb = doc["diagnostics"]["fallbacks"][0]
+ self.assertEqual((fb["code"], fb["stage"]), ("stale_lock_taken_over", "cut"))
+ for name in (".lk_out.mp4.ffskill-audit.jsonl", ".lk_plan_out.mp4.ffskill-audit.jsonl"):
+ self.out(name).unlink()
+
def test_fifth_review_regressions(self):
"""Review 5 (2026-09-12): render refuses an output that is one of its clip sources (it
overwrote the source and said completed); fit --method trim drops a subtitle track whose
@@ -3118,11 +3241,13 @@ class ContractTests(unittest.TestCase):
self.assertEqual(json.loads(proc.stdout)["error"]["kind"], kind)
self.assertTrue(keep.exists(), f"failed run deleted a pre-existing output ({extra})")
self.assertEqual((keep.stat().st_size, keep.stat().st_mtime_ns), before, "pre-existing output was modified")
- self.assertEqual([p.name for p in keep.parent.glob(".*ffskill*")], [], "temp file left behind after a failure")
+ # a lock takeover's audit log (another test's) is a deliberate record, not a temp file
+ self.assertEqual([p.name for p in keep.parent.glob(".*ffskill*") if not p.name.endswith(".ffskill-audit.jsonl")], [],
+ "temp file left behind after a failure")
# success replaces the file, through the same temp path, and leaves no temp behind
self.assertEqual(tool("cut", self.src, "--start", "0", "--end", "2", "--overwrite", "-o", keep).returncode, 0)
self.assertNotEqual(keep.stat().st_size, before[0])
- self.assertEqual([p.name for p in keep.parent.glob(".*ffskill*")], [])
+ self.assertEqual([p.name for p in keep.parent.glob(".*ffskill*") if not p.name.endswith(".ffskill-audit.jsonl")], [])
# a fresh path that the failed run created is still cleaned up
fresh = self.out("fresh.mp4")
proc = tool("color", self.src, "--lut", bad, "--json", "-o", fresh, check=False)
@@ -3468,6 +3593,151 @@ class ContractTests(unittest.TestCase):
self.assertEqual(before, after, f"{p.name} modified")
+class SilentPathTests(unittest.TestCase):
+ """docs/design-decisions.md, "Nothing happens silently": the output lock's decisions, in
+ process, and the source-level checks that keep the runner's refusals structured."""
+
+ class Refused(Exception):
+ pass
+
+ def setUp(self):
+ self.tmp = tempfile.mkdtemp(prefix="ffskill_silent_")
+ self.out = os.path.join(self.tmp, "out.mp4")
+ self.lock = Path(self.tmp, ".out.mp4.ffskill-lock")
+ self.audit = Path(self.tmp, ".out.mp4.ffskill-audit.jsonl")
+ self.saved = (_common.STATE.timeout, _common.STATE.dry_run, _common.STATE.audit_log)
+ _common.STATE.dry_run, _common.STATE.audit_log = False, None
+ for bucket in ("warnings", "fallbacks", "skipped", "notes"):
+ setattr(_common.STATE, bucket, [])
+
+ def tearDown(self):
+ _common.STATE.timeout, _common.STATE.dry_run, _common.STATE.audit_log = self.saved
+ for bucket in ("warnings", "fallbacks", "skipped", "notes"):
+ setattr(_common.STATE, bucket, [])
+ shutil.rmtree(self.tmp, ignore_errors=True)
+
+ def enter(self, at=None):
+ """Take the lock (time.time() reads `at` when given); return the refusal's kwargs or None."""
+ from unittest import mock
+
+ def refuse(kind, reason, msg, **kw):
+ raise self.Refused(dict(kw, kind=kind, reason=reason, message=msg))
+ import contextlib
+ try:
+ with contextlib.ExitStack() as stack:
+ stack.enter_context(mock.patch("_common.runner.refuse", refuse))
+ if at is not None:
+ stack.enter_context(mock.patch("time.time", lambda: at))
+ with _common._OutputLock(self.out):
+ return None
+ except self.Refused as r:
+ return r.args[0]
+
+ def test_a_live_holder_keeps_its_lock_past_an_hour_while_inside_its_own_limit(self):
+ """Through 2.10 a lock older than an hour was taken over even while its pid ran, so a
+ legitimate encode longer than an hour lost its lock to a second writer. The writer now
+ dates the lock to its own deadline (child_limit(): 4x --timeout + 60 s)."""
+ _common.STATE.timeout = 1800.0
+ lk = _common._OutputLock(self.out)
+ lk.__enter__()
+ try:
+ self.assertEqual(self.lock.read_text(), str(os.getpid()))
+ refused = self.enter(at=time.time() + 5400) # 1.5 h later, inside 4 x 1800 + 60 s
+ self.assertIsNotNone(refused, "a live holder inside its own limit lost its lock")
+ self.assertEqual((refused["reason"], refused["pid"], refused["pid_alive"], refused["why"]),
+ ("output_locked", os.getpid(), True, "held"))
+ self.assertEqual(refused["lock"], str(self.lock))
+ late = self.enter(at=time.time() + 7400)
+ if platform.system() == "Windows":
+ # Windows cannot delete a lock file its holder still has open: the takeover fails,
+ # and the refusal says why rather than the failure passing silently
+ self.assertIsNotNone(late, "an open lock cannot be deleted on Windows")
+ self.assertEqual((late["reason"], late["why"]), ("output_locked", "past_deadline"))
+ self.assertIn("takeover_error", late)
+ return
+ self.assertIsNone(late, "past its limit, a live pid is taken over")
+ fb = _common.STATE.fallbacks[-1]
+ self.assertEqual((fb["code"], fb["why"], fb["pid_alive"]), ("stale_lock_taken_over", "past_deadline", True))
+ self.assertEqual(json.loads(self.audit.read_text(encoding="utf-8"))["event"], "lock_takeover")
+ finally:
+ lk.__exit__()
+
+ def test_a_takeover_that_cannot_delete_the_lock_is_refused_with_the_reason(self):
+ """On Windows a lock its live holder still has open cannot be deleted; that failure used to
+ be swallowed (`except OSError: pass`). It now refuses with `takeover_error`."""
+ from unittest import mock
+ _common.STATE.timeout = 1800.0
+ lk = _common._OutputLock(self.out)
+ lk.__enter__()
+ try:
+ with mock.patch("os.remove", side_effect=PermissionError("in use by another process")):
+ late = self.enter(at=time.time() + 7400)
+ self.assertIsNotNone(late)
+ self.assertEqual((late["reason"], late["why"]), ("output_locked", "past_deadline"))
+ self.assertIn("in use by another process", late["takeover_error"])
+ finally:
+ lk.__exit__()
+
+ def test_with_no_time_limit_a_live_holders_lock_is_never_taken_over(self):
+ _common.STATE.timeout = 0.0
+ lk = _common._OutputLock(self.out)
+ lk.__enter__()
+ try:
+ self.assertIsNotNone(self.enter(at=time.time() + 5 * 365 * 86400))
+ finally:
+ lk.__exit__()
+
+ def test_a_lock_an_older_version_wrote_is_taken_over_after_an_hour_as_before(self):
+ self.lock.write_text(str(os.getpid()))
+ self.assertIsNotNone(self.enter(at=time.time() + 600))
+ self.assertIsNone(self.enter(at=time.time() + 3700))
+ self.assertEqual(_common.STATE.fallbacks[-1]["why"], "past_deadline")
+
+ def test_running_without_a_lock_is_a_recorded_warning(self):
+ """An unlockable location (os.open fails with something other than EEXIST) ran with no
+ lock and no record. It is now a diagnostics.warnings entry and an audit attempt."""
+ from unittest import mock
+ real_open = os.open
+
+ def no_lock(path, *a, **k):
+ if str(path).endswith(".ffskill-lock"):
+ raise PermissionError(13, "Permission denied", path)
+ return real_open(path, *a, **k)
+ with mock.patch("os.open", no_lock):
+ with _common._OutputLock(self.out):
+ pass
+ w = _common.STATE.warnings[-1]
+ self.assertEqual((w["code"], w["lock"]), ("output_unlocked", str(self.lock)))
+ self.assertEqual(json.loads(self.audit.read_text(encoding="utf-8"))["event"], "output_unlocked")
+
+ def test_a_failed_audit_write_is_a_warning_not_a_failure(self):
+ res = _common.audit("lock_takeover", {"x": 1}, default_path=os.path.join(self.tmp, "missing", "a.jsonl"))
+ self.assertEqual((res["written"], res["reason"]), (False, "unwritable"))
+ self.assertEqual(_common.STATE.warnings[-1]["code"], "audit_log_unwritable")
+ _common.STATE.dry_run = True
+ self.assertEqual(_common.audit("x", {}, default_path=str(self.audit))["reason"], "dry_run")
+ self.assertFalse(self.audit.exists())
+
+ def test_every_refusal_reason_is_in_the_contract_and_the_runner_guards_use_refuse(self):
+ """error.reason is a contract enum (json_output.error_reasons): every refuse(kind, "reason",
+ ...) literal under scripts/ must be listed there, and the runner's output guards must refuse
+ (structured), not die (prose only)."""
+ import inspect
+ reasons = set()
+ for path in script_sources():
+ text = path.read_text(encoding="utf-8")
+ reasons |= set(re.findall(r'\brefuse\(\s*"[a-z_]+",\s*"([a-z_]+)"', text))
+ reasons |= set(re.findall(r'_reject_plan\(path, digest, "([a-z_]+)"', text))
+ self.assertTrue(reasons)
+ self.assertEqual(sorted(reasons - set(_contract.ERROR_REASONS)), [], "refusal reasons missing from _contract.ERROR_REASONS")
+ self.assertEqual(sorted(set(_contract.ERROR_REASONS) - reasons), [], "_contract.ERROR_REASONS lists a reason nothing refuses with")
+ runner = sys.modules["_common.runner"]
+ for fn in (runner._check_no_overwrite_input, runner.refuse_output_is_input, runner._check_output_path,
+ runner.refuse_existing_outputs, runner._OutputLock.__enter__):
+ src = inspect.getsource(fn)
+ self.assertNotRegex(src, r"(?<![\w.])die\(", f"{fn.__qualname__} refuses in prose only (use refuse())")
+
+
class FfmpegSecurityTableTests(unittest.TestCase):
"""doctor's `ffmpeg_security` table and comparison, without running ffmpeg (every OS)."""