Free, open-source AI video generator for Mac. Run LTX-2.5 and MiniMax H3 (Hailuo) locally on Apple Silicon: text-to-video and image-to-video with sound, lip-sync, character LoRA training, songs from lyrics (YuE2) and image editing (Qwen-Image-Edit). One-click install via Pinokio. Offline: no cloud, no API key, no NVIDIA GPU.
Python ai-filmmakingai-videoai-video-generatorapple-siliconhailuoimage-to-videolip-synclocal-ailora-trainingltx-2ltx-2-5ltx-videomacosminimax-h3mlxmusic-generationoffline-aipinokiotext-to-videovideo-generation
release(v4.1.1): the Metal-watchdog retry arms on the machines that needed it
mrbizarro342048459fa4
diff --git a/VERSION b/VERSIONindex ee74734..627a3f4 100644--- a/VERSION+++ b/VERSION@@ -1 +1 @@-4.1.0+4.1.1diff --git a/docs/STATE.md b/docs/STATE.mdindex 802e618..a46fbef 100644--- a/docs/STATE.md+++ b/docs/STATE.md@@ -1,5 +1,13 @@# Phosphene — project state, history, open work+> **🩹 2026-08-15 — v4.1.1: the Metal-watchdog retry finally arms on the machines that needed it.** Every render dying with `helper exited from SIGABRT` on an M2 Max, and the error text inviting the user to file an issue about a retry that could not fire.+>+> - **The mitigation was right and never ran.** `_METAL_TIMEOUT_RX` matched `kIOGPUCommandBufferCallbackErrorTimeout` and `Caused GPU Timeout Error`. macOS names the watchdog kill by how it decided to kill you, and an M2 Max on macOS 26.5.2 kills a long prompt-encode command buffer as **`ImpactingInteractivity`** instead. Same kill, different word, so the Gemma-encode fallback (truncate `LTX2_GEMMA_MAX_LENGTH`, retry once) sat armed-but-unreachable while the render aborted at `[Encoding prompt]`.+> - **Root-caused by the reporter, not by us.** @ybekocak captured the crash line by running the helper's job by hand, proved the engine CLI succeeded on the same weights/venv/machine, and identified the regex as the gap — with the fix confirmed working on their hardware before it was written down here. #44 (M1 Max, v3.2.6) is the same failure wearing the other code.+> - **Deliberately not the whole family.** `OutOfMemory` and `InnocentVictim` are different failures; a shorter prompt is not their fix, and arming a retry on them spends a second render to reach the same end. `test_metal_watchdog_signature.py` (5) pins both positives and all three negatives, and extracts the pattern FROM the panel source so the test cannot drift away from what ships.+>+> **Gates:** `check_ltx_pin` PASS (pin unmoved at `v0.14.19+ltx25.6`) · all 11 panel suites green · `check_output_codec` PASS · `py_compile` panel + helper + image engine.+> **🔁 2026-08-15 — v4.1.0: the forming take MOVES.** The live preview has always decoded a short window of latent frames per publish — 17 on LTX, 14 on H3 — and then thrown all but one away, so the stage showed a still that twitched every few seconds. It now keeps them. Same decode, same cost, one extra encode: an animated WebP the browser loops natively.>> - **THE PIN MOVES: `v0.14.19+ltx25.5` → `v0.14.19+ltx25.6`**, and it is the LTX half of the loop. `live_preview.py` encodes the frames it already had into `preview_latest.webp` (42 ms/frame) alongside the PNG it always wrote, and announces it as `preview_loop` / `preview_loop_frames` in `status.json`. The still is still written and still served as `still_url`, so nothing that consumed the old contract breaks. The H3 half rides its own runner (`codex/live-preview`, `c45a8d4`, 40 ms/frame) and reaches installs by re-running Install Hailuo H3.@@ -8,7 +16,7 @@>> **Gates at the promote tree:** `check_ltx_pin` PASS at `v0.14.19+ltx25.6` (tag verified to CONTAIN the three pyproject bumps, checkout HEAD == tag, installed packages report `0.14.19+ltx25.6` == `_LTX_EXPECTED_VERSION` — the skew class that bit the 25.5 bump, checked explicitly this time) · vendored engine `test_live_preview` + `test_repin_masked_sample` 19 · all 10 panel suites green (`test_stage_live_preview` 18, `test_schedule_preset` 19, `test_geometry_grid` 6, `test_h3_turbo_adapter` 11, `test_character_roundtrip` 35, `test_lora_compat`, `test_spicy_contract`, `test_storyboard`, `test_prompt_enhance_endpoint`, analytics) · `check_output_codec` PASS (yuv420p per sidecar, faststart present) · `py_compile` panel + helper + image engine · `node -c` install.js + update.js · no plain-pip mflux.>-> **Known and NOT fixed here — the forming take can land in the Now card instead of the stage.** Owner-reported on v4.0.9. The stage is refused whenever `_liveStageMediaHeld()` is true — a video playing, or playback controls touched within `LIVE_STAGE_PLAYBACK_HOLD_MS` — which is v4.0.4's deliberate "a clip you are watching is never stolen" rule. The finished clip auto-selects and **autoplays** when a render completes, so starting the next render while it plays hands the stage to the chip and the preview to the corner. Diagnosed, not guessed: `normalizeLivePreview` returns `eligible: true` for an H3 job the moment frames exist, so this is not an eligibility gap. The fix belongs in the hold rule (an autoplaying auto-selected clip is not "a clip you are watching"), and it needs a reproduction on the shipped build before it is changed.+> **Known and NOT fixed here — the forming take can land in the Now card instead of the stage.** The stage is refused whenever `_liveStageMediaHeld()` is true — a video playing, or playback controls touched within `LIVE_STAGE_PLAYBACK_HOLD_MS` — which is v4.0.4's deliberate "a clip you are watching is never stolen" rule. The finished clip auto-selects and **autoplays** when a render completes, so starting the next render while it plays hands the stage to the chip and the preview to the Now card. `normalizeLivePreview` returns `eligible: true` for an H3 job the moment frames exist, so this is not an eligibility gap. The fix belongs in the hold rule (an autoplaying auto-selected clip is not "a clip you are watching"), and it needs a reproduction on the shipped build before it is changed.> **🎯 2026-08-15 — v4.0.7: the reference image is honoured on every tier, "Inspire" becomes a mode, Ingredients stops pretending, and the analytics finally answer per-machine questions.** Four owner reports in one release, plus the schema-v2 render events.>@@ -23,7 +31,7 @@> **🔧 2026-08-14 night — v4.0.5 ASSEMBLED on dev/beta (`65880b3` + version `66f0dbd`), HELD for the owner's word.** The two recovered audits' punch-list, closed in one campaign. (1) **H3 Turbo install is LIVE again**: the v1.0 runner-layout repack is a published release asset on `weights-ltx25-v1` (sha256 `d51d626f…`, verified byte-identical against the local repack the selection was graded on); `/h3/turbo/install` streams it partial→verify→atomic-rename, `install_h3.js` fetches the same asset best-effort via `scripts/fetch_h3_turbo.py` (resumable, digest-checked, short Pinokio-safe lines), and the retired **ckpt500-EMA is accepted again as the LAST fallback** — dropping it in 4.0.4 un-Turboed installs that rendered fine on it. Raw v0.1 stays refused by name. Turbo gate 5→11 tests incl. script↔panel pin parity. (2) **The dead HQ Speed control is gone** (skip-step fields were dropped at the engine boundary; both pills ran identical settings under a "~12% faster" claim), and the collapsed summary stops printing "exact speed" from the retired accel input. (3) **The real speed control**: the engine's 2.5-only `fast` schedule preset (F6S2, 5+2 forwards, owner-graded, **a different take**) wired end to end — Speed pills → `schedule_preset` → make_job (lane-gated: dropped with a sentence on 2.3/HQ/H3) → helper → `generate_two_stage`; tier cells carry `fast_eta` (cost-model 5+2/8+2 ratio, so fixed costs aren't discounted), chips re-price while Fast is armed, and Load Params round-trips it from day one (`test_schedule_preset.py`, 8). (4) **Geometry truth**: make_job floors LTX canvases to the two-stage engine's /64 grid and frames to 8k+1 (floored, never raised) and pushes a sentence when it does; dims inputs step by 64; warnings state what WILL render; H3 tier-cell geometry untouched (`test_geometry_grid.py`, 6). (5) **Load Params restores the cast** (owner-reported): the fire-and-forget IIFE is replaced by an awaited restore through `applyCharacterSelection()` — the new non-toggling hydrator — with visible failure when the character left the library; quality chip by the sidecar's own token, explicit `no_voice` with touched provenance, Extend's four fields, STG (absence = Off), and the Length chip repaints after programmatic frame assignment. A hidden checked No-voice can no longer rewrite a plain render (submit gated on a cast; every hide path clears the box). (6) **One health cluster**: the six machine-state pills live in a single bordered container that is the header's only shrinkable region — narrow windows degrade it to colored state dots, then collapse it; the creator avatar is never clipped again. (7) **Stale-process pill**: `/version` compares HEAD-on-disk to HEAD-at-boot each poll; a promote landing under a running panel renders "Restart to finish update" — **validated live**: the dev panel booted at `90b8a6e` reported `stale_process: true` the moment `65880b3` landed. (8) Keyframe sidecars stamp the resolved q8 pack, not the 2.3-defaulting `MODEL_ID_HQ`. **Gates all green at the tree** (pinokio scripts 377/500 worst · ltx pin unmoved at `v0.14.19+ltx25.4` · post_update · registry 169 · schedules 45 · character round-trip 35 · lora_compat · stage_live_preview · spicy · turbo 11 · schedule_preset 8 · geometry 6 · storyboard suites · py_compile · page parse). **Known-remaining / descoped**: the owner's "stray −1 bar" did not reproduce in either live panel's idle state (needs a screenshot the moment it appears); HDR Load-Params restore is moot while the HDR pill ships commented out (roadmap-parked); render validation of the fast preset + Turbo v1.0 timing run deferred until the GPU frees (the eltrumpo retrain holds both locks overnight). **Audit sources**: `~/AI/projects/phosphene/notes/customize_audit_2026-08-14.md` + `journey_audit_2026-08-14.md` (recovered from the sandboxed sessions' stdout; the FIELD REGISTRY campaign spec in the journey audit is the post-4.0.5 systemic fix).-**Public release: `v4.1.0`** (2026-08-14 — the live preview leaves the postage stamp and plays full-size on the main stage; the Spicy option follows the Settings switch on both engines; H3's Turbo adapter selection moves to LightX2V v1.0 768p and its automatic install is paused until the repack is published. Carries v4.0.3's picker-LoRA crash fix and the v4.0.2 content: the 720p High tier, i2v that animates the image you gave it, characters on the engine's native runtime-LoRA route, and the vendored pin at `v0.14.19+ltx25.4`; see the entries below. The narrative in this paragraph stops at v3.1.1 and is kept as history). **The 2.3/2.5 divergence that ran through the v3.8.x line was closed by v4.0.0** — LTX-2.5 is the registered default and the vendored port is pinned at the tag `v0.14.19+ltx25.4`. v3.8.x registered 2.3 only and pinned `871694d`, because the 2.5 packs were not publicly downloadable at cut time; they are now. See `CLAUDE.md`'s pin table. The entire v3.0 line shipped — v3.0.0 (Characters / Voice / Image Studio / A2V, May 23) → v3.0.1 (FFLF crash fix) → v3.0.2 (Boost/Turbo accel restored after a 2-month silent regression) → v3.0.3 (HiDream hidden, #15) → v3.0.4 (CivitAI SSL) → v3.0.5 (A2V kwarg signature shim, #5) → v3.0.6 (deep-review hardening) → v3.0.7 (GPU-race fix + /status version surfacing) → **v3.0.8 (ltx-2-mlx v0.14.8 catch-up — codec-only patches, I2V "mosaic"/Metal-watchdog fix #17, ASCII version-skew handshake fix)** → **v3.0.9 (custom I2V Width×height inputs restored for power users; Train-tab dev-transformer Download "unknown install key" fix — form-encoding, reported by @cocktailpeanut)** → **v3.0.10 (model-integrity self-heal — boot scan + `/status.model_integrity` + a one-click Repair banner for corrupt/partial weights, the leading *non-Metal* "mosaic" garbage-decode cause; safetensors header+size check)** → **v3.0.11 (latent helper-handshake hang fixed: `_read_until` mixed `select()` on the raw fd with a buffered `readline()`, so any event the helper emitted *right before* `ready` — a version-skew line, the new runtime fingerprint — got stranded in the TextIOWrapper buffer and the panel hung 120 s → "helper failed to start: None". This is the true root cause behind the mislabeled "⚠️-emoji" handshake break. Now reads the raw fd via `os.read()` into a carry buffer and drains every complete line before re-`select()`-ing. PLUS: every render log self-documents its runtime — mlx/mlx-metal version + Apple chip + macOS — surfaced in `/status`, the `helper ready ·` line, and a standalone `runtime |` line, the exact data needed to triangulate the remaining MLX-numerical "mosaic")** → **v3.0.12 (the REAL mosaic fix + 3 more. The mosaic is NOT a code bug or the missing-upscaler theory (both disproved by local repro — the bare 6-file Q4 set renders byte-identical-clean on M4 Max); it's stale/corrupt weight *content* — right size, wrong bytes — that v3.0.10's header+size check can't see (confirmed: ronyeoh/#18 + claude3d/#5 fixed it only by a fresh full re-download). Added **deep checksum verify vs upstream**: Settings → Model files → "Verify model files (checksum)" / `/models/verify-deep` hashes every weight + compares to HuggingFace's published SHA-256 (proven to match local), mismatches feed the existing one-click Repair re-download. Validated: flipped 16 bytes mid-file → header+size passed, deep-verify caught it. ALSO in v3.0.12: character-LoRA-shows-as-"style-only" fix (#5 — `list_user_loras` recovers trigger+kind from the `_v2` filename when the sidecar is missing); `cgi`→`email` multipart migration (Python-3.13-ready, removes the boot DeprecationWarning, adopted from @ssfeather's PR #22); dynamic 3–8 multi-keyframe UI (adopted from @youngbee12's PR #20, ported clean with the integrity feature preserved + a JS ReferenceError fixed). Folds in v3.0.11's handshake fix + runtime fingerprint.)** → **v3.0.13 (character LoRA fixes, #5 @claude3d: a multi-word/spaced character filename no longer mislabels as style-only — the id regex gated out spaces before the sidecar was read; and the Character tab no longer silently falls back to plain T2V — a UI desync left the avatar's selection ring lit after `character_id` was cleared. Plus an `enhancePrompt` element-id fix. Cherry-picked to prod `ce03139`, release `814dc2d`.)** → **v3.1.0 (Ideogram 4 — open-weight 9.3B text-rendering image model + a visual text-placement canvas that places exact text via normalized-bbox captions. mflux 0.18 `mflux-generate-ideogram4`; gated `ideogram-ai/ideogram-4-fp8` ~26 GB, needs a HF Read token + license. Fixed the gated-download 401 that would have hit EVERY user — the panel now exports its configured token to subprocess env via `_sync_hf_token_to_env` so a stale `hf auth login` cache can't shadow it; proven clean-room. Validated end-to-end: 8-text-element field-guide poster rendered clean, every label placed per its bbox.)** → **v3.1.1 (Ideogram output-card buttons hotfix, reported by Mr Bizarro: an Ideogram image's prompt is a caption JSON, which broke every output surface that fed it into a plain box / inline onclick — the Recent-list Animate button was worked-but-dead because apostrophes/&/< in the caption terminated the single-quoted onclick. Display-layer fix: `_displayPromptFor` derives `high_level_description`, `escapeHtml` the onclick args, friendly `_imgEngineLabel`, and `loadParams` restores an Ideogram image into the visual canvas instead of dumping JSON. `params.prompt` stays the caption JSON; renders unaffected. Validated live.)** All validated end-to-end on the dev panel (deep-verify corrupt-detect + repair, /loras, byte-identical upload, multi-keyframe render, FFLF regression). See §4 for the full version history. The May-17 Codex C+ UI restructure (capability tiers, Q4 surface, Character as 5th mode pill, accel kill, HQ-speed move) and the Train-tab / LoRA-chrome work that followed are all baseline now, folded into v3.0.x. There is no `v2.0.6` tag.+**Public release: `v4.1.1`** (2026-08-14 — the live preview leaves the postage stamp and plays full-size on the main stage; the Spicy option follows the Settings switch on both engines; H3's Turbo adapter selection moves to LightX2V v1.0 768p and its automatic install is paused until the repack is published. Carries v4.0.3's picker-LoRA crash fix and the v4.0.2 content: the 720p High tier, i2v that animates the image you gave it, characters on the engine's native runtime-LoRA route, and the vendored pin at `v0.14.19+ltx25.4`; see the entries below. The narrative in this paragraph stops at v3.1.1 and is kept as history). **The 2.3/2.5 divergence that ran through the v3.8.x line was closed by v4.0.0** — LTX-2.5 is the registered default and the vendored port is pinned at the tag `v0.14.19+ltx25.4`. v3.8.x registered 2.3 only and pinned `871694d`, because the 2.5 packs were not publicly downloadable at cut time; they are now. See `CLAUDE.md`'s pin table. The entire v3.0 line shipped — v3.0.0 (Characters / Voice / Image Studio / A2V, May 23) → v3.0.1 (FFLF crash fix) → v3.0.2 (Boost/Turbo accel restored after a 2-month silent regression) → v3.0.3 (HiDream hidden, #15) → v3.0.4 (CivitAI SSL) → v3.0.5 (A2V kwarg signature shim, #5) → v3.0.6 (deep-review hardening) → v3.0.7 (GPU-race fix + /status version surfacing) → **v3.0.8 (ltx-2-mlx v0.14.8 catch-up — codec-only patches, I2V "mosaic"/Metal-watchdog fix #17, ASCII version-skew handshake fix)** → **v3.0.9 (custom I2V Width×height inputs restored for power users; Train-tab dev-transformer Download "unknown install key" fix — form-encoding, reported by @cocktailpeanut)** → **v3.0.10 (model-integrity self-heal — boot scan + `/status.model_integrity` + a one-click Repair banner for corrupt/partial weights, the leading *non-Metal* "mosaic" garbage-decode cause; safetensors header+size check)** → **v3.0.11 (latent helper-handshake hang fixed: `_read_until` mixed `select()` on the raw fd with a buffered `readline()`, so any event the helper emitted *right before* `ready` — a version-skew line, the new runtime fingerprint — got stranded in the TextIOWrapper buffer and the panel hung 120 s → "helper failed to start: None". This is the true root cause behind the mislabeled "⚠️-emoji" handshake break. Now reads the raw fd via `os.read()` into a carry buffer and drains every complete line before re-`select()`-ing. PLUS: every render log self-documents its runtime — mlx/mlx-metal version + Apple chip + macOS — surfaced in `/status`, the `helper ready ·` line, and a standalone `runtime |` line, the exact data needed to triangulate the remaining MLX-numerical "mosaic")** → **v3.0.12 (the REAL mosaic fix + 3 more. The mosaic is NOT a code bug or the missing-upscaler theory (both disproved by local repro — the bare 6-file Q4 set renders byte-identical-clean on M4 Max); it's stale/corrupt weight *content* — right size, wrong bytes — that v3.0.10's header+size check can't see (confirmed: ronyeoh/#18 + claude3d/#5 fixed it only by a fresh full re-download). Added **deep checksum verify vs upstream**: Settings → Model files → "Verify model files (checksum)" / `/models/verify-deep` hashes every weight + compares to HuggingFace's published SHA-256 (proven to match local), mismatches feed the existing one-click Repair re-download. Validated: flipped 16 bytes mid-file → header+size passed, deep-verify caught it. ALSO in v3.0.12: character-LoRA-shows-as-"style-only" fix (#5 — `list_user_loras` recovers trigger+kind from the `_v2` filename when the sidecar is missing); `cgi`→`email` multipart migration (Python-3.13-ready, removes the boot DeprecationWarning, adopted from @ssfeather's PR #22); dynamic 3–8 multi-keyframe UI (adopted from @youngbee12's PR #20, ported clean with the integrity feature preserved + a JS ReferenceError fixed). Folds in v3.0.11's handshake fix + runtime fingerprint.)** → **v3.0.13 (character LoRA fixes, #5 @claude3d: a multi-word/spaced character filename no longer mislabels as style-only — the id regex gated out spaces before the sidecar was read; and the Character tab no longer silently falls back to plain T2V — a UI desync left the avatar's selection ring lit after `character_id` was cleared. Plus an `enhancePrompt` element-id fix. Cherry-picked to prod `ce03139`, release `814dc2d`.)** → **v3.1.0 (Ideogram 4 — open-weight 9.3B text-rendering image model + a visual text-placement canvas that places exact text via normalized-bbox captions. mflux 0.18 `mflux-generate-ideogram4`; gated `ideogram-ai/ideogram-4-fp8` ~26 GB, needs a HF Read token + license. Fixed the gated-download 401 that would have hit EVERY user — the panel now exports its configured token to subprocess env via `_sync_hf_token_to_env` so a stale `hf auth login` cache can't shadow it; proven clean-room. Validated end-to-end: 8-text-element field-guide poster rendered clean, every label placed per its bbox.)** → **v3.1.1 (Ideogram output-card buttons hotfix, reported by Mr Bizarro: an Ideogram image's prompt is a caption JSON, which broke every output surface that fed it into a plain box / inline onclick — the Recent-list Animate button was worked-but-dead because apostrophes/&/< in the caption terminated the single-quoted onclick. Display-layer fix: `_displayPromptFor` derives `high_level_description`, `escapeHtml` the onclick args, friendly `_imgEngineLabel`, and `loadParams` restores an Ideogram image into the visual canvas instead of dumping JSON. `params.prompt` stays the caption JSON; renders unaffected. Validated live.)** All validated end-to-end on the dev panel (deep-verify corrupt-detect + repair, /loras, byte-identical upload, multi-keyframe render, FFLF regression). See §4 for the full version history. The May-17 Codex C+ UI restructure (capability tiers, Q4 surface, Character as 5th mode pill, accel kill, HQ-speed move) and the Train-tab / LoRA-chrome work that followed are all baseline now, folded into v3.0.x. There is no `v2.0.6` tag.> **🎬 2026-08-14 — v4.0.4: the forming take plays on the MAIN STAGE.** The live preview existed since v3.7 and lived in a postage stamp next to the queue, which is not where anyone looks while they wait. It now takes the big player.>diff --git a/mlx_ltx_panel.py b/mlx_ltx_panel.pyindex 29fada4..1600621 100644--- a/mlx_ltx_panel.py+++ b/mlx_ltx_panel.py@@ -12115,8 +12115,24 @@ class WarmHelper:# normal prompt is untouched. This is NOT applied pre-emptively — it# costs conditioning headroom on very long prompts, so it only arms# itself after this machine has actually proven it needs it.+ # The watchdog does NOT report one code. It reports whichever one fits how+ # it decided to kill you, and the choice varies by chip and macOS build:+ # Timeout — the classic long-command-buffer kill (M1 Max, #44)+ # ImpactingInteractivity — same kill, named for the UI stall it caused,+ # which is what an M2 Max on macOS 26.5.2 emits+ # (#59, root-caused by @ybekocak with the fix+ # confirmed on their machine)+ # Matching only the first meant the mitigation below existed, was correct,+ # and never armed on the second — the render just died with SIGABRT and the+ # error text told the user to open an issue about a retry that could not+ # happen. Deliberately NOT the whole kIOGPUCommandBufferCallbackError family:+ # OutOfMemory and InnocentVictim are different failures and a shorter prompt+ # is not their fix._METAL_TIMEOUT_RX = re.compile(- r"kIOGPUCommandBufferCallbackErrorTimeout|Caused GPU Timeout Error",+ r"kIOGPUCommandBufferCallbackErrorTimeout"+ r"|kIOGPUCommandBufferCallbackErrorImpactingInteractivity"+ r"|Caused GPU Timeout Error"+ r"|Impacting Interactivity",re.IGNORECASE)# Anything proving the run got PAST prompt encoding. If the watchdog# fires later than that (denoise, decode), a shorter prompt encode isdiff --git a/test_metal_watchdog_signature.py b/test_metal_watchdog_signature.pynew file mode 100644index 0000000..03dcd16--- /dev/null+++ b/test_metal_watchdog_signature.py@@ -0,0 +1,68 @@+"""The Metal GPU-watchdog signature the Gemma-encode fallback arms on.++The mitigation (truncate LTX2_GEMMA_MAX_LENGTH and retry once) was correct and+present since v3.x, but its matcher only knew the `Timeout` spelling. macOS+picks the code by how it decided to kill you, and on an M2 Max / macOS 26.5.2 it+kills a long prompt-encode command buffer as `ImpactingInteractivity` instead —+so the fallback never armed, the render died with SIGABRT, and the error text+invited the user to file a bug about a retry that could not fire. Reported and+root-caused by @ybekocak in #59; #44 is the same failure with the other code.++The negative cases matter as much: OutOfMemory and InnocentVictim are different+failures. Arming a shorter-prompt retry on those burns a second render to reach+the same end.+"""++import re+import unittest+from pathlib import Path++PANEL = Path(__file__).with_name("mlx_ltx_panel.py")+++def _panel_regex() -> re.Pattern:+ """The REAL pattern out of the panel source, so this cannot drift from it."""+ src = PANEL.read_text()+ i = src.index("_METAL_TIMEOUT_RX = re.compile(")+ body = src[i:src.index("re.IGNORECASE)", i)]+ parts = re.findall(r'r"([^"]+)"', body)+ assert parts, "could not extract the pattern from the panel source"+ return re.compile("".join(parts), re.IGNORECASE)+++class MetalWatchdogSignature(unittest.TestCase):+ def setUp(self):+ self.rx = _panel_regex()++ def test_arms_on_impacting_interactivity(self):+ """#59 — M2 Max / macOS 26.5.2, the line that used to be missed."""+ line = ("libc++abi: terminating due to uncaught exception of type "+ "std::runtime_error: [METAL] Command buffer execution failed: "+ "Impacting Interactivity "+ "(0000000e:kIOGPUCommandBufferCallbackErrorImpactingInteractivity)")+ self.assertTrue(self.rx.search(line))++ def test_arms_on_classic_timeout(self):+ """#44 — M1 Max, the spelling that always worked. Must keep working."""+ line = ("[METAL] Command buffer execution failed: Caused GPU Timeout "+ "Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout)")+ self.assertTrue(self.rx.search(line))++ def test_does_not_arm_on_out_of_memory(self):+ line = ("[METAL] Command buffer execution failed: Out Of Memory "+ "(00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)")+ self.assertFalse(self.rx.search(line))++ def test_does_not_arm_on_innocent_victim(self):+ line = "[METAL] failed: (00000004:kIOGPUCommandBufferCallbackErrorInnocentVictim)"+ self.assertFalse(self.rx.search(line))++ def test_ordinary_render_output_never_arms(self):+ for line in ("Denoising: 50%|##### | 4/8 [00:21<00:21, 5.4s/it]",+ "[Decoding video + audio + muxing] done in 15.6s",+ "step:generate done"):+ self.assertFalse(self.rx.search(line), line)+++if __name__ == "__main__":+ unittest.main()