Skip to content

Free, open-source AI video generator for Mac. Run LTX-2.5 and MiniMax H3 (Hailuo) locally on Apple Silicon: text-to-video and image-to-video with sound, lip-sync, character LoRA training, songs from lyrics (YuE2) and image editing (Qwen-Image-Edit). One-click install via Pinokio. Offline: no cloud, no API key, no NVIDIA GPU.

v4.13.2 — Stop means stop

mrbizarroe1d500e36d2f
diff --git a/README.md b/README.md
index 17aab86..fc3acb9 100644
--- a/README.md
+++ b/README.md
@@ -19,7 +19,7 @@
</p>
-> **Current release: v4.13.1.** **Hailuo H3 that looks right**: High clips run twice the sampler steps, so faces and motion come out sharp (a 5 s High clip now takes about 34 min instead of 19); every clip keeps its last frame; Turbo applies its whole adapter; Upscale ×2 no longer flashes white at the end; twelve more H3 fixes. Full notes on the [releases page](https://github.com/mrbizarro/Phosphene/releases).
+> **Current release: v4.13.2.** **Stop means stop**: pressing Stop while Turbo is still preparing no longer lets the render start afterwards; H3 LoRAs imported before 4.13.1 are applied at their trained strength again; very large Upscale ×2 canvases decode safely (or are refused up front); a failed Turbo companion download is not retried by every job; a silent clip stays silent after Upscale ×2. Full notes on the [releases page](https://github.com/mrbizarro/Phosphene/releases).
## Overview
diff --git a/VERSION b/VERSION
index 56d2fb5..650edfe 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-4.13.1
+4.13.2
diff --git a/docs/STATE.md b/docs/STATE.md
index fd8feaa..e838b55 100644
--- a/docs/STATE.md
+++ b/docs/STATE.md
@@ -1,6 +1,66 @@
# Phosphene — project state, history, open work
-> **🚀 2026-09-16 — v4.13.1 released: the H3 + ×2 fix wave (owner's order: "fix it and ship it").**
+> **🛑 2026-09-17 — v4.13.2 released (public, tag `v4.13.2`): "Stop means stop" — five fixes from an independent review of 4.13.1.**
+> **Why:** Codex re-reviewed the shipped 4.13.1 (PM hub `notes/ship-review-0917/REVIEW.md`): NOT ship-safe, 1 P1 + 4 P2.
+> Each was reproduced on `efe5fcf` with CPU-only probes before fixing (`scratchpad/hotfix/repro_shipped.py`).
+> **Fixes (dev `0e4ec45` `f39222c` `d44d36c`; public = one release commit cherry-picked onto `efe5fcf`, same patch-id):**
+> - **P1 Stop.** `run_tracked_subprocess` (own group, registered, killed by Stop, timeout kills the group);
+> `_register_job_pgid` publishes a pgid and reads the flag in ONE `LOCK` section, and `stop_current_job` sets the flag
+> inside the section that reads the pgids. Used by the H3 renderer, both trainers, post-process mux, the Turbo
+> companion fetch and the ×2 hold-tail/finish. Dispatch re-checks Stop after the fetch and before the spawn; the warm
+> helper refuses a stopped job. Cancellation OWNER = the job the worker thread runs (`_JOB_CTX`, set around
+> `run_job_inner`): children (One Shot parts `<take>-p1`, retakes) obey their take's Stop; off-queue work (editor
+> proxies, film assembly) never inherits a render's Stop. `JobCancelled` lets best-effort steps re-raise.
+> - **P2 LoRA strength.** Files whose per-module alphas are (or will be) folded list at 1.0; `_h3_lora_prepare` moves the
+> sidecar to 1.0 and keeps `recommended_strength_before_fold`; a job without `h3_lora_scale_v` (queued by an older
+> build) at that old number renders at 1.0 and the migrated value is written back to ITS `params.loras` entry (so Load
+> Params / Finish replay it). `make_job` stamps `h3_lora_scale_v = 2`. Other strengths are kept.
+> - **P3 int32 decode guard.** `_decode_tiling_peak_elements` checks the tiles the pinned tiler really cuts (causal extra
+> frame, per-axis spatial size, last tile, 3×frames×canvas blend buffer). Temporal-only when its tiles are ≥4 latent
+> frames; otherwise spatial + temporal (blended seams); refuse when nothing fits. ×2 admission refuses such canvases
+> up front (`upscale_canvas_decodable`, `_decode_min_group_frames`). **Behaviour change:** 4096×2304 now decodes in
+> 2048 px spatial + 64-frame tiles instead of 16-frame unblended temporal tiles. CPU check, real LTX-2.5 VAE, 384×512
+> latent: spatial 256 px + 32-frame tiles 40.5 dB vs single pass; 16-frame temporal 31.5 dB.
+> - **P4 companion fetch.** One fetch at a time (non-blocking lock, all callers), uuid temp file, failure remembered
+> 6 h per target (in memory; Install Turbo / the managed download retry at once), Stop is not a failure,
+> `/status` `turbo.adaln_fetch`.
+> - **P5 ×2 sound.** The result always carries the source's sound or none (silent 121-frame sources kept the model's
+> invented track before); `_probe_audio_state` tells "none" from "unknown" (unknown → optional-map mux + re-probe);
+> the `.mux.mp4` temp is removed on every exit.
+> **Codex review of the exact diff (3 rounds, gpt-6-astra xhigh):** R1 NOT SHIP-SAFE (recipe kept 0.5; temp file on
+> Stop; editor proxy inherited a render's Stop; 17-frame admission too strict) → fixed. R2 NOT SHIP-SAFE (take parts
+> ignored Stop; repeated-path LoRA writeback) → fixed. **R3: "SHIP-SAFE … No new serious defect or remaining release
+> blocker was found in this diff."** Reports: PM hub `notes/ship-review-0917/HOTFIX_REVIEW{_R1,_R2,}.md`.
+> Known, not fixed: `_join_take_parts` still uses plain `subprocess.run` (CPU join, pre-existing); failure memory resets
+> on panel restart; a render meeting an in-flight companion fetch renders without it.
+> **Gates:** new `test_h3_stop_and_companion` 22, `test_h3_lora_strength_migration` 14, `test_upscale_x2_tail` 31.
+> Full `release_gates.sh` 95 PASS / 0 FAIL / 0 SKIP in the release worktree, MLX forced to the CPU (8199 rendering).
+> **Clean room:** fresh clone of the release commit, fresh GitHub clone of the engine, `ltx_checkout.sh` + `ltx_venv.sh`
+> + install's uv pip steps, no weights, empty state/outputs/HF; booted on :8398 → `/version` 4.13.2, UI 200, integrity
+> OK, no tracebacks; stopped by PID, removed. Runner unchanged (`codex/h3-engine-v2` tip `11b90a0`).
+> **Not run:** any render, update-path gate. **Pinokio post: NOT posted** — draft `launch/09_pinokio_4132_stop_fixes.md`
+> (covers 4.13.1 + 4.13.2; 4.13.1 was never posted). **Next promote:** parent on the v4.13.2 release commit.
+
+> **🧹 2026-09-17 — GitHub triage: PR #82 merged; #80 #81 closed (for the next release).**
+> **PR #82** (Viktorminator): the CivitAI modal's Search button used `.pill-btn` (`width:100%`), so the query field
+> shrank to 43 px and the button took 823 px of the row. Reproduced on 4.13.1 and verified fixed with static
+> servers on :8471/:8472/:8473: at 1017 px the field is 799 px and the button 67 px; at 375 px, 233/69; the
+> Hugging Face source is fine too; Enter and the button both call `/civitai/search`. `release_gates.sh --fast`
+> in the PR worktree: 87 PASS / 0 FAIL / 2 SKIP (MLX on the CPU). **Squash-merged on public main as `efe5fcf`**,
+> on top of the v4.13.1 release commit. VERSION is unchanged, so Pinokio users who press Update get this CSS fix
+> under the 4.13.1 label. Cherry-picked to dev as `13bfebd`. **Next promote:** the snapshot commit's parent is
+> `efe5fcf` (origin/main), and the dev tree already carries the change. **Release notes:** include "the CivitAI
+> search field is full width again (thanks @Viktorminator, #82)".
+> **Issues:** #80 and #81 are closed (fixed in 4.12.4; the reporter was silent after we asked). #62: Piotr's
+> package is reviewed. The spec is the High recipe, all 5200 steps ran, all 52 captions carry `cjhtrn`, and the
+> zip's `training_data/` is empty. Next step, promised on the thread: retrain his dataset here and measure
+> the adapter's identity-family delta_rms (his is 7.8e-04; working characters are ~1.7e-03). This needs about 6 h of
+> GPU time. Minor finding: `lora_lab/train_character.py` ignores the spec's `checkpoint_interval` and uses
+> steps//5 instead; this does not change what gets learned. #63: cross-linked to #83. The staged runner still
+> hardcodes `anchors = ("first",)`, so a last-frame anchor is not wired. #24: corrected the old Avoid description.
+> Left alone: #78 and #48 (nudged 09-16 and 09-08), #61 (asked 09-13), #83 (answered 09-16).
+
+> **🚀 2026-09-16 — v4.13.1 released (public `ba83d51`, tag `v4.13.1`): the H3 + ×2 fix wave (owner's order: "fix it and ship it").**
> **Shipped (cherry-picked onto public 4.13.0, NOT a dev read-tree):** `3a5c5cc` autoplay only on click,
> `85a7098` H3 reference image validated with PIL, `745dc9c` Turbo adaLN + provenance, `8eafdc5` High = 16
> sigma points, `1e1ec51` ×2 int32 tail + full frame count, `3a4d9ad`, `a1c9808` the twelve round-2 fixes,
@@ -12,8 +72,15 @@
> `minimax-h3-mlx codex/h3-engine-v2`, which is `11b90a0` (mux keeps every frame, LoRA accounting). So H3
> users get the runner fix by pressing Update; no reinstall.
> **Gates:** full `release_gates.sh` 93 PASS / 0 FAIL / 0 SKIP in the release worktree, MLX forced to the CPU
-> device via a `sitecustomize` shim (the GPU was rendering the owner's jobs on 8199). Clean-room boot of a
-> fresh clone on a spare port: see the release report in the PM hub.
+> device via a `sitecustomize` shim (the GPU was rendering the owner's jobs on 8199). **Clean room:** fresh clone of `ba83d51` in a temp dir; `ltx_checkout.sh` + `ltx_venv.sh` + the install's
+> uv pip steps (ltx25.7 packages) with no weights, empty `HF_HOME`/state/outputs; panel booted on :8397 — `/version`
+> 4.13.1 / `ba83d51`, UI 200 with the TEST PANEL badge, every High cell `steps 16 / forwards 15` (High 5 s ~34 min),
+> model integrity OK, no tracebacks; `install_h3.js` branch clone + `h3_checkout.sh` → `11b90a0`. Stopped by PID,
+> clone removed. **Not run:** update-path gate on a previous install and any render (no GPU tonight).
+> **Pinokio post: NOT posted** — draft in the PM hub `launch/08_pinokio_4131_h3_fixes.md`, waiting for the owner's go.
+> **Next promote:** public main is again a cherry-pick of dev; a later read-tree of dev is fine (it carries these
+> commits) but will ship the held Editor audio work — only on the owner's word after he has used it.
+
> **🔎 2026-09-16 — H3 round-2 review: High runs 15 forwards, twelve bugs fixed (UNRELEASED, `dev`).**
> **High = 16 points (`8eafdc5`).** The owner ran a matched gym A/B: High 1024×576 was clearly better at 15
diff --git a/mlx_ltx_panel.py b/mlx_ltx_panel.py
index cac47a3..64fdb45 100644
--- a/mlx_ltx_panel.py
+++ b/mlx_ltx_panel.py
@@ -2961,7 +2961,31 @@ def _h3_turbo_asset(key: str | None = None) -> dict:
"sha256": H3_TURBO_ASSET_SHA256, "bytes": H3_TURBO_ASSET_BYTES}
-def _h3_turbo_fetch_embedder(target_dir, push_log, runner=None) -> bool:
+# ONE companion fetch at a time, from any caller (the install button, the
+# managed download, the first Turbo render): they share this lock instead of
+# racing each other onto the same file. A failed attempt is remembered per
+# target so a queue of Turbo jobs does not pay the same network timeout once
+# per job; the Turbo install button (force=True) retries at any time.
+_h3_turbo_embedder_lock = threading.Lock()
+_h3_turbo_embedder_failures: dict[str, dict] = {}
+H3_TURBO_EMBEDDER_RETRY_SEC = 6 * 3600
+
+
+def _h3_turbo_embedder_retry_state(target_dir=None) -> dict | None:
+ """The remembered failure for the companion under `target_dir`, while its
+ cooldown lasts: {"failed_at", "retry_at", "error"}; else None."""
+ try:
+ target = Path(target_dir if target_dir is not None else _h3_turbo_dir())
+ except Exception: # noqa: BLE001
+ return None
+ rec = _h3_turbo_embedder_failures.get(str(target / H3_TURBO_EMBEDDER_FILE))
+ if not rec or time.time() >= rec["retry_at"]:
+ return None
+ return dict(rec)
+
+
+def _h3_turbo_fetch_embedder(target_dir, push_log, runner=None, *,
+ force: bool = False) -> bool:
"""Fetch Turbo's adaLN companion (H3_TURBO_EMBEDDER_FILE, ~63 MB) with the
H3 pack's own `scripts/fetch_time_embedder.py`, when the resolved adapter
carries adaLN pairs and the file is not already there.
@@ -2969,7 +2993,13 @@ def _h3_turbo_fetch_embedder(target_dir, push_log, runner=None) -> bool:
Best-effort by design: the adapter works without it (208 of 259 pairs),
so a failure here is logged loudly and the render log repeats it — it
never fails the adapter download. Returns True when the file is present
- afterwards (or was never needed)."""
+ afterwards (or was never needed).
+
+ Coordinated: one fetch at a time (a second caller returns False at once
+ rather than waiting or writing the same file), each attempt writes its
+ own temp file, and a failure is not retried for
+ H3_TURBO_EMBEDDER_RETRY_SEC unless `force` (the install button). A
+ JobCancelled from the runner — Stop during the fetch — is re-raised."""
import subprocess
target_dir = Path(target_dir)
target = target_dir / H3_TURBO_EMBEDDER_FILE
@@ -2978,6 +3008,16 @@ def _h3_turbo_fetch_embedder(target_dir, push_log, runner=None) -> bool:
return True
if _h3_real_file(target, H3_TURBO_EMBEDDER_MIN_BYTES):
return True
+ key = str(target)
+ if not force:
+ rec = _h3_turbo_embedder_retry_state(target_dir)
+ if rec is not None:
+ mins = max(1, int((rec["retry_at"] - time.time()) // 60))
+ push_log(f"[h3:turbo] not fetching {H3_TURBO_EMBEDDER_FILE} again yet — "
+ f"the last attempt failed ({rec['error']}); next automatic try "
+ f"in ~{mins} min, or press Install Turbo to retry now. "
+ f"Turbo's {resolved['adaln_pairs']} adaLN pairs will be skipped")
+ return False
python = _h3_python()
fetcher = H3_ROOT / "scripts" / "fetch_time_embedder.py"
if python is None or not fetcher.is_file():
@@ -2985,40 +3025,63 @@ def _h3_turbo_fetch_embedder(target_dir, push_log, runner=None) -> bool:
f"(H3 pack venv or {fetcher.name} missing) — Turbo's "
f"{resolved['adaln_pairs']} adaLN pairs will be skipped")
return False
- env = dict(os.environ)
- hf_token = _active_hf_token()
- if hf_token:
- env["HF_TOKEN"] = hf_token
- env["HUGGING_FACE_HUB_TOKEN"] = hf_token
- # One small index JSON goes through the hub cache; keep it inside the H3
- # models tree instead of ~/.cache/huggingface.
- env.setdefault("HF_HOME", str(H3_MODELS / "hf_home"))
- env["PYTHONUNBUFFERED"] = "1"
- tmp = target.with_name(target.name + ".partial")
- push_log(f"[h3:turbo] fetching {H3_TURBO_EMBEDDER_FILE} (~60 MB, four "
- f"tensors range-read from MiniMaxAI/MiniMax-H3) for the "
- f"adapter's {resolved['adaln_pairs']} adaLN pairs…")
+ if not _h3_turbo_embedder_lock.acquire(blocking=False):
+ push_log(f"[h3:turbo] {H3_TURBO_EMBEDDER_FILE} is already being fetched — "
+ f"not starting a second download; Turbo's "
+ f"{resolved['adaln_pairs']} adaLN pairs are skipped until it lands")
+ return False
+ import uuid as _uuid
+ tmp = target.with_name(f"{target.name}.{_uuid.uuid4().hex}.partial")
try:
- target_dir.mkdir(parents=True, exist_ok=True)
- proc = (runner or subprocess.run)(
- [str(python), str(fetcher), "--out", str(tmp)],
- capture_output=True, text=True, env=env, timeout=1800)
- if proc.returncode != 0:
- raise RuntimeError((proc.stderr or proc.stdout or "")[-300:].strip()
- or f"exit {proc.returncode}")
- if not _h3_real_file(tmp, H3_TURBO_EMBEDDER_MIN_BYTES):
- raise RuntimeError("fetch wrote no usable file")
- tmp.replace(target)
- except Exception as e: # noqa: BLE001
+ if _h3_real_file(target, H3_TURBO_EMBEDDER_MIN_BYTES):
+ return True # landed while we waited for the lock
+ env = dict(os.environ)
+ hf_token = _active_hf_token()
+ if hf_token:
+ env["HF_TOKEN"] = hf_token
+ env["HUGGING_FACE_HUB_TOKEN"] = hf_token
+ # One small index JSON goes through the hub cache; keep it inside the
+ # H3 models tree instead of ~/.cache/huggingface.
+ env.setdefault("HF_HOME", str(H3_MODELS / "hf_home"))
+ env["PYTHONUNBUFFERED"] = "1"
+ push_log(f"[h3:turbo] fetching {H3_TURBO_EMBEDDER_FILE} (~60 MB, four "
+ f"tensors range-read from MiniMaxAI/MiniMax-H3) for the "
+ f"adapter's {resolved['adaln_pairs']} adaLN pairs…")
+ try:
+ target_dir.mkdir(parents=True, exist_ok=True)
+ proc = (runner or subprocess.run)(
+ [str(python), str(fetcher), "--out", str(tmp)],
+ capture_output=True, text=True, env=env, timeout=1800)
+ if proc.returncode != 0:
+ raise RuntimeError((proc.stderr or proc.stdout or "")[-300:].strip()
+ or f"exit {proc.returncode}")
+ if not _h3_real_file(tmp, H3_TURBO_EMBEDDER_MIN_BYTES):
+ raise RuntimeError("fetch wrote no usable file")
+ tmp.replace(target)
+ except JobCancelled:
+ # The user stopped the job; that is not a failed download.
+ push_log(f"[h3:turbo] {H3_TURBO_EMBEDDER_FILE} fetch stopped")
+ raise
+ except Exception as e: # noqa: BLE001
+ now = time.time()
+ _h3_turbo_embedder_failures[key] = {
+ "failed_at": now, "retry_at": now + H3_TURBO_EMBEDDER_RETRY_SEC,
+ "error": (str(e) or type(e).__name__)[:200]}
+ push_log(f"[h3:turbo] WARNING: {H3_TURBO_EMBEDDER_FILE} fetch failed "
+ f"({e}) — Turbo still works, but its adaLN pairs are "
+ f"skipped; next automatic try in "
+ f"{H3_TURBO_EMBEDDER_RETRY_SEC // 3600} h, or press "
+ f"Install Turbo to retry now")
+ return False
+ _h3_turbo_embedder_failures.pop(key, None)
+ push_log(f"[h3:turbo] adaLN companion installed → {target}")
+ return True
+ finally:
try:
tmp.unlink()
except OSError:
pass
- push_log(f"[h3:turbo] WARNING: {H3_TURBO_EMBEDDER_FILE} fetch failed "
- f"({e}) — Turbo still works, but its adaLN pairs are skipped")
- return False
- push_log(f"[h3:turbo] adaLN companion installed → {target}")
- return True
+ _h3_turbo_embedder_lock.release()
def _h3_turbo_download_bg(target_dir, push_log, asset: dict | None = None) -> None:
@@ -3032,7 +3095,7 @@ def _h3_turbo_download_bg(target_dir, push_log, asset: dict | None = None) -> No
target_dir.mkdir(parents=True, exist_ok=True)
target = target_dir / asset['file']
if target.is_file():
- _h3_turbo_fetch_embedder(target_dir, push_log)
+ _h3_turbo_fetch_embedder(target_dir, push_log, force=True)
_set_h3_turbo_dl(status="done", mb=total_mb, total_mb=total_mb,
error=None)
return
@@ -3070,7 +3133,7 @@ def _h3_turbo_download_bg(target_dir, push_log, asset: dict | None = None) -> No
"please retry")
tmp.replace(target)
push_log(f"[h3:turbo] adapter installed → {target}")
- _h3_turbo_fetch_embedder(target_dir, push_log)
+ _h3_turbo_fetch_embedder(target_dir, push_log, force=True)
_set_h3_turbo_dl(status="done", mb=total_mb, total_mb=total_mb,
error=None)
except Exception as e: # noqa: BLE001
@@ -3119,7 +3182,11 @@ def _h3_install_turbo(push_log, download_fn=None, embedder_fn=None) -> dict:
total_mb=60, error=None)
def _embedder_only():
- ok = (embedder_fn or _h3_turbo_fetch_embedder)(target, push_log)
+ if embedder_fn is not None:
+ ok = embedder_fn(target, push_log)
+ else:
+ # The button is the explicit retry: past any cooldown.
+ ok = _h3_turbo_fetch_embedder(target, push_log, force=True)
_set_h3_turbo_dl(status="done" if ok else "error", mb=60,
total_mb=60,
error=None if ok else
@@ -10533,6 +10600,9 @@ def h3_turbo_status() -> dict:
"missing": paths["missing"],
"note": h3_turbo_note(paths),
"label": "Turbo",
+ # A remembered companion-fetch failure (and when Turbo retries it by
+ # itself); None when there is none. The install button retries now.
+ "adaln_fetch": _h3_turbo_embedder_retry_state(paths.get("dir")),
"adaln": {"pairs": paths.get("adaln_pairs", 0),
"embedder": (str(paths["embedder"])
if paths.get("embedder") else None),
@@ -11456,6 +11526,101 @@ def _h3_lora_convert_kohya(path: Path) -> dict:
"trigger_words": [trigger] if trigger else []}
+# Bumped when the meaning of a picked H3 LoRA strength changes. 2 = per-module
+# alphas are folded into lora_B (4.13.1), so a folded file reads as trained at
+# 1.0. make_job stamps it on every H3 job; a job without it was queued by an
+# older build, with the strength that build recommended.
+H3_LORA_SCALE_VERSION = 2
+
+
+def _h3_lora_scale_state(path: Path) -> str:
+ """How this file's PEFT scale is carried, from its header alone:
+ "folded" the scale is inside lora_B (our conversions, exporters that say so)
+ "mixed" per-module alphas that disagree — `_h3_lora_prepare` folds them
+ "plain" anything else: the sidecar's strength applies as written
+ """
+ try:
+ header, buf_start = _safetensors_header(path)
+ except Exception: # noqa: BLE001
+ return "plain"
+ try:
+ if _h3_lora_scale_report(path, header, buf_start)["source"] == "folded":
+ return "folded"
+ ratios = _h3_lora_alpha_ratios(path, header, buf_start)
+ except Exception: # noqa: BLE001
+ return "plain"
+ if ratios and (max(ratios) - min(ratios)) >= 1e-6:
+ return "mixed"
+ return "plain"
+
+
+def _h3_lora_sidecar_json(path: Path) -> dict | None:
+ try:
+ data = json.loads(path.with_suffix(".json").read_text(encoding="utf-8"))
+ except (OSError, ValueError):
+ return None
+ return data if isinstance(data, dict) else None
+
+
+def _h3_lora_migrate_sidecar(path: Path) -> float | None:
+ """Move a folded file's sidecar to strength 1.0, keeping what it said.
+
+ An adapter imported before its scale was folded into lora_B carries the
+ strength that USED to be right (alpha/rank, e.g. 0.5) next to a file that
+ now already contains it — applied again, the adapter is scaled twice. The
+ old number is kept as `recommended_strength_before_fold`, which is what
+ lets a job queued with it be recognised (see `_h3_lora_migrate_strength`).
+ Returns that pre-fold strength (None when there is none)."""
+ data = _h3_lora_sidecar_json(path)
+ if data is None:
+ return None
+ before = data.get("recommended_strength_before_fold")
+ if before is not None:
+ try:
+ return float(before)
+ except (TypeError, ValueError):
+ return None
+ try:
+ current = float(data.get("recommended_strength") or 1.0)
+ except (TypeError, ValueError):
+ return None
+ if math.isclose(current, 1.0, abs_tol=1e-6):
+ return None
+ if _h3_lora_scale_state(path) != "folded":
+ return None
+ data["recommended_strength_before_fold"] = current
+ data["recommended_strength"] = 1.0
+ data["strength_migrated_at"] = iso_now()
+ try:
+ atomic_write_text(path.with_suffix(".json"), json.dumps(data, indent=2))
+ except Exception as exc: # noqa: BLE001
+ push(f"[h3:lora] WARN: could not update {path.with_suffix('.json').name} ({exc})")
+ push(f"[h3:lora] {path.name}: its scale is now inside the file, so its "
+ f"recommended strength is 1.0 (was {current:g})")
+ return current
+
+
+def _h3_lora_migrate_strength(path: Path, strength: float, params: dict) -> float:
+ """The strength to render a picked, already-prepared H3 LoRA at.
+
+ A job queued by a build before the fold (no `h3_lora_scale_v`) that still
+ carries the file's old automatic recommendation gets 1.0 — the same
+ adapter as trained, instead of that scale applied a second time. Any
+ other strength is the user's own choice and is kept."""
+ before = _h3_lora_migrate_sidecar(path)
+ if before is None or not math.isclose(strength, before, abs_tol=1e-4):
+ return strength
+ if params.get("h3_lora_scale_v"):
+ push(f"[h3:lora] note: {path.name} is picked at {strength:g}, its old "
+ f"recommended strength. The file now carries that scale itself and "
+ f"reads as trained at 1.0 — {strength:g} renders it weaker.")
+ return strength
+ push(f"[h3:lora] {path.name}: this job was queued at the file's old "
+ f"recommended strength {strength:g}; that scale is now inside the file, "
+ f"so it renders at 1.0 (the same adapter, not scaled twice)")
+ return 1.0
+
+
def _h3_lora_prepare(path: Path) -> dict:
"""Make `path` loadable by the H3 runner, or raise saying why it can't be.
@@ -11464,6 +11629,17 @@ def _h3_lora_prepare(path: Path) -> dict:
before a render (so a file dropped in by hand, or a pre-existing download,
gets the same treatment instead of failing inside the runner). Idempotent:
a bare-layout file is inspected and returned untouched."""
+ try:
+ return _h3_lora_prepare_file(path)
+ finally:
+ # A file whose scale is (now) inside lora_B keeps no stale strength.
+ try:
+ _h3_lora_migrate_sidecar(path)
+ except Exception: # noqa: BLE001
+ pass
+
+
+def _h3_lora_prepare_file(path: Path) -> dict:
info = _h3_lora_layout(path)
if info["ok"]:
folded = _h3_lora_fold_mixed_alphas(path)
@@ -11598,6 +11774,7 @@ def _h3_lora_layout_cached(path: Path) -> dict:
except Exception as exc:
info = {"layout": "not_a_lora", "prefix": "", "pairs": 0, "ok": False,
"convertible": False, "reason": str(exc)}
+ info["scale_state"] = _h3_lora_scale_state(path)
# Drop stale entries for the same path (an edited file leaves one behind).
for stale in [k for k in _H3_LORA_LAYOUT_CACHE if k.split("|")[0] == str(path)]:
_H3_LORA_LAYOUT_CACHE.pop(stale, None)
@@ -11648,12 +11825,16 @@ def list_h3_user_loras() -> list[dict]:
"size_bytes": size_bytes,
"lane": "h3",
"trigger_words": list(meta.get("trigger_words") or []),
- # A kohya file is always converted with alpha/rank folded into
- # lora_B, so 1.0 is right. Imports before 2026-09-16 saved the
- # trainer's alpha/rank here as well — a double scale — and are
- # corrected on read rather than trusted.
- "recommended_strength": (1.0 if meta.get("lora_layout") == "kohya"
- else float(meta.get("recommended_strength") or 1.0)),
+ # A file whose scale is inside lora_B (a kohya conversion, a
+ # folded mixed-alpha file) or is about to be (mixed per-module
+ # alphas, folded at the next render) reads as trained at 1.0.
+ # Imports before 2026-09-16 saved alpha/rank here as well — a
+ # double scale — so the sidecar is corrected on read, not trusted.
+ "recommended_strength": (
+ 1.0 if (meta.get("lora_layout") == "kohya"
+ or info.get("layout") == "kohya"
+ or info.get("scale_state") in ("folded", "mixed"))
+ else float(meta.get("recommended_strength") or 1.0)),
"preview_url": preview_url,
"preview_type": preview_type,
"base_model": meta.get("base_model") or "MiniMax H3",
@@ -14259,16 +14440,27 @@ def _probe_video_frames(path: str) -> int:
return 0
-def _video_has_audio(path: str) -> bool:
+def _probe_audio_state(path: str) -> str:
+ """"audio", "none", or "unknown" (ffprobe failed or could not read it).
+
+ The difference between the last two matters wherever the answer decides
+ whether a soundtrack is dropped: "none" is a fact about the file,
+ "unknown" is a fact about the probe."""
try:
- out = subprocess.run(
- [str(FFPROBE), "-v", "error", "-select_streams", "a:0",
+ r = subprocess.run(
+ [str(FFPROBE), "-v", "error", "-select_streams", "a",
"-show_entries", "stream=codec_type", "-of", "csv=p=0", path],
- capture_output=True, text=True, timeout=10,
- ).stdout.strip()
- return "audio" in out
- except Exception:
- return False
+ capture_output=True, text=True, timeout=30,
+ )
+ except Exception: # noqa: BLE001
+ return "unknown"
+ if r.returncode != 0:
+ return "unknown"
+ return "audio" if "audio" in (r.stdout or "") else "none"
+
+
+def _video_has_audio(path: str) -> bool:
+ return _probe_audio_state(path) == "audio"
def ltx_grid_frames_up(frames: int) -> int:
@@ -14302,6 +14494,38 @@ def upscale_frame_plan(src_frames: int, form_frames=None) -> tuple[int, int]:
return out_frames, grid
+# mlx_warm_helper._DECODE_MAX_ELEMENTS: the decoder's int32 limit, with headroom.
+UPSCALE_DECODE_MAX_ELEMENTS = int(0.9 * 2 ** 31)
+
+
+def _decode_min_group_frames(frames: int) -> int:
+ """Output frames of the longest temporal group when the decode is cut into
+ the SMALLEST temporal tiles the pinned tiler allows (2 latent frames, no
+ overlap): split_with_symmetric_overlaps + split_temporal_latents' causal
+ shift, in plain arithmetic. 17 for long clips, 9 for a 17-frame clip."""
+ f = max(1, (max(1, int(frames)) - 1) // 8 + 1)
+ size = 2
+ if f <= size:
+ return 1 + (f - 1) * 8
+ amount = (f + size - 1) // size
+ starts = [i * size for i in range(amount)]
+ ends = [s + size for s in starts]
+ ends[-1] = f
+ starts = starts[:1] + [s - 1 for s in starts[1:]]
+ return max(1 + (e - 1) * 8 - s * 8 for s, e in zip(starts, ends))
+
+
+def upscale_canvas_decodable(width: int, height: int, frames: int) -> bool:
+ """Whether ANY decode tiling keeps this canvas under the decoder's int32
+ limit. Tiling cannot shrink the accumulation buffer the tiler keeps per
+ temporal group — 3 × its frames × the full canvas — below what the
+ shortest temporal tiles give (`_decode_min_group_frames`); every other
+ tensor a spatial tile can shrink. Mirrors the refusal in the helper's
+ `_int32_safe_decode_tiling`."""
+ return (3 * _decode_min_group_frames(frames) * int(width) * int(height)
+ <= UPSCALE_DECODE_MAX_ELEMENTS)
+
+
def _upscale_hold_tail_cmd(src: str, out: str, frames: int, grid: int) -> list[str]:
"""ffmpeg argv: `src` with its last frame held until it is `grid` frames.
@@ -14347,36 +14571,66 @@ def _upscale_finish_cmd(video: str, audio_source: str | None, out: str, *,
def _upscale_finish(video_path: Path, audio_source: str, *, frames: int,
- fps: float, trim: bool) -> bool:
+ fps: float, trim: bool, job: dict | None = None) -> bool:
"""Bring the ×2 render back to the source clip's length and soundtrack.
Returns whether the source's audio is on the result. The whole point of
the H3 → LTX ×2 lane: H3 wrote the dialogue and the sound; the upscaler
- must not throw them away, nor a single frame of the picture."""
- has_audio = _video_has_audio(audio_source)
- if not (has_audio or trim):
- return False
+ must not throw them away, nor a single frame of the picture.
+
+ THE SOUND POLICY IS UNCONDITIONAL: the result carries the source's sound
+ or no sound. The IC pipeline writes a soundtrack of its own, and it used
+ to survive whenever the source was silent AND no trim was needed — so a
+ silent 121-frame clip came back with invented audio while a silent
+ 124-frame clip came back silent. A source whose audio could not be
+ PROBED is not treated as silent: its sound is muxed in if it has any."""
+ state = _probe_audio_state(audio_source)
+ want_source = state in ("audio", "unknown")
tmp = video_path.with_name(video_path.stem + ".mux.mp4")
- cmd = _upscale_finish_cmd(str(video_path), audio_source if has_audio else None,
- str(tmp), frames=frames, fps=fps, trim=trim,
- codec=output_codec_settings())
- try:
- r = subprocess.run(cmd, capture_output=True, text=True, timeout=900)
+ codec = output_codec_settings()
+
+ def _run(with_source: bool) -> tuple[bool, str]:
+ cmd = _upscale_finish_cmd(str(video_path),
+ audio_source if with_source else None,
+ str(tmp), frames=frames, fps=fps, trim=trim,
+ codec=codec)
+ if with_source and state == "unknown":
+ # Optional map: a source that really has no audio stream yields a
+ # silent result instead of an ffmpeg error.
+ cmd[cmd.index("1:a:0")] = "1:a:0?"
+ try:
+ r = run_tracked_subprocess(cmd, pgid_key="mux_pgid",
+ label="the ×2 finish", job=job,
+ timeout=900)
+ except subprocess.TimeoutExpired:
+ tmp.unlink(missing_ok=True)
+ return False, "timed out"
if r.returncode != 0 or not tmp.is_file():
tmp.unlink(missing_ok=True)
- if trim:
- raise RuntimeError("Upscale ×2: could not cut the render back to "
- f"{frames} frames: {(r.stderr or '')[-200:].strip()}")
- push(f"Upscale: audio mux failed ({(r.stderr or '')[-160:].strip()}); clip kept silent")
- return False
+ return False, (r.stderr or "")[-200:].strip()
+ return True, ""
+
+ try:
+ ok, err = _run(want_source)
+ kept = want_source and ok
+ if ok and state == "unknown":
+ kept = _probe_audio_state(str(tmp)) == "audio"
+ if not ok and want_source:
+ push(f"Upscale ×2: could not carry the source sound over ({err}) — "
+ "delivering the clip without sound")
+ ok, err = _run(False)
+ if not ok:
+ raise RuntimeError(
+ (f"Upscale ×2: could not cut the render back to {frames} frames: {err}"
+ if trim else
+ f"Upscale ×2: could not finish the clip (removing the model's own "
+ f"soundtrack failed): {err}"))
tmp.replace(video_path)
- return has_audio
- except subprocess.TimeoutExpired:
+ return kept
+ finally:
+ # Stop, a spawn error, anything: the half-written temp sits NEXT TO
+ # the output, where the gallery would list it.
tmp.unlink(missing_ok=True)
- if trim:
- raise RuntimeError("Upscale ×2: cutting the render back to length timed out")
- push("Upscale: audio mux timed out; clip kept silent")
- return False
def _native_render_for(src: Path) -> Path:
@@ -15564,6 +15818,23 @@ class WarmHelper:
self._ensure()
with self.lock:
assert self.proc is not None and self.proc.stdin is not None
+ # A Stop for THIS job that landed before the job reached the
+ # helper (stop_current_job kills the helper, and _ensure just
+ # respawned it) must not start the render anyway. Stop sets
+ # the flag before it kills, so a kill that lands after this
+ # check still ends the job the ordinary way.
+ with LOCK:
+ _cur = STATE.get("current")
+ _stopped = bool(_cur is not None and job_spec.get("id")
+ and _cur.get("id") == job_spec.get("id")
+ and _cur.get("cancel_requested"))
+ # A child of the worker's queue job (a take part) carries
+ # its own id; its owner is the job this thread runs.
+ _owner = _thread_job()
+ _stopped = _stopped or bool(_owner is not None
+ and _owner.get("cancel_requested"))
+ if _stopped:
+ raise JobCancelled("Stopped before the render started.")
try:
self.proc.stdin.write(json.dumps(job_spec) + "\n")
self.proc.stdin.flush()
@@ -16099,23 +16370,128 @@ def compute_upscale_plan(w: int, h: int, mode: str | None,
}
+class JobCancelled(RuntimeError):
+ """Stop reached the job before (or while) one of its subprocesses ran.
+
+ The worker already files any exception raised under `cancel_requested` as
+ `cancelled`; the type exists so a best-effort step (the Turbo companion
+ fetch) can tell "the user stopped this" apart from "the step failed" and
+ re-raise instead of carrying on to the GPU."""
+
+
+# The queue job the CURRENT THREAD is running, set by the worker around
+# run_job_inner. Cancellation belongs to that job only: editor proxies, film
+# assembly and other HTTP-thread work run outside the queue, and must not
+# inherit a Stop that was meant for whatever render happens to be current.
+_JOB_CTX = threading.local()
+
+
+def _thread_job() -> dict | None:
+ return getattr(_JOB_CTX, "job", None)
+
+
+def _cancel_requested(job: dict | None = None) -> bool:
+ """Whether Stop was pressed for `job` — or for the queue job this thread
+ is running, which OWNS every child it dispatches synchronously (a One
+ Shot take's parts and retakes are their own dicts with their own ids,
+ and Stop only marks the queued parent). No job = no cancellation owner."""
+ if job is None:
+ return False
+ owner = _thread_job()
+ with LOCK:
+ return bool(job.get("cancel_requested")
+ or (owner is not None and owner is not job
+ and owner.get("cancel_requested")))
+
+
+def _raise_if_cancelled(job: dict | None, before: str) -> None:
+ if _cancel_requested(job):
+ raise JobCancelled(f"Stopped before {before}.")
+
+
+def _register_job_pgid(key: str, pgid: int, job: dict | None = None) -> None:
+ """Publish a freshly spawned process group where Stop looks for it.
+
+ THE ORDERING IS THE FIX. `stop_current_job` sets `cancel_requested` and
+ reads every pgid inside one `LOCK` section; this registers the pgid and
+ reads the flag inside another. Whichever section runs first, the other
+ sees its effect: either Stop finds this pgid and kills it, or this finds
+ the flag and kills the group itself. Registering first and checking a
+ moment later outside the lock (the old shape) left a window where Stop
+ saw neither, and the render it had just cancelled started anyway."""
+ with LOCK:
+ STATE[key] = pgid
+ cancelled = _cancel_requested(job)
+ if cancelled:
+ try:
+ os.killpg(pgid, signal.SIGKILL)
+ except (ProcessLookupError, PermissionError, OSError):
+ pass
+ raise JobCancelled("Stopped before this step could start.")
+
+
+def _clear_job_pgid(key: str, pgid: int | None) -> None:
+ with LOCK:
+ if pgid is None or STATE.get(key) == pgid:
+ STATE[key] = None
+
+
+def run_tracked_subprocess(cmd: list[str], *, pgid_key: str, label: str,
+ job: dict | None = None, timeout: float | None = None,
+ **popen_kw) -> subprocess.CompletedProcess:
+ """`subprocess.run(capture_output=True, text=True)`, but Stop can end it.
+
+ Refuses to start after Stop, runs in its own process group registered
+ under `pgid_key` (which `stop_current_job` kills), and kills the whole
+ group on timeout (then re-raises TimeoutExpired, like `subprocess.run`).
+ A process that exits after Stop is returned as-is; callers that go on to
+ more work check `_raise_if_cancelled` themselves. Raises JobCancelled
+ when Stop ended the process."""
+ _raise_if_cancelled(job, label)
+ popen_kw.pop("capture_output", None)
+ popen_kw.pop("text", None)
+ proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+ text=True, start_new_session=True, **popen_kw)
+ # start_new_session: the child is its own group leader, pgid == pid (and
+ # os.getpgid would race a child that has already exited).
+ pgid = proc.pid
+ try:
+ _register_job_pgid(pgid_key, pgid, job)
+ try:
+ stdout, stderr = proc.communicate(timeout=timeout)
+ except subprocess.TimeoutExpired:
+ try:
+ os.killpg(pgid, signal.SIGKILL)
+ except (ProcessLookupError, PermissionError, OSError):
+ pass
+ proc.communicate()
+ raise
+ except BaseException:
+ if proc.poll() is None:
+ try:
+ os.killpg(pgid, signal.SIGKILL)
+ except (ProcessLookupError, PermissionError, OSError):
+ pass
+ try:
+ proc.wait(timeout=5)
+ except Exception: # noqa: BLE001
+ pass
+ raise
+ finally:
+ _clear_job_pgid(pgid_key, pgid)
+ if proc.returncode != 0 and _cancel_requested(job):
+ raise JobCancelled(f"Stopped during {label}.")
+ return subprocess.CompletedProcess(cmd, proc.returncode, stdout or "", stderr or "")
+
+
def run_postprocess_tracked(cmd: list[str], label: str) -> tuple[str, str]:
"""Run a post-process in its own process group so /stop can kill it."""
push(f"{label}: " + " ".join(shlex.quote(c) for c in cmd))
env = os.environ.copy()
env["PATH"] = f"{FFMPEG_BIN}:{env.get('PATH', '')}"
- proc = subprocess.Popen(
- cmd, env=env, text=True,
- stdout=subprocess.PIPE, stderr=subprocess.PIPE,
- start_new_session=True,
- )
- with LOCK:
- STATE["mux_pgid"] = os.getpgid(proc.pid)
- try:
- stdout, stderr = proc.communicate()
- finally:
- with LOCK:
- STATE["mux_pgid"] = None
+ proc = run_tracked_subprocess(cmd, pgid_key="mux_pgid", label=label,
+ job=_thread_job(), env=env)
+ stdout, stderr = proc.stdout, proc.stderr
if proc.returncode != 0:
push((stderr or stdout or "").strip())
raise RuntimeError(f"{label.lower()} exited with code {proc.returncode}")
@@ -16168,11 +16544,15 @@ def stop_current_job(timeout: float = 5.0) -> None:
subprocess). Worker advances."""
with LOCK:
cur = STATE["current"]
+ # Set BEFORE the pgids are read, in the same section: a worker that
+ # spawns right now either registers its pgid first (read below and
+ # killed) or sees this flag when it registers (and kills itself) —
+ # see _register_job_pgid.
+ if cur is not None:
+ cur["cancel_requested"] = True
mux_pgid = STATE.get("mux_pgid")
train_pgid = STATE.get("train_pgid")
h3_pgid = STATE.get("h3_pgid")
- if cur is not None:
- cur["cancel_requested"] = True
push("Stop requested — killing helper + ffmpeg post-process + training subprocess to abort current job.")
HELPER.kill()
# Image-engine subprocesses (HiDream's BF16 helper, mflux's per-family
@@ -22280,6 +22660,8 @@ def make_job(form: dict[str, list[str]] | dict[str, str], *,
"to spend it on your LoRA. Rendering at this shape's own "
f"{_h3_steps or H3_TIERS[_h3_tier]['steps']} sigma points.")
_tier_cfg = H3_TIERS[_h3_tier]
+ # The picked LoRA strengths mean what this build's picker shows.
+ job["params"]["h3_lora_scale_v"] = H3_LORA_SCALE_VERSION
# Portrait = the cell's own canvas, rotated. Identical pixel count and
# packed-row count, so the tier's measured estimate stays honest.
if _h3_orientation == "portrait":
@@ -23456,7 +23838,8 @@ def run_train_job_inner(job: dict) -> None:
)
with LOCK:
STATE["pid"] = proc.pid
- STATE["train_pgid"] = os.getpgid(proc.pid)
+ # Registered and checked against Stop in one lock section.
+ _register_job_pgid("train_pgid", proc.pid, job)
# Stream stdout. Each line is either JSON (a structured progress
# event) or plain text (forwarded as-is to the log).
assert proc.stdout is not None
@@ -23845,7 +24228,7 @@ def run_train_job_inner(job: dict) -> None:
)
with LOCK:
STATE["pid"] = audio_proc.pid
- STATE["train_pgid"] = os.getpgid(audio_proc.pid)
+ _register_job_pgid("train_pgid", audio_proc.pid, job)
assert audio_proc.stdout is not None
for raw in audio_proc.stdout:
line = raw.rstrip("\n")
@@ -23918,6 +24301,8 @@ def run_train_job_inner(job: dict) -> None:
f"{audio_rc}")
except FileNotFoundError as e:
audio_failed_reason = f"audio trainer not available: {e}"
+ except JobCancelled:
+ raise
except Exception as e:
audio_failed_reason = f"audio trainer raised: {e}"
finally:
@@ -24255,6 +24640,7 @@ def run_h3_job_inner(job: dict) -> None:
user_lora_stack: list[tuple[Path, float]] = []
_h3_lora_root = _h3_loras_dir()
_picked: list[tuple[Path, float]] = []
+ _picked_entries: list[dict] = [] # the recipe entry each pick came from
_foreign: list[str] = []
for _entry in (p.get("loras") or []):
_raw = str((_entry or {}).get("path") or "").strip()
@@ -24278,6 +24664,7 @@ def run_h3_job_inner(job: dict) -> None:
except (TypeError, ValueError):
_st = 1.0
_picked.append((_rp, max(-2.0, min(2.0, _st))))
+ _picked_entries.append(_entry)
if _foreign:
# Never silent: an LTX LoRA handed to H3's loader matches zero modules
# and renders as though no adapter were attached at all.
@@ -24312,8 +24699,19 @@ def run_h3_job_inner(job: dict) -> None:
# files never went through the download path); a diffusers-namespace
# file raises with the sentence naming the alpha problem.
_stack_layouts = []
- for _pp, _ps in _picked:
+ for _i, (_pp, _ps) in enumerate(_picked):
_stack_layouts.append(_h3_lora_prepare(_pp).get("layout"))
+ # After prepare, so a file folded just now is recognised too.
+ _new = _h3_lora_migrate_strength(_pp, _ps, p)
+ if _new != _ps and isinstance(_picked_entries[_i], dict):
+ # Into the recipe as well — THIS entry only (a stack may list
+ # the same file twice at different strengths): the sidecar
+ # copies `loras`, and Load Params / Finish replay it with this
+ # build's stamp, so a stale 0.5 there would read as deliberate.
+ _picked_entries[_i]["strength"] = _new
+ _picked[_i] = (_pp, _new)
+ # The recipe now means what this build means by a strength.
+ p["h3_lora_scale_v"] = H3_LORA_SCALE_VERSION
user_lora, user_lora_strength = _picked[0]
user_lora_stack = list(_picked)
p["h3_lora_path"] = str(user_lora)
@@ -24375,11 +24773,17 @@ def run_h3_job_inner(job: dict) -> None:
f"folded v0.1 fallback) under {turbo_paths['dir']}.")
if h3_turbo_adaln_state(turbo_paths) == "no_embedder":
# An adapter downloaded before the adaLN companion came back.
- # ~60 MB, once; a failure leaves the render running without the
- # pairs, and the dispatch log below says so.
+ # ~60 MB, once: a failure is remembered (retried after a cooldown
+ # or from the Turbo install button), and the render runs without
+ # the pairs, which the dispatch log below says out loud. The fetch
+ # runs as a tracked process group, so Stop ends it — and a Stop
+ # that lands during it ends the JOB, not just the download.
_h3_turbo_fetch_embedder(
turbo_paths["dir"], push,
- runner=lambda *a, **kw: subprocess.run(*a, **{**kw, "timeout": 300}))
+ runner=lambda cmd, **kw: run_tracked_subprocess(
+ cmd, pgid_key="h3_pgid", label="the Turbo companion download",
+ job=job, **{**kw, "timeout": 300}))
+ _raise_if_cancelled(job, "the H3 render")
turbo_paths = h3_turbo_paths()
steps = h3_turbo_steps(turbo_paths)
@@ -24698,6 +25102,9 @@ def run_h3_job_inner(job: dict) -> None:
_decode_est = (H3_DECODE_SEC_PER_PX_FRAME * int(width) * int(height)
* int(window_frames) * _hw_speed_factor("h3"))
last_step, total_steps = 0, max(1, steps - 1)
+ # Last look before the GPU: a Stop pressed during any of the preparation
+ # above (the companion fetch, the reference crop) ends the job here.
+ _raise_if_cancelled(job, "the H3 render")
try:
proc = subprocess.Popen(
cmd,
@@ -24713,7 +25120,9 @@ def run_h3_job_inner(job: dict) -> None:
)
with LOCK:
STATE["pid"] = proc.pid
- STATE["h3_pgid"] = os.getpgid(proc.pid)
+ # Registered and checked against Stop in ONE lock section, so a Stop
+ # racing this spawn either finds the pgid or is found by it.
+ _register_job_pgid("h3_pgid", proc.pid, job)
# Crash guard — see reap_orphan_subprocesses(). start_new_session means
# a SIGKILLed panel leaves this 40 GiB render running; the next boot
# reaps it from this file before the queue resumes the same job.
@@ -26126,6 +26535,13 @@ def run_job_inner(job: dict) -> None:
# The form's generation length is not an upscale control; it only
# widens the ceiling past the length this lane is measured at.
out_frames, frames = upscale_frame_plan(src_frames, p.get("frames"))
+ if not upscale_canvas_decodable(up_w, up_h, frames):
+ # The helper refuses this too, but only after the whole render.
+ raise RenderRefused(
+ "hardware_tier",
+ f"Upscale ×2 to {up_w}×{up_h} is past what the video decoder can "
+ f"write safely, even in tiles (the GPU kernels' 2^31-element "
+ f"limit). Use a smaller source clip.")
if src_frames and out_frames < src_frames:
push(f"Upscale ×2: {source.name} is {src_frames} frames; the ×2 covers "
f"its first {out_frames}")
@@ -26178,8 +26594,14 @@ def run_job_inner(job: dict) -> None:
model_src = src
if frames > (src_frames or frames):
model_src = str(work_dir / f"{source.stem}_hold{frames}.mp4")
- _r = subprocess.run(_upscale_hold_tail_cmd(src, model_src, src_frames, frames),
- capture_output=True, text=True, timeout=600)
+ try:
+ _r = run_tracked_subprocess(
+ _upscale_hold_tail_cmd(src, model_src, src_frames, frames),
+ pgid_key="mux_pgid", label="the ×2 preparation", job=job,
+ timeout=600)
+ except BaseException:
+ shutil.rmtree(work_dir, ignore_errors=True)
+ raise
if _r.returncode != 0 or _probe_video_frames(model_src) != frames:
shutil.rmtree(work_dir, ignore_errors=True)
raise RuntimeError(
@@ -26223,12 +26645,13 @@ def run_job_inner(job: dict) -> None:
+ (f"from the clip's own latent, {refine_steps}-step refine + Pixel Spatial Upscaler"
if start_from == "source" else "full re-render from noise + Pixel Spatial Upscaler"))
try:
+ _raise_if_cancelled(job, "the ×2 render")
result = HELPER.run(job_spec)
if "seed_used" in result:
push(f"seed used: {result['seed_used']}")
p["seed_used"] = result["seed_used"]
muxed = _upscale_finish(final_out, src, frames=out_frames, fps=src_fps,
- trim=frames != out_frames)
+ trim=frames != out_frames, job=job)
finally:
shutil.rmtree(work_dir, ignore_errors=True)
delivered = _probe_video_frames(str(final_out))
@@ -27669,7 +28092,11 @@ def worker_loop() -> None:
# concurrently. Blocking acquire is correct here — the worker is
# already serialized; it only waits if an inline image is mid-flight.
with _GPU_LOCK:
- run_job_inner(job)
+ _JOB_CTX.job = job
+ try:
+ run_job_inner(job)
+ finally:
+ _JOB_CTX.job = None
job["status"] = "done"
_CONSEC_FAIL.update(sig="", n=0)
except JobStopped as stop:
diff --git a/mlx_warm_helper.py b/mlx_warm_helper.py
index f5b66a5..da5bdab 100644
--- a/mlx_warm_helper.py
+++ b/mlx_warm_helper.py
@@ -797,36 +797,130 @@ def _vae_decode_max_latent_frames(h_lat: int, w_lat: int) -> int:
return n
-def _int32_safe_decode_tiling(orig, latent_shape, frame_rate=24.0):
+def _decode_tiling_peak_elements(tiling, latent_shape, T=None) -> int:
+ """The largest tensor a decode with `tiling` builds, from the tile
+ extents the pinned tiler will ACTUALLY cut — the causal extra latent frame
+ on every temporal tile after the first, the per-axis spatial tile size
+ `prepare_tiles_for_decoding` derives, and the last tile running to the
+ edge — plus the float32 accumulation buffer `tiled_decode` allocates per
+ temporal group (3 × frames × the FULL canvas). Pure arithmetic over the
+ tiler's own split functions; no tensor is built. `T` is the tiling module."""
+ _, _, f_lat, h_lat, w_lat = (int(v) for v in latent_shape)
+ if tiling is None:
+ return _vae_decode_peak_elements(f_lat, h_lat, w_lat)
+ if T is None:
+ from ltx_core_mlx.model.video_vae import tiling as T
+ t_starts, t_ends = [0], [f_lat]
+ h_max, w_max = h_lat, w_lat
+ sc = getattr(tiling, "spatial_config", None)
+ if sc is not None:
+ long_side = max(h_lat, w_lat)
+ size_lat = sc.tile_size_in_pixels // 32
+ overlap = sc.tile_overlap_in_pixels // 32
+ lower = max(2, overlap + 1)
+ maxes = []
+ for n in (h_lat, w_lat):
+ adjusted = max(lower, round(size_lat * n / long_side))
+ iv = T.split_with_symmetric_overlaps(adjusted, overlap)(n)
+ maxes.append(max(e - s for s, e in zip(iv.starts, iv.ends)))
+ h_max, w_max = maxes
+ tc = getattr(tiling, "temporal_config", None)
+ if tc is not None:
+ iv = T.split_temporal_latents(tc.tile_size_in_frames // 8,
+ tc.tile_overlap_in_frames // 8)(f_lat)
+ t_starts, t_ends = list(iv.starts), list(iv.ends)
+ elif sc is not None:
+ # tiled_decode needs a temporal slice with integer bounds; spatial
+ # tiling without temporal tiling cannot run at all.
+ return 1 << 62
+ t_max = max(e - s for s, e in zip(t_starts, t_ends))
+ peak = _vae_decode_peak_elements(t_max, h_max, w_max)
+ out_frames = max(1 + (e - 1) * 8 - s * 8 for s, e in zip(t_starts, t_ends))
+ return max(peak, 3 * out_frames * (32 * h_lat) * (32 * w_lat))
+
+
+def _decode_temporal_config(T, latent_frames: int, frame_rate: float):
+ tile_frames = int(latent_frames) * 8
+ one_second = max(8, (int(frame_rate) // 8) * 8)
+ overlap = min(one_second, (tile_frames // 32) * 8)
+ if overlap >= tile_frames:
+ overlap = tile_frames - 8
+ return T.TemporalTilingConfig(tile_size_in_frames=tile_frames,
+ tile_overlap_in_frames=overlap)
+
+
+def _int32_safe_decode_tiling(orig, latent_shape, frame_rate=24.0, T=None):
"""`_compute_decode_tiling`, plus an element cap next to its byte budget.
Upstream only tiles when a decode would exceed a MEMORY budget (8 GB by
default), so a 2048×1152 × 121-frame clip — 2.3 GB of block-3 activation —
decodes in one pass and runs straight into the int32 wrap above. When the
- pass is too big, this returns the same temporal tiling upstream uses for
- long clips, with tiles small enough to stay under the cap. Everything
+ pass is too big, this returns upstream's own temporal tiling with tiles
+ small enough to stay under the cap; when no temporal tiling can (a canvas
+ so large that two latent frames plus the causal third already wrap), it
+ ALSO tiles in space. Every candidate is checked against the extents the
+ tiler will really cut (`_decode_tiling_peak_elements`), and a canvas no
+ tiling can fit is refused rather than decoded into garbage. Everything
that already fits keeps upstream's answer untouched."""
cfg = orig(latent_shape, frame_rate=frame_rate)
+ if T is None:
+ from ltx_core_mlx.model.video_vae import tiling as T
+ cap = _DECODE_MAX_ELEMENTS
+
+ def fits(c) -> bool:
+ return _decode_tiling_peak_elements(c, latent_shape, T) <= cap
+
+ if fits(cfg):
+ return cfg
_, _, f_lat, h_lat, w_lat = (int(v) for v in latent_shape)
+ spatial = getattr(cfg, "spatial_config", None) if cfg is not None else None
+ # 1. Temporal only (the 4.13.1 answer, now verified): split_temporal_latents
+ # moves every tile after the first one latent frame earlier (causal
+ # continuity), so those tiles are size + 1 long. Taken as-is when its
+ # tiles are 4+ latent frames: those are 32+ frames with an 8-frame blend
+ # between them. Shorter tiles have NO blend (overlap rounds to 0), and on
+ # the real decoder that seam costs more than a blended spatial one
+ # (CPU, LTX-2.5 VAE, 384×512: 16-frame temporal tiles 31.5 dB vs the
+ # single pass; 256 px spatial + 32-frame temporal tiles 40.5 dB).
max_lat = _vae_decode_max_latent_frames(h_lat, w_lat)
- if f_lat <= max_lat:
- return cfg
- # split_temporal_latents moves every tile after the first one latent
- # frame earlier (causal continuity), so those tiles are size + 1 long.
size = max(2, max_lat - 1)
- tc = getattr(cfg, "temporal_config", None) if cfg is not None else None
- if tc is not None and tc.tile_size_in_frames // 8 <= size:
- return cfg
- from ltx_core_mlx.model.video_vae.tiling import TemporalTilingConfig, TilingConfig
- tile_frames = size * 8
- one_second = max(8, (int(frame_rate) // 8) * 8)
- overlap = min(one_second, (tile_frames // 32) * 8)
- if overlap >= tile_frames:
- overlap = tile_frames - 8
- return TilingConfig(
- spatial_config=getattr(cfg, "spatial_config", None) if cfg is not None else None,
- temporal_config=TemporalTilingConfig(
- tile_size_in_frames=tile_frames, tile_overlap_in_frames=overlap))
+ temporal_only = T.TilingConfig(
+ spatial_config=spatial,
+ temporal_config=_decode_temporal_config(T, size, frame_rate))
+ if size >= 4 and fits(temporal_only):
+ return temporal_only
+ # 2. Space as well. Largest spatial tile first (fewest seams), each with
+ # the longest temporal tile that fits; prefer temporal tiles of 4+.
+ long_px = max(h_lat, w_lat) * 32
+ overlap_px = 64
+ spatial_sizes = []
+ for k in range(2, 129):
+ per_tile = -(-long_px // k) # ceil(long / k)
+ px = -(-per_tile // 32) * 32 # up to a multiple of 32
+ if px <= overlap_px + 32:
+ break
+ if px < long_px and px not in spatial_sizes:
+ spatial_sizes.append(px)
+ fallback = None
+ for px in spatial_sizes:
+ sc = T.SpatialTilingConfig(tile_size_in_pixels=px, tile_overlap_in_pixels=overlap_px)
+ for t in range(max(2, f_lat), 1, -1):
+ c = T.TilingConfig(spatial_config=sc,
+ temporal_config=_decode_temporal_config(T, t, frame_rate))
+ if fits(c):
+ if t >= min(4, max(2, f_lat)):
+ return c
+ fallback = fallback or c
+ break
+ if fits(temporal_only):
+ return temporal_only
+ if fallback is not None:
+ return fallback
+ raise RuntimeError(
+ f"This canvas ({w_lat * 32}×{h_lat * 32}) is too large for the video "
+ f"decoder to write safely, even in tiles: its tensors would pass the "
+ f"2^31-element limit of the GPU kernels and the frames would come out "
+ f"corrupted. Use a smaller size.")
def _install_decode_int32_guard() -> None:
@@ -845,16 +939,21 @@ def _install_decode_int32_guard() -> None:
return
def _guarded(latent_shape, frame_rate: float = 24.0):
- cfg = _int32_safe_decode_tiling(orig, latent_shape, frame_rate=frame_rate)
- tc =getattr(cfg, "temporal_config", None) if cfg is not None else None
- if tc is not None:
+ up = orig(latent_shape, frame_rate=frame_rate)
+ cfg = _int32_safe_decode_tiling(lambda *_a, **_k: up, latent_shape,
+ frame_rate=frame_rate)
+ if cfg is not up:
_, _, f_lat, h_lat, w_lat = (int(v) for v in latent_shape)
- if f_lat > _vae_decode_max_latent_frames(h_lat, w_lat):
- emit({"event": "log",
- "line": f"vae-decode int32 guard: {f_lat} latent frames at "
- f"{w_lat * 32}×{h_lat * 32} decode in tiles of "
- f"{tc.tile_size_in_frames} frames "
- f"(overlap {tc.tile_overlap_in_frames})"})
+ tc = getattr(cfg, "temporal_config", None)
+ sc = getattr(cfg, "spatial_config", None)
+ emit({"event": "log",
+ "line": f"vae-decode int32 guard: {f_lat} latent frames at "
+ f"{w_lat * 32}×{h_lat * 32} decode in tiles of "
+ f"{tc.tile_size_in_frames} frames "
+ f"(overlap {tc.tile_overlap_in_frames})"
+ + (f" × {sc.tile_size_in_pixels}px squares "
+ f"(overlap {sc.tile_overlap_in_pixels}px)"
+ if sc is not None else "")})
return cfg
_guarded._phosphene_int32_guard = True
diff --git a/test_h3_lora_strength_migration.py b/test_h3_lora_strength_migration.py
new file mode 100644
index 0000000..c70712a
--- /dev/null
+++ b/test_h3_lora_strength_migration.py
@@ -0,0 +1,331 @@
+#!/usr/bin/env python3
+"""An H3 LoRA imported before its scale was folded keeps its strength (ship review 2026-09-17).
+
+Before 4.13.1, importing an adapter whose per-module alpha/rank were 0.5 and
+1.0 saved a recommended strength of 0.5. 4.13.1 folds those ratios into each
+lora_B at render time, so the file itself now carries them — but the sidecar
+(and every job already queued with that recommendation) still said 0.5, and
+the modules rendered at 0.25 and 0.5: the adapter, weakened. The listing
+only corrected kohya imports.
+
+Now: a folded (or about-to-be-folded) file lists at 1.0; its sidecar is moved
+to 1.0 with the old number kept; a job queued by an older build at that old
+number renders at 1.0; a strength anyone chose deliberately is kept.
+
+MLX runs on the CPU (tiny tensors); no model, no GPU."""
+from __future__ import annotations
+
+import json
+import os
+import struct
+import sys
+import tempfile
+import unittest
+import unittest.mock
+from contextlib import ExitStack
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent
+_TMP = Path(tempfile.mkdtemp(prefix="phos-h3-lora-mig-"))
+for _k, _d in (("LTX_STATE_DIR", "state"), ("LTX_OUTPUT_DIR", "out"),
+ ("LTX_UPLOADS_DIR", "uploads")):
+ (_TMP / _d).mkdir(parents=True, exist_ok=True)
+ os.environ[_k] = str(_TMP / _d)
+os.environ["PHOSPHENE_ANALYTICS_DISABLED"] = "1"
+os.environ["PHOSPHENE_DISABLE_VERSION_CHECK"] = "1"
+os.environ["LTX_H3_FORCE_CAPABLE"] = "1"
+os.environ.setdefault("LTX_PORT", "8299")
+sys.path.insert(0, str(ROOT))
+
+import mlx_ltx_panel as P # noqa: E402
+
+
+def _safetensors(path: Path, tensors: dict, meta: dict | None = None) -> None:
+ header, blobs, off = {}, [], 0
+ for name, (shape, vals) in tensors.items():
+ raw = struct.pack(f"<{len(vals)}f", *vals)
+ header[name] = {"dtype": "F32", "shape": list(shape),
+ "data_offsets": [off, off + len(raw)]}
+ blobs.append(raw)
+ off += len(raw)
+ if meta:
+ header["__metadata__"] = {k: str(v) for k, v in meta.items()}
+ hb = json.dumps(header).encode()
+ hb += b" " * (-len(hb) % 8)
+ path.write_bytes(struct.pack("<Q", len(hb)) + hb + b"".join(blobs))
+
+
+def _module(name: str, rank: int, alpha: float | None) -> dict:
+ t = {f"{name}.lora_A.weight": ((rank, 3), [1.0] * (rank * 3)),
+ f"{name}.lora_B.weight": ((4, rank), [1.0] * (4 * rank))}
+ if alpha is not None:
+ t[f"{name}.alpha"] = ((), [alpha])
+ return t
+
+
+M1, M2 = "blocks.1.attn.qkv_proj", "blocks.2.attn.qkv_proj"
+
+
+class _LoraDir(unittest.TestCase):
+ def setUp(self):
+ import mlx.core as mx
+ mx.set_default_device(mx.cpu)
+ self.dir = Path(tempfile.mkdtemp(prefix="h3loras-", dir=_TMP))
+ self.st = ExitStack()
+ self.st.enter_context(unittest.mock.patch.object(P, "_h3_loras_dir", lambda: self.dir))
+ self.logs = []
+ self.st.enter_context(unittest.mock.patch.object(P, "push", self.logs.append))
+ P._H3_LORA_LAYOUT_CACHE.clear()
+
+ def tearDown(self):
+ self.st.close()
+ P._H3_LORA_LAYOUT_CACHE.clear()
+
+ def imported(self, name, tensors, strength, meta=None, layout="bare"):
+ """A file + the sidecar a pre-4.13.1 import wrote for it."""
+ f = self.dir / f"{name}.safetensors"
+ _safetensors(f, tensors, meta)
+ f.with_suffix(".json").write_text(json.dumps({
+ "name": name, "kind": "import", "lora_layout": layout,
+ "recommended_strength": strength}))
+ return f
+
+ def mixed(self, name="mixed"):
+ # alpha/rank 0.5 and 1.0; the old import picked 0.5 (farthest from 1).
+ return self.imported(name, {**_module(M1, 4, 2.0), **_module(M2, 4, 4.0)}, 0.5)
+
+ def listed(self, f):
+ P._H3_LORA_LAYOUT_CACHE.clear()
+ return next(e for e in P.list_h3_user_loras()
+ if e["filename"] == f.name)["recommended_strength"]
+
+ def sidecar(self, f):
+ return json.loads(f.with_suffix(".json").read_text())
+
+ def effective(self, f, strength):
+ import mlx.core as mx
+ w = mx.load(str(f))
+ return [round(float(w[f"{m}.lora_B.weight"][0, 0].item()) * strength, 6)
+ for m in (M1, M2)]
+
+ def dispatch(self, f, strength, params):
+ P._h3_lora_prepare(f)
+ return P._h3_lora_migrate_strength(f, strength, params)
+
+
+class AFoldedFileReadsAsTrained(_LoraDir):
+ def test_the_old_import_lists_at_one_before_any_render(self):
+ f = self.mixed()
+ self.assertEqual(P._h3_lora_scale_state(f), "mixed")
+ self.assertEqual(self.listed(f), 1.0)
+ self.assertEqual(self.sidecar(f)["recommended_strength"], 0.5) # untouched by a read
+
+ def test_a_job_queued_before_the_update_renders_as_trained(self):
+ f = self.mixed()
+ st = self.dispatch(f, 0.5, {}) # no h3_lora_scale_v: old build
+ self.assertEqual(st, 1.0)
+ self.assertEqual(self.effective(f, st), [0.5, 1.0]) # as trained, not [0.25, 0.5]
+ sc = self.sidecar(f)
+ self.assertEqual((sc["recommended_strength"], sc["recommended_strength_before_fold"]),
+ (1.0, 0.5))
+ self.assertTrue(sc.get("strength_migrated_at"))
+ self.assertEqual(self.listed(f), 1.0)
+ self.assertTrue(any("renders at 1.0" in m for m in self.logs))
+
+ def test_a_file_4_13_1_already_folded_is_corrected_too(self):
+ f = self.mixed()
+ P._h3_lora_fold_mixed_alphas(f) # what 4.13.1 did, sidecar untouched
+ self.assertEqual(P._h3_lora_scale_state(f), "folded")
+ self.assertEqual(self.sidecar(f)["recommended_strength"], 0.5)
+ self.assertEqual(self.listed(f), 1.0)
+ self.assertEqual(self.dispatch(f, 0.5, {}), 1.0)
+ # and a second job from the same old queue, after the sidecar moved
+ self.assertEqual(self.dispatch(f, 0.5, {}), 1.0)
+
+ def test_a_deliberate_strength_is_kept(self):
+ f = self.mixed()
+ self.assertEqual(self.dispatch(f, 0.8, {}), 0.8)
+ self.assertEqual(self.dispatch(f, 1.0, {}), 1.0)
+ self.assertEqual(self.dispatch(f, -0.5, {}), -0.5)
+
+ def test_a_job_queued_by_this_build_keeps_its_strength_and_says_so(self):
+ f = self.mixed()
+ self.logs.clear()
+ v = {"h3_lora_scale_v": P.H3_LORA_SCALE_VERSION}
+ self.assertEqual(self.dispatch(f, 0.5, v), 0.5)
+ self.assertTrue(any("renders it weaker" in m for m in self.logs))
+ self.assertEqual(self.dispatch(f, 1.0, v), 1.0)
+
+ def test_a_uniform_alpha_keeps_its_strength(self):
+ # Not folded: the strength IS where alpha/rank gets applied.
+ f = self.imported("uniform", {**_module(M1, 4, 2.0), **_module(M2, 4, 2.0)}, 0.5)
+ before = f.read_bytes()
+ self.assertEqual(P._h3_lora_scale_state(f), "plain")
+ self.assertEqual(self.listed(f), 0.5)
+ self.assertEqual(self.dispatch(f, 0.5, {}), 0.5)
+ self.assertEqual(f.read_bytes(), before)
+ self.assertNotIn("recommended_strength_before_fold", self.sidecar(f))
+ self.assertEqual(self.effective(f, 0.5), [0.5, 0.5])
+
+ def test_a_kohya_conversion_saved_at_its_alpha_is_corrected(self):
+ f = self.imported("kohya", {**_module(M1, 4, None), **_module(M2, 4, None)}, 0.25,
+ meta={"converted_by": "phosphene", "converted_from": "kohya",
+ "ss_network_alpha": "1"},
+ layout="kohya")
+ self.assertEqual(self.listed(f), 1.0)
+ self.assertEqual(self.dispatch(f, 0.25, {}), 1.0)
+ self.assertEqual(self.sidecar(f)["recommended_strength_before_fold"], 0.25)
+
+ def test_an_exporter_folded_file_at_one_is_left_alone(self):
+ f = self.imported("baked", {**_module(M1, 4, None)}, 1.0,
+ meta={"baked_scale": "0.0625"})
+ before = f.with_suffix(".json").read_text()
+ self.assertEqual(self.dispatch(f, 1.0, {}), 1.0)
+ self.assertEqual(self.dispatch(f, 0.3, {}), 0.3)
+ self.assertEqual(f.with_suffix(".json").read_text(), before)
+
+ def test_no_sidecar_is_fine(self):
+ f = self.dir / "bare.safetensors"
+ _safetensors(f, {**_module(M1, 4, 2.0), **_module(M2, 4, 4.0)})
+ self.assertEqual(self.listed(f), 1.0)
+ self.assertEqual(self.dispatch(f, 0.5, {}), 0.5)
+ self.assertFalse(f.with_suffix(".json").exists())
+
+
+class TheRenderPathUsesIt(_LoraDir):
+ def test_the_runner_gets_one_for_an_old_queued_job(self):
+ f = self.mixed()
+ job = {"id": "lora-old", "params": {
+ "engine": "h3", "mode": "t2v", "h3_tier": "standard_5s",
+ "prompt": "a dancer", "seed": "7", "h3_turbo": False,
+ "loras": [{"path": str(f), "strength": 0.5}]}}
+ seen = {}
+
+ class Intercept(Exception):
+ pass
+
+ def popen(cmd, **kw):
+ seen["cmd"] = cmd
+ raise Intercept()
+
+ paths = dict(missing=[], repairable=False, dit=_TMP / "dit", python=sys.executable,
+ runner=_TMP / "gen.py", compact_root=_TMP / "c", text_config=_TMP / "t")
+ ff = _TMP / "ffmpeg"
+ ff.write_text("")
+ with ExitStack() as st:
+ for name, val in {
+ "h3_paths": lambda: paths, "h3_capable": lambda: True,
+ "h3_supports_lora": lambda: True, "h3_supports_lora_stack": lambda: False,
+ "h3_lora_max_stack": lambda: 1, "h3_dit_choice": lambda: ("bf16", None),
+ "output_codec_settings": lambda: {"crf": "18", "pix_fmt": "yuv420p"},
+ "_h3_runner_has_flag": lambda flag: False, "FFMPEG": ff}.items():
+ st.enter_context(unittest.mock.patch.object(P, name, val))
+ for name in ("h3_supports_memory_limits", "h3_supports_prompt_cache",
+ "h3_live_preview_ready", "h3_supports_stage_a",
+ "h3_supports_tae_draft"):
+ st.enter_context(unittest.mock.patch.object(P, name, lambda: False))
+ st.enter_context(unittest.mock.patch.object(P.HELPER, "is_alive", lambda: False))
+ st.enter_context(unittest.mock.patch.object(P.subprocess, "Popen", popen))
+ with self.assertRaises(Intercept):
+ P.run_h3_job_inner(job)
+ cmd = seen["cmd"]
+ spec = cmd[cmd.index("--lora") + 1]
+ self.assertEqual(spec, f"{f.resolve()}:1")
+ self.assertEqual(job["params"]["h3_lora_strength"], 1.0)
+ self.assertEqual(self.effective(f, 1.0), [0.5, 1.0])
+
+ def _dispatch_argv(self, job, stack=False):
+ seen = {}
+
+ class Intercept(Exception):
+ pass
+
+ def popen(cmd, **kw):
+ seen["cmd"] = cmd
+ raise Intercept()
+
+ paths = dict(missing=[], repairable=False, dit=_TMP / "dit", python=sys.executable,
+ runner=_TMP / "gen.py", compact_root=_TMP / "c", text_config=_TMP / "t")
+ ff = _TMP / "ffmpeg"
+ ff.write_text("")
+ with ExitStack() as st:
+ for name, val in {
+ "h3_paths": lambda: paths, "h3_capable": lambda: True,
+ "h3_supports_lora": lambda: True,
+ "h3_supports_lora_stack": lambda: stack,
+ "h3_lora_max_stack": lambda: 4 if stack else 1,
+ "h3_dit_choice": lambda: ("bf16", None),
+ "output_codec_settings": lambda: {"crf": "18", "pix_fmt": "yuv420p"},
+ "_h3_runner_has_flag": lambda flag: False, "FFMPEG": ff}.items():
+ st.enter_context(unittest.mock.patch.object(P, name, val))
+ for name in ("h3_supports_memory_limits", "h3_supports_prompt_cache",
+ "h3_live_preview_ready", "h3_supports_stage_a",
+ "h3_supports_tae_draft"):
+ st.enter_context(unittest.mock.patch.object(P, name, lambda: False))
+ st.enter_context(unittest.mock.patch.object(P.HELPER, "is_alive", lambda: False))
+ st.enter_context(unittest.mock.patch.object(P.subprocess, "Popen", popen))
+ with self.assertRaises(Intercept):
+ P.run_h3_job_inner(job)
+ cmd = seen["cmd"]
+ got = [float(cmd[i + 1].rsplit(":", 1)[1]) for i, a in enumerate(cmd) if a == "--lora"]
+ return got if stack else got[0]
+
+ def test_a_repeated_file_in_a_stack_migrates_only_its_own_entry(self):
+ # Codex hotfix review round 2: the same file twice, once at the old
+ # recommendation and once deliberately.
+ f = self.mixed()
+ job = {"id": "lora-dup", "params": {
+ "engine": "h3", "mode": "t2v", "h3_tier": "standard_5s",
+ "prompt": "a dancer", "seed": "7", "h3_turbo": False,
+ "loras": [{"path": str(f), "strength": 0.5},
+ {"path": str(f), "strength": 0.7}]}}
+ self.assertEqual(self._dispatch_argv(job, stack=True), [1.0, 0.7])
+ self.assertEqual([l["strength"] for l in job["params"]["loras"]], [1.0, 0.7])
+ replay = P.make_job({k: [v] for k, v in {
+ "mode": "t2v", "engine": "h3", "prompt": "a dancer", "h3_turbo": "0",
+ "loras": json.dumps(job["params"]["loras"])}.items()})
+ replay["id"] = "lora-dup2"
+ self.assertEqual(self._dispatch_argv(replay, stack=True), [1.0, 0.7])
+
+ def test_the_corrected_strength_survives_load_params_and_finish(self):
+ # Codex hotfix review #1: the recipe the sidecar copies (and Load
+ # Params / Finish replay) must carry the migrated strength.
+ f = self.mixed()
+ job = {"id": "lora-rt", "params": {
+ "engine": "h3", "mode": "t2v", "h3_tier": "standard_5s",
+ "prompt": "a dancer", "seed": "7", "h3_turbo": False,
+ "loras": [{"path": str(f), "strength": 0.5, "name": "mixed"}]}}
+ self.assertEqual(self._dispatch_argv(job), 1.0)
+ rec = job["params"]
+ self.assertEqual(rec["loras"][0]["strength"], 1.0)
+ self.assertEqual(rec["loras"][0]["name"], "mixed") # entry kept
+ self.assertEqual(rec["h3_lora_scale_v"], P.H3_LORA_SCALE_VERSION)
+ # What queue.js restores from the sidecar, through a real make_job.
+ replay = P.make_job({k: [v] for k, v in {
+ "mode": "t2v", "engine": "h3", "prompt": "a dancer", "h3_turbo": "0",
+ "loras": json.dumps([{"path": l["path"], "strength": l["strength"]}
+ for l in rec["loras"]])}.items()})
+ self.assertEqual(replay["params"]["loras"][0]["strength"], 1.0)
+ replay["id"] = "lora-rt2"
+ self.assertEqual(self._dispatch_argv(replay), 1.0)
+
+ def test_a_deliberate_strength_is_not_rewritten_in_the_recipe(self):
+ f = self.mixed()
+ job = {"id": "lora-keep", "params": {
+ "engine": "h3", "mode": "t2v", "h3_tier": "standard_5s",
+ "prompt": "a dancer", "seed": "7", "h3_turbo": False,
+ "loras": [{"path": str(f), "strength": 0.7}]}}
+ self.assertEqual(self._dispatch_argv(job), 0.7)
+ self.assertEqual(job["params"]["loras"][0]["strength"], 0.7)
+
+ def test_make_job_stamps_the_scale_version(self):
+ j = P.make_job({k: [v] for k, v in {
+ "mode": "t2v", "engine": "h3", "prompt": "a dancer", "h3_turbo": "0"}.items()})
+ self.assertEqual(j["params"].get("h3_lora_scale_v"), P.H3_LORA_SCALE_VERSION)
+ ltx = P.make_job({k: [v] for k, v in {
+ "mode": "t2v", "engine": "ltx", "prompt": "a dancer"}.items()})
+ self.assertNotIn("h3_lora_scale_v", ltx["params"])
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/test_h3_stop_and_companion.py b/test_h3_stop_and_companion.py
new file mode 100644
index 0000000..be185ec
--- /dev/null
+++ b/test_h3_stop_and_companion.py
@@ -0,0 +1,542 @@
+#!/usr/bin/env python3
+"""Stop means stop, and the Turbo companion is fetched once (ship review 2026-09-17).
+
+P1: an H3 Turbo job whose adapter lacks its adaLN companion fetched it with a
+plain subprocess, outside anything Stop knew about, and then spawned the
+renderer without looking at `cancel_requested`. A Stop pressed during the
+fetch was followed by a full GPU render. Registration of every tracked
+process group also raced Stop: the pgid was published, and the flag read,
+in separate moments.
+
+P4: a failed "once" fetch was retried by every Turbo job (up to 300 s each,
+holding the GPU gate), outside the installer's lock, on a shared .partial.
+
+No GPU, no network, no model: the renderer's Popen is replaced by a short
+`sleep` (or refused outright), and the fetch runner is a fake."""
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+import unittest
+import unittest.mock
+from contextlib import ExitStack
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent
+_TMP = Path(tempfile.mkdtemp(prefix="phos-h3-stop-"))
+for _k, _d in (("LTX_STATE_DIR", "state"), ("LTX_OUTPUT_DIR", "out"),
+ ("LTX_UPLOADS_DIR", "uploads")):
+ (_TMP / _d).mkdir(parents=True, exist_ok=True)
+ os.environ[_k] = str(_TMP / _d)
+os.environ["PHOSPHENE_ANALYTICS_DISABLED"] = "1"
+os.environ["PHOSPHENE_DISABLE_VERSION_CHECK"] = "1"
+os.environ.setdefault("LTX_PORT", "8298")
+sys.path.insert(0, str(ROOT))
+
+import mlx_ltx_panel as P # noqa: E402
+
+
+def _alive(pid: int) -> bool:
+ try:
+ os.kill(pid, 0)
+ except ProcessLookupError:
+ return False
+ # A zombie still answers kill(0); ask ps for its state.
+ out = subprocess.run(["ps", "-o", "stat=", "-p", str(pid)],
+ capture_output=True, text=True).stdout.strip()
+ return bool(out) and not out.startswith("Z")
+
+
+class _CurrentJob:
+ """Make `job` the worker's current job for the duration."""
+
+ def __init__(self, job):
+ self.job = job
+
+ def __enter__(self):
+ with P.LOCK:
+ self.saved = (P.STATE.get("current"), P.STATE.get("h3_pgid"),
+ P.STATE.get("mux_pgid"))
+ P.STATE["current"] = self.job
+ return self.job
+
+ def __exit__(self, *exc):
+ with P.LOCK:
+ P.STATE["current"], P.STATE["h3_pgid"], P.STATE["mux_pgid"] = self.saved
+ return False
+
+
+class TrackedProcessesObeyStop(unittest.TestCase):
+ def test_a_stopped_job_never_spawns(self):
+ job = {"id": "t-pre", "cancel_requested": True}
+ with _CurrentJob(job), unittest.mock.patch.object(
+ P.subprocess, "Popen", side_effect=AssertionError("spawned")):
+ with self.assertRaises(P.JobCancelled):
+ P.run_tracked_subprocess(["sleep", "30"], pgid_key="h3_pgid",
+ label="x", job=job)
+
+ def test_stop_between_spawn_and_registration_kills_the_group(self):
+ # The race the review named: the process exists, Stop has already
+ # read the (empty) pgid slot, and only then is the pgid published.
+ job = {"id": "t-race"}
+ spawned = []
+ real_popen = subprocess.Popen
+
+ def popen_then_stop(*a, **kw):
+ proc = real_popen(*a, **kw)
+ spawned.append(proc)
+ job["cancel_requested"] = True # Stop, right here
+ return proc
+
+ with _CurrentJob(job), unittest.mock.patch.object(
+ P.subprocess, "Popen", side_effect=popen_then_stop):
+ t0 = time.time()
+ with self.assertRaises(P.JobCancelled):
+ P.run_tracked_subprocess(["sleep", "30"], pgid_key="mux_pgid",
+ label="x", job=job)
+ self.assertLess(time.time() - t0, 5)
+ self.assertIsNotNone(spawned[0].poll())
+ self.assertFalse(_alive(spawned[0].pid))
+ self.assertIsNone(P.STATE.get("mux_pgid"))
+
+ def test_stop_current_job_ends_a_running_tracked_process(self):
+ job = {"id": "t-live"}
+ result = {}
+
+ def run():
+ try:
+ P.run_tracked_subprocess(["sleep", "30"], pgid_key="mux_pgid",
+ label="the fetch", job=job)
+ result["r"] = "finished"
+ except P.JobCancelled as e:
+ result["r"] = e
+
+ with _CurrentJob(job):
+ th = threading.Thread(target=run)
+ t0 = time.time()
+ th.start()
+ deadline = time.time() + 5
+ while P.STATE.get("mux_pgid") is None and time.time() < deadline:
+ time.sleep(0.02)
+ pgid = P.STATE.get("mux_pgid")
+ self.assertIsNotNone(pgid)
+ P.stop_current_job()
+ th.join(timeout=10)
+ self.assertFalse(th.is_alive())
+ self.assertIsInstance(result["r"], P.JobCancelled)
+ self.assertLess(time.time() - t0, 8)
+ self.assertFalse(_alive(pgid))
+
+ def test_a_finished_process_is_returned_like_subprocess_run(self):
+ r = P.run_tracked_subprocess([sys.executable, "-c",
+ "import sys; print('out'); print('err', file=sys.stderr); sys.exit(3)"],
+ pgid_key="mux_pgid", label="x", job={"id": "t-ok"})
+ self.assertEqual((r.returncode, r.stdout.strip(), r.stderr.strip()), (3, "out", "err"))
+
+ def test_timeout_kills_the_group_and_raises(self):
+ t0 = time.time()
+ with self.assertRaises(subprocess.TimeoutExpired):
+ P.run_tracked_subprocess(["sleep", "30"], pgid_key="mux_pgid",
+ label="x", job={"id": "t-to"}, timeout=0.5)
+ self.assertLess(time.time() - t0, 5)
+
+ def test_stop_sets_the_flag_before_it_reads_the_pgids(self):
+ src = (ROOT / "mlx_ltx_panel.py").read_text(encoding="utf-8")
+ body = src[src.index("def stop_current_job("):]
+ body = body[:body.index('push("Stop requested')]
+ self.assertLess(body.index('cur["cancel_requested"] = True'),
+ body.index('h3_pgid = STATE.get("h3_pgid")'))
+ self.assertIn("with LOCK:", body[:body.index('cur["cancel_requested"] = True')])
+
+ def test_work_outside_the_queue_ignores_a_stop_meant_for_the_render(self):
+ # Codex hotfix review #3: an editor proxy built on an HTTP thread
+ # while a stopped render unwinds must still run.
+ stopped = {"id": "t-render", "cancel_requested": True}
+ with _CurrentJob(stopped):
+ out, _err = P.run_postprocess_tracked(
+ [sys.executable, "-c", "print('proxy built')"], "Timeline proxy")
+ self.assertIn("proxy built", out)
+
+ def test_the_worker_threads_own_job_is_the_owner(self):
+ job = {"id": "t-owner", "cancel_requested": True}
+ P._JOB_CTX.job = job
+ try:
+ with unittest.mock.patch.object(P.subprocess, "Popen",
+ side_effect=AssertionError("spawned")):
+ with self.assertRaises(P.JobCancelled):
+ P.run_postprocess_tracked(["true"], "Mux")
+ finally:
+ P._JOB_CTX.job = None
+ # and another thread's view is unaffected
+ seen = []
+ th = threading.Thread(target=lambda: seen.append(P._thread_job()))
+ P._JOB_CTX.job = job
+ try:
+ th.start(); th.join()
+ finally:
+ P._JOB_CTX.job = None
+ self.assertEqual(seen, [None])
+ src = (ROOT / "mlx_ltx_panel.py").read_text(encoding="utf-8")
+ loop = src[src.index("with _GPU_LOCK:\n _JOB_CTX.job = job"):]
+ self.assertLess(loop.index("run_job_inner(job)"), loop.index("_JOB_CTX.job = None"))
+
+ def test_every_pgid_is_published_through_the_race_free_helper(self):
+ src = (ROOT / "mlx_ltx_panel.py").read_text(encoding="utf-8")
+ for key in ("h3_pgid", "train_pgid", "mux_pgid"):
+ self.assertNotIn(f'STATE["{key}"] = os.getpgid', src, key)
+
+
+def _h3_dispatch_patches(stack, fetch, popen):
+ tmp = _TMP / "pack"
+ paths = dict(missing=[], repairable=False, dit=tmp / "dit", python=sys.executable,
+ runner=tmp / "generate_staged.py", compact_root=tmp / "compact",
+ text_config=tmp / "text")
+ turbo = dict(files_ok=True, adaln_pairs=51, embedder=None, lora=tmp / "turbo.safetensors",
+ dir=tmp / "turbo", version="v4-600-EMA", fallback=False, missing=[])
+ for name, val in {
+ "h3_paths": lambda: paths, "h3_capable": lambda: True,
+ "h3_supports_lora": lambda: True, "h3_turbo_paths": lambda: dict(turbo),
+ "_h3_turbo_fetch_embedder": fetch, "h3_dit_choice": lambda: ("bf16", None),
+ "output_codec_settings": lambda: {"crf": "18", "pix_fmt": "yuv420p"},
+ "push": lambda msg: None, "_h3_runner_has_flag": lambda flag: False,
+ "h3_supports_lora_adaln": lambda: True}.items():
+ stack.enter_context(unittest.mock.patch.object(P, name, val))
+ for name in ("h3_supports_memory_limits", "h3_supports_prompt_cache",
+ "h3_live_preview_ready", "h3_supports_stage_a", "h3_supports_tae_draft"):
+ stack.enter_context(unittest.mock.patch.object(P, name, lambda: False))
+ stack.enter_context(unittest.mock.patch.object(P.HELPER, "is_alive", lambda: False))
+ stack.enter_context(unittest.mock.patch.object(P.subprocess, "Popen", popen))
+ ff = _TMP / "ffmpeg"
+ ff.write_text("")
+ stack.enter_context(unittest.mock.patch.object(P, "FFMPEG", ff))
+
+
+def _turbo_job(jid):
+ return {"id": jid, "params": {"engine": "h3", "mode": "t2v", "h3_tier": "standard_5s",
+ "prompt": "a lighthouse at dusk", "seed": "42",
+ "h3_turbo": True, "loras": []}}
+
+
+class TheTurboDispatchObeysStop(unittest.TestCase):
+ def test_stop_during_the_companion_fetch_never_reaches_the_renderer(self):
+ job = _turbo_job("t-fetch-stop")
+ fetches = []
+
+ def fetch(target_dir, push_log, runner=None, **kw):
+ fetches.append(runner)
+ job["cancel_requested"] = True # Stop, mid-fetch
+ return False
+
+ with ExitStack() as st, _CurrentJob(job):
+ _h3_dispatch_patches(st, fetch, unittest.mock.Mock(
+ side_effect=AssertionError("renderer spawned after Stop")))
+ with self.assertRaises(P.JobCancelled):
+ P.run_h3_job_inner(job)
+ self.assertEqual(len(fetches), 1)
+ self.assertIsNotNone(fetches[0], "the fetch must get the tracked runner")
+
+ def test_the_fetch_runs_as_a_tracked_group_stop_can_kill(self):
+ job = _turbo_job("t-fetch-kill")
+ seen = {}
+
+ def fetch(target_dir, push_log, runner=None, **kw):
+ def stop_soon():
+ deadline = time.time() + 5
+ while P.STATE.get("h3_pgid") is None and time.time() < deadline:
+ time.sleep(0.02)
+ seen["pgid"] = P.STATE.get("h3_pgid")
+ P.stop_current_job()
+ th = threading.Thread(target=stop_soon)
+ th.start()
+ try:
+ runner(["sleep", "30"], capture_output=True, text=True, env=None,
+ timeout=1800)
+ finally:
+ th.join()
+
+ real_popen = subprocess.Popen
+ spawned = []
+
+ def popen(cmd, **kw):
+ if cmd and cmd[0] == "caffeinate":
+ raise AssertionError("renderer spawned after Stop")
+ p = real_popen(cmd, **kw)
+ spawned.append(p)
+ return p
+
+ with ExitStack() as st, _CurrentJob(job):
+ _h3_dispatch_patches(st, fetch, popen)
+ t0 = time.time()
+ with self.assertRaises(P.JobCancelled):
+ P.run_h3_job_inner(job)
+ self.assertLess(time.time() - t0, 8)
+ self.assertEqual(seen["pgid"], spawned[0].pid)
+ self.assertFalse(_alive(spawned[0].pid))
+
+ def test_stop_racing_the_renderer_spawn_kills_it(self):
+ job = _turbo_job("t-spawn-race")
+ real_popen = subprocess.Popen
+ spawned = []
+
+ def popen(cmd, **kw):
+ self.assertEqual(cmd[0], "caffeinate")
+ kw.pop("cwd", None)
+ p = real_popen(["sleep", "30"], **{k: v for k, v in kw.items()
+ if k in ("stdout", "stderr", "text",
+ "bufsize", "start_new_session",
+ "env")})
+ spawned.append(p)
+ job["cancel_requested"] = True # Stop lands mid-spawn
+ return p
+
+ with ExitStack() as st, _CurrentJob(job):
+ _h3_dispatch_patches(st, lambda *a, **k: True, popen)
+ st.enter_context(unittest.mock.patch.object(
+ P, "h3_turbo_paths", lambda: dict(
+ files_ok=True, adaln_pairs=51, embedder=_TMP / "emb",
+ lora=_TMP / "turbo.safetensors", dir=_TMP / "turbo",
+ version="v4-600-EMA", fallback=False, missing=[])))
+ with self.assertRaises(P.JobCancelled):
+ P.run_h3_job_inner(job)
+ self.assertEqual(len(spawned), 1)
+ spawned[0].wait(timeout=5)
+ self.assertFalse(_alive(spawned[0].pid))
+ self.assertIsNone(P.STATE.get("h3_pgid"))
+
+ def test_the_x2_lane_checks_stop_before_the_helper(self):
+ src = (ROOT / "mlx_ltx_panel.py").read_text(encoding="utf-8")
+ i = src.index(' if mode == "upscale":')
+ body = src[i:src.index(' if mode == "ingredients":', i)]
+ self.assertIn('_raise_if_cancelled(job, "the ×2 render")', body)
+ self.assertLess(body.index('_raise_if_cancelled(job, "the ×2 render")'),
+ body.index("HELPER.run(job_spec)"))
+ self.assertIn('pgid_key="mux_pgid", label="the ×2 preparation"', body)
+ self.assertNotIn("subprocess.run(_upscale_hold_tail_cmd", body)
+
+ def test_the_helper_refuses_a_job_stopped_before_it_arrived(self):
+ job = {"id": "t-helper", "cancel_requested": True}
+ writes = []
+
+ class FakeStdin:
+ def write(self, s):
+ writes.append(s)
+
+ def flush(self):
+ pass
+
+ class FakeProc:
+ stdin = FakeStdin()
+
+ def poll(self):
+ return None
+
+ h = P.WarmHelper()
+ h.proc = FakeProc()
+ with _CurrentJob(job), unittest.mock.patch.object(h, "_ensure", lambda: None):
+ with self.assertRaises(P.JobCancelled):
+ h._run_once({"action": "generate", "id": "t-helper"})
+ self.assertEqual(writes, [])
+ h.proc = None
+
+
+class AOneShotPartBelongsToItsTake(unittest.TestCase):
+ """Codex hotfix review round 2: a take's parts are their own dicts
+ (`<take>-p1`) and Stop marks only the queued take."""
+
+ def setUp(self):
+ self.parent = {"id": "take-x", "params": {}}
+ P._JOB_CTX.job = self.parent
+ P._h3_turbo_embedder_failures.clear()
+
+ def tearDown(self):
+ P._JOB_CTX.job = None
+ P._h3_turbo_embedder_failures.clear()
+
+ def test_the_owner_cancels_its_children(self):
+ child = _turbo_job("take-x-p1")
+ self.assertFalse(P._cancel_requested(child))
+ self.parent["cancel_requested"] = True
+ self.assertTrue(P._cancel_requested(child))
+ self.assertFalse(child.get("cancel_requested"))
+ with self.assertRaises(P.JobCancelled):
+ P._raise_if_cancelled(child, "the part")
+
+ def test_the_helper_refuses_a_part_of_a_stopped_take(self):
+ self.parent["cancel_requested"] = True
+ writes = []
+
+ class FakeProc:
+ class stdin:
+ @staticmethod
+ def write(s):
+ writes.append(s)
+
+ @staticmethod
+ def flush():
+ pass
+
+ def poll(self):
+ return None
+
+ h = P.WarmHelper()
+ h.proc = FakeProc()
+ with _CurrentJob(self.parent), unittest.mock.patch.object(h, "_ensure", lambda: None):
+ with self.assertRaises(P.JobCancelled):
+ h._run_once({"action": "generate", "id": "take-x-p1"})
+ self.assertEqual(writes, [])
+ h.proc = None
+
+ def test_stop_during_a_parts_companion_fetch_ends_the_take_and_is_no_failure(self):
+ child = _turbo_job("take-x-p1")
+ root = _TMP / "pack-take"
+ (root / "scripts").mkdir(parents=True, exist_ok=True)
+ fetcher = root / "scripts" / "fetch_time_embedder.py"
+ fetcher.write_text("import time\ntime.sleep(30)\n")
+ turbo_dir = _TMP / "turbo-take"
+ real_fetch = P._h3_turbo_fetch_embedder
+ spawned = []
+ real_popen = subprocess.Popen
+
+ def popen(cmd, **kw):
+ if cmd and cmd[0] == "caffeinate":
+ raise AssertionError("renderer spawned after Stop")
+ proc = real_popen(cmd, **kw)
+ spawned.append(proc)
+ return proc
+
+ def stop_soon():
+ deadline = time.time() + 10
+ while P.STATE.get("h3_pgid") is None and time.time() < deadline:
+ time.sleep(0.02)
+ P.stop_current_job()
+
+ with ExitStack() as st, _CurrentJob(self.parent):
+ _h3_dispatch_patches(st, real_fetch, popen)
+ st.enter_context(unittest.mock.patch.object(P, "H3_ROOT", root))
+ st.enter_context(unittest.mock.patch.object(P, "_h3_python", lambda: Path(sys.executable)))
+ st.enter_context(unittest.mock.patch.object(
+ P, "h3_turbo_paths", lambda: dict(
+ files_ok=True, adaln_pairs=51, embedder=None, lora=turbo_dir / "l.safetensors",
+ dir=turbo_dir, version="v4-600-EMA", fallback=False, missing=[])))
+ th = threading.Thread(target=stop_soon)
+ th.start()
+ t0 = time.time()
+ with self.assertRaises(P.JobCancelled):
+ P.run_h3_job_inner(child)
+ th.join()
+ self.assertLess(time.time() - t0, 10)
+ self.assertTrue(self.parent.get("cancel_requested"))
+ self.assertEqual(len(spawned), 1)
+ self.assertFalse(_alive(spawned[0].pid))
+ self.assertEqual(P._h3_turbo_embedder_failures, {})
+ self.assertFalse(list(turbo_dir.glob("*.partial")) if turbo_dir.exists() else [])
+
+
+class TheCompanionIsFetchedOnce(unittest.TestCase):
+ def setUp(self):
+ self.dir = Path(tempfile.mkdtemp(prefix="phos-emb-"))
+ root = self.dir / "pack"
+ (root / "scripts").mkdir(parents=True)
+ (root / "scripts" / "fetch_time_embedder.py").write_text("# fixture")
+ self.st = ExitStack()
+ for name, val in {"H3_ROOT": root, "_h3_python": lambda: Path(sys.executable),
+ "H3_TURBO_EMBEDDER_MIN_BYTES": 1,
+ "h3_turbo_paths": lambda: {"adaln_pairs": 51}}.items():
+ self.st.enter_context(unittest.mock.patch.object(P, name, val))
+ P._h3_turbo_embedder_failures.clear()
+ self.logs = []
+ self.calls = []
+
+ def tearDown(self):
+ self.st.close()
+ P._h3_turbo_embedder_failures.clear()
+
+ def failing(self, cmd, **kw):
+ self.calls.append(cmd[cmd.index("--out") + 1])
+ Path(cmd[cmd.index("--out") + 1]).write_bytes(b"half")
+
+ class R:
+ returncode, stdout, stderr = 1, "", "network is unreachable"
+ return R()
+
+ def ok(self, cmd, **kw):
+ self.calls.append(cmd[cmd.index("--out") + 1])
+ Path(cmd[cmd.index("--out") + 1]).write_bytes(b"tensors")
+
+ class R:
+ returncode, stdout, stderr = 0, "", ""
+ return R()
+
+ def test_a_failure_is_not_retried_by_the_next_job(self):
+ f = P._h3_turbo_fetch_embedder
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.failing))
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.failing))
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.failing))
+ self.assertEqual(len(self.calls), 1)
+ self.assertTrue(any("not fetching" in m and "Install Turbo" in m for m in self.logs))
+ self.assertFalse(list(self.dir.glob("*.partial")))
+ state = P._h3_turbo_embedder_retry_state(self.dir)
+ self.assertIn("network is unreachable", state["error"])
+ # The cooldown ends by itself...
+ P._h3_turbo_embedder_failures[str(self.dir / P.H3_TURBO_EMBEDDER_FILE)]["retry_at"] = 0
+ self.assertIsNone(P._h3_turbo_embedder_retry_state(self.dir))
+ self.assertTrue(f(self.dir, self.logs.append, runner=self.ok))
+ self.assertEqual(len(self.calls), 2)
+ self.assertIsNone(P._h3_turbo_embedder_retry_state(self.dir))
+
+ def test_the_install_button_retries_at_once(self):
+ f = P._h3_turbo_fetch_embedder
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.failing))
+ self.assertTrue(f(self.dir, self.logs.append, runner=self.ok, force=True))
+ self.assertEqual(len(self.calls), 2)
+ self.assertTrue((self.dir / P.H3_TURBO_EMBEDDER_FILE).is_file())
+ src = (ROOT / "mlx_ltx_panel.py").read_text(encoding="utf-8")
+ install = src[src.index("def _h3_install_turbo("):]
+ install = install[:install.index("\ndef ")]
+ self.assertIn("_h3_turbo_fetch_embedder(target, push_log, force=True)", install)
+ bg = src[src.index("def _h3_turbo_download_bg("):src.index("def _h3_install_turbo(")]
+ self.assertEqual(bg.count("_h3_turbo_fetch_embedder(target_dir, push_log, force=True)"), 2)
+
+ def test_one_fetch_at_a_time_and_each_writes_its_own_file(self):
+ f = P._h3_turbo_fetch_embedder
+ with P._h3_turbo_embedder_lock:
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.ok, force=True))
+ self.assertEqual(self.calls, [])
+ self.assertTrue(any("already being fetched" in m for m in self.logs))
+ self.assertIsNone(P._h3_turbo_embedder_retry_state(self.dir)) # not a failure
+ self.assertFalse(f(self.dir, self.logs.append, runner=self.failing))
+ self.assertTrue(f(self.dir, self.logs.append, runner=self.ok, force=True))
+ self.assertEqual(len(set(self.calls)), 2)
+ for c in self.calls:
+ self.assertTrue(c.endswith(".partial"))
+ self.assertNotEqual(Path(c).name, P.H3_TURBO_EMBEDDER_FILE + ".partial")
+ self.assertFalse(list(self.dir.glob("*.partial")))
+
+ def test_stop_during_the_fetch_is_not_a_failed_download(self):
+ def stopped(cmd, **kw):
+ raise P.JobCancelled("Stopped during the fetch.")
+ with self.assertRaises(P.JobCancelled):
+ P._h3_turbo_fetch_embedder(self.dir, self.logs.append, runner=stopped)
+ self.assertIsNone(P._h3_turbo_embedder_retry_state(self.dir))
+ self.assertFalse(P._h3_turbo_embedder_lock.locked())
+
+ def test_status_names_a_remembered_failure(self):
+ self.assertIsNone(P._h3_turbo_embedder_retry_state(self.dir))
+ P._h3_turbo_fetch_embedder(self.dir, self.logs.append, runner=self.failing)
+ with unittest.mock.patch.object(P, "h3_turbo_paths", lambda: {
+ "adaln_pairs": 51, "dir": self.dir, "files_ok": True, "lora": None,
+ "version": "v4-600-EMA", "fallback": False, "missing": [],
+ "embedder": None}), \
+ unittest.mock.patch.object(P, "h3_supports_lora", lambda: True), \
+ unittest.mock.patch.object(P, "_h3_retune_turbo_estimates", lambda: None), \
+ unittest.mock.patch.object(P, "h3_turbo_note", lambda paths: ""):
+ status = P.h3_turbo_status()
+ self.assertIn("network is unreachable", status["adaln_fetch"]["error"])
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/test_upscale_x2_tail.py b/test_upscale_x2_tail.py
index 4632678..c02b519 100644
--- a/test_upscale_x2_tail.py
+++ b/test_upscale_x2_tail.py
@@ -48,7 +48,8 @@ except Exception: # noqa: BLE001
def _helper_ns() -> dict:
tree = ast.parse(HELPER_SRC)
names = {"_vae_decode_peak_elements", "_vae_decode_max_latent_frames",
- "_int32_safe_decode_tiling"}
+ "_int32_safe_decode_tiling", "_decode_tiling_peak_elements",
+ "_decode_temporal_config"}
picked = [n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name in names]
assert {n.name for n in picked} == names
consts = [n for n in tree.body if isinstance(n, ast.Assign)
@@ -183,6 +184,207 @@ class TheGuardIsWired(unittest.TestCase):
self.assertIn("_compute_decode_tiling(", src)
+def _pure_tiling():
+ """The vendored tiler's config classes and split functions, exec'd from
+ source without its MLX import — so the tile-extent checks below run on any
+ machine, against the exact interval code the decoder uses."""
+ import dataclasses
+ import types
+ src_dir = None
+ for base in sys.path + [str(ROOT / "ltx-2-mlx" / "env" / "lib" / "python3.11" / "site-packages")]:
+ cand = Path(base) / "ltx_core_mlx" / "model" / "video_vae"
+ if (cand / "tiling.py").is_file():
+ src_dir = cand
+ break
+ if src_dir is None:
+ return None, None
+ names = {"SpatialTilingConfig", "TemporalTilingConfig", "TilingConfig",
+ "DimensionIntervals", "default_split_operation",
+ "split_with_symmetric_overlaps", "split_temporal_latents"}
+ tree = ast.parse((src_dir / "tiling.py").read_text(encoding="utf-8"))
+ body = [ast.ImportFrom(module="__future__", names=[ast.alias(name="annotations")], level=0)]
+ body += [n for n in tree.body if isinstance(n, (ast.ClassDef, ast.FunctionDef))
+ and n.name in names]
+ ns = {"dataclass": dataclasses.dataclass, "replace": dataclasses.replace,
+ "field": dataclasses.field}
+ exec(compile(ast.fix_missing_locations(ast.Module(body=body, type_ignores=[])),
+ "tiling", "exec"), ns)
+ ns["DEFAULT_SPLIT_OPERATION"] = ns["default_split_operation"]
+ orig = [n for n in ast.parse((src_dir / "video_vae.py").read_text(encoding="utf-8")).body
+ if isinstance(n, ast.FunctionDef) and n.name == "_compute_decode_tiling"]
+ ons = {"os": os, "TilingConfig": ns["TilingConfig"],
+ "TemporalTilingConfig": ns["TemporalTilingConfig"]}
+ exec(compile(ast.Module(body=orig, type_ignores=[]), "orig", "exec"), ons)
+ return types.SimpleNamespace(**ns), ons["_compute_decode_tiling"]
+
+
+# (width, height, latent frames): the review's matrix plus the canvases the
+# 4.13.1 temporal-only guard got wrong (2 latent frames + the causal third).
+GUARD_CASES = [(2048, 1152, 17), (2688, 1536, 17), (4096, 2304, 17),
+ (4096, 4096, 4), (4096, 4096, 17), (4096, 4096, 31),
+ (4480, 4480, 17), (6144, 3456, 17), (1280, 768, 16),
+ (2048, 1152, 16), (4096, 4096, 1), (4096, 4096, 2), (4096, 4096, 3)]
+
+
+class TheGuardChecksTheTilesItReturns(unittest.TestCase):
+ """Ship review 2026-09-17, P2: `size = max(2, max_lat - 1)` returned
+ [2, 3]-latent tiles for a 4096×4096 decode — 2.56e9 elements, past the cap
+ the guard exists to keep."""
+
+ def setUp(self):
+ self.T, self.orig = _pure_tiling()
+ if self.T is None:
+ self.skipTest("vendored ltx_core_mlx tiling.py not found")
+ os.environ.pop("LTX2_VAE_DECODE_BUDGET_GB", None)
+ self.ns = _helper_ns()
+
+ def guard(self, shape):
+ return self.ns["_int32_safe_decode_tiling"](self.orig, shape, 24.0, T=self.T)
+
+ def extents(self, cfg, shape):
+ """Tile lengths per axis, straight from the tiler's split functions,
+ the way prepare_tiles_for_decoding derives them."""
+ T = self.T
+ _, _, f, h, w = shape
+ out = {"t": [(0, f)], "h": [(0, h)], "w": [(0, w)]}
+ sc = cfg.spatial_config
+ if sc is not None:
+ long_side, size, ov = max(h, w), sc.tile_size_in_pixels // 32, sc.tile_overlap_in_pixels // 32
+ for axis, n in (("h", h), ("w", w)):
+ iv = T.split_with_symmetric_overlaps(
+ max(max(2, ov + 1), round(size * n / long_side)), ov)(n)
+ out[axis] = list(zip(iv.starts, iv.ends))
+ tc = cfg.temporal_config
+ iv = T.split_temporal_latents(tc.tile_size_in_frames // 8, tc.tile_overlap_in_frames // 8)(f)
+ out["t"] = list(zip(iv.starts, iv.ends))
+ return out
+
+ def test_the_4_13_1_answer_overflowed_on_a_square_4k_decode(self):
+ shape = (1, 128, 4, 128, 128)
+ max_lat = self.ns["_vae_decode_max_latent_frames"](128, 128)
+ self.assertEqual(max_lat, 2)
+ old = self.T.TilingConfig(temporal_config=self.T.TemporalTilingConfig(16, 0))
+ lengths = [e - s for s, e in self.extents(old, shape)["t"]]
+ self.assertEqual(lengths, [2, 3])
+ self.assertGreater(self.ns["_vae_decode_peak_elements"](3, 128, 128), INT32)
+ self.assertGreater(self.ns["_decode_tiling_peak_elements"](old, shape, self.T), INT32)
+
+ def test_every_returned_tiling_is_under_the_cap_on_every_tile(self):
+ cap = self.ns["_DECODE_MAX_ELEMENTS"]
+ peak = self.ns["_vae_decode_peak_elements"]
+ for w, h, f in GUARD_CASES:
+ shape = (1, 128, f, h // 32, w // 32)
+ cfg = self.guard(shape)
+ if cfg is None:
+ self.assertLessEqual(peak(f, h // 32, w // 32), cap, (w, h, f))
+ continue
+ ext = self.extents(cfg, shape)
+ for ts, te in ext["t"]:
+ out_frames = 1 + (te - 1) * 8 - ts * 8
+ self.assertLessEqual(3 * out_frames * h * w, cap, (w, h, f)) # the blend buffer
+ for hs, he in ext["h"]:
+ for ws, we in ext["w"]:
+ self.assertLessEqual(peak(te - ts, he - hs, we - ws), cap,
+ (w, h, f, (ts, te), (hs, he), (ws, we)))
+ # The whole clip and the whole canvas are covered.
+ self.assertEqual(ext["t"][0][0], 0)
+ self.assertEqual(ext["t"][-1][1], f)
+ for axis, n in (("h", h // 32), ("w", w // 32)):
+ covered = set()
+ for a, b in ext[axis]:
+ covered.update(range(a, b))
+ self.assertEqual(covered, set(range(n)), (w, h, f, axis))
+
+ def test_the_canvases_4_13_1_got_right_keep_their_tiles(self):
+ # Same answers as the shipped guard where its tiles were blended.
+ for (w, h, f), frames in [((2048, 1152, 17), 96), ((2688, 1536, 17), 48)]:
+ cfg = self.guard((1, 128, f, h // 32, w // 32))
+ self.assertIsNone(cfg.spatial_config)
+ self.assertEqual(cfg.temporal_config.tile_size_in_frames, frames)
+
+ def test_unblended_temporal_tiles_give_way_to_blended_spatial_ones(self):
+ # 4096×2304: 4.13.1 cut 16-frame temporal tiles with no blend at all.
+ cfg = self.guard((1, 128, 17, 72, 128))
+ self.assertIsNotNone(cfg.spatial_config)
+ self.assertGreaterEqual(cfg.temporal_config.tile_size_in_frames, 32)
+ self.assertGreater(cfg.temporal_config.tile_overlap_in_frames, 0)
+ self.assertGreater(cfg.spatial_config.tile_overlap_in_pixels, 0)
+
+ def test_only_a_canvas_temporal_tiles_cannot_fit_is_tiled_in_space(self):
+ cfg = self.guard((1, 128, 17, 128, 128))
+ self.assertIsNotNone(cfg.spatial_config)
+ self.assertLess(cfg.spatial_config.tile_size_in_pixels, 4096)
+ self.assertGreaterEqual(cfg.temporal_config.tile_size_in_frames, 32) # blended seams
+
+ def test_a_canvas_no_tiling_can_fit_is_refused(self):
+ with self.assertRaises(RuntimeError) as ctx:
+ self.guard((1, 128, 17, 256, 256)) # 8192×8192
+ self.assertIn("too large", str(ctx.exception))
+ # The panel refuses the same canvas BEFORE the render, and admits the
+ # ones the helper can tile.
+ self.assertFalse(panel.upscale_canvas_decodable(8192, 8192, 129))
+ for w, h, f in GUARD_CASES:
+ self.assertTrue(panel.upscale_canvas_decodable(w, h, 8 * f - 7), (w, h, f))
+ for w, h in [(6144, 6144), (8192, 4608), (7680, 4320)]:
+ refused = not panel.upscale_canvas_decodable(w, h, 129)
+ try:
+ self.guard((1, 128, 17, h // 32, w // 32))
+ helper_refused = False
+ except RuntimeError:
+ helper_refused = True
+ if refused:
+ self.assertTrue(helper_refused, (w, h)) # never admit what the helper refuses
+ if not refused:
+ self.assertFalse(helper_refused, (w, h))
+
+ def test_admission_and_the_helper_agree_on_every_length(self):
+ # Codex hotfix review #4: a 17-frame clip's shortest groups are 9
+ # frames, not 17 — 8192×8192 × 17 tiles fine and must be admitted.
+ self.assertEqual(panel._decode_min_group_frames(17), 9)
+ self.assertEqual(panel._decode_min_group_frames(129), 17)
+ self.assertEqual(panel._decode_min_group_frames(9), 9)
+ self.assertEqual(panel._decode_min_group_frames(1), 1)
+ self.assertTrue(panel.upscale_canvas_decodable(8192, 8192, 17))
+ for w, h in [(8192, 8192), (6144, 6144), (8192, 4608), (9600, 9600),
+ (12800, 7200), (16384, 16384)]:
+ for frames in (1, 9, 17, 25, 33, 41, 121, 129, 241):
+ f = (frames - 1) // 8 + 1
+ try:
+ self.guard((1, 128, f, h // 32, w // 32))
+ helper_ok = True
+ except RuntimeError:
+ helper_ok = False
+ self.assertEqual(panel.upscale_canvas_decodable(w, h, frames), helper_ok,
+ (w, h, frames))
+
+ @unittest.skipUnless(HAVE_LTX, "ltx_core_mlx not importable (run with the repo venv)")
+ def test_the_real_tiler_cuts_what_the_check_measured(self):
+ cap = self.ns["_DECODE_MAX_ELEMENTS"]
+ peak = self.ns["_vae_decode_peak_elements"]
+ from ltx_core_mlx.model.video_vae import tiling as RT
+ for w, h, f in GUARD_CASES:
+ shape = (1, 128, f, h // 32, w // 32)
+ cfg = self.ns["_int32_safe_decode_tiling"](_vv._compute_decode_tiling, shape, 24.0)
+ if cfg is None:
+ continue
+ real = RT.TilingConfig(
+ spatial_config=(RT.SpatialTilingConfig(cfg.spatial_config.tile_size_in_pixels,
+ cfg.spatial_config.tile_overlap_in_pixels)
+ if cfg.spatial_config else None),
+ temporal_config=RT.TemporalTilingConfig(cfg.temporal_config.tile_size_in_frames,
+ cfg.temporal_config.tile_overlap_in_frames))
+ worst = 0
+ for t in prepare_tiles_for_decoding(shape, real):
+ n = [t.in_coords[i].stop - t.in_coords[i].start for i in (2, 3, 4)]
+ worst = max(worst, peak(*n))
+ o = t.out_coords[2]
+ self.assertIsInstance(o.stop, int) # tiled_decode needs a bounded slice
+ self.assertLessEqual(3 * (o.stop - o.start) * h * w, cap)
+ self.assertLessEqual(worst, cap, (w, h, f))
+ # The pure check never under-counts what the real tiler cuts.
+ self.assertLessEqual(worst, self.ns["_decode_tiling_peak_elements"](real, shape, RT))
+
+
import mlx_ltx_panel as panel # noqa: E402
@@ -322,5 +524,133 @@ class TheClipGoesInAndComesOutTheSameLength(unittest.TestCase):
+def _tiny(path, frames, tone):
+ cmd = [str(panel.FFMPEG), "-y", "-v", "error", "-f", "lavfi", "-i", "testsrc2=size=32x32:rate=24"]
+ if tone:
+ cmd += ["-f", "lavfi", "-i", "sine=frequency=440:sample_rate=48000",
+ "-map", "0:v", "-map", "1:a", "-af", f"atrim=end={frames / 24 + 0.3:.4f}",
+ "-c:a", "aac"]
+ cmd += ["-frames:v", str(frames), "-c:v", "libx264", "-threads", "1",
+ "-pix_fmt", "yuv420p", str(path)]
+ subprocess.run(cmd, check=True, capture_output=True)
+ return path
+
+
[email protected](Path(str(panel.FFMPEG)).is_file(), "ffmpeg not installed")
+class TheSoundIsTheSourcesOrNone(unittest.TestCase):
+ """Ship review 2026-09-17, P2: the ×2 model writes a soundtrack of its own.
+ A SILENT source kept it when no trim was needed (9/17/73/121/129 frames)
+ and lost it when one was (124) — the result's sound depended on the
+ frame count. And a failed probe read as "silent"."""
+
+ def setUp(self):
+ self.tmp = Path(tempfile.mkdtemp(prefix="x2sound_"))
+ self.codec = unittest.mock.patch.object(
+ panel, "output_codec_settings",
+ return_value={"pix_fmt": "yuv420p", "crf": "18", "preset": "medium"})
+ self.codec.start()
+
+ def tearDown(self):
+ import shutil
+ self.codec.stop()
+ shutil.rmtree(self.tmp, ignore_errors=True)
+
+ def test_a_silent_source_comes_back_silent_at_every_length(self):
+ for n in (1, 8, 9, 10, 17, 72, 73, 121, 124, 129, 130, 243):
+ src = _tiny(self.tmp / f"src{n}.mp4", n, tone=False)
+ deliver, grid = panel.upscale_frame_plan(n, 121)
+ model = _tiny(self.tmp / f"model{n}.mp4", grid, tone=True) # the model's own sound
+ self.assertTrue(panel._video_has_audio(str(model)))
+ kept = panel._upscale_finish(model, str(src), frames=deliver, fps=24.0,
+ trim=grid != deliver)
+ self.assertFalse(kept, n)
+ self.assertFalse(panel._video_has_audio(str(model)), n)
+ self.assertEqual(panel._probe_video_frames(str(model)), deliver, n)
+ self.assertFalse(list(self.tmp.glob("*.mux.mp4")))
+
+ def test_a_source_with_sound_keeps_it_at_every_length(self):
+ for n in (9, 121, 124, 129):
+ src = _tiny(self.tmp / f"s{n}.mp4", n, tone=True)
+ deliver, grid = panel.upscale_frame_plan(n, 121)
+ model = _tiny(self.tmp / f"m{n}.mp4", grid, tone=False)
+ self.assertTrue(panel._upscale_finish(model, str(src), frames=deliver, fps=24.0,
+ trim=grid != deliver), n)
+ self.assertTrue(panel._video_has_audio(str(model)), n)
+ self.assertEqual(panel._probe_video_frames(str(model)), deliver, n)
+
+ def test_a_failed_probe_does_not_throw_the_sound_away(self):
+ src = _tiny(self.tmp / "s.mp4", 121, tone=True)
+ model = _tiny(self.tmp / "m.mp4", 121, tone=True)
+ real = panel._probe_audio_state
+ with unittest.mock.patch.object(
+ panel, "_probe_audio_state",
+ side_effect=lambda p: "unknown" if p == str(src) else real(p)):
+ self.assertTrue(panel._upscale_finish(model, str(src), frames=121, fps=24.0,
+ trim=False))
+ self.assertTrue(panel._video_has_audio(str(model)))
+ # ...and a silent source behind a failed probe still ends silent.
+ silent = _tiny(self.tmp / "q.mp4", 121, tone=False)
+ model2 = _tiny(self.tmp / "m2.mp4", 121, tone=True)
+ with unittest.mock.patch.object(
+ panel, "_probe_audio_state",
+ side_effect=lambda p: "unknown" if p == str(silent) else real(p)):
+ self.assertFalse(panel._upscale_finish(model2, str(silent), frames=121,
+ fps=24.0, trim=False))
+ self.assertFalse(panel._video_has_audio(str(model2)))
+
+ def test_the_probe_tells_none_from_unknown(self):
+ self.assertEqual(panel._probe_audio_state(str(_tiny(self.tmp / "a.mp4", 9, True))), "audio")
+ self.assertEqual(panel._probe_audio_state(str(_tiny(self.tmp / "b.mp4", 9, False))), "none")
+ self.assertEqual(panel._probe_audio_state(str(self.tmp / "missing.mp4")), "unknown")
+ junk = self.tmp / "junk.mp4"
+ junk.write_bytes(b"not a video")
+ self.assertEqual(panel._probe_audio_state(str(junk)), "unknown")
+
+ def test_an_unmuxable_source_still_drops_the_models_sound(self):
+ src = _tiny(self.tmp / "s.mp4", 121, tone=True)
+ model = _tiny(self.tmp / "m.mp4", 121, tone=True)
+ broken = self.tmp / "broken.mp4"
+ broken.write_bytes(b"x")
+ with unittest.mock.patch.object(panel, "_probe_audio_state",
+ side_effect=lambda p: "audio" if p == str(broken)
+ else "none"):
+ self.assertFalse(panel._upscale_finish(model, str(broken), frames=121,
+ fps=24.0, trim=False))
+ self.assertFalse(panel._video_has_audio(str(model)))
+ self.assertEqual(panel._probe_video_frames(str(model)), 121)
+
+ def test_a_stopped_finish_leaves_no_temp_file_for_the_gallery(self):
+ # Codex hotfix review #2: ffmpeg had opened <out>.mux.mp4 when Stop hit.
+ src = _tiny(self.tmp / "s.mp4", 9, tone=False)
+ model = _tiny(self.tmp / "m.mp4", 9, tone=True)
+
+ def stopped_mid_write(cmd, **kw):
+ Path(cmd[-1]).write_bytes(b"half an mp4")
+ raise panel.JobCancelled("Stopped during the ×2 finish.")
+
+ with unittest.mock.patch.object(panel, "run_tracked_subprocess", stopped_mid_write):
+ with self.assertRaises(panel.JobCancelled):
+ panel._upscale_finish(model, str(src), frames=9, fps=24.0, trim=False)
+ self.assertEqual(sorted(p.name for p in self.tmp.iterdir()), ["m.mp4", "s.mp4"])
+
+ def boom(cmd, **kw):
+ Path(cmd[-1]).write_bytes(b"x")
+ raise OSError("spawn failed")
+
+ with unittest.mock.patch.object(panel, "run_tracked_subprocess", boom):
+ with self.assertRaises(OSError):
+ panel._upscale_finish(model, str(src), frames=9, fps=24.0, trim=False)
+ self.assertFalse(list(self.tmp.glob("*.mux.mp4")))
+
+ def test_stop_ends_the_finish(self):
+ src = _tiny(self.tmp / "s.mp4", 9, tone=False)
+ model = _tiny(self.tmp / "m.mp4", 9, tone=True)
+ before = model.read_bytes()
+ with self.assertRaises(panel.JobCancelled):
+ panel._upscale_finish(model, str(src), frames=9, fps=24.0, trim=False,
+ job={"id": "x", "cancel_requested": True})
+ self.assertEqual(model.read_bytes(), before)
+
+
if __name__ == "__main__":
unittest.main()