Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Studio: MCP agents follow long work in one call and see every model a load unloads (#14089)
Etherle2a214bc08ea
diff --git a/studio/MCP.md b/studio/MCP.mdindex 9b3b5a62189..0393274293e 100644--- a/studio/MCP.md+++ b/studio/MCP.md@@ -276,7 +276,7 @@ api_key_format = "Bearer {token}"|---|---|| `studio_status` | What is loaded, loading and running: chat, image, video, speech-to-text and embedding models, training, export and GPUs. || `list_models` | Models Unsloth Studio can serve, by kind or by Audio page `workflow`. With `model`, its training defaults. With `source="catalog"`, the image, video and audio models it can download, with their size and whether they fit on the GPU now. |-| `load_model` | Load a chat, image, video or speech-to-text model, downloading it first if needed. Reports progress. |+| `load_model` | Load a chat, image, video or speech-to-text model, downloading it first if needed. Reports progress, waits up to `wait_s` (240 s by default) and lists what it unloaded. || `unload_model` | Unload a model. || `chat` | Ask the loaded model, with optional images, thinking controls and a `response_format`. Reports progress and returns a `cancel_id`. || `embed` | Embed up to 2048 texts, or rank them against a `query` and return the `top_k` best matches. |@@ -286,7 +286,7 @@ api_key_format = "Bearer {token}"| `transcribe` | Speech to text, or translation to English; `speakers` labels who speaks, and `expected_text` adds a match score. || `analyze_audio` | A clip's duration, loudness, true peak, clipping and speech span, plus a match score against `expected_text`. || `generate_video` | Start a video, with a seed, frame count, last frame and image or audio references. One renders at a time. Poll it with `get_job`. |-| `get_job` | The state of a video, recipe or export job. |+| `get_job` | The state of a video, recipe, export, training or load job. With `wait_s` it waits for the job to finish or move on. || `list_media` | Earlier images, audio clips, videos, transcripts and saved voices, a page at a time. || `voices` | List, save, rename and delete saved voices. || `run_recipe` | Validate, preview or run a Data Recipe. |@@ -296,10 +296,12 @@ api_key_format = "Bearer {token}"| `export_model` | Export a checkpoint to GGUF, merged weights, a LoRA adapter or the base model. || `cancel` | Stop running work of a given kind; the kinds are listed in the tool's schema. |-Long work returns at once. Start it, then poll `get_job` or `studio_status`,-and stop it with `cancel`. Audio runs cannot be cancelled. Unsloth Studio runs one-export at a time, so `export_model` is refused while another export runs, and-cancelling an export job stops only that job's own step.+Long work keeps running in Unsloth Studio when a tool call times out.+Follow every long job with `get_job(kind=…, id=…, wait_s=…)`: it checks every 2 s and returns once the job finishes, its status changes (for an export or a training run, also its phase), or `wait_s` runs out.+`wait_s` is at most 240; 50 stays under Codex's 60 s default tool timeout, and longer waits need a longer client timeout.+`load_model` waits up to `wait_s` too (240 s when unset), then returns `status: "loading"` and a `job_id` for `get_job(kind="load")`.+Stop work with `cancel`.+Unsloth Studio runs one export at a time, so `export_model` is refused while another export runs, and cancelling an export job stops only that job's own step.### Media and filesdiff --git a/studio/backend/studio_mcp/export_jobs.py b/studio/backend/studio_mcp/export_jobs.pyindex b49a2a8a58b..d467fcd552d 100644--- a/studio/backend/studio_mcp/export_jobs.py+++ b/studio/backend/studio_mcp/export_jobs.py@@ -1,60 +1,38 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Export jobs started over MCP. Loading a checkpoint and exporting it can take many minutes, longer than an agent waits on one tool call, so export_model runs them as a background task and get_job reports on it. Jobs are keyed by account, and an id from another account reads as unknown."""+"""Export jobs started over MCP. Loading a checkpoint and exporting it can take many minutes, longer than an agent waits on one tool call, so export_model runs them as a background task in the job store and get_job reports on it. Jobs are keyed by account, and an id from another account reads as unknown."""from __future__ import annotations-import asyncioimport re-import secrets-import time-from collections import OrderedDict-from dataclasses import dataclass, field+from dataclasses import dataclassfrom pathlib import Pathfrom typing import Any, Awaitable, Callable, Optional-from fastmcp.exceptions import ToolError--from hub.utils.host_paths import redact_paths_in_text+from studio_mcp import job_storefrom studio_mcp.tools import leaf_nameCAPACITY = 64@dataclass-class ExportJob:- job_id: str- account_id: str- format: str- status: str = "running"- phase: str = "starting"+class ExportJob(job_store.Job):+ kind: str = "export"+ format: str = ""# The op id sent with the step in flight; cancel stops the worker only while that op runs.op_id: Optional[str] = Noneoutput: Optional[str] = None- error: Optional[str] = None- finished_at: Optional[float] = None- # Held here so the task outlives the tool call that started it.- task: Optional[asyncio.Task] = field(default = None, repr = False)-- @property- def finished(self) -> bool:- return self.status != "running"---_jobs: "OrderedDict[str, ExportJob]" = OrderedDict()-def _key(account_id: str, job_id: str) -> str:- return f"{account_id}:{job_id}"+STORE = job_store.JobStore(+ capacity = CAPACITY, unknown = "No such export job", failure = "The export stopped unexpectedly."+)+_jobs = STORE.jobsdef _evict() -> None:- while len(_jobs) > CAPACITY:- oldest = next((key for key, job in _jobs.items() if job.finished), None)- if oldest is None:- return- _jobs.pop(oldest)+ STORE.evict()def output_name(path: Any) -> Optional[str]:@@ -70,69 +48,23 @@ def output_name(path: Any) -> Optional[str]:return f"{leaf_name(path)} (outside exports folder)"-def finish(- job: ExportJob,- status: str,- *,- error: Optional[str] = None,-) -> None:- if job.finished:- return- job.status = status- job.phase = "done"- job.error = redact_paths_in_text(error) if error else job.error- job.finished_at = time.monotonic()---async def settle(job: ExportJob, timeout: float) -> None:- """Wait up to ``timeout`` seconds for the job's task to finish on its own, without cancelling it."""- if job.task is not None and not job.task.done():- await asyncio.wait({job.task}, timeout = timeout)---async def _drive(job: ExportJob, run: Callable[[ExportJob], Awaitable[None]]) -> None:- try:- await run(job)- except asyncio.CancelledError:- finish(job, "cancelled")- raise- except ToolError as exc:- finish(job, "failed", error = str(exc))- except Exception:- finish(job, "failed", error = "The export stopped unexpectedly.")- else:- finish(job, "completed")--def start(account_id: str, format: str, run: Callable[[ExportJob], Awaitable[None]]) -> ExportJob:- job = ExportJob(job_id = secrets.token_urlsafe(12), account_id = account_id, format = format)- _jobs[_key(account_id, job.job_id)] = job- _evict()- job.task = asyncio.get_running_loop().create_task(_drive(job, run))- return job+ job = ExportJob(job_id = job_store.new_id(), account_id = account_id, format = format)+ return STORE.start(job, run)def lookup(account_id: str, job_id: str) -> ExportJob:- job = _jobs.get(_key(account_id, job_id))- if job is None:- raise ToolError("No such export job")- return job+ return STORE.lookup(account_id, job_id)def any_running() -> bool:- return any(not job.finished for job in _jobs.values())+ return bool(STORE.running())def jobs_of(account_id: str) -> list[ExportJob]:- return [job for job in _jobs.values() if job.account_id == account_id]---def mark_cancelled(job: ExportJob) -> None:- if job.task is not None and not job.task.done():- job.task.cancel()- finish(job, "cancelled")+ return STORE.jobs_of(account_id)def _reset() -> None:"""Test hook."""- _jobs.clear()+ STORE.reset()diff --git a/studio/backend/studio_mcp/job_store.py b/studio/backend/studio_mcp/job_store.pynew file mode 100644index 00000000000..abc915b5fdc--- /dev/null+++ b/studio/backend/studio_mcp/job_store.py@@ -0,0 +1,125 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0++"""Background jobs started over MCP. Some work (an export, a model load) outlasts an agent's tool call, so it runs as a task held here and get_job reports on it; a client that gives up on the call never stops the work. Jobs are keyed by account, and an id from another account reads as unknown."""++from __future__ import annotations++import asyncio+import secrets+import time+from collections import OrderedDict+from dataclasses import dataclass, field+from typing import Any, Awaitable, Callable, Optional++from fastmcp.exceptions import ToolError++from hub.utils.host_paths import redact_paths_in_text+++@dataclass+class Job:+ job_id: str+ account_id: str+ kind: str = ""+ status: str = "running"+ phase: str = "starting"+ error: Optional[str] = None+ result: Any = None+ finished_at: Optional[float] = None+ # Held here so the task outlives the tool call that started it.+ task: Optional[asyncio.Task] = field(default = None, repr = False)++ @property+ def finished(self) -> bool:+ return self.status != "running"+++def new_id() -> str:+ return secrets.token_urlsafe(12)+++class JobStore:+ """Jobs of one kind, oldest first. Past ``capacity`` the oldest finished job is dropped; a running one never is."""++ def __init__(self, *, capacity: int, unknown: str, failure: str) -> None:+ self.capacity, self.unknown, self.failure = capacity, unknown, failure+ self.jobs: "OrderedDict[str, Job]" = OrderedDict()++ def add(self, job: Job) -> Job:+ self.jobs[f"{job.account_id}:{job.job_id}"] = job+ self.evict()+ return job++ def evict(self) -> None:+ while len(self.jobs) > self.capacity:+ oldest = next((key for key, job in self.jobs.items() if job.finished), None)+ if oldest is None:+ return+ self.jobs.pop(oldest)++ def start(self, job: Job, run: Callable[[Job], Awaitable[Any]]) -> Job:+ """Run ``run(job)`` as a task the store holds; its return value becomes ``job.result``."""+ self.add(job)+ job.task = asyncio.get_running_loop().create_task(self._drive(job, run))+ return job++ async def _drive(self, job: Job, run: Callable[[Job], Awaitable[Any]]) -> None:+ try:+ result = await run(job)+ except asyncio.CancelledError:+ finish(job, "cancelled")+ raise+ except ToolError as exc:+ finish(job, "failed", error = str(exc))+ except Exception:+ finish(job, "failed", error = self.failure)+ else:+ job.result = result if result is not None else job.result+ finish(job, "completed")++ def lookup(self, account_id: str, job_id: str) -> Job:+ job = self.jobs.get(f"{account_id}:{job_id}")+ if job is None:+ raise ToolError(self.unknown)+ return job++ def jobs_of(self, account_id: str) -> list[Job]:+ return [job for job in self.jobs.values() if job.account_id == account_id]++ def running(self, account_id: Optional[str] = None) -> list[Job]:+ return [+ job+ for job in self.jobs.values()+ if not job.finished and (account_id is None or job.account_id == account_id)+ ]++ def reset(self) -> None:+ """Test hook."""+ self.jobs.clear()+++def finish(+ job: Job,+ status: str,+ *,+ error: Optional[str] = None,+) -> None:+ if job.finished:+ return+ job.status = status+ job.phase = "done"+ job.error = redact_paths_in_text(error) if error else job.error+ job.finished_at = time.monotonic()+++async def settle(job: Job, timeout: float) -> None:+ """Wait up to ``timeout`` seconds for the job's task to finish on its own, without cancelling it."""+ if job.task is not None and not job.task.done() and timeout > 0:+ await asyncio.wait({job.task}, timeout = timeout)+++def mark_cancelled(job: Job) -> None:+ if job.task is not None and not job.task.done():+ job.task.cancel()+ finish(job, "cancelled")diff --git a/studio/backend/studio_mcp/loading.py b/studio/backend/studio_mcp/loading.pyindex 98611f8cc16..0cab29bb820 100644--- a/studio/backend/studio_mcp/loading.py+++ b/studio/backend/studio_mcp/loading.py@@ -1,31 +1,46 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Load and unload models per kind. A load call blocks until the model is ready (a big download can take an hour), so progress comes from polling the progress routes in a sibling task and reporting it to the agent."""+"""Load and unload models per kind. A load (a big download can take an hour) runs as a job in the job store, so a client that stops waiting never stops it: load_model waits up to ``wait_s`` while reporting progress, then returns the job id for get_job(kind="load")."""from __future__ import annotationsimport asyncioimport dataclasses+import hashlib+import os+import time+from dataclasses import dataclass, field+from pathlib import Pathfrom typing import Any, Awaitable, Callable, Optionalfrom fastmcp import Contextfrom fastmcp.exceptions import ToolError+from studio_mcp import job_storefrom studio_mcp.caller import Callerfrom studio_mcp.errors import tool_error-from studio_mcp.outputs import LoadResult, UnloadResult+from studio_mcp.outputs import EvictedSlot, LoadResult, UnloadResultfrom studio_mcp.tools import as_dict, number, opt_text, route_json, strings, try_jsonPOLL_INTERVAL_S = 2.0+# The longest a load_model or get_job call waits before returning (user decision); clients must allow more.+MAX_WAIT_S = 240+# What a caller still waiting on a job is told to wait before its next get_job without wait_s.+POLL_AFTER_S = 10+# Forwarded reads have no HTTP timeout; a status or progress read taking longer counts as failed.+READ_S = 15.0Progress = Optional[tuple[float, str]]async def with_progress(- ctx: Optional[Context], work: Awaitable[Any], poll: Callable[[], Awaitable[Progress]]+ ctx: Optional[Context],+ work: Awaitable[Any],+ poll: Callable[[], Awaitable[Progress]],+ job: Optional[LoadJob] = None,) -> Any:- """Await ``work`` while reporting what ``poll`` sees. A failed poll is skipped, never fatal."""+ """Await ``work`` while reporting what ``poll`` sees, to the agent and onto ``job``. A failed poll is skipped, never fatal."""task = asyncio.ensure_future(work)try:while True:@@ -33,9 +48,11 @@ async def with_progress(if done:return task.result()try:- progress = await poll()+ progress = await asyncio.wait_for(poll(), timeout = READ_S)except Exception:progress = None+ if progress is not None and job is not None:+ job.note(*progress)if progress is not None and ctx is not None:await ctx.report_progress(progress[0], 1.0, progress[1])finally:@@ -43,6 +60,273 @@ async def with_progress(task.cancel()+@dataclass+class LoadJob(job_store.Job):+ kind: str = "load"+ # The slot kind asked for (llm, tts, audio, image, video) and the model as asked.+ slot: str = "llm"+ model: str = ""+ variant: Optional[str] = None+ # Any other load settings; a repeat call with different ones starts its own load.+ options: Any = None+ fraction: Optional[float] = None+ # Each distinct (fraction, phase) the load went through, for every caller following it.+ marks: list[tuple[float, str]] = field(default_factory = list, repr = False)+ # The STT download this load started, so unload_model can stop it, and whether the STT load+ # request is out (it then runs to the end: cancelling its in-process route does not stop it).+ download: Optional[dict] = None+ posted: bool = False+ # A media load POST, shielded: cancelling the job leaves it running, since the route registers its load from a thread.+ post: Optional[asyncio.Future] = field(default = None, repr = False)++ def note(self, fraction: Optional[float], phase: str) -> None:+ # A step without a fraction keeps the last one, so progress never drops back.+ self.phase = phase+ self.fraction = self.fraction if fraction is None else fraction+ if fraction is not None and (not self.marks or self.marks[-1] != (fraction, phase)):+ self.marks.append((fraction, phase))++ async def report_progress(+ self,+ progress: float,+ total: Optional[float] = None,+ message: Optional[str] = None,+ ) -> None:+ """Stands in for the agent's Context, so load code that reports progress records it on the job."""+ self.note(progress, message or self.phase)+++LOADS = job_store.JobStore(+ capacity = 32, unknown = "No such load job", failure = "The load stopped unexpectedly."+)+++def deadline_after(wait_s: Optional[float]) -> float:+ """When a wait of ``wait_s`` seconds (None: the longest) that starts now ends, on the monotonic clock."""+ wait = MAX_WAIT_S if wait_s is None else max(0, min(wait_s, MAX_WAIT_S))+ return time.monotonic() + wait+++def _is_path(name: str) -> bool:+ if name.startswith(("/", "~", ".")) or "\\" in name or name[1:3] in (":/", ":\\"):+ return True+ # A relative path (models/Foo.gguf) reads like a Hub id; one that exists on disk is a path.+ try:+ return Path(name).expanduser().exists()+ except (OSError, ValueError):+ return False+++def _same(left: Optional[str], right: Optional[str]) -> bool:+ # Hub ids match in any case; a local path keeps the file system's case.+ if not left or not right:+ return False+ return left == right if _is_path(left) or _is_path(right) else left.lower() == right.lower()+++def _repo(name: str) -> str:+ # A GGUF is listed as repo:variant in one place and the bare repo in another. A local path+ # (C:\models\a.gguf) has a drive letter there, not a variant; a ref:<digest> handle is one local model.+ if name.startswith("ref:"):+ return name+ head, sep, tail = name.partition(":")+ return head if sep and len(head) > 1 and "/" not in tail and "\\" not in tail else name+++def _same_id(left: str, right: str) -> bool:+ return _same(left, right) or _same(_repo(left), _repo(right))+++def _in_flight(caller: Caller, asked: LoadJob) -> Optional[LoadJob]:+ for job in LOADS.running(caller.account_id):+ if (+ job.slot == asked.slot+ and _same(job.model, asked.model)+ and (job.variant or "").lower() == (asked.variant or "").lower()+ and job.options == asked.options+ ):+ return job+ return None+++def _stop_loads(+ caller: Caller,+ slots: Any,+ model: Optional[str] = None,+) -> list[LoadJob]:+ """Mark this caller's loads of ``slots`` still running in the job store cancelled. cancel has no load kind, so unload_model is how an agent stops one; the caller's unload route stops the load server-side."""+ jobs = [+ job+ for job in LOADS.running(caller.account_id)+ if job.slot in slots and (model is None or _same_id(job.model, model))+ ]+ for job in jobs:+ job_store.mark_cancelled(job)+ return jobs+++def _early(job: LoadJob) -> LoadResult:+ return LoadResult(+ kind = job.slot,+ model = job.model,+ loaded = False,+ status = "loading",+ fraction = job.fraction,+ job_id = job.job_id,+ poll_after_s = POLL_AFTER_S,+ )+++def outcome(job: LoadJob) -> LoadResult:+ """A finished load's result; a failed or cancelled one raises its error."""+ if job.status == "completed" and isinstance(job.result, LoadResult):+ return job.result+ if job.status == "cancelled":+ raise ToolError(f"The load of {job.model} was cancelled")+ raise ToolError(job.error or f"Unsloth Studio did not load {job.model}")+++async def wait_load(ctx: Optional[Context], job: LoadJob, deadline: float) -> LoadResult:+ """Wait for ``job`` until ``deadline`` (monotonic), reporting each step it makes; never cancels it."""+ seen = 0 if job.task is None or not job.marks else len(job.marks) - 1+ while True:+ if ctx is not None:+ for fraction, phase in job.marks[seen:]:+ await ctx.report_progress(min(fraction, 1.0), 1.0, phase)+ seen = len(job.marks)+ left = deadline - time.monotonic()+ if job.finished or job.task is None or left <= 0:+ break+ await asyncio.wait({job.task}, timeout = min(POLL_INTERVAL_S, left))+ return outcome(job) if job.finished else _early(job)+++async def start_load(+ caller: Caller,+ ctx: Optional[Context],+ *,+ slot: str,+ model: str,+ variant: Optional[str],+ deadline: float,+ run: Callable[[LoadJob, dict[str, list[str]]], Awaitable[LoadResult]],+ options: Any = None,+ hub_token: Optional[str] = None,+) -> LoadResult:+ """Run a load as a job and wait for it until ``deadline``. A repeat call for the same load while it runs follows it; one with another Hub token (a retry with a fixed token, say) starts its own."""+ token = hub_token or caller.hf_token+ asked = LoadJob(+ job_id = job_store.new_id(),+ account_id = caller.account_id,+ slot = slot,+ model = model,+ variant = variant,+ options = (options, hashlib.sha256(token.encode()).hexdigest()[:16] if token else None),+ )++ async def drive(job: LoadJob) -> LoadResult:+ # Every route read runs inside the job, so a slow one never holds the call past its deadline.+ before, chat = await _resident(caller)+ # A chat load Unsloth Studio is already running (started elsewhere) is followed, not started+ # again. The status names only the repo, so a load asking for a variant always posts.+ loading_now = strings(chat.get("loading"))+ if slot in CHAT_ROUTE_KINDS and not variant and any(_same(n, model) for n in loading_now):+ return await _watch_llm(caller, slot, model, before, job)+ # K16, after the attach: M5 waits on a download of the same model here, then M7 refuses a partial one.+ return await run(job, before)++ job = _in_flight(caller, asked) or LOADS.start(asked, drive)+ return await wait_load(ctx, job, deadline)+++# The load_model kinds that load through the chat route.+CHAT_ROUTE_KINDS = ("llm", "tts", "audio")+# Route slot kinds as MCP kinds.+ROUTE_KINDS = {"chat": "llm", "audio": "audio", "image": "image", "video": "video"}+++def _media_name(status: Any) -> Optional[str]:+ if not isinstance(status, dict) or status.get("loaded") is not True:+ return None+ return opt_text(status.get("display_repo_id")) or opt_text(status.get("repo_id"))+++async def _read_json(caller: Caller, path: str) -> Any:+ """``try_json`` within READ_S: a read that stalls is None, like one that failed."""+ try:+ return await asyncio.wait_for(try_json(caller, path), timeout = READ_S)+ except asyncio.TimeoutError:+ return None+++async def _resident(caller: Caller) -> tuple[dict[str, list[str]], dict]:+ chat, image, video = await asyncio.gather(+ _read_json(caller, "/api/inference/status"),+ _read_json(caller, MEDIA_ROUTES["image"]["status"]),+ _read_json(caller, MEDIA_ROUTES["video"]["status"]),+ )+ slots: dict[str, list[str]] = {}+ # A slot that could not be read is left out: this only feeds a report.+ if isinstance(chat, dict):+ slots["llm"] = list(+ dict.fromkeys([*strings(chat.get("serving")), *strings(chat.get("loaded"))])+ )+ for kind, status in (("image", image), ("video", video)):+ if isinstance(status, dict):+ name = _media_name(status)+ slots[kind] = [name] if name else []+ return slots, as_dict(chat)+++async def resident_slots(caller: Caller) -> dict[str, list[str]]:+ """What each slot holds now, by MCP kind: llm (chat models), image and video."""+ return (await _resident(caller))[0]+++def _slot_rows(rows: Any) -> list[EvictedSlot]:+ found = []+ for row in rows if isinstance(rows, list) else []:+ if isinstance(row, dict) and opt_text(row.get("model")):+ kind = opt_text(row.get("kind")) or "chat"+ found.append(EvictedSlot(kind = ROUTE_KINDS.get(kind, kind), model = row["model"]))+ return found+++async def finish_load(+ caller: Caller, kind: str, model: str, before: dict[str, list[str]], answer: Any, **extras: Any+) -> LoadResult:+ """The LoadResult for a finished load, with what it unloaded: the load answer's ``evicted_slots`` (else its ``evicted`` ids placed by the ``before`` snapshot), plus a before/after diff for what the answer leaves out."""+ answer = as_dict(answer)+ named = strings(answer.get("evicted"))+ slots = _slot_rows(answer.get("evicted_slots"))+ if not slots:+ slots = [+ EvictedSlot(+ kind = next(+ (k for k, models in before.items() if any(_same_id(name, m) for m in models)),+ "llm",+ ),+ model = name,+ )+ for name in named+ ]+ # The route lists what it unloaded to make room, never the model this load replaced in its own+ # slot (and nothing at all before it had a list), so whatever else left is found by a diff.+ if any(before.values()):+ after = await resident_slots(caller)+ slots += [+ EvictedSlot(kind = slot, model = name)+ for slot, models in before.items()+ if slot in after+ for name in models+ if not any(_same(name, m) for m in after[slot])+ and not _same_id(name, model)+ and not any(_same_id(name, s.model) for s in slots)+ ]+ evicted = [slot.model for slot in slots] if slots else named+ return LoadResult(kind = kind, model = model, evicted = evicted, evicted_slots = slots, **extras)++def _llm_poll(caller: Caller, model: str) -> Callable[[], Awaitable[Progress]]:async def poll() -> Progress:loading = await route_json("GET", "/api/inference/load-progress", caller = caller)@@ -74,6 +358,7 @@ async def load_llm(load_in_4bit: bool,hf_token: Optional[str],kind: str = "llm",+ deadline: Optional[float] = None,) -> LoadResult:body: dict[str, Any] = {"model_path": model,@@ -86,34 +371,64 @@ async def load_llm(token = hf_token or caller.hf_tokenif token:body["hf_token"] = token- before = await _serving_or_empty(caller)- payload = await with_progress(++ async def run(job: LoadJob, before: dict[str, list[str]]) -> LoadResult:+ payload = await with_progress(+ None,+ route_json("POST", "/api/inference/load", caller = caller, json_body = body),+ _llm_poll(caller, model),+ job,+ )+ if not isinstance(payload, dict):+ raise ToolError("Unsloth Studio did not confirm the load")+ return await finish_load(+ caller,+ kind,+ opt_text(payload.get("model")) or model,+ before,+ payload,+ display_name = opt_text(payload.get("display_name")),+ )++ return await start_load(+ caller,ctx,- route_json("POST", "/api/inference/load", caller = caller, json_body = body),- _llm_poll(caller, model),- )- if not isinstance(payload, dict):- raise ToolError("Unsloth Studio did not confirm the load")- loaded = opt_text(payload.get("model")) or model- evicted = strings(payload.get("evicted"))- # Some loads (an audio engine switch) drop the chat model unlisted; only then, since the route lists its own drops by another id form.- if before and not evicted:- after = set(await _serving_or_empty(caller))- evicted += [- m for m in before if m not in after and m not in evicted and m not in (model, loaded)- ]- return LoadResult(- kind = kind, model = loaded, display_name = opt_text(payload.get("display_name")), evicted = evicted+ slot = kind,+ model = model,+ variant = variant,+ deadline = deadline or deadline_after(None),+ run = run,+ options = (max_seq_length or 0, load_in_4bit),+ hub_token = token,)-async def _serving_or_empty(caller: Caller) -> list[str]:- try:- serving, _checkpoints, _resident = await _llm_resident(caller)- except Exception:- # Only a report: a status read that fails must not turn a finished load into an error.- return []- return serving+async def _watch_llm(+ caller: Caller, kind: str, model: str, before: dict[str, list[str]], job: LoadJob+) -> LoadResult:+ """Follow a chat load Unsloth Studio is already running, until it serves ``model`` or stops loading it."""+ poll = _llm_poll(caller, model)+ misses = 0+ while True:+ await asyncio.sleep(POLL_INTERVAL_S)+ status = await _read_json(caller, "/api/inference/status")+ if not isinstance(status, dict):+ # A failed read is skipped; only a run of them ends the follow.+ misses += 1+ if misses >= _IDLE_POLLS:+ raise ToolError(f"Unsloth Studio stopped answering while loading {model}")+ continue+ misses = 0+ if not any(_same(name, model) for name in strings(status.get("loading"))):+ if any(_same_id(name, model) for name in strings(status.get("serving"))):+ return await finish_load(caller, kind, model, before, {})+ raise ToolError(f"Unsloth Studio stopped loading {model} before it was ready")+ try:+ progress = await asyncio.wait_for(poll(), timeout = READ_S)+ except Exception:+ progress = None+ if progress is not None:+ job.note(*progress)async def _llm_resident(caller: Caller) -> tuple[list[str], list[str], set[str]]:@@ -125,11 +440,15 @@ async def _llm_resident(caller: Caller) -> tuple[list[str], list[str], set[str]]return serving, checkpoints, {*serving, *checkpoints, *strings(status.get("loaded"))}-async def unload_llm(caller: Caller, model: Optional[str]) -> UnloadResult:+async def unload_llm(+ caller: Caller,+ model: Optional[str],+ kind: str = "llm",+) -> UnloadResult:serving, checkpoints, before = await _llm_resident(caller)if model is None:if not checkpoints:- return UnloadResult(kind = "llm", unloaded = False)+ return UnloadResult(kind = kind, unloaded = False)target, model = checkpoints[0], (serving[0] if serving else checkpoints[0])else:# The route unloads by checkpoint id, listed in the same order as serving.@@ -141,7 +460,7 @@ async def unload_llm(caller: Caller, model: Optional[str]) -> UnloadResult:_serving, _checkpoints, after = await _llm_resident(caller)was_loaded = bool({model, target} & before)return UnloadResult(- kind = "llm", model = model, unloaded = was_loaded and not ({model, target} & after)+ kind = kind, model = model, unloaded = was_loaded and not ({model, target} & after))@@ -168,11 +487,48 @@ def _media_fraction(progress: dict) -> Optional[float]:return number(progress.get("fraction"))-def _resident_matches(status: Any, model: str) -> bool:+def _resident_matches(+ status: Any,+ model: str,+ body: Optional[dict] = None,+ *,+ settled: bool = True,+) -> bool:if not isinstance(status, dict) or status.get("loaded") is not True:return Falsenames = {opt_text(status.get("repo_id")), opt_text(status.get("display_repo_id"))}- return model.lower() in {name.lower() for name in names if name}+ if model.lower() not in {name.lower() for name in names if name}:+ return False+ # Another file of the same repo is another model: a missing field reads as not loaded yet.+ wanted = opt_text((body or {}).get("gguf_filename"))+ if not wanted:+ return True+ kind = opt_text((body or {}).get("model_kind"))+ if kind and opt_text(status.get("model_kind")) not in (None, kind):+ return False+ resident = opt_text(status.get("gguf_filename"))+ if resident:+ return os.path.normcase(resident) == os.path.normcase(wanted)+ from hub.utils.gguf import extract_quant_token, quant_token_with_bpw++ # Video status names only a GGUF's quant; a plain checkpoint has none to compare.+ quant = extract_quant_token(wanted) if wanted.lower().endswith(".gguf") else None+ if not quant:+ return True+ if (opt_text(status.get("gguf_variant")) or "").lower() != quant.lower():+ return False+ # Status drops a -3.97bpw style suffix, so until the load reports ready a sibling build matches too.+ if not settled and quant_token_with_bpw(wanted) != quant:+ return False+ # MiniMax-H3 ships its fl2va and ref2va partitions at the same quant; status names the one loaded.+ h3_task = opt_text(status.get("h3_task"))+ if not h3_task:+ return True+ try:+ from core.inference.video_minimax_h3 import h3_transformer_task+ except Exception:+ return False+ return h3_task.lower() == h3_transformer_task(wanted).lower()async def _gguf_filename(caller: Caller, model: str, variant: Optional[str]) -> Optional[str]:@@ -199,6 +555,7 @@ async def load_media(model: str,variant: Optional[str],hf_token: Optional[str],+ deadline: Optional[float] = None,) -> LoadResult:routes = MEDIA_ROUTES[kind]if hf_token:@@ -208,29 +565,74 @@ async def load_media(token = hf_token or caller.hf_tokenif token:body["hf_token"] = token- if variant or "gguf" in model.lower():- filename = await _gguf_filename(caller, model, variant)- if filename:- body["gguf_filename"] = filename- body["model_kind"] = "gguf"- # The plan validates the pick the way the load does, before any download.- plan = await route_json("POST", routes["plan"], caller = caller, json_body = body)- if isinstance(plan, dict):- if opt_text(plan.get("incompatible_reason")):- raise tool_error(plan["incompatible_reason"])- if "gguf_filename" not in body:- for entry in plan.get("entries") or []:- if (- isinstance(entry, dict)- and entry.get("checkpoint")- and opt_text(entry.get("gguf_filename"))- ):- body["gguf_filename"] = entry["gguf_filename"]- break- # The GPU arbiter may unload chat models to make room; diffed after the load like an LLM load.- before = await _serving_or_empty(caller)- # Starts the load in the background; its answer describes whatever was resident before.- await route_json("POST", routes["load"], caller = caller, json_body = body)++ async def prepare() -> None:+ # Hub and plan reads run inside the job, so a slow one never holds the call past ``wait_s``.+ if variant or "gguf" in model.lower():+ filename = await _gguf_filename(caller, model, variant)+ if filename:+ body["gguf_filename"] = filename+ body["model_kind"] = "gguf"+ # The plan validates the pick the way the load does, before any download.+ plan = await route_json("POST", routes["plan"], caller = caller, json_body = body)+ if isinstance(plan, dict):+ if opt_text(plan.get("incompatible_reason")):+ raise tool_error(plan["incompatible_reason"])+ if "gguf_filename" not in body:+ for entry in plan.get("entries") or []:+ if (+ isinstance(entry, dict)+ and entry.get("checkpoint")+ and opt_text(entry.get("gguf_filename"))+ ):+ body["gguf_filename"] = entry["gguf_filename"]+ break++ async def run(job: LoadJob, before: dict[str, list[str]]) -> LoadResult:+ await prepare()+ # The answer names what the arbiter unloaded to make room; the load itself goes on in the background.+ job.post = asyncio.ensure_future(+ route_json("POST", routes["load"], caller = caller, json_body = body)+ )+ job.post.add_done_callback(lambda post: post.cancelled() or post.exception())+ answer = await with_progress(+ None, asyncio.shield(job.post), _media_poll(caller, routes), job+ )+ status = await _media_ready(caller, kind, model, routes, job, body)+ name = _media_name(status) or model+ return await finish_load(caller, kind, name, before, answer)++ return await start_load(+ caller,+ ctx,+ slot = kind,+ model = model,+ variant = variant,+ deadline = deadline or deadline_after(None),+ run = run,+ )+++def _media_poll(caller: Caller, routes: dict[str, str]) -> Callable[[], Awaitable[Progress]]:+ # While the load POST waits (the arbiter freeing the GPU, a chat context being saved), its phase still shows.+ async def poll() -> Progress:+ progress = as_dict(await route_json("GET", routes["progress"], caller = caller))+ phase = opt_text(progress.get("phase"))+ if phase in (None, "ready", "error"):+ return None+ return _media_fraction(progress) or 0.0, "Downloading" if phase == "downloading" else phase++ return poll+++async def _media_ready(+ caller: Caller,+ kind: str,+ model: str,+ routes: dict[str, str],+ job: LoadJob,+ body: Optional[dict] = None,+) -> dict:idle = 0while True:progress = as_dict(await route_json("GET", routes["progress"], caller = caller))@@ -239,10 +641,9 @@ async def load_media(raise tool_error(opt_text(progress.get("error")) or f"The {kind} model failed to load")if phase == "ready":break- status = Noneif phase is None:status = await route_json("GET", routes["status"], caller = caller)- if _resident_matches(status, model):+ if _resident_matches(status, model, body, settled = False):breakidle += 1if idle >= _IDLE_POLLS:@@ -252,30 +653,33 @@ async def load_media()else:idle = 0- fraction = _media_fraction(progress)- if ctx is not None and fraction is not None:- await ctx.report_progress(- fraction, 1.0, "Downloading" if phase == "downloading" else str(phase)- )+ job.note(+ _media_fraction(progress), "Downloading" if phase == "downloading" else str(phase)+ )await asyncio.sleep(POLL_INTERVAL_S)status = await route_json("GET", routes["status"], caller = caller)- if not _resident_matches(status, model):+ if not _resident_matches(status, model, body):raise ToolError(f"Unsloth Studio did not finish loading {model} as the {kind} model")- after = set(await _serving_or_empty(caller)) if before else set()- return LoadResult(- kind = kind,- model = opt_text(status.get("display_repo_id")) or opt_text(status.get("repo_id")) or model,- evicted = [name for name in before if name not in after],- )+ return statusasync def unload_media(caller: Caller, kind: str) -> UnloadResult:routes = MEDIA_ROUTES[kind]+ # A load_model still loading runs on in the job store; the unload route cancels its load too.+ stopped = _stop_loads(caller, (kind,))+ # A stopped load whose POST is still out registers its load once that returns: unload after it, so the unload cancels it.+ posts = {job.post for job in stopped if job.post is not None and not job.post.done()}+ if posts:+ await asyncio.wait(posts, timeout = MAX_WAIT_S)before = as_dict(await route_json("GET", routes["status"], caller = caller))- if before.get("loaded") is not True:+ if before.get("loaded") is not True and not stopped:return UnloadResult(kind = kind, unloaded = False)after = await route_json("POST", routes["unload"], caller = caller)- model = opt_text(before.get("display_repo_id")) or opt_text(before.get("repo_id"))+ model = (+ opt_text(before.get("display_repo_id"))+ or opt_text(before.get("repo_id"))+ or (stopped[-1].model if stopped else None)+ )unloaded = isinstance(after, dict) and after.get("loaded") is not Truereturn UnloadResult(kind = kind, model = model, unloaded = unloaded)@@ -340,6 +744,8 @@ async def download_stt("""Download an STT model and wait for it; the download route reads the Hub token from its header."""if hf_token:caller = dataclasses.replace(caller, hf_token = hf_token)+ if isinstance(ctx, LoadJob):+ ctx.download = {"model": model, "engine": engine}started = await route_json("POST","/api/inference/audio/stt/download",@@ -352,6 +758,8 @@ async def download_stt(hub_header = True,)download_id = started.get("download_id") if isinstance(started, dict) else None+ if isinstance(ctx, LoadJob) and download_id is not None:+ ctx.download = {"model": model, "engine": engine, "download_id": download_id}while True:await asyncio.sleep(POLL_INTERVAL_S)state = (await stt_status(caller, model)).get(engine)@@ -393,6 +801,8 @@ async def load_stt(body: dict[str, Any] = {"model": model, "engine": engine}if variant:body["gguf_variant"] = variant+ if isinstance(ctx, LoadJob):+ ctx.posted = Trueloaded = await route_json("POST", "/api/inference/audio/stt/load", caller = caller, json_body = body)@@ -404,7 +814,52 @@ async def load_stt(return LoadResult(kind = "stt", model = resident)+# Unloads waiting on an STT load that outlasted the unload call, held so they are not collected.+_LATE_UNLOADS: set[asyncio.Task] = set()+++async def _unload_once_loaded(caller: Caller, job: LoadJob) -> None:+ if job.task is not None:+ await asyncio.wait({job.task})+ if job.status == "completed" and isinstance(job.result, LoadResult):+ try:+ await unload_stt(caller, job.result.model)+ except Exception:+ pass+++async def _stop_stt_loads(caller: Caller, model: Optional[str]) -> Optional[str]:+ """Stop this caller's STT loads still downloading, and their downloads, and let one already loading finish so it can be unloaded (later, if it outlasts this call); the last model stopped, if any."""+ stopped = None+ for job in LOADS.running(caller.account_id):+ if job.slot == "stt" and job.posted and (model is None or _same_id(job.model, model)):+ await job_store.settle(job, MAX_WAIT_S)+ if not job.finished:+ late = asyncio.get_running_loop().create_task(_unload_once_loaded(caller, job))+ _LATE_UNLOADS.add(late)+ late.add_done_callback(_LATE_UNLOADS.discard)+ jobs = [+ job+ for job in LOADS.running(caller.account_id)+ if job.slot == "stt" and not job.posted and (model is None or _same_id(job.model, model))+ ]+ for job in jobs:+ job_store.mark_cancelled(job)+ stopped = job.model+ # Partial files stay cached, so a later load resumes the download.+ if job.download is not None:+ await route_json(+ "POST",+ "/api/inference/audio/stt/download/cancel",+ caller = caller,+ json_body = job.download,+ )+ return stopped++async def unload_stt(caller: Caller, model: Optional[str]) -> UnloadResult:+ # A load_model still downloading or loading runs on in the job store, so unloading stops it too.+ stopped = await _stop_stt_loads(caller, model)status = await stt_status(caller)resident = {engine: status[engine]["loaded_model"]@@ -413,7 +868,7 @@ async def unload_stt(caller: Caller, model: Optional[str]) -> UnloadResult:}targets = [(e, m) for e, m in resident.items() if model is None or m == model]if not targets:- return UnloadResult(kind = "stt", model = model, unloaded = False)+ return UnloadResult(kind = "stt", model = stopped or model, unloaded = stopped is not None)engine, name = targets[0]await route_json("POST",diff --git a/studio/backend/studio_mcp/outputs.py b/studio/backend/studio_mcp/outputs.pyindex f65d9839774..57e393dd000 100644--- a/studio/backend/studio_mcp/outputs.py+++ b/studio/backend/studio_mcp/outputs.py@@ -189,6 +189,14 @@ class TrainingSlot(ToolOutput):elapsed_seconds: Optional[float] = None+class TrainingJob(TrainingSlot):+ # Once completed: the run folder and its checkpoints by the names export_model and load_model take.+ run_folder: Optional[RouteText] = None+ checkpoints: list[RouteText] = []+ # The call that returns the run's full loss curve.+ loss_history: Optional[str] = None++class ExportSlot(ToolOutput):active: bool = Falseop_kind: Optional[RouteText] = None@@ -254,8 +262,20 @@ class LoadResult(ToolOutput):kind: RouteTextmodel: RouteTextloaded: bool = True+ # "loading" when wait_s ran out first: the load goes on, and get_job(kind="load", id=job_id) follows it.+ status: RouteText = "loaded"+ fraction: Optional[float] = None+ job_id: Optional[RouteText] = None+ poll_after_s: Optional[float] = Nonedisplay_name: Optional[RouteText] = Noneevicted: list[RouteText] = []+ evicted_slots: list[EvictedSlot] = []+++class EvictedSlot(ToolOutput):+ # llm, audio, image or video.+ kind: RouteText+ model: RouteTextclass UnloadResult(ToolOutput):@@ -570,6 +590,10 @@ class JobStatus(ToolOutput):seed: Optional[int] = Nonestatus: Optional[RouteText] = Noneprogress_percent: Optional[float] = None+ # While the job runs: seconds to wait before the next get_job call without wait_s.+ poll_after_s: Optional[float] = None+ training: Optional[TrainingJob] = None+ load: Optional[LoadResult] = Noneerror: Optional[RouteText] = Noneerror_code: Optional[RouteText] = Nonevideo: Optional[VideoInfo] = Nonediff --git a/studio/backend/studio_mcp/tools/cancel.py b/studio/backend/studio_mcp/tools/cancel.pyindex 1fe9437e0c3..6fdc65ecab8 100644--- a/studio/backend/studio_mcp/tools/cancel.py+++ b/studio/backend/studio_mcp/tools/cancel.py@@ -10,7 +10,7 @@ from urllib.parse import quotefrom fastmcp.exceptions import ToolError-from studio_mcp import export_jobs+from studio_mcp import export_jobs, job_storefrom studio_mcp.caller import current_callerfrom studio_mcp.outputs import CancelResultfrom studio_mcp.tools import DESTRUCTIVE, EXPORT_STATUS, as_dict, opt_text, route_json@@ -91,15 +91,15 @@ async def _cancel_export_job(caller: Any, job: export_jobs.ExportJob) -> tuple[b"POST", "/api/export/cancel", caller = caller, json_body = {"expected_op_id": op_id})if _stopped("export", answer):- export_jobs.mark_cancelled(job)+ job_store.mark_cancelled(job)return True, "The export was stopped and its checkpoint unloaded."# Its step just ended, or is queued behind someone else's: let a finished job report itself.- await export_jobs.settle(job, SETTLE_S)+ await job_store.settle(job, SETTLE_S)if job.finished:return False, f"The export already {job.status}."if job.op_id == op_id:break- export_jobs.mark_cancelled(job)+ job_store.mark_cancelled(job)return True, "The export job was stopped before its next step."diff --git a/studio/backend/studio_mcp/tools/jobs.py b/studio/backend/studio_mcp/tools/jobs.pyindex 37e2549f7ed..e869016842c 100644--- a/studio/backend/studio_mcp/tools/jobs.py+++ b/studio/backend/studio_mcp/tools/jobs.py@@ -1,17 +1,22 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0-"""Long-running Unsloth Studio jobs: ``run_recipe`` starts a data recipe and ``get_job`` reports on any job."""+"""Long-running Unsloth Studio jobs: ``run_recipe`` starts a data recipe and ``get_job`` reports on any job, waiting for it to move when asked."""from __future__ import annotations+import asyncio+import timefrom typing import Annotated, Any, Literal, Optionalfrom urllib.parse import quote+from fastmcp import Contextfrom fastmcp.exceptions import ToolErrorfrom fastmcp.tools import ToolResultfrom pydantic import Field+from studio_mcp import checkpoints as named+from studio_mcp import export_jobs, loadingfrom studio_mcp.caller import Caller, current_callerfrom studio_mcp.errors import raise_for_routefrom studio_mcp.forward import forward@@ -26,7 +31,6 @@ from studio_mcp.media import (signed_url,video_frame_path,)-from studio_mcp import export_jobsfrom studio_mcp.outputs import (ExportInfo,ExportJobSummary,@@ -35,10 +39,12 @@ from studio_mcp.outputs import (RecipeError,RecipeInfo,RecipeResult,+ TrainingJob,VideoInfo,)from studio_mcp.tools import READ_ONLY, WRITES, as_dict, integer, leaf_name, number, opt_textfrom studio_mcp.tools import route_json+from studio_mcp.tools.status import SLOT_ROUTES, _diffusion_training, _trainingdef _video_path(video_id: str, suffix: str = "") -> str:@@ -189,9 +195,147 @@ async def _export_job(caller: Caller, job_id: Optional[str]) -> ToolResult:return media_result([], _export_status(export_jobs.lookup(caller.account_id, job_id)))+# Training phases by job status; any other phase means the run is still going.+TRAINING_STATUS = {+ "completed": "completed",+ "error": "failed",+ "stopped": "cancelled",+ "idle": "idle",+}+++async def _training_job(caller: Caller, kind: str, job_id: Optional[str]) -> ToolResult:+ payload = as_dict(await route_json("GET", SLOT_ROUTES[kind], caller = caller))+ slot = (_training if kind == "training" else _diffusion_training)(payload)+ # An id that is not the current job (or one with nothing running) is never echoed back as idle.+ if job_id and job_id != slot.job_id:+ # list_training_runs reads LLM runs only.+ where = (+ f'; list_training_runs(run_id="{job_id}") has its result' if kind == "training" else ""+ )+ raise ToolError(f"{job_id} is not the current training job{where}")+ phase = "running" if kind == "diffusion_training" and slot.phase == "busy" else slot.phase+ status = TRAINING_STATUS.get(phase, "running")+ training = TrainingJob(**slot.model_dump())+ if status == "completed":+ details = as_dict(payload.get("details")) if kind == "training" else payload+ run, found = await _run_folder(caller, opt_text(details.get("output_dir")))+ training = training.model_copy(+ update = {+ "run_folder": run,+ "checkpoints": [c.name for c in found if run and c.run == run],+ # list_training_runs reads LLM runs only.+ "loss_history": (+ f'list_training_runs(run_id="{slot.job_id}")'+ if kind == "training" and slot.job_id+ else None+ ),+ }+ )+ return media_result(+ [],+ JobStatus(+ kind = kind,+ id = slot.job_id or job_id,+ status = status,+ progress_percent = slot.progress_percent,+ error = slot.error,+ training = training,+ ),+ )+++async def _run_folder(caller: Caller, output: Optional[str]) -> tuple[Optional[str], list]:+ """The run folder name for a status ``output_dir``: an API key gets an opaque reference, matched like list_training_runs does."""+ from hub.utils.host_paths import cache_reference++ found, folders = await named.list_checkpoints(caller)+ if not output:+ return None, found+ for folder, path in folders.items():+ if output in (path, cache_reference(path)):+ return folder, found+ return (None if output.startswith("ref:") else leaf_name(output)), found+++def _load_status(job: loading.LoadJob) -> JobStatus:+ return JobStatus(+ kind = "load",+ id = job.job_id,+ status = job.status,+ progress_percent = (+ 100.0+ if job.status == "completed"+ else round(100.0 * job.fraction, 1)+ if job.fraction is not None+ else None+ ),+ error = job.error,+ load = job.result if job.status == "completed" else None,+ )+++async def _load_job(caller: Caller, job_id: Optional[str]) -> ToolResult:+ if job_id is None:+ jobs = [+ JobSummary(id = job.job_id, status = job.status)+ for job in loading.LOADS.jobs_of(caller.account_id)+ ]+ return media_result([], JobStatus(kind = "load", jobs = jobs))+ return media_result([], _load_status(loading.LOADS.lookup(caller.account_id, job_id)))+++async def _read_once(+ caller: Caller,+ kind: str,+ job_id: Optional[str],+ rows: Optional[int],+ offset: int,+ return_media: ReturnMedia = "inline",+ preview_px: Optional[int] = None,+) -> ToolResult:+ if kind == "recipe":+ return await _recipe_job(caller, job_id, rows, offset)+ if kind == "export":+ return await _export_job(caller, job_id)+ if kind == "load":+ return await _load_job(caller, job_id)+ if kind in ("training", "diffusion_training"):+ return await _training_job(caller, kind, job_id)+ if job_id is None:+ return await _video_jobs(caller)+ return await _video_job(caller, job_id, return_media, preview_px)+++# The least time get_job gives its first read and, at the end of a wait, its rows read.+FIRST_READ_S = 30+LAST_READ_S = 5+# Job statuses that mean the job is still going; get_job(wait_s) waits only on these.+RUNNING = {"running", "queued", "in_progress", "created", "pending", "active", "cancelling"}+++def _moment(structured: Any) -> tuple[Any, Any, Any]:+ # (status, phase, percent): a wait ends when the first two change. Export has no percent, only a phase.+ job = as_dict(structured)+ phase = as_dict(job.get("export")).get("phase") or as_dict(job.get("training")).get("phase")+ return job.get("status"), phase, job.get("progress_percent")++async def get_job(- kind: Literal["video", "recipe", "export"],+ kind: Literal["video", "recipe", "export", "training", "diffusion_training", "load"],id: Optional[str] = None,+ wait_s: Annotated[+ int,+ Field(+ ge = 0,+ le = loading.MAX_WAIT_S,+ description = (+ "Seconds to wait for the job to finish or move on, 0 to 240. "+ "50 stays under Codex's 60 s default tool timeout; above that, raise the client's timeout."+ ),+ ),+ ] = 0,+ ctx: Optional[Context] = None,rows: Annotated[Optional[int], Field(ge = 1, le = 500)] = None,offset: Annotated[int, Field(ge = 0)] = 0,return_media: Annotated[@@ -206,15 +350,84 @@ async def get_job(Field(ge = 32, le = 1024, description = "Long side of a video preview in pixels; default 1024."),] = None,) -> ToolResult:- """The state of a long-running job. kind "video": with ``id``, its status and progress, and once completed a thumbnail plus the video's URL; without ``id``, recent video jobs. kind "recipe": a data recipe job's status and progress (without ``id``, the current one), and with ``rows`` that many generated rows from ``offset``. Unsloth Studio runs one recipe at a time; a recipe job id is not tied to an account. kind "export": an export_model job by its id (without ``id``, this key's export jobs), with its output named relative to Unsloth Studio's exports folder."""+ """The state of a long-running job.+ Follow every long job with ``wait_s`` instead of polling: the call checks every 2 s, reports progress, and returns once the job finishes, its status changes (for an export or a training run, also its phase), or ``wait_s`` runs out; a running job then gives ``poll_after_s``.+ kind "video": with ``id``, its status and progress, and once completed a thumbnail plus the video's URL; without ``id``, recent video jobs.+ kind "recipe": a data recipe job's status and progress (without ``id``, the current one), and with ``rows`` that many generated rows from ``offset``.+ Unsloth Studio runs one recipe at a time; a recipe job id is not tied to an account.+ kind "export": an export_model job by its id (without ``id``, this key's export jobs), with its output named relative to Unsloth Studio's exports folder.+ kind "training" or "diffusion_training": the current run started by start_training (``id`` is its job id), and once completed its run folder and checkpoint names; ``loss_history`` names the list_training_runs call with the full loss curve.+ kind "load": a load_model call that returned ``status`` "loading", by its ``job_id``; once completed, ``load`` is the load's result with what it unloaded."""caller = current_caller()- if kind == "recipe":- return await _recipe_job(caller, id, rows, offset)- if kind == "export":- return await _export_job(caller, id)- if id is None:- return await _video_jobs(caller)- return await _video_job(caller, id, return_media, preview_px)+ deadline = time.monotonic() + min(wait_s, loading.MAX_WAIT_S)+ # Forwarded reads have no HTTP timeout, so each one here is bounded: the first by the wait (at+ # least FIRST_READ_S), the polls and the final rows read by what is left of it.+ first = max(min(wait_s, loading.MAX_WAIT_S), FIRST_READ_S)+ try:+ result = await asyncio.wait_for(+ _read_once(caller, kind, id, rows, offset, return_media, preview_px), timeout = first+ )+ except asyncio.TimeoutError:+ raise ToolError(+ f"Unsloth Studio did not report on the {kind} job within {first:g} s; call get_job again"+ ) from None+ start = _moment(result.structured_content)+ # A listing has nothing to wait on.+ waits = id is not None or kind in ("recipe", "training", "diffusion_training")+ moved = polled = False+ while waits and start[0] in RUNNING:+ left = deadline - time.monotonic()+ if left <= 0:+ break+ await asyncio.sleep(min(loading.POLL_INTERVAL_S, left))+ try:+ # Recipe rows are read once, for the answer, not on every poll; a read that stalls past+ # the deadline ends the wait with the last answer.+ latest = await asyncio.wait_for(+ _read_once(caller, kind, id, None, offset, return_media, preview_px),+ timeout = max(deadline - time.monotonic(), 0.01),+ )+ except asyncio.TimeoutError:+ break+ except Exception:+ # Like a load's progress poll: a failed read during the wait is skipped.+ continue+ result, polled = latest, True+ now = _moment(result.structured_content)+ if now[:2] != start[:2]:+ moved = True+ break+ if ctx is not None and now[2] is not None:+ await ctx.report_progress(min(now[2] / 100.0, 1.0), 1.0, str(now[1] or now[0]))+ if polled and rows:+ try:+ result = await asyncio.wait_for(+ _read_once(caller, kind, id, rows, offset, return_media, preview_px),+ timeout = max(deadline - time.monotonic(), LAST_READ_S),+ )+ except asyncio.TimeoutError:+ pass # The answer goes out without rows; the next get_job reads them.+ # A wait that ended because the job moved on can be followed again at once.+ return _with_poll_after(kind, result, 0 if moved else POLL_AFTER_S[kind])+++# Seconds a caller not using wait_s should leave before asking again, by kind.+POLL_AFTER_S = {+ "video": 15,+ "recipe": 10,+ "export": 15,+ "training": 30,+ "diffusion_training": 30,+ "load": 10,+}+++def _with_poll_after(kind: str, result: ToolResult, after: float) -> ToolResult:+ structured = as_dict(result.structured_content)+ if structured.get("status") not in RUNNING or structured.get("jobs") is not None:+ return result+ status = JobStatus.model_validate({**structured, "poll_after_s": after})+ return media_result(list(result.content[1:]), status)async def run_recipe(diff --git a/studio/backend/studio_mcp/tools/models.py b/studio/backend/studio_mcp/tools/models.pyindex 15aa52adeee..d72ab5cc198 100644--- a/studio/backend/studio_mcp/tools/models.py+++ b/studio/backend/studio_mcp/tools/models.py@@ -262,15 +262,55 @@ async def load_model(max_seq_length: Optional[int] = None,load_in_4bit: bool = True,hf_token: Optional[str] = None,+ wait_s: Annotated[+ Optional[int],+ Field(+ description = (+ "Seconds to wait for the load, 0 to 240 (unset: 240). "+ "Raise the client's tool timeout above 240 s, or pass 50 under Codex's 60 s default. "+ "The load keeps running in Unsloth Studio either way."+ )+ ),+ ] = None,ctx: Optional[Context] = None,) -> LoadResult:- """Load a model into Unsloth Studio, downloading it first if needed, and wait until it is ready; progress is reported while it loads. ``kind`` "llm" covers chat, vision, embedding and audio models, and the "tts" and "audio" kinds list_models reports load the same way; "image" and "video" load the generation models and "stt" a speech-to-text model (downloaded first when needed). ``model`` is an id from list_models or a Hugging Face repo id; ``variant`` picks a GGUF quantization such as Q4_K_M. ``max_seq_length`` 0 or unset lets Unsloth Studio choose the context, and ``load_in_4bit`` picks 4-bit weights (both llm only). ``hf_token`` is for gated repos. Loading may unload other models to make room, for example a chat model to fit an image model; they are listed in ``evicted``."""+ """Load a model into Unsloth Studio, downloading it first if needed, and report progress while it loads.+ ``kind`` "llm" covers chat, vision, embedding and audio models, and the "tts" and "audio" kinds list_models reports load the same way; "image" and "video" load the generation models and "stt" a speech-to-text model (downloaded first when needed).+ ``model`` is an id from list_models or a Hugging Face repo id; ``variant`` picks a GGUF quantization such as Q4_K_M.+ ``max_seq_length`` 0 or unset lets Unsloth Studio choose the context, and ``load_in_4bit`` picks 4-bit weights (both llm only).+ ``hf_token`` is for gated repos.+ The call waits up to ``wait_s`` seconds (at most 240); set the client's tool timeout above that, or pass wait_s=50 under Codex's 60 s default.+ A load that is still running then returns ``loaded`` false, ``status`` "loading" and a ``job_id``: it keeps running in Unsloth Studio, and get_job(kind="load", id=job_id, wait_s=…) follows it to the same result.+ Calling load_model again with the same arguments while that load runs follows it instead of starting another, and so does a call (without ``variant``) for a chat model Unsloth Studio is already loading.+ Loading may unload other models to make room, for example a chat model to fit an image model; ``evicted`` lists them and ``evicted_slots`` gives each one's kind."""caller = current_caller()+ deadline = loading.deadline_after(wait_s)if kind == "stt":- return await loading.load_stt(caller, ctx, model = model, variant = variant, hf_token = hf_token)++ async def load_stt(job: loading.LoadJob, _before: Any) -> LoadResult:+ return await loading.load_stt(+ caller, job, model = model, variant = variant, hf_token = hf_token+ )++ return await loading.start_load(+ caller,+ ctx,+ slot = kind,+ model = model,+ variant = variant,+ deadline = deadline,+ run = load_stt,+ hub_token = hf_token,+ )if kind in loading.MEDIA_ROUTES:return await loading.load_media(- caller, ctx, kind = kind, model = model, variant = variant, hf_token = hf_token+ caller,+ ctx,+ kind = kind,+ model = model,+ variant = variant,+ hf_token = hf_token,+ deadline = deadline,)return await loading.load_llm(caller,@@ -281,16 +321,17 @@ async def load_model(load_in_4bit = load_in_4bit,hf_token = hf_token,kind = kind,+ deadline = deadline,)async def unload_model(kind: ModelKind = "llm", model: Optional[str] = None) -> UnloadResult:- """Unload a model to free memory. Without ``model`` the active one of that kind is unloaded; image and video have one slot each, so ``model`` is ignored there. The embedding model is managed by Unsloth Studio and is not unloaded here. ``unloaded`` is false when nothing matching was loaded."""+ """Unload a model to free memory. Without ``model`` the active one of that kind is unloaded; image and video have one slot each, so ``model`` is ignored there. The embedding model is managed by Unsloth Studio and is not unloaded here. ``unloaded`` is false when nothing matching was loaded. For "image", "video" and "stt", a load_model of that kind still running is stopped too (a speech-to-text load past its download is unloaded once it finishes, after this call if it takes longer than 240 s)."""if kind == "stt":return await loading.unload_stt(current_caller(), model)if kind in loading.MEDIA_ROUTES:return await loading.unload_media(current_caller(), kind)- return await loading.unload_llm(current_caller(), model)+ return await loading.unload_llm(current_caller(), model, kind)TOOLS = ((list_models, READ_ONLY), (load_model, WRITES), (unload_model, DESTRUCTIVE))diff --git a/studio/backend/tests/mcp_harness.py b/studio/backend/tests/mcp_harness.pyindex 05d06f430f4..32351d172b9 100644--- a/studio/backend/tests/mcp_harness.py+++ b/studio/backend/tests/mcp_harness.py@@ -12,9 +12,10 @@ import inspectimport jsonimport secretsimport time+from contextlib import contextmanagerfrom pathlib import Pathfrom types import SimpleNamespace-from typing import Any, Callable, Optional+from typing import Any, Callable, Iterator, Optionalfrom urllib.parse import parse_qslimport pytest@@ -355,6 +356,14 @@ def run_export_job(monkeypatch: Any, studio: FastAPI, args: dict) -> tuple[dict,time.sleep(0.02)+@contextmanager+def session(monkeypatch: Any, studio: FastAPI) -> Iterator[Callable[..., dict]]:+ """One served MCP server for several calls, so a background job one call starts is still there for the next."""+ from mcp_server import create_studio_mcp+ with TestClient(served(create_studio_mcp(), studio, monkeypatch = monkeypatch)) as http:+ yield lambda name, args = None: call_tool(http, name, args)++def route_studio(router: Any = None,prefix: str = "",diff --git a/studio/backend/tests/test_mcp_server.py b/studio/backend/tests/test_mcp_server.pyindex e079d3ef19f..fe25dcc82f6 100644--- a/studio/backend/tests/test_mcp_server.py+++ b/studio/backend/tests/test_mcp_server.py@@ -76,8 +76,8 @@ SCHEMA_FACTS = [("load_model","parameters/properties",- lambda props: set(props)- == {"model", "kind", "variant", "max_seq_length", "load_in_4bit", "hf_token"},+ lambda props: {"model", "kind", "variant", "max_seq_length", "load_in_4bit", "hf_token"}+ <= set(props),),("load_model", "parameters/properties/kind/default", "llm"),(diff --git a/studio/backend/tests/test_mcp_tools_cancel.py b/studio/backend/tests/test_mcp_tools_cancel.pyindex 86f8a7e62a6..1557fe1f7b1 100644--- a/studio/backend/tests/test_mcp_tools_cancel.py+++ b/studio/backend/tests/test_mcp_tools_cancel.py@@ -7,7 +7,7 @@ import typesimport pytestfrom fastapi.responses import JSONResponse-from studio_mcp import export_jobs+from studio_mcp import export_jobs, job_storefrom .mcp_harness import bodies, register_export_job, run_tool, sequence@@ -183,7 +183,7 @@ def _cancel_directly(monkeypatch, job_run, answers):await asyncio.sleep(0)result = await cancel_module.cancel("export", job.job_id)status = job.status- export_jobs.mark_cancelled(job)+ job_store.mark_cancelled(job)await asyncio.gather(job.task, return_exceptions = True)job.status = statusreturn job, result, callsdiff --git a/studio/backend/tests/test_mcp_tools_export.py b/studio/backend/tests/test_mcp_tools_export.pyindex 0c99587cc98..8e3dbbce399 100644--- a/studio/backend/tests/test_mcp_tools_export.py+++ b/studio/backend/tests/test_mcp_tools_export.py@@ -8,7 +8,7 @@ from pathlib import Pathimport pytestfrom fastmcp.exceptions import ToolError-from studio_mcp import export_jobs+from studio_mcp import export_jobs, job_storefrom .mcp_harness import (OUT,@@ -63,9 +63,7 @@ def test_a_running_job_is_not_settled_from_the_export_status(monkeypatch):def test_a_failed_job_reports_its_error_scrubbed(monkeypatch):- export_jobs.finish(- register_export_job(), "failed", error = "Disk full writing /srv/exports/x.gguf"- )+ job_store.finish(register_export_job(), "failed", error = "Disk full writing /srv/exports/x.gguf")result, _studio = _get(monkeypatch, {"kind": "export", "id": "job-a"})assert result["structuredContent"]["status"] == "failed"assert result["structuredContent"]["error"].startswith("Disk full writing")@@ -97,12 +95,12 @@ def test_the_registry_runs_settles_fails_and_cancels_tasks():done = export_jobs.start("owner", "gguf", quick)assert done.task is not None and done.status == "running"failed = export_jobs.start("owner", "gguf", fails)- await export_jobs.settle(done, 1.0)+ await job_store.settle(done, 1.0)stuck = export_jobs.start("owner", "lora", hangs)- await export_jobs.settle(stuck, 0.01)+ await job_store.settle(stuck, 0.01)# settle never cancels: a job still busy after the wait is left running.assert stuck.status == "running"- export_jobs.mark_cancelled(stuck)+ job_store.mark_cancelled(stuck)await asyncio.gather(failed.task, stuck.task, return_exceptions = True)return done, failed, stuckdiff --git a/studio/backend/tests/test_mcp_tools_jobs.py b/studio/backend/tests/test_mcp_tools_jobs.pynew file mode 100644index 00000000000..a4981c451d5--- /dev/null+++ b/studio/backend/tests/test_mcp_tools_jobs.py@@ -0,0 +1,263 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0++"""get_job(wait_s): one call follows a job until it finishes or moves on; and the training and load kinds."""++import asyncio+import threading+import time++import pytest+from fastapi.responses import Response++from studio_mcp import export_jobs, loading++from .mcp_harness import (+ OUT,+ RUN_CHECKPOINTS,+ call_with_progress,+ fake_studio,+ fast_polls, # noqa: F401 (fixture)+ register_export_job,+ run_tool,+ sequence,+ session,+ slow,+)++VIDEO = ("GET", "/v1/videos/vid-1")+TRAIN = ("GET", "/api/train/status")+CHECKPOINTS = ("GET", "/api/models/checkpoints")+DIFFUSION = ("GET", "/api/train/diffusion/status")++[email protected](autouse = True)+def _clean_stores():+ loading.LOADS.reset()+ export_jobs._reset()+ yield+ loading.LOADS.reset()+ export_jobs._reset()+++def _video(status, progress):+ return {"id": "vid-1", "status": status, "progress": progress}+++def _counted(*answers):+ answer, reads = sequence(*answers), []++ def handler(request, body):+ reads.append(1)+ return answer(request, body)++ return handler, reads+++def test_a_wait_returns_once_the_job_finishes_and_reports_progress(monkeypatch, fast_polls):+ handler, reads = _counted(+ _video("in_progress", 10), _video("in_progress", 40), _video("completed", 100)+ )+ studio = fake_studio(+ {VIDEO: handler, ("GET", "/v1/videos/vid-1/content"): Response(status_code = 404)}+ )+ progress, result = call_with_progress(+ monkeypatch, studio, "get_job", {"kind": "video", "id": "vid-1", "wait_s": 30}+ )+ assert result["structuredContent"]["status"] == "completed"+ assert result["structuredContent"]["poll_after_s"] is None+ assert len(reads) == 3+ assert [p["progress"] for p in progress] == [0.4]++[email protected]("wait_s,reads_at_least", [(0, 1), (1, 3)])+def test_a_running_job_says_when_to_look_again(monkeypatch, fast_polls, wait_s, reads_at_least):+ monkeypatch.setattr(loading, "MAX_WAIT_S", 0.1)+ handler, reads = _counted(_video("in_progress", 10))+ result, _studio = run_tool(+ monkeypatch, {VIDEO: handler}, "get_job", {"kind": "video", "id": "vid-1", "wait_s": wait_s}+ )+ assert result["structuredContent"]["status"] == "in_progress"+ assert result["structuredContent"]["poll_after_s"] == 15+ assert len(reads) >= reads_at_least+ assert wait_s or len(reads) == 1+++def test_a_wait_returns_on_a_status_change(monkeypatch, fast_polls):+ handler, reads = _counted(_video("queued", 0), _video("in_progress", 0))+ result, _studio = run_tool(+ monkeypatch, {VIDEO: handler}, "get_job", {"kind": "video", "id": "vid-1", "wait_s": 30}+ )+ assert result["structuredContent"]["status"] == "in_progress"+ # It moved on, so the next wait can start at once.+ assert result["structuredContent"]["poll_after_s"] == 0+ assert len(reads) == 2+++def test_an_export_wait_returns_on_a_phase_change(monkeypatch, fast_polls):+ # Export has no percent: a wait ends when the job moves to its next step.+ job = register_export_job(phase = "loading")+ threading.Timer(0.1, lambda: setattr(job, "phase", "exporting")).start()+ with session(monkeypatch, fake_studio({})) as call:+ result = call("get_job", {"kind": "export", "id": job.job_id, "wait_s": 30})+ assert result["structuredContent"]["export"]["phase"] == "exporting"+ assert result["structuredContent"]["status"] == "running"+++TRAINING = {+ "job_id": "job-7",+ "phase": "training",+ "is_training_running": True,+ "message": "Training",+ "details": {"step": 10, "total_steps": 30, "loss": 1.2},+}+++def _done(output_dir):+ details = {**TRAINING["details"], "step": 30, "output_dir": output_dir}+ return {**TRAINING, "phase": "completed", "is_training_running": False, "details": details}+++# An API key sees output_dir as an opaque reference; a local caller may see the path.[email protected]("opaque", [True, False])+def test_a_training_wait_returns_the_run_folder_and_its_checkpoints(+ monkeypatch, fast_polls, opaque+):+ from hub.utils.host_paths import cache_reference++ path = f"{OUT}/qwen-lora"+ routes = {+ TRAIN: sequence(TRAINING, _done(cache_reference(path) if opaque else path)),+ CHECKPOINTS: RUN_CHECKPOINTS,+ }+ result, _studio = run_tool(monkeypatch, routes, "get_job", {"kind": "training", "wait_s": 30})+ job = result["structuredContent"]+ assert (job["kind"], job["id"], job["status"]) == ("training", "job-7", "completed")+ assert job["training"]["run_folder"] == "qwen-lora"+ assert job["training"]["checkpoints"] == ["qwen-lora", "qwen-lora/checkpoint-30"]+ assert job["training"]["loss_history"] == 'list_training_runs(run_id="job-7")'+ assert "/srv" not in str(result)+++def test_a_diffusion_training_run_has_no_llm_history_pointer(monkeypatch):+ routes = {+ ("GET", "/api/train/diffusion/status"): {+ "active": False,+ "job_id": "d-1",+ "status": "completed",+ "step": 5,+ "total_steps": 5,+ "output_dir": f"{OUT}/my-lora",+ },+ CHECKPOINTS: RUN_CHECKPOINTS,+ }+ result, _studio = run_tool(monkeypatch, routes, "get_job", {"kind": "diffusion_training"})+ job = result["structuredContent"]+ assert (job["status"], job["training"]["run_folder"]) == ("completed", "my-lora")+ assert job["training"]["loss_history"] is None++[email protected]("idle", [False, True])[email protected]("kind,route", [("training", TRAIN), ("diffusion_training", DIFFUSION)])+def test_a_training_job_that_is_not_current_points_at_its_history(monkeypatch, kind, route, idle):+ if idle:+ current = {"phase": "idle", "is_training_running": False} if kind == "training" else {}+ elif kind == "training":+ current = TRAINING+ else:+ current = {"active": True, "job_id": "d-2", "status": "running"}+ result, _studio = run_tool(+ monkeypatch, {route: current}, "get_job", {"kind": kind, "id": "job-1"}+ )+ assert result["isError"] is True+ text = result["content"][0]["text"]+ assert "job-1 is not the current training job" in text+ # list_training_runs reads LLM runs only.+ assert ('list_training_runs(run_id="job-1")' in text) is (kind == "training")+++def test_an_unknown_load_job_and_the_listing(monkeypatch):+ with session(monkeypatch, fake_studio({})) as call:+ missing = call("get_job", {"kind": "load", "id": "nope"})+ listing = call("get_job", {"kind": "load"})+ assert missing["isError"] is True+ assert "No such load job" in missing["content"][0]["text"]+ assert listing["structuredContent"]["jobs"] == []+++SCHEMA = [+ ("parameters/properties/wait_s/default", 0),+ ("parameters/properties/wait_s/maximum", 240),+ (+ "parameters/properties/kind/enum",+ ["video", "recipe", "export", "training", "diffusion_training", "load"],+ ),+ ("output_schema/type", "object"),+]++[email protected]("path,expected", SCHEMA)+def test_get_job_schema(path, expected):+ from mcp_server import create_studio_mcp++ tool = asyncio.run(create_studio_mcp().get_tool("get_job"))+ node = {"parameters": tool.parameters, "output_schema": tool.output_schema}+ for part in path.split("/"):+ node = node[part]+ assert node == expected+++def test_a_recipe_wait_reads_its_rows_once(monkeypatch, fast_polls):+ status = "/api/data-recipe/jobs/rec-1/status"+ running = {"job_id": "rec-1", "status": "running", "progress": {"percent": 10}}+ done = {"job_id": "rec-1", "status": "completed", "progress": {"percent": 100}}+ pages, page = [], {"dataset": [{"q": "a"}], "total": 1}++ def dataset(request, body):+ pages.append(1)+ return page++ studio = fake_studio(+ {+ ("GET", status): sequence(running, running, running, done),+ ("GET", "/api/data-recipe/jobs/rec-1/dataset"): dataset,+ }+ )+ result, _studio = run_tool(+ monkeypatch, studio, "get_job", {"kind": "recipe", "id": "rec-1", "rows": 5, "wait_s": 30}+ )+ assert result["structuredContent"]["status"] == "completed"+ # The first read and the answer; none of the polls between.+ assert len(pages) == 2+++def test_a_status_read_that_stalls_never_holds_a_wait_past_wait_s(monkeypatch, fast_polls):+ reads = []++ async def handler(request, body):+ reads.append(1)+ if len(reads) > 1:+ await asyncio.sleep(10)+ return _video("in_progress", 10)++ studio = fake_studio(+ {VIDEO: handler, ("GET", "/v1/videos/vid-1/content"): Response(status_code = 404)}+ )+ started = time.monotonic()+ result, _studio = run_tool(+ monkeypatch, studio, "get_job", {"kind": "video", "id": "vid-1", "wait_s": 1}+ )+ assert time.monotonic() - started < 5+ assert result["structuredContent"]["status"] == "in_progress"+++def test_a_first_read_that_stalls_ends_with_a_retry_hint(monkeypatch):+ from studio_mcp.tools import jobs++ monkeypatch.setattr(jobs, "FIRST_READ_S", 0.2)+ studio = fake_studio({VIDEO: slow(_video("in_progress", 10), 10)})+ started = time.monotonic()+ result, _studio = run_tool(monkeypatch, studio, "get_job", {"kind": "video", "id": "vid-1"})+ assert time.monotonic() - started < 5+ assert result["isError"] is True+ assert "call get_job again" in result["content"][0]["text"]diff --git a/studio/backend/tests/test_mcp_tools_loading.py b/studio/backend/tests/test_mcp_tools_loading.pyindex da518c7110d..ff5d8fbffa4 100644--- a/studio/backend/tests/test_mcp_tools_loading.py+++ b/studio/backend/tests/test_mcp_tools_loading.py@@ -1,11 +1,15 @@# SPDX-License-Identifier: AGPL-3.0-only# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0+import asyncioimport json+import timeimport pytestfrom fastapi.responses import JSONResponse, Response+from studio_mcp import loading+from .mcp_harness import (DEFERRED,WHISPER,@@ -14,9 +18,11 @@ from .mcp_harness import (call_with_progress,fake_studio,fast_polls, # noqa: F401 (fixture)+ make_caller,queries,run_tool,sequence,+ session,slow,stt_state,)@@ -33,6 +39,8 @@ LOADED = {STATUS = {"loaded": [LLM], "serving": [LLM], "serving_checkpoints": ["/srv/models/llama.gguf"]}EMPTY_STATUS = {"loaded": [], "serving": [], "serving_checkpoints": []}LOAD = ("POST", "/api/inference/load")+# A load that finished inside its wait.+SETTLED = {"status": "loaded", "fraction": None, "job_id": None, "poll_after_s": None}STT = "/api/inference/audio/stt"STATUS_ROUTE = ("GET", "/api/inference/status")@@ -49,6 +57,14 @@ PAYLOADS = {}[email protected](autouse = True)+def _no_loads_left():+ # A job left running by one test would be followed by the next one loading the same model.+ loading.LOADS.reset()+ yield+ loading.LOADS.reset()++# tts / audio load through the LLM route but report the kind asked for. STATUS serves LLM# before and after, so only the route's own evicted list counts.@pytest.mark.parametrize("kind", [None, "tts", "audio"])@@ -59,8 +75,10 @@ def test_load_sends_the_body_and_reports_the_result(monkeypatch, kind):"kind": kind or "llm","model": LLM,"loaded": True,+ **SETTLED,"display_name": "Llama 3.2 1B","evicted": ["unsloth/Qwen3-0.6B"],+ "evicted_slots": [{"kind": "llm", "model": "unsloth/Qwen3-0.6B"}],}assert bodies(studio, "/api/inference/load") == [{"model_path": LLM, "load_in_4bit": True, "max_seq_length": 0, "gguf_variant": "Q4_K_M"}@@ -217,13 +235,13 @@ def test_a_gguf_image_repo_sends_its_gguf_filename(monkeypatch, fast_polls, extrstudio = _media_studio("image",progress = [{"phase": "downloading", "bytes_downloaded": 1, "bytes_total": 2}, READY],- statuses = [{"loaded": True, "repo_id": FLUX_GGUF, "family": "flux"}],+ statuses = [{"loaded": True, "repo_id": FLUX_GGUF, "gguf_filename": filename}],)result, _studio = run_tool(monkeypatch, studio, "load_model", {"model": FLUX_GGUF, "kind": "image", **extra})- loaded = {"kind": "image", "model": FLUX_GGUF, "loaded": True, "display_name": None}- assert result["structuredContent"] == {**loaded, "evicted": []}+ loaded = {"kind": "image", "model": FLUX_GGUF, "loaded": True, **SETTLED, "display_name": None}+ assert result["structuredContent"] == {**loaded, "evicted": [], "evicted_slots": []}expected = {"model_path": FLUX_GGUF, "gguf_filename": filename, "model_kind": "gguf"}expected.update({"hf_token": extra["hf_token"]} if "hf_token" in extra else {})assert bodies(studio, "/api/inference/images/download-plan") == [expected]@@ -243,6 +261,18 @@ def test_a_plan_entry_supplies_the_checkpoint_file(monkeypatch, fast_polls):assert [c for c in studio.state.calls if c[1] == "/api/hub/gguf-variants"] == []+def test_a_video_checkpoint_with_a_dtype_in_its_name_still_loads(monkeypatch, fast_polls):+ # Video status publishes no file and no quant for a plain checkpoint.+ entry = {"checkpoint": True, "gguf_filename": "ltx-video-2b-v0.9.5-bf16.safetensors"}+ studio = _media_studio(+ "video",+ statuses = [{"loaded": True, "repo_id": LTX, "model_kind": "single_file"}],+ plan = {"entries": [entry]},+ )+ result, _studio = run_tool(monkeypatch, studio, "load_model", {"model": LTX, "kind": "video"})+ assert result["structuredContent"]["loaded"] is True++PREVIOUS = {"loaded": True, "repo_id": "old/model"}AB = {"model": "a/b", "kind": "image"}DISK_FULL = [{"phase": "downloading"}, {"phase": "error", "error": "Out of disk space"}]@@ -291,13 +321,61 @@ def test_a_previous_model_answer_then_the_new_one_loads(monkeypatch, fast_polls)assert result["structuredContent"]["model"] == "new/model"+# Image status names the file, video only its quant: either way the old variant is not the new one.+ "kind,old,new,filename",+ [+ (+ "image",+ {"gguf_filename": "flux1-schnell-Q4_K_S.gguf"},+ {"gguf_filename": "flux1-schnell-Q8_0.gguf"},+ None,+ ),+ ("video", {"gguf_variant": "Q4_K_S"}, {"gguf_variant": "Q8_0"}, None),+ # Same quant, other MiniMax-H3 partition: only h3_task tells them apart.+ (+ "video",+ {"gguf_variant": "Q8_0", "h3_task": "fl2va"},+ {"gguf_variant": "Q8_0", "h3_task": "ref2va"},+ "minimax_h3_ref2va-Q8_0.gguf",+ ),+ # Same quant, other bpw build: status cannot tell them apart before the load is ready.+ ("video", {"gguf_variant": "IQ4_XS"}, {"gguf_variant": "IQ4_XS"}, "m-IQ4_XS-3.97bpw.gguf"),+ ],+)+def test_another_variant_of_the_resident_repo_waits_for_its_own_load(+ monkeypatch, fast_polls, kind, old, new, filename+):+ if filename:++ async def pick(*_args):+ return filename++ monkeypatch.setattr(loading, "_gguf_filename", pick)+ resident = {"loaded": True, "repo_id": FLUX_GGUF, "model_kind": "gguf"}+ old, unnamed, new = {**resident, **old}, {**resident, "gguf_variant": None}, {**resident, **new}+ studio = _media_studio(+ kind,+ progress = [{"phase": None}, {"phase": None}, READY],+ # Read before the load, at each idle poll, then once it is ready.+ statuses = [old, old, unnamed, new],+ )+ result, _studio = run_tool(+ monkeypatch, studio, "load_model", {"model": FLUX_GGUF, "kind": kind, "variant": "Q8_0"}+ )+ assert result["structuredContent"]["loaded"] is True+ # The old variant and a status naming no variant are both "not loaded yet".+ assert len([c for c in studio.state.calls if c[1].endswith("/load-progress")]) == 3++@pytest.mark.parametrize("kind,model", [("image", FLUX_GGUF), ("video", LTX)])def test_a_media_load_reports_the_chat_model_the_gpu_arbiter_unloaded(monkeypatch, fast_polls, kind, model):studio = _media_studio(kind,- statuses = [{"loaded": True, "repo_id": model}],+ # The default variant, as the Hub listing names it.+ statuses = [{"loaded": True, "repo_id": model, "gguf_variant": "Q4_K_S"}],chat_statuses = [STATUS, EMPTY_STATUS],)result, _studio = run_tool(monkeypatch, studio, "load_model", {"model": model, "kind": kind})@@ -483,6 +561,474 @@ def test_stt_unload_names_the_engine_and_model_in_the_query(assert unloads == ([b""] if unloaded else [])+IMAGE_STATUS = ("GET", "/api/inference/images/status")+VIDEO_STATUS = ("GET", "/api/inference/video/status")+FLUX_LOADED = {"loaded": True, "repo_id": FLUX_GGUF}+LTX_LOADED = {"loaded": True, "repo_id": LTX}+UNLOADED = {"loaded": False}+++def _video_over_image(load_answer, *, delay = 0.0):+ """A video load while an image model is resident; the arbiter unloads the image model."""+ return fake_studio(+ {+ ("POST", "/api/inference/video/download-plan"): {"entries": []},+ ("POST", "/api/inference/video/load"): slow(load_answer, delay),+ ("GET", "/api/inference/video/load-progress"): READY,+ # Read before the load, then by the load itself, then after it.+ VIDEO_STATUS: sequence(UNLOADED, LTX_LOADED),+ IMAGE_STATUS: sequence(FLUX_LOADED, UNLOADED),+ }+ )++[email protected]("wait_s,expected", [(None, 240), (300, 240), (50, 50), (0, 0), (-5, 0)])+def test_wait_s_defaults_to_the_cap_and_is_clamped(wait_s, expected):+ assert round(loading.deadline_after(wait_s) - time.monotonic()) == expected+++def test_a_slow_load_returns_early_and_get_job_returns_its_result(monkeypatch, fast_polls):+ studio = _video_over_image(UNLOADED, delay = 0.5)+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": LTX, "kind": "video", "wait_s": 0})[+ "structuredContent"+ ]+ assert early["loaded"] is False+ assert early["status"] == "loading"+ assert early["poll_after_s"] == loading.POLL_AFTER_S+ # A repeat load_model follows the load in flight instead of starting another.+ again = call("load_model", {"model": LTX, "kind": "video", "wait_s": 0})[+ "structuredContent"+ ]+ assert again["job_id"] == early["job_id"]+ job = call("get_job", {"kind": "load", "id": early["job_id"], "wait_s": 30})+ assert job["structuredContent"]["status"] == "completed"+ assert job["structuredContent"]["progress_percent"] == 100.0+ assert job["structuredContent"]["poll_after_s"] is None+ load = job["structuredContent"]["load"]+ assert (load["loaded"], load["status"], load["model"]) == (True, "loaded", LTX)+ assert load["evicted_slots"] == [{"kind": "image", "model": FLUX_GGUF}]+ assert len(bodies(studio, "/api/inference/video/load")) == 1+++def test_a_client_that_gives_up_mid_post_does_not_lose_the_load(monkeypatch, fast_polls):+ studio = fake_studio({**PAYLOADS, LOAD: slow(LOADED, 0.3)})+ caller = make_caller(studio_app = studio)++ async def give_up() -> loading.LoadJob:+ call = asyncio.ensure_future(+ loading.load_llm(+ caller,+ None,+ model = LLM,+ variant = None,+ max_seq_length = None,+ load_in_4bit = True,+ hf_token = None,+ )+ )+ await asyncio.sleep(0.1)+ call.cancel()+ (job,) = loading.LOADS.jobs_of("owner")+ await asyncio.wait({job.task}, timeout = 5)+ return job++ job = asyncio.run(give_up())+ assert job.status == "completed"+ assert (job.result.model, job.result.loaded) == (LLM, True)+++def test_a_chat_load_already_in_flight_is_followed_not_started_again(monkeypatch, fast_polls):+ loading_now = {"loading": [LLM], "serving": [], "loaded": []}+ routes = {**PAYLOADS, STATUS_ROUTE: sequence(loading_now, loading_now, STATUS)}+ progress, result = call_with_progress(+ monkeypatch, fake_studio(routes), "load_model", {"model": LLM}+ )+ assert result["structuredContent"]["loaded"] is True+ assert result["structuredContent"]["model"] == LLM+ assert progress and "mmap" in progress[0]["message"]+++# The load answer's evicted_slots come first, else its evicted placed by what was resident; a+# before/after diff adds what the answer leaves out (here the image model) or, on load routes from+# before evicted_slots, finds everything.+ "answer,expected",+ [+ (+ {+ "evicted": [LLM],+ "evicted_slots": [+ {"kind": "chat", "model": LLM},+ {"kind": "audio", "model": "a/tts"},+ ],+ },+ [+ {"kind": "llm", "model": LLM},+ {"kind": "audio", "model": "a/tts"},+ {"kind": "image", "model": FLUX_GGUF},+ ],+ ),+ ({"evicted": [FLUX_GGUF]}, [{"kind": "image", "model": FLUX_GGUF}]),+ ({}, [{"kind": "image", "model": FLUX_GGUF}]),+ ],+)+def test_evicted_slots_read_the_answer_then_fall_back(monkeypatch, fast_polls, answer, expected):+ studio = _video_over_image({**UNLOADED, **answer})+ result, _studio = run_tool(monkeypatch, studio, "load_model", {"model": LTX, "kind": "video"})+ assert result["structuredContent"]["evicted_slots"] == expected+ assert result["structuredContent"]["evicted"] == [slot["model"] for slot in expected]+++def test_a_tts_load_reports_the_video_model_it_unloaded(monkeypatch):+ routes = {+ **PAYLOADS,+ LOAD: {**LOADED, "evicted": []},+ STATUS_ROUTE: EMPTY_STATUS,+ VIDEO_STATUS: sequence(LTX_LOADED, UNLOADED),+ }+ result, _studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM, "kind": "tts"})+ assert result["structuredContent"]["kind"] == "tts"+ assert result["structuredContent"]["evicted_slots"] == [{"kind": "video", "model": LTX}]++[email protected]("kind", ["tts", "audio"])+def test_unload_echoes_the_kind_asked_for(monkeypatch, kind):+ routes = {**PAYLOADS, STATUS_ROUTE: sequence(STATUS, EMPTY_STATUS)}+ result, _studio = run_tool(monkeypatch, routes, "unload_model", {"kind": kind})+ assert result["structuredContent"] == {"kind": kind, "model": LLM, "unloaded": True}+++def test_saving_the_chat_context_counts_as_progress(monkeypatch, fast_polls):+ posted = []++ async def load(request, body):+ await asyncio.sleep(0.2)+ posted.append(True)+ return UNLOADED++ studio = fake_studio(+ {+ ("POST", "/api/inference/video/download-plan"): {"entries": []},+ ("POST", "/api/inference/video/load"): load,+ ("GET", "/api/inference/video/load-progress"): lambda request, body: (+ READY if posted else {"phase": "saving_chat_context"}+ ),+ VIDEO_STATUS: LTX_LOADED,+ }+ )+ progress, result = call_with_progress(+ monkeypatch, studio, "load_model", {"model": LTX, "kind": "video"}+ )+ assert result["structuredContent"]["loaded"] is True+ assert [p["message"] for p in progress] == ["saving_chat_context"]+++def test_a_chat_load_asking_for_a_variant_never_follows_another_load(monkeypatch, fast_polls):+ # The status names only the repo, so another variant of it loading elsewhere is not this load.+ loading_now = {"loading": [LLM], "serving": [], "loaded": []}+ routes = {**PAYLOADS, STATUS_ROUTE: sequence(loading_now, STATUS)}+ _result, studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM, "variant": "Q8_0"})+ assert len(bodies(studio, "/api/inference/load")) == 1+++def test_one_failed_status_read_does_not_end_a_followed_load(monkeypatch, fast_polls):+ loading_now = {"loading": [LLM], "serving": [], "loaded": []}+ busy = Response(status_code = 503)+ routes = {**PAYLOADS, STATUS_ROUTE: sequence(loading_now, busy, loading_now, STATUS)}+ result, studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM})+ assert result["structuredContent"]["loaded"] is True+ assert bodies(studio, "/api/inference/load") == []+++def test_a_listed_chat_drop_does_not_hide_a_media_eviction(monkeypatch):+ routes = {+ **PAYLOADS,+ STATUS_ROUTE: sequence({**STATUS, "serving": [LLM, "unsloth/Qwen3-0.6B"]}, STATUS),+ VIDEO_STATUS: sequence(LTX_LOADED, UNLOADED),+ }+ result, _studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM})+ assert result["structuredContent"]["evicted_slots"] == [+ {"kind": "llm", "model": "unsloth/Qwen3-0.6B"},+ {"kind": "video", "model": LTX},+ ]+++def test_progress_never_drops_back():+ job = loading.LoadJob(job_id = "j", account_id = "owner")+ job.note(0.5, "Downloading")+ job.note(None, "saving_chat_context")+ assert (job.fraction, job.phase, job.marks) == (+ 0.5,+ "saving_chat_context",+ [(0.5, "Downloading")],+ )+++def test_the_model_a_load_replaced_is_reported_beside_what_the_route_lists(monkeypatch):+ # The route lists the image model the arbiter unloaded, not the chat model this load replaced.+ answer = {+ **LOADED,+ "evicted": [FLUX_GGUF],+ "evicted_slots": [{"kind": "image", "model": FLUX_GGUF}],+ }+ routes = {+ **PAYLOADS,+ LOAD: answer,+ STATUS_ROUTE: sequence({**STATUS, "serving": ["unsloth/Qwen3-0.6B"], "loaded": []}, STATUS),+ IMAGE_STATUS: sequence(FLUX_LOADED, UNLOADED),+ }+ result, _studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM})+ assert result["structuredContent"]["evicted_slots"] == [+ {"kind": "image", "model": FLUX_GGUF},+ {"kind": "llm", "model": "unsloth/Qwen3-0.6B"},+ ]+++def test_a_speech_to_text_load_never_follows_a_chat_load(monkeypatch, fast_polls):+ chat = {"loading": [WHISPER], "serving": [], "loaded": []}+ studio = fake_studio(+ {+ STATUS_ROUTE: chat,+ ("GET", f"{STT}/status"): {"transformers": stt_state(downloaded = [WHISPER])},+ ("POST", f"{STT}/load"): {"loaded_model": WHISPER},+ }+ )+ result, studio = run_tool(monkeypatch, studio, "load_model", {"model": WHISPER, "kind": "stt"})+ assert result["structuredContent"]["kind"] == "stt"+ assert len(bodies(studio, f"{STT}/load")) == 1+++def test_unloading_stt_stops_a_load_still_downloading(monkeypatch, fast_polls):+ # cancel has no load kind, so unload_model is how an agent stops a load it left running.+ downloading = {"downloading": True, "bytes_done": 1, "bytes_total": 10}+ studio = fake_studio(+ {+ ("GET", f"{STT}/status"): {"transformers": stt_state(download = downloading)},+ ("POST", f"{STT}/download"): {"downloading": True, "download_id": "d1"},+ ("POST", f"{STT}/download/cancel"): {"downloading": False, "cancelled": True},+ ("POST", f"{STT}/load"): {"loaded_model": WHISPER},+ }+ )+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": WHISPER, "kind": "stt", "wait_s": 0})+ assert early["structuredContent"]["status"] == "loading"+ job_id = early["structuredContent"]["job_id"]+ unloaded = call("unload_model", {"kind": "stt"})["structuredContent"]+ job = call("get_job", {"kind": "load", "id": job_id})+ assert unloaded == {"kind": "stt", "model": WHISPER, "unloaded": True}+ assert job["structuredContent"]["status"] == "cancelled"+ assert bodies(studio, f"{STT}/download/cancel") == [+ {"model": WHISPER, "engine": "transformers", "download_id": "d1"}+ ]+ assert bodies(studio, f"{STT}/load") == []+++def test_unloading_media_stops_a_load_still_running(monkeypatch, fast_polls):+ # Nothing is resident yet, so before this the unload answered "nothing loaded" and posted nothing.+ unload = "/api/inference/video/unload"+ studio = fake_studio(+ {+ ("POST", "/api/inference/video/download-plan"): {"entries": []},+ ("POST", "/api/inference/video/load"): slow(LTX_LOADED, 0.5),+ ("GET", "/api/inference/video/load-progress"): READY,+ VIDEO_STATUS: UNLOADED,+ IMAGE_STATUS: UNLOADED,+ ("POST", unload): UNLOADED,+ }+ )+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": LTX, "kind": "video", "wait_s": 0})+ job_id = early["structuredContent"]["job_id"]+ unloaded = call("unload_model", {"kind": "video"})["structuredContent"]+ job = call("get_job", {"kind": "load", "id": job_id})+ assert unloaded == {"kind": "video", "model": LTX, "unloaded": True}+ assert job["structuredContent"]["status"] == "cancelled"+ assert len([c for c in studio.state.calls if c[1] == unload]) == 1+++def test_unloading_media_waits_for_a_load_post_still_freeing_the_gpu(monkeypatch, fast_polls):+ # The route registers its load from a thread once the arbiter freed the GPU; cancelling the+ # awaiting job does not stop that thread, so an unload posted earlier finds nothing to cancel.+ unload = "/api/inference/video/unload"+ registered = []+ seen_at_unload = []++ def handoff() -> None:+ time.sleep(0.5)+ registered.append(True)++ async def load(request, body):+ await asyncio.to_thread(handoff)+ return LTX_LOADED++ def unload_route(request, body):+ seen_at_unload.append(bool(registered))+ return UNLOADED++ studio = fake_studio(+ {+ ("POST", "/api/inference/video/download-plan"): {"entries": []},+ ("POST", "/api/inference/video/load"): load,+ ("GET", "/api/inference/video/load-progress"): READY,+ VIDEO_STATUS: UNLOADED,+ IMAGE_STATUS: UNLOADED,+ ("POST", unload): unload_route,+ }+ )+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": LTX, "kind": "video", "wait_s": 0})+ job_id = early["structuredContent"]["job_id"]+ unloaded = call("unload_model", {"kind": "video"})["structuredContent"]+ job = call("get_job", {"kind": "load", "id": job_id})+ assert unloaded["unloaded"] is True+ assert job["structuredContent"]["status"] == "cancelled"+ assert seen_at_unload == [True]+++def test_unloading_stt_lets_a_load_past_its_download_finish_then_unloads_it(+ monkeypatch, fast_polls+):+ # Cancelling the in-process load route would not stop the sidecar load, so it is waited for.+ studio = fake_studio(+ {+ ("GET", f"{STT}/status"): sequence(+ {"transformers": stt_state(downloaded = [WHISPER])},+ {"transformers": stt_state(downloaded = [WHISPER], loaded = WHISPER)},+ {"transformers": stt_state(downloaded = [WHISPER])},+ ),+ ("POST", f"{STT}/load"): slow({"loaded_model": WHISPER}, 0.5),+ ("POST", f"{STT}/unload"): {"loaded_model": None},+ }+ )+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": WHISPER, "kind": "stt", "wait_s": 0})+ job_id = early["structuredContent"]["job_id"]+ unloaded = call("unload_model", {"kind": "stt"})["structuredContent"]+ job = call("get_job", {"kind": "load", "id": job_id})+ assert unloaded == {"kind": "stt", "model": WHISPER, "unloaded": True}+ assert job["structuredContent"]["status"] == "completed"+ assert bodies(studio, f"{STT}/download/cancel") == []+ assert len(bodies(studio, f"{STT}/load")) == 1+++def test_an_stt_load_that_outlasts_the_unload_is_unloaded_once_it_finishes(monkeypatch, fast_polls):+ monkeypatch.setattr(loading, "MAX_WAIT_S", 0.05)+ on_disk = {"transformers": stt_state(downloaded = [WHISPER])}+ studio = fake_studio(+ {+ ("GET", f"{STT}/status"): sequence(+ on_disk,+ on_disk,+ {"transformers": stt_state(downloaded = [WHISPER], loaded = WHISPER)},+ on_disk,+ ),+ ("POST", f"{STT}/load"): slow({"loaded_model": WHISPER}, 0.5),+ ("POST", f"{STT}/unload"): {"loaded_model": None},+ }+ )+ with session(monkeypatch, studio) as call:+ early = call("load_model", {"model": WHISPER, "kind": "stt", "wait_s": 0})+ unloaded = call("unload_model", {"kind": "stt"})["structuredContent"]+ assert unloaded["unloaded"] is False+ deadline = time.monotonic() + 5+ while not queries(studio, f"{STT}/unload") and time.monotonic() < deadline:+ time.sleep(0.05)+ job = call("get_job", {"kind": "load", "id": early["structuredContent"]["job_id"]})+ assert job["structuredContent"]["status"] == "completed"+ assert queries(studio, f"{STT}/unload") == [+ {"engine": "transformers", "model": WHISPER, "wait": "true"}+ ]+++# The resident-model snapshot and the media Hub and plan reads all run inside the job.[email protected]("slow_route", ["plan", "status"])+def test_a_slow_read_before_the_load_does_not_hold_the_call_past_wait_s(+ monkeypatch, fast_polls, slow_route+):+ studio = fake_studio(+ {+ ("POST", "/api/inference/video/download-plan"): (+ slow({"entries": []}, 1.0) if slow_route == "plan" else {"entries": []}+ ),+ ("POST", "/api/inference/video/load"): LTX_LOADED,+ ("GET", "/api/inference/video/load-progress"): READY,+ VIDEO_STATUS: sequence(UNLOADED, LTX_LOADED),+ IMAGE_STATUS: slow(UNLOADED, 1.0) if slow_route == "status" else UNLOADED,+ }+ )+ with session(monkeypatch, studio) as call:+ started = time.monotonic()+ early = call("load_model", {"model": LTX, "kind": "video", "wait_s": 0})+ assert time.monotonic() - started < 0.9+ assert early["structuredContent"]["status"] == "loading"+ job = call(+ "get_job", {"kind": "load", "id": early["structuredContent"]["job_id"], "wait_s": 30}+ )+ assert job["structuredContent"]["status"] == "completed"+ assert job["structuredContent"]["load"]["model"] == LTX+++ "left,right,same",+ [+ ("unsloth/Q-GGUF:Q4_K_M", "unsloth/Q-GGUF", True),+ ("C:\\models\\old.gguf", "C:\\models\\new.gguf", False),+ ("C:/models/old.gguf", "C:/models/new.gguf", False),+ ],+)+def test_only_a_repo_id_drops_its_variant_when_matching(left, right, same):+ assert loading._same_id(left, right) is same+++def test_a_retry_with_another_hub_token_starts_its_own_load(monkeypatch, fast_polls):+ studio = fake_studio({**PAYLOADS, LOAD: slow(LOADED, 1.0), STATUS_ROUTE: EMPTY_STATUS})+ with session(monkeypatch, studio) as call:+ first = call("load_model", {"model": LLM, "wait_s": 0, "hf_token": "hf_old"})+ same = call("load_model", {"model": LLM, "wait_s": 0, "hf_token": "hf_old"})+ retry = call("load_model", {"model": LLM, "wait_s": 0, "hf_token": "hf_new"})+ ids = [r["structuredContent"]["job_id"] for r in (first, same, retry)]+ assert ids[0] == ids[1] != ids[2]+++ "left,right,same",+ [+ ("unsloth/Qwen3-0.6B", "Unsloth/qwen3-0.6b", True),+ ("/models/Foo.gguf", "/models/foo.gguf", False),+ ("C:\\models\\Foo.gguf", "c:\\models\\foo.gguf", False),+ ("models/Foo.gguf", "models/foo.gguf", False),+ ],+)+def test_hub_ids_match_in_any_case_and_local_paths_keep_theirs(+ tmp_path, monkeypatch, left, right, same+):+ # A relative path that exists on disk is a path, though it reads like a Hub id.+ (tmp_path / "models").mkdir()+ (tmp_path / "models" / "Foo.gguf").write_text("")+ monkeypatch.chdir(tmp_path)+ assert loading._same(left, right) is same+++def test_a_status_route_that_stalls_never_holds_up_a_load(monkeypatch, fast_polls):+ # The snapshot only feeds the eviction report, so a slot that does not answer is left out.+ monkeypatch.setattr(loading, "READ_S", 0.2)+ routes = {**PAYLOADS, STATUS_ROUTE: EMPTY_STATUS, IMAGE_STATUS: slow(UNLOADED, 10)}+ started = time.monotonic()+ result, _studio = run_tool(monkeypatch, routes, "load_model", {"model": LLM, "wait_s": 30})+ assert time.monotonic() - started < 5+ assert result["structuredContent"]["loaded"] is True+++def test_two_local_ref_handles_are_different_models():+ from studio_mcp import loading++ a, b = "ref:" + "1" * 32, "ref:" + "2" * 32+ assert not loading._same_id(a, b)+ assert loading._same_id(a, a)+ assert loading._same_id("org/repo:Q4_K_M", "org/repo")++def test_a_padded_keep_loaded_refusal_names_unload_model(monkeypatch):refusal = ("Loading this model would unload the chat model unsloth/Qwen3-0.6B, and Keep multiple"diff --git a/studio/backend/tests/test_mcp_tools_video.py b/studio/backend/tests/test_mcp_tools_video.pyindex 0a0f5fdbdba..b1d10384ba3 100644--- a/studio/backend/tests/test_mcp_tools_video.py+++ b/studio/backend/tests/test_mcp_tools_video.py@@ -111,6 +111,9 @@ def test_a_completed_job_returns_its_thumbnail_and_a_link_never_the_mp4(monkeypa"seed": None,"status": "completed","progress_percent": 100.0,+ "poll_after_s": None,+ "training": None,+ "load": None,"error": None,"error_code": None,"video": {