Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Docker Studio: stop disabling Unsloth's TRL patches on GPU hosts (#10825)
oobabooga4e926dba5001
diff --git a/docker/DOCKERHUB.md b/docker/DOCKERHUB.mdindex e2dadd3489e..b51841f2a89 100644--- a/docker/DOCKERHUB.md+++ b/docker/DOCKERHUB.md@@ -101,11 +101,16 @@ Turing has no bfloat16; Unsloth falls back to float16 there. AMD GPUs are not su| `JUPYTER_PASSWORD` | JupyterLab password. Unset: generated once and printed in the logs. || `JUPYTER_PORT` | JupyterLab port inside the container. Default `8888`. || `SSH_KEY` or `PUBLIC_KEY` | OpenSSH public key for root login. Enables sshd on port 22. Password login is never enabled. |-| `UNSLOTH_ALLOW_CPU=1` | Allow starting without a GPU. |+| `UNSLOTH_ALLOW_CPU=1` | Allow starting without a GPU (`latest` already does). Dropped when a GPU is visible, where it would turn off Unsloth's training patches. || `UNSLOTH_JUPYTER_CLOUDFLARE=1` | Publish JupyterLab through a Cloudflare quick tunnel and print the URL. || `UNSLOTH_SKIP_NOTEBOOK_SYNC=1` | Do not refresh the notebooks from GitHub on start. || `HF_TOKEN`, `WANDB_API_KEY` | Forwarded to Hugging Face and Weights and Biases. |+On a host with no GPU, Studio, JupyterLab and its kernels, and login shells all run in+CPU mode. A non-login `docker exec` is built from the image, not the container's first+process, so `docker exec <c> python -c "import unsloth"` still reports that it needs a+GPU: use `docker exec <c> bash -lc '...'` or `docker exec -e UNSLOTH_ALLOW_CPU=1 <c> ...`.+## VolumesThe working directory is `/workspace`. Mount what you want to keep:diff --git a/docker/Dockerfile.studio b/docker/Dockerfile.studioindex 05570927a1e..54e747a7a51 100644--- a/docker/Dockerfile.studio+++ b/docker/Dockerfile.studio@@ -11,8 +11,7 @@## Studio on :8000 (user unsloth; UNSLOTH_STUDIO_PASSWORD env, else a generated one# is printed in the logs; persisted under /opt/unsloth-studio/auth/); JupyterLab on-# :8888 (JUPYTER_PASSWORD env, else a random one is printed). Without GPU passthrough add -e UNSLOTH_ALLOW_CPU=1:-# training is unavailable but Studio chat / Data Recipes / GGUF / Jupyter work.+# :8888 (JUPYTER_PASSWORD env, else a random one is printed). Without a GPU it starts on CPU: no training, but Studio chat / Data Recipes / GGUF / Jupyter work.# CI pins BASE_IMAGE to the published base digest so both images ship the same stack.ARG BASE_IMAGE=unsloth/unsloth:core@@ -51,16 +50,10 @@ ARG TARGETARCH# disabled unless PUBLIC_KEY/SSH_KEY is set (see studio_launch.sh). The# JUPYTER_PORT / UNSLOTH_ENABLE_SSHD defaults let supervisord's %(ENV_*)s resolve.USER root-# UNSLOTH_ALLOW_CPU defaults to 1 on THIS image only. It publishes as :latest, so a-# plain `docker run unsloth/unsloth` on a CPU-only, AMD or Docker-Desktop host would-# otherwise exit 1 from entrypoint.sh before Studio ever starts. Most of what this-# image offers (Studio chat over llama.cpp/GGUF, Data Recipes, JupyterLab, GGUF-# tooling) needs no GPU. The entrypoint only takes the CPU path when no GPU is-# visible, so this does not weaken the checks on a GPU host, and `-e-# UNSLOTH_ALLOW_CPU=0` restores the strict behaviour. The base/training image keeps-# the strict check: FastLanguageModel genuinely requires a GPU there.+# UNSLOTH_IMAGE_ALLOW_CPU=1 lets THIS image (:latest) start without a GPU. The entrypoint turns it into+# UNSLOTH_ALLOW_CPU=1 only when no GPU is visible, since that one disables the TRL patches. :core never opts in.ENV UNSLOTH_STUDIO_HOME=/opt/unsloth-studio \- UNSLOTH_ALLOW_CPU=1 \+ UNSLOTH_IMAGE_ALLOW_CPU=1 \JUPYTER_PORT=8888 \UNSLOTH_ENABLE_SSHD=false \DEBIAN_FRONTEND=noninteractive@@ -89,6 +82,8 @@ RUN apt-get update \# nvidia-*-cu12 wheels are byte-identical and the dedup below can symlink them.## fetch+checkout FETCH_HEAD, not `clone --branch`: CI passes a commit SHA.+#+# UNSLOTH_ALLOW_CPU=1 is inline for install.sh only: the build has no GPU, and the image ENV must not carry it (see above).RUN set -eux \&& case "${TARGETARCH:-amd64}" in \amd64|arm64) TORCH_FAMILY="cu128" ;; \@@ -108,6 +103,7 @@ RUN set -eux \UNSLOTH_LLAMA_TAG="${LLAMA_PREBUILT_TAG}" \UNSLOTH_LLAMA_KEEP_PREBUILT=1 \UNSLOTH_PYTHON=3.12 \+ UNSLOTH_ALLOW_CPU=1 \bash install.sh --local \# Fail loud unless the Studio venv torch EXACTLY matches the base (version AND# CUDA family) before the dedup symlinks their CUDA libs. Compare to the base's@@ -229,6 +225,11 @@ RUN chmod +x /usr/local/bin/unsloth-studio-launch \# Studio, JupyterLab, sshd. All bind 0.0.0.0 in the container; publish with -p.EXPOSE 8000 8888 22+# BASE_IMAGE defaults to the PUBLISHED :core, whose baked entrypoint predates+# UNSLOTH_IMAGE_ALLOW_CPU: without this copy a standalone build opts in above and still exits 1 on a CPU-only host.+COPY entrypoint.sh /usr/local/bin/unsloth-entrypoint+RUN chmod +x /usr/local/bin/unsloth-entrypoint+# The base ENTRYPOINT (unsloth-entrypoint) still runs its GPU pre-flight# first, then hands off to the service launcher.CMD ["/usr/local/bin/unsloth-studio-launch"]diff --git a/docker/entrypoint.sh b/docker/entrypoint.shindex 2da169502d7..8cd5ae8a667 100755--- a/docker/entrypoint.sh+++ b/docker/entrypoint.sh@@ -52,29 +52,52 @@ sync_notebooks() {fi}+err() { printf "\033[1;31mERROR:\033[0m %s\n" "$*" >&2; }+warn() { printf "\033[1;33mWARN:\033[0m %s\n" "$*" >&2; }++# nvidia-smi is injected on a GPU request, so a missing binary means "no GPU attached"+gpu_visible() {+ local listing+ # nvidia-smi -L ignores CUDA_VISIBLE_DEVICES but torch honours it: "" and -1 leave device_count() == 0, so they are no GPU+ case "${CUDA_VISIBLE_DEVICES-unset}" in+ ""|-1) return 1 ;;+ esac+ command -v nvidia-smi >/dev/null 2>&1 || return 1+ listing="$(nvidia-smi -L 2>/dev/null || true)"+ grep -q '^GPU' <<< "${listing}"+}++# The library reads UNSLOTH_ALLOW_CPU=1 as CPU-only CI and skips its TRL trainer patches, so it may only reach processes with no GPU.+# Images opt in via UNSLOTH_IMAGE_ALLOW_CPU; `-`, not `:-`, so an explicitly empty UNSLOTH_ALLOW_CPU means off.+allow_cpu="${UNSLOTH_ALLOW_CPU-${UNSLOTH_IMAGE_ALLOW_CPU:-0}}"+if gpu_visible; then+ has_gpu=1+ if [[ "${UNSLOTH_ALLOW_CPU:-}" == "1" ]]; then+ warn "Ignoring UNSLOTH_ALLOW_CPU=1: a GPU is visible, and the variable would turn off Unsloth's TRL trainer patches."+ warn "Drop it from docker run; shells opened with docker exec still see the value you passed."+ fi+ unset UNSLOTH_ALLOW_CPU+else+ has_gpu=0+ if [[ "${allow_cpu}" == "1" ]]; then+ export UNSLOTH_ALLOW_CPU=1+ fi+fi+if [[ "${UNSLOTH_SKIP_GPU_CHECK:-0}" == "1" ]]; thensync_notebooksexec "$@"fi-err() { printf "\033[1;31mERROR:\033[0m %s\n" "$*" >&2; }-warn() { printf "\033[1;33mWARN:\033[0m %s\n" "$*" >&2; }--# CPU mode covers Jupyter, GGUF tooling and Studio chat, but NOT training or loading-# a model. A visible GPU still runs the checks below.-if [[ "${UNSLOTH_ALLOW_CPU:-0}" == "1" ]]; then- if ! command -v nvidia-smi >/dev/null 2>&1 || ! nvidia-smi -L 2>/dev/null | grep -q '^GPU'; then- warn "UNSLOTH_ALLOW_CPU=1 and no GPU visible -- continuing on CPU."- warn "CPU mode covers Jupyter, GGUF tooling and llama.cpp (GGUF) Studio chat."- warn "Training and loading Unsloth models (FastLanguageModel) still require an NVIDIA GPU."- sync_notebooks- exec "$@"- fi+if [[ "${has_gpu}" == "0" && "${allow_cpu}" == "1" ]]; then+ warn "UNSLOTH_ALLOW_CPU=1 and no GPU visible -- continuing on CPU."+ warn "CPU mode covers Jupyter, GGUF tooling and llama.cpp (GGUF) Studio chat."+ warn "Training and loading Unsloth models (FastLanguageModel) still require an NVIDIA GPU."+ sync_notebooks+ exec "$@"fi-# nvidia-smi is injected on a GPU request, not baked in, so a missing binary means-# "no GPU attached", same as an empty -L-if ! command -v nvidia-smi >/dev/null 2>&1 || ! nvidia-smi -L 2>/dev/null | grep -q '^GPU'; then+if [[ "${has_gpu}" == "0" ]]; thenerr "No GPU visible inside the container."cat >&2 <<'MSG'diff --git a/docker/run.sh b/docker/run.shindex 771a6fc19f7..d590e341040 100755--- a/docker/run.sh+++ b/docker/run.sh@@ -61,10 +61,12 @@ mkdir -p "$HF_CACHE" "$TRITON_CACHE"# regression tests can stage a fake device tree; leave it unset in normal use.DEV_ROOT="${UNSLOTH_DEV_ROOT:-}"host_has_nvidia() {+ local listing[[ -e "$DEV_ROOT/dev/nvidiactl" ]] && return 0- command -v nvidia-smi >/dev/null 2>&1 \- && nvidia-smi -L 2>/dev/null | grep -q '^GPU' && return 0- return 1+ command -v nvidia-smi >/dev/null 2>&1 || return 1+ # buffer before grep -q: under pipefail the producer's SIGPIPE can become the pipeline status, which reads as "no GPU" and silently drops --gpus+ listing="$(nvidia-smi -L 2>/dev/null || true)"+ grep -q '^GPU' <<< "${listing}"}if [[ ${#GPU_FLAG[@]} -gt 0 ]] && ! host_has_nvidia; thendiff --git a/tests/python/test_docker_cpu_fallback.py b/tests/python/test_docker_cpu_fallback.pyindex 86945773f9a..90b756b89fd 100644--- a/tests/python/test_docker_cpu_fallback.py+++ b/tests/python/test_docker_cpu_fallback.py@@ -11,6 +11,7 @@ Two independent failure points, one per class below:"""import os+import reimport shutilimport statimport subprocess@@ -26,6 +27,13 @@ _BASE_DF = os.path.join(_DOCKER, "Dockerfile")_ENTRYPOINT = os.path.join(_DOCKER, "entrypoint.sh")+# Git Bash satisfies which("bash") on Windows but breaks on path translation and the exec bit; upstream only runs this file on ubuntu.+_posix_shell = pytest.mark.skipif(+ os.name != "posix" or shutil.which("bash") is None,+ reason = "POSIX shell required",+)++def _stub(path, body):with open(path, "w") as f:f.write("#!/usr/bin/env bash\n" + body)@@ -107,7 +115,7 @@ def _invoke_run_sh(return argv_log.read_text().splitlines(), proc.stderr[email protected](shutil.which("bash") is None, reason = "bash required")+@_posix_shellclass TestRunShDegradesWithoutNvidia:def test_gpus_flag_is_dropped_when_the_host_has_no_nvidia_gpu(self, tmp_path):"""`--gpus all` on an NVIDIA-less host is exit 125 AT THE DAEMON, so the@@ -157,16 +165,37 @@ class TestRunShDegradesWithoutNvidia:class TestStudioImageAllowsCpu:- def test_studio_image_defaults_allow_cpu_on(self):- """:latest is the Studio image. Without this default every CPU-only, AMD- and Docker-Desktop user goes from working Studio to an exit 1."""+ def test_studio_image_opts_in_through_its_own_variable(self):+ """:latest is the Studio image: without an opt-in every CPU-only, AMD and Docker-Desktop user gets an exit 1."""body = open(_STUDIO_DF, encoding = "utf-8").read()env_lines = [+ ln.strip()+ for ln in body.splitlines()+ if "UNSLOTH_IMAGE_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")+ ]+ assert env_lines, "Dockerfile.studio does not default UNSLOTH_IMAGE_ALLOW_CPU=1"++ def test_studio_image_env_never_carries_allow_cpu(self):+ """An image ENV reaches every process, so UNSLOTH_ALLOW_CPU=1 there broke training on GPU hosts. Only install.sh may see it, inline."""+ body = open(_STUDIO_DF, encoding = "utf-8").read()+ env_block = body[body.index("ENV UNSLOTH_STUDIO_HOME") :]+ env_block = env_block[: env_block.index("\n\n")]+ assert "UNSLOTH_ALLOW_CPU" not in env_block.replace("UNSLOTH_IMAGE_ALLOW_CPU", "")+ inline = [ln.strip()for ln in body.splitlines()if "UNSLOTH_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")]- assert env_lines, "Dockerfile.studio does not default UNSLOTH_ALLOW_CPU=1"+ assert inline == ["UNSLOTH_ALLOW_CPU=1 \\"], inline++ def test_the_studio_image_bundles_the_entrypoint_that_reads_its_opt_in(self):+ """The published :core entrypoint predates UNSLOTH_IMAGE_ALLOW_CPU, so without its own copy a standalone build opts in and still exits 1."""+ body = open(_STUDIO_DF, encoding = "utf-8").read()+ assert re.search(+ r"^COPY\s+entrypoint\.sh\s+/usr/local/bin/unsloth-entrypoint\s*$",+ body,+ re.M,+ ), "Dockerfile.studio does not bundle its own entrypoint"def test_the_base_training_image_keeps_the_strict_check(self):"""FastLanguageModel genuinely needs a GPU, so :core must NOT default it."""@@ -174,20 +203,156 @@ class TestStudioImageAllowsCpu:offenders = [ln.strip()for ln in body.splitlines()- if "UNSLOTH_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")+ if "ALLOW_CPU=1" in ln and not ln.strip().startswith("#")]assert not offenders, f"base image weakened the GPU check: {offenders}"- def test_entrypoint_reads_allow_cpu_from_the_environment(self):- """An image-level ENV and a `-e` flag reach the process identically, so the- entrypoint must read it from the environment with no `-e`-only handling."""- body = open(_ENTRYPOINT, encoding = "utf-8").read()- assert '"${UNSLOTH_ALLOW_CPU:-0}" == "1"' in body-- def test_allow_cpu_only_applies_when_no_gpu_is_visible(self):- """The default must not weaken a GPU host: the CPU branch has to be gated- on nvidia-smi finding nothing, otherwise it would skip the torch checks."""- body = open(_ENTRYPOINT, encoding = "utf-8").read()- idx = body.index('"${UNSLOTH_ALLOW_CPU:-0}" == "1"')- branch = body[idx : idx + 400]- assert "nvidia-smi" in branch and "grep -q '^GPU'" in branch++def _studio_image_env():+ """Read the image ENV from the Dockerfile: a test that spells the name itself would pass against an entrypoint that never heard of it."""+ body = open(_STUDIO_DF, encoding = "utf-8").read()+ env = {}+ for block in re.findall(r"^ENV\s+((?:.*\\\n)*.*)$", body, re.M):+ for key, value in re.findall(r"(\w*ALLOW_CPU)=(\S+)", block):+ env[key] = value.rstrip("\\").strip()+ return env+++def _run_entrypoint(tmp_path, *, gpu, env_extra):+ bindir = tmp_path / "bin"+ bindir.mkdir()+ if gpu:+ _stub(+ str(bindir / "nvidia-smi"),+ 'case "$*" in\n'+ ' *compute_cap*) echo "8.9" ;;\n'+ ' *driver_version*) echo "610.43.02" ;;\n'+ ' *) echo "GPU 0: NVIDIA RTX 6000 Ada Generation (UUID: GPU-abc)" ;;\n'+ "esac\n",+ )+ else:+ _stub(str(bindir / "nvidia-smi"), "exit 1\n")+ # the GPU path runs two torch heredocs; accept them without torch+ _stub(str(bindir / "python"), "cat > /dev/null\nexit 0\n")+ dump = tmp_path / "child_env"+ env = {+ "PATH": str(bindir) + ":/usr/bin:/bin",+ "HOME": str(tmp_path),+ "UNSLOTH_STUDIO_HOME": str(tmp_path / "studio"),+ }+ env.update(env_extra)+ proc = subprocess.run(+ [shutil.which("bash") or "/bin/bash", _ENTRYPOINT, "bash", "-c", f"env > {dump}"],+ env = env,+ capture_output = True,+ text = True,+ timeout = 60,+ )+ child = {}+ if dump.exists():+ for line in dump.read_text().splitlines():+ key, _, value = line.partition("=")+ child[key] = value+ return proc.returncode, child, proc.stderr+++@_posix_shell+class TestEntrypointAllowCpu:+ def test_studio_image_on_a_cpu_host_starts_and_exports_allow_cpu(self, tmp_path):+ """Studio's own processes need UNSLOTH_ALLOW_CPU=1 to import unsloth without a GPU."""+ rc, child, stderr = _run_entrypoint(+ tmp_path, gpu = False, env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1"}+ )+ assert rc == 0, stderr+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"+ assert "continuing on CPU" in stderr++ def test_studio_image_on_a_gpu_host_hides_allow_cpu(self, tmp_path):+ """The regression: children that see it train on stock TRL. Env comes from the Dockerfile, so image and entrypoint are exercised as a PAIR."""+ image_env = _studio_image_env()+ assert image_env, "Dockerfile.studio bakes no *ALLOW_CPU image ENV"+ rc, child, stderr = _run_entrypoint(tmp_path, gpu = True, env_extra = image_env)+ assert rc == 0, stderr+ assert "UNSLOTH_ALLOW_CPU" not in child++ def test_an_explicit_allow_cpu_on_a_gpu_host_is_dropped_with_a_warning(self, tmp_path):+ """run.sh forwards a host-shell UNSLOTH_ALLOW_CPU and the docs tell CPU users to pass it, so a GPU host can receive it too."""+ rc, child, stderr = _run_entrypoint(+ tmp_path, gpu = True, env_extra = {"UNSLOTH_ALLOW_CPU": "1"}+ )+ assert rc == 0, stderr+ assert "UNSLOTH_ALLOW_CPU" not in child+ assert "Ignoring UNSLOTH_ALLOW_CPU=1" in stderr++ def test_an_explicit_zero_restores_the_strict_check(self, tmp_path):+ rc, _, stderr = _run_entrypoint(+ tmp_path,+ gpu = False,+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_ALLOW_CPU": "0"},+ )+ assert rc == 1+ assert "No GPU visible" in stderr++ def test_core_without_an_opt_in_still_refuses(self, tmp_path):+ rc, _, stderr = _run_entrypoint(tmp_path, gpu = False, env_extra = {})+ assert rc == 1+ assert "No GPU visible" in stderr++ def test_core_with_an_explicit_opt_in_starts_on_cpu(self, tmp_path):+ rc, child, stderr = _run_entrypoint(+ tmp_path, gpu = False, env_extra = {"UNSLOTH_ALLOW_CPU": "1"}+ )+ assert rc == 0, stderr+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"++ @pytest.mark.parametrize("gpu", [True, False])+ def test_skipping_the_gpu_check_applies_the_same_rule(self, tmp_path, gpu):+ rc, child, stderr = _run_entrypoint(+ tmp_path,+ gpu = gpu,+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_SKIP_GPU_CHECK": "1"},+ )+ assert rc == 0, stderr+ if gpu:+ assert "UNSLOTH_ALLOW_CPU" not in child+ else:+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"++ def test_skipping_the_gpu_check_does_not_invent_an_opt_in(self, tmp_path):+ """The only way to reach exec on a CPU host with no opt-in, so the only place an unconditional export would hide and disable the TRL patches."""+ rc, child, stderr = _run_entrypoint(+ tmp_path, gpu = False, env_extra = {"UNSLOTH_SKIP_GPU_CHECK": "1"}+ )+ assert rc == 0, stderr+ assert "UNSLOTH_ALLOW_CPU" not in child++ @pytest.mark.parametrize("mask", ["", "-1"])+ def test_cuda_masked_to_nothing_counts_as_no_gpu(self, tmp_path, mask):+ """nvidia-smi -L ignores CUDA_VISIBLE_DEVICES but torch honours it, so the GPU path would strip the opt-in from a process with no device."""+ rc, child, stderr = _run_entrypoint(+ tmp_path,+ gpu = True,+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "CUDA_VISIBLE_DEVICES": mask},+ )+ assert rc == 0, stderr+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"+ assert "continuing on CPU" in stderr++ def test_a_real_device_mask_is_still_a_gpu(self, tmp_path):+ rc, child, stderr = _run_entrypoint(+ tmp_path,+ gpu = True,+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "CUDA_VISIBLE_DEVICES": "0"},+ )+ assert rc == 0, stderr+ assert "UNSLOTH_ALLOW_CPU" not in child++ def test_an_explicit_empty_value_is_not_an_opt_in(self, tmp_path):+ """`-e UNSLOTH_ALLOW_CPU=` means off; it must not fall through to the image opt-in, which `:-` would do."""+ rc, _, stderr = _run_entrypoint(+ tmp_path,+ gpu = False,+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_ALLOW_CPU": ""},+ )+ assert rc == 1+ assert "No GPU visible" in stderr