Skip to content

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

Docker Studio: stop disabling Unsloth's TRL patches on GPU hosts (#10825)

oobabooga4e926dba5001
diff --git a/docker/DOCKERHUB.md b/docker/DOCKERHUB.md
index e2dadd3489e..b51841f2a89 100644
--- a/docker/DOCKERHUB.md
+++ b/docker/DOCKERHUB.md
@@ -101,11 +101,16 @@ Turing has no bfloat16; Unsloth falls back to float16 there. AMD GPUs are not su
| `JUPYTER_PASSWORD` | JupyterLab password. Unset: generated once and printed in the logs. |
| `JUPYTER_PORT` | JupyterLab port inside the container. Default `8888`. |
| `SSH_KEY` or `PUBLIC_KEY` | OpenSSH public key for root login. Enables sshd on port 22. Password login is never enabled. |
-| `UNSLOTH_ALLOW_CPU=1` | Allow starting without a GPU. |
+| `UNSLOTH_ALLOW_CPU=1` | Allow starting without a GPU (`latest` already does). Dropped when a GPU is visible, where it would turn off Unsloth's training patches. |
| `UNSLOTH_JUPYTER_CLOUDFLARE=1` | Publish JupyterLab through a Cloudflare quick tunnel and print the URL. |
| `UNSLOTH_SKIP_NOTEBOOK_SYNC=1` | Do not refresh the notebooks from GitHub on start. |
| `HF_TOKEN`, `WANDB_API_KEY` | Forwarded to Hugging Face and Weights and Biases. |
+On a host with no GPU, Studio, JupyterLab and its kernels, and login shells all run in
+CPU mode. A non-login `docker exec` is built from the image, not the container's first
+process, so `docker exec <c> python -c "import unsloth"` still reports that it needs a
+GPU: use `docker exec <c> bash -lc '...'` or `docker exec -e UNSLOTH_ALLOW_CPU=1 <c> ...`.
+
## Volumes
The working directory is `/workspace`. Mount what you want to keep:
diff --git a/docker/Dockerfile.studio b/docker/Dockerfile.studio
index 05570927a1e..54e747a7a51 100644
--- a/docker/Dockerfile.studio
+++ b/docker/Dockerfile.studio
@@ -11,8 +11,7 @@
#
# Studio on :8000 (user unsloth; UNSLOTH_STUDIO_PASSWORD env, else a generated one
# is printed in the logs; persisted under /opt/unsloth-studio/auth/); JupyterLab on
-# :8888 (JUPYTER_PASSWORD env, else a random one is printed). Without GPU passthrough add -e UNSLOTH_ALLOW_CPU=1:
-# training is unavailable but Studio chat / Data Recipes / GGUF / Jupyter work.
+# :8888 (JUPYTER_PASSWORD env, else a random one is printed). Without a GPU it starts on CPU: no training, but Studio chat / Data Recipes / GGUF / Jupyter work.
# CI pins BASE_IMAGE to the published base digest so both images ship the same stack.
ARG BASE_IMAGE=unsloth/unsloth:core
@@ -51,16 +50,10 @@ ARG TARGETARCH
# disabled unless PUBLIC_KEY/SSH_KEY is set (see studio_launch.sh). The
# JUPYTER_PORT / UNSLOTH_ENABLE_SSHD defaults let supervisord's %(ENV_*)s resolve.
USER root
-# UNSLOTH_ALLOW_CPU defaults to 1 on THIS image only. It publishes as :latest, so a
-# plain `docker run unsloth/unsloth` on a CPU-only, AMD or Docker-Desktop host would
-# otherwise exit 1 from entrypoint.sh before Studio ever starts. Most of what this
-# image offers (Studio chat over llama.cpp/GGUF, Data Recipes, JupyterLab, GGUF
-# tooling) needs no GPU. The entrypoint only takes the CPU path when no GPU is
-# visible, so this does not weaken the checks on a GPU host, and `-e
-# UNSLOTH_ALLOW_CPU=0` restores the strict behaviour. The base/training image keeps
-# the strict check: FastLanguageModel genuinely requires a GPU there.
+# UNSLOTH_IMAGE_ALLOW_CPU=1 lets THIS image (:latest) start without a GPU. The entrypoint turns it into
+# UNSLOTH_ALLOW_CPU=1 only when no GPU is visible, since that one disables the TRL patches. :core never opts in.
ENV UNSLOTH_STUDIO_HOME=/opt/unsloth-studio \
- UNSLOTH_ALLOW_CPU=1 \
+ UNSLOTH_IMAGE_ALLOW_CPU=1 \
JUPYTER_PORT=8888 \
UNSLOTH_ENABLE_SSHD=false \
DEBIAN_FRONTEND=noninteractive
@@ -89,6 +82,8 @@ RUN apt-get update \
# nvidia-*-cu12 wheels are byte-identical and the dedup below can symlink them.
#
# fetch+checkout FETCH_HEAD, not `clone --branch`: CI passes a commit SHA.
+#
+# UNSLOTH_ALLOW_CPU=1 is inline for install.sh only: the build has no GPU, and the image ENV must not carry it (see above).
RUN set -eux \
&& case "${TARGETARCH:-amd64}" in \
amd64|arm64) TORCH_FAMILY="cu128" ;; \
@@ -108,6 +103,7 @@ RUN set -eux \
UNSLOTH_LLAMA_TAG="${LLAMA_PREBUILT_TAG}" \
UNSLOTH_LLAMA_KEEP_PREBUILT=1 \
UNSLOTH_PYTHON=3.12 \
+ UNSLOTH_ALLOW_CPU=1 \
bash install.sh --local \
# Fail loud unless the Studio venv torch EXACTLY matches the base (version AND
# CUDA family) before the dedup symlinks their CUDA libs. Compare to the base's
@@ -229,6 +225,11 @@ RUN chmod +x /usr/local/bin/unsloth-studio-launch \
# Studio, JupyterLab, sshd. All bind 0.0.0.0 in the container; publish with -p.
EXPOSE 8000 8888 22
+# BASE_IMAGE defaults to the PUBLISHED :core, whose baked entrypoint predates
+# UNSLOTH_IMAGE_ALLOW_CPU: without this copy a standalone build opts in above and still exits 1 on a CPU-only host.
+COPY entrypoint.sh /usr/local/bin/unsloth-entrypoint
+RUN chmod +x /usr/local/bin/unsloth-entrypoint
+
# The base ENTRYPOINT (unsloth-entrypoint) still runs its GPU pre-flight
# first, then hands off to the service launcher.
CMD ["/usr/local/bin/unsloth-studio-launch"]
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
index 2da169502d7..8cd5ae8a667 100755
--- a/docker/entrypoint.sh
+++ b/docker/entrypoint.sh
@@ -52,29 +52,52 @@ sync_notebooks() {
fi
}
+err() { printf "\033[1;31mERROR:\033[0m %s\n" "$*" >&2; }
+warn() { printf "\033[1;33mWARN:\033[0m %s\n" "$*" >&2; }
+
+# nvidia-smi is injected on a GPU request, so a missing binary means "no GPU attached"
+gpu_visible() {
+ local listing
+ # nvidia-smi -L ignores CUDA_VISIBLE_DEVICES but torch honours it: "" and -1 leave device_count() == 0, so they are no GPU
+ case "${CUDA_VISIBLE_DEVICES-unset}" in
+ ""|-1) return 1 ;;
+ esac
+ command -v nvidia-smi >/dev/null 2>&1 || return 1
+ listing="$(nvidia-smi -L 2>/dev/null || true)"
+ grep -q '^GPU' <<< "${listing}"
+}
+
+# The library reads UNSLOTH_ALLOW_CPU=1 as CPU-only CI and skips its TRL trainer patches, so it may only reach processes with no GPU.
+# Images opt in via UNSLOTH_IMAGE_ALLOW_CPU; `-`, not `:-`, so an explicitly empty UNSLOTH_ALLOW_CPU means off.
+allow_cpu="${UNSLOTH_ALLOW_CPU-${UNSLOTH_IMAGE_ALLOW_CPU:-0}}"
+if gpu_visible; then
+ has_gpu=1
+ if [[ "${UNSLOTH_ALLOW_CPU:-}" == "1" ]]; then
+ warn "Ignoring UNSLOTH_ALLOW_CPU=1: a GPU is visible, and the variable would turn off Unsloth's TRL trainer patches."
+ warn "Drop it from docker run; shells opened with docker exec still see the value you passed."
+ fi
+ unset UNSLOTH_ALLOW_CPU
+else
+ has_gpu=0
+ if [[ "${allow_cpu}" == "1" ]]; then
+ export UNSLOTH_ALLOW_CPU=1
+ fi
+fi
+
if [[ "${UNSLOTH_SKIP_GPU_CHECK:-0}" == "1" ]]; then
sync_notebooks
exec "$@"
fi
-err() { printf "\033[1;31mERROR:\033[0m %s\n" "$*" >&2; }
-warn() { printf "\033[1;33mWARN:\033[0m %s\n" "$*" >&2; }
-
-# CPU mode covers Jupyter, GGUF tooling and Studio chat, but NOT training or loading
-# a model. A visible GPU still runs the checks below.
-if [[ "${UNSLOTH_ALLOW_CPU:-0}" == "1" ]]; then
- if ! command -v nvidia-smi >/dev/null 2>&1 || ! nvidia-smi -L 2>/dev/null | grep -q '^GPU'; then
- warn "UNSLOTH_ALLOW_CPU=1 and no GPU visible -- continuing on CPU."
- warn "CPU mode covers Jupyter, GGUF tooling and llama.cpp (GGUF) Studio chat."
- warn "Training and loading Unsloth models (FastLanguageModel) still require an NVIDIA GPU."
- sync_notebooks
- exec "$@"
- fi
+if [[ "${has_gpu}" == "0" && "${allow_cpu}" == "1" ]]; then
+ warn "UNSLOTH_ALLOW_CPU=1 and no GPU visible -- continuing on CPU."
+ warn "CPU mode covers Jupyter, GGUF tooling and llama.cpp (GGUF) Studio chat."
+ warn "Training and loading Unsloth models (FastLanguageModel) still require an NVIDIA GPU."
+ sync_notebooks
+ exec "$@"
fi
-# nvidia-smi is injected on a GPU request, not baked in, so a missing binary means
-# "no GPU attached", same as an empty -L
-if ! command -v nvidia-smi >/dev/null 2>&1 || ! nvidia-smi -L 2>/dev/null | grep -q '^GPU'; then
+if [[ "${has_gpu}" == "0" ]]; then
err "No GPU visible inside the container."
cat >&2 <<'MSG'
diff --git a/docker/run.sh b/docker/run.sh
index 771a6fc19f7..d590e341040 100755
--- a/docker/run.sh
+++ b/docker/run.sh
@@ -61,10 +61,12 @@ mkdir -p "$HF_CACHE" "$TRITON_CACHE"
# regression tests can stage a fake device tree; leave it unset in normal use.
DEV_ROOT="${UNSLOTH_DEV_ROOT:-}"
host_has_nvidia() {
+ local listing
[[ -e "$DEV_ROOT/dev/nvidiactl" ]] && return 0
- command -v nvidia-smi >/dev/null 2>&1 \
- && nvidia-smi -L 2>/dev/null | grep -q '^GPU' && return 0
- return 1
+ command -v nvidia-smi >/dev/null 2>&1 || return 1
+ # buffer before grep -q: under pipefail the producer's SIGPIPE can become the pipeline status, which reads as "no GPU" and silently drops --gpus
+ listing="$(nvidia-smi -L 2>/dev/null || true)"
+ grep -q '^GPU' <<< "${listing}"
}
if [[ ${#GPU_FLAG[@]} -gt 0 ]] && ! host_has_nvidia; then
diff --git a/tests/python/test_docker_cpu_fallback.py b/tests/python/test_docker_cpu_fallback.py
index 86945773f9a..90b756b89fd 100644
--- a/tests/python/test_docker_cpu_fallback.py
+++ b/tests/python/test_docker_cpu_fallback.py
@@ -11,6 +11,7 @@ Two independent failure points, one per class below:
"""
import os
+import re
import shutil
import stat
import subprocess
@@ -26,6 +27,13 @@ _BASE_DF = os.path.join(_DOCKER, "Dockerfile")
_ENTRYPOINT = os.path.join(_DOCKER, "entrypoint.sh")
+# Git Bash satisfies which("bash") on Windows but breaks on path translation and the exec bit; upstream only runs this file on ubuntu.
+_posix_shell = pytest.mark.skipif(
+ os.name != "posix" or shutil.which("bash") is None,
+ reason = "POSIX shell required",
+)
+
+
def _stub(path, body):
with open(path, "w") as f:
f.write("#!/usr/bin/env bash\n" + body)
@@ -107,7 +115,7 @@ def _invoke_run_sh(
return argv_log.read_text().splitlines(), proc.stderr
[email protected](shutil.which("bash") is None, reason = "bash required")
+@_posix_shell
class TestRunShDegradesWithoutNvidia:
def test_gpus_flag_is_dropped_when_the_host_has_no_nvidia_gpu(self, tmp_path):
"""`--gpus all` on an NVIDIA-less host is exit 125 AT THE DAEMON, so the
@@ -157,16 +165,37 @@ class TestRunShDegradesWithoutNvidia:
class TestStudioImageAllowsCpu:
- def test_studio_image_defaults_allow_cpu_on(self):
- """:latest is the Studio image. Without this default every CPU-only, AMD
- and Docker-Desktop user goes from working Studio to an exit 1."""
+ def test_studio_image_opts_in_through_its_own_variable(self):
+ """:latest is the Studio image: without an opt-in every CPU-only, AMD and Docker-Desktop user gets an exit 1."""
body = open(_STUDIO_DF, encoding = "utf-8").read()
env_lines = [
+ ln.strip()
+ for ln in body.splitlines()
+ if "UNSLOTH_IMAGE_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")
+ ]
+ assert env_lines, "Dockerfile.studio does not default UNSLOTH_IMAGE_ALLOW_CPU=1"
+
+ def test_studio_image_env_never_carries_allow_cpu(self):
+ """An image ENV reaches every process, so UNSLOTH_ALLOW_CPU=1 there broke training on GPU hosts. Only install.sh may see it, inline."""
+ body = open(_STUDIO_DF, encoding = "utf-8").read()
+ env_block = body[body.index("ENV UNSLOTH_STUDIO_HOME") :]
+ env_block = env_block[: env_block.index("\n\n")]
+ assert "UNSLOTH_ALLOW_CPU" not in env_block.replace("UNSLOTH_IMAGE_ALLOW_CPU", "")
+ inline = [
ln.strip()
for ln in body.splitlines()
if "UNSLOTH_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")
]
- assert env_lines, "Dockerfile.studio does not default UNSLOTH_ALLOW_CPU=1"
+ assert inline == ["UNSLOTH_ALLOW_CPU=1 \\"], inline
+
+ def test_the_studio_image_bundles_the_entrypoint_that_reads_its_opt_in(self):
+ """The published :core entrypoint predates UNSLOTH_IMAGE_ALLOW_CPU, so without its own copy a standalone build opts in and still exits 1."""
+ body = open(_STUDIO_DF, encoding = "utf-8").read()
+ assert re.search(
+ r"^COPY\s+entrypoint\.sh\s+/usr/local/bin/unsloth-entrypoint\s*$",
+ body,
+ re.M,
+ ), "Dockerfile.studio does not bundle its own entrypoint"
def test_the_base_training_image_keeps_the_strict_check(self):
"""FastLanguageModel genuinely needs a GPU, so :core must NOT default it."""
@@ -174,20 +203,156 @@ class TestStudioImageAllowsCpu:
offenders = [
ln.strip()
for ln in body.splitlines()
- if "UNSLOTH_ALLOW_CPU=1" in ln and not ln.strip().startswith("#")
+ if "ALLOW_CPU=1" in ln and not ln.strip().startswith("#")
]
assert not offenders, f"base image weakened the GPU check: {offenders}"
- def test_entrypoint_reads_allow_cpu_from_the_environment(self):
- """An image-level ENV and a `-e` flag reach the process identically, so the
- entrypoint must read it from the environment with no `-e`-only handling."""
- body = open(_ENTRYPOINT, encoding = "utf-8").read()
- assert '"${UNSLOTH_ALLOW_CPU:-0}" == "1"' in body
-
- def test_allow_cpu_only_applies_when_no_gpu_is_visible(self):
- """The default must not weaken a GPU host: the CPU branch has to be gated
- on nvidia-smi finding nothing, otherwise it would skip the torch checks."""
- body = open(_ENTRYPOINT, encoding = "utf-8").read()
- idx = body.index('"${UNSLOTH_ALLOW_CPU:-0}" == "1"')
- branch = body[idx : idx + 400]
- assert "nvidia-smi" in branch and "grep -q '^GPU'" in branch
+
+def _studio_image_env():
+ """Read the image ENV from the Dockerfile: a test that spells the name itself would pass against an entrypoint that never heard of it."""
+ body = open(_STUDIO_DF, encoding = "utf-8").read()
+ env = {}
+ for block in re.findall(r"^ENV\s+((?:.*\\\n)*.*)$", body, re.M):
+ for key, value in re.findall(r"(\w*ALLOW_CPU)=(\S+)", block):
+ env[key] = value.rstrip("\\").strip()
+ return env
+
+
+def _run_entrypoint(tmp_path, *, gpu, env_extra):
+ bindir = tmp_path / "bin"
+ bindir.mkdir()
+ if gpu:
+ _stub(
+ str(bindir / "nvidia-smi"),
+ 'case "$*" in\n'
+ ' *compute_cap*) echo "8.9" ;;\n'
+ ' *driver_version*) echo "610.43.02" ;;\n'
+ ' *) echo "GPU 0: NVIDIA RTX 6000 Ada Generation (UUID: GPU-abc)" ;;\n'
+ "esac\n",
+ )
+ else:
+ _stub(str(bindir / "nvidia-smi"), "exit 1\n")
+ # the GPU path runs two torch heredocs; accept them without torch
+ _stub(str(bindir / "python"), "cat > /dev/null\nexit 0\n")
+ dump = tmp_path / "child_env"
+ env = {
+ "PATH": str(bindir) + ":/usr/bin:/bin",
+ "HOME": str(tmp_path),
+ "UNSLOTH_STUDIO_HOME": str(tmp_path / "studio"),
+ }
+ env.update(env_extra)
+ proc = subprocess.run(
+ [shutil.which("bash") or "/bin/bash", _ENTRYPOINT, "bash", "-c", f"env > {dump}"],
+ env = env,
+ capture_output = True,
+ text = True,
+ timeout = 60,
+ )
+ child = {}
+ if dump.exists():
+ for line in dump.read_text().splitlines():
+ key, _, value = line.partition("=")
+ child[key] = value
+ return proc.returncode, child, proc.stderr
+
+
+@_posix_shell
+class TestEntrypointAllowCpu:
+ def test_studio_image_on_a_cpu_host_starts_and_exports_allow_cpu(self, tmp_path):
+ """Studio's own processes need UNSLOTH_ALLOW_CPU=1 to import unsloth without a GPU."""
+ rc, child, stderr = _run_entrypoint(
+ tmp_path, gpu = False, env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1"}
+ )
+ assert rc == 0, stderr
+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"
+ assert "continuing on CPU" in stderr
+
+ def test_studio_image_on_a_gpu_host_hides_allow_cpu(self, tmp_path):
+ """The regression: children that see it train on stock TRL. Env comes from the Dockerfile, so image and entrypoint are exercised as a PAIR."""
+ image_env = _studio_image_env()
+ assert image_env, "Dockerfile.studio bakes no *ALLOW_CPU image ENV"
+ rc, child, stderr = _run_entrypoint(tmp_path, gpu = True, env_extra = image_env)
+ assert rc == 0, stderr
+ assert "UNSLOTH_ALLOW_CPU" not in child
+
+ def test_an_explicit_allow_cpu_on_a_gpu_host_is_dropped_with_a_warning(self, tmp_path):
+ """run.sh forwards a host-shell UNSLOTH_ALLOW_CPU and the docs tell CPU users to pass it, so a GPU host can receive it too."""
+ rc, child, stderr = _run_entrypoint(
+ tmp_path, gpu = True, env_extra = {"UNSLOTH_ALLOW_CPU": "1"}
+ )
+ assert rc == 0, stderr
+ assert "UNSLOTH_ALLOW_CPU" not in child
+ assert "Ignoring UNSLOTH_ALLOW_CPU=1" in stderr
+
+ def test_an_explicit_zero_restores_the_strict_check(self, tmp_path):
+ rc, _, stderr = _run_entrypoint(
+ tmp_path,
+ gpu = False,
+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_ALLOW_CPU": "0"},
+ )
+ assert rc == 1
+ assert "No GPU visible" in stderr
+
+ def test_core_without_an_opt_in_still_refuses(self, tmp_path):
+ rc, _, stderr = _run_entrypoint(tmp_path, gpu = False, env_extra = {})
+ assert rc == 1
+ assert "No GPU visible" in stderr
+
+ def test_core_with_an_explicit_opt_in_starts_on_cpu(self, tmp_path):
+ rc, child, stderr = _run_entrypoint(
+ tmp_path, gpu = False, env_extra = {"UNSLOTH_ALLOW_CPU": "1"}
+ )
+ assert rc == 0, stderr
+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"
+
+ @pytest.mark.parametrize("gpu", [True, False])
+ def test_skipping_the_gpu_check_applies_the_same_rule(self, tmp_path, gpu):
+ rc, child, stderr = _run_entrypoint(
+ tmp_path,
+ gpu = gpu,
+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_SKIP_GPU_CHECK": "1"},
+ )
+ assert rc == 0, stderr
+ if gpu:
+ assert "UNSLOTH_ALLOW_CPU" not in child
+ else:
+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"
+
+ def test_skipping_the_gpu_check_does_not_invent_an_opt_in(self, tmp_path):
+ """The only way to reach exec on a CPU host with no opt-in, so the only place an unconditional export would hide and disable the TRL patches."""
+ rc, child, stderr = _run_entrypoint(
+ tmp_path, gpu = False, env_extra = {"UNSLOTH_SKIP_GPU_CHECK": "1"}
+ )
+ assert rc == 0, stderr
+ assert "UNSLOTH_ALLOW_CPU" not in child
+
+ @pytest.mark.parametrize("mask", ["", "-1"])
+ def test_cuda_masked_to_nothing_counts_as_no_gpu(self, tmp_path, mask):
+ """nvidia-smi -L ignores CUDA_VISIBLE_DEVICES but torch honours it, so the GPU path would strip the opt-in from a process with no device."""
+ rc, child, stderr = _run_entrypoint(
+ tmp_path,
+ gpu = True,
+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "CUDA_VISIBLE_DEVICES": mask},
+ )
+ assert rc == 0, stderr
+ assert child.get("UNSLOTH_ALLOW_CPU") == "1"
+ assert "continuing on CPU" in stderr
+
+ def test_a_real_device_mask_is_still_a_gpu(self, tmp_path):
+ rc, child, stderr = _run_entrypoint(
+ tmp_path,
+ gpu = True,
+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "CUDA_VISIBLE_DEVICES": "0"},
+ )
+ assert rc == 0, stderr
+ assert "UNSLOTH_ALLOW_CPU" not in child
+
+ def test_an_explicit_empty_value_is_not_an_opt_in(self, tmp_path):
+ """`-e UNSLOTH_ALLOW_CPU=` means off; it must not fall through to the image opt-in, which `:-` would do."""
+ rc, _, stderr = _run_entrypoint(
+ tmp_path,
+ gpu = False,
+ env_extra = {"UNSLOTH_IMAGE_ALLOW_CPU": "1", "UNSLOTH_ALLOW_CPU": ""},
+ )
+ assert rc == 1
+ assert "No GPU visible" in stderr