Skip to content

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

docker: stop the toolkit installer reporting a healthy driver as missing (#11106)

Daniel Hand6dd50c322ef
diff --git a/docker/DOCKERHUB-ROCM.md b/docker/DOCKERHUB-ROCM.md
index c106411ba39..3a0a8eaa056 100644
--- a/docker/DOCKERHUB-ROCM.md
+++ b/docker/DOCKERHUB-ROCM.md
@@ -6,6 +6,12 @@ This is the AMD counterpart to [`unsloth/unsloth`](https://hub.docker.com/r/unsl
## Quick start
+Needs Docker and a working amdgpu driver. Without Docker, start here:
+
+```bash
+curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
+```
+
AMD GPUs are reached through the kernel driver's device nodes, not through a container toolkit, so the run command differs from the NVIDIA one:
```bash
diff --git a/docker/DOCKERHUB.md b/docker/DOCKERHUB.md
index 75723b01b56..eda37c231f0 100644
--- a/docker/DOCKERHUB.md
+++ b/docker/DOCKERHUB.md
@@ -17,7 +17,13 @@ Source: [`docker/`](https://github.com/unslothai/unsloth/tree/main/docker) in th
## Quick start
-Needs an NVIDIA driver of 570.26 or newer and, on Linux, the NVIDIA Container Toolkit. One command installs the toolkit (Ubuntu, Debian, RHEL, Fedora, Rocky, Amazon Linux, SUSE) and checks a container can see the GPU:
+Needs an NVIDIA driver of 570.26 or newer and, on Linux, Docker plus the NVIDIA Container Toolkit. Without Docker, start here:
+
+```bash
+curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
+```
+
+Then one command installs the toolkit (Ubuntu, Debian, RHEL, Fedora, Rocky, Amazon Linux, SUSE) and checks a container can see the GPU:
```bash
curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh
@@ -39,11 +45,11 @@ docker run -d --gpus all --ipc=host \
`docker run -d` returns at once; follow the startup with `docker logs -f <container>`, which ends with a ready block once both services answer (Studio takes about a minute). Then open Studio at `http://localhost:8000` (user `unsloth`) and JupyterLab at `http://localhost:8888`. Leave either password variable unset and a random one is generated and printed in that log.
-The `docker/run.sh` helper in the repository sets these flags, including the `unsloth-studio` volume, for you:
+Or let `run.sh` set these flags, including the `unsloth-studio` volume, for you. It offers to install the NVIDIA Container Toolkit if the daemon has no `nvidia` runtime:
```bash
-git clone https://github.com/unslothai/unsloth && cd unsloth
-UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash docker/run.sh
+curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/run.sh -o run.sh
+UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash run.sh
```
### Notebooks only (`core`)
diff --git a/docker/install_nvidia_toolkit.sh b/docker/install_nvidia_toolkit.sh
index 0d006afa96f..ebf8b5803a8 100755
--- a/docker/install_nvidia_toolkit.sh
+++ b/docker/install_nvidia_toolkit.sh
@@ -16,7 +16,9 @@ say() { printf '%s\n' "$*"; }
fail() { printf 'ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
command -v docker >/dev/null 2>&1 \
- || fail "docker is not installed. Install Docker Engine first (https://docs.docker.com/engine/install/), then run this again." 2
+ || fail "docker is not installed. Install Docker Engine first, on Linux with
+ curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
+ (https://docs.docker.com/engine/install/ covers Docker Desktop and the distro packages), then run this again." 2
# Every grep of a command's output below reads a variable, not a pipe. `grep -q` exits on the first
# match and closes the pipe; `set -o pipefail` above then promotes the producer's SIGPIPE (141) to the
@@ -127,9 +129,42 @@ fi
# On WSL 2 the Windows driver puts nvidia-smi under /usr/lib/wsl/lib, which sudo's secure_path drops from PATH.
NVSMI="$(command -v nvidia-smi 2>/dev/null || true)"
[[ -z "$NVSMI" && -x "${WSL_LIB_DIR}/nvidia-smi" ]] && NVSMI="${WSL_LIB_DIR}/nvidia-smi"
+
+# sudo strips every LD_* variable unconditionally, -E or not, as an anti-preload
+# measure. A host that keeps the NVIDIA userspace libraries outside ldconfig and
+# reaches them through LD_LIBRARY_PATH therefore loses them the moment this script
+# elevates: nvidia-smi is on PATH, runs, and exits with "couldn't find
+# libnvidia-ml.so", which carries no GPU line and used to be reported here as a
+# missing driver. Measured on Colab (T4, driver 580.82.07): libs in
+# /usr/lib64-nvidia, absent from `ldconfig -p`, LD_LIBRARY_PATH set for the user
+# and UNSET under sudo. Put the directory back when it is the only thing wrong.
+nvsmi() { LD_LIBRARY_PATH="${NV_LIB_DIR}${NV_LIB_DIR:+${LD_LIBRARY_PATH:+:}}${LD_LIBRARY_PATH:-}" "$NVSMI" "$@"; }
+NV_LIB_DIR=""
gpu_list=""
-[[ -n "$NVSMI" ]] && gpu_list="$("$NVSMI" -L 2>/dev/null || true)"
+# 2>&1, not 2>/dev/null: the loader's complaint is the evidence that separates a
+# missing driver from a driver this shell cannot reach, and it goes to stderr.
+[[ -n "$NVSMI" ]] && gpu_list="$(nvsmi -L 2>&1 || true)"
+if [[ -n "$NVSMI" ]] && grep -qi 'libnvidia-ml' <<<"$gpu_list"; then
+ for _dir in /usr/lib64-nvidia /usr/local/nvidia/lib64 "$WSL_LIB_DIR"; do
+ [[ -e "${DESTDIR}${_dir}/libnvidia-ml.so.1" ]] || continue
+ NV_LIB_DIR="${DESTDIR}${_dir}"
+ gpu_list="$(nvsmi -L 2>&1 || true)"
+ grep -q '^GPU' <<<"$gpu_list" && break
+ NV_LIB_DIR=""
+ done
+fi
if [[ -z "$NVSMI" ]] || ! grep -q '^GPU' <<<"$gpu_list"; then
+ # Still a library complaint after the search: the driver is there and only the
+ # loader cannot see it, so saying "no driver" would send the reader to install
+ # one they already have.
+ if grep -qi 'libnvidia-ml' <<<"$gpu_list"; then
+ fail "nvidia-smi cannot load libnvidia-ml.so, so this script cannot read the driver:
+ ${gpu_list%%$'\n'*}
+ sudo drops every LD_* variable, so a driver reached through LD_LIBRARY_PATH (Colab
+ puts its libraries in /usr/lib64-nvidia) disappears when this script elevates.
+ Find the directory holding libnvidia-ml.so.1 and name it on the command line:
+ sudo LD_LIBRARY_PATH=/usr/lib64-nvidia bash install_nvidia_toolkit.sh" 2
+ fi
if grep -qi microsoft "$PROC_VERSION" 2>/dev/null; then
fail "no NVIDIA GPU is visible in this WSL 2 distro. Install a current NVIDIA Windows driver
from nvidia.com (never a Linux driver inside WSL), restart WSL (wsl --shutdown), then run
@@ -144,7 +179,7 @@ MIN_DRIVER=570.26
# `sed -n 1p`, not `head -1`: head closes the pipe after one line, and nvidia-smi prints one per GPU,
# so on a multi-GPU host the SIGPIPE becomes the substitution's status and `set -e` kills the script
# here with nothing printed. sed reads to the end. (Measured: exit 141, no output, 8 GPUs.)
-DRIVER="$("$NVSMI" --query-gpu=driver_version --format=csv,noheader 2>/dev/null | sed -n 1p | tr -d '[:space:]')"
+DRIVER="$(nvsmi --query-gpu=driver_version --format=csv,noheader 2>/dev/null | sed -n 1p | tr -d '[:space:]')"
driver_ok() {
[[ -n "$DRIVER" ]] && [[ "$(printf '%s\n' "$MIN_DRIVER" "$DRIVER" | sort -V | head -1)" == "$MIN_DRIVER" ]]
}
diff --git a/docker/run.sh b/docker/run.sh
index 3f0d8dfad8e..e1ab70456ea 100755
--- a/docker/run.sh
+++ b/docker/run.sh
@@ -61,6 +61,10 @@ fi
# regression tests can stage a fake device tree; leave it unset in normal use.
DEV_ROOT="${UNSLOTH_DEV_ROOT:-}"
+# Named once: the NVIDIA toolkit installer, both as the fallback download below
+# and in the message that tells you to run it yourself.
+TOOLKIT_URL="${UNSLOTH_TOOLKIT_URL:-https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh}"
+
# --group-add needs NUMERIC gids: a name is resolved INSIDE the container, where
# the host's video/render groups do not exist.
amd_device_flags() {
@@ -259,22 +263,34 @@ if [[ -n "$DOCKER_ERR" ]]; then
printf " Start the Docker daemon, or add yourself to the docker group (newgrp docker).\n\n" >&2
elif [[ $ROCM -eq 0 && ${#GPU_FLAG[@]} -gt 0 ]] && host_has_nvidia \
&& ! grep -qi 'Runtimes:.*nvidia' <<<"$DOCKER_INFO"; then
+ # run.sh is also published on its own, so the sibling installer is missing
+ # whenever it was curled rather than cloned. Fetch it in that case: offering
+ # to run a path that does not exist is worse than not offering at all.
INSTALLER="$(dirname "${BASH_SOURCE[0]}")/install_nvidia_toolkit.sh"
+ # Download it next to run.sh rather than into a scratch file: the path is then
+ # the one the message names, and a second run reuses it instead of refetching.
+ if [[ ! -f "$INSTALLER" ]]; then
+ curl -fsSL "$TOOLKIT_URL" -o "$INSTALLER" 2>/dev/null || { rm -f "$INSTALLER"; INSTALLER=""; }
+ fi
printf "\033[1;33mWARN:\033[0m 'docker info' does not list 'nvidia' as a runtime: the NVIDIA\n" >&2
printf " Container Toolkit is not set up, so --gpus %s would fail at the daemon.\n" "$GPUS" >&2
answer="${UNSLOTH_INSTALL_TOOLKIT:-}"
- if [[ -z "$answer" && -t 0 && -t 1 ]]; then
+ if [[ -n "$INSTALLER" && -z "$answer" && -t 0 && -t 1 ]]; then
read -r -p " Install it now with sudo (bash $INSTALLER)? [Y/n] " answer </dev/tty || answer=n
answer="${answer:-y}"
fi
+ # Nothing on disk to run: a forced UNSLOTH_INSTALL_TOOLKIT=1 would otherwise
+ # select the branch below and `bash ""` would fail into `|| true`, leaving the
+ # user with no toolkit, no error, and a docker run that still lacks the runtime.
+ [[ -n "$INSTALLER" ]] || answer=n
case "$answer" in
1|[Yy]*)
# -E keeps UNSLOTH_TOOLKIT_VERIFY and the proxy settings through env_reset; a failed, cancelled or driver-too-old install (exit 3) must not stop the docker run below.
if [[ "$(id -u)" = 0 ]]; then bash "$INSTALLER" || true; else sudo -E bash "$INSTALLER" || true; fi
;;
*)
- printf " Install it with one command (Linux, needs sudo):\n" >&2
- printf " curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh\n\n" >&2
+ printf " Install it with one command (Linux, needs root):\n" >&2
+ printf " curl -fsSL %s -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh\n\n" "$TOOLKIT_URL" >&2
;;
esac
fi