Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
docker: stop the toolkit installer reporting a healthy driver as missing (#11106)
Daniel Hand6dd50c322ef
diff --git a/docker/DOCKERHUB-ROCM.md b/docker/DOCKERHUB-ROCM.mdindex c106411ba39..3a0a8eaa056 100644--- a/docker/DOCKERHUB-ROCM.md+++ b/docker/DOCKERHUB-ROCM.md@@ -6,6 +6,12 @@ This is the AMD counterpart to [`unsloth/unsloth`](https://hub.docker.com/r/unsl## Quick start+Needs Docker and a working amdgpu driver. Without Docker, start here:++```bash+curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh+```+AMD GPUs are reached through the kernel driver's device nodes, not through a container toolkit, so the run command differs from the NVIDIA one:```bashdiff --git a/docker/DOCKERHUB.md b/docker/DOCKERHUB.mdindex 75723b01b56..eda37c231f0 100644--- a/docker/DOCKERHUB.md+++ b/docker/DOCKERHUB.md@@ -17,7 +17,13 @@ Source: [`docker/`](https://github.com/unslothai/unsloth/tree/main/docker) in th## Quick start-Needs an NVIDIA driver of 570.26 or newer and, on Linux, the NVIDIA Container Toolkit. One command installs the toolkit (Ubuntu, Debian, RHEL, Fedora, Rocky, Amazon Linux, SUSE) and checks a container can see the GPU:+Needs an NVIDIA driver of 570.26 or newer and, on Linux, Docker plus the NVIDIA Container Toolkit. Without Docker, start here:++```bash+curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh+```++Then one command installs the toolkit (Ubuntu, Debian, RHEL, Fedora, Rocky, Amazon Linux, SUSE) and checks a container can see the GPU:```bashcurl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh@@ -39,11 +45,11 @@ docker run -d --gpus all --ipc=host \`docker run -d` returns at once; follow the startup with `docker logs -f <container>`, which ends with a ready block once both services answer (Studio takes about a minute). Then open Studio at `http://localhost:8000` (user `unsloth`) and JupyterLab at `http://localhost:8888`. Leave either password variable unset and a random one is generated and printed in that log.-The `docker/run.sh` helper in the repository sets these flags, including the `unsloth-studio` volume, for you:+Or let `run.sh` set these flags, including the `unsloth-studio` volume, for you. It offers to install the NVIDIA Container Toolkit if the daemon has no `nvidia` runtime:```bash-git clone https://github.com/unslothai/unsloth && cd unsloth-UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash docker/run.sh+curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/run.sh -o run.sh+UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash run.sh```### Notebooks only (`core`)diff --git a/docker/install_nvidia_toolkit.sh b/docker/install_nvidia_toolkit.shindex 0d006afa96f..ebf8b5803a8 100755--- a/docker/install_nvidia_toolkit.sh+++ b/docker/install_nvidia_toolkit.sh@@ -16,7 +16,9 @@ say() { printf '%s\n' "$*"; }fail() { printf 'ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }command -v docker >/dev/null 2>&1 \- || fail "docker is not installed. Install Docker Engine first (https://docs.docker.com/engine/install/), then run this again." 2+ || fail "docker is not installed. Install Docker Engine first, on Linux with+ curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh+ (https://docs.docker.com/engine/install/ covers Docker Desktop and the distro packages), then run this again." 2# Every grep of a command's output below reads a variable, not a pipe. `grep -q` exits on the first# match and closes the pipe; `set -o pipefail` above then promotes the producer's SIGPIPE (141) to the@@ -127,9 +129,42 @@ fi# On WSL 2 the Windows driver puts nvidia-smi under /usr/lib/wsl/lib, which sudo's secure_path drops from PATH.NVSMI="$(command -v nvidia-smi 2>/dev/null || true)"[[ -z "$NVSMI" && -x "${WSL_LIB_DIR}/nvidia-smi" ]] && NVSMI="${WSL_LIB_DIR}/nvidia-smi"++# sudo strips every LD_* variable unconditionally, -E or not, as an anti-preload+# measure. A host that keeps the NVIDIA userspace libraries outside ldconfig and+# reaches them through LD_LIBRARY_PATH therefore loses them the moment this script+# elevates: nvidia-smi is on PATH, runs, and exits with "couldn't find+# libnvidia-ml.so", which carries no GPU line and used to be reported here as a+# missing driver. Measured on Colab (T4, driver 580.82.07): libs in+# /usr/lib64-nvidia, absent from `ldconfig -p`, LD_LIBRARY_PATH set for the user+# and UNSET under sudo. Put the directory back when it is the only thing wrong.+nvsmi() { LD_LIBRARY_PATH="${NV_LIB_DIR}${NV_LIB_DIR:+${LD_LIBRARY_PATH:+:}}${LD_LIBRARY_PATH:-}" "$NVSMI" "$@"; }+NV_LIB_DIR=""gpu_list=""-[[ -n "$NVSMI" ]] && gpu_list="$("$NVSMI" -L 2>/dev/null || true)"+# 2>&1, not 2>/dev/null: the loader's complaint is the evidence that separates a+# missing driver from a driver this shell cannot reach, and it goes to stderr.+[[ -n "$NVSMI" ]] && gpu_list="$(nvsmi -L 2>&1 || true)"+if [[ -n "$NVSMI" ]] && grep -qi 'libnvidia-ml' <<<"$gpu_list"; then+ for _dir in /usr/lib64-nvidia /usr/local/nvidia/lib64 "$WSL_LIB_DIR"; do+ [[ -e "${DESTDIR}${_dir}/libnvidia-ml.so.1" ]] || continue+ NV_LIB_DIR="${DESTDIR}${_dir}"+ gpu_list="$(nvsmi -L 2>&1 || true)"+ grep -q '^GPU' <<<"$gpu_list" && break+ NV_LIB_DIR=""+ done+fiif [[ -z "$NVSMI" ]] || ! grep -q '^GPU' <<<"$gpu_list"; then+ # Still a library complaint after the search: the driver is there and only the+ # loader cannot see it, so saying "no driver" would send the reader to install+ # one they already have.+ if grep -qi 'libnvidia-ml' <<<"$gpu_list"; then+ fail "nvidia-smi cannot load libnvidia-ml.so, so this script cannot read the driver:+ ${gpu_list%%$'\n'*}+ sudo drops every LD_* variable, so a driver reached through LD_LIBRARY_PATH (Colab+ puts its libraries in /usr/lib64-nvidia) disappears when this script elevates.+ Find the directory holding libnvidia-ml.so.1 and name it on the command line:+ sudo LD_LIBRARY_PATH=/usr/lib64-nvidia bash install_nvidia_toolkit.sh" 2+ fiif grep -qi microsoft "$PROC_VERSION" 2>/dev/null; thenfail "no NVIDIA GPU is visible in this WSL 2 distro. Install a current NVIDIA Windows driverfrom nvidia.com (never a Linux driver inside WSL), restart WSL (wsl --shutdown), then run@@ -144,7 +179,7 @@ MIN_DRIVER=570.26# `sed -n 1p`, not `head -1`: head closes the pipe after one line, and nvidia-smi prints one per GPU,# so on a multi-GPU host the SIGPIPE becomes the substitution's status and `set -e` kills the script# here with nothing printed. sed reads to the end. (Measured: exit 141, no output, 8 GPUs.)-DRIVER="$("$NVSMI" --query-gpu=driver_version --format=csv,noheader 2>/dev/null | sed -n 1p | tr -d '[:space:]')"+DRIVER="$(nvsmi --query-gpu=driver_version --format=csv,noheader 2>/dev/null | sed -n 1p | tr -d '[:space:]')"driver_ok() {[[ -n "$DRIVER" ]] && [[ "$(printf '%s\n' "$MIN_DRIVER" "$DRIVER" | sort -V | head -1)" == "$MIN_DRIVER" ]]}diff --git a/docker/run.sh b/docker/run.shindex 3f0d8dfad8e..e1ab70456ea 100755--- a/docker/run.sh+++ b/docker/run.sh@@ -61,6 +61,10 @@ fi# regression tests can stage a fake device tree; leave it unset in normal use.DEV_ROOT="${UNSLOTH_DEV_ROOT:-}"+# Named once: the NVIDIA toolkit installer, both as the fallback download below+# and in the message that tells you to run it yourself.+TOOLKIT_URL="${UNSLOTH_TOOLKIT_URL:-https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh}"+# --group-add needs NUMERIC gids: a name is resolved INSIDE the container, where# the host's video/render groups do not exist.amd_device_flags() {@@ -259,22 +263,34 @@ if [[ -n "$DOCKER_ERR" ]]; thenprintf " Start the Docker daemon, or add yourself to the docker group (newgrp docker).\n\n" >&2elif [[ $ROCM -eq 0 && ${#GPU_FLAG[@]} -gt 0 ]] && host_has_nvidia \&& ! grep -qi 'Runtimes:.*nvidia' <<<"$DOCKER_INFO"; then+ # run.sh is also published on its own, so the sibling installer is missing+ # whenever it was curled rather than cloned. Fetch it in that case: offering+ # to run a path that does not exist is worse than not offering at all.INSTALLER="$(dirname "${BASH_SOURCE[0]}")/install_nvidia_toolkit.sh"+ # Download it next to run.sh rather than into a scratch file: the path is then+ # the one the message names, and a second run reuses it instead of refetching.+ if [[ ! -f "$INSTALLER" ]]; then+ curl -fsSL "$TOOLKIT_URL" -o "$INSTALLER" 2>/dev/null || { rm -f "$INSTALLER"; INSTALLER=""; }+ fiprintf "\033[1;33mWARN:\033[0m 'docker info' does not list 'nvidia' as a runtime: the NVIDIA\n" >&2printf " Container Toolkit is not set up, so --gpus %s would fail at the daemon.\n" "$GPUS" >&2answer="${UNSLOTH_INSTALL_TOOLKIT:-}"- if [[ -z "$answer" && -t 0 && -t 1 ]]; then+ if [[ -n "$INSTALLER" && -z "$answer" && -t 0 && -t 1 ]]; thenread -r -p " Install it now with sudo (bash $INSTALLER)? [Y/n] " answer </dev/tty || answer=nanswer="${answer:-y}"fi+ # Nothing on disk to run: a forced UNSLOTH_INSTALL_TOOLKIT=1 would otherwise+ # select the branch below and `bash ""` would fail into `|| true`, leaving the+ # user with no toolkit, no error, and a docker run that still lacks the runtime.+ [[ -n "$INSTALLER" ]] || answer=ncase "$answer" in1|[Yy]*)# -E keeps UNSLOTH_TOOLKIT_VERIFY and the proxy settings through env_reset; a failed, cancelled or driver-too-old install (exit 3) must not stop the docker run below.if [[ "$(id -u)" = 0 ]]; then bash "$INSTALLER" || true; else sudo -E bash "$INSTALLER" || true; fi;;*)- printf " Install it with one command (Linux, needs sudo):\n" >&2- printf " curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/docker/install_nvidia_toolkit.sh -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh\n\n" >&2+ printf " Install it with one command (Linux, needs root):\n" >&2+ printf " curl -fsSL %s -o install_nvidia_toolkit.sh && sudo -E bash install_nvidia_toolkit.sh\n\n" "$TOOLKIT_URL" >&2;;esacfi