Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Docker ROCm Studio image: run GGUF chat on the GPU through Vulkan (#13852)
Daniel Hand65cc7ba563c
diff --git a/docker/DOCKERHUB-ROCM.md b/docker/DOCKERHUB-ROCM.mdindex 1af9cda8451..dcdfe8d43cd 100644--- a/docker/DOCKERHUB-ROCM.md+++ b/docker/DOCKERHUB-ROCM.md@@ -113,7 +113,7 @@ Included: PyTorch with ROCm, Unsloth, unsloth-zoo, transformers, TRL, PEFT, acceNot included in `latest`, unlike `unsloth/unsloth`: Unsloth Studio and its web UI, JupyterLab, prebuilt llama.cpp and whisper.cpp, vLLM and xformers. `latest` is a training image.-The `studio` tag adds Unsloth Studio, JupyterLab with the Unsloth notebooks (the `AMD-*` set first) and a CPU llama.cpp for GGUF chat, still without whisper.cpp, vLLM or xformers. Training, the notebooks and the UI use the GPU; GGUF chat runs on the CPU, since a ROCm llama.cpp bundle is per-architecture and would pin the image to one card. Studio's data (accounts, chats, project folders, outputs) lives at `/opt/unsloth-studio`, so mount a volume there to keep it, and one on `/workspace` for the notebooks, your edits and what they save (projects created by an image from before project folders moved to the volume keep their folder under `/root/Documents` inside the container, so copy their files out before `docker rm`), with `GPU_FLAGS` set as in the quick start:+The `studio` tag adds Unsloth Studio, JupyterLab with the Unsloth notebooks (the `AMD-*` set first) and a Vulkan llama.cpp for GGUF chat, still without whisper.cpp, vLLM or xformers. Training, the notebooks, the UI and GGUF chat all use the GPU. GGUF chat reaches the card through Vulkan (Mesa's RADV driver) on `/dev/dri`, which covers RDNA cards without pinning the image to one architecture; on CDNA (which RADV does not drive) or on the WSL2 bridge, it runs on the CPU instead. Vulkan does not read `HIP_VISIBLE_DEVICES`: on a host with several AMD GPUs, pick the chat GPU in Studio or pass only that card's `/dev/dri/renderD*` node. Studio's data (accounts, chats, project folders, outputs) lives at `/opt/unsloth-studio`, so mount a volume there to keep it, and one on `/workspace` for the notebooks, your edits and what they save (projects created by an image from before project folders moved to the volume keep their folder under `/root/Documents` inside the container, so copy their files out before `docker rm`), with `GPU_FLAGS` set as in the quick start:```bashdocker run --rm $GPU_FLAGS --ipc=host -p 127.0.0.1:8000:8000 -p 127.0.0.1:8888:8888 \diff --git a/docker/Dockerfile.studio-rocm b/docker/Dockerfile.studio-rocmindex 988807d8c02..a5dd1656872 100644--- a/docker/Dockerfile.studio-rocm+++ b/docker/Dockerfile.studio-rocm@@ -8,9 +8,9 @@# also ships JupyterLab, the notebook tooling and the notebooks, so Dockerfile.studio# inherits them; the ROCm base ships none of that, so it is all installed here.#-# llama.cpp is installed as the CPU bundle: the builder has no GPU, and a ROCm-# bundle is per-gfx-family, so baking one would pin this image to one arch. GGUF-# chat therefore runs on the CPU here; training, the notebooks and the UI use the GPU.+# llama.cpp is the Vulkan bundle (Mesa RADV): a ROCm bundle is per-gfx-family and would+# pin this image to one arch. Without a RADV device (CDNA, the WSL2 DXG bridge) GGUF+# chat falls back to the CPU.## Build (on the published ROCm base, or one built from docker/Dockerfile.rocm):# docker buildx build --build-arg BASE_IMAGE=unsloth/unsloth-rocm:latest \@@ -96,10 +96,11 @@ ENV UNSLOTH_STUDIO_HOME=/opt/unsloth-studio \# install.sh needs curl + git; supervisor runs Studio and JupyterLab together;# ffmpeg is what the notebooks' audio decode shells out to (docker/Dockerfile bakes-# it for the same reason), and it is not in the ROCm base.+# it for the same reason), and it is not in the ROCm base. libvulkan1 and+# mesa-vulkan-drivers (RADV) let the Vulkan llama.cpp reach the card.RUN apt-get update \&& apt-get install -y --no-install-recommends \- curl git ca-certificates supervisor ffmpeg \+ curl git ca-certificates supervisor ffmpeg libvulkan1 mesa-vulkan-drivers \&& rm -rf /var/lib/apt/lists/*# JupyterLab and the notebooks' runtime packages into the base venv, so the kernel@@ -151,7 +152,7 @@ RUN set -eux \UNSLOTH_TORCH_INDEX_URL="${TORCH_INDEX_URL}" \UNSLOTH_ROCM_GFX_ARCH="${ROCM_GFX}" \UNSLOTH_ZOO_REF="${UNSLOTH_STUDIO_ZOO_REF}" \- UNSLOTH_LLAMA_CPP_BACKEND=cpu \+ UNSLOTH_LLAMA_CPP_BACKEND=vulkan \UNSLOTH_SKIP_WHISPER_INSTALL=1 \UNSLOTH_PYTHON=3.12 \UNSLOTH_ALLOW_CPU=1 \@@ -167,6 +168,8 @@ RUN set -eux \# The UI is the point of this image, so a missing frontend build is fatal here# rather than a 404 at runtime.&& test -f "${UNSLOTH_STUDIO_HOME}/src/studio/frontend/dist/index.html" \+ # Shipping the CPU bundle here would put GGUF chat back on the CPU unnoticed.+ && "${UNSLOTH_STUDIO_HOME}/unsloth_studio/bin/python" -c "import json; m = json.load(open('${UNSLOTH_STUDIO_HOME}/llama.cpp/UNSLOTH_PREBUILT_INFO.json')); assert m.get('backend') == 'vulkan', 'llama.cpp installed as ' + repr(m.get('backend')) + ', not vulkan'; print('llama.cpp', m.get('release_tag'), m.get('asset'))" \&& rm -rf "${UNSLOTH_STUDIO_HOME}/src/.git" \"${UNSLOTH_STUDIO_HOME}/src/studio/frontend/node_modules" \"${UV_CACHE_DIR}" \diff --git a/tests/python/test_docker_rocm.py b/tests/python/test_docker_rocm.pyindex 840395313bf..6dd5cda6945 100644--- a/tests/python/test_docker_rocm.py+++ b/tests/python/test_docker_rocm.py@@ -3,6 +3,7 @@"""Drives run.sh, build.sh and the entrypoint with stub docker, rocm-smi and python: no AMD GPU."""import os+import reimport shutilimport statimport subprocess@@ -1050,3 +1051,27 @@ class TestStudioLaunchRocm:assert 'UNSLOTH_ROCM_GFX_ARCH="${ROCM_GFX}"' in bodyignore = open(os.path.join(_DOCKER, ".dockerignore"), encoding = "utf-8").read()assert "!studio_launch.sh" in ignore and "!studio_run.sh" in ignore+++class TestStudioRocmGgufOnTheGpu:+ """#5405: GGUF chat in the ROCm Studio image ran on the CPU (CPU llama.cpp bundle)."""++ def _body(self):+ return open(_STUDIO_DOCKERFILE, encoding = "utf-8").read()++ def test_llama_cpp_is_installed_as_the_vulkan_bundle(self):+ body = self._body()+ install = body[+ body.index(". /etc/unsloth-rocm-build") : body.index("bash install.sh --local")+ ]+ assert "UNSLOTH_LLAMA_CPP_BACKEND=vulkan" in install, install+ assert "UNSLOTH_LLAMA_CPP_BACKEND=cpu" not in body+ # install.sh falls back to the CPU bundle on a failed Vulkan install+ assert "assert m.get('backend') == 'vulkan'" in body++ def test_the_radv_driver_is_installed(self):+ body = self._body()+ apt = re.search(r"apt-get install -y --no-install-recommends \\\n\s+([^\n]+)", body).group(+ 1+ )+ assert "libvulkan1" in apt and "mesa-vulkan-drivers" in apt, apt