Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
rocm: update linux to v7.2 (#14391)
Daniel Hiltgenebb1b9ec14dd
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yamlindex b614d2f04..a47156516 100644--- a/.github/workflows/test.yaml+++ b/.github/workflows/test.yaml@@ -51,7 +51,7 @@ jobs:container: nvidia/cuda:13.0.0-devel-ubuntu22.04flags: '-DCMAKE_CUDA_ARCHITECTURES=87'- preset: ROCm- container: rocm/dev-ubuntu-22.04:6.1.2+ container: rocm/dev-ubuntu-22.04:7.2extra-packages: rocm-libsflags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'- preset: Vulkandiff --git a/CMakeLists.txt b/CMakeLists.txtindex 7d53531be..a9e4471d8 100644--- a/CMakeLists.txt+++ b/CMakeLists.txt@@ -148,7 +148,7 @@ if(CMAKE_HIP_COMPILER))install(RUNTIME_DEPENDENCY_SET rocmDIRECTORIES ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR}- PRE_INCLUDE_REGEXES hipblas rocblas amdhip64 rocsolver amd_comgr hsa-runtime64 rocsparse tinfo rocprofiler-register drm drm_amdgpu numa elf+ PRE_INCLUDE_REGEXES hipblas rocblas amdhip64 rocsolver amd_comgr hsa-runtime64 rocsparse tinfo rocprofiler-register roctx64 rocroller drm drm_amdgpu numa elfPRE_EXCLUDE_REGEXES ".*"POST_EXCLUDE_REGEXES "system32"RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIPdiff --git a/CMakePresets.json b/CMakePresets.jsonindex 64b7fd58a..0d643038a 100644--- a/CMakePresets.json+++ b/CMakePresets.json@@ -77,6 +77,15 @@"OLLAMA_RUNNER_DIR": "rocm"}},+ {+ "name": "ROCm 7",+ "inherits": [ "ROCm" ],+ "cacheVariables": {+ "CMAKE_HIP_FLAGS": "-parallel-jobs=4",+ "AMDGPU_TARGETS": "gfx942;gfx950;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",+ "OLLAMA_RUNNER_DIR": "rocm"+ }+ },{"name": "Vulkan","inherits": [ "Default" ],@@ -158,6 +167,11 @@"inherits": [ "ROCm" ],"configurePreset": "ROCm 6"},+ {+ "name": "ROCm 7",+ "inherits": [ "ROCm" ],+ "configurePreset": "ROCm 7"+ },{"name": "Vulkan","targets": [ "ggml-vulkan" ],diff --git a/Dockerfile b/Dockerfileindex 1057dd0f9..cabb6cc82 100644--- a/Dockerfile+++ b/Dockerfile@@ -2,7 +2,7 @@ARG FLAVOR=${TARGETARCH}-ARG ROCMVERSION=6.3.3+ARG ROCMVERSION=7.2ARG JETPACK5VERSION=r35.4.1ARG JETPACK6VERSION=r36.4.0ARG CMAKEVERSION=3.31.2@@ -77,13 +77,13 @@ RUN --mount=type=cache,target=/root/.ccache \&& cmake --install build --component CUDA --strip-FROM base AS rocm-6+FROM base AS rocm-7ENV PATH=/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin:$PATHCOPY CMakeLists.txt CMakePresets.json .COPY ml/backend/ggml/ggml ml/backend/ggml/ggmlRUN --mount=type=cache,target=/root/.ccache \- cmake --preset 'ROCm 6' \- && cmake --build --preset 'ROCm 6' -- -l $(nproc) \+ cmake --preset 'ROCm 7' \+ && cmake --build --preset 'ROCm 7' -- -l $(nproc) \&& cmake --install build --component HIP --stripRUN rm -f dist/lib/ollama/rocm/rocblas/library/*gfx90[06]*@@ -194,7 +194,7 @@ COPY --from=jetpack-5 dist/lib/ollama/ /lib/ollama/COPY --from=jetpack-6 dist/lib/ollama/ /lib/ollama/FROM scratch AS rocm-COPY --from=rocm-6 dist/lib/ollama /lib/ollama+COPY --from=rocm-7 dist/lib/ollama /lib/ollamaFROM ${FLAVOR} AS archiveCOPY --from=cpu dist/lib/ollama /lib/ollamadiff --git a/docs/gpu.mdx b/docs/gpu.mdxindex 60e2f5e3c..e129cdce2 100644--- a/docs/gpu.mdx+++ b/docs/gpu.mdx@@ -61,11 +61,13 @@ Ollama supports the following AMD GPUs via the ROCm library:### Linux Support-| Family | Cards and accelerators |-| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |-| AMD Radeon RX | `7900 XTX` `7900 XT` `7900 GRE` `7800 XT` `7700 XT` `7600 XT` `7600` `6950 XT` `6900 XTX` `6900XT` `6800 XT` `6800` `Vega 64` |-| AMD Radeon PRO | `W7900` `W7800` `W7700` `W7600` `W7500` `W6900X` `W6800X Duo` `W6800X` `W6800` `V620` `V420` `V340` `V320` `Vega II Duo` `Vega II` `SSG` |-| AMD Instinct | `MI300X` `MI300A` `MI300` `MI250X` `MI250` `MI210` `MI200` `MI100` `MI60` |+| Family | Cards and accelerators |+| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| AMD Radeon RX | `9070 XT` `9070 GRE` `9070` `9060 XT` `9060 XT LP` `9060` `7900 XTX` `7900 XT` `7900 GRE` `7800 XT` `7700 XT` `7700` `7600 XT` `7600` `6950 XT` `6900 XTX` `6900XT` `6800 XT` `6800` `5700 XT` `5700` `5600 XT` `5500 XT` |+| AMD Radeon AI PRO | `R9700` `R9600D` |+| AMD Radeon PRO | `W7900` `W7800` `W7700` `W7600` `W7500` `W6900X` `W6800X Duo` `W6800X` `W6800` `V620` |+| AMD Ryzen AI | `Ryzen AI Max+ 395` `Ryzen AI Max 390` `Ryzen AI Max 385` `Ryzen AI 9 HX 475` `Ryzen AI 9 HX 470` `Ryzen AI 9 465` `Ryzen AI 9 HX 375` `Ryzen AI 9 HX 370` `Ryzen AI 9 365` |+| AMD Instinct | `MI350X` `MI300X` `MI300A` `MI250X` `MI250` `MI210` `MI100` |### Windows Support@@ -97,17 +99,20 @@ This table shows some example GPUs that map to these LLVM targets:| **LLVM Target** | **An Example GPU** ||-----------------|---------------------|| gfx908 | Radeon Instinct MI100 |-| gfx90a | Radeon Instinct MI210 |-| gfx940 | Radeon Instinct MI300 |-| gfx941 | |-| gfx942 | |+| gfx90a | Radeon Instinct MI210/MI250 |+| gfx942 | Radeon Instinct MI300X/MI300A |+| gfx950 | Radeon Instinct MI350X |+| gfx1010 | Radeon RX 5700 XT |+| gfx1012 | Radeon RX 5500 XT || gfx1030 | Radeon PRO V620 || gfx1100 | Radeon PRO W7900 || gfx1101 | Radeon PRO W7700 || gfx1102 | Radeon RX 7600 |--AMD is working on enhancing ROCm v6 to broaden support for families of GPUs in a-future release which should increase support for more GPUs.+| gfx1103 | Radeon 780M |+| gfx1150 | Ryzen AI 9 HX 375 |+| gfx1151 | Ryzen AI Max+ 395 |+| gfx1200 | Radeon RX 9070 |+| gfx1201 | Radeon RX 9070 XT |Reach out on [Discord](https://discord.gg/ollama) or file an[issue](https://github.com/ollama/ollama/issues) for additional help.diff --git a/docs/linux.mdx b/docs/linux.mdxindex 49b58a45c..66c6813d1 100644--- a/docs/linux.mdx+++ b/docs/linux.mdx@@ -101,7 +101,7 @@ nvidia-smi### Install AMD ROCm drivers (optional)-[Download and Install](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html) ROCm v6.+[Download and Install](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html) ROCm v7.### Start Ollama