Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

docs: clarify Windows CUDA GPU architecture support

zhifu gaoa2a6ba3ddf1b
diff --git a/README.md b/README.md
index 95198c8b8..dd2cd7a63 100644
--- a/README.md
+++ b/README.md
@@ -264,6 +264,11 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda
```
+The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell
+GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or
+build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA
+asset is published.
+
**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)
diff --git a/README_zh.md b/README_zh.md
index 0860c62db..601c4d682 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -303,6 +303,10 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda
```
+当前 Windows CUDA 包面向 CUDA architecture 86。RTX 50 / Blackwell GPU 会报告
+compute capability 12.0(`sm_120`),在专用 CUDA 产物发布前,请使用 CPU 包,或从
+源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。
+
**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)
[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)
diff --git a/runtime/llama.cpp/README.md b/runtime/llama.cpp/README.md
index 2439551fd..f38460507 100644
--- a/runtime/llama.cpp/README.md
+++ b/runtime/llama.cpp/README.md
@@ -59,7 +59,7 @@ cmake -B build -DCMAKE_BUILD_TYPE=Release # fetches pinned llama.cpp; stati
cmake --build build -j # -> build/bin/llama-funasr-* (all tools)
```
-### Optional CUDA backend for SenseVoiceSmall
+### Optional Windows CUDA backend for SenseVoiceSmall
The CPU release ZIPs are portable packages. Tagged releases also publish
`funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on
@@ -73,15 +73,21 @@ then select the backend at runtime:
-m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda
```
-Build from source to target a different GPU architecture:
+Build from source to target other GPU architectures:
```bash
-cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
+cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
+ -DCMAKE_CUDA_ARCHITECTURES=120
cmake --build build-cuda -j --target llama-funasr-sensevoice
./build-cuda/bin/llama-funasr-sensevoice \
-m sensevoice-small-f16.gguf -a sample.wav --backend cuda
```
+Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /
+Blackwell cards report compute capability 12.0 (`sm_120`), so the current
+`windows-x64-cuda` prebuilt package for architecture 86 will not cover those
+cards.
+
`--backend cpu` remains the default and is what the portable cross-platform
prebuilt binaries use. The CUDA package requires an NVIDIA driver compatible
with the CUDA Toolkit version configured by the release workflow. A binary built
diff --git a/runtime/llama.cpp/tests/test_release_workflow.py b/runtime/llama.cpp/tests/test_release_workflow.py
index 18f749952..201c97698 100644
--- a/runtime/llama.cpp/tests/test_release_workflow.py
+++ b/runtime/llama.cpp/tests/test_release_workflow.py
@@ -39,3 +39,5 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets():
assert "CUDA architecture 86" in readme
assert "Build from source" in readme
assert "other GPU architectures" in readme
+ assert "CMAKE_CUDA_ARCHITECTURES=120" in readme
+ assert "sm_120" in readme