Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

feat(runtime): add Windows Blackwell CUDA asset

LauraGPTc56759590019
diff --git a/.github/workflows/build-llamacpp-binaries.yml b/.github/workflows/build-llamacpp-binaries.yml
index 121094893..9001b9317 100644
--- a/.github/workflows/build-llamacpp-binaries.yml
+++ b/.github/workflows/build-llamacpp-binaries.yml
@@ -145,6 +145,30 @@ jobs:
-DGGML_FMA=OFF
-DGGML_F16C=OFF
-DGGML_BMI2=OFF
+ - os: windows-latest
+ name: windows-x64-cuda-blackwell
+ cuda: true
+ cuda_version: '13.2.0'
+ cuda_architectures: '120'
+ build_target: llama-funasr-sensevoice
+ timeout_minutes: 90
+ cmake_flags: >-
+ -DGGML_NATIVE=OFF
+ -DGGML_CUDA=ON
+ -DGGML_CUDA_FORCE_CUBLAS=ON
+ -DGGML_CUDA_FA=OFF
+ -DGGML_CUDA_NCCL=OFF
+ -DGGML_CUDA_GRAPHS=OFF
+ -DGGML_AVX=OFF
+ -DGGML_AVX2=OFF
+ -DGGML_AVX_VNNI=OFF
+ -DGGML_AVX512=OFF
+ -DGGML_AVX512_VBMI=OFF
+ -DGGML_AVX512_VNNI=OFF
+ -DGGML_AVX512_BF16=OFF
+ -DGGML_FMA=OFF
+ -DGGML_F16C=OFF
+ -DGGML_BMI2=OFF
runs-on: ${{ matrix.os }}
timeout-minutes: ${{ matrix.timeout_minutes || 30 }}
defaults:
@@ -257,4 +281,4 @@ jobs:
gh release create "$tag" dist/* \
--repo "${{ github.repository }}" \
--title "FunASR llama.cpp runtime $version" \
- --notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA asset is \`windows-x64-cuda\`; it requires an NVIDIA driver compatible with the CUDA Toolkit version configured by the release workflow, targets CUDA architecture 86, and enables SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. No Python ASR runtime or local build is required. Docs: $docs"
+ --notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA assets are \`windows-x64-cuda\` for CUDA architecture 86 and \`windows-x64-cuda-blackwell\` for CUDA architecture 120 (RTX 50 / Blackwell); both require a compatible NVIDIA driver and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. A successful CI build verifies code generation and archive integrity, not execution on physical Blackwell hardware. No Python ASR runtime or local build is required. Docs: $docs"
diff --git a/runtime/llama.cpp/README.md b/runtime/llama.cpp/README.md
index 22a405a96..7070a8780 100644
--- a/runtime/llama.cpp/README.md
+++ b/runtime/llama.cpp/README.md
@@ -93,11 +93,11 @@ FSMN-VAD without Python.
### Optional Windows CUDA backend for SenseVoiceSmall
-The CPU release ZIPs are portable packages. Tagged releases also publish
-`funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on
-NVIDIA GPUs that match CUDA architecture 86. Download the CUDA ZIP from
-[runtime-llamacpp-v0.1.9](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.9),
-then select the backend at runtime:
+The CPU release ZIPs are portable packages. Tagged releases publish separate
+SenseVoiceSmall CUDA packages: `funasr-llamacpp-windows-x64-cuda.zip` targets
+CUDA architecture 86, while `funasr-llamacpp-windows-x64-cuda-blackwell.zip`
+targets CUDA architecture 120 (`sm_120`) for RTX 50 / Blackwell GPUs. Select the
+matching archive, then enable the backend at runtime:
```bash
# From the extracted windows-x64-cuda package:
@@ -105,7 +105,9 @@ then select the backend at runtime:
-m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda
```
-Build from source to target other GPU architectures:
+The Blackwell package uses the same command from its extracted directory. Build
+from source to target other GPU architectures or to reproduce the architecture
+120 build locally:
```bash
cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
@@ -115,10 +117,11 @@ cmake --build build-cuda -j --target llama-funasr-sensevoice
-m sensevoice-small-f16.gguf -a sample.wav --backend cuda
```
-Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /
-Blackwell cards report compute capability 12.0 (`sm_120`), so the current
-`windows-x64-cuda` prebuilt package for architecture 86 will not cover those
-cards.
+Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. A successful
+release workflow build verifies architecture 120 code generation and ZIP
+integrity; it does not prove execution on physical Blackwell hardware. Keep
+hardware-specific reports open until the matching archive is retested on the
+reported GPU.
`--backend cpu` remains the default and is what the portable cross-platform
prebuilt binaries use. The CUDA package requires an NVIDIA driver compatible
diff --git a/runtime/llama.cpp/tests/test_release_workflow.py b/runtime/llama.cpp/tests/test_release_workflow.py
index 590a34477..d6542c462 100644
--- a/runtime/llama.cpp/tests/test_release_workflow.py
+++ b/runtime/llama.cpp/tests/test_release_workflow.py
@@ -16,6 +16,19 @@ def test_windows_cuda_release_asset_is_in_matrix():
assert "timeout_minutes: 90" in workflow
+def test_windows_blackwell_cuda_release_asset_is_in_matrix():
+ workflow = WORKFLOW.read_text(encoding="utf-8")
+ entry = workflow.split("name: windows-x64-cuda-blackwell", maxsplit=1)[1].split(
+ " - os:", maxsplit=1
+ )[0]
+
+ assert "cuda: true" in entry
+ assert "cuda_version: '13.2.0'" in entry
+ assert "cuda_architectures: '120'" in entry
+ assert "build_target: llama-funasr-sensevoice" in entry
+ assert "timeout_minutes: 90" in entry
+
+
def test_linux_vulkan_release_asset_is_in_matrix():
workflow = WORKFLOW.read_text(encoding="utf-8")
@@ -88,10 +101,13 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets():
assert "--backend cuda" in readme
assert "Windows CUDA" in readme
assert "CUDA architecture 86" in readme
+ assert "windows-x64-cuda-blackwell" in readme
+ assert "CUDA architecture 120" in readme
assert "Build from source" in readme
assert "other GPU architectures" in readme
assert "CMAKE_CUDA_ARCHITECTURES=120" in readme
assert "sm_120" in readme
+ assert "does not prove execution on physical Blackwell hardware" in readme
def test_release_notes_explain_vulkan_linux_asset():
@@ -134,6 +150,16 @@ def test_github_release_notes_mention_vulkan_asset():
assert "--backend vulkan" in release_notes
assert "GGML_VULKAN=ON" in release_notes
+
+def test_github_release_notes_distinguish_cuda_architecture_assets():
+ workflow = WORKFLOW.read_text(encoding="utf-8")
+ release_notes = workflow.split('--notes "', maxsplit=1)[1]
+
+ assert "windows-x64-cuda-blackwell" in release_notes
+ assert "CUDA architecture 86" in release_notes
+ assert "CUDA architecture 120" in release_notes
+
+
def test_readme_documents_lightweight_http_server():
readme = (ROOT / "runtime" / "llama.cpp" / "README.md").read_text(encoding="utf-8")