Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
feat(runtime): add Windows Blackwell CUDA asset
LauraGPTc56759590019
diff --git a/.github/workflows/build-llamacpp-binaries.yml b/.github/workflows/build-llamacpp-binaries.ymlindex 121094893..9001b9317 100644--- a/.github/workflows/build-llamacpp-binaries.yml+++ b/.github/workflows/build-llamacpp-binaries.yml@@ -145,6 +145,30 @@ jobs:-DGGML_FMA=OFF-DGGML_F16C=OFF-DGGML_BMI2=OFF+ - os: windows-latest+ name: windows-x64-cuda-blackwell+ cuda: true+ cuda_version: '13.2.0'+ cuda_architectures: '120'+ build_target: llama-funasr-sensevoice+ timeout_minutes: 90+ cmake_flags: >-+ -DGGML_NATIVE=OFF+ -DGGML_CUDA=ON+ -DGGML_CUDA_FORCE_CUBLAS=ON+ -DGGML_CUDA_FA=OFF+ -DGGML_CUDA_NCCL=OFF+ -DGGML_CUDA_GRAPHS=OFF+ -DGGML_AVX=OFF+ -DGGML_AVX2=OFF+ -DGGML_AVX_VNNI=OFF+ -DGGML_AVX512=OFF+ -DGGML_AVX512_VBMI=OFF+ -DGGML_AVX512_VNNI=OFF+ -DGGML_AVX512_BF16=OFF+ -DGGML_FMA=OFF+ -DGGML_F16C=OFF+ -DGGML_BMI2=OFFruns-on: ${{ matrix.os }}timeout-minutes: ${{ matrix.timeout_minutes || 30 }}defaults:@@ -257,4 +281,4 @@ jobs:gh release create "$tag" dist/* \--repo "${{ github.repository }}" \--title "FunASR llama.cpp runtime $version" \- --notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA asset is \`windows-x64-cuda\`; it requires an NVIDIA driver compatible with the CUDA Toolkit version configured by the release workflow, targets CUDA architecture 86, and enables SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. No Python ASR runtime or local build is required. Docs: $docs"+ --notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA assets are \`windows-x64-cuda\` for CUDA architecture 86 and \`windows-x64-cuda-blackwell\` for CUDA architecture 120 (RTX 50 / Blackwell); both require a compatible NVIDIA driver and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. A successful CI build verifies code generation and archive integrity, not execution on physical Blackwell hardware. No Python ASR runtime or local build is required. Docs: $docs"diff --git a/runtime/llama.cpp/README.md b/runtime/llama.cpp/README.mdindex 22a405a96..7070a8780 100644--- a/runtime/llama.cpp/README.md+++ b/runtime/llama.cpp/README.md@@ -93,11 +93,11 @@ FSMN-VAD without Python.### Optional Windows CUDA backend for SenseVoiceSmall-The CPU release ZIPs are portable packages. Tagged releases also publish-`funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on-NVIDIA GPUs that match CUDA architecture 86. Download the CUDA ZIP from-[runtime-llamacpp-v0.1.9](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.9),-then select the backend at runtime:+The CPU release ZIPs are portable packages. Tagged releases publish separate+SenseVoiceSmall CUDA packages: `funasr-llamacpp-windows-x64-cuda.zip` targets+CUDA architecture 86, while `funasr-llamacpp-windows-x64-cuda-blackwell.zip`+targets CUDA architecture 120 (`sm_120`) for RTX 50 / Blackwell GPUs. Select the+matching archive, then enable the backend at runtime:```bash# From the extracted windows-x64-cuda package:@@ -105,7 +105,9 @@ then select the backend at runtime:-m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda```-Build from source to target other GPU architectures:+The Blackwell package uses the same command from its extracted directory. Build+from source to target other GPU architectures or to reproduce the architecture+120 build locally:```bashcmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \@@ -115,10 +117,11 @@ cmake --build build-cuda -j --target llama-funasr-sensevoice-m sensevoice-small-f16.gguf -a sample.wav --backend cuda```-Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /-Blackwell cards report compute capability 12.0 (`sm_120`), so the current-`windows-x64-cuda` prebuilt package for architecture 86 will not cover those-cards.+Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. A successful+release workflow build verifies architecture 120 code generation and ZIP+integrity; it does not prove execution on physical Blackwell hardware. Keep+hardware-specific reports open until the matching archive is retested on the+reported GPU.`--backend cpu` remains the default and is what the portable cross-platformprebuilt binaries use. The CUDA package requires an NVIDIA driver compatiblediff --git a/runtime/llama.cpp/tests/test_release_workflow.py b/runtime/llama.cpp/tests/test_release_workflow.pyindex 590a34477..d6542c462 100644--- a/runtime/llama.cpp/tests/test_release_workflow.py+++ b/runtime/llama.cpp/tests/test_release_workflow.py@@ -16,6 +16,19 @@ def test_windows_cuda_release_asset_is_in_matrix():assert "timeout_minutes: 90" in workflow+def test_windows_blackwell_cuda_release_asset_is_in_matrix():+ workflow = WORKFLOW.read_text(encoding="utf-8")+ entry = workflow.split("name: windows-x64-cuda-blackwell", maxsplit=1)[1].split(+ " - os:", maxsplit=1+ )[0]++ assert "cuda: true" in entry+ assert "cuda_version: '13.2.0'" in entry+ assert "cuda_architectures: '120'" in entry+ assert "build_target: llama-funasr-sensevoice" in entry+ assert "timeout_minutes: 90" in entry++def test_linux_vulkan_release_asset_is_in_matrix():workflow = WORKFLOW.read_text(encoding="utf-8")@@ -88,10 +101,13 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets():assert "--backend cuda" in readmeassert "Windows CUDA" in readmeassert "CUDA architecture 86" in readme+ assert "windows-x64-cuda-blackwell" in readme+ assert "CUDA architecture 120" in readmeassert "Build from source" in readmeassert "other GPU architectures" in readmeassert "CMAKE_CUDA_ARCHITECTURES=120" in readmeassert "sm_120" in readme+ assert "does not prove execution on physical Blackwell hardware" in readmedef test_release_notes_explain_vulkan_linux_asset():@@ -134,6 +150,16 @@ def test_github_release_notes_mention_vulkan_asset():assert "--backend vulkan" in release_notesassert "GGML_VULKAN=ON" in release_notes++def test_github_release_notes_distinguish_cuda_architecture_assets():+ workflow = WORKFLOW.read_text(encoding="utf-8")+ release_notes = workflow.split('--notes "', maxsplit=1)[1]++ assert "windows-x64-cuda-blackwell" in release_notes+ assert "CUDA architecture 86" in release_notes+ assert "CUDA architecture 120" in release_notes++def test_readme_documents_lightweight_http_server():readme = (ROOT / "runtime" / "llama.cpp" / "README.md").read_text(encoding="utf-8")