Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
Document Fun-ASR-Nano Ascend NPU runtime bounds
zhifu gao7fed1f61baf9
diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.mdindex f323161a5..ccf6dbdca 100644--- a/docs/deployment_matrix.md+++ b/docs/deployment_matrix.md@@ -87,7 +87,11 @@ Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution a### I want to run Fun-ASR-Nano on Ascend NPU-Fun-ASR-Nano's LLM-based path is currently documented and validated for CUDA/vLLM, standard PyTorch CPU/GPU runs, and CPU/edge GGUF runtimes. Ascend NPU (`torch_npu`) is not an officially validated runtime for this model yet. Do not assume that a SenseVoice or Paraformer NPU deployment means Fun-ASR-Nano will also work, because Nano also exercises the Qwen decoder, `inputs_embeds`, and autocast path. If you are adapting it, start with `torch.bfloat16`, capture the `torch` / `torch_npu` / CANN versions, and open a focused PR or deployment issue with a minimal command and full stack trace.+Fun-ASR-Nano's LLM-based path is documented and validated for CUDA/vLLM, standard PyTorch CPU/GPU runs, and CPU/edge GGUF runtimes. Ascend NPU (`torch_npu`) is still not an official production runtime for this model. Do not assume that a SenseVoice or Paraformer NPU deployment means Fun-ASR-Nano will also work, because Nano also exercises the Qwen decoder, `inputs_embeds`, and backend-specific autocast/operator paths.++A community smoke test in [#3034](https://github.com/modelscope/FunASR/issues/3034) confirmed that the PyTorch `AutoModel(..., device="npu:*")` path can now enter the NPU backend after the autocast device fix, and produced the expected transcript on a 310P3 with `torch_npu 2.5.1` / CANN 8.5.1. That run was much slower than CPU (`rtf_avg` about 124 on NPU vs about 1.9 on CPU), so treat it as compatibility evidence, not a performance recommendation. The same report showed `AutoModelVLLM` on `vllm_ascend 0.9.2rc1` failing inside Qwen3 rotary embedding / `TransData` operator support; debug that path as a vLLM-Ascend runtime/operator issue and capture logs with `ASCEND_LAUNCH_BLOCKING=1`.++If you are adapting this backend, keep the first PR narrow: start with `torch.bfloat16`, capture `torch` / `torch_npu` / CANN / driver / NPU model, separate PyTorch `AutoModel` from `AutoModelVLLM`, and attach the minimal command plus full stack trace.## Readiness checklistdiff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.mdindex e0e9b0047..af7ae16d9 100644--- a/docs/deployment_matrix_zh.md+++ b/docs/deployment_matrix_zh.md@@ -70,7 +70,11 @@ Fun-ASR-Nano 走 vLLM 路径。请用自己的音频分布做 benchmark,并关### 我想在昇腾 NPU 上跑 Fun-ASR-Nano-Fun-ASR-Nano 的 LLM-based 路径目前主要按 CUDA/vLLM、标准 PyTorch CPU/GPU,以及 CPU/边缘 GGUF runtime 记录和验证;Ascend NPU(`torch_npu`)还不是这个模型的官方验证运行时。不要因为 SenseVoice 或 Paraformer 能在 NPU 上跑,就默认 Fun-ASR-Nano 也能直接跑通,因为 Nano 还会经过 Qwen 解码器、`inputs_embeds` 和 autocast 路径。若要适配,请先从 `torch.bfloat16` 开始,记录 `torch` / `torch_npu` / CANN 版本,并在最小 PR 或 deployment issue 里附上最小命令和完整错误栈。+Fun-ASR-Nano 的 LLM-based 路径目前主要按 CUDA/vLLM、标准 PyTorch CPU/GPU,以及 CPU/边缘 GGUF runtime 记录和验证;Ascend NPU(`torch_npu`)仍不是这个模型的官方生产运行时。不要因为 SenseVoice 或 Paraformer 能在 NPU 上跑,就默认 Fun-ASR-Nano 也能直接跑通,因为 Nano 还会经过 Qwen 解码器、`inputs_embeds`,以及后端相关的 autocast / 算子路径。++[#3034](https://github.com/modelscope/FunASR/issues/3034) 里的社区复测表明:修复 autocast device 之后,PyTorch `AutoModel(..., device="npu:*")` 路径已经能进入 NPU 后端,并在 310P3、`torch_npu 2.5.1`、CANN 8.5.1 环境里产出正确文本。但该 smoke run 明显慢于 CPU(NPU `rtf_avg` 约 124,CPU 约 1.9),所以这只能作为兼容性证据,不是性能推荐。同一报告里,`AutoModelVLLM` + `vllm_ascend 0.9.2rc1` 仍在 Qwen3 rotary embedding / `TransData` 算子支持处失败;这条路径应按 vLLM-Ascend runtime / 算子兼容问题继续排查,并建议带 `ASCEND_LAUNCH_BLOCKING=1` 复现日志。++如果你正在适配这个后端,请保持第一版范围很小:先从 `torch.bfloat16` 开始,记录 `torch` / `torch_npu` / CANN / 驱动 / NPU 型号,把 PyTorch `AutoModel` 和 `AutoModelVLLM` 分开验证,并在 PR 或 deployment issue 里附上最小命令和完整 stack trace。## 上线检查清单diff --git a/docs/model_selection.md b/docs/model_selection.mdindex 39f1f0d86..d2ac10ab0 100644--- a/docs/model_selection.md+++ b/docs/model_selection.md@@ -53,7 +53,7 @@ The `examples/openai_api` server exposes short aliases so application teams do n| `paraformer-en` | `paraformer-en` with VAD | You want a compact English route in OpenAI-style clients. || `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | You are evaluating LLM-based ASR, 31-language coverage, or vLLM acceleration. |-For Ascend NPU deployments, treat `fun-asr-nano` separately from SenseVoice / Paraformer. The Fun-ASR-Nano path is not officially validated on `torch_npu` yet; use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime unless you are doing a backend adaptation.+For Ascend NPU deployments, treat `fun-asr-nano` separately from SenseVoice / Paraformer. The Fun-ASR-Nano PyTorch `AutoModel` path has community compatibility evidence on 310P3 after the NPU autocast fix, but it was much slower than CPU in that smoke test; `AutoModelVLLM` still depends on vLLM-Ascend operator support and has hit Qwen3 rotary / `TransData` failures. Use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime for production unless you are actively validating an Ascend backend.Check the live service before wiring clients:diff --git a/docs/model_selection_zh.md b/docs/model_selection_zh.mdindex 287fc3392..a669733ab 100644--- a/docs/model_selection_zh.md+++ b/docs/model_selection_zh.md@@ -53,7 +53,7 @@ result = model.generate(input="meeting.wav")| `paraformer-en` | `paraformer-en` + VAD | OpenAI 风格客户端里的英文轻量路由。 || `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | 评估 LLM-based ASR、31 语种覆盖或 vLLM 加速。 |-如果部署目标是昇腾 NPU,请把 `fun-asr-nano` 和 SenseVoice / Paraformer 分开看。Fun-ASR-Nano 路径目前尚未在 `torch_npu` 上官方验证;除非正在做后端适配,否则优先使用 CUDA/vLLM、标准 PyTorch CPU/GPU 或 GGUF runtime。+如果部署目标是昇腾 NPU,请把 `fun-asr-nano` 和 SenseVoice / Paraformer 分开看。Fun-ASR-Nano 的 PyTorch `AutoModel` 路径在修复 NPU autocast 后已有 310P3 社区兼容性 smoke 结果,但该测试明显慢于 CPU;`AutoModelVLLM` 仍依赖 vLLM-Ascend 算子支持,并已遇到 Qwen3 rotary / `TransData` 失败。生产部署优先使用 CUDA/vLLM、标准 PyTorch CPU/GPU 或 GGUF runtime,除非你正在主动验证 Ascend 后端。接入客户端前先检查在线服务: