Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

Document Fun-ASR-Nano Ascend NPU runtime bounds

zhifu gao7fed1f61baf9
diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.md
index f323161a5..ccf6dbdca 100644
--- a/docs/deployment_matrix.md
+++ b/docs/deployment_matrix.md
@@ -87,7 +87,11 @@ Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution a
### I want to run Fun-ASR-Nano on Ascend NPU
-Fun-ASR-Nano's LLM-based path is currently documented and validated for CUDA/vLLM, standard PyTorch CPU/GPU runs, and CPU/edge GGUF runtimes. Ascend NPU (`torch_npu`) is not an officially validated runtime for this model yet. Do not assume that a SenseVoice or Paraformer NPU deployment means Fun-ASR-Nano will also work, because Nano also exercises the Qwen decoder, `inputs_embeds`, and autocast path. If you are adapting it, start with `torch.bfloat16`, capture the `torch` / `torch_npu` / CANN versions, and open a focused PR or deployment issue with a minimal command and full stack trace.
+Fun-ASR-Nano's LLM-based path is documented and validated for CUDA/vLLM, standard PyTorch CPU/GPU runs, and CPU/edge GGUF runtimes. Ascend NPU (`torch_npu`) is still not an official production runtime for this model. Do not assume that a SenseVoice or Paraformer NPU deployment means Fun-ASR-Nano will also work, because Nano also exercises the Qwen decoder, `inputs_embeds`, and backend-specific autocast/operator paths.
+
+A community smoke test in [#3034](https://github.com/modelscope/FunASR/issues/3034) confirmed that the PyTorch `AutoModel(..., device="npu:*")` path can now enter the NPU backend after the autocast device fix, and produced the expected transcript on a 310P3 with `torch_npu 2.5.1` / CANN 8.5.1. That run was much slower than CPU (`rtf_avg` about 124 on NPU vs about 1.9 on CPU), so treat it as compatibility evidence, not a performance recommendation. The same report showed `AutoModelVLLM` on `vllm_ascend 0.9.2rc1` failing inside Qwen3 rotary embedding / `TransData` operator support; debug that path as a vLLM-Ascend runtime/operator issue and capture logs with `ASCEND_LAUNCH_BLOCKING=1`.
+
+If you are adapting this backend, keep the first PR narrow: start with `torch.bfloat16`, capture `torch` / `torch_npu` / CANN / driver / NPU model, separate PyTorch `AutoModel` from `AutoModelVLLM`, and attach the minimal command plus full stack trace.
## Readiness checklist
diff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.md
index e0e9b0047..af7ae16d9 100644
--- a/docs/deployment_matrix_zh.md
+++ b/docs/deployment_matrix_zh.md
@@ -70,7 +70,11 @@ Fun-ASR-Nano 走 vLLM 路径。请用自己的音频分布做 benchmark,并关
### 我想在昇腾 NPU 上跑 Fun-ASR-Nano
-Fun-ASR-Nano 的 LLM-based 路径目前主要按 CUDA/vLLM、标准 PyTorch CPU/GPU,以及 CPU/边缘 GGUF runtime 记录和验证;Ascend NPU(`torch_npu`)还不是这个模型的官方验证运行时。不要因为 SenseVoice 或 Paraformer 能在 NPU 上跑,就默认 Fun-ASR-Nano 也能直接跑通,因为 Nano 还会经过 Qwen 解码器、`inputs_embeds` 和 autocast 路径。若要适配,请先从 `torch.bfloat16` 开始,记录 `torch` / `torch_npu` / CANN 版本,并在最小 PR 或 deployment issue 里附上最小命令和完整错误栈。
+Fun-ASR-Nano 的 LLM-based 路径目前主要按 CUDA/vLLM、标准 PyTorch CPU/GPU,以及 CPU/边缘 GGUF runtime 记录和验证;Ascend NPU(`torch_npu`)仍不是这个模型的官方生产运行时。不要因为 SenseVoice 或 Paraformer 能在 NPU 上跑,就默认 Fun-ASR-Nano 也能直接跑通,因为 Nano 还会经过 Qwen 解码器、`inputs_embeds`,以及后端相关的 autocast / 算子路径。
+
+[#3034](https://github.com/modelscope/FunASR/issues/3034) 里的社区复测表明:修复 autocast device 之后,PyTorch `AutoModel(..., device="npu:*")` 路径已经能进入 NPU 后端,并在 310P3、`torch_npu 2.5.1`、CANN 8.5.1 环境里产出正确文本。但该 smoke run 明显慢于 CPU(NPU `rtf_avg` 约 124,CPU 约 1.9),所以这只能作为兼容性证据,不是性能推荐。同一报告里,`AutoModelVLLM` + `vllm_ascend 0.9.2rc1` 仍在 Qwen3 rotary embedding / `TransData` 算子支持处失败;这条路径应按 vLLM-Ascend runtime / 算子兼容问题继续排查,并建议带 `ASCEND_LAUNCH_BLOCKING=1` 复现日志。
+
+如果你正在适配这个后端,请保持第一版范围很小:先从 `torch.bfloat16` 开始,记录 `torch` / `torch_npu` / CANN / 驱动 / NPU 型号,把 PyTorch `AutoModel` 和 `AutoModelVLLM` 分开验证,并在 PR 或 deployment issue 里附上最小命令和完整 stack trace。
## 上线检查清单
diff --git a/docs/model_selection.md b/docs/model_selection.md
index 39f1f0d86..d2ac10ab0 100644
--- a/docs/model_selection.md
+++ b/docs/model_selection.md
@@ -53,7 +53,7 @@ The `examples/openai_api` server exposes short aliases so application teams do n
| `paraformer-en` | `paraformer-en` with VAD | You want a compact English route in OpenAI-style clients. |
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | You are evaluating LLM-based ASR, 31-language coverage, or vLLM acceleration. |
-For Ascend NPU deployments, treat `fun-asr-nano` separately from SenseVoice / Paraformer. The Fun-ASR-Nano path is not officially validated on `torch_npu` yet; use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime unless you are doing a backend adaptation.
+For Ascend NPU deployments, treat `fun-asr-nano` separately from SenseVoice / Paraformer. The Fun-ASR-Nano PyTorch `AutoModel` path has community compatibility evidence on 310P3 after the NPU autocast fix, but it was much slower than CPU in that smoke test; `AutoModelVLLM` still depends on vLLM-Ascend operator support and has hit Qwen3 rotary / `TransData` failures. Use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime for production unless you are actively validating an Ascend backend.
Check the live service before wiring clients:
diff --git a/docs/model_selection_zh.md b/docs/model_selection_zh.md
index 287fc3392..a669733ab 100644
--- a/docs/model_selection_zh.md
+++ b/docs/model_selection_zh.md
@@ -53,7 +53,7 @@ result = model.generate(input="meeting.wav")
| `paraformer-en` | `paraformer-en` + VAD | OpenAI 风格客户端里的英文轻量路由。 |
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | 评估 LLM-based ASR、31 语种覆盖或 vLLM 加速。 |
-如果部署目标是昇腾 NPU,请把 `fun-asr-nano` 和 SenseVoice / Paraformer 分开看。Fun-ASR-Nano 路径目前尚未在 `torch_npu` 上官方验证;除非正在做后端适配,否则优先使用 CUDA/vLLM、标准 PyTorch CPU/GPU 或 GGUF runtime。
+如果部署目标是昇腾 NPU,请把 `fun-asr-nano` 和 SenseVoice / Paraformer 分开看。Fun-ASR-Nano 的 PyTorch `AutoModel` 路径在修复 NPU autocast 后已有 310P3 社区兼容性 smoke 结果,但该测试明显慢于 CPU;`AutoModelVLLM` 仍依赖 vLLM-Ascend 算子支持,并已遇到 Qwen3 rotary / `TransData` 失败。生产部署优先使用 CUDA/vLLM、标准 PyTorch CPU/GPU 或 GGUF runtime,除非你正在主动验证 Ascend 后端。
接入客户端前先检查在线服务: