Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

docs: clarify Fun-ASR-Nano vLLM precision boundary

LauraGPT331369fd64ed
diff --git a/docs/vllm_guide.md b/docs/vllm_guide.md
index 5aedf56ae..1894bf29b 100644
--- a/docs/vllm_guide.md
+++ b/docs/vllm_guide.md
@@ -872,6 +872,13 @@ Streaming: `serve_realtime_ws.RealtimeASRSession`
**Q: Slow first startup?**
vLLM initialization takes 60–90 s (KV Cache + CUDA Graph warmup). Subsequent inferences are instant.
+**Q: What happens when Fun-ASR-Nano vLLM uses `dtype="fp16"`?**
+The audio frontend and adaptor remain float16, but FunASR runs the Qwen3 decoder
+in bfloat16 because vLLM float16 decoding can produce degraded repeated output.
+This is automatic and keeps the two-byte decoder weight footprint. On hardware
+without BF16 support, use `dtype="fp32"`; the vLLM path does not claim an
+end-to-end FP16 decoder mode.
+
**Q: vLLM returns repeated punctuation such as `!!!!!!!!` but PyTorch/HF generate is normal. What should I check?**
This usually means the audio frontend and checkpoint can work, but the vLLM
prompt-embedding path or decoding parameters differ from the upstream runner.
diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.md
index 7b65fc621..0568fcd42 100644
--- a/docs/vllm_guide_zh.md
+++ b/docs/vllm_guide_zh.md
@@ -866,6 +866,12 @@ RTFx 102 → 46。CER 不变。默认关闭。
**Q: 首次慢?**
vLLM 初始化 60-90s,之后即时。
+**Q: Fun-ASR-Nano vLLM 使用 `dtype="fp16"` 时实际会怎样?**
+音频 frontend 与 adaptor 仍使用 FP16,但 FunASR 会让 Qwen3 decoder 使用 BF16,
+因为 vLLM 的 FP16 decoder 可能产生退化的重复输出。该行为自动生效,decoder 权重
+仍是两字节精度;不支持 BF16 的硬件请使用 `dtype="fp32"`。vLLM 路径不宣称支持
+端到端 FP16 decoder。
+
**Q: vLLM 输出连续标点(例如 `!!!!!!!!`),但 PyTorch/HF generate 正常,应该先查什么?**
这通常说明音频 frontend 和 checkpoint 本身能工作,但 vLLM prompt-embedding
路径或解码参数和 upstream runner 不一致。改模型前先检查这些项:
diff --git a/tests/test_docs_funasr_install_commands.py b/tests/test_docs_funasr_install_commands.py
index c5eec5990..2f22d83f4 100644
--- a/tests/test_docs_funasr_install_commands.py
+++ b/tests/test_docs_funasr_install_commands.py
@@ -341,8 +341,8 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime():
}
for name, text in readmes.items():
- assert 'python -m pip install -U "funasr==1.4.12"' in text, name
- assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.12" in text, name
+ assert 'python -m pip install -U "funasr==1.4.13"' in text, name
+ assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.13" in text, name
assert "runtime-llamacpp-v0.2.6" in text, name
assert "https://www.funasr.com/en/deploy/llama-cpp.html" in readmes["README.md"]