Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs: add vLLM repeated punctuation troubleshooting (#3118)
zhifu gaoa6ebdb7fed42
diff --git a/docs/vllm_guide.md b/docs/vllm_guide.mdindex 567865fb9..eaac27a5c 100644--- a/docs/vllm_guide.md+++ b/docs/vllm_guide.md@@ -775,3 +775,23 @@ Streaming: `serve_realtime_ws.RealtimeASRSession`**Q: Slow first startup?**vLLM initialization takes 60–90 s (KV Cache + CUDA Graph warmup). Subsequent inferences are instant.++**Q: vLLM returns repeated punctuation such as `!!!!!!!!` but PyTorch/HF generate is normal. What should I check?**+This usually means the audio frontend and checkpoint can work, but the vLLM+prompt-embedding path or decoding parameters differ from the upstream runner.+Check these items before changing the model:++- Pass prompt embeddings to vLLM as float32:+ `EmbedsPrompt(prompt_embeds=input_embeds.float())`.+- Use ASR-style deterministic decoding. The Fun-ASR-Nano vLLM path defaults to+ `temperature=0.0`, `top_p=1.0`, and `skip_special_tokens=True`. In+ prompt-embeds mode, keep `repetition_penalty` at the neutral `1.0` unless you+ are using a token-prompt path; other values are normalized by FunASR's vLLM+ helpers to avoid vLLM CUDA scatter errors.+- Verify that `model_dir` and `vllm_model_dir` are the matching Fun-ASR-Nano+ pair. If clearing `vllm_model_dir` makes the same audio work through HF+ generate, keep debugging the vLLM path rather than the audio file.+- Log vLLM `finish_reason`, generated token ids, prompt embedding dtype, and+ prompt embedding shape for one failing sample. Repeated punctuation with+ `finish_reason="length"` usually points to decode/prompt mismatch rather than+ VAD or audio loading.diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.mdindex 90964c979..7b9381438 100644--- a/docs/vllm_guide_zh.md+++ b/docs/vllm_guide_zh.md@@ -779,3 +779,20 @@ RTFx 102 → 46。CER 不变。默认关闭。**Q: 首次慢?**vLLM 初始化 60-90s,之后即时。++**Q: vLLM 输出连续标点(例如 `!!!!!!!!`),但 PyTorch/HF generate 正常,应该先查什么?**+这通常说明音频 frontend 和 checkpoint 本身能工作,但 vLLM prompt-embedding+路径或解码参数和 upstream runner 不一致。改模型前先检查这些项:++- 传给 vLLM 的 prompt embeddings 要显式转成 float32:+ `EmbedsPrompt(prompt_embeds=input_embeds.float())`。+- 使用 ASR 更合适的确定性解码。Fun-ASR-Nano vLLM 路径默认使用+ `temperature=0.0`、`top_p=1.0` 和 `skip_special_tokens=True`。在+ prompt-embeds 模式下,`repetition_penalty` 保持中性的 `1.0`,除非你走的是+ token prompt 路径;FunASR 的 vLLM helper 会把其他值归一化,避免 vLLM CUDA scatter+ 错误。+- 确认 `model_dir` 和 `vllm_model_dir` 是匹配的一组 Fun-ASR-Nano 模型。如果清空+ `vllm_model_dir` 后同一音频走 HF generate 正常,就继续排查 vLLM 路径,而不是音频文件。+- 对一个失败样本记录 vLLM `finish_reason`、生成 token ids、prompt embedding dtype+ 和 shape。连续标点且 `finish_reason="length"` 时,通常更像解码/prompt 不匹配,而不是+ VAD 或音频读取问题。