Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

docs: add vLLM API repeat troubleshooting

zhifu gao0573e013d338
diff --git a/docs/vllm_guide.md b/docs/vllm_guide.md
index 056798695..1b3d9b576 100644
--- a/docs/vllm_guide.md
+++ b/docs/vllm_guide.md
@@ -309,6 +309,31 @@ for result in engine.streaming_generate("audio.wav", language="中文"):
> **Note: `repetition_penalty` cannot be used with EmbedsPrompt.** Here the prompt is a sequence of embedding vectors with no corresponding token IDs, whereas `repetition_penalty` needs the prompt's token IDs to down-weight already-seen tokens in the logits; applying it under EmbedsPrompt **indexes out of bounds and triggers a CUDA device-side assert**.
+### Production API Stability Checklist
+
+When wrapping `AutoModelVLLM` in a long-running API service, keep request state isolated and pin safe decoding defaults:
+
+```python
+common = dict(
+ language="auto",
+ temperature=0.0,
+ repetition_penalty=1.0,
+ max_new_tokens=200,
+)
+
+for _ in range(2):
+ results = model.generate(["vad_segment_01.wav", "vad_segment_02.wav"], **common)
+ print([r["text"] for r in results])
+```
+
+If the same audio is normal on the first request but repeats on the second request:
+
+1. Run the minimal script above outside the API layer with the same VAD segments.
+2. If the script is stable, check whether the API wrapper reuses per-request variables, previous VAD segment lists, previous `results`, or accumulated text across requests.
+3. If the script also repeats, capture the exact `funasr`, `vllm`, and `torch` versions, plus the first and second outputs, before tuning any decoding parameter.
+
+Do not increase `repetition_penalty` to suppress repeats on Fun-ASR-Nano vLLM. The prompt-embeds path should stay at the neutral value `1.0`.
+
---
## 5. Offline Speech Recognition Service
diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.md
index a502df415..d453697ca 100644
--- a/docs/vllm_guide_zh.md
+++ b/docs/vllm_guide_zh.md
@@ -304,6 +304,31 @@ for result in engine.streaming_generate("audio.wav", language="中文"):
**注意:EmbedsPrompt 下不能用 `repetition_penalty`。** 此时 prompt 是 embedding 向量、没有对应的 token ID,而 `repetition_penalty` 要靠 prompt 的 token ID 在 logits 上给已出现的词降分;用在 EmbedsPrompt 上会**索引越界、触发 CUDA device-side assert**。
+### 生产 API 稳定性清单
+
+把 `AutoModelVLLM` 封装成长驻 API 服务时,请隔离每次请求的状态,并固定安全的解码默认值:
+
+```python
+common = dict(
+ language="auto",
+ temperature=0.0,
+ repetition_penalty=1.0,
+ max_new_tokens=200,
+)
+
+for _ in range(2):
+ results = model.generate(["vad_segment_01.wav", "vad_segment_02.wav"], **common)
+ print([r["text"] for r in results])
+```
+
+如果同一个音频第一次请求正常、第二次请求开始重复:
+
+1. 先把 API 层拿掉,用相同 VAD 分段跑上面的最小脚本。
+2. 如果最小脚本稳定,优先检查 API 封装是否复用了请求级变量、上一轮 VAD 分段列表、上一轮 `results` 或累积文本。
+3. 如果最小脚本也重复,再记录完整的 `funasr`、`vllm`、`torch` 版本,以及第一次和第二次输出文本,再调整其它解码参数。
+
+不要通过调大 `repetition_penalty` 来压制 Fun-ASR-Nano vLLM 重复输出;prompt-embeds 路径应保持中性值 `1.0`。
+
### 输出特性
| 累积音频 | 输出质量 |
diff --git a/docs/vllm_guide_zh_v2.md b/docs/vllm_guide_zh_v2.md
index a502df415..d453697ca 100644
--- a/docs/vllm_guide_zh_v2.md
+++ b/docs/vllm_guide_zh_v2.md
@@ -304,6 +304,31 @@ for result in engine.streaming_generate("audio.wav", language="中文"):
**注意:EmbedsPrompt 下不能用 `repetition_penalty`。** 此时 prompt 是 embedding 向量、没有对应的 token ID,而 `repetition_penalty` 要靠 prompt 的 token ID 在 logits 上给已出现的词降分;用在 EmbedsPrompt 上会**索引越界、触发 CUDA device-side assert**。
+### 生产 API 稳定性清单
+
+把 `AutoModelVLLM` 封装成长驻 API 服务时,请隔离每次请求的状态,并固定安全的解码默认值:
+
+```python
+common = dict(
+ language="auto",
+ temperature=0.0,
+ repetition_penalty=1.0,
+ max_new_tokens=200,
+)
+
+for _ in range(2):
+ results = model.generate(["vad_segment_01.wav", "vad_segment_02.wav"], **common)
+ print([r["text"] for r in results])
+```
+
+如果同一个音频第一次请求正常、第二次请求开始重复:
+
+1. 先把 API 层拿掉,用相同 VAD 分段跑上面的最小脚本。
+2. 如果最小脚本稳定,优先检查 API 封装是否复用了请求级变量、上一轮 VAD 分段列表、上一轮 `results` 或累积文本。
+3. 如果最小脚本也重复,再记录完整的 `funasr`、`vllm`、`torch` 版本,以及第一次和第二次输出文本,再调整其它解码参数。
+
+不要通过调大 `repetition_penalty` 来压制 Fun-ASR-Nano vLLM 重复输出;prompt-embeds 路径应保持中性值 `1.0`。
+
### 输出特性
| 累积音频 | 输出质量 |