Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

fix(realtime): reduce default partial decode window (#3632)

zhifu gao649b5e5218d2
diff --git a/docs/vllm_guide.md b/docs/vllm_guide.md
index aa4111d61..8758a84d9 100644
--- a/docs/vllm_guide.md
+++ b/docs/vllm_guide.md
@@ -795,7 +795,7 @@ Because each refresh re-encodes from the sentence start, the longer a sentence,
**Usage guidance**
- Normal conversational speech has natural pauses, so VAD splits it into relatively short utterances and each partial's cost is naturally bounded — **usually nothing to worry about**.
-- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower. `serve_realtime_ws.py` bounds provisional previews with `--partial-window-sec 15` by default; under multi-client continuous-monologue load, reduce the window and lengthen `--decode-interval` based on measured output lag. This only affects provisional `partial`; VAD-locked sentences and STOP final output still run on the full audio. See the measured L20 starting point in §6.7.
+- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower. `serve_realtime_ws.py` bounds provisional previews with `--partial-window-sec 8` by default; raise the window only after measuring headroom for multi-client continuous-monologue load. This only affects provisional `partial`; VAD-locked sentences and STOP final output still run on the full audio. See the measured L20 starting point in §6.7.
### 6.6 Cost of speaker diarization (SPK) and how to enable it
diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.md
index 102cd006e..2bb2d25ef 100644
--- a/docs/vllm_guide_zh.md
+++ b/docs/vllm_guide_zh.md
@@ -788,7 +788,7 @@ Fun-ASR-Nano 的声学编码器(SenseVoice)是**全上下文、非流式**
**使用建议**
- 正常对话语音有自然停顿,VAD 会把它切成一句句较短的语音,每句 partial 的开销自然受限,**通常无需关注**。
-- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口;多客户端连续独白负载应根据实测输出滞后缩短窗口并增大 `--decode-interval`。这只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。可先参考 §6.7 的 L20 实测起点。
+- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 8` 限制临时预览窗口;多客户端连续独白负载应先从该值开始,并只在实测有余量时再提高窗口。它只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。可先参考 §6.7 的 L20 实测起点。
### 6.6 说话人分离(SPK)的代价与开关
diff --git a/docs/vllm_guide_zh_v2.md b/docs/vllm_guide_zh_v2.md
index edbe4e45f..a6345bc31 100644
--- a/docs/vllm_guide_zh_v2.md
+++ b/docs/vllm_guide_zh_v2.md
@@ -760,7 +760,7 @@ Fun-ASR-Nano 的声学编码器(SenseVoice)是**全上下文、非流式**
**使用建议**
- 正常对话语音有自然停顿,VAD 会把它切成一句句较短的语音,每句 partial 的开销自然受限,**通常无需关注**。
-- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口;多客户端或连续独白压测时可降到 `8-10`,并把 `--decode-interval` 提高到 `0.8-1.0`。这只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。
+- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 8` 限制临时预览窗口;多客户端或连续独白压测应先从该值开始,并只在实测有余量时再提高窗口。它只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。
### 6.6 说话人分离(SPK)的代价与开关
diff --git a/funasr/bin/realtime_ws.py b/funasr/bin/realtime_ws.py
index f49d01dc4..27f73232d 100644
--- a/funasr/bin/realtime_ws.py
+++ b/funasr/bin/realtime_ws.py
@@ -694,7 +694,7 @@ class RealtimeASRSession:
spk_tracker=None,
sample_rate=16000,
chunk_ms=960,
- partial_window_sec=15.0,
+ partial_window_sec=8.0,
audio_lookback_sec=5.0,
endpoint_mode="server",
):
@@ -1455,7 +1455,7 @@ async def handle_client(websocket, args):
asr_kwargs,
vad,
spk_tracker=spk_tracker,
- partial_window_sec=getattr(args, 'partial_window_sec', 15.0),
+ partial_window_sec=getattr(args, 'partial_window_sec', 8.0),
endpoint_mode=endpoint_mode,
)
logger.info(f"Client connected: {websocket.remote_address}")
@@ -1650,11 +1650,11 @@ def build_arg_parser():
"without loading the VAD model."
),
)
- parser.add_argument("--partial-window-sec", type=float, default=15.0,
+ parser.add_argument("--partial-window-sec", type=float, default=8.0,
help="Cap the interim partial re-decode window to the most recent N seconds. "
"A long ongoing speech segment is otherwise re-encoded from its start on "
"every chunk (O(L^2)), which saturates the GPU under concurrency and times "
- "out long-segment requests. Lower it (e.g. 8-10) for high-concurrency "
+ "out long-segment requests. Raise it only after measuring headroom for your "
"self-hosting; <=0 disables (legacy behaviour). Final transcripts are unaffected.")
parser.add_argument("--enable-spk", action="store_true", help="Enable streaming speaker diarization.")
parser.add_argument("--spk-model", type=str, default="iic/speech_eres2netv2_sv_zh-cn_16k-common")
diff --git a/tests/test_realtime_ws_service.py b/tests/test_realtime_ws_service.py
index 6d52b98b4..ebdda6500 100644
--- a/tests/test_realtime_ws_service.py
+++ b/tests/test_realtime_ws_service.py
@@ -111,7 +111,7 @@ def test_cli_defaults_disable_speaker_and_bound_partial_window():
args = module.build_arg_parser().parse_args([])
assert args.enable_spk is False
- assert args.partial_window_sec == 15.0
+ assert args.partial_window_sec == 8.0
assert args.endpoint_mode == "server"
assert args.log_session_stats_interval == 0.0
assert args.decode_batch_wait_ms == 10.0