Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
fix(realtime): reduce default partial decode window (#3632)
zhifu gao649b5e5218d2
diff --git a/docs/vllm_guide.md b/docs/vllm_guide.mdindex aa4111d61..8758a84d9 100644--- a/docs/vllm_guide.md+++ b/docs/vllm_guide.md@@ -795,7 +795,7 @@ Because each refresh re-encodes from the sentence start, the longer a sentence,**Usage guidance**- Normal conversational speech has natural pauses, so VAD splits it into relatively short utterances and each partial's cost is naturally bounded — **usually nothing to worry about**.-- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower. `serve_realtime_ws.py` bounds provisional previews with `--partial-window-sec 15` by default; under multi-client continuous-monologue load, reduce the window and lengthen `--decode-interval` based on measured output lag. This only affects provisional `partial`; VAD-locked sentences and STOP final output still run on the full audio. See the measured L20 starting point in §6.7.+- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower. `serve_realtime_ws.py` bounds provisional previews with `--partial-window-sec 8` by default; raise the window only after measuring headroom for multi-client continuous-monologue load. This only affects provisional `partial`; VAD-locked sentences and STOP final output still run on the full audio. See the measured L20 starting point in §6.7.### 6.6 Cost of speaker diarization (SPK) and how to enable itdiff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.mdindex 102cd006e..2bb2d25ef 100644--- a/docs/vllm_guide_zh.md+++ b/docs/vllm_guide_zh.md@@ -788,7 +788,7 @@ Fun-ASR-Nano 的声学编码器(SenseVoice)是**全上下文、非流式****使用建议**- 正常对话语音有自然停顿,VAD 会把它切成一句句较短的语音,每句 partial 的开销自然受限,**通常无需关注**。-- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口;多客户端连续独白负载应根据实测输出滞后缩短窗口并增大 `--decode-interval`。这只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。可先参考 §6.7 的 L20 实测起点。+- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 8` 限制临时预览窗口;多客户端连续独白负载应先从该值开始,并只在实测有余量时再提高窗口。它只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。可先参考 §6.7 的 L20 实测起点。### 6.6 说话人分离(SPK)的代价与开关diff --git a/docs/vllm_guide_zh_v2.md b/docs/vllm_guide_zh_v2.mdindex edbe4e45f..a6345bc31 100644--- a/docs/vllm_guide_zh_v2.md+++ b/docs/vllm_guide_zh_v2.md@@ -760,7 +760,7 @@ Fun-ASR-Nano 的声学编码器(SenseVoice)是**全上下文、非流式****使用建议**- 正常对话语音有自然停顿,VAD 会把它切成一句句较短的语音,每句 partial 的开销自然受限,**通常无需关注**。-- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口;多客户端或连续独白压测时可降到 `8-10`,并把 `--decode-interval` 提高到 `0.8-1.0`。这只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。+- 只有**超长、不停顿的连续语音**(如长篇朗读)会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 8` 限制临时预览窗口;多客户端或连续独白压测应先从该值开始,并只在实测有余量时再提高窗口。它只影响临时 `partial`,VAD 锁定句和 STOP 最终结果仍走完整音频。### 6.6 说话人分离(SPK)的代价与开关diff --git a/funasr/bin/realtime_ws.py b/funasr/bin/realtime_ws.pyindex f49d01dc4..27f73232d 100644--- a/funasr/bin/realtime_ws.py+++ b/funasr/bin/realtime_ws.py@@ -694,7 +694,7 @@ class RealtimeASRSession:spk_tracker=None,sample_rate=16000,chunk_ms=960,- partial_window_sec=15.0,+ partial_window_sec=8.0,audio_lookback_sec=5.0,endpoint_mode="server",):@@ -1455,7 +1455,7 @@ async def handle_client(websocket, args):asr_kwargs,vad,spk_tracker=spk_tracker,- partial_window_sec=getattr(args, 'partial_window_sec', 15.0),+ partial_window_sec=getattr(args, 'partial_window_sec', 8.0),endpoint_mode=endpoint_mode,)logger.info(f"Client connected: {websocket.remote_address}")@@ -1650,11 +1650,11 @@ def build_arg_parser():"without loading the VAD model."),)- parser.add_argument("--partial-window-sec", type=float, default=15.0,+ parser.add_argument("--partial-window-sec", type=float, default=8.0,help="Cap the interim partial re-decode window to the most recent N seconds. ""A long ongoing speech segment is otherwise re-encoded from its start on ""every chunk (O(L^2)), which saturates the GPU under concurrency and times "- "out long-segment requests. Lower it (e.g. 8-10) for high-concurrency "+ "out long-segment requests. Raise it only after measuring headroom for your ""self-hosting; <=0 disables (legacy behaviour). Final transcripts are unaffected.")parser.add_argument("--enable-spk", action="store_true", help="Enable streaming speaker diarization.")parser.add_argument("--spk-model", type=str, default="iic/speech_eres2netv2_sv_zh-cn_16k-common")diff --git a/tests/test_realtime_ws_service.py b/tests/test_realtime_ws_service.pyindex 6d52b98b4..ebdda6500 100644--- a/tests/test_realtime_ws_service.py+++ b/tests/test_realtime_ws_service.py@@ -111,7 +111,7 @@ def test_cli_defaults_disable_speaker_and_bound_partial_window():args = module.build_arg_parser().parse_args([])assert args.enable_spk is False- assert args.partial_window_sec == 15.0+ assert args.partial_window_sec == 8.0assert args.endpoint_mode == "server"assert args.log_session_stats_interval == 0.0assert args.decode_batch_wait_ms == 10.0