Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

feat(moss): support vLLM diarized responses

LauraGPT14e154ad2abd
diff --git a/docs/moss_transcribe_diarize.md b/docs/moss_transcribe_diarize.md
index 6467de591..02ed5d460 100644
--- a/docs/moss_transcribe_diarize.md
+++ b/docs/moss_transcribe_diarize.md
@@ -21,7 +21,9 @@ This guide was verified against:
- source: `OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
- model: `OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
- license: Apache-2.0 in the pinned upstream source and model metadata
-- vLLM nightly index: `68b4a1d582818e67adc903bf1b8fc5a5447da2fa`
+- vLLM release: `v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`
+- vLLM CUDA 12.9 x86_64 wheel SHA-256:
+ `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`
Pin all three revisions. The model uses `trust_remote_code`; do not execute a
floating model revision in a production service.
@@ -76,36 +78,48 @@ model = AutoModel(
backend="vllm",
vllm_base_url="http://127.0.0.1:8898/v1",
vllm_model="moss-transcribe-diarize",
+ vllm_response_format="diarized_json",
disable_update=True,
)
result = model.generate("audio.wav")[0]
```
The vLLM adapter sends the documented OpenAI-compatible multipart request and
-normalizes the returned raw text. Authentication can be supplied with
-`vllm_api_key`; keep it in a secret store rather than source code.
+normalizes the official speaker-attributed `segments` directly into
+`sentence_info`. Authentication can be supplied with `vllm_api_key`; keep it
+in a secret store rather than source code.
+
+`vllm_response_format="diarized_json"` is the structured production path on
+the pinned vLLM revision. The compatibility default remains `json` so existing
+clients keep the exact tagged generation in `raw_text`; in structured mode,
+`raw_text` is the cleaned `text` returned by vLLM and the authoritative speaker
+metadata is in `sentence_info`.
## Choose a serving path
| Path | Environment | Response contract | Use when |
|---|---|---|---|
-| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=json` returns raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |
+| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=diarized_json` returns OpenAI-compatible speaker segments; `response_format=json` keeps raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |
| SGLang Omni | CUDA 13 in the current upstream guide | `response_format=verbose_json` returns parsed segments | You need structured speaker segments directly from the API |
| Transformers | PyTorch process | Python objects and raw tagged text | Evaluation, debugging, or custom preprocessing |
The two HTTP backends share `/v1/audio/transcriptions`, but their documented
-response formats are not interchangeable. Do not promise vLLM `segments`
-without testing the exact vLLM revision.
+response formats are not interchangeable. The pinned vLLM revision supports
+`diarized_json`; test the exact server revision before promising `segments`.
## vLLM
-Create an isolated environment and install the upstream-pinned nightly build:
+Create an isolated environment and install the verified vLLM release:
```bash
uv venv --python 3.12 .venv-moss
-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
- --torch-backend=auto \
- --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
+curl -fL \
+ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \
+ -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl
+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
+ | sha256sum -c -
+uv pip install --python .venv-moss/bin/python --torch-backend=auto \
+ "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"
```
The `audio` extra is required. A plain `vllm` install can start the server but
@@ -124,13 +138,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
--port 8898
```
-Submit audio and validate the speaker-tagged text:
+Submit audio and validate the official speaker-attributed response:
```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F model=moss-transcribe-diarize \
- -F response_format=json \
+ -F response_format=diarized_json \
-F temperature=0 \
| tee moss-transcription.json
@@ -140,19 +154,36 @@ import json
with open("moss-transcription.json", encoding="utf-8") as stream:
payload = json.load(stream)
text = payload.get("text", "")
+segments = payload.get("segments", [])
assert text.strip(), payload
-assert "[S01]" in text, text
-print(text)
+assert segments, payload
+assert all(
+ isinstance(item.get("speaker"), str)
+ and item.get("text")
+ and item.get("start") <= item.get("end")
+ for item in segments
+), payload
+print(text, sorted({item["speaker"] for item in segments}))
PY
```
+For an audit of the model's compact tagged generation, repeat the request with
+`response_format=json` and verify `[S01]` appears in the returned `text`.
+
For long recordings, test `max_completion_tokens` against the longest expected
meeting. A larger limit can increase memory use and tail latency.
### Reproduced vLLM contract
-The FunASR ecosystem validation used vLLM
-`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB:
+The structured-response validation used vLLM 0.27.1, Torch
+`2.13.0+cu129`, and one H100 80GB. The 15.1685-second two-speaker probe
+(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)
+returned two `diarized_json` segments labelled `S01` and `S02`; the same
+request through the FunASR adapter returned two monotonic `sentence_info`
+entries with the same speakers.
+
+Earlier raw-response validation on vLLM
+`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB found:
- the bundled 6.000-second sample
(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)
@@ -165,6 +196,12 @@ This proves the pinned API and speaker-return contract on those inputs. It is
not a diarization accuracy result, overlap test, throughput benchmark, or
production capacity claim.
+The older nightly commit
+`68b4a1d582818e67adc903bf1b8fc5a5447da2fa` predates vLLM
+[`#48543`](https://github.com/vllm-project/vllm/pull/48543). It accepts
+`response_format=json` but returns HTTP 400 for `diarized_json`; upgrade to the
+verified release instead of silently retrying an expensive transcription.
+
The FunASR adapter was also tested on both `backend="hf"` and
`backend="vllm"` with model revision
`e8681d68e7042738ffca8ac8212bc8fcb1131ab8`. The 15.1685-second two-speaker
diff --git a/docs/moss_transcribe_diarize_zh.md b/docs/moss_transcribe_diarize_zh.md
index c0409a759..324d8d68e 100644
--- a/docs/moss_transcribe_diarize_zh.md
+++ b/docs/moss_transcribe_diarize_zh.md
@@ -19,7 +19,9 @@ MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话
- 源码:`OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
- 模型:`OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
- 许可证:固定上游源码和模型元数据中的 Apache-2.0
-- vLLM nightly 索引:`68b4a1d582818e67adc903bf1b8fc5a5447da2fa`
+- vLLM 发布版:`v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`
+- vLLM CUDA 12.9 x86_64 wheel SHA-256:
+ `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`
三者都要固定。模型依赖 `trust_remote_code`,生产服务不要执行浮动的模型 revision。
@@ -68,34 +70,46 @@ model = AutoModel(
backend="vllm",
vllm_base_url="http://127.0.0.1:8898/v1",
vllm_model="moss-transcribe-diarize",
+ vllm_response_format="diarized_json",
disable_update=True,
)
result = model.generate("audio.wav")[0]
```
-vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求,并统一解析返回文本。
-认证可通过 `vllm_api_key` 提供;请放在 secret store,不要写入源码。
+vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求,并把官方返回的
+speaker `segments` 直接映射到 `sentence_info`。认证可通过 `vllm_api_key` 提供;
+请放在 secret store,不要写入源码。
+
+固定 vLLM revision 的生产结构化路径是
+`vllm_response_format="diarized_json"`。为保持已有客户端兼容,默认值仍是 `json`,
+此时 `raw_text` 保留原始标签生成;结构化模式下,`raw_text` 是 vLLM 返回的清理后
+`text`,权威说话人信息位于 `sentence_info`。
## 选择服务后端
| 路径 | 环境 | 响应契约 | 适用场景 |
|---|---|---|---|
-| vLLM | CUDA 12(`cu129`)或 CUDA 13(`cu130`) | `response_format=json` 返回原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM,或需要其调度能力 |
+| vLLM | CUDA 12(`cu129`)或 CUDA 13(`cu130`) | `response_format=diarized_json` 返回 OpenAI-compatible 说话人 segments;`response_format=json` 保留原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM,或需要其调度能力 |
| SGLang Omni | 当前上游指南为 CUDA 13 | `response_format=verbose_json` 返回解析后的 segments | API 需要直接返回结构化说话人分段 |
| Transformers | PyTorch 进程 | Python 对象和原始标签文本 | 评测、排障或自定义预处理 |
两个 HTTP 后端都使用 `/v1/audio/transcriptions`,但官方文档中的响应格式不能混为
-一谈。没有对 exact vLLM revision 做验证时,不要承诺 vLLM 返回 `segments`。
+一谈。固定的 vLLM revision 支持 `diarized_json`;承诺 `segments` 前仍要验证实际
+部署的 exact server revision。
## vLLM
-创建隔离环境并安装上游固定的 nightly build:
+创建隔离环境并安装已验证的 vLLM 发布版:
```bash
uv venv --python 3.12 .venv-moss
-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
- --torch-backend=auto \
- --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
+curl -fL \
+ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \
+ -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl
+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
+ | sha256sum -c -
+uv pip install --python .venv-moss/bin/python --torch-backend=auto \
+ "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"
```
必须安装 `audio` extra。只安装 plain `vllm` 时服务可以启动,但因为没有音频解码器,
@@ -113,13 +127,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
--port 8898
```
-提交音频并验证说话人标签文本:
+提交音频并验证官方结构化说话人响应:
```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F model=moss-transcribe-diarize \
- -F response_format=json \
+ -F response_format=diarized_json \
-F temperature=0 \
| tee moss-transcription.json
@@ -129,18 +143,34 @@ import json
with open("moss-transcription.json", encoding="utf-8") as stream:
payload = json.load(stream)
text = payload.get("text", "")
+segments = payload.get("segments", [])
assert text.strip(), payload
-assert "[S01]" in text, text
-print(text)
+assert segments, payload
+assert all(
+ isinstance(item.get("speaker"), str)
+ and item.get("text")
+ and item.get("start") <= item.get("end")
+ for item in segments
+), payload
+print(text, sorted({item["speaker"] for item in segments}))
PY
```
+如需审计模型原始紧凑标签生成,请把请求改为 `response_format=json`,并验证返回
+`text` 中包含 `[S01]`。
+
长音频需要按业务最长会议验证 `max_completion_tokens`。增大上限也会影响显存和尾延迟。
### 已复现的 vLLM 契约
-FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
-`2.11.0+cu129` 和一张 H100 80GB:
+结构化响应验证使用 vLLM 0.27.1、Torch `2.13.0+cu129` 和一张 H100 80GB。
+15.1685 秒双说话人样例
+(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)
+返回两个 `diarized_json` segments,标签为 `S01`、`S02`;同一请求通过 FunASR
+适配器后,返回两个时间单调且说话人一致的 `sentence_info`。
+
+较早的 raw-response 验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
+`2.11.0+cu129` 和一张 H100 80GB,结果如下:
- 仓库内 6.000 秒样例
(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)
@@ -152,6 +182,11 @@ FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
这证明固定版本对这些输入满足 API 和说话人返回契约,不是 diarization 精度、重叠语音、
吞吐或生产容量结论。
+旧 nightly commit `68b4a1d582818e67adc903bf1b8fc5a5447da2fa` 早于 vLLM
+[`#48543`](https://github.com/vllm-project/vllm/pull/48543):它支持
+`response_format=json`,但 `diarized_json` 会返回 HTTP 400。应升级到已验证发布版,
+不要静默重试一次成本较高的转写请求。
+
FunASR 适配器还分别使用 `backend="hf"` 和 `backend="vllm"`,按模型 revision
`e8681d68e7042738ffca8ac8212bc8fcb1131ab8` 做了真实测试。15.1685 秒的双说话人
样例(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)通过
diff --git a/funasr/models/moss_transcribe_diarize/model.py b/funasr/models/moss_transcribe_diarize/model.py
index fcc302be5..0f00dfb1a 100644
--- a/funasr/models/moss_transcribe_diarize/model.py
+++ b/funasr/models/moss_transcribe_diarize/model.py
@@ -7,6 +7,7 @@ https://github.com/OpenMOSS/MOSS-Transcribe-Diarize
import copy
import io
+import math
import mimetypes
import os
import re
@@ -127,6 +128,66 @@ def _result_from_transcript(key, transcript):
}
+def _result_from_diarized_response(key, response):
+ """Normalize vLLM's OpenAI-compatible diarized JSON response."""
+ text = response.get("text") if isinstance(response, dict) else None
+ segments = response.get("segments") if isinstance(response, dict) else None
+ if not isinstance(text, str) or not isinstance(segments, list):
+ raise RuntimeError(
+ "vLLM diarized_json response requires text and segments fields"
+ )
+
+ sentence_info = []
+ timestamps = []
+ for index, segment in enumerate(segments):
+ if not isinstance(segment, dict):
+ raise RuntimeError(
+ "vLLM diarized_json segment %d must be an object" % index
+ )
+ start = segment.get("start")
+ end = segment.get("end")
+ segment_text = segment.get("text")
+ speaker = segment.get("speaker")
+ numeric = (
+ isinstance(start, (int, float))
+ and not isinstance(start, bool)
+ and isinstance(end, (int, float))
+ and not isinstance(end, bool)
+ )
+ if (
+ not numeric
+ or not math.isfinite(start)
+ or not math.isfinite(end)
+ or start < 0
+ or end < start
+ or not isinstance(segment_text, str)
+ or not isinstance(speaker, str)
+ or not speaker
+ ):
+ raise RuntimeError(
+ "vLLM diarized_json segment %d has an invalid contract" % index
+ )
+ timestamp = [int(round(start * 1000)), int(round(end * 1000))]
+ timestamps.append(timestamp)
+ sentence_info.append(
+ {
+ "start": timestamp[0],
+ "end": timestamp[1],
+ "text": segment_text,
+ "sentence": segment_text,
+ "spk": speaker,
+ "timestamp": [timestamp],
+ }
+ )
+ return {
+ "key": key,
+ "text": text,
+ "raw_text": text,
+ "timestamp": timestamps,
+ "sentence_info": sentence_info,
+ }
+
+
@tables.register("model_classes", "MOSS-Transcribe-Diarize")
@tables.register("model_classes", DEFAULT_MODEL)
class MossTranscribeDiarize(nn.Module):
@@ -157,6 +218,9 @@ class MossTranscribeDiarize(nn.Module):
self.vllm_model = kwargs.get("vllm_model", DEFAULT_MODEL)
self.vllm_api_key = kwargs.get("vllm_api_key", "EMPTY")
self.vllm_timeout = float(kwargs.get("vllm_timeout", 600.0))
+ self.vllm_response_format = kwargs.get("vllm_response_format", "json").lower()
+ if self.vllm_response_format not in {"json", "diarized_json"}:
+ raise ValueError("vllm_response_format must be 'json' or 'diarized_json'")
self.http_session = kwargs.get("http_session")
self.hf_model = None
@@ -240,10 +304,18 @@ class MossTranscribeDiarize(nn.Module):
results = []
for index, audio in enumerate(inputs):
if self.backend == "vllm":
- transcript = self._transcribe_vllm(audio, **kwargs)
+ response = self._transcribe_vllm(audio, **kwargs)
+ if self.vllm_response_format == "diarized_json":
+ results.append(
+ _result_from_diarized_response(keys[index], response)
+ )
+ else:
+ results.append(
+ _result_from_transcript(keys[index], response["text"])
+ )
else:
transcript = self._transcribe_hf(audio, **kwargs)
- results.append(_result_from_transcript(keys[index], transcript))
+ results.append(_result_from_transcript(keys[index], transcript))
return results, {
"batch_data_time": max(audio_duration, 1e-9),
"forward": time.perf_counter() - started,
@@ -324,7 +396,7 @@ class MossTranscribeDiarize(nn.Module):
filename, payload, content_type = self._multipart_audio(audio)
data = {
"model": self.vllm_model,
- "response_format": "json",
+ "response_format": self.vllm_response_format,
"temperature": str(kwargs.get("temperature", 0)),
}
prompt = kwargs.get("prompt")
@@ -345,11 +417,15 @@ class MossTranscribeDiarize(nn.Module):
)
response.raise_for_status()
result = response.json()
- if not isinstance(result, dict) or not isinstance(result.get("text"), str):
+ if not isinstance(result, dict):
+ raise RuntimeError("vLLM transcription response must be a JSON object")
+ if self.vllm_response_format == "json" and not isinstance(
+ result.get("text"), str
+ ):
raise RuntimeError(
"vLLM transcription response did not contain a text field"
)
- return result["text"]
+ return result
@staticmethod
def _multipart_audio(audio):
diff --git a/tests/test_moss_transcribe_diarize_docs.py b/tests/test_moss_transcribe_diarize_docs.py
index 8b2a956de..219d663c2 100644
--- a/tests/test_moss_transcribe_diarize_docs.py
+++ b/tests/test_moss_transcribe_diarize_docs.py
@@ -25,15 +25,19 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
"cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3",
"OpenMOSS-Team/MOSS-Transcribe-Diarize",
"e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
- "68b4a1d582818e67adc903bf1b8fc5a5447da2fa",
+ "6e448d0ea9bf3d88d898b65449ca6dc2aec170ac",
+ "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b",
"vllm[audio]",
- "0.23.1rc1.dev949+g68b4a1d58",
+ "vLLM 0.27.1",
+ "68b4a1d582818e67adc903bf1b8fc5a5447da2fa",
"dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912",
"43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8",
"S01 -> S02 -> S01",
"/v1/audio/transcriptions",
"response_format=json",
+ "response_format=diarized_json",
"response_format=verbose_json",
+ 'vllm_response_format="diarized_json"',
"[S01]",
"Apache-2.0",
"sentence_info",
diff --git a/tests/test_moss_transcribe_diarize_model.py b/tests/test_moss_transcribe_diarize_model.py
index 938a6fa93..55176dae3 100644
--- a/tests/test_moss_transcribe_diarize_model.py
+++ b/tests/test_moss_transcribe_diarize_model.py
@@ -138,6 +138,89 @@ class MossVllmBackendTest(unittest.TestCase):
self.assertEqual(content_type, "audio/wav")
response.raise_for_status.assert_called_once_with()
+ def test_normalizes_official_vllm_diarized_json_response(self):
+ session = MagicMock()
+ response = MagicMock()
+ response.json.return_value = {
+ "task": "transcribe",
+ "duration": 3.5,
+ "text": "hello again",
+ "segments": [
+ {
+ "type": "transcript.text.segment",
+ "id": "seg_0",
+ "start": 0.25,
+ "end": 1.5,
+ "text": "hello",
+ "speaker": "S01",
+ },
+ {
+ "type": "transcript.text.segment",
+ "id": "seg_1",
+ "start": 2.0,
+ "end": 3.5,
+ "text": "again",
+ "speaker": "S02",
+ },
+ ],
+ }
+ session.post.return_value = response
+
+ model = MossTranscribeDiarize(
+ backend="vllm",
+ vllm_base_url="http://vllm.test:8000/v1",
+ vllm_response_format="diarized_json",
+ http_session=session,
+ )
+ results, _ = model.inference(
+ [np.zeros(1600, dtype=np.float32)], key=["meeting"]
+ )
+
+ self.assertEqual(
+ session.post.call_args.kwargs["data"]["response_format"],
+ "diarized_json",
+ )
+ self.assertEqual(results[0]["text"], "hello again")
+ self.assertEqual(results[0]["raw_text"], "hello again")
+ self.assertEqual(results[0]["timestamp"], [[250, 1500], [2000, 3500]])
+ self.assertEqual(
+ [(item["spk"], item["text"]) for item in results[0]["sentence_info"]],
+ [("S01", "hello"), ("S02", "again")],
+ )
+
+ def test_rejects_unsupported_vllm_response_format(self):
+ with self.assertRaisesRegex(ValueError, "json.*diarized_json"):
+ MossTranscribeDiarize(
+ backend="vllm",
+ vllm_base_url="http://vllm.test:8000/v1",
+ vllm_response_format="verbose_json",
+ )
+
+ def test_rejects_malformed_vllm_diarized_segment(self):
+ session = MagicMock()
+ response = MagicMock()
+ response.json.return_value = {
+ "text": "broken",
+ "segments": [
+ {
+ "start": 2.0,
+ "end": 1.0,
+ "text": "broken",
+ "speaker": "S01",
+ }
+ ],
+ }
+ session.post.return_value = response
+ model = MossTranscribeDiarize(
+ backend="vllm",
+ vllm_base_url="http://vllm.test:8000/v1",
+ vllm_response_format="diarized_json",
+ http_session=session,
+ )
+
+ with self.assertRaisesRegex(RuntimeError, "segment 0.*invalid contract"):
+ model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
+
if __name__ == "__main__":
unittest.main()
diff --git a/web-pages/product-site/data/deployments.json b/web-pages/product-site/data/deployments.json
index 80bdd6afc..a83d79487 100644
--- a/web-pages/product-site/data/deployments.json
+++ b/web-pages/product-site/data/deployments.json
@@ -1,6 +1,6 @@
{
"schema_version": 1,
- "verified": "2026-08-29",
+ "verified": "2026-08-30",
"deployments": [
{
"id": "moss-transcribe-diarize",
@@ -13,10 +13,10 @@
"models": ["OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)"],
"operating_systems": ["Linux"],
"interfaces": ["OpenAI-compatible HTTP", "vLLM", "SGLang Omni", "Transformers"],
- "tested": {"funasr": "AutoModel adapter@a3a2de0a; third-party model@e8681d68", "runtime": "Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB", "verified": "2026-08-29"},
+ "tested": {"funasr": "AutoModel diarized_json adapter; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
"commands": {
"install": [
- "uv venv --python 3.12 .venv-moss && uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' --torch-backend=auto --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129",
+ "uv venv --python 3.12 .venv-moss && curl -fL https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl && echo 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl' | sha256sum -c - && uv pip install --python .venv-moss/bin/python --torch-backend=auto \"vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl\"",
"HF_HUB_ENABLE_HF_TRANSFER=1 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize --revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 --local-dir .models/moss-transcribe-diarize"
],
"launch": [
@@ -27,9 +27,9 @@
"curl -fsS http://127.0.0.1:8898/v1/models"
],
"smoke": [
- "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=json -F temperature=0 | tee /tmp/moss-transcription.json",
- "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\ntext = payload.get('text', '')\nassert text.strip(), payload\nassert '[S01]' in text, text\nprint(text)\nPY",
- "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
+ "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=diarized_json -F temperature=0 | tee /tmp/moss-transcription.json",
+ "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\ntext = payload.get('text', '')\nsegments = payload.get('segments', [])\nassert text.strip() and segments, payload\nassert all(isinstance(item.get('speaker'), str) and item.get('text') and item.get('start') <= item.get('end') for item in segments), payload\nprint(text, sorted({item['speaker'] for item in segments}))\nPY",
+ "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', vllm_response_format='diarized_json', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
]
},
"evidence": [
@@ -37,22 +37,24 @@
{"label": "upstream source pinned at cb765f2", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/commit/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3"},
{"label": "official model snapshot pinned at e8681d68", "url": "https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize/tree/e8681d68e7042738ffca8ac8212bc8fcb1131ab8"},
{"label": "official vLLM and SGLang Omni serving guide", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3/README.md"},
+ {"label": "vLLM official diarized_json response contract", "url": "https://github.com/vllm-project/vllm/pull/48543"},
+ {"label": "verified vLLM 0.27.1 release", "url": "https://github.com/vllm-project/vllm/releases/tag/v0.27.1"},
{"label": "FunASR ecosystem integration guide", "url": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md"},
{"label": "FunASR AutoModel adapter merge and exact-head CI", "url": "https://github.com/modelscope/FunASR/pull/3558"}
],
"benchmarks": [
{
"model": "OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68",
- "runtime": "vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129",
+ "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129",
"hardware": "NVIDIA H100 80GB HBM3",
- "workload": "OpenAI-compatible transcription API smoke test plus synthetic A-to-B-to-A speaker-return probe",
- "audio": "Bundled 6.000 s Mandarin sample; synthetic 19.147 s file built as sample A + 0.8 s silence + sample B + 0.8 s silence + sample A",
- "settings": "Pinned model and vLLM revisions; vllm[audio]; response_format=json; temperature=0; one request at a time",
- "timing_scope": "Client wall time after /health became ready; local HTTP and decoding included; model download and engine startup excluded",
- "result": "Both requests returned HTTP 200. The 6.000 s sample produced S01 with timestamps 0.96-5.94. The 19.147 s probe produced S01 -> S02 -> S01 with timestamps covering 0.96-19.08. Observed client times were 1.034 s and 0.171 s respectively.",
- "qualification": "A functional API contract and speaker-return probe on synthetic concatenation, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.",
+ "workload": "OpenAI-compatible diarized_json API smoke plus FunASR AutoModel structured-response mapping",
+ "audio": "15.1685 s two-speaker probe; SHA256 43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8",
+ "settings": "Pinned model revision; vLLM 0.27.1; vllm[audio]; response_format=diarized_json; temperature=0; max_completion_tokens=768",
+ "timing_scope": "Functional response validation after /health became ready; no latency or throughput claim",
+ "result": "Direct vLLM HTTP and the FunASR adapter each returned two non-empty monotonic segments with speakers S01 and S02.",
+ "qualification": "A structured response contract smoke, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.",
"source": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md",
- "verified": "2026-08-29"
+ "verified": "2026-08-30"
}
],
"translations": {
@@ -62,11 +64,11 @@
"fit": ["会议、访谈、播客和通话的多人长音频", "希望通过同一个模型输出文本、时间戳与说话人编号", "已有 NVIDIA GPU,并需要 OpenAI 兼容的音频转写接口"],
"not_fit": ["实时流式字幕或低延迟端点检测", "CPU、桌面边缘设备或 Windows 原生部署", "要求 FunASR 自有模型权重的项目"],
"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers 与 vLLM 提供统一的结构化结果。",
- "primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 nightly commit,升级前必须重跑真实多人长音频。",
+ "primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 0.27.1 发布版,升级前必须重跑真实多人长音频。",
"status_label": "社区验证",
- "operations": ["固定模型 revision、FunASR adapter merge、vLLM nightly commit、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 保留 raw_text,并统一返回 text、timestamp 和带 spk 的 sentence_info;直接调用 vLLM 时 json 仍是原始 [Sxx] 标记文本"],
+ "operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 可用 vLLM diarized_json 直接统一 text、timestamp 和带 spk 的 sentence_info;json 兼容路径仍在 raw_text 保留原始 [Sxx] 标记文本"],
"security": ["把服务绑定内网,由网关完成认证、TLS、限流及音频大小/时长限制", "固定并审计 trust_remote_code 对应的模型 revision,不执行浮动 main 代码", "隔离模型缓存、上传临时目录和生成日志,按数据保留策略清理原始音频"],
- "troubleshooting": ["启动前确认 CUDA 12 使用 cu129 nightly 源,CUDA 13 使用上游文档列出的 cu130 源", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "直接调用 vLLM 时 json 没有 segments 字段是当前接口边界;使用 FunASR AutoModel 归一化结果,或改用并单独验证 SGLang Omni verbose_json"]
+ "troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;其他 CUDA 栈需按上游发布资产重新验证", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "需要结构化说话人 segments 时使用 vLLM diarized_json;旧 68b4a1d5 nightly 会返回 HTTP 400,json 只返回原始 [Sxx] 标签文本"]
},
"en": {
"name": "MOSS unified transcription and diarization",
@@ -74,11 +76,11 @@
"fit": ["Multi-speaker meetings, interviews, podcasts, and calls", "One model output for text, timestamps, and speaker identifiers", "NVIDIA GPU deployments that need an OpenAI-compatible audio transcription endpoint"],
"not_fit": ["Realtime streaming captions or low-latency endpoint detection", "CPU, desktop-edge, or native Windows deployments", "Projects that require FunASR-owned model weights"],
"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers and vLLM.",
- "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins a nightly commit and must be revalidated on real long multi-speaker audio before upgrades.",
+ "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins release 0.27.1 and must be revalidated on real long multi-speaker audio before upgrades.",
"status_label": "Community verified",
- "operations": ["Pin the model revision, FunASR adapter merge, vLLM nightly commit, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel preserves raw_text and normalizes text, timestamp, and sentence_info with spk; direct vLLM json still returns raw [Sxx]-tagged text"],
+ "operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel can map vLLM diarized_json directly into text, timestamp, and sentence_info with spk; the json compatibility path still preserves raw [Sxx]-tagged text in raw_text"],
"security": ["Bind workers to a private network and put authentication, TLS, rate limits, and audio size/duration limits at the gateway", "Pin and audit the trust_remote_code model revision instead of executing a floating main revision", "Isolate model caches, upload directories, and generation logs; remove source audio according to retention policy"],
- "troubleshooting": ["Use the cu129 nightly source for CUDA 12 and the upstream cu130 source for CUDA 13", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "A direct vLLM json response without segments is the current API boundary; normalize it with FunASR AutoModel or use and separately validate SGLang Omni verbose_json"]
+ "troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; revalidate another CUDA stack against its upstream release asset", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "Use vLLM diarized_json for structured speaker segments; the old 68b4a1d5 nightly returns HTTP 400, while json only returns raw [Sxx]-tagged text"]
}
}
},
diff --git a/web-pages/product-site/tests/browser/product-site.spec.ts b/web-pages/product-site/tests/browser/product-site.spec.ts
index ce9b933c3..28bf45818 100644
--- a/web-pages/product-site/tests/browser/product-site.spec.ts
+++ b/web-pages/product-site/tests/browser/product-site.spec.ts
@@ -186,7 +186,9 @@ for (const viewport of [
await expect(page.getByText('FunASR AutoModel', { exact: false }).first()).toBeVisible();
await expect(page.getByText('OpenMOSS', { exact: false }).first()).toBeVisible();
await expect(page.getByText('sentence_info', { exact: false }).first()).toBeVisible();
+ await expect(page.getByText('diarized_json', { exact: false }).first()).toBeVisible();
await expect(page.locator('a[href="https://github.com/modelscope/FunASR/pull/3558"]')).toBeVisible();
+ await expect(page.locator('a[href="https://github.com/vllm-project/vllm/pull/48543"]')).toBeVisible();
await expect(page.locator('a[href="/en/deploy/moss-transcribe-diarize.html"]')).toBeVisible();
const layout = await page.evaluate(() => ({
diff --git a/web-pages/product-site/tests/test_registry.py b/web-pages/product-site/tests/test_registry.py
index b5a2297ea..9e01efdad 100644
--- a/web-pages/product-site/tests/test_registry.py
+++ b/web-pages/product-site/tests/test_registry.py
@@ -107,14 +107,15 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
'OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)'
]
assert entry['tested'] == {
- 'funasr': 'AutoModel adapter@a3a2de0a; third-party model@e8681d68',
- 'runtime': 'Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB',
- 'verified': '2026-08-29',
+ 'funasr': 'AutoModel diarized_json adapter; third-party model@e8681d68',
+ 'runtime': 'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB',
+ 'verified': '2026-08-30',
}
install = '\n'.join(entry['commands']['install'])
assert 'vllm[audio]' in install
- assert 'wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129' in install
+ assert 'vllm-0.27.1%2Bcu129' in install
+ assert 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b' in install
assert 'OpenMOSS-Team/MOSS-Transcribe-Diarize' in install
assert 'e8681d68e7042738ffca8ac8212bc8fcb1131ab8' in install
@@ -123,10 +124,11 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
assert '--trust-remote-code' in launch
smoke = '\n'.join(entry['commands']['smoke'])
assert '/v1/audio/transcriptions' in smoke
- assert 'response_format=json' in smoke
- assert '[S01]' in smoke
+ assert 'response_format=diarized_json' in smoke
+ assert "payload.get('segments'" in smoke
assert "from funasr import AutoModel" in smoke
assert "backend='vllm'" in smoke
+ assert "vllm_response_format='diarized_json'" in smoke
assert "result['raw_text']" in smoke
assert "result['sentence_info']" in smoke
@@ -141,6 +143,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
'e8681d68e7042738ffca8ac8212bc8fcb1131ab8'
) in evidence_urls
assert 'https://github.com/modelscope/FunASR/pull/3558' in evidence_urls
+ assert 'https://github.com/vllm-project/vllm/pull/48543' in evidence_urls
english = entry['translations']['en']
assert 'OpenMOSS' in english['summary']
@@ -151,14 +154,16 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
assert 'FunASR AutoModel API' not in ' '.join(english['not_fit'])
assert 'sentence_info' in english['operations'][-1]
assert 'raw_text' in english['operations'][-1]
+ assert 'diarized_json' in english['operations'][-1]
+ assert 'diarized_json' in english['troubleshooting'][-1]
assert 'internal segmentation' in english['primary_limitation']
assert 'FunASR model' in english['primary_limitation']
assert any(
benchmark['hardware'] == 'NVIDIA H100 80GB HBM3'
- and '6.000 s' in benchmark['audio']
- and '19.147 s' in benchmark['audio']
- and 'S01 -> S02 -> S01' in benchmark['result']
- and 'functional API contract' in benchmark['qualification']
+ and '15.1685 s' in benchmark['audio']
+ and '43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8' in benchmark['audio']
+ and 'S01 and S02' in benchmark['result']
+ and 'contract smoke' in benchmark['qualification']
for benchmark in entry['benchmarks']
)
@@ -167,7 +172,7 @@ def test_audio_cpp_contract_tracks_mainline_nano_and_sensevoice(valid_registry):
entry = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')
llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')
- assert valid_registry['verified'] == '2026-08-29'
+ assert valid_registry['verified'] == '2026-08-30'
assert entry['maturity'] == 'community-verified'
assert entry['selector_rank'] > llama_cpp['selector_rank']
assert entry['tested'] == {
@@ -333,7 +338,7 @@ def test_sensevoice_native_server_contract_tracks_merged_runtime(valid_registry)
llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')
audio_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')
- assert valid_registry['verified'] == '2026-08-29'
+ assert valid_registry['verified'] == '2026-08-30'
assert entry['maturity'] == 'production-verified'
assert llama_cpp['selector_rank'] < entry['selector_rank'] < audio_cpp['selector_rank']
assert entry['tested'] == {