Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
feat(moss): support vLLM diarized responses
LauraGPT14e154ad2abd
diff --git a/docs/moss_transcribe_diarize.md b/docs/moss_transcribe_diarize.mdindex 6467de591..02ed5d460 100644--- a/docs/moss_transcribe_diarize.md+++ b/docs/moss_transcribe_diarize.md@@ -21,7 +21,9 @@ This guide was verified against:- source: `OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`- model: `OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`- license: Apache-2.0 in the pinned upstream source and model metadata-- vLLM nightly index: `68b4a1d582818e67adc903bf1b8fc5a5447da2fa`+- vLLM release: `v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`+- vLLM CUDA 12.9 x86_64 wheel SHA-256:+ `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`Pin all three revisions. The model uses `trust_remote_code`; do not execute afloating model revision in a production service.@@ -76,36 +78,48 @@ model = AutoModel(backend="vllm",vllm_base_url="http://127.0.0.1:8898/v1",vllm_model="moss-transcribe-diarize",+ vllm_response_format="diarized_json",disable_update=True,)result = model.generate("audio.wav")[0]```The vLLM adapter sends the documented OpenAI-compatible multipart request and-normalizes the returned raw text. Authentication can be supplied with-`vllm_api_key`; keep it in a secret store rather than source code.+normalizes the official speaker-attributed `segments` directly into+`sentence_info`. Authentication can be supplied with `vllm_api_key`; keep it+in a secret store rather than source code.++`vllm_response_format="diarized_json"` is the structured production path on+the pinned vLLM revision. The compatibility default remains `json` so existing+clients keep the exact tagged generation in `raw_text`; in structured mode,+`raw_text` is the cleaned `text` returned by vLLM and the authoritative speaker+metadata is in `sentence_info`.## Choose a serving path| Path | Environment | Response contract | Use when ||---|---|---|---|-| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=json` returns raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |+| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=diarized_json` returns OpenAI-compatible speaker segments; `response_format=json` keeps raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler || SGLang Omni | CUDA 13 in the current upstream guide | `response_format=verbose_json` returns parsed segments | You need structured speaker segments directly from the API || Transformers | PyTorch process | Python objects and raw tagged text | Evaluation, debugging, or custom preprocessing |The two HTTP backends share `/v1/audio/transcriptions`, but their documented-response formats are not interchangeable. Do not promise vLLM `segments`-without testing the exact vLLM revision.+response formats are not interchangeable. The pinned vLLM revision supports+`diarized_json`; test the exact server revision before promising `segments`.## vLLM-Create an isolated environment and install the upstream-pinned nightly build:+Create an isolated environment and install the verified vLLM release:```bashuv venv --python 3.12 .venv-moss-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \- --torch-backend=auto \- --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129+curl -fL \+ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \+ -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \+ | sha256sum -c -+uv pip install --python .venv-moss/bin/python --torch-backend=auto \+ "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"```The `audio` extra is required. A plain `vllm` install can start the server but@@ -124,13 +138,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \--port 8898```-Submit audio and validate the speaker-tagged text:+Submit audio and validate the official speaker-attributed response:```bashcurl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \-F [email protected] \-F model=moss-transcribe-diarize \- -F response_format=json \+ -F response_format=diarized_json \-F temperature=0 \| tee moss-transcription.json@@ -140,19 +154,36 @@ import jsonwith open("moss-transcription.json", encoding="utf-8") as stream:payload = json.load(stream)text = payload.get("text", "")+segments = payload.get("segments", [])assert text.strip(), payload-assert "[S01]" in text, text-print(text)+assert segments, payload+assert all(+ isinstance(item.get("speaker"), str)+ and item.get("text")+ and item.get("start") <= item.get("end")+ for item in segments+), payload+print(text, sorted({item["speaker"] for item in segments}))PY```+For an audit of the model's compact tagged generation, repeat the request with+`response_format=json` and verify `[S01]` appears in the returned `text`.+For long recordings, test `max_completion_tokens` against the longest expectedmeeting. A larger limit can increase memory use and tail latency.### Reproduced vLLM contract-The FunASR ecosystem validation used vLLM-`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB:+The structured-response validation used vLLM 0.27.1, Torch+`2.13.0+cu129`, and one H100 80GB. The 15.1685-second two-speaker probe+(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)+returned two `diarized_json` segments labelled `S01` and `S02`; the same+request through the FunASR adapter returned two monotonic `sentence_info`+entries with the same speakers.++Earlier raw-response validation on vLLM+`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB found:- the bundled 6.000-second sample(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)@@ -165,6 +196,12 @@ This proves the pinned API and speaker-return contract on those inputs. It isnot a diarization accuracy result, overlap test, throughput benchmark, orproduction capacity claim.+The older nightly commit+`68b4a1d582818e67adc903bf1b8fc5a5447da2fa` predates vLLM+[`#48543`](https://github.com/vllm-project/vllm/pull/48543). It accepts+`response_format=json` but returns HTTP 400 for `diarized_json`; upgrade to the+verified release instead of silently retrying an expensive transcription.+The FunASR adapter was also tested on both `backend="hf"` and`backend="vllm"` with model revision`e8681d68e7042738ffca8ac8212bc8fcb1131ab8`. The 15.1685-second two-speakerdiff --git a/docs/moss_transcribe_diarize_zh.md b/docs/moss_transcribe_diarize_zh.mdindex c0409a759..324d8d68e 100644--- a/docs/moss_transcribe_diarize_zh.md+++ b/docs/moss_transcribe_diarize_zh.md@@ -19,7 +19,9 @@ MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话- 源码:`OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`- 模型:`OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`- 许可证:固定上游源码和模型元数据中的 Apache-2.0-- vLLM nightly 索引:`68b4a1d582818e67adc903bf1b8fc5a5447da2fa`+- vLLM 发布版:`v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`+- vLLM CUDA 12.9 x86_64 wheel SHA-256:+ `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`三者都要固定。模型依赖 `trust_remote_code`,生产服务不要执行浮动的模型 revision。@@ -68,34 +70,46 @@ model = AutoModel(backend="vllm",vllm_base_url="http://127.0.0.1:8898/v1",vllm_model="moss-transcribe-diarize",+ vllm_response_format="diarized_json",disable_update=True,)result = model.generate("audio.wav")[0]```-vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求,并统一解析返回文本。-认证可通过 `vllm_api_key` 提供;请放在 secret store,不要写入源码。+vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求,并把官方返回的+speaker `segments` 直接映射到 `sentence_info`。认证可通过 `vllm_api_key` 提供;+请放在 secret store,不要写入源码。++固定 vLLM revision 的生产结构化路径是+`vllm_response_format="diarized_json"`。为保持已有客户端兼容,默认值仍是 `json`,+此时 `raw_text` 保留原始标签生成;结构化模式下,`raw_text` 是 vLLM 返回的清理后+`text`,权威说话人信息位于 `sentence_info`。## 选择服务后端| 路径 | 环境 | 响应契约 | 适用场景 ||---|---|---|---|-| vLLM | CUDA 12(`cu129`)或 CUDA 13(`cu130`) | `response_format=json` 返回原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM,或需要其调度能力 |+| vLLM | CUDA 12(`cu129`)或 CUDA 13(`cu130`) | `response_format=diarized_json` 返回 OpenAI-compatible 说话人 segments;`response_format=json` 保留原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM,或需要其调度能力 || SGLang Omni | 当前上游指南为 CUDA 13 | `response_format=verbose_json` 返回解析后的 segments | API 需要直接返回结构化说话人分段 || Transformers | PyTorch 进程 | Python 对象和原始标签文本 | 评测、排障或自定义预处理 |两个 HTTP 后端都使用 `/v1/audio/transcriptions`,但官方文档中的响应格式不能混为-一谈。没有对 exact vLLM revision 做验证时,不要承诺 vLLM 返回 `segments`。+一谈。固定的 vLLM revision 支持 `diarized_json`;承诺 `segments` 前仍要验证实际+部署的 exact server revision。## vLLM-创建隔离环境并安装上游固定的 nightly build:+创建隔离环境并安装已验证的 vLLM 发布版:```bashuv venv --python 3.12 .venv-moss-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \- --torch-backend=auto \- --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129+curl -fL \+ https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \+ -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \+ | sha256sum -c -+uv pip install --python .venv-moss/bin/python --torch-backend=auto \+ "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"```必须安装 `audio` extra。只安装 plain `vllm` 时服务可以启动,但因为没有音频解码器,@@ -113,13 +127,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \--port 8898```-提交音频并验证说话人标签文本:+提交音频并验证官方结构化说话人响应:```bashcurl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \-F [email protected] \-F model=moss-transcribe-diarize \- -F response_format=json \+ -F response_format=diarized_json \-F temperature=0 \| tee moss-transcription.json@@ -129,18 +143,34 @@ import jsonwith open("moss-transcription.json", encoding="utf-8") as stream:payload = json.load(stream)text = payload.get("text", "")+segments = payload.get("segments", [])assert text.strip(), payload-assert "[S01]" in text, text-print(text)+assert segments, payload+assert all(+ isinstance(item.get("speaker"), str)+ and item.get("text")+ and item.get("start") <= item.get("end")+ for item in segments+), payload+print(text, sorted({item["speaker"] for item in segments}))PY```+如需审计模型原始紧凑标签生成,请把请求改为 `response_format=json`,并验证返回+`text` 中包含 `[S01]`。+长音频需要按业务最长会议验证 `max_completion_tokens`。增大上限也会影响显存和尾延迟。### 已复现的 vLLM 契约-FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch-`2.11.0+cu129` 和一张 H100 80GB:+结构化响应验证使用 vLLM 0.27.1、Torch `2.13.0+cu129` 和一张 H100 80GB。+15.1685 秒双说话人样例+(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)+返回两个 `diarized_json` segments,标签为 `S01`、`S02`;同一请求通过 FunASR+适配器后,返回两个时间单调且说话人一致的 `sentence_info`。++较早的 raw-response 验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch+`2.11.0+cu129` 和一张 H100 80GB,结果如下:- 仓库内 6.000 秒样例(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)@@ -152,6 +182,11 @@ FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch这证明固定版本对这些输入满足 API 和说话人返回契约,不是 diarization 精度、重叠语音、吞吐或生产容量结论。+旧 nightly commit `68b4a1d582818e67adc903bf1b8fc5a5447da2fa` 早于 vLLM+[`#48543`](https://github.com/vllm-project/vllm/pull/48543):它支持+`response_format=json`,但 `diarized_json` 会返回 HTTP 400。应升级到已验证发布版,+不要静默重试一次成本较高的转写请求。+FunASR 适配器还分别使用 `backend="hf"` 和 `backend="vllm"`,按模型 revision`e8681d68e7042738ffca8ac8212bc8fcb1131ab8` 做了真实测试。15.1685 秒的双说话人样例(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)通过diff --git a/funasr/models/moss_transcribe_diarize/model.py b/funasr/models/moss_transcribe_diarize/model.pyindex fcc302be5..0f00dfb1a 100644--- a/funasr/models/moss_transcribe_diarize/model.py+++ b/funasr/models/moss_transcribe_diarize/model.py@@ -7,6 +7,7 @@ https://github.com/OpenMOSS/MOSS-Transcribe-Diarizeimport copyimport io+import mathimport mimetypesimport osimport re@@ -127,6 +128,66 @@ def _result_from_transcript(key, transcript):}+def _result_from_diarized_response(key, response):+ """Normalize vLLM's OpenAI-compatible diarized JSON response."""+ text = response.get("text") if isinstance(response, dict) else None+ segments = response.get("segments") if isinstance(response, dict) else None+ if not isinstance(text, str) or not isinstance(segments, list):+ raise RuntimeError(+ "vLLM diarized_json response requires text and segments fields"+ )++ sentence_info = []+ timestamps = []+ for index, segment in enumerate(segments):+ if not isinstance(segment, dict):+ raise RuntimeError(+ "vLLM diarized_json segment %d must be an object" % index+ )+ start = segment.get("start")+ end = segment.get("end")+ segment_text = segment.get("text")+ speaker = segment.get("speaker")+ numeric = (+ isinstance(start, (int, float))+ and not isinstance(start, bool)+ and isinstance(end, (int, float))+ and not isinstance(end, bool)+ )+ if (+ not numeric+ or not math.isfinite(start)+ or not math.isfinite(end)+ or start < 0+ or end < start+ or not isinstance(segment_text, str)+ or not isinstance(speaker, str)+ or not speaker+ ):+ raise RuntimeError(+ "vLLM diarized_json segment %d has an invalid contract" % index+ )+ timestamp = [int(round(start * 1000)), int(round(end * 1000))]+ timestamps.append(timestamp)+ sentence_info.append(+ {+ "start": timestamp[0],+ "end": timestamp[1],+ "text": segment_text,+ "sentence": segment_text,+ "spk": speaker,+ "timestamp": [timestamp],+ }+ )+ return {+ "key": key,+ "text": text,+ "raw_text": text,+ "timestamp": timestamps,+ "sentence_info": sentence_info,+ }++@tables.register("model_classes", "MOSS-Transcribe-Diarize")@tables.register("model_classes", DEFAULT_MODEL)class MossTranscribeDiarize(nn.Module):@@ -157,6 +218,9 @@ class MossTranscribeDiarize(nn.Module):self.vllm_model = kwargs.get("vllm_model", DEFAULT_MODEL)self.vllm_api_key = kwargs.get("vllm_api_key", "EMPTY")self.vllm_timeout = float(kwargs.get("vllm_timeout", 600.0))+ self.vllm_response_format = kwargs.get("vllm_response_format", "json").lower()+ if self.vllm_response_format not in {"json", "diarized_json"}:+ raise ValueError("vllm_response_format must be 'json' or 'diarized_json'")self.http_session = kwargs.get("http_session")self.hf_model = None@@ -240,10 +304,18 @@ class MossTranscribeDiarize(nn.Module):results = []for index, audio in enumerate(inputs):if self.backend == "vllm":- transcript = self._transcribe_vllm(audio, **kwargs)+ response = self._transcribe_vllm(audio, **kwargs)+ if self.vllm_response_format == "diarized_json":+ results.append(+ _result_from_diarized_response(keys[index], response)+ )+ else:+ results.append(+ _result_from_transcript(keys[index], response["text"])+ )else:transcript = self._transcribe_hf(audio, **kwargs)- results.append(_result_from_transcript(keys[index], transcript))+ results.append(_result_from_transcript(keys[index], transcript))return results, {"batch_data_time": max(audio_duration, 1e-9),"forward": time.perf_counter() - started,@@ -324,7 +396,7 @@ class MossTranscribeDiarize(nn.Module):filename, payload, content_type = self._multipart_audio(audio)data = {"model": self.vllm_model,- "response_format": "json",+ "response_format": self.vllm_response_format,"temperature": str(kwargs.get("temperature", 0)),}prompt = kwargs.get("prompt")@@ -345,11 +417,15 @@ class MossTranscribeDiarize(nn.Module):)response.raise_for_status()result = response.json()- if not isinstance(result, dict) or not isinstance(result.get("text"), str):+ if not isinstance(result, dict):+ raise RuntimeError("vLLM transcription response must be a JSON object")+ if self.vllm_response_format == "json" and not isinstance(+ result.get("text"), str+ ):raise RuntimeError("vLLM transcription response did not contain a text field")- return result["text"]+ return result@staticmethoddef _multipart_audio(audio):diff --git a/tests/test_moss_transcribe_diarize_docs.py b/tests/test_moss_transcribe_diarize_docs.pyindex 8b2a956de..219d663c2 100644--- a/tests/test_moss_transcribe_diarize_docs.py+++ b/tests/test_moss_transcribe_diarize_docs.py@@ -25,15 +25,19 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->"cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3","OpenMOSS-Team/MOSS-Transcribe-Diarize","e8681d68e7042738ffca8ac8212bc8fcb1131ab8",- "68b4a1d582818e67adc903bf1b8fc5a5447da2fa",+ "6e448d0ea9bf3d88d898b65449ca6dc2aec170ac",+ "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b","vllm[audio]",- "0.23.1rc1.dev949+g68b4a1d58",+ "vLLM 0.27.1",+ "68b4a1d582818e67adc903bf1b8fc5a5447da2fa","dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912","43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8","S01 -> S02 -> S01","/v1/audio/transcriptions","response_format=json",+ "response_format=diarized_json","response_format=verbose_json",+ 'vllm_response_format="diarized_json"',"[S01]","Apache-2.0","sentence_info",diff --git a/tests/test_moss_transcribe_diarize_model.py b/tests/test_moss_transcribe_diarize_model.pyindex 938a6fa93..55176dae3 100644--- a/tests/test_moss_transcribe_diarize_model.py+++ b/tests/test_moss_transcribe_diarize_model.py@@ -138,6 +138,89 @@ class MossVllmBackendTest(unittest.TestCase):self.assertEqual(content_type, "audio/wav")response.raise_for_status.assert_called_once_with()+ def test_normalizes_official_vllm_diarized_json_response(self):+ session = MagicMock()+ response = MagicMock()+ response.json.return_value = {+ "task": "transcribe",+ "duration": 3.5,+ "text": "hello again",+ "segments": [+ {+ "type": "transcript.text.segment",+ "id": "seg_0",+ "start": 0.25,+ "end": 1.5,+ "text": "hello",+ "speaker": "S01",+ },+ {+ "type": "transcript.text.segment",+ "id": "seg_1",+ "start": 2.0,+ "end": 3.5,+ "text": "again",+ "speaker": "S02",+ },+ ],+ }+ session.post.return_value = response++ model = MossTranscribeDiarize(+ backend="vllm",+ vllm_base_url="http://vllm.test:8000/v1",+ vllm_response_format="diarized_json",+ http_session=session,+ )+ results, _ = model.inference(+ [np.zeros(1600, dtype=np.float32)], key=["meeting"]+ )++ self.assertEqual(+ session.post.call_args.kwargs["data"]["response_format"],+ "diarized_json",+ )+ self.assertEqual(results[0]["text"], "hello again")+ self.assertEqual(results[0]["raw_text"], "hello again")+ self.assertEqual(results[0]["timestamp"], [[250, 1500], [2000, 3500]])+ self.assertEqual(+ [(item["spk"], item["text"]) for item in results[0]["sentence_info"]],+ [("S01", "hello"), ("S02", "again")],+ )++ def test_rejects_unsupported_vllm_response_format(self):+ with self.assertRaisesRegex(ValueError, "json.*diarized_json"):+ MossTranscribeDiarize(+ backend="vllm",+ vllm_base_url="http://vllm.test:8000/v1",+ vllm_response_format="verbose_json",+ )++ def test_rejects_malformed_vllm_diarized_segment(self):+ session = MagicMock()+ response = MagicMock()+ response.json.return_value = {+ "text": "broken",+ "segments": [+ {+ "start": 2.0,+ "end": 1.0,+ "text": "broken",+ "speaker": "S01",+ }+ ],+ }+ session.post.return_value = response+ model = MossTranscribeDiarize(+ backend="vllm",+ vllm_base_url="http://vllm.test:8000/v1",+ vllm_response_format="diarized_json",+ http_session=session,+ )++ with self.assertRaisesRegex(RuntimeError, "segment 0.*invalid contract"):+ model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])+if __name__ == "__main__":unittest.main()diff --git a/web-pages/product-site/data/deployments.json b/web-pages/product-site/data/deployments.jsonindex 80bdd6afc..a83d79487 100644--- a/web-pages/product-site/data/deployments.json+++ b/web-pages/product-site/data/deployments.json@@ -1,6 +1,6 @@{"schema_version": 1,- "verified": "2026-08-29",+ "verified": "2026-08-30","deployments": [{"id": "moss-transcribe-diarize",@@ -13,10 +13,10 @@"models": ["OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)"],"operating_systems": ["Linux"],"interfaces": ["OpenAI-compatible HTTP", "vLLM", "SGLang Omni", "Transformers"],- "tested": {"funasr": "AutoModel adapter@a3a2de0a; third-party model@e8681d68", "runtime": "Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB", "verified": "2026-08-29"},+ "tested": {"funasr": "AutoModel diarized_json adapter; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},"commands": {"install": [- "uv venv --python 3.12 .venv-moss && uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' --torch-backend=auto --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129",+ "uv venv --python 3.12 .venv-moss && curl -fL https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl && echo 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl' | sha256sum -c - && uv pip install --python .venv-moss/bin/python --torch-backend=auto \"vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl\"","HF_HUB_ENABLE_HF_TRANSFER=1 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize --revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 --local-dir .models/moss-transcribe-diarize"],"launch": [@@ -27,9 +27,9 @@"curl -fsS http://127.0.0.1:8898/v1/models"],"smoke": [- "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=json -F temperature=0 | tee /tmp/moss-transcription.json",- "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\ntext = payload.get('text', '')\nassert text.strip(), payload\nassert '[S01]' in text, text\nprint(text)\nPY",- "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"+ "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=diarized_json -F temperature=0 | tee /tmp/moss-transcription.json",+ "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\ntext = payload.get('text', '')\nsegments = payload.get('segments', [])\nassert text.strip() and segments, payload\nassert all(isinstance(item.get('speaker'), str) and item.get('text') and item.get('start') <= item.get('end') for item in segments), payload\nprint(text, sorted({item['speaker'] for item in segments}))\nPY",+ "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', vllm_response_format='diarized_json', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"]},"evidence": [@@ -37,22 +37,24 @@{"label": "upstream source pinned at cb765f2", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/commit/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3"},{"label": "official model snapshot pinned at e8681d68", "url": "https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize/tree/e8681d68e7042738ffca8ac8212bc8fcb1131ab8"},{"label": "official vLLM and SGLang Omni serving guide", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3/README.md"},+ {"label": "vLLM official diarized_json response contract", "url": "https://github.com/vllm-project/vllm/pull/48543"},+ {"label": "verified vLLM 0.27.1 release", "url": "https://github.com/vllm-project/vllm/releases/tag/v0.27.1"},{"label": "FunASR ecosystem integration guide", "url": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md"},{"label": "FunASR AutoModel adapter merge and exact-head CI", "url": "https://github.com/modelscope/FunASR/pull/3558"}],"benchmarks": [{"model": "OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68",- "runtime": "vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129",+ "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129","hardware": "NVIDIA H100 80GB HBM3",- "workload": "OpenAI-compatible transcription API smoke test plus synthetic A-to-B-to-A speaker-return probe",- "audio": "Bundled 6.000 s Mandarin sample; synthetic 19.147 s file built as sample A + 0.8 s silence + sample B + 0.8 s silence + sample A",- "settings": "Pinned model and vLLM revisions; vllm[audio]; response_format=json; temperature=0; one request at a time",- "timing_scope": "Client wall time after /health became ready; local HTTP and decoding included; model download and engine startup excluded",- "result": "Both requests returned HTTP 200. The 6.000 s sample produced S01 with timestamps 0.96-5.94. The 19.147 s probe produced S01 -> S02 -> S01 with timestamps covering 0.96-19.08. Observed client times were 1.034 s and 0.171 s respectively.",- "qualification": "A functional API contract and speaker-return probe on synthetic concatenation, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.",+ "workload": "OpenAI-compatible diarized_json API smoke plus FunASR AutoModel structured-response mapping",+ "audio": "15.1685 s two-speaker probe; SHA256 43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8",+ "settings": "Pinned model revision; vLLM 0.27.1; vllm[audio]; response_format=diarized_json; temperature=0; max_completion_tokens=768",+ "timing_scope": "Functional response validation after /health became ready; no latency or throughput claim",+ "result": "Direct vLLM HTTP and the FunASR adapter each returned two non-empty monotonic segments with speakers S01 and S02.",+ "qualification": "A structured response contract smoke, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.","source": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md",- "verified": "2026-08-29"+ "verified": "2026-08-30"}],"translations": {@@ -62,11 +64,11 @@"fit": ["会议、访谈、播客和通话的多人长音频", "希望通过同一个模型输出文本、时间戳与说话人编号", "已有 NVIDIA GPU,并需要 OpenAI 兼容的音频转写接口"],"not_fit": ["实时流式字幕或低延迟端点检测", "CPU、桌面边缘设备或 Windows 原生部署", "要求 FunASR 自有模型权重的项目"],"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers 与 vLLM 提供统一的结构化结果。",- "primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 nightly commit,升级前必须重跑真实多人长音频。",+ "primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 0.27.1 发布版,升级前必须重跑真实多人长音频。","status_label": "社区验证",- "operations": ["固定模型 revision、FunASR adapter merge、vLLM nightly commit、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 保留 raw_text,并统一返回 text、timestamp 和带 spk 的 sentence_info;直接调用 vLLM 时 json 仍是原始 [Sxx] 标记文本"],+ "operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 可用 vLLM diarized_json 直接统一 text、timestamp 和带 spk 的 sentence_info;json 兼容路径仍在 raw_text 保留原始 [Sxx] 标记文本"],"security": ["把服务绑定内网,由网关完成认证、TLS、限流及音频大小/时长限制", "固定并审计 trust_remote_code 对应的模型 revision,不执行浮动 main 代码", "隔离模型缓存、上传临时目录和生成日志,按数据保留策略清理原始音频"],- "troubleshooting": ["启动前确认 CUDA 12 使用 cu129 nightly 源,CUDA 13 使用上游文档列出的 cu130 源", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "直接调用 vLLM 时 json 没有 segments 字段是当前接口边界;使用 FunASR AutoModel 归一化结果,或改用并单独验证 SGLang Omni verbose_json"]+ "troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;其他 CUDA 栈需按上游发布资产重新验证", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "需要结构化说话人 segments 时使用 vLLM diarized_json;旧 68b4a1d5 nightly 会返回 HTTP 400,json 只返回原始 [Sxx] 标签文本"]},"en": {"name": "MOSS unified transcription and diarization",@@ -74,11 +76,11 @@"fit": ["Multi-speaker meetings, interviews, podcasts, and calls", "One model output for text, timestamps, and speaker identifiers", "NVIDIA GPU deployments that need an OpenAI-compatible audio transcription endpoint"],"not_fit": ["Realtime streaming captions or low-latency endpoint detection", "CPU, desktop-edge, or native Windows deployments", "Projects that require FunASR-owned model weights"],"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers and vLLM.",- "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins a nightly commit and must be revalidated on real long multi-speaker audio before upgrades.",+ "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins release 0.27.1 and must be revalidated on real long multi-speaker audio before upgrades.","status_label": "Community verified",- "operations": ["Pin the model revision, FunASR adapter merge, vLLM nightly commit, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel preserves raw_text and normalizes text, timestamp, and sentence_info with spk; direct vLLM json still returns raw [Sxx]-tagged text"],+ "operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel can map vLLM diarized_json directly into text, timestamp, and sentence_info with spk; the json compatibility path still preserves raw [Sxx]-tagged text in raw_text"],"security": ["Bind workers to a private network and put authentication, TLS, rate limits, and audio size/duration limits at the gateway", "Pin and audit the trust_remote_code model revision instead of executing a floating main revision", "Isolate model caches, upload directories, and generation logs; remove source audio according to retention policy"],- "troubleshooting": ["Use the cu129 nightly source for CUDA 12 and the upstream cu130 source for CUDA 13", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "A direct vLLM json response without segments is the current API boundary; normalize it with FunASR AutoModel or use and separately validate SGLang Omni verbose_json"]+ "troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; revalidate another CUDA stack against its upstream release asset", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "Use vLLM diarized_json for structured speaker segments; the old 68b4a1d5 nightly returns HTTP 400, while json only returns raw [Sxx]-tagged text"]}}},diff --git a/web-pages/product-site/tests/browser/product-site.spec.ts b/web-pages/product-site/tests/browser/product-site.spec.tsindex ce9b933c3..28bf45818 100644--- a/web-pages/product-site/tests/browser/product-site.spec.ts+++ b/web-pages/product-site/tests/browser/product-site.spec.ts@@ -186,7 +186,9 @@ for (const viewport of [await expect(page.getByText('FunASR AutoModel', { exact: false }).first()).toBeVisible();await expect(page.getByText('OpenMOSS', { exact: false }).first()).toBeVisible();await expect(page.getByText('sentence_info', { exact: false }).first()).toBeVisible();+ await expect(page.getByText('diarized_json', { exact: false }).first()).toBeVisible();await expect(page.locator('a[href="https://github.com/modelscope/FunASR/pull/3558"]')).toBeVisible();+ await expect(page.locator('a[href="https://github.com/vllm-project/vllm/pull/48543"]')).toBeVisible();await expect(page.locator('a[href="/en/deploy/moss-transcribe-diarize.html"]')).toBeVisible();const layout = await page.evaluate(() => ({diff --git a/web-pages/product-site/tests/test_registry.py b/web-pages/product-site/tests/test_registry.pyindex b5a2297ea..9e01efdad 100644--- a/web-pages/product-site/tests/test_registry.py+++ b/web-pages/product-site/tests/test_registry.py@@ -107,14 +107,15 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi'OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)']assert entry['tested'] == {- 'funasr': 'AutoModel adapter@a3a2de0a; third-party model@e8681d68',- 'runtime': 'Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB',- 'verified': '2026-08-29',+ 'funasr': 'AutoModel diarized_json adapter; third-party model@e8681d68',+ 'runtime': 'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB',+ 'verified': '2026-08-30',}install = '\n'.join(entry['commands']['install'])assert 'vllm[audio]' in install- assert 'wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129' in install+ assert 'vllm-0.27.1%2Bcu129' in install+ assert 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b' in installassert 'OpenMOSS-Team/MOSS-Transcribe-Diarize' in installassert 'e8681d68e7042738ffca8ac8212bc8fcb1131ab8' in install@@ -123,10 +124,11 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regiassert '--trust-remote-code' in launchsmoke = '\n'.join(entry['commands']['smoke'])assert '/v1/audio/transcriptions' in smoke- assert 'response_format=json' in smoke- assert '[S01]' in smoke+ assert 'response_format=diarized_json' in smoke+ assert "payload.get('segments'" in smokeassert "from funasr import AutoModel" in smokeassert "backend='vllm'" in smoke+ assert "vllm_response_format='diarized_json'" in smokeassert "result['raw_text']" in smokeassert "result['sentence_info']" in smoke@@ -141,6 +143,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi'e8681d68e7042738ffca8ac8212bc8fcb1131ab8') in evidence_urlsassert 'https://github.com/modelscope/FunASR/pull/3558' in evidence_urls+ assert 'https://github.com/vllm-project/vllm/pull/48543' in evidence_urlsenglish = entry['translations']['en']assert 'OpenMOSS' in english['summary']@@ -151,14 +154,16 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regiassert 'FunASR AutoModel API' not in ' '.join(english['not_fit'])assert 'sentence_info' in english['operations'][-1]assert 'raw_text' in english['operations'][-1]+ assert 'diarized_json' in english['operations'][-1]+ assert 'diarized_json' in english['troubleshooting'][-1]assert 'internal segmentation' in english['primary_limitation']assert 'FunASR model' in english['primary_limitation']assert any(benchmark['hardware'] == 'NVIDIA H100 80GB HBM3'- and '6.000 s' in benchmark['audio']- and '19.147 s' in benchmark['audio']- and 'S01 -> S02 -> S01' in benchmark['result']- and 'functional API contract' in benchmark['qualification']+ and '15.1685 s' in benchmark['audio']+ and '43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8' in benchmark['audio']+ and 'S01 and S02' in benchmark['result']+ and 'contract smoke' in benchmark['qualification']for benchmark in entry['benchmarks'])@@ -167,7 +172,7 @@ def test_audio_cpp_contract_tracks_mainline_nano_and_sensevoice(valid_registry):entry = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')- assert valid_registry['verified'] == '2026-08-29'+ assert valid_registry['verified'] == '2026-08-30'assert entry['maturity'] == 'community-verified'assert entry['selector_rank'] > llama_cpp['selector_rank']assert entry['tested'] == {@@ -333,7 +338,7 @@ def test_sensevoice_native_server_contract_tracks_merged_runtime(valid_registry)llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')audio_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')- assert valid_registry['verified'] == '2026-08-29'+ assert valid_registry['verified'] == '2026-08-30'assert entry['maturity'] == 'production-verified'assert llama_cpp['selector_rank'] < entry['selector_rank'] < audio_cpp['selector_rank']assert entry['tested'] == {