Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
Document custom speaker model support (#3079)
zhifu gaobfa3aa4c2bff
diff --git a/docs/reference/FQA.md b/docs/reference/FQA.mdindex e551f522d..4dcb397ca 100644--- a/docs/reference/FQA.md+++ b/docs/reference/FQA.md@@ -180,6 +180,27 @@ result = model.generate(input="meeting.wav")Then inspect `result[0]["sentence_info"]`. Each sentence should include fields such as `text`, `start`, `end`, and `spk` when diarization is available.+## Can I use a speaker model other than cam++?++Yes. `spk_model` can be any FunASR `AutoModel` speaker embedding model, including a ModelScope model ID or a local model path. For example, ERes2NetV2 can be used directly:++```python+from funasr import AutoModel++model = AutoModel(+ model="paraformer-zh",+ vad_model="fsmn-vad",+ punc_model="ct-punc",+ spk_model="iic/speech_eres2netv2_sv_zh-cn_16k-common",+ device="cuda",+)+result = model.generate(input="meeting.wav")+```++For a fully custom speaker model, make sure the model can be loaded by FunASR `AutoModel` and that its `inference()` method returns `spk_embedding`; the user-facing call still goes through `AutoModel.generate()`. The diarization post-processing and clustering steps use those embeddings to assign speaker labels. If your diarization model emits speaker segments directly instead of embeddings, add an adapter layer that converts its output to the current speaker-label structure.++The tutorial has a longer ERes2NetV2 example: [Speaker Verification / Diarization](../tutorial/README.md#speaker-verification--diarization-eres2netv2).+## The same command works on CPU but fails on CUDAThis usually points to a CUDA, driver, PyTorch, or GPU memory mismatch. Include these checks in your issue:diff --git a/docs/tutorial/README.md b/docs/tutorial/README.mdindex f75fd4760..269ce7fdb 100644--- a/docs/tutorial/README.md+++ b/docs/tutorial/README.md@@ -241,6 +241,7 @@ for sentence in res:print(f"[Speaker {sentence['spk']}] {sentence['text']}")```Notes: `spk_model` can also be `"cam++"` (CAM++ model). ERes2NetV2 provides improved short-duration speaker feature extraction.+Custom `spk_model` values can be ModelScope model ids or local model paths. The model must load through FunASR `AutoModel` and return `spk_embedding` from its `inference()` method; users still call `AutoModel.generate()`. Downstream speaker clustering uses those embeddings to assign `spk` labels.#### Multi-language ASR (Qwen3-ASR)```pythondiff --git a/docs/tutorial/README_zh.md b/docs/tutorial/README_zh.mdindex 7ad75345c..fdbf0e761 100644--- a/docs/tutorial/README_zh.md+++ b/docs/tutorial/README_zh.md@@ -220,6 +220,7 @@ for sentence in res:print(f"[说话人 {sentence['spk']}] {sentence['text']}")```注:`spk_model` 也可以使用 `"cam++"` (CAM++ 模型)。ERes2NetV2 在短时说话人特征提取方面有所改进。+自定义 `spk_model` 可以传 ModelScope 模型 ID 或本地模型路径;模型需要能通过 FunASR `AutoModel` 加载,并在自身 `inference()` 方法中返回 `spk_embedding`,用户侧仍调用 `AutoModel.generate()`。后续说话人聚类会使用这些 embedding 分配 `spk` 标签。#### 多语言语音识别 (Qwen3-ASR)```python