Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

docs: make multilingual README entrypoints task-oriented

zhifu gao1aff355786e0
diff --git a/README.md b/README.md
index b75b1fd03..4d744c797 100644
--- a/README.md
+++ b/README.md
@@ -63,17 +63,16 @@ from funasr import AutoModel
model = AutoModel(model="FunAudioLLM/Fun-ASR-Nano-2512", device="cuda")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
print(result[0]["text"])
-# 欢迎大家来体验达摩院推出的语音识别模型。
```
For the separate 31-language checkpoint, use
[Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512).
Language coverage is checkpoint-specific, so Nano and MLT-Nano should be treated as distinct model choices.
-On CPU (or for five-language ASR plus emotion and audio-event tags), use
-**SenseVoiceSmall**. The pipeline below composes SenseVoiceSmall with FSMN-VAD
-and CAM++; diarization is provided by the separate CAM++ model, not by the
-SenseVoiceSmall checkpoint:
+For a CPU-first example with five-language ASR plus emotion and audio-event
+tags, use **SenseVoiceSmall**. The pipeline below combines it with FSMN-VAD and
+CAM++ for speaker-aware VAD segments; these are not native speaker outputs of
+the SenseVoiceSmall checkpoint.
See the [SenseVoice paper](https://arxiv.org/abs/2407.04051),
[Hugging Face checkpoint](https://huggingface.co/FunAudioLLM/SenseVoiceSmall),
and [GGUF edge checkpoint](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF).
@@ -82,7 +81,7 @@ and [GGUF edge checkpoint](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GG
from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess
-model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cuda") # use device="cpu" if you don't have a GPU
+model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cpu")
result = model.generate(
input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav",
batch_size_s=300,
@@ -93,13 +92,15 @@ for seg in result[0]["sentence_info"]:
print(f"[{seg['start']/1000:.1f}s] Speaker {seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")
```
-**Output** — structured text with speaker labels, timestamps, and punctuation:
-```
-[0.6s] Speaker 0: 欢迎大家来体验达摩院推出的语音识别模型
-```
+This prints each returned segment's start time in seconds, anonymous speaker
+index, and text with SenseVoice tags removed. Text and segment boundaries depend
+on the audio and checkpoint; no fixed transcript is asserted here.
-One `AutoModel` pipeline call coordinates the configured ASR, VAD, and speaker
-models and returns the combined result.
+CAM++ extracts `spk_embedding` vectors. `AutoModel` clusters those embeddings
+and assigns speaker indices to VAD segments. Indices are local to a recording,
+not known-person identities. See the [SDK contract](./docs/python_api.md) for
+the component and result boundaries. Change to `device="cuda"` only after
+verifying a compatible GPU environment as described above.
### Scale & deploy the flagship
@@ -120,23 +121,22 @@ results = model.generate(["audio1.wav", "audio2.wav"], language="auto")
### Why FunASR?
-Whisper is a single model; **FunASR is a toolkit** — you pick the right model
-per job: **Fun-ASR-Nano** (Chinese, English, Japanese, and Chinese dialects;
-GPU), **Fun-ASR-MLT-Nano** (31 languages), **SenseVoiceSmall** (five-language
-ASR plus emotion and audio events), and **Paraformer** (low-latency streaming).
-The table shows toolkit-level capabilities and names the model or pipeline that
-provides each one:
+FunASR is a toolkit: choose the task, checkpoint, and runtime separately.
+Support in one model or adapter does not imply support in every serving backend.
-| | FunASR (toolkit) | Whisper | Cloud APIs |
+| Task | Checkpoint or pipeline | Runtime entrypoint | Important limitation |
|---|---|---|---|
-| Top speed | **340x realtime** (Fun-ASR-Nano + vLLM) | 13x realtime | ~1x realtime |
-| Speaker ID | ✅ via VAD + CAM++ pipeline | ❌ Needs pyannote | ✅ Extra cost |
-| Emotion | ✅ via SenseVoice | ❌ | ❌ |
-| Languages | Checkpoint-specific (for example Qwen3-ASR 52, MLT-Nano 31, Nano zh/en/ja) | 57 | Varies |
-| Streaming | ✅ WebSocket (Paraformer) | ❌ | ✅ |
-| CPU viable | ✅ 17x realtime (SenseVoice) | ❌ Too slow | N/A |
-| Self-hosted | ✅ Yes (toolkit: MIT; model licenses vary) | ✅ MIT license | ❌ Cloud only |
-| Cost | Free | Free | $0.006/min+ |
+| File transcription with emotion/event tags | SenseVoiceSmall | Python `AutoModel`, CPU or GPU | Five-language checkpoint; tags do not identify speakers. |
+| LLM-based file transcription | Fun-ASR-Nano | `AutoModel`; split-engine `AutoModelVLLM` for the documented GPU path | Base Nano covers zh/en/ja and Chinese dialects/accents; timestamp support depends on checkpoint and path. |
+| Broader multilingual transcription | Fun-ASR-MLT-Nano | Python `AutoModel` | Separate 31-language checkpoint; do not transfer its coverage to base Nano. |
+| Chunked live transcription | Paraformer-zh-streaming | Streaming SDK or runtime WebSocket service | Use the streaming checkpoint and per-session cache, not an offline checkpoint. |
+| Speaker-aware file transcription | SenseVoiceSmall + FSMN-VAD + CAM++ | `AutoModel` with VAD and embedding clustering | Anonymous indices within a recording, not enrolled-speaker identification. |
+| Joint text, timestamps, and speakers | MOSS-Transcribe-Diarize, third-party OpenMOSS | FunASR adapter or upstream backend in the MOSS guide | Offline, recording-local anonymous labels; no external VAD/speaker pipeline for its unified path. |
+| Native CPU/edge transcription | Fun-ASR-Nano or SenseVoiceSmall GGUF | llama.cpp runtime | Requires matching converted weights; GGUF is not a Python `AutoModel` checkpoint. |
+
+See the [Model Zoo](./model_zoo/readme.md) and [deployment matrix](./docs/deployment_matrix.md)
+for checkpoint, interface, and licensing boundaries. Benchmark on your own audio
+and hardware before choosing a runtime.
Trying FunASR for the first time? Use the [Colab quickstart](./examples/colab/) before setting up a local environment. Choosing a first model? Start with the [model selection guide](./docs/model_selection.md). Planning a switch from Whisper or a cloud ASR provider? Use the [migration guide](./docs/migration_from_whisper.md) and [benchmark example](./examples/migration/) to test representative audio, map features, and roll out safely.
@@ -162,12 +162,17 @@ Requirements: Python ≥ 3.8. Install PyTorch + torchaudio first ([pytorch.org](
## Model Zoo
+This list includes third-party models. **OpenMOSS** publishes MOSS-Transcribe-Diarize;
+FunASR provides an adapter, not ownership of its weights. Its unified path is
+offline, with anonymous labels scoped to each recording, not realtime or
+known-person identification. Model licenses are separate from the toolkit's MIT license.
+
| Model | Task | Languages | Params | Links |
|-------|------|-----------|--------|-------|
| **Fun-ASR-Nano** | ASR | zh/en/ja + Chinese dialects and accents | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) [GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) |
| **Fun-ASR-MLT-Nano** | ASR | 31 languages | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) |
| **SenseVoiceSmall** | ASR + emotion + events | zh/en/ja/ko/yue | 234M | [⭐](https://www.modelscope.cn/models/iic/SenseVoiceSmall) [🤗](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) [GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF) [paper](https://arxiv.org/abs/2407.04051) |
-| **MOSS-Transcribe-Diarize** | Offline ASR + timestamps + anonymous speakers | See official card | See official card | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [guide](./docs/moss_transcribe_diarize.md) |
+| **MOSS-Transcribe-Diarize** | Third-party OpenMOSS: offline ASR + timestamps + anonymous speakers | See official card | See official card | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [guide](./docs/moss_transcribe_diarize.md) |
| **Paraformer-zh** | ASR + timestamps | zh/en | 220M | [⭐](https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) [🤗](https://huggingface.co/funasr/paraformer-zh) |
| Paraformer-zh-streaming | Streaming ASR | zh/en | 220M | [⭐](https://modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online/summary) [🤗](https://huggingface.co/funasr/paraformer-zh-streaming) |
| Qwen3-ASR | ASR, 52 languages | multilingual | 1.7B | [usage](examples/industrial_data_pretraining/qwen3_asr) |
@@ -176,7 +181,7 @@ Requirements: Python ≥ 3.8. Install PyTorch + torchaudio first ([pytorch.org](
| Whisper-large-v3-turbo | ASR + translation | multilingual | 809M | [usage](examples/industrial_data_pretraining/whisper) |
| ct-punc | Punctuation | zh/en | 290M | [⭐](https://modelscope.cn/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large/summary) [🤗](https://huggingface.co/funasr/ct-punc) |
| fsmn-vad | VAD | zh/en | 0.4M | [⭐](https://modelscope.cn/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/summary) [🤗](https://huggingface.co/funasr/fsmn-vad) |
-| cam++ | Speaker diarization | — | 7.2M | [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) |
+| cam++ | Speaker embeddings (pipeline component) | — | 7.2M | [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) |
| emotion2vec+large | Emotion recognition | — | 300M | [⭐](https://modelscope.cn/models/iic/emotion2vec_plus_large/summary) [🤗](https://huggingface.co/emotion2vec/emotion2vec_plus_large) |
---
@@ -324,17 +329,16 @@ inference on physical Blackwell hardware.
## Benchmark
-> 184 long-form audio files (192 min). [Full report →](https://modelscope.github.io/FunASR/benchmark.html) · [RTFx and reproducibility notes →](./docs/benchmark/rtf_reproducibility.md)
-
-| Model | Chinese CER ↓ | GPU Speed | CPU Speed | vs Whisper-large-v3 |
-|-------|------|-----------|-----------|-------------------|
-| **Fun-ASR-Nano** (vLLM) | **8.20%** | **340x** realtime | — | 🚀 **26x faster** |
-| **SenseVoice-Small** | **7.81%** | **170x** realtime | **17x** realtime | 🚀 **13x faster** |
-| **Paraformer-Large** | 10.18% | **120x** realtime | **15x** realtime | 🚀 **9x faster** |
-| Whisper-large-v3-turbo | 21.71% | 46x realtime | ❌ | 3.4x faster |
-| Whisper-large-v3 | 20.02% | 13x realtime | ❌ | baseline |
+The [historical benchmark report](https://modelscope.github.io/FunASR/benchmark.html)
+and [split-engine measurements](./docs/vllm_guide.md#benchmark) retain their
+original results. They are separate records, not universal speed rankings or
+production capacity guarantees.
-> **Key takeaway:** FunASR models run on CPU faster than Whisper runs on GPU.
+Use the [RTFx and reproducibility notes](./docs/benchmark/rtf_reproducibility.md)
+to compare checkpoint/revision, audio set, hardware, batching, warmup, timing
+scope, and CER/WER. Offline throughput is not streaming latency. The
+[migration benchmark example](./examples/migration/) helps measure your own
+recordings with the same evaluation scope.
---
diff --git a/README_ja.md b/README_ja.md
index ca9a3065e..ee5233cce 100644
--- a/README_ja.md
+++ b/README_ja.md
@@ -31,26 +31,34 @@
## クイックスタート
```bash
-pip install funasr
+python -m pip install torch torchaudio
+python -m pip install funasr
```
+以下は公開サンプルを使う CPU 向けの例です。GPU を使う場合は
+[インストールガイド](./docs/installation/installation.md) に従って互換性のある
+PyTorch/CUDA 環境を用意し、`torch.cuda.is_available()` を確認してから
+`device="cuda"` に変更してください。
+
```python
from funasr import AutoModel
+from funasr.utils.postprocess_utils import rich_transcription_postprocess
-model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cuda")
-result = model.generate(input="meeting.wav")
-```
+model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cpu")
+result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
-**出力** — 話者ラベル・タイムスタンプ・句読点付きの構造化テキスト:
-```
-[00:00.4 → 00:03.8] 話者0: Q3の計画について話し合いましょう。
-[00:04.2 → 00:07.1] 話者1: いいですね。3つのポイントがあります。
-[00:07.5 → 00:12.3] 話者0: どうぞ。あと30分あります。
+for seg in result[0]["sentence_info"]:
+ print(f"[{seg['start']/1000:.1f}s] 話者{seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")
```
-これは1回の `AutoModel` パイプライン呼び出しですが、SenseVoiceSmall、
-FSMN-VAD、CAM++という独立したモデルを組み合わせています。話者分離は
-SenseVoiceSmall自体ではなく、CAM++によって提供されます。
+実際に返された VAD 区間の開始時刻(秒)、匿名話者番号、SenseVoice タグを
+除いたテキストを表示します。テキストや区間は音声と checkpoint に依存し、
+ここでは固定の認識結果を示していません。
+
+CAM++ は `spk_embedding` ベクトルを抽出し、`AutoModel` がクラスタリングと
+VAD 区間への話者割り当てを行います。番号は録音内だけで有効で、既知の人物の
+識別や SenseVoiceSmall 単体の出力ではありません。
+詳細は [SDK 契約](./docs/python_api.md) を参照してください。
初めて使う場合は [Colab クイックスタート](./examples/colab/README_ja.md) から試せます。どのモデルを選ぶか迷う場合は [モデル選択ガイド](./docs/model_selection_ja.md) を参照してください。
@@ -60,22 +68,21 @@ SenseVoiceSmall自体ではなく、CAM++によって提供されます。
### なぜFunASRを選ぶのか?
-Whisper は単一モデルですが、**FunASR はツールキット**です——用途に応じて
-**Fun-ASR-Nano**(中・英・日と中国語方言・地域アクセント、GPU)、
-**Fun-ASR-MLT-Nano**(31言語)、**SenseVoiceSmall**(5言語ASRと感情・
-音声イベント)、**Paraformer**(低遅延ストリーミング)を選べます。
-下の表はツールキット全体の機能と、それを提供するモデルまたはパイプラインを示します:
+FunASR はツールキットです。タスク、checkpoint、ランタイムを別々に選びます。
+あるモデルやアダプターの機能が、すべての配信バックエンドで使えるとは限りません。
-| | FunASR(ツールキット) | Whisper | クラウドAPI |
+| タスク | Checkpoint またはパイプライン | ランタイムの入口 | 主な制約 |
|---|---|---|---|
-| 最高速度 | **340倍リアルタイム**(Fun-ASR-Nano + vLLM) | 13倍リアルタイム | 〜1倍リアルタイム |
-| 話者認識 | ✅ VAD + CAM++パイプライン | ❌ pyannoteが必要 | ✅ 追加料金 |
-| 感情認識 | ✅ SenseVoice による | ❌ | ❌ |
-| 言語数 | チェックポイントごとに異なる(例:Qwen3-ASR 52、MLT-Nano 31、Nano 中/英/日) | 57 | サービスにより異なる |
-| ストリーミング | ✅ WebSocket(Paraformer) | ❌ | ✅ |
-| CPU対応 | ✅ 17倍リアルタイム(SenseVoice) | ❌ 遅すぎる | 該当なし |
-| セルフホスト | ✅ 対応(ツールキット: MIT、モデルごとに異なる) | ✅ MITライセンス | ❌ クラウドのみ |
-| コスト | 無料 | 無料 | $0.006/分〜 |
+| ファイル認識と感情・イベントタグ | SenseVoiceSmall | Python `AutoModel`、CPU または GPU | 5言語の checkpoint。タグは話者の身元を示しません。 |
+| LLM によるファイル認識 | Fun-ASR-Nano | `AutoModel`、または文書化された GPU 分離エンジン `AutoModelVLLM` | 基本 Nano は中・英・日と中国語方言・アクセント。timestamp は checkpoint と経路に依存します。 |
+| より多くの言語のファイル認識 | Fun-ASR-MLT-Nano | Python `AutoModel` | 独立した31言語 checkpoint。その対応言語を基本 Nano に当てはめないでください。 |
+| チャンク単位のライブ認識 | Paraformer-zh-streaming | ストリーミング SDK または runtime WebSocket | ストリーミング checkpoint とセッション別 cache が必要です。 |
+| 話者付きファイル認識 | SenseVoiceSmall + FSMN-VAD + CAM++ | `AutoModel` の VAD と埋め込みクラスタリング | 番号は録音内の匿名ラベルで、登録済み人物の識別ではありません。 |
+| テキスト・時刻・話者の同時生成 | 第三者 OpenMOSS の MOSS-Transcribe-Diarize | MOSS ガイドの FunASR adapter または上流バックエンド | オフライン、録音内の匿名ラベル。統合経路に外部 VAD/話者モデルは付けません。 |
+| ネイティブ CPU/エッジ認識 | Fun-ASR-Nano または SenseVoiceSmall GGUF | llama.cpp runtime | 対応する変換済み重みが必要。GGUF は Python `AutoModel` 用 checkpoint ではありません。 |
+
+[Model Zoo](./model_zoo/readme.md) と [デプロイ方式](./docs/deployment_matrix_ja.md)
+でインターフェースとライセンスの制約を確認し、対象音声・ハードウェアで評価してください。
---
@@ -83,17 +90,14 @@ Whisper は単一モデルですが、**FunASR はツールキット**です—
## ベンチマーク
-> 184件の長時間音声(計192分)。[詳細レポート →](https://modelscope.github.io/FunASR/benchmark.html)
+[過去の評価レポート](https://modelscope.github.io/FunASR/benchmark.html) と
+[分離エンジンの測定](./docs/vllm_guide.md#benchmark) に元の結果を残しています。
+別々の記録であり、一般的な速度順位や本番容量を保証するものではありません。
-| モデル | 中国語 CER ↓ | GPU速度 | CPU速度 | Whisper-large-v3比 |
-|--------|------|---------|---------|-------------------|
-| **Fun-ASR-Nano**(vLLM) | **8.20%** | **340倍**リアルタイム | — | 🚀 **26倍高速** |
-| **SenseVoice-Small** | **7.81%** | **170倍**リアルタイム | **17倍**リアルタイム | 🚀 **13倍高速** |
-| **Paraformer-Large** | 10.18% | **120倍**リアルタイム | **15倍**リアルタイム | 🚀 **9倍高速** |
-| Whisper-large-v3-turbo | 21.71% | 46倍リアルタイム | ❌ | 3.4倍高速 |
-| Whisper-large-v3 | 20.02% | 13倍リアルタイム | ❌ | ベースライン |
-
-> **ポイント:** FunASRのCPU速度は、WhisperのGPU速度より速い。
+[RTFx と再現性の説明](./docs/benchmark/rtf_reproducibility.md) に従い、
+checkpoint/revision、音声セット、ハードウェア、バッチ、ウォームアップ、計測範囲、
+CER/WER をそろえて比較してください。オフラインのスループットはストリーミングの
+遅延ではありません。[移行評価の例](./examples/migration/) で自分の録音を評価できます。
---
@@ -121,12 +125,17 @@ pip install funasr
## モデル一覧
+第三者モデルも含みます。MOSS-Transcribe-Diarize の公開元は **OpenMOSS** で、
+FunASR はアダプターを提供します。統合経路はオフラインで、匿名話者ラベルは
+録音内だけで有効です。リアルタイム処理や既知の人物の識別ではありません。
+モデルのライセンスはツールキットの MIT ライセンスとは別に確認してください。
+
| モデル | タスク | 言語 | パラメータ | リンク |
|--------|--------|------|-----------|--------|
| **Fun-ASR-Nano** | 認識 | 中/英/日 + 中国語方言 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) [GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) |
| **Fun-ASR-MLT-Nano** | 認識 | 31言語 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) |
| **SenseVoiceSmall** | 認識 + 感情 + イベント | 中/英/日/韓/粤 | 234M | [⭐](https://www.modelscope.cn/models/iic/SenseVoiceSmall) [🤗](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) [GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF) |
-| **MOSS-Transcribe-Diarize** | オフライン認識 + タイムスタンプ + 匿名話者 | 公式モデルカードを参照 | 公式モデルカードを参照 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [ガイド](./docs/moss_transcribe_diarize.md) |
+| **MOSS-Transcribe-Diarize** | 第三者 OpenMOSS:オフライン認識 + タイムスタンプ + 匿名話者 | 公式モデルカードを参照 | 公式モデルカードを参照 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [ガイド](./docs/moss_transcribe_diarize.md) |
| **Paraformer-zh** | 認識 + タイムスタンプ | 中/英 | 220M | [⭐](https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) [🤗](https://huggingface.co/funasr/paraformer-zh) |
| Qwen3-ASR | 認識、52言語 | 多言語 | 1.7B | [使用法](examples/industrial_data_pretraining/qwen3_asr) |
| GLM-ASR-Nano | 認識、17言語 | 多言語 | 1.5B | [使用法](examples/industrial_data_pretraining/glm_asr) |
diff --git a/README_ko.md b/README_ko.md
index d37a62e53..3e48b611b 100644
--- a/README_ko.md
+++ b/README_ko.md
@@ -31,26 +31,33 @@
## 빠른 시작
```bash
-pip install funasr
+python -m pip install torch torchaudio
+python -m pip install funasr
```
+아래는 공개 샘플을 사용하는 CPU 우선 예제입니다. GPU를 사용하려면
+[설치 가이드](./docs/installation/installation.md)에 따라 호환되는 PyTorch/CUDA
+환경을 준비하고 `torch.cuda.is_available()`을 확인한 뒤 `device="cuda"`로 바꾸세요.
+
```python
from funasr import AutoModel
+from funasr.utils.postprocess_utils import rich_transcription_postprocess
-model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cuda")
-result = model.generate(input="meeting.wav")
-```
+model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cpu")
+result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
-**출력** — 화자 라벨, 타임스탬프, 구두점이 포함된 구조화된 텍스트:
-```
-[00:00.4 → 00:03.8] 화자0: Q3 계획에 대해 논의하겠습니다.
-[00:04.2 → 00:07.1] 화자1: 좋습니다. 세 가지 포인트가 있습니다.
-[00:07.5 → 00:12.3] 화자0: 말씀하세요. 30분 남았습니다.
+for seg in result[0]["sentence_info"]:
+ print(f"[{seg['start']/1000:.1f}s] 화자{seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")
```
-한 번의 `AutoModel` 파이프라인 호출이지만, SenseVoiceSmall, FSMN-VAD,
-CAM++라는 독립된 모델을 조합합니다. 화자 분리는 SenseVoiceSmall 자체가 아니라
-CAM++에서 제공합니다.
+실제로 반환된 VAD 구간의 시작 시각(초), 익명 화자 번호, SenseVoice 태그를 제거한
+텍스트를 출력합니다. 텍스트와 구간은 오디오와 checkpoint에 따라 달라지므로
+고정된 인식 결과를 제시하지 않습니다.
+
+CAM++는 `spk_embedding` 벡터를 추출하고, `AutoModel`이 클러스터링하여 VAD
+구간에 화자 번호를 할당합니다. 번호는 해당 녹음 안에서만 유효하며 알려진 인물의
+신원이나 SenseVoiceSmall 단독 출력이 아닙니다. 자세한 내용은
+[SDK 계약](./docs/python_api.md)을 참고하세요.
처음 사용한다면 [Colab 빠른 시작](./examples/colab/README_ko.md)으로 먼저 확인할 수 있습니다. 어떤 모델을 선택할지 고민된다면 [모델 선택 가이드](./docs/model_selection_ko.md)를 참고하세요.
@@ -60,22 +67,21 @@ CAM++에서 제공합니다.
### 왜 FunASR인가?
-Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞게
-**Fun-ASR-Nano**(중국어/영어/일본어 및 중국어 방언/지역 억양, GPU),
-**Fun-ASR-MLT-Nano**(31개 언어), **SenseVoiceSmall**(5개 언어 ASR와
-감정·오디오 이벤트), **Paraformer**(저지연 스트리밍)를 선택하세요.
-아래 표는 툴킷 전체의 기능과 이를 제공하는 모델 또는 파이프라인을 보여 줍니다:
+FunASR는 툴킷입니다. 작업, checkpoint, 런타임을 각각 선택해야 합니다.
+한 모델이나 어댑터가 지원하는 기능이 모든 서빙 백엔드에서 지원되는 것은 아닙니다.
-| | FunASR(툴킷) | Whisper | 클라우드 API |
+| 작업 | Checkpoint 또는 파이프라인 | 런타임 진입점 | 주요 제한 |
|---|---|---|---|
-| 최고 속도 | **340배 실시간**(Fun-ASR-Nano + vLLM) | 13배 실시간 | ~1배 실시간 |
-| 화자 인식 | ✅ VAD + CAM++ 파이프라인 | ❌ pyannote 필요 | ✅ 추가 비용 |
-| 감정 인식 | ✅ SenseVoice 제공 | ❌ | ❌ |
-| 언어 수 | 체크포인트별 상이(예: Qwen3-ASR 52, MLT-Nano 31, Nano 중/영/일) | 57개 | 서비스마다 다름 |
-| 스트리밍 | ✅ WebSocket(Paraformer) | ❌ | ✅ |
-| CPU 사용 | ✅ 17배 실시간(SenseVoice) | ❌ 너무 느림 | 해당 없음 |
-| 자체 호스팅 | ✅ 지원 (툴킷: MIT, 모델별 상이) | ✅ MIT 라이선스 | ❌ 클라우드만 |
-| 비용 | 무료 | 무료 | $0.006/분~ |
+| 파일 전사와 감정/이벤트 태그 | SenseVoiceSmall | Python `AutoModel`, CPU 또는 GPU | 5개 언어 checkpoint이며 태그는 화자 신원을 나타내지 않습니다. |
+| LLM 기반 파일 전사 | Fun-ASR-Nano | `AutoModel`, 또는 문서의 GPU 분리 엔진 `AutoModelVLLM` | 기본 Nano는 중/영/일 및 중국어 방언/억양을 지원하며 timestamp는 checkpoint와 경로에 따라 다릅니다. |
+| 더 많은 언어의 파일 전사 | Fun-ASR-MLT-Nano | Python `AutoModel` | 별도의 31개 언어 checkpoint이며 기본 Nano에 같은 범위를 적용하지 않습니다. |
+| 청크 단위 실시간 전사 | Paraformer-zh-streaming | 스트리밍 SDK 또는 runtime WebSocket | 스트리밍 checkpoint와 세션별 cache가 필요하며 오프라인 checkpoint로 대체할 수 없습니다. |
+| 화자별 파일 전사 | SenseVoiceSmall + FSMN-VAD + CAM++ | `AutoModel`의 VAD와 임베딩 클러스터링 | 녹음 안의 익명 번호이며 등록된 인물의 신원 식별이 아닙니다. |
+| 텍스트, 시각, 화자 공동 생성 | 제3자 OpenMOSS의 MOSS-Transcribe-Diarize | MOSS 가이드의 FunASR adapter 또는 업스트림 백엔드 | 오프라인, 녹음 내 익명 라벨이며 통합 경로에 외부 VAD/화자 모델을 붙이지 않습니다. |
+| 네이티브 CPU/엣지 전사 | Fun-ASR-Nano 또는 SenseVoiceSmall GGUF | llama.cpp runtime | 호환되는 변환 가중치가 필요하며 GGUF는 Python `AutoModel`용 checkpoint가 아닙니다. |
+
+[Model Zoo](./model_zoo/readme.md)와 [배포 매트릭스](./docs/deployment_matrix_ko.md)에서
+인터페이스와 라이선스 제한을 확인하고 대상 오디오와 하드웨어로 평가하세요.
---
@@ -83,17 +89,14 @@ Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞
## 벤치마크
-> 184개 장시간 오디오(총 192분). [상세 보고서 →](https://modelscope.github.io/FunASR/benchmark.html)
+[기존 평가 보고서](https://modelscope.github.io/FunASR/benchmark.html)와
+[분리 엔진 측정](./docs/vllm_guide.md#benchmark)에 원래 결과를 보존합니다.
+서로 다른 기록이며 보편적인 속도 순위나 운영 용량을 보장하지 않습니다.
-| 모델 | 중국어 CER ↓ | GPU 속도 | CPU 속도 | Whisper-large-v3 대비 |
-|------|------|----------|----------|---------------------|
-| **Fun-ASR-Nano**(vLLM) | **8.20%** | **340배** 실시간 | — | 🚀 **26배 빠름** |
-| **SenseVoice-Small** | **7.81%** | **170배** 실시간 | **17배** 실시간 | 🚀 **13배 빠름** |
-| **Paraformer-Large** | 10.18% | **120배** 실시간 | **15배** 실시간 | 🚀 **9배 빠름** |
-| Whisper-large-v3-turbo | 21.71% | 46배 실시간 | ❌ | 3.4배 빠름 |
-| Whisper-large-v3 | 20.02% | 13배 실시간 | ❌ | 기준선 |
-
-> **핵심:** FunASR의 CPU 속도가 Whisper의 GPU 속도보다 빠릅니다.
+[RTFx와 재현성 설명](./docs/benchmark/rtf_reproducibility.md)에 따라
+checkpoint/revision, 오디오 집합, 하드웨어, 배치, 워밍업, 측정 범위, CER/WER를
+맞춰 비교하세요. 오프라인 처리량은 스트리밍 지연이 아닙니다.
+[마이그레이션 평가 예제](./examples/migration/)로 자신의 녹음을 측정할 수 있습니다.
---
@@ -121,12 +124,17 @@ pip install funasr
## 모델 목록
+제3자 모델도 포함합니다. MOSS-Transcribe-Diarize의 배포 주체는 **OpenMOSS**이며
+FunASR는 어댑터를 제공합니다. 통합 경로는 오프라인이고 익명 화자 라벨은 해당 녹음
+안에서만 유효합니다. 실시간 처리나 알려진 인물의 신원 식별이 아닙니다.
+모델 라이선스는 툴킷의 MIT 라이선스와 별도로 확인해야 합니다.
+
| 모델 | 작업 | 언어 | 파라미터 | 링크 |
|------|------|------|---------|------|
| **Fun-ASR-Nano** | 인식 | 중/영/일 + 중국어 방언 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) [GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) |
| **Fun-ASR-MLT-Nano** | 인식 | 31개 언어 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) |
| **SenseVoiceSmall** | 인식 + 감정 + 이벤트 | 중/영/일/한/광둥어 | 234M | [⭐](https://www.modelscope.cn/models/iic/SenseVoiceSmall) [🤗](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) [GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF) |
-| **MOSS-Transcribe-Diarize** | 오프라인 인식 + 타임스탬프 + 익명 화자 | 공식 모델 카드 참조 | 공식 모델 카드 참조 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [가이드](./docs/moss_transcribe_diarize.md) |
+| **MOSS-Transcribe-Diarize** | 제3자 OpenMOSS: 오프라인 인식 + 타임스탬프 + 익명 화자 | 공식 모델 카드 참조 | 공식 모델 카드 참조 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [가이드](./docs/moss_transcribe_diarize.md) |
| **Paraformer-zh** | 인식 + 타임스탬프 | 중/영 | 220M | [⭐](https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) [🤗](https://huggingface.co/funasr/paraformer-zh) |
| Qwen3-ASR | 인식, 52개 언어 | 다국어 | 1.7B | [사용법](examples/industrial_data_pretraining/qwen3_asr) |
| GLM-ASR-Nano | 인식, 17개 언어 | 다국어 | 1.5B | [사용법](examples/industrial_data_pretraining/glm_asr) |
diff --git a/README_zh.md b/README_zh.md
index c7196d88d..46bb71b46 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -53,11 +53,13 @@ PY
只有这里输出 `True` 时才使用 `device="cuda"`;否则请先使用
`device="cpu"`,或重新安装匹配 CUDA 的 PyTorch wheel。
+以下 SenseVoiceSmall 示例先使用 CPU,并组合 FSMN-VAD 与 CAM++。
+
```python
from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess
-model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cuda")
+model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cpu")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
# AutoModel pipeline 返回带说话人 id 和时间戳的 VAD 分段:
@@ -65,14 +67,13 @@ for seg in result[0]["sentence_info"]:
print(f"[{seg['start']/1000:.1f}s] 说话人{seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")
```
-**输出** — 带说话人标签、时间戳和标点的结构化文本:
-```
-[0.6s] 说话人0: 欢迎大家来体验达摩院推出的语音识别模型
-```
+代码打印实际返回的 VAD 分段起点(秒)、匿名说话人编号和去除 SenseVoice
+标签后的文本。文本与分段边界取决于音频和 checkpoint,这里不预设转写结果。
-这是一次 `AutoModel` pipeline 调用,实际组合了 SenseVoiceSmall、FSMN-VAD
-和 CAM++ 三个独立模型;说话人分离由 CAM++ 提供,并非 SenseVoiceSmall
-checkpoint 的内置输出。
+CAM++ 提取 `spk_embedding` 说话人嵌入,`AutoModel` 再聚类并为 VAD 分段分配
+说话人编号;编号仅在当前录音内有效,不是已知人物身份,也不是 SenseVoiceSmall
+checkpoint 的内置输出。组件和返回字段见 [SDK 契约](./docs/python_api_zh.md)。
+需要 GPU 时,先按上文验证环境,再将 `device` 改为 `"cuda"`。
SenseVoice 论文见 [arXiv:2407.04051](https://arxiv.org/abs/2407.04051),
模型见 [Hugging Face checkpoint](https://huggingface.co/FunAudioLLM/SenseVoiceSmall),
边缘部署可用 [GGUF checkpoint](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF)。
@@ -110,22 +111,21 @@ results = model.generate(["audio1.wav", "audio2.wav"], language="auto")
### 为什么选 FunASR?
-Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型:
-**Fun-ASR-Nano**(中/英/日及中文方言,需 GPU)、
-**Fun-ASR-MLT-Nano**(31 语种)、**SenseVoiceSmall**(五语种 ASR,
-并返回情感与音频事件标签)、**Paraformer**(低延迟流式)。下表展示的是
-工具箱级能力,并标明由哪个模型或 pipeline 提供:
+FunASR 是工具箱,需要分别选择任务、checkpoint 和运行时。
+一个模型或适配器支持的能力,不代表所有服务后端都支持。
-| | FunASR(工具箱) | Whisper | 云端 API |
+| 任务 | Checkpoint 或 pipeline | 运行时入口 | 主要边界 |
|---|---|---|---|
-| 最高速度 | **340 倍实时**(Fun-ASR-Nano + vLLM) | 13 倍实时 | ~1 倍实时 |
-| 说话人识别 | ✅ 由 VAD + CAM++ pipeline 提供 | ❌ 需要 pyannote | ✅ 额外付费 |
-| 情感识别 | ✅ 由 SenseVoice 提供 | ❌ | ❌ |
-| 语言数 | 取决于 checkpoint(例如 Qwen3-ASR 52、MLT-Nano 31、Nano 中/英/日) | 57 | 因服务而异 |
-| 流式识别 | ✅ WebSocket(Paraformer) | ❌ | ✅ |
-| CPU 可用 | ✅ 17 倍实时(SenseVoice) | ❌ 太慢 | 不适用 |
-| 私有部署 | ✅ 支持(工具箱 MIT;模型协议各异) | ✅ MIT 开源 | ❌ 仅云端 |
-| 费用 | 免费 | 免费 | ¥0.04/分钟起 |
+| 文件转写及情感/事件标签 | SenseVoiceSmall | Python `AutoModel`,CPU 或 GPU | 五语种 checkpoint;标签不代表说话人身份。 |
+| LLM 文件转写 | Fun-ASR-Nano | `AutoModel`;文档指定的 GPU 拆分引擎 `AutoModelVLLM` | 基础 Nano 覆盖中/英/日及中文方言/口音;时间戳取决于 checkpoint 和路径。 |
+| 更多语种的文件转写 | Fun-ASR-MLT-Nano | Python `AutoModel` | 独立的 31 语种 checkpoint,不能把其覆盖范围归给基础 Nano。 |
+| 分块实时转写 | Paraformer-zh-streaming | 流式 SDK 或 runtime WebSocket 服务 | 使用流式 checkpoint 和会话独立 cache,不能替换为离线 checkpoint。 |
+| 带说话人的文件转写 | SenseVoiceSmall + FSMN-VAD + CAM++ | `AutoModel` 的 VAD 与嵌入聚类 | 编号仅在当前录音内有效,不是已注册人物身份识别。 |
+| 联合文本、时间戳与说话人 | 第三方 OpenMOSS 的 MOSS-Transcribe-Diarize | MOSS 指南中的 FunASR adapter 或上游后端 | 离线、录音内匿名标签;统一路径不外接 VAD/说话人 pipeline。 |
+| 原生 CPU/端侧转写 | Fun-ASR-Nano 或 SenseVoiceSmall GGUF | llama.cpp runtime | 需要匹配的转换权重,GGUF 不能作为 Python `AutoModel` checkpoint 加载。 |
+
+Checkpoint、接口与协议边界见 [Model Zoo](./model_zoo/readme_zh.md) 和
+[部署矩阵](./docs/deployment_matrix_zh.md)。请用目标音频与硬件评测后再选运行时。
第一次试用 FunASR?可以先跑 [Colab 快速体验](./examples/colab/README_zh.md),再配置本地环境。还不确定先用哪个模型?先看 [模型选择指南](./docs/model_selection_zh.md)。计划从 Whisper 或云端 ASR 切换?请按 [迁移指南](./docs/migration_from_whisper_zh.md) 和 [评测示例](./examples/migration/) 用代表性音频评测、映射功能并安全上线。
@@ -135,17 +135,14 @@ Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型
## 性能评测
-> 184 条长音频(共 192 分钟)。[完整报告 →](https://modelscope.github.io/FunASR/zh/benchmark.html)
-
-| 模型 | 中文 CER ↓ | GPU 速度 | CPU 速度 | 对比 Whisper-large-v3 |
-|------|------|----------|----------|---------------------|
-| **Fun-ASR-Nano**(vLLM) | **8.20%** | **340 倍**实时 | — | 🚀 **快 26 倍** |
-| **SenseVoice-Small** | **7.81%** | **170 倍**实时 | **17 倍**实时 | 🚀 **快 13 倍** |
-| **Paraformer-Large** | 10.18% | **120 倍**实时 | **15 倍**实时 | 🚀 **快 9 倍** |
-| Whisper-large-v3-turbo | 21.71% | 46 倍实时 | ❌ | 快 3.4 倍 |
-| Whisper-large-v3 | 20.02% | 13 倍实时 | ❌ | 基准 |
+[历史评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) 与
+[拆分引擎测量](./docs/vllm_guide_zh.md#benchmark) 保留原始结果。
+两者是独立记录,不能合并成通用速度排名或生产容量承诺。
-> **一句话:** FunASR 在 CPU 上的速度,比 Whisper 在 GPU 上还快。
+请按 [RTFx 与可复现评测说明](./docs/benchmark/rtf_reproducibility.md) 对齐
+checkpoint/revision、音频集、硬件、批量大小、预热、计时范围与 CER/WER。
+离线吞吐量不等于流式延迟;可使用 [迁移评测示例](./examples/migration/)
+以相同口径测量自己的录音。
---
@@ -181,12 +178,16 @@ pip install -e ./
## 模型列表
+列表包含第三方模型。MOSS-Transcribe-Diarize 由 **OpenMOSS** 发布,FunASR
+提供适配器,并不拥有其权重。统一路径为离线处理,匿名说话人标签仅在当前录音
+内有效,不是实时流式或已知人物身份识别。模型协议与工具箱 MIT 协议分别适用。
+
| 模型 | 任务 | 语言 | 参数量 | 链接 |
|------|------|------|--------|------|
| **Fun-ASR-Nano** | 识别 | 中/英/日 + 中文方言 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) [GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) |
| **Fun-ASR-MLT-Nano** | 识别 | 31 种语言 | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) |
| **SenseVoiceSmall** | 识别 + 情感 + 事件 | 中/英/日/韩/粤 | 234M | [⭐](https://www.modelscope.cn/models/iic/SenseVoiceSmall) [🤗](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) [GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF) [论文](https://arxiv.org/abs/2407.04051) |
-| **MOSS-Transcribe-Diarize** | 离线识别 + 时间戳 + 匿名说话人 | 以官方模型卡为准 | 以官方模型卡为准 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [指南](./docs/moss_transcribe_diarize_zh.md) |
+| **MOSS-Transcribe-Diarize** | 第三方 OpenMOSS:离线识别 + 时间戳 + 匿名说话人 | 以官方模型卡为准 | 以官方模型卡为准 | [🤗](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) [指南](./docs/moss_transcribe_diarize_zh.md) |
| **Paraformer-zh** | 识别 + 时间戳 | 中/英 | 220M | [⭐](https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) [🤗](https://huggingface.co/funasr/paraformer-zh) |
| Paraformer-zh-streaming | 流式识别 | 中/英 | 220M | [⭐](https://modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online/summary) [🤗](https://huggingface.co/funasr/paraformer-zh-streaming) |
| Qwen3-ASR | 识别,52 种语言 | 多语言 | 1.7B | [使用](examples/industrial_data_pretraining/qwen3_asr) |
@@ -195,7 +196,7 @@ pip install -e ./
| Whisper-large-v3-turbo | 识别 + 翻译 | 多语言 | 809M | [使用](examples/industrial_data_pretraining/whisper) |
| ct-punc | 标点恢复 | 中/英 | 290M | [⭐](https://modelscope.cn/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large/summary) [🤗](https://huggingface.co/funasr/ct-punc) |
| fsmn-vad | 语音检测 | 中/英 | 0.4M | [⭐](https://modelscope.cn/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/summary) [🤗](https://huggingface.co/funasr/fsmn-vad) |
-| cam++ | 说话人分离 | — | 7.2M | [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) |
+| cam++ | 说话人嵌入(speaker embeddings,pipeline 组件) | — | 7.2M | [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) |
| emotion2vec+large | 情感识别 | — | 300M | [⭐](https://modelscope.cn/models/iic/emotion2vec_plus_large/summary) [🤗](https://huggingface.co/emotion2vec/emotion2vec_plus_large) |
---
diff --git a/tests/test_readme_task_entrypoints.py b/tests/test_readme_task_entrypoints.py
new file mode 100644
index 000000000..261d27727
--- /dev/null
+++ b/tests/test_readme_task_entrypoints.py
@@ -0,0 +1,169 @@
+"""README entrypoint contracts; parse examples without importing or running models."""
+
+import ast
+from pathlib import Path
+import re
+import subprocess
+from urllib.parse import unquote, urlsplit
+
+from bs4 import BeautifulSoup
+from markdown import Markdown
+import pytest
+
+
+ROOT = Path(__file__).resolve().parents[1]
+LANGUAGES = {
+ "README.md": ("Why FunASR?", "Benchmark", "What's new", "Model Zoo", "offline", "anonymous", "recording"),
+ "README_zh.md": ("为什么选 FunASR?", "性能评测", "最新动态", "模型列表", "离线", "匿名", "录音"),
+ "README_ja.md": ("なぜFunASRを選ぶのか?", "ベンチマーク", "最新情報", "モデル一覧", "オフライン", "匿名", "録音"),
+ "README_ko.md": ("왜 FunASR인가?", "벤치마크", "최신 소식", "모델 목록", "오프라인", "익명", "녹음"),
+}
+
+
+def read(name):
+ return (ROOT / name).read_text(encoding="utf-8")
+
+
+def fences(text):
+ language = None
+ lines = []
+ for line in text.splitlines():
+ if line.startswith("```"):
+ if language is None:
+ language, lines = line[3:].strip(), []
+ else:
+ yield language, "\n".join(lines) + "\n"
+ language = None
+ elif language is not None:
+ lines.append(line)
+ assert language is None, "Unclosed code fence"
+
+
+def render(text):
+ def slugify(value, separator):
+ return re.sub(r"[^\w\- ]", "", value.lower()).replace(" ", separator)
+
+ return BeautifulSoup(Markdown(
+ extensions=["extra", "toc"],
+ extension_configs={"toc": {"slugify": slugify}},
+ ).convert(text), "html.parser")
+
+
+def section(name, title):
+ soup = render(read(name))
+ heading = soup.find(re.compile(r"^h[1-6]$"), string=title)
+ assert heading is not None, title
+ nodes = []
+ for node in heading.next_siblings:
+ if node.name and re.fullmatch(r"h[1-6]", node.name) and node.name <= heading.name:
+ break
+ nodes.append(str(node))
+ return BeautifulSoup("".join(nodes), "html.parser")
+
+
[email protected]("name", LANGUAGES)
+def test_task_table_names_checkpoint_runtime_and_boundaries(name):
+ body = section(name, LANGUAGES[name][0])
+ table = body.find("table")
+ assert table is not None
+ assert len(table.select("thead th")) == 4
+ assert not any(word in table.get_text() for word in ("Whisper", "Cloud APIs", "$0.006", "¥0.04"))
+ for checkpoint in ("SenseVoiceSmall", "Fun-ASR-Nano", "Fun-ASR-MLT-Nano", "Paraformer", "MOSS-Transcribe-Diarize", "llama.cpp"):
+ assert checkpoint in table.get_text(), checkpoint
+ assert "OpenMOSS" in table.get_text()
+ assert "CPU" in table.get_text() and "WebSocket" in table.get_text()
+ assert all(len(row.find_all("td")) == 4 for row in table.select("tbody tr"))
+
+
[email protected]("name", LANGUAGES)
+def test_benchmark_links_evidence_instead_of_generalizing_vendor_performance(name):
+ body = section(name, LANGUAGES[name][1])
+ assert not body.find("table"), "Keep historical numeric tables in their evidence reports"
+ links = [a.get("href", "") for a in body.find_all("a")]
+ assert any("docs/benchmark/rtf_reproducibility.md" in href for href in links)
+ assert any("benchmark.html" in href for href in links)
+ assert "Whisper" not in body.get_text(), "Do not promote a cross-hardware comparison into a blanket claim"
+ assert not re.search(r"[$¥]\s*0\.\d+", read(name))
+
+
[email protected]("name", LANGUAGES)
+def test_first_sensevoice_example_is_cpu_and_prints_real_return_fields(name):
+ examples = [code for language, code in fences(read(name))
+ if language == "python" and "SenseVoiceSmall" in code]
+ assert examples
+ tree = ast.parse(examples[0])
+ calls = [node for node in ast.walk(tree) if isinstance(node, ast.Call)]
+ constructor = next(node for node in calls if isinstance(node.func, ast.Name) and node.func.id == "AutoModel")
+ options = {kw.arg: ast.literal_eval(kw.value) for kw in constructor.keywords}
+ assert options["device"] == "cpu"
+ assert options["vad_model"] == "fsmn-vad" and options["spk_model"] == "cam++"
+ generate = next(node for node in calls if isinstance(node.func, ast.Attribute) and node.func.attr == "generate")
+ sample = next(ast.literal_eval(kw.value) for kw in generate.keywords if kw.arg == "input")
+ assert sample.startswith("https://") and "asr_example_zh.wav" in sample
+ assert "asr_example_zh.wav" in read("examples/industrial_data_pretraining/paraformer/demo.py")
+ assert any(isinstance(node.func, ast.Name) and node.func.id == "print" for node in calls)
+ fields = {node.slice.value for node in ast.walk(tree)
+ if isinstance(node, ast.Subscript) and isinstance(node.slice, ast.Constant)}
+ assert {"sentence_info", "start", "spk", "sentence"} <= fields
+ assert "spk_embedding" in read(name), "Explain embeddings separately from pipeline diarization"
+
+
[email protected]("name", LANGUAGES)
+def test_model_zoo_retains_third_party_moss_and_recording_scope(name):
+ body = section(name, LANGUAGES[name][3])
+ moss = next(row for row in body.select("tr") if "MOSS-Transcribe-Diarize" in row.get_text())
+ assert "OpenMOSS" in moss.get_text()
+ for boundary in LANGUAGES[name][4:]:
+ assert boundary in body.get_text().lower(), boundary
+ assert any("docs/moss_transcribe_diarize" in a.get("href", "") for a in moss.find_all("a"))
+ for row in body.select("tr"):
+ if row.find("td") and row.find("td").get_text(strip=True) == "cam++":
+ assert "embedding" in row.find_all("td")[1].get_text().lower()
+
+
[email protected]("name", LANGUAGES)
+def test_news_stays_three_items_and_keeps_release_history(name):
+ body = section(name, LANGUAGES[name][2])
+ assert len(body.select("li")) == 3
+ assert any(a.get("href") == "https://github.com/modelscope/FunASR/releases" for a in body.find_all("a"))
+
+
[email protected]("name", LANGUAGES)
+def test_python_and_shell_examples_have_valid_syntax_without_execution(name):
+ for language, code in fences(read(name)):
+ if language == "python":
+ compile(code, name, "exec")
+ elif language in {"bash", "sh", "shell"}:
+ result = subprocess.run(["bash", "-n"], input=code, text=True, capture_output=True)
+ assert result.returncode == 0, result.stderr
+ for embedded in re.findall(r"<<'PY'\n(.*?)\nPY", code, re.S):
+ ast.parse(embedded)
+
+
[email protected]("name", LANGUAGES)
+def test_relative_links_and_internal_navigation_resolve(name):
+ soup = render(read(name))
+ for link in soup.select("a[href]"):
+ target = urlsplit(link["href"])
+ if target.scheme or target.netloc:
+ continue
+ resolved = (ROOT / unquote(target.path)).resolve() if target.path else ROOT / name
+ assert resolved.is_relative_to(ROOT)
+ assert resolved.exists(), (name, link["href"])
+ if target.fragment and resolved.suffix == ".md":
+ destination = render(resolved.read_text(encoding="utf-8"))
+ anchors = {node.get("id") for node in destination.select("[id]")}
+ anchors.update(node.get("name") for node in destination.select("a[name]"))
+ assert unquote(target.fragment) in anchors, (name, link["href"])
+
+
+def test_speaker_field_explanation_has_implementation_evidence():
+ camp = ast.parse(read("funasr/models/campplus/model.py"))
+ assert any(isinstance(node, ast.Constant) and node.value == "spk_embedding" for node in ast.walk(camp))
+ auto = ast.parse(read("funasr/auto/auto_model.py"))
+ strings = {node.value for node in ast.walk(auto) if isinstance(node, ast.Constant) and isinstance(node.value, str)}
+ assert {"sentence_info", "start", "sentence"} <= strings
+ assert any(isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "distribute_spk" for node in ast.walk(auto))
+ utility = ast.parse(read("funasr/models/campplus/utils.py"))
+ distribute = next(node for node in utility.body if isinstance(node, ast.FunctionDef) and node.name == "distribute_spk")
+ assert any(isinstance(node, ast.Constant) and node.value == "spk" for node in ast.walk(distribute))