Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs: add ASR migration guide
zhifu gao5daa601f1ece
diff --git a/README.md b/README.mdindex 15ae1e03a..378f66b13 100644--- a/README.md+++ b/README.md@@ -21,7 +21,7 @@</p><p align="center">- <a href="#quick-start">Quick Start</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>+ <a href="#quick-start">Quick Start</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a></p>---@@ -66,6 +66,8 @@ That's it. **One model, one call** — VAD segmentation, speech recognition, pun| Cost | Free | Free | $0.006/min+ || CPU viable | ✅ 17x realtime | ❌ Too slow | N/A |+Planning a switch from Whisper or a cloud ASR provider? Use the [migration guide](./docs/migration_from_whisper.md) to benchmark representative audio, map features, and roll out safely.+---<a name="benchmark"></a>diff --git a/README_ja.md b/README_ja.mdindex 91151acf9..62bcbc185 100644--- a/README_ja.md+++ b/README_ja.md@@ -21,7 +21,7 @@</p><p align="center">- <a href="#クイックスタート">クイックスタート</a> · <a href="#ベンチマーク">ベンチマーク</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#モデル一覧">モデル一覧</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent連携</a> · <a href="https://modelscope.github.io/FunASR/">ドキュメント</a>+ <a href="#クイックスタート">クイックスタート</a> · <a href="#ベンチマーク">ベンチマーク</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#モデル一覧">モデル一覧</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent連携</a> · <a href="https://modelscope.github.io/FunASR/">ドキュメント</a></p>---diff --git a/README_ko.md b/README_ko.mdindex 2c53bd8e2..66cf207ce 100644--- a/README_ko.md+++ b/README_ko.md@@ -21,7 +21,7 @@</p><p align="center">- <a href="#빠른-시작">빠른 시작</a> · <a href="#벤치마크">벤치마크</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#모델-목록">모델 목록</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 연동</a> · <a href="https://modelscope.github.io/FunASR/">문서</a>+ <a href="#빠른-시작">빠른 시작</a> · <a href="#벤치마크">벤치마크</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#모델-목록">모델 목록</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 연동</a> · <a href="https://modelscope.github.io/FunASR/">문서</a></p>---diff --git a/README_zh.md b/README_zh.mdindex 782a46f27..62c41de43 100644--- a/README_zh.md+++ b/README_zh.md@@ -21,7 +21,7 @@</p><p align="center">- <a href="#快速开始">快速开始</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>+ <a href="#快速开始">快速开始</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a></p>---@@ -65,6 +65,8 @@ result = model.generate(input="meeting.wav")| 费用 | 免费 | 免费 | ¥0.04/分钟起 || CPU 可用 | ✅ 17 倍实时 | ❌ 太慢 | 不适用 |+计划从 Whisper 或云端 ASR 切换?请按 [迁移指南](./docs/migration_from_whisper_zh.md) 用代表性音频评测、映射功能并安全上线。+---<a name="性能评测"></a>diff --git a/docs/index.rst b/docs/index.rstindex fdbe8dc7d..551b30437 100644--- a/docs/index.rst+++ b/docs/index.rst@@ -106,6 +106,8 @@ Overview:maxdepth: 1:caption: Application+ ./migration_from_whisper.md+ ./migration_from_whisper_zh.md./use_case_showcase.md./use_case_showcase_zh.md./reference/application.mddiff --git a/docs/migration_from_whisper.md b/docs/migration_from_whisper.mdnew file mode 100644index 000000000..d38668f77--- /dev/null+++ b/docs/migration_from_whisper.md@@ -0,0 +1,96 @@+# Migrate from Whisper or Cloud ASR to FunASR++Use this guide when you already have a Whisper, OpenAI/Cloud ASR, or custom speech pipeline and want to decide whether FunASR is worth switching to. The goal is not to prove a benchmark with one sample file; it is to compare quality, speed, cost, and deployment fit on audio that looks like your real workload.++## When FunASR is a good fit++FunASR is usually worth evaluating when you need one or more of these properties:++- Private or self-hosted transcription where audio should stay inside your environment.+- High-throughput long-form transcription for meetings, archives, media, or call recordings.+- Speaker-aware transcripts with VAD, punctuation, timestamps, and diarization in one pipeline.+- An OpenAI-compatible audio endpoint for agents, Dify, LangChain, AutoGen, or internal apps.+- Streaming ASR or live captions with WebSocket/runtime service support.+- CPU-viable smoke tests before moving to GPU deployment.++Stay on your current pipeline if you need a managed service with no operations work, a vendor SLA, or a language/domain that your own benchmark shows FunASR does not handle well enough yet.++## Fast evaluation plan++1. Pick 20-50 representative audio files. Include short clips, long recordings, noisy samples, different speakers, and the languages or dialects you care about.+2. Run your current Whisper or cloud ASR pipeline exactly as you use it in production. Save transcripts, latency, cost, and failure cases.+3. Run FunASR locally with the README quick start, then choose a deployment path from the [deployment matrix](./deployment_matrix.md).+4. Compare output with human review or your normal WER/CER process. Do not compare only one clean demo file.+5. Run the OpenAI-compatible API smoke test if your application already uses OpenAI-style clients.+6. Record warmup time, model download time, device, GPU/CPU type, batch size, and audio duration separately from steady-state throughput.++## Feature mapping++| Existing workflow | FunASR path | What to validate |+|---|---|---|+| Whisper file transcription | [README quick start](../README.md#quick-start) with SenseVoice, Paraformer, or Fun-ASR-Nano | Transcript quality, timestamps, speed, model download, CPU/GPU behavior. |+| Whisper plus pyannote | `spk_model="cam++"` with VAD and punctuation | Speaker labels, speaker changes, overlapping speech, long silences. |+| OpenAI audio API or cloud batch ASR | [OpenAI-compatible API example](../examples/openai_api/) | `/v1/audio/transcriptions`, response format, client compatibility, upload limits. |+| Dify/LangChain/AutoGen agent audio | [Client recipes](../examples/openai_api/CLIENTS.md) or [MCP server](../examples/mcp_server/) | Tool latency, file handling, auth boundary, error reporting. |+| Live captions or call-center streaming | [Runtime service docs](../runtime/readme.md) | Chunking, endpointing, reconnects, backpressure, partial/final result behavior. |+| Subtitle generation | [Subtitle example](../examples/subtitle/) | Segment readability, line length, speaker labels, SRT/VTT compatibility. |+| Offline archive processing | [Batch ASR example](../examples/batch_asr_improved.py) | Manifest handling, retries, progress logs, throughput, failed-file recovery. |++## Minimal local comparison++Install FunASR and run the same file you used for your baseline:++```bash+pip install funasr+```++```python+from funasr import AutoModel++model = AutoModel(+ model="iic/SenseVoiceSmall",+ vad_model="fsmn-vad",+ spk_model="cam++",+ device="cuda", # use "cpu" for a portable smoke test+)+result = model.generate(input="sample.wav")+print(result)+```++For an API-style comparison:++```bash+pip install funasr fastapi uvicorn python-multipart+funasr-server --model sensevoice --device cuda++curl http://localhost:8000/v1/audio/transcriptions \+ -F [email protected] \+ -F model=sensevoice \+ -F response_format=verbose_json+```++If you want a repeatable container smoke test, start from `examples/openai_api/docker-compose.yml` and verify it with `BASE_URL=http://localhost:8000 bash examples/openai_api/smoke_test.sh`.++## Quality and speed checklist++Track these fields for both the old pipeline and FunASR:++- Audio duration, language, domain, sample rate, channel count, and speaker count.+- Model name, model version, FunASR version, Python/PyTorch/CUDA versions, and Docker image tag if used.+- Hardware, device mode, batch size, streaming chunk size, and whether warmup/model download is excluded.+- WER/CER or human review notes for names, numbers, punctuation, diarization, timestamps, and domain terms.+- Latency, throughput, GPU/CPU memory, cost per hour of audio, and failed-file rate.+- Operational requirements: authentication, upload limits, TLS, logs, monitoring, retries, and retention rules.++## Rollout checklist++- Keep the old pipeline available until FunASR passes your representative benchmark.+- Start with an internal endpoint or batch job before exposing a public API.+- Add request IDs and log audio duration, model, device, latency, and error type for every request.+- Pin the model alias and deployment command in your runbook.+- Test noisy audio, silence, overlapping speakers, long files, non-UTF-8 filenames, and network interruptions.+- Open a [Deployment Help issue](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) with your command, logs, model, device, and sample characteristics if you hit a blocker.++## Share the result++If FunASR replaces or complements your existing ASR stack, consider opening a [showcase issue](https://github.com/modelscope/FunASR/issues/new?template=showcase.md). Migration reports with hardware, speed, quality notes, and deployment details help new users choose the right path faster.diff --git a/docs/migration_from_whisper_zh.md b/docs/migration_from_whisper_zh.mdnew file mode 100644index 000000000..5e593d90c--- /dev/null+++ b/docs/migration_from_whisper_zh.md@@ -0,0 +1,96 @@+# 从 Whisper 或云端 ASR 迁移到 FunASR++当你已经有 Whisper、OpenAI/云端 ASR 或自研语音流水线,并想判断是否值得切到 FunASR 时,可以按这个指南评估。目标不是用一个样例音频证明结论,而是在真实业务音频上比较质量、速度、成本和部署可行性。++## 什么时候值得评估 FunASR++如果你有以下需求,FunASR 通常值得优先测试:++- 音频需要留在私有环境内,不能上传到外部云服务。+- 会议、归档、媒体、客服录音等长音频需要高吞吐转写。+- 希望一条流水线内完成 VAD、标点、时间戳和说话人分离。+- 需要 OpenAI 兼容音频接口,接入 Agent、Dify、LangChain、AutoGen 或内部应用。+- 需要 WebSocket/runtime 服务支撑流式识别或实时字幕。+- 希望先用 CPU 做可复现 smoke test,再迁移到 GPU 服务。++如果你更需要完全托管服务、厂商 SLA,或你的自有评测显示目标语言/领域还不够好,可以暂时保留现有方案。++## 快速评估计划++1. 选择 20-50 条有代表性的音频,覆盖短音频、长录音、噪声、多说话人、目标语言和方言。+2. 按生产方式运行当前 Whisper 或云端 ASR,保存转写结果、延迟、成本和失败样例。+3. 用 README 快速开始跑 FunASR,再根据 [部署选型表](./deployment_matrix_zh.md) 选择服务路径。+4. 用人工审阅或现有 WER/CER 流程比较结果,不要只看一个干净 demo 音频。+5. 如果应用已经使用 OpenAI 风格客户端,运行 OpenAI 兼容 API smoke test。+6. 分开记录 warmup、模型下载、设备、GPU/CPU 型号、batch size、音频时长和稳定吞吐。++## 功能映射++| 现有流程 | FunASR 路径 | 需要验证什么 |+|---|---|---|+| Whisper 文件转写 | 使用 SenseVoice、Paraformer 或 Fun-ASR-Nano 的 [README 快速开始](../README_zh.md#快速开始) | 转写质量、时间戳、速度、模型下载、CPU/GPU 行为。 |+| Whisper + pyannote | VAD、标点和 `spk_model="cam++"` | 说话人标签、换人位置、重叠说话、长静音。 |+| OpenAI 音频 API 或云端批量 ASR | [OpenAI 兼容 API 示例](../examples/openai_api/) | `/v1/audio/transcriptions`、响应格式、客户端兼容性、上传限制。 |+| Dify/LangChain/AutoGen Agent 音频 | [客户端配方](../examples/openai_api/CLIENTS.md) 或 [MCP 服务](../examples/mcp_server/) | 工具延迟、文件处理、鉴权边界、错误返回。 |+| 实时字幕或客服流式识别 | [Runtime 服务文档](../runtime/readme_cn.md) | 分块、断句、重连、背压、中间/最终结果行为。 |+| 字幕生成 | [字幕示例](../examples/subtitle/) | 分段可读性、行长、说话人标签、SRT/VTT 兼容性。 |+| 离线归档处理 | [批处理示例](../examples/batch_asr_improved.py) | manifest、重试、进度日志、吞吐、失败文件恢复。 |++## 最小本地对比++安装 FunASR,并用你基线评测里的同一条音频运行:++```bash+pip install funasr+```++```python+from funasr import AutoModel++model = AutoModel(+ model="iic/SenseVoiceSmall",+ vad_model="fsmn-vad",+ spk_model="cam++",+ device="cuda", # 便携 smoke test 可改成 "cpu"+)+result = model.generate(input="sample.wav")+print(result)+```++如果要按 API 服务方式对比:++```bash+pip install funasr fastapi uvicorn python-multipart+funasr-server --model sensevoice --device cuda++curl http://localhost:8000/v1/audio/transcriptions \+ -F [email protected] \+ -F model=sensevoice \+ -F response_format=verbose_json+```++如果需要可复现的容器 smoke test,可以从 `examples/openai_api/docker-compose.yml` 开始,并用 `BASE_URL=http://localhost:8000 bash examples/openai_api/smoke_test.sh` 验证。++## 质量与速度检查清单++对旧流水线和 FunASR 都记录这些字段:++- 音频时长、语言、领域、采样率、声道数和说话人数。+- 模型名、模型版本、FunASR 版本、Python/PyTorch/CUDA 版本,以及 Docker 镜像 tag。+- 硬件、设备模式、batch size、流式 chunk size,以及是否排除 warmup/模型下载时间。+- WER/CER 或人工审阅记录:姓名、数字、标点、说话人分离、时间戳、领域词。+- 延迟、吞吐、GPU/CPU 内存、每小时音频成本、失败文件比例。+- 运维要求:鉴权、上传限制、TLS、日志、监控、重试和数据留存规则。++## 上线检查清单++- 在代表性评测通过前,保留旧流水线作为回退。+- 先做内部 endpoint 或离线批处理,再对外暴露 API。+- 为每个请求记录 request id、音频时长、模型、设备、延迟和错误类型。+- 在 runbook 中固定模型别名和部署命令。+- 测试噪声、静音、多人重叠、长文件、非 UTF-8 文件名和网络中断。+- 遇到阻塞时,通过 [Deployment Help issue](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) 提交命令、日志、模型、设备和样本特征。++## 分享迁移结果++如果 FunASR 替代或补充了你的现有 ASR 栈,欢迎开一个 [showcase issue](https://github.com/modelscope/FunASR/issues/new?template=showcase.md)。包含硬件、速度、质量记录和部署细节的迁移报告,能帮助新用户更快选型。diff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.mdindex 0cb1dae92..0da372e0b 100644--- a/docs/use_case_showcase.md+++ b/docs/use_case_showcase.md@@ -8,6 +8,7 @@ FunASR is useful far beyond a single offline transcription command. This page co|---|---|---|| Transcribe one file locally | [README quick start](../README.md#quick-start) | Verify install and model download in minutes. || Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. |+| Migrate from Whisper/cloud ASR | [Migration guide](./migration_from_whisper.md) | Map existing pipelines to FunASR, benchmark representative audio, and plan a safe rollout. || Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. || Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. || Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, WebSocket, vLLM, MCP, batch, subtitles, and Triton. |@@ -60,6 +61,7 @@ Use this path when partial results and low perceived latency matter more than aUse this path when deciding whether FunASR is a good replacement for Whisper or a cloud ASR provider.+- Follow the [migration guide](./migration_from_whisper.md) to map features and benchmark representative audio.- Read the [public benchmark report](https://modelscope.github.io/FunASR/benchmark.html).- Benchmark your own sample set before migration; include both short clips and long-form recordings.- Track cost and throughput together: GPU speed, CPU viability, model download size, and deployment complexity.diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.mdindex 84264b7d7..c44c0ba67 100644--- a/docs/use_case_showcase_zh.md+++ b/docs/use_case_showcase_zh.md@@ -8,6 +8,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部|---|---|---|| 本地转写一个文件 | [README 快速开始](../README_zh.md#快速开始) | 几分钟内验证安装、模型下载和首次推理。 || 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 |+| 从 Whisper/云端 ASR 迁移 | [迁移指南](./migration_from_whisper_zh.md) | 将现有流水线映射到 FunASR,用代表性音频评测并规划安全上线。 || 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 || 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 || 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 |@@ -60,6 +61,7 @@ curl http://localhost:8000/v1/audio/transcriptions \当你在评估是否用 FunASR 替代 Whisper 或云端 ASR 时,使用这个路径。+- 按 [迁移指南](./migration_from_whisper_zh.md) 映射功能并评测代表性音频。- 阅读 [公开性能评测](https://modelscope.github.io/FunASR/zh/benchmark.html)。- 用自己的样本集再测一次;同时包含短音频和长音频。- 同时记录成本和吞吐:GPU 速度、CPU 可用性、模型下载体积、部署复杂度。