Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs: add deployment decision matrix
zhifu gao200998ab6d5b
diff --git a/README.md b/README.mdindex 6d3c646f2..15ae1e03a 100644--- a/README.md+++ b/README.md@@ -196,7 +196,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/README_zh.md b/README_zh.mdindex cb11511c8..782a46f27 100644--- a/README_zh.md+++ b/README_zh.md@@ -195,7 +195,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.mdnew file mode 100644index 000000000..83f19667c--- /dev/null+++ b/docs/deployment_matrix.md@@ -0,0 +1,61 @@+# FunASR Deployment Matrix++Use this page to choose the shortest deployment path for a product, demo, benchmark, or internal workflow. Start with the smallest surface that satisfies the job, then move to heavier runtimes only when throughput, latency, or integration requirements demand it.++## Quick decision table++| Path | Best for | Start here | Operational notes |+|---|---|---|---|+| Python API | Notebooks, offline jobs, first model evaluation | [README quick start](../README.md#quick-start) | Lowest ceremony; caller owns batching, retries, and files. |+| OpenAI-compatible API | Private speech API, agents, Dify/LangChain/AutoGen-style clients | [OpenAI API example](../examples/openai_api/) | Easiest integration for apps that already support OpenAI audio APIs. |+| Docker Compose API | Reproducible local smoke test or small internal service | [OpenAI API Docker docs](../examples/openai_api/#docker-deployment) | CPU by default; adapt the image before using CUDA in containers. |+| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |+| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |+| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |+| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |+| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |+| Triton runtime | Specialized high-performance serving | [Triton runtime docs](../runtime/triton_gpu/README.md) | Heavier setup; choose when your team already operates Triton/GPU serving. |++## Common choices++### I want to try FunASR in five minutes++Use the Python API from the README. It is the shortest route for validating installation, model download, device selection, and basic output shape.++### I want a local replacement for cloud transcription++Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md).++### I want a repeatable container demo++Use `examples/openai_api/docker-compose.yml` for a CPU-mode smoke test:++```bash+cd examples/openai_api+cp .env.example .env+docker compose up --build+```++Keep CPU mode until you have a CUDA-capable PyTorch/FunASR image. After that, set `FUNASR_DEVICE=cuda` and verify with the same smoke test.++### I need streaming or live captioning++Use the runtime WebSocket service. Validate chunk size, VAD, endpointing, punctuation, speaker diarization, reconnect behavior, and client backpressure with real audio before production rollout.++### I need more LLM-based ASR throughput++Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution and watch GPU memory, tensor parallel size, first-token latency, and warmup time.++## Readiness checklist++- Pick a model alias and pin it in deployment notes.+- Record FunASR version, model version, device, CUDA/PyTorch version, Docker image tag, and command line.+- Run a short public smoke sample and at least one realistic private sample.+- Log audio duration, model, device, latency, response format, and error type for every request.+- Add upload-size limits, authentication, TLS, and rate limits before exposing an API outside a trusted network.+- For streaming, test silence, noise, overlapping speakers, long sessions, reconnects, and slow clients.+- For benchmark claims, include input duration, hardware, batch size, model, runtime path, and whether model download/warmup time is excluded.++## When to open an issue++Use [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) for runtime, Docker, vLLM, Triton, Android, browser, or agent integration problems. Include your deployment path, exact command/config, logs, model, device, and audio characteristics.diff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.mdnew file mode 100644index 000000000..613f56e69--- /dev/null+++ b/docs/deployment_matrix_zh.md@@ -0,0 +1,61 @@+# FunASR 部署选型表++这个页面帮助你为产品、demo、benchmark 或内部工作流选择最短部署路径。先选择能满足目标的最小方案,只有在吞吐、延迟或集成方式有明确要求时,再切换到更重的运行时。++## 快速决策表++| 路径 | 适合场景 | 从这里开始 | 运维提示 |+|---|---|---|---|+| Python API | Notebook、离线任务、首次模型评测 | [README 快速开始](../README_zh.md#快速开始) | 最简单;调用方自己负责批处理、重试和文件管理。 |+| OpenAI 兼容 API | 私有语音 API、Agent、Dify/LangChain/AutoGen 风格客户端 | [OpenAI API 示例](../examples/openai_api/) | 已支持 OpenAI audio API 的应用最容易接入。 |+| Docker Compose API | 可复现本地 smoke test 或小型内部服务 | [OpenAI API Docker 文档](../examples/openai_api/#docker-deployment) | 默认 CPU;容器里使用 CUDA 前需要先适配 CUDA-capable 镜像。 |+| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |+| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 |+| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |+| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |+| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |+| Triton Runtime | 专门的高性能推理服务 | [Triton 文档](../runtime/triton_gpu/README.md) | 配置更重;适合已经在运维 Triton/GPU serving 的团队。 |++## 常见选择++### 我想五分钟内试跑 FunASR++使用 README 里的 Python API。它是验证安装、模型下载、设备选择和基础输出格式的最短路径。++### 我想替代云端转写服务++使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions`、`/v1/models`、`/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh`,再根据 [客户端配方](../examples/openai_api/CLIENTS.md) 接入 SDK 或 HTTP 客户端。++### 我想要可复现的容器 demo++使用 `examples/openai_api/docker-compose.yml` 跑 CPU smoke test:++```bash+cd examples/openai_api+cp .env.example .env+docker compose up --build+```++在没有 CUDA-capable PyTorch/FunASR 镜像前保持 CPU 模式。准备好 CUDA 镜像后,再设置 `FUNASR_DEVICE=cuda` 并用同一个 smoke test 验证。++### 我需要流式识别或实时字幕++使用 Runtime WebSocket 服务。上线前请用真实音频验证 chunk size、VAD、断句、标点、说话人分离、重连行为和客户端背压。++### 我需要更高的 LLM-based ASR 吞吐++Fun-ASR-Nano 走 vLLM 路径。请用自己的音频分布做 benchmark,并关注 GPU 显存、tensor parallel size、首 token 延迟和 warmup 时间。++## 上线检查清单++- 选择模型 alias,并写入部署说明。+- 记录 FunASR 版本、模型版本、设备、CUDA/PyTorch 版本、Docker 镜像 tag 和启动命令。+- 跑一个公开短音频 smoke sample,再跑至少一个真实私有样本。+- 每次请求记录音频时长、模型、设备、延迟、响应格式和错误类型。+- API 暴露到可信网络外之前,增加上传大小限制、鉴权、TLS 和限流。+- 流式场景需要测试静音、噪声、多人重叠、长连接、重连和慢客户端。+- 发布 benchmark 结论时,说明输入时长、硬件、batch size、模型、运行路径,以及是否排除模型下载和 warmup 时间。++## 什么时候开 issue++Runtime、Docker、vLLM、Triton、Android、浏览器或 Agent 集成问题,请使用 [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md)。请附上部署路径、完整命令/config、日志、模型、设备和音频特征。diff --git a/docs/index.rst b/docs/index.rstindex a80518c1a..fdbe8dc7d 100644--- a/docs/index.rst+++ b/docs/index.rst@@ -71,6 +71,8 @@ Overview:maxdepth: 1:caption: Runtime and Service+ ./deployment_matrix.md+ ./deployment_matrix_zh.md./runtime/readme.md./runtime/docs/SDK_tutorial_online.md./runtime/docs/SDK_tutorial.mddiff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.mdindex 8ff30c27a..0cb1dae92 100644--- a/docs/use_case_showcase.md+++ b/docs/use_case_showcase.md@@ -10,6 +10,7 @@ FunASR is useful far beyond a single offline transcription command. This page co| Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. || Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. || Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. |+| Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, WebSocket, vLLM, MCP, batch, subtitles, and Triton. || Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. || Accelerate LLM-based ASR | [vLLM guide](./vllm_guide.md) | Use tensor parallel decoding and streaming service support for Fun-ASR-Nano. || Generate subtitles | [Subtitle example](../examples/subtitle/) | Turn long audio or video into subtitle files for media workflows. |diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.mdindex d111a0cc7..84264b7d7 100644--- a/docs/use_case_showcase_zh.md+++ b/docs/use_case_showcase_zh.md@@ -10,6 +10,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部| 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 || 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 || 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 |+| 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 || 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 || 加速 LLM-based ASR | [vLLM 指南](./vllm_guide.md) | 为 Fun-ASR-Nano 使用 tensor parallel 解码和流式服务能力。 || 生成字幕 | [字幕示例](../examples/subtitle/) | 将长音频或视频转成字幕文件。 |