Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs: add OpenAI API client recipes
zhifu gao8877ef704248
diff --git a/README.md b/README.mdindex 12f3e1b99..2f623f67a 100644--- a/README.md+++ b/README.md@@ -196,7 +196,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API example →](./examples/openai_api/) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/README_zh.md b/README_zh.mdindex c20892ca2..e5ee25047 100644--- a/README_zh.md+++ b/README_zh.md@@ -195,7 +195,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API 示例 →](./examples/openai_api/) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.mdindex 1974094c1..6ba5f2e0e 100644--- a/docs/use_case_showcase.md+++ b/docs/use_case_showcase.md@@ -8,7 +8,7 @@ FunASR is useful far beyond a single offline transcription command. This page co|---|---|---|| Transcribe one file locally | [README quick start](../README.md#quick-start) | Verify install and model download in minutes. || Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. |-| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. |+| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. || Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. || Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. || Accelerate LLM-based ASR | [vLLM guide](./vllm_guide.md) | Use tensor parallel decoding and streaming service support for Fun-ASR-Nano. |diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.mdindex 9568b4144..75a76de4a 100644--- a/docs/use_case_showcase_zh.md+++ b/docs/use_case_showcase_zh.md@@ -8,7 +8,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部|---|---|---|| 本地转写一个文件 | [README 快速开始](../README_zh.md#快速开始) | 几分钟内验证安装、模型下载和首次推理。 || 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 |-| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 |+| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 || 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 || 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 || 加速 LLM-based ASR | [vLLM 指南](./vllm_guide.md) | 为 Fun-ASR-Nano 使用 tensor parallel 解码和流式服务能力。 |diff --git a/examples/openai_api/CLIENTS.md b/examples/openai_api/CLIENTS.mdnew file mode 100644index 000000000..62b98f11e--- /dev/null+++ b/examples/openai_api/CLIENTS.md@@ -0,0 +1,141 @@+# Client Recipes for the FunASR OpenAI-Compatible API++Use this page when `funasr-server` is already running and you want to connect an existing application, agent tool, or workflow engine to local speech recognition.++## Preflight++```bash+export BASE_URL=http://localhost:8000+curl -fsS "$BASE_URL/health"+curl -fsS "$BASE_URL/v1/models"+```++If the server is on another machine, replace `localhost` with the reachable host name or service address. Keep `/v1` in SDK base URLs, and omit `/v1` for direct endpoint checks like `/health`.++## Model aliases++| Alias | Good first use | Notes |+|---|---|---|+| `sensevoice` | Private multilingual API | Fast default with language, emotion, and event tags. |+| `paraformer` | Mandarin production transcription | Includes VAD and punctuation. |+| `paraformer-en` | English transcription | Smaller English-only route. |+| `fun-asr-nano` | LLM-based ASR experiments | Pair with vLLM for higher throughput deployments. |++## Python OpenAI SDK++```python+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")++with open("meeting.wav", "rb") as audio:+ result = client.audio.transcriptions.create(+ model="sensevoice",+ file=audio,+ response_format="verbose_json",+ )++print(result.text)+for segment in getattr(result, "segments", []):+ print(segment)+```++Most OpenAI SDKs require an API key value even when the local FunASR server does not check it. Use any placeholder for local development, then add real authentication at your gateway if the service is shared.++## Plain Python requests++```python+import requests++with open("meeting.wav", "rb") as audio:+ response = requests.post(+ "http://localhost:8000/v1/audio/transcriptions",+ files={"file": ("meeting.wav", audio, "audio/wav")},+ data={"model": "sensevoice", "response_format": "verbose_json"},+ timeout=300,+ )+response.raise_for_status()+print(response.json()["text"])+```++This is the most portable pattern for internal services, queues, notebooks, and low-code tools that can issue multipart HTTP requests.++## Agent tool pattern++Expose transcription as a regular tool function. The agent does not need to know FunASR internals; it only needs a file path or uploaded audio object.++```python+from pathlib import Path+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")++def transcribe_audio(audio_path: str) -> str:+ """Transcribe a local audio file with FunASR and return plain text."""+ path = Path(audio_path)+ with path.open("rb") as audio:+ result = client.audio.transcriptions.create(+ model="sensevoice",+ file=audio,+ )+ return result.text+```++For LangChain, LlamaIndex, AutoGen, CrewAI, Semantic Kernel, and similar frameworks, register the function above using that framework's normal tool or function-calling mechanism.++## Dify, workflow engines, and HTTP nodes++Use a multipart HTTP node or custom tool:++| Setting | Value |+|---|---|+| Method | `POST` |+| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |+| Body type | `multipart/form-data` |+| File field | `file` |+| Text fields | `model=sensevoice`, `response_format=verbose_json` |+| Result path | `text` for transcript, `segments` for timestamps/speakers |++When the workflow system cannot send files directly, upload audio to an internal object store first, then run a small worker that downloads the object and calls FunASR with the `requests` recipe above.++## Response formats++`response_format=json` returns a compact response:++```json+{"text": "recognized speech"}+```++`response_format=verbose_json` adds operational fields useful for agents and subtitles:++```json+{+ "text": "recognized speech",+ "segments": [+ {"start": 0.0, "end": 3.2, "text": "recognized speech", "speaker": 0}+ ],+ "language": "auto",+ "duration": 0.42,+ "model": "sensevoice"+}+```++## Production checklist++- Put TLS, authentication, rate limits, and upload-size limits in front of the service before exposing it outside a trusted network.+- Preload the default model at startup and use `/health` for readiness checks.+- Set client timeouts based on maximum audio duration; long recordings need longer HTTP timeouts.+- Log audio duration, model alias, device, latency, response format, and error type for every request.+- Pin model aliases and deployment images in production notes so benchmark results remain reproducible.+- For GPU hosts, keep one worker per GPU until you have measured memory headroom and concurrency behavior.++## Troubleshooting quick checks++| Symptom | Check |+|---|---|+| SDK says authentication is missing | Pass any placeholder `api_key` for local development. |+| 400 unknown model | Call `/v1/models` and use one of the listed aliases. |+| Request times out | Increase client timeout or split very long recordings. |+| First request is slow | The model may be loading; preload with `--model sensevoice`. |+| CUDA is unavailable | Start with `--device cpu` to verify the API path, then fix GPU drivers/runtime. |+| Port conflict | Start with `--port 9000` and set `BASE_URL=http://localhost:9000`. |diff --git a/examples/openai_api/README.md b/examples/openai_api/README.mdindex b9160b274..49e3a6555 100644--- a/examples/openai_api/README.md+++ b/examples/openai_api/README.md@@ -11,6 +11,8 @@ python server.py --model sensevoice --device cuda --port 8000Server starts in ~20s (model loading). Health check: `GET /health`+Need copy-paste integration snippets for Python SDK, HTTP clients, agent tools, or Dify-style workflows? See [Client recipes](CLIENTS.md).+### End-to-end smoke testIn another terminal, download a public sample and verify both health and transcription:@@ -87,7 +89,7 @@ curl http://localhost:8000/v1/audio/transcriptions \## Agent Framework Integration-Works with: **LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, or any framework using OpenAI audio API.+Works with: **LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, or any framework using OpenAI audio API. See [Client recipes](CLIENTS.md) for copy-paste SDK, HTTP, and agent-tool patterns.### LangChain Example```python