Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs: add OpenAI API workflow recipes
zhifu gaofa2e1a7f80f7
diff --git a/README.md b/README.mdindex e176b3374..28f39eaae 100644--- a/README.md+++ b/README.md@@ -198,7 +198,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/README_zh.md b/README_zh.mdindex 795eab3a6..7e3501b37 100644--- a/README_zh.md+++ b/README_zh.md@@ -197,7 +197,7 @@ curl http://localhost:8000/v1/audio/transcriptions \docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12```-[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)+[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)---diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.mdindex 83f19667c..0074bc6fa 100644--- a/docs/deployment_matrix.md+++ b/docs/deployment_matrix.md@@ -24,7 +24,7 @@ Use the Python API from the README. It is the shortest route for validating inst### I want a local replacement for cloud transcription-Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md).+Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md). For Dify, n8n, HTTP nodes, or webhook workers, follow the [workflow recipes](../examples/openai_api/WORKFLOWS.md).### I want a repeatable container demodiff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.mdindex 613f56e69..8180dc3ab 100644--- a/docs/deployment_matrix_zh.md+++ b/docs/deployment_matrix_zh.md@@ -24,7 +24,7 @@### 我想替代云端转写服务-使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions`、`/v1/models`、`/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh`,再根据 [客户端配方](../examples/openai_api/CLIENTS.md) 接入 SDK 或 HTTP 客户端。+使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions`、`/v1/models`、`/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh`,再根据 [客户端配方](../examples/openai_api/CLIENTS.md) 接入 SDK 或 HTTP 客户端。Dify、n8n、HTTP 节点或 webhook worker 可参考 [工作流配方](../examples/openai_api/WORKFLOWS.md)。### 我想要可复现的容器 demodiff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.mdindex 4bef32251..029439b47 100644--- a/docs/use_case_showcase.md+++ b/docs/use_case_showcase.md@@ -9,7 +9,7 @@ FunASR is useful far beyond a single offline transcription command. This page co| Transcribe one file locally | [README quick start](../README.md#quick-start) | Verify install and model download in minutes. || Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. || Migrate from Whisper/cloud ASR | [Migration guide](./migration_from_whisper.md) | Map existing pipelines to FunASR, benchmark representative audio, and plan a safe rollout. |-| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. |+| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/), [client recipes](../examples/openai_api/CLIENTS.md), and [workflow recipes](../examples/openai_api/WORKFLOWS.md) | Reuse LangChain, Dify, n8n, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. || Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. || Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, WebSocket, vLLM, MCP, batch, subtitles, and Triton. || Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. |diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.mdindex 9d502423a..8fd4f94b5 100644--- a/docs/use_case_showcase_zh.md+++ b/docs/use_case_showcase_zh.md@@ -9,7 +9,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部| 本地转写一个文件 | [README 快速开始](../README_zh.md#快速开始) | 几分钟内验证安装、模型下载和首次推理。 || 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 || 从 Whisper/云端 ASR 迁移 | [迁移指南](./migration_from_whisper_zh.md) | 将现有流水线映射到 FunASR,用代表性音频评测并规划安全上线。 |-| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 |+| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/)、[客户端配方](../examples/openai_api/CLIENTS.md) 和 [工作流配方](../examples/openai_api/WORKFLOWS.md) | 复用 LangChain、Dify、n8n、AutoGen 等 OpenAI 风格客户端,音频不出内网。 || 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 || 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 || 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 |diff --git a/examples/openai_api/CLIENTS.md b/examples/openai_api/CLIENTS.mdindex 62b98f11e..85c32dc6e 100644--- a/examples/openai_api/CLIENTS.md+++ b/examples/openai_api/CLIENTS.md@@ -1,6 +1,6 @@# Client Recipes for the FunASR OpenAI-Compatible API-Use this page when `funasr-server` is already running and you want to connect an existing application, agent tool, or workflow engine to local speech recognition.+Use this page when `funasr-server` is already running and you want to connect an existing application, agent tool, or workflow engine to local speech recognition. For Dify, n8n, HTTP nodes, and webhook workers, see the [workflow recipes](WORKFLOWS.md).## Preflight@@ -96,7 +96,7 @@ Use a multipart HTTP node or custom tool:| Text fields | `model=sensevoice`, `response_format=verbose_json` || Result path | `text` for transcript, `segments` for timestamps/speakers |-When the workflow system cannot send files directly, upload audio to an internal object store first, then run a small worker that downloads the object and calls FunASR with the `requests` recipe above.+When the workflow system cannot send files directly, upload audio to an internal object store first, then run a small worker that downloads the object and calls FunASR with the `requests` recipe above. See [workflow recipes](WORKFLOWS.md) for Dify, n8n, and webhook-worker patterns.## Response formatsdiff --git a/examples/openai_api/README.md b/examples/openai_api/README.mdindex 08077f504..33f1304b4 100644--- a/examples/openai_api/README.md+++ b/examples/openai_api/README.md@@ -11,7 +11,7 @@ python server.py --model sensevoice --device cuda --port 8000Server starts in ~20s (model loading). Health check: `GET /health`-Need copy-paste integration snippets for Python SDK, HTTP clients, agent tools, or Dify-style workflows? See [Client recipes](CLIENTS.md).+Need copy-paste integration snippets for Python SDK, HTTP clients, agent tools, or Dify/n8n-style workflows? See [Client recipes](CLIENTS.md) and [workflow recipes](WORKFLOWS.md).### End-to-end smoke test@@ -89,7 +89,7 @@ curl http://localhost:8000/v1/audio/transcriptions \## Agent Framework Integration-Works with: **LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, or any framework using OpenAI audio API. See [Client recipes](CLIENTS.md) for copy-paste SDK, HTTP, and agent-tool patterns.+Works with: **LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, **n8n**, or any framework using OpenAI audio API. See [Client recipes](CLIENTS.md) for SDK and agent-tool patterns, plus [workflow recipes](WORKFLOWS.md) for low-code HTTP nodes and webhook workers.### LangChain Example```pythondiff --git a/examples/openai_api/WORKFLOWS.md b/examples/openai_api/WORKFLOWS.mdnew file mode 100644index 000000000..8bb0c03dc--- /dev/null+++ b/examples/openai_api/WORKFLOWS.md@@ -0,0 +1,160 @@+# Low-code Workflow Recipes for the FunASR OpenAI-Compatible API++Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. Start with the local smoke test in this directory, then replace `localhost` with the reachable service name inside your network.++## Server preflight++```bash+cd examples/openai_api+python server.py --model sensevoice --device cuda --port 8000+```++From the workflow host or container:++```bash+export FUNASR_BASE_URL=http://<funasr-host>:8000+curl -fsS "$FUNASR_BASE_URL/health"+curl -fsS "$FUNASR_BASE_URL/v1/models"+```++If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. Use a Docker Compose service name, Kubernetes service name, or LAN host name instead.++## Multipart HTTP request++Every workflow engine eventually needs to send this request shape:++| Field | Value |+|---|---|+| Method | `POST` |+| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |+| Body type | `multipart/form-data` |+| File field | `file` |+| Text field | `model=sensevoice` |+| Text field | `response_format=verbose_json` |+| Timeout | Set according to maximum audio duration, for example 300 seconds for long files. |++Equivalent curl command:++```bash+curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \+ -F [email protected] \+ -F model=sensevoice \+ -F response_format=verbose_json+```++Typical JSON fields to map downstream:++| Path | Use |+|---|---|+| `text` | Plain transcript for a chatbot, ticket, or knowledge-base step. |+| `segments` | Timestamps and speaker labels when `verbose_json` is requested. |+| `duration` | Audio processing time reported by the API, useful for logs. |+| `model` | Model alias used for the request. |++## Dify custom tool or HTTP node++Use this pattern when a Dify application receives an uploaded audio file or a URL to internal audio storage.++### Direct file upload path++Configure an HTTP request node or custom tool with:++- Method: `POST`+- URL: `http://<funasr-host>:8000/v1/audio/transcriptions`+- Body: `multipart/form-data`+- File part: `file`, bound to the uploaded audio variable+- Text parts: `model=sensevoice`, `response_format=verbose_json`+- Output variable: map `text` as the transcript, and keep `segments` when timestamps or speaker labels matter++### Audio URL path++Some workflow tools pass a file URL rather than raw multipart bytes. In that case, add a small internal worker:++1. Dify sends the audio URL and metadata to the worker.+2. The worker downloads the file from trusted storage.+3. The worker posts multipart data to FunASR.+4. The worker returns `text`, `segments`, and operational logs to Dify.++```python+import requests++FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions"++def transcribe_from_url(audio_url: str) -> dict:+ audio_response = requests.get(audio_url, timeout=120)+ audio_response.raise_for_status()+ files = {"file": ("audio.wav", audio_response.content, "audio/wav")}+ data = {"model": "sensevoice", "response_format": "verbose_json"}+ response = requests.post(FUNASR_URL, files=files, data=data, timeout=300)+ response.raise_for_status()+ return response.json()+```++Keep this worker inside your trusted network and validate allowed URL domains before downloading user-provided links.++## n8n HTTP Request node++A common n8n flow is: trigger -> binary audio data -> HTTP Request -> transcript consumer.++Recommended HTTP Request settings:++| n8n setting | Value |+|---|---|+| Method | `POST` |+| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |+| Send Body | enabled |+| Body Content Type | `Form-Data` / multipart |+| Binary file field | `file` |+| Additional form fields | `model=sensevoice`, `response_format=verbose_json` |+| Response Format | JSON |+| Timeout | Increase for long recordings. |++After the request, use `{{$json.text}}` as the transcript. If `verbose_json` is enabled, route `{{$json.segments}}` to subtitle, speaker, or QA steps.++## Webhook worker pattern++Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing.++```python+from pathlib import Path+import tempfile+import requests++FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions"++def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict:+ with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp:+ tmp.write(payload)+ tmp.flush()+ with open(tmp.name, "rb") as audio:+ response = requests.post(+ FUNASR_URL,+ files={"file": (filename, audio, content_type)},+ data={"model": "sensevoice", "response_format": "verbose_json"},+ timeout=300,+ )+ response.raise_for_status()+ return response.json()+```++This worker is also the right place to add audio conversion, file-size checks, request IDs, authentication, and retries.++## Production guardrails++- Put authentication, TLS, upload-size limits, and rate limits in front of FunASR before sharing it across teams.+- Use `/health` for workflow readiness checks and `/v1/models` to validate model aliases.+- Log request id, audio duration, model alias, response format, device, latency, and error type.+- Set workflow timeouts according to maximum audio duration; split very long recordings before sending them through low-code tools.+- Keep private audio in trusted storage and avoid putting signed URLs, credentials, or transcripts into public logs.+- Run the same workflow with at least one public smoke sample and one realistic private sample before production use.++## Troubleshooting++| Symptom | Fix |+|---|---|+| Workflow can call `/health` but transcription fails | Confirm the request is `multipart/form-data` and the binary field is named `file`. |+| `localhost` connection fails from Dify or n8n | Use the host, Compose service, or Kubernetes service reachable from the workflow runtime. |+| Response has no `segments` | Set `response_format=verbose_json`. |+| Requests time out | Increase HTTP timeout or split long recordings. |+| First request is slow | Preload the model with `--model sensevoice` and use `/health` as a readiness check. |+| Unknown model alias | Call `/v1/models` and use one of the returned aliases. |