Skip to content

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

docs: maintain bilingual historical benchmark sources and routes

zhifu gaoe7adf031a2ae
diff --git a/docs/README.md b/docs/README.md
index 292ea9834..44c2c7236 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -51,6 +51,7 @@ interfaces. Choose the matching protocol before selecting a client.
## Evaluate and Operate
+- [Historical ASR benchmark](benchmark/historical_asr.md) / [中文](benchmark/historical_asr_zh.md): preserved results and provenance limits
- [Performance methodology](benchmark/rtf_reproducibility.md) and [realtime WebSocket benchmarks](benchmark/realtime_ws_benchmark.md)
- [Troubleshooting](troubleshooting.md) / [中文](troubleshooting_zh.md)
- [Use cases](use_case_showcase.md), [community integrations](community_projects.md), and [repository responsibilities](repository_roles.md)
diff --git a/docs/benchmark/historical_asr.md b/docs/benchmark/historical_asr.md
new file mode 100644
index 000000000..492c3a8e5
--- /dev/null
+++ b/docs/benchmark/historical_asr.md
@@ -0,0 +1,113 @@
+# Historical ASR Benchmark
+
+[中文](historical_asr_zh.md)
+
+This is a historical record with incomplete provenance, retained for readers of
+earlier FunASR comparisons. It is not a new measurement, a universal leaderboard,
+or a guarantee for a current checkpoint, machine or deployment. For a new
+evaluation, start with the [performance methodology](rtf_reproducibility.md).
+
+## Historical Summary
+
+The table below preserves the original report's wording and numbers. Its
+"best" labels refer only to that report, not to all available models or hardware.
+
+| Metric | Result |
+| --- | --- |
+| Dataset | 184 long-form Chinese audio files, 11,539 s total, 192.3 min. |
+| GPU | NVIDIA H100 80GB HBM3. |
+| Best GPU speed | SenseVoice-Small: 169.6x realtime in the full benchmark, 211.8x in the initial run. |
+| Best CPU speed | SenseVoice-Small: 17.2x realtime; Paraformer-Large: 15.6x realtime. |
+| Baseline | OpenAI Whisper-large-v3: 13.4x realtime on GPU. |
+
+The **169.6x full run and 211.8x initial run are separate reported results**.
+The original page does not disclose the measurement date. The source was
+checked on **2026-09-07**, which is a snapshot audit date, not the measurement date.
+
+## Historical Results
+
+All values and notes in this table are archived claims from the original report.
+The notes are not current API capability guarantees. In particular, a model's
+raw tags do not imply that an HTTP endpoint returns those tags, and an old
+timestamp limitation must not replace the current [model selection guide](../model_selection.md).
+
+| Model | Device | RTF | Speed | CER | Notes |
+| --- | --- | --- | --- | --- | --- |
+| SenseVoice-Small | GPU | 0.005896 | 169.6x | 7.81% | ASR + language / emotion / event tags; CER after tag stripping. |
+| Paraformer-Large | GPU | 0.008359 | 119.6x | 10.18% | Fast non-autoregressive Chinese ASR with VAD/punctuation pipeline. |
+| Fun-ASR-Nano | GPU | 0.058803 | 17.0x | 8.06% | LLM-based ASR for Chinese, English, Japanese, seven Chinese dialect groups, and 26 regional accents; supports hotwords. Reliable checkpoint-native timestamps are not available ([#106](https://github.com/QwenAudio/Fun-ASR/issues/106)). |
+| GLM-ASR-Nano | GPU | 0.026974 | 37.1x | 31.07% | LLM-based multilingual ASR. |
+| Whisper-large-v3-turbo (OpenAI) | GPU | 0.021708 | 46.1x | 21.71% | OpenAI Whisper implementation. |
+| Whisper-large-v3 (OpenAI) | GPU | 0.074694 | 13.4x | 20.02% | Baseline for large Whisper quality. |
+| SenseVoice-Small | CPU | 0.057988 | 17.2x | 7.81% | CPU run from the remaining benchmark script. |
+| Paraformer-Large | CPU | 0.064056 | 15.6x | 10.18% | CPU viable for batch jobs. |
+| Fun-ASR-Nano | CPU | 0.274318 | 3.6x | 8.06% | LLM-based model is heavier but still above realtime. |
+
+The repeated CPU/GPU CER values do not establish independent per-device scoring.
+The raw predictions, references and scoring program are not available in the
+audited record. The tag-stripping statement is preserved as a historical claim,
+not as newly verified scorer output. Numerical precision and rounded speed/RTF
+pairs are retained without recalculation.
+
+## Provenance and Limitations
+
+The [original English HTML](https://github.com/modelscope/FunASR/blob/67d63b80a246dc33749e43904c294e0409cd9183/benchmark.html)
+is pinned to its historical GitHub Pages commit. Its bytes matched the archived
+public page during the source audit. That establishes the source of this table,
+not the correctness or reproducibility of the underlying measurements.
+
+The old report describes RTF as total inference time divided by total audio
+duration, and speed as its reciprocal. The latter is also called RTFx:
+
+```text
+RTF = total inference time / total audio duration
+RTFx = total audio duration / total inference time = 1 / RTF
+```
+
+The following commands are **historical text and cannot be run directly from
+the audited checkout**. All three referenced files were absent at FunASR source
+revision `386f6f9106684ba5a114e796147db4396a09eab5`; no replacement scripts or
+reproduction data are supplied by this document.
+
+```text
+python benchmark/run_full_benchmark.py
+python benchmark/run_remaining.py
+python benchmark/fix_sensevoice_cer.py
+```
+
+The audited report does not provide a CPU model/thread count, dataset membership
+and reference manifest, exact checkpoint revisions, software/driver versions,
+per-file predictions or timing logs, or a complete timing scope covering warmup,
+I/O and preprocessing. Without these materials, the old table is not directly
+reproducible and its CPU-versus-GPU headline is not a production-wide guarantee.
+
+This record's **11,539 seconds** must remain separate from the **11,541 seconds**
+reported in the [vLLM methodology](rtf_reproducibility.md). Both mention 184 files,
+but that alone does not prove identical dataset membership. Do not merge their
+rows or silently normalize the two-second discrepancy.
+
+## Choosing a Current Path
+
+The following is the **original recommendation table, retained as historical
+context only**. It is not a newly validated recommendation or performance ranking.
+
+| Need | Recommended model |
+| --- | --- |
+| Fastest production transcription | SenseVoice-Small or Paraformer-Large. |
+| CPU batch transcription | SenseVoice-Small first; Paraformer-Large for Chinese production pipelines. |
+| Chinese/English/Japanese LLM-style recognition with dialect and accent coverage | Fun-ASR-Nano; use the separate [Fun-ASR-MLT-Nano](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint for 31 languages, and use [vLLM](../vllm_guide.md) for higher LLM decoding throughput. |
+| OpenAI-compatible local endpoint | [funasr-server](../agent_integration.md) with model alias `sensevoice`, `paraformer`, or `fun-asr-nano`. |
+
+For current decisions, use the [model and capability guide](../model_selection.md),
+[Agent integration contracts](../agent_integration.md), and
+[vLLM deployment guide](../vllm_guide.md). The separate MLT checkpoint's language
+coverage must not be attributed to Fun-ASR-Nano. Evaluate your own audio, runtime
+and end-to-end latency before selecting a deployment.
+
+Use the [performance methodology](rtf_reproducibility.md) for new measurements
+and the [WebSocket benchmark guide](realtime_ws_benchmark.md) for concurrent
+realtime services. The current
+[migration timing helper](../../examples/migration/benchmark_funasr.py) measures
+FunASR on your own audio; it does not compute CER/WER, run Whisper or reproduce
+the missing historical scripts. Keep your timing scope, failed files and quality
+evaluation separate and explicit.
diff --git a/docs/benchmark/historical_asr_zh.md b/docs/benchmark/historical_asr_zh.md
new file mode 100644
index 000000000..5bbfe7b8f
--- /dev/null
+++ b/docs/benchmark/historical_asr_zh.md
@@ -0,0 +1,98 @@
+# 历史 ASR 评测
+
+[English](historical_asr.md)
+
+这是一份**来源不完整的历史记录**,用于帮助读者理解早期 FunASR 对比结果。
+它不是新测量、通用排行榜,也不是对当前 checkpoint、机器或部署的保证。
+开展新的评测请先阅读[性能评测方法](rtf_reproducibility.md)。
+
+## 历史概览
+
+下表保留原报告的措辞与数字。“最佳”等标签仅指该份报告,不能推广到所有模型或硬件。
+
+| 指标 | 结果 |
+| --- | --- |
+| 数据集 | 184 条中文长音频,总时长 11,539 秒,约 192.3 分钟。 |
+| GPU | NVIDIA H100 80GB HBM3. |
+| 最佳 GPU 速度 | SenseVoice-Small: 169.6x realtime in the full benchmark, 211.8x in the initial run. |
+| 最佳 CPU 速度 | SenseVoice-Small: 17.2x realtime; Paraformer-Large: 15.6x realtime. |
+| 基线 | OpenAI Whisper-large-v3:GPU 上 13.4 倍实时。 |
+
+**完整运行的 169.6x 与初次运行的 211.8x 是两个独立报告的结果**。
+原页面没有披露测量日期。**2026-09-07 是本次来源快照核对日期,不是测量日期**。
+
+## 历史结果
+
+下表全部数值和说明都是原报告的历史表述,**不是当前 API 能力保证**。
+模型原始输出包含标签,不代表 HTTP 接口一定返回这些标签;旧时间戳限制也不能代替
+当前的[模型选型说明](../model_selection_zh.md)。
+
+| 模型 | 设备 | RTF | 速度 | CER | 说明 |
+| --- | --- | --- | --- | --- | --- |
+| SenseVoice-Small | GPU | 0.005896 | 169.6x | 7.81% | ASR + 语种 / 情感 / 事件标签;CER 已去除标签后计算。 |
+| Paraformer-Large | GPU | 0.008359 | 119.6x | 10.18% | 高速非自回归中文 ASR,适合 VAD/标点生产流水线。 |
+| Fun-ASR-Nano | GPU | 0.058803 | 17.0x | 8.06% | LLM-based 中/英/日 ASR,另覆盖 7 类中文方言和 26 种地域口音,支持热词;不提供可靠的 checkpoint 原生时间戳([#106](https://github.com/QwenAudio/Fun-ASR/issues/106))。 |
+| GLM-ASR-Nano | GPU | 0.026974 | 37.1x | 31.07% | LLM-based 多语种 ASR。 |
+| Whisper-large-v3-turbo (OpenAI) | GPU | 0.021708 | 46.1x | 21.71% | OpenAI Whisper 实现。 |
+| Whisper-large-v3 (OpenAI) | GPU | 0.074694 | 13.4x | 20.02% | 基线 for large Whisper quality. |
+| SenseVoice-Small | CPU | 0.057988 | 17.2x | 7.81% | CPU 结果来自 remaining benchmark 脚本。 |
+| Paraformer-Large | CPU | 0.064056 | 15.6x | 10.18% | CPU 上可用于批量任务。 |
+| Fun-ASR-Nano | CPU | 0.274318 | 3.6x | 8.06% | LLM-based 模型更重,但仍高于实时。 |
+
+CPU/GPU 行重复出现相同 CER,不能证明曾分别独立评分。审计材料中没有原始预测、
+参考文本或评分程序。“去除标签后计算”保留为历史说法,不是本次验证过的评分输出。
+原始精度以及经过舍入的速度/RTF 数值对均原样保留,不重新计算。
+
+## 来源与限制
+
+[原中文 HTML](https://github.com/modelscope/FunASR/blob/67d63b80a246dc33749e43904c294e0409cd9183/zh/benchmark.html)
+固定到历史 GitHub Pages 提交;本次核对时,该文件与旧公网快照逐字节一致。
+这能确认表格出处,但不能证明原始测量正确或可复现。
+
+原报告将 RTF 定义为总推理时间除以总音频时长,速度为其倒数,后者也称 RTFx:
+
+```text
+RTF = total inference time / total audio duration
+RTFx = total audio duration / total inference time = 1 / RTF
+```
+
+以下命令仅是**历史文本,不能直接执行**:在本次审计的 FunASR 源码版本
+`386f6f9106684ba5a114e796147db4396a09eab5` 中,三个文件都不存在。
+本页没有提供替代脚本或复现数据。
+
+```text
+python benchmark/run_full_benchmark.py
+python benchmark/run_remaining.py
+python benchmark/fix_sensevoice_cer.py
+```
+
+审计的原报告未披露 CPU 型号/线程数、数据成员与参考文本清单、精确 checkpoint
+revision、软件/驱动版本、逐文件预测和计时日志,以及是否包含预热、I/O、预处理等
+完整计时范围。缺少这些材料,旧表无法直接复现,“CPU 对比 GPU”的标题也不能视作
+面向所有生产环境的保证。
+
+这份记录的 **11,539 秒**必须与 [vLLM 评测方法](rtf_reproducibility.md) 中的
+**11,541 秒**分开引用。两者都提到 184 个文件,并不能证明数据成员完全一致;
+不要合并两份表格的结果,也不要自行消除两秒差异。
+
+## 当前选型
+
+下面是**原报告的建议表,仅保留为历史上下文**,不是新验证的推荐或性能排名。
+
+| 需求 | 推荐模型 |
+| --- | --- |
+| 最快生产转写 | SenseVoice-Small 或 Paraformer-Large。 |
+| CPU 批量转写 | 优先 SenseVoice-Small;中文生产流水线可选 Paraformer-Large。 |
+| 中/英/日及中文方言、口音的 LLM-style 识别 | Fun-ASR-Nano;需要 31 语种时使用独立的 [Fun-ASR-MLT-Nano](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint,并使用 [vLLM](../vllm_guide_zh.md) 提升 LLM 解码吞吐。 |
+| OpenAI 兼容本地服务 | 使用 [funasr-server](../agent_integration_zh.md),模型别名为 `sensevoice`、`paraformer` 或 `fun-asr-nano`。 |
+
+当前决策请参考[模型与能力说明](../model_selection_zh.md)、
+[Agent 接口契约](../agent_integration_zh.md)和 [vLLM 部署说明](../vllm_guide_zh.md)。
+独立 MLT checkpoint 的语种覆盖不能归到 Fun-ASR-Nano;选型前请使用自己的音频、
+运行时与端到端延迟要求进行评估。
+
+新的测量按[性能评测方法](rtf_reproducibility.md)记录;并发实时服务参考
+[WebSocket 压测说明](realtime_ws_benchmark.md)。当前的
+[迁移计时工具](../../examples/migration/benchmark_funasr.py)只测量 FunASR 在自有音频
+上的表现,**不计算 CER/WER**、不运行 Whisper,也不复现缺失的历史脚本。
+请分别记录计时范围、失败文件和质量评估。
diff --git a/docs/index.rst b/docs/index.rst
index 91e660c49..8671321b7 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -60,6 +60,8 @@ entry to these guides. Source Markdown remains in this repository.
:maxdepth: 1
:caption: Evaluate and Operate
+ benchmark/historical_asr
+ benchmark/historical_asr_zh
benchmark/rtf_reproducibility
benchmark/realtime_ws_benchmark
troubleshooting
diff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.md
index d4747a729..09a5cb64e 100644
--- a/docs/use_case_showcase.md
+++ b/docs/use_case_showcase.md
@@ -8,7 +8,7 @@ FunASR is useful far beyond a single offline transcription command. This page co
|---|---|---|
| Try FunASR in a browser | [Colab quickstart](../examples/colab/) | Run a public sample and upload your own audio before setting up a local environment. |
| Transcribe one file locally | [README quick start](../README.md#quick-start) and [model selection guide](./model_selection.md) | Verify install, model choice, and model download in minutes. |
-| Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. |
+| Compare accuracy and speed | [Historical benchmark](./benchmark/historical_asr.md) and [current methodology](./benchmark/rtf_reproducibility.md) | Read the historical results and provenance limits, then benchmark representative local audio before choosing a model. |
| Migrate from Whisper/cloud ASR | [Migration guide](./migration_from_whisper.md) | Map existing pipelines to FunASR, benchmark representative audio, and plan a safe rollout. |
| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/), [Gradio browser demo](../examples/openai_api/GRADIO.md), [client recipes](../examples/openai_api/CLIENTS.md), [JavaScript/TypeScript recipes](../examples/openai_api/JAVASCRIPT.md), and [workflow recipes](../examples/openai_api/WORKFLOWS.md) | Reuse LangChain, Dify, n8n, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. |
| Reuse an existing integration | [Community integrations](./community_projects.md) | Start from verified upstream paths for voice agents, local assistants, desktop subtitles, model serving, and Rust VAD. |
diff --git a/web-pages/product-site/assets/css/experience.css b/web-pages/product-site/assets/css/experience.css
index d76fbbd46..8e39eb294 100644
--- a/web-pages/product-site/assets/css/experience.css
+++ b/web-pages/product-site/assets/css/experience.css
@@ -158,6 +158,8 @@ td, th { padding: 14px; }
.docs-article pre code { background: none; color: inherit; white-space: pre; padding: 0; }
.docs-article :not(pre) > code { padding: 2px 5px; border-radius: 4px; background: #f0f4f1; font-size: .88em; }
.docs-article table { display: block; overflow-x: auto; max-width: 100%; width: 100%; overflow-wrap: normal; }
+.docs-article #historical-results ~ table td:nth-child(6),
+.docs-article #\5386\53f2\7ed3\679c ~ table td:nth-child(6) { min-width: 260px; }
.docs-article blockquote { border-left: 3px solid #7ea98e; background: #f3f8f4; margin: 20px 0; padding: 8px 20px; color: #52655a; }
.docs-toc { position: sticky; top: 104px; max-height: calc(100vh - 130px); overflow-y: auto; border-left: 1px solid var(--line); padding-left: 18px; font-size: 12px; }
.docs-toc strong { color: var(--ink); }
diff --git a/web-pages/product-site/data/documentation.json b/web-pages/product-site/data/documentation.json
index 4e5c503e6..37859ef65 100644
--- a/web-pages/product-site/data/documentation.json
+++ b/web-pages/product-site/data/documentation.json
@@ -33,6 +33,7 @@
{"slug": "troubleshooting", "group": "operate", "zh": "安装与推理故障排查", "en": "Troubleshooting", "source_zh": "docs/troubleshooting_zh.md", "source_en": "docs/troubleshooting.md"},
{"slug": "security", "group": "operate", "zh": "服务安全边界", "en": "Service security boundaries", "source_zh": "examples/openai_api/SECURITY_zh.md", "source_en": "examples/openai_api/SECURITY.md"},
{"slug": "benchmark-method", "group": "operate", "zh": "可复现性能评测", "en": "Reproducible performance tests", "source_zh": "docs/benchmark/rtf_reproducibility.md", "source_en": "docs/benchmark/rtf_reproducibility.md"},
+ {"slug": "historical-asr-benchmark", "group": "operate", "zh": "历史 ASR 评测", "en": "Historical ASR benchmark", "source_zh": "docs/benchmark/historical_asr_zh.md", "source_en": "docs/benchmark/historical_asr.md"},
{"slug": "realtime-benchmark", "group": "operate", "zh": "实时 WebSocket 压测", "en": "Realtime WebSocket benchmarking", "source_zh": "docs/benchmark/realtime_ws_benchmark.md", "source_en": "docs/benchmark/realtime_ws_benchmark.md"},
{"slug": "agent-integration", "group": "ecosystem", "zh": "Agent 集成", "en": "Agent integration", "source_zh": "docs/agent_integration_zh.md", "source_en": "docs/agent_integration.md"},
{"slug": "use-cases", "group": "ecosystem", "zh": "应用场景与示例", "en": "Use cases & examples", "source_zh": "docs/use_case_showcase_zh.md", "source_en": "docs/use_case_showcase.md"},
diff --git a/web-pages/product-site/data/legacy_doc_anchors.json b/web-pages/product-site/data/legacy_doc_anchors.json
index a0bc12711..862b73ebf 100644
--- a/web-pages/product-site/data/legacy_doc_anchors.json
+++ b/web-pages/product-site/data/legacy_doc_anchors.json
@@ -1,6 +1,10 @@
{
"source_ref": "origin/gh-pages",
"source_revision": "871ac766876c2f511550478d88c86aa3daab3172",
+ "additional_source_revisions": {
+ "benchmark.html": "67d63b80a246dc33749e43904c294e0409cd9183",
+ "zh/benchmark.html": "67d63b80a246dc33749e43904c294e0409cd9183"
+ },
"notes": "Published fragments map to the nearest current topic heading. Consolidated quickstart topics use workflow sections; removed standalone runtime diagnostics use the deployment readiness checklist. Source articles and API reference pages are not rewritten by this compatibility layer.",
"pages": {
"tutorial.html": {
@@ -148,6 +152,18 @@
"voice": "desktop-voice-input",
"subtitle": "subtitle-generation"
},
+ "benchmark.html": {
+ "summary": "historical-summary",
+ "table": "historical-results",
+ "method": "provenance-and-limitations",
+ "choose": "choosing-a-current-path"
+ },
+ "zh/benchmark.html": {
+ "summary": "历史概览",
+ "table": "历史结果",
+ "method": "来源与限制",
+ "choose": "当前选型"
+ },
"zh/agent.html": {
"server": "http-服务",
"sdk": "sdk-与-curl",
diff --git a/web-pages/product-site/export_docs.py b/web-pages/product-site/export_docs.py
index 83a78ad14..4d4b80d71 100644
--- a/web-pages/product-site/export_docs.py
+++ b/web-pages/product-site/export_docs.py
@@ -18,6 +18,7 @@ ALIASES = {
'vllm': 'vllm.html',
'deployment-matrix': 'deployment-matrix.html',
'agent-integration': 'agent.html',
+ 'historical-asr-benchmark': 'benchmark.html',
}
diff --git a/web-pages/product-site/tests/browser/documentation.spec.ts b/web-pages/product-site/tests/browser/documentation.spec.ts
index 593d1e72f..3650f3773 100644
--- a/web-pages/product-site/tests/browser/documentation.spec.ts
+++ b/web-pages/product-site/tests/browser/documentation.spec.ts
@@ -93,6 +93,46 @@ test('public speech sample is playable and waveform has actual pixels', async ({
expect(await page.locator('.hero-image').evaluate((image: HTMLImageElement) => image.naturalWidth)).toBeGreaterThan(1000);
});
+for (const prefix of ['', '/en']) {
+ for (const width of [390, 1440]) {
+ test(`Historical ASR benchmark is discoverable and readable ${prefix || '/zh'} ${width}`, async ({ page }) => {
+ await page.setViewportSize({ width, height: 900 });
+ const errors: string[] = [];
+ page.on('pageerror', error => errors.push(error.message));
+ await page.goto(`${prefix}/docs/`);
+ await page.locator('[data-doc-search] input').fill(prefix ? 'Historical ASR' : '历史 ASR');
+ const route = `${prefix}/docs/historical-asr-benchmark.html`;
+ await page.locator(`.search-results a[href="${route}"]`).click();
+ await expect(page).toHaveURL(new RegExp(`${route}$`));
+ await expect(page.locator('.docs-article h2')).toHaveCount(4);
+ await expect(page.locator('.docs-article table')).toHaveCount(3);
+ const results = page.locator('.docs-article table').nth(1);
+ await expect(results.locator('tr')).toHaveCount(10);
+ const notes = await results.locator('td:nth-child(6)').evaluateAll(nodes => nodes.map(node => ({
+ width: node.getBoundingClientRect().width,
+ rowHeight: node.parentElement!.getBoundingClientRect().height,
+ })));
+ expect(notes.every(note => note.width >= 260 && note.rowHeight < 240)).toBeTruthy();
+ const cell = results.locator('td').filter({ hasText: /^0\.005896$/ });
+ await cell.scrollIntoViewIfNeeded();
+ const numberHeight = await cell.evaluate(node => {
+ const range = document.createRange();
+ range.selectNodeContents(node);
+ return range.getBoundingClientRect().height;
+ });
+ expect(numberHeight).toBeLessThan(32);
+ for (const table of await page.locator('.docs-article table').all()) {
+ await table.scrollIntoViewIfNeeded();
+ expect(await page.evaluate(() => document.documentElement.scrollWidth - innerWidth)).toBeLessThanOrEqual(1);
+ }
+ const peer = `${prefix ? '' : '/en'}/docs/historical-asr-benchmark.html`;
+ await page.locator(`.docs-article a[href="${peer}"]`).click();
+ await expect(page).toHaveURL(new RegExp(`${peer}$`));
+ expect(errors).toEqual([]);
+ });
+ }
+}
+
for (const prefix of ['', '/en']) {
for (const width of [390, 1440]) {
test(`Agent integration is discoverable and readable ${prefix || '/zh'} ${width}`, async ({ page, context }) => {
diff --git a/web-pages/product-site/tests/fixtures/historical_asr_tables.json b/web-pages/product-site/tests/fixtures/historical_asr_tables.json
new file mode 100644
index 000000000..57e4791a6
--- /dev/null
+++ b/web-pages/product-site/tests/fixtures/historical_asr_tables.json
@@ -0,0 +1,807 @@
+{
+ "schema_version": 1,
+ "source_commit": "67d63b80a246dc33749e43904c294e0409cd9183",
+ "status": "Frozen historical HTML cells, not new measurements",
+ "pages": [
+ {
+ "language": "en",
+ "requested_url": "https://modelscope.github.io/FunASR/benchmark.html",
+ "fetched_at_utc": "2026-09-07T11:03:49.943245+00:00",
+ "sha256": "e11c0d3842555f8f376d8ee658ee1dc4c45e92c08f2d420b3b60913fde6d8186",
+ "tables": [
+ {
+ "section_id": "summary",
+ "rows": [
+ [
+ {
+ "text": "Metric",
+ "links": []
+ },
+ {
+ "text": "Result",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Dataset",
+ "links": []
+ },
+ {
+ "text": "184 long-form Chinese audio files, 11,539 s total, 192.3 min.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "NVIDIA H100 80GB HBM3.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Best GPU speed",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small: 169.6x realtime in the full benchmark, 211.8x in the initial run.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Best CPU speed",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small: 17.2x realtime; Paraformer-Large: 15.6x realtime.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Baseline",
+ "links": []
+ },
+ {
+ "text": "OpenAI Whisper-large-v3: 13.4x realtime on GPU.",
+ "links": []
+ }
+ ]
+ ]
+ },
+ {
+ "section_id": "table",
+ "rows": [
+ [
+ {
+ "text": "Model",
+ "links": []
+ },
+ {
+ "text": "Device",
+ "links": []
+ },
+ {
+ "text": "RTF",
+ "links": []
+ },
+ {
+ "text": "Speed",
+ "links": []
+ },
+ {
+ "text": "CER",
+ "links": []
+ },
+ {
+ "text": "Notes",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "SenseVoice-Small",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.005896",
+ "links": []
+ },
+ {
+ "text": "169.6x",
+ "links": []
+ },
+ {
+ "text": "7.81%",
+ "links": []
+ },
+ {
+ "text": "ASR + language / emotion / event tags; CER after tag stripping.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Paraformer-Large",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.008359",
+ "links": []
+ },
+ {
+ "text": "119.6x",
+ "links": []
+ },
+ {
+ "text": "10.18%",
+ "links": []
+ },
+ {
+ "text": "Fast non-autoregressive Chinese ASR with VAD/punctuation pipeline.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Fun-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.058803",
+ "links": []
+ },
+ {
+ "text": "17.0x",
+ "links": []
+ },
+ {
+ "text": "8.06%",
+ "links": []
+ },
+ {
+ "text": "LLM-based ASR for Chinese, English, Japanese, seven Chinese dialect groups, and 26 regional accents; supports hotwords. Reliable checkpoint-native timestamps are not available (#106).",
+ "links": [
+ "https://github.com/QwenAudio/Fun-ASR/issues/106"
+ ]
+ }
+ ],
+ [
+ {
+ "text": "GLM-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.026974",
+ "links": []
+ },
+ {
+ "text": "37.1x",
+ "links": []
+ },
+ {
+ "text": "31.07%",
+ "links": []
+ },
+ {
+ "text": "LLM-based multilingual ASR.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Whisper-large-v3-turbo (OpenAI)",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.021708",
+ "links": []
+ },
+ {
+ "text": "46.1x",
+ "links": []
+ },
+ {
+ "text": "21.71%",
+ "links": []
+ },
+ {
+ "text": "OpenAI Whisper implementation.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Whisper-large-v3 (OpenAI)",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.074694",
+ "links": []
+ },
+ {
+ "text": "13.4x",
+ "links": []
+ },
+ {
+ "text": "20.02%",
+ "links": []
+ },
+ {
+ "text": "Baseline for large Whisper quality.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "SenseVoice-Small",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.057988",
+ "links": []
+ },
+ {
+ "text": "17.2x",
+ "links": []
+ },
+ {
+ "text": "7.81%",
+ "links": []
+ },
+ {
+ "text": "CPU run from the remaining benchmark script.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Paraformer-Large",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.064056",
+ "links": []
+ },
+ {
+ "text": "15.6x",
+ "links": []
+ },
+ {
+ "text": "10.18%",
+ "links": []
+ },
+ {
+ "text": "CPU viable for batch jobs.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Fun-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.274318",
+ "links": []
+ },
+ {
+ "text": "3.6x",
+ "links": []
+ },
+ {
+ "text": "8.06%",
+ "links": []
+ },
+ {
+ "text": "LLM-based model is heavier but still above realtime.",
+ "links": []
+ }
+ ]
+ ]
+ },
+ {
+ "section_id": "choose",
+ "rows": [
+ [
+ {
+ "text": "Need",
+ "links": []
+ },
+ {
+ "text": "Recommended model",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Fastest production transcription",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small or Paraformer-Large.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "CPU batch transcription",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small first; Paraformer-Large for Chinese production pipelines.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Chinese/English/Japanese LLM-style recognition with dialect and accent coverage",
+ "links": []
+ },
+ {
+ "text": "Fun-ASR-Nano; use the separate Fun-ASR-MLT-Nano checkpoint for 31 languages, and use vLLM for higher LLM decoding throughput.",
+ "links": [
+ "https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512",
+ "vllm.html"
+ ]
+ }
+ ],
+ [
+ {
+ "text": "OpenAI-compatible local endpoint",
+ "links": []
+ },
+ {
+ "text": "funasr-server with model alias sensevoice, paraformer, or fun-asr-nano.",
+ "links": [
+ "agent.html"
+ ]
+ }
+ ]
+ ]
+ }
+ ]
+ },
+ {
+ "language": "zh",
+ "requested_url": "https://modelscope.github.io/FunASR/zh/benchmark.html",
+ "fetched_at_utc": "2026-09-07T11:03:50.243350+00:00",
+ "sha256": "af8cf07e9cf06c9ce5865b54f34581a2911ede699f750776634d8b6d22183462",
+ "tables": [
+ {
+ "section_id": "summary",
+ "rows": [
+ [
+ {
+ "text": "指标",
+ "links": []
+ },
+ {
+ "text": "结果",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "数据集",
+ "links": []
+ },
+ {
+ "text": "184 条中文长音频,总时长 11,539 秒,约 192.3 分钟。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "NVIDIA H100 80GB HBM3.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "最佳 GPU 速度",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small: 169.6x realtime in the full benchmark, 211.8x in the initial run.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "最佳 CPU 速度",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small: 17.2x realtime; Paraformer-Large: 15.6x realtime.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "基线",
+ "links": []
+ },
+ {
+ "text": "OpenAI Whisper-large-v3:GPU 上 13.4 倍实时。",
+ "links": []
+ }
+ ]
+ ]
+ },
+ {
+ "section_id": "table",
+ "rows": [
+ [
+ {
+ "text": "模型",
+ "links": []
+ },
+ {
+ "text": "设备",
+ "links": []
+ },
+ {
+ "text": "RTF",
+ "links": []
+ },
+ {
+ "text": "速度",
+ "links": []
+ },
+ {
+ "text": "CER",
+ "links": []
+ },
+ {
+ "text": "说明",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "SenseVoice-Small",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.005896",
+ "links": []
+ },
+ {
+ "text": "169.6x",
+ "links": []
+ },
+ {
+ "text": "7.81%",
+ "links": []
+ },
+ {
+ "text": "ASR + 语种 / 情感 / 事件标签;CER 已去除标签后计算。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Paraformer-Large",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.008359",
+ "links": []
+ },
+ {
+ "text": "119.6x",
+ "links": []
+ },
+ {
+ "text": "10.18%",
+ "links": []
+ },
+ {
+ "text": "高速非自回归中文 ASR,适合 VAD/标点生产流水线。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Fun-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.058803",
+ "links": []
+ },
+ {
+ "text": "17.0x",
+ "links": []
+ },
+ {
+ "text": "8.06%",
+ "links": []
+ },
+ {
+ "text": "LLM-based 中/英/日 ASR,另覆盖 7 类中文方言和 26 种地域口音,支持热词;不提供可靠的 checkpoint 原生时间戳(#106)。",
+ "links": [
+ "https://github.com/QwenAudio/Fun-ASR/issues/106"
+ ]
+ }
+ ],
+ [
+ {
+ "text": "GLM-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.026974",
+ "links": []
+ },
+ {
+ "text": "37.1x",
+ "links": []
+ },
+ {
+ "text": "31.07%",
+ "links": []
+ },
+ {
+ "text": "LLM-based 多语种 ASR。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Whisper-large-v3-turbo (OpenAI)",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.021708",
+ "links": []
+ },
+ {
+ "text": "46.1x",
+ "links": []
+ },
+ {
+ "text": "21.71%",
+ "links": []
+ },
+ {
+ "text": "OpenAI Whisper 实现。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Whisper-large-v3 (OpenAI)",
+ "links": []
+ },
+ {
+ "text": "GPU",
+ "links": []
+ },
+ {
+ "text": "0.074694",
+ "links": []
+ },
+ {
+ "text": "13.4x",
+ "links": []
+ },
+ {
+ "text": "20.02%",
+ "links": []
+ },
+ {
+ "text": "基线 for large Whisper quality.",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "SenseVoice-Small",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.057988",
+ "links": []
+ },
+ {
+ "text": "17.2x",
+ "links": []
+ },
+ {
+ "text": "7.81%",
+ "links": []
+ },
+ {
+ "text": "CPU 结果来自 remaining benchmark 脚本。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Paraformer-Large",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.064056",
+ "links": []
+ },
+ {
+ "text": "15.6x",
+ "links": []
+ },
+ {
+ "text": "10.18%",
+ "links": []
+ },
+ {
+ "text": "CPU 上可用于批量任务。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "Fun-ASR-Nano",
+ "links": []
+ },
+ {
+ "text": "CPU",
+ "links": []
+ },
+ {
+ "text": "0.274318",
+ "links": []
+ },
+ {
+ "text": "3.6x",
+ "links": []
+ },
+ {
+ "text": "8.06%",
+ "links": []
+ },
+ {
+ "text": "LLM-based 模型更重,但仍高于实时。",
+ "links": []
+ }
+ ]
+ ]
+ },
+ {
+ "section_id": "choose",
+ "rows": [
+ [
+ {
+ "text": "需求",
+ "links": []
+ },
+ {
+ "text": "推荐模型",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "最快生产转写",
+ "links": []
+ },
+ {
+ "text": "SenseVoice-Small 或 Paraformer-Large。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "CPU 批量转写",
+ "links": []
+ },
+ {
+ "text": "优先 SenseVoice-Small;中文生产流水线可选 Paraformer-Large。",
+ "links": []
+ }
+ ],
+ [
+ {
+ "text": "中/英/日及中文方言、口音的 LLM-style 识别",
+ "links": []
+ },
+ {
+ "text": "Fun-ASR-Nano;需要 31 语种时使用独立的 Fun-ASR-MLT-Nano checkpoint,并使用 vLLM 提升 LLM 解码吞吐。",
+ "links": [
+ "https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512",
+ "vllm.html"
+ ]
+ }
+ ],
+ [
+ {
+ "text": "OpenAI 兼容本地服务",
+ "links": []
+ },
+ {
+ "text": "使用 funasr-server,模型别名为 sensevoice、paraformer 或 fun-asr-nano。",
+ "links": [
+ "agent.html"
+ ]
+ }
+ ]
+ ]
+ }
+ ]
+ }
+ ]
+}
diff --git a/web-pages/product-site/tests/test_documentation.py b/web-pages/product-site/tests/test_documentation.py
index 6f6433e0b..3eaa73455 100644
--- a/web-pages/product-site/tests/test_documentation.py
+++ b/web-pages/product-site/tests/test_documentation.py
@@ -209,3 +209,58 @@ def test_agent_recipes_are_bilingual_and_use_existing_script_options():
'examples/voice_input/funasr_input.py',
'examples/subtitle/generate_subtitle.py'}
assert recipes[0] == recipes[1]
+
+
[email protected]('language,prefix,suffix', [('en', 'en/', ''), ('zh', '', '_zh')])
+def test_historical_benchmark_preserves_frozen_tables_and_discovery(output, language, prefix, suffix):
+ frozen = json.loads((SITE_ROOT / 'tests/fixtures/historical_asr_tables.json').read_text())
+ record = next(item for item in frozen['pages'] if item['language'] == language)
+ route = '/' + prefix + 'docs/historical-asr-benchmark.html'
+ path = output / route.lstrip('/')
+ assert path.is_file(), route
+ page = BeautifulSoup(path.read_text(), 'html.parser')
+ article = page.select_one('.docs-article')
+ tables = article.select('table')
+ assert len(tables) == 3
+ assert len(article.select('h2')) == 4
+ for table, expected in zip(tables, record['tables']):
+ actual = [[' '.join(cell.get_text().split()) for cell in row.select('th,td')]
+ for row in table.select('tr')]
+ assert actual == [[cell['text'] for cell in row] for row in expected['rows']]
+ assert page.select_one('[data-source-link]')['href'].endswith(
+ f'/docs/benchmark/historical_asr{suffix}.md')
+ links = {link['href'] for link in article.select('a[href]')}
+ for slug in ('agent-integration', 'vllm', 'model-selection', 'benchmark-method', 'realtime-benchmark'):
+ assert '/' + prefix + 'docs/' + slug + '.html' in links
+ assert ('/docs/' if prefix else '/en/docs/') + 'historical-asr-benchmark.html' in links
+ assert 'https://github.com/QwenAudio/Fun-ASR/issues/106' in links
+ assert 'https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512' in links
+ original_path = urlsplit(record['requested_url']).path.removeprefix('/FunASR/')
+ assert f"https://github.com/modelscope/FunASR/blob/{frozen['source_commit']}/{original_path}" in links
+ index = BeautifulSoup((output / prefix / 'docs/index.html').read_text(), 'html.parser')
+ assert index.select_one(f'a[href="{route}"]')
+ search = json.loads((output / f'search-{language}.json').read_text())
+ assert any(row['url'] == route for row in search)
+
+
[email protected]('suffix', ['', '_zh'])
+def test_historical_benchmark_does_not_advertise_missing_scripts_as_runnable(suffix):
+ root = SITE_ROOT.parents[1]
+ source_path = root / f'docs/benchmark/historical_asr{suffix}.md'
+ assert source_path.is_file()
+ source = source_path.read_text()
+ blocks = re.findall(r'^```(\w+)\n(.*?)^```', source, re.M | re.S)
+ assert len(blocks) == 2 and all(language == 'text' for language, _ in blocks)
+ commands = [line for _, block in blocks for line in block.splitlines() if line.startswith('python ')]
+ assert commands == ['python benchmark/run_full_benchmark.py',
+ 'python benchmark/run_remaining.py',
+ 'python benchmark/fix_sensevoice_cer.py']
+ for command in commands:
+ assert not (root / command.split()[1]).exists()
+ for marker in ('11,539', '11,541', '169.6x', '211.8x', 'RTFx', '2026-09-07'):
+ assert marker in source
+ for marker in (('来源不完整', '不是测量日期', '不能直接执行', '不计算 CER/WER') if suffix
+ else ('incomplete provenance', 'not the measurement date', 'cannot be run directly', 'does not compute CER/WER')):
+ assert marker in source
+ if not suffix:
+ assert 'Reproduce the 184-file long-audio benchmark' not in (root / 'docs/use_case_showcase.md').read_text()
diff --git a/web-pages/product-site/tests/test_legacy_doc_anchors.py b/web-pages/product-site/tests/test_legacy_doc_anchors.py
index 56ce76686..f1c953b1e 100644
--- a/web-pages/product-site/tests/test_legacy_doc_anchors.py
+++ b/web-pages/product-site/tests/test_legacy_doc_anchors.py
@@ -21,6 +21,7 @@ OLD_IDS = {
'deployment-matrix.html': 'matrix choices runtime-diagnostics checklist help',
# Independently inventoried from public EN/ZH pages on 2026-09-07.
'agent.html': 'server sdk workflows mcp voice subtitle',
+ 'benchmark.html': 'summary table method choose',
}
PAGES = [prefix + name for name in OLD_IDS for prefix in ('', 'zh/')]
@@ -151,6 +152,15 @@ def test_agent_legacy_fragments_keep_their_topics(page, targets):
assert pages[page] == dict(zip('server sdk workflows mcp voice subtitle'.split(), targets))
[email protected]('page,targets', [
+ ('benchmark.html', ['historical-summary', 'historical-results', 'provenance-and-limitations', 'choosing-a-current-path']),
+ ('zh/benchmark.html', ['历史概览', '历史结果', '来源与限制', '当前选型']),
+])
+def test_benchmark_legacy_fragments_keep_their_topics(page, targets):
+ pages = json.loads((SITE / 'data/legacy_doc_anchors.json').read_text())['pages']
+ assert pages[page] == dict(zip('summary table method choose'.split(), targets))
+
+
@pytest.mark.parametrize('heading', ('<h2 id="different">Topic</h2>', '<p id="topic">Not a heading</p>'))
def test_missing_or_nonheading_target_fails_explicitly(heading):
from export_docs import insert_legacy_anchors