Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

Update alg combination news (#2227)

Wenhua Chengbd9a13ceec95
diff --git a/README.md b/README.md
index 0831747ce..1e3f4e1a7 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@
</p>
-<h3> Advanced Quantization Algorithm for LLMs</h3>
+<h3> Advanced Quantization Toolkit for LLMs</h3>
[![python](https://img.shields.io/badge/python-3.10%2B-blue)](https://github.com/intel/auto-round)
[![version](https://img.shields.io/badge/release-0.14.0-green)](https://github.com/intel/auto-round/releases)
@@ -36,7 +36,7 @@ See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2]
## 🆕 What's New
-* [2026/08] We experimentally support **algorithm composition**, e.g., `--algs awq,signround`[./docs/awq_details.md](./docs/awq_details.md) and `--algs hadamard,awq,signround`[./docs/rotation_details.md](./docs/rotation_details.md), to improve accuracy, especially in scenarios with activation quantization.
+* [2026/08] We experimentally support **algorithm composition** (e.g., `--algs awq,signround` or `--algs hadamard,awq,signround`) to improve accuracy [*Overview*](./docs/algorithm_combinations.md). We welcome any practical, deployable algorithms that are ready for real-world use. Feel free to submit a PR or leave a comment in Issues.
* [2026/08] AutoScheme WOQ deployment on vLLM is experimentally restored, thanks to Humming Kernel: [*vLLM PR*](https://github.com/vllm-project/vllm/pull/52890), [*Example Model*](https://huggingface.co/Intel/Qwen3.8-27B-bpw2.8-AutoRound). Note: shared layers must be configured per vLLM's fusion patterns.
diff --git a/README_CN.md b/README_CN.md
index e8a9664f0..6d33063ab 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -35,7 +35,7 @@ AutoRound 是专为大语言模型(LLMs)和视觉-语言模型(VLMs)设
## 🆕 最新进展
-* [2026/08] 我们实验性地支持**算法组合**,例如 AWQ + SignRound(`--algorithm awq,signround`,[./docs/awq_details.md](./docs/awq_details.md))以及 Hadamard(research 特性)+ AWQ + SignRound(`--algorithm hadamard,awq,signround`,[./docs/rotation_details.md](./docs/rotation_details.md)),以提升精度,尤其是在带有激活量化的场景中。
+* [2026/08] 我们实验性地支持**算法组合**(例如 `--algs awq,signround` 或 `--algs hadamard,awq,signround`),以提升精度:[*总览*](./docs/algorithm_combinations_CN.md). 我们欢迎能真正落地的任何算法组合,欢迎提交 PR 或在 Issues 中留言。
* [2026/08] 感谢 Humming Kernel,AutoScheme WOQ 在 vLLM 上的部署已实验性恢复:[*vLLM PR*](https://github.com/vllm-project/vllm/pull/52890),[*示例模型*](https://huggingface.co/Intel/Qwen3.8-27B-bpw2.8-AutoRound)。注意:共享层需按 vLLM 的融合模式进行配置。
diff --git a/docs/algorithm_combinations.md b/docs/algorithm_combinations.md
new file mode 100644
index 000000000..1beb2f0a6
--- /dev/null
+++ b/docs/algorithm_combinations.md
@@ -0,0 +1,35 @@
+# Algorithm Combinations
+
+AutoRound can be combined with several algorithms
+before (or during) quantization. This page summarizes each combination and rates
+it along two dimensions:
+
+- **Accuracy Gain** — does the transform improve the accuracy of the quantized
+ model compared with plain AutoRound?
+- **Deployment** — can the resulting model actually be deployed/served today
+ (kernel support, export path, real inference engine)?
+
+## Legend
+
+| Light | Meaning |
+|:-----:|:----------------------------------------------------------------|
+| 🟢 | Good — clear benefit / ready to deploy |
+| 🟡 | Partial — conditional benefit / limited or experimental support |
+| 🔴 | Poor — no measurable benefit / not deployable yet |
+
+> **Note:** A Partial/Poor **Accuracy Gain** rating may stem from two factors:
+> (1) limitations in our current implementation, and (2) our own internal,
+> subjective evaluation. Both are subject to change as the implementation matures
+> and more benchmarks become available.
+
+## Matrix
+
+| Combination | Accuracy Gain | Deployment | Details | CLI Usage | Comments | Reference |
+|:-------------------------------------------------------|:-------------:|:----------:|:----------------------------------------|:---------------------------------|:----------------------------------------------------------------------------------------|:-----------------------------------------------------|
+| AutoRound + AWQ (activation-aware scaling) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | Recommended when activations are quantized (e.g., W4A4). | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |
+| AutoRound + Hadamard rotation | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | Especially helpful for INT4 (W4A4) and some MXFP4 scenarios. no production kernel. | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |
+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | Python API only | Learns rotation matrices; higher accuracy at extra training cost. no production kernel. | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |
+| AutoRound + LFQ (logit-aware final-block quantization) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
+| AutoRound + MX Attention (mxfp4 variant) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | Adopt 7.25 as the denominator for scale calculation | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |
+| AutoRound + SVDQuant (low-rank outlier absorption) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | Recommended for diffusion models; currently only FLUX is supported. | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |
+
diff --git a/docs/algorithm_combinations_CN.md b/docs/algorithm_combinations_CN.md
new file mode 100644
index 000000000..01d9885d2
--- /dev/null
+++ b/docs/algorithm_combinations_CN.md
@@ -0,0 +1,32 @@
+# 算法组合
+
+AutoRound 可以在量化之前(或量化过程中)与多种算法结合使用。
+本页对每种组合进行汇总,并从两个维度进行评级:
+
+- **精度收益(Accuracy Gain)** —— 相比原始 AutoRound,该变换是否能提升量化模型的精度?
+- **可部署性(Deployment)** —— 得到的模型当前是否能够实际部署/服务
+ (内核支持、导出路径、真实推理引擎)?
+
+## 图例
+
+| 信号灯 | 含义 |
+|:---:|:--------------------------|
+| 🟢 | 良好 —— 收益明显 / 可直接部署 |
+| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |
+| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |
+
+> **说明:** 「精度收益」被评为部分/较差可能有两方面原因:
+> (1)当前AutoRound实现尚存在局限;(2)来自我们内部的主观评估。
+> 二者都会随着实现完善与更多评测数据的补充而变化。
+
+## 矩阵
+
+| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |
+|:-----------------------------------|:----:|:----:|:----------------------------------------|:---------------------------------|:-------------------------------------|:-----------------------------------------------------|
+| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |
+| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |
+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |
+| AutoRound + LFQ(logit 感知的末层块量化) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
+| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |
+| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |
+
diff --git a/docs/lfq_acc.md b/docs/lfq_acc.md
new file mode 100644
index 000000000..79ffa6343
--- /dev/null
+++ b/docs/lfq_acc.md
@@ -0,0 +1,13 @@
+
+iters 200 enable_alg_ext is turned on Qwen3-8B
+
+lfq causes large drop, maybe some bugs in our implementation
+
+| Model | ARC-c | ARC-e | BoolQ | Hella | LAMB | MMLU | OBQA | PIQA | Truth. | Wino | GSM8K | if-eval stirct | mmlu-pro | AVG. |
+| :--------- | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -------------- | -------- | ---: |
+| LFQ W2G64 | 0.3046 | 0.5396 | 0.6783 | 0.3856 | 0.2340 | 0.2822 | 0.2000 | 0.6436 | 0.3011 | 0.5564 | 0.3419 | 0.2206+0.1183 | 0.2334 | |
+| ours W2G64 | 0.3942 | 0.7412 | 0.7841 | 0.4282 | 0.5242 | 0.5498 | 0.2540 | 0.7051 | 0.3097 | 0.6464 | 0.5239 | 0.2542+0.1201 | 0.2745 | |
+| | | | | | | | | | | | | | | |
+| LFQ w3g128 | 0.4949 | 0.7605 | 0.8061 | 0.5276 | 0.4883 | 0.6558 | 0.2860 | 0.7530 | 0.3452 | 0.6717 | 0.8241 | 0.3777+0.2366 | 0.4076 | |
+| oursw3g128 | 0.5213 | 0.8093 | 0.8667 | 0.5319 | 0.6445 | 0.6949 | 0.3140 | 0.7546 | 0.3623 | 0.7088 | 0.8597 | 0.3333+0.1904 | 0.4269 | |
+| | | | | | | | | | | | | | | |
\ No newline at end of file