PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Update alg combination news (#2227)
Wenhua Chengbd9a13ceec95
diff --git a/README.md b/README.mdindex 0831747ce..1e3f4e1a7 100644--- a/README.md+++ b/README.md@@ -7,7 +7,7 @@</p>-<h3> Advanced Quantization Algorithm for LLMs</h3>+<h3> Advanced Quantization Toolkit for LLMs</h3>[](https://github.com/intel/auto-round)[](https://github.com/intel/auto-round/releases)@@ -36,7 +36,7 @@ See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2]## 🆕 What's New-* [2026/08] We experimentally support **algorithm composition**, e.g., `--algs awq,signround`[./docs/awq_details.md](./docs/awq_details.md) and `--algs hadamard,awq,signround`[./docs/rotation_details.md](./docs/rotation_details.md), to improve accuracy, especially in scenarios with activation quantization.+* [2026/08] We experimentally support **algorithm composition** (e.g., `--algs awq,signround` or `--algs hadamard,awq,signround`) to improve accuracy [*Overview*](./docs/algorithm_combinations.md). We welcome any practical, deployable algorithms that are ready for real-world use. Feel free to submit a PR or leave a comment in Issues.* [2026/08] AutoScheme WOQ deployment on vLLM is experimentally restored, thanks to Humming Kernel: [*vLLM PR*](https://github.com/vllm-project/vllm/pull/52890), [*Example Model*](https://huggingface.co/Intel/Qwen3.8-27B-bpw2.8-AutoRound). Note: shared layers must be configured per vLLM's fusion patterns.diff --git a/README_CN.md b/README_CN.mdindex e8a9664f0..6d33063ab 100644--- a/README_CN.md+++ b/README_CN.md@@ -35,7 +35,7 @@ AutoRound 是专为大语言模型(LLMs)和视觉-语言模型(VLMs)设## 🆕 最新进展-* [2026/08] 我们实验性地支持**算法组合**,例如 AWQ + SignRound(`--algorithm awq,signround`,[./docs/awq_details.md](./docs/awq_details.md))以及 Hadamard(research 特性)+ AWQ + SignRound(`--algorithm hadamard,awq,signround`,[./docs/rotation_details.md](./docs/rotation_details.md)),以提升精度,尤其是在带有激活量化的场景中。+* [2026/08] 我们实验性地支持**算法组合**(例如 `--algs awq,signround` 或 `--algs hadamard,awq,signround`),以提升精度:[*总览*](./docs/algorithm_combinations_CN.md). 我们欢迎能真正落地的任何算法组合,欢迎提交 PR 或在 Issues 中留言。* [2026/08] 感谢 Humming Kernel,AutoScheme WOQ 在 vLLM 上的部署已实验性恢复:[*vLLM PR*](https://github.com/vllm-project/vllm/pull/52890),[*示例模型*](https://huggingface.co/Intel/Qwen3.8-27B-bpw2.8-AutoRound)。注意:共享层需按 vLLM 的融合模式进行配置。diff --git a/docs/algorithm_combinations.md b/docs/algorithm_combinations.mdnew file mode 100644index 000000000..1beb2f0a6--- /dev/null+++ b/docs/algorithm_combinations.md@@ -0,0 +1,35 @@+# Algorithm Combinations++AutoRound can be combined with several algorithms+before (or during) quantization. This page summarizes each combination and rates+it along two dimensions:++- **Accuracy Gain** — does the transform improve the accuracy of the quantized+ model compared with plain AutoRound?+- **Deployment** — can the resulting model actually be deployed/served today+ (kernel support, export path, real inference engine)?++## Legend++| Light | Meaning |+|:-----:|:----------------------------------------------------------------|+| 🟢 | Good — clear benefit / ready to deploy |+| 🟡 | Partial — conditional benefit / limited or experimental support |+| 🔴 | Poor — no measurable benefit / not deployable yet |++> **Note:** A Partial/Poor **Accuracy Gain** rating may stem from two factors:+> (1) limitations in our current implementation, and (2) our own internal,+> subjective evaluation. Both are subject to change as the implementation matures+> and more benchmarks become available.++## Matrix++| Combination | Accuracy Gain | Deployment | Details | CLI Usage | Comments | Reference |+|:-------------------------------------------------------|:-------------:|:----------:|:----------------------------------------|:---------------------------------|:----------------------------------------------------------------------------------------|:-----------------------------------------------------|+| AutoRound + AWQ (activation-aware scaling) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | Recommended when activations are quantized (e.g., W4A4). | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |+| AutoRound + Hadamard rotation | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | Especially helpful for INT4 (W4A4) and some MXFP4 scenarios. no production kernel. | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | Python API only | Learns rotation matrices; higher accuracy at extra training cost. no production kernel. | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |+| AutoRound + LFQ (logit-aware final-block quantization) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |+| AutoRound + MX Attention (mxfp4 variant) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | Adopt 7.25 as the denominator for scale calculation | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |+| AutoRound + SVDQuant (low-rank outlier absorption) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | Recommended for diffusion models; currently only FLUX is supported. | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |+diff --git a/docs/algorithm_combinations_CN.md b/docs/algorithm_combinations_CN.mdnew file mode 100644index 000000000..01d9885d2--- /dev/null+++ b/docs/algorithm_combinations_CN.md@@ -0,0 +1,32 @@+# 算法组合++AutoRound 可以在量化之前(或量化过程中)与多种算法结合使用。+本页对每种组合进行汇总,并从两个维度进行评级:++- **精度收益(Accuracy Gain)** —— 相比原始 AutoRound,该变换是否能提升量化模型的精度?+- **可部署性(Deployment)** —— 得到的模型当前是否能够实际部署/服务+ (内核支持、导出路径、真实推理引擎)?++## 图例++| 信号灯 | 含义 |+|:---:|:--------------------------|+| 🟢 | 良好 —— 收益明显 / 可直接部署 |+| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |+| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |++> **说明:** 「精度收益」被评为部分/较差可能有两方面原因:+> (1)当前AutoRound实现尚存在局限;(2)来自我们内部的主观评估。+> 二者都会随着实现完善与更多评测数据的补充而变化。++## 矩阵++| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |+|:-----------------------------------|:----:|:----:|:----------------------------------------|:---------------------------------|:-------------------------------------|:-----------------------------------------------------|+| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |+| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |+| AutoRound + LFQ(logit 感知的末层块量化) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |+| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |+| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |+diff --git a/docs/lfq_acc.md b/docs/lfq_acc.mdnew file mode 100644index 000000000..79ffa6343--- /dev/null+++ b/docs/lfq_acc.md@@ -0,0 +1,13 @@++iters 200 enable_alg_ext is turned on Qwen3-8B++lfq causes large drop, maybe some bugs in our implementation++| Model | ARC-c | ARC-e | BoolQ | Hella | LAMB | MMLU | OBQA | PIQA | Truth. | Wino | GSM8K | if-eval stirct | mmlu-pro | AVG. |+| :--------- | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -------------- | -------- | ---: |+| LFQ W2G64 | 0.3046 | 0.5396 | 0.6783 | 0.3856 | 0.2340 | 0.2822 | 0.2000 | 0.6436 | 0.3011 | 0.5564 | 0.3419 | 0.2206+0.1183 | 0.2334 | |+| ours W2G64 | 0.3942 | 0.7412 | 0.7841 | 0.4282 | 0.5242 | 0.5498 | 0.2540 | 0.7051 | 0.3097 | 0.6464 | 0.5239 | 0.2542+0.1201 | 0.2745 | |+| | | | | | | | | | | | | | | |+| LFQ w3g128 | 0.4949 | 0.7605 | 0.8061 | 0.5276 | 0.4883 | 0.6558 | 0.2860 | 0.7530 | 0.3452 | 0.6717 | 0.8241 | 0.3777+0.2366 | 0.4076 | |+| oursw3g128 | 0.5213 | 0.8093 | 0.8667 | 0.5319 | 0.6445 | 0.6949 | 0.3140 | 0.7546 | 0.3623 | 0.7088 | 0.8597 | 0.3333+0.1904 | 0.4269 | |+| | | | | | | | | | | | | | | |\ No newline at end of file