PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update lfq acc data (#2529)
Wenhua Cheng5ce470f491fd
diff --git a/docs/algorithm_combinations.md b/docs/algorithm_combinations.mdindex 1beb2f0a6..8f04d5a10 100644--- a/docs/algorithm_combinations.md+++ b/docs/algorithm_combinations.md@@ -6,8 +6,8 @@ it along two dimensions:- **Accuracy Gain** — does the transform improve the accuracy of the quantizedmodel compared with plain AutoRound?-- **Deployment** — can the resulting model actually be deployed/served today- (kernel support, export path, real inference engine)?+- **Deployment** — can the resulting model actually be deployed/served today (kernel support, export path, real+ inference engine)?## Legend@@ -29,7 +29,7 @@ it along two dimensions:| AutoRound + AWQ (activation-aware scaling) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | Recommended when activations are quantized (e.g., W4A4). | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) || AutoRound + Hadamard rotation | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | Especially helpful for INT4 (W4A4) and some MXFP4 scenarios. no production kernel. | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) || AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | Python API only | Learns rotation matrices; higher accuracy at extra training cost. no production kernel. | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |-| AutoRound + LFQ (logit-aware final-block quantization) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |+| AutoRound + LFQ (logit-aware final-block quantization) | 🟡 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) || AutoRound + MX Attention (mxfp4 variant) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | Adopt 7.25 as the denominator for scale calculation | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) || AutoRound + SVDQuant (low-rank outlier absorption) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | Recommended for diffusion models; currently only FLUX is supported. | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |diff --git a/docs/algorithm_combinations_CN.md b/docs/algorithm_combinations_CN.mdindex 01d9885d2..50d2bc076 100644--- a/docs/algorithm_combinations_CN.md+++ b/docs/algorithm_combinations_CN.md@@ -9,11 +9,11 @@ AutoRound 可以在量化之前(或量化过程中)与多种算法结合使## 图例-| 信号灯 | 含义 |-|:---:|:--------------------------|-| 🟢 | 良好 —— 收益明显 / 可直接部署 |-| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |-| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |+| 信号灯 | 含义 |+|:------:|:--------------------------------------------|+| 🟢 | 良好 —— 收益明显 / 可直接部署 |+| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |+| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |> **说明:** 「精度收益」被评为部分/较差可能有两方面原因:> (1)当前AutoRound实现尚存在局限;(2)来自我们内部的主观评估。@@ -21,12 +21,12 @@ AutoRound 可以在量化之前(或量化过程中)与多种算法结合使## 矩阵-| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |-|:-----------------------------------|:----:|:----:|:----------------------------------------|:---------------------------------|:-------------------------------------|:-----------------------------------------------------|-| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |-| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |-| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |-| AutoRound + LFQ(logit 感知的末层块量化) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |-| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |-| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |+| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |+|:------------------------------------------|:--------:|:--------:|:----------------------------------------|:---------------------------------|:---------------------------------------------------------|:-----------------------------------------------------|+| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |+| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |+| AutoRound + LFQ(logit 感知的末层块量化) | 🟡 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |+| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |+| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |diff --git a/docs/lfq_acc.md b/docs/lfq_acc.mdindex 79ffa6343..8216269da 100644--- a/docs/lfq_acc.md+++ b/docs/lfq_acc.md@@ -1,4 +1,23 @@+Qwen3-8B with signround iters 200+### W2G64+| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |+| ----- | ----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: |+| V1 | 0.4959 | 0.2596 | 0.4064 | 0.5331 | 0.3720 | 0.7016 | 0.8131 | 0.4437 | 0.4822 | 0.2740 | 0.7051 | 0.3280 | 0.6314 |+| V1+lfq new | 0.5103 | 0.2547 | 0.4420 | 0.5570 | 0.4036 | 0.7344 | 0.7920 | 0.4556 | 0.5267 | 0.2860 | 0.7155 | 0.3182 | 0.6377 |+| V2 | 0.5275 | 0.2921 | 0.5807 | 0.5739 | 0.4317 | 0.7529 | 0.7550 | 0.4495 | 0.5251 | 0.2740 | 0.7171 | 0.3280 | 0.6496 |+| V2+lfq new | 0.5292| 0.2965 | 0.5595 | 0.5695 | 0.4369 | 0.7668 | 0.7636 | 0.4558 | 0.5255 | 0.2680 | 0.7193 | 0.3305 | 0.6590 |++### W3G128++| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |+| ----- | ----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: |+| V1 | 0.6198 | 0.4221 | 0.8423 | 0.6971 | 0.5213 | 0.8136 | 0.8648 | 0.5374 | 0.6220 | 0.3180 | 0.7573 | 0.3611 | 0.6803 |+| V1+lfq new | 0.6151 | 0.4252 | 0.8150 | 0.6967 | 0.5188 | 0.8047 | 0.8676 | 0.5394 | 0.6179 | 0.3120 | 0.7557 | 0.3599 | 0.6685 |+| V2 | 0.6230 | 0.4301 | 0.8696 | 0.6980 | 0.5119 | 0.8114 | 0.8563 | 0.5338 | 0.6447 | 0.3120 | 0.7601 | 0.3574 | 0.6906 |+| V2+lfq new | 0.6253 | 0.4279 | 0.8560 | 0.7012 | 0.5299 | 0.8211 | 0.8615 | 0.5364 | 0.6297 | 0.3140 | 0.7639 | 0.3660 | 0.6961 |++### legacy result with AutoRound version<=0.16.1iters 200 enable_alg_ext is turned on Qwen3-8Blfq causes large drop, maybe some bugs in our implementation