Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

update lfq acc data (#2529)

Wenhua Cheng5ce470f491fd
diff --git a/docs/algorithm_combinations.md b/docs/algorithm_combinations.md
index 1beb2f0a6..8f04d5a10 100644
--- a/docs/algorithm_combinations.md
+++ b/docs/algorithm_combinations.md
@@ -6,8 +6,8 @@ it along two dimensions:
- **Accuracy Gain** — does the transform improve the accuracy of the quantized
model compared with plain AutoRound?
-- **Deployment** — can the resulting model actually be deployed/served today
- (kernel support, export path, real inference engine)?
+- **Deployment** — can the resulting model actually be deployed/served today (kernel support, export path, real
+ inference engine)?
## Legend
@@ -29,7 +29,7 @@ it along two dimensions:
| AutoRound + AWQ (activation-aware scaling) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | Recommended when activations are quantized (e.g., W4A4). | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |
| AutoRound + Hadamard rotation | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | Especially helpful for INT4 (W4A4) and some MXFP4 scenarios. no production kernel. | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |
| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | Python API only | Learns rotation matrices; higher accuracy at extra training cost. no production kernel. | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |
-| AutoRound + LFQ (logit-aware final-block quantization) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
+| AutoRound + LFQ (logit-aware final-block quantization) | 🟡 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | Refines the final block to lift low-bit generation quality | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
| AutoRound + MX Attention (mxfp4 variant) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | Adopt 7.25 as the denominator for scale calculation | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |
| AutoRound + SVDQuant (low-rank outlier absorption) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | Recommended for diffusion models; currently only FLUX is supported. | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |
diff --git a/docs/algorithm_combinations_CN.md b/docs/algorithm_combinations_CN.md
index 01d9885d2..50d2bc076 100644
--- a/docs/algorithm_combinations_CN.md
+++ b/docs/algorithm_combinations_CN.md
@@ -9,11 +9,11 @@ AutoRound 可以在量化之前(或量化过程中)与多种算法结合使
## 图例
-| 信号灯 | 含义 |
-|:---:|:--------------------------|
-| 🟢 | 良好 —— 收益明显 / 可直接部署 |
-| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |
-| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |
+| 信号灯 | 含义 |
+|:------:|:--------------------------------------------|
+| 🟢 | 良好 —— 收益明显 / 可直接部署 |
+| 🟡 | 部分 —— 收益有条件 / 支持有限或处于实验阶段 |
+| 🔴 | 较差 —— 无明显收益 / 暂时无法部署 |
> **说明:** 「精度收益」被评为部分/较差可能有两方面原因:
> (1)当前AutoRound实现尚存在局限;(2)来自我们内部的主观评估。
@@ -21,12 +21,12 @@ AutoRound 可以在量化之前(或量化过程中)与多种算法结合使
## 矩阵
-| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |
-|:-----------------------------------|:----:|:----:|:----------------------------------------|:---------------------------------|:-------------------------------------|:-----------------------------------------------------|
-| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |
-| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |
-| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |
-| AutoRound + LFQ(logit 感知的末层块量化) | 🔴 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
-| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |
-| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |
+| 组合 | 精度收益 | 可部署性 | 详情 | 命令行用法 | 备注 | 参考文献 |
+|:------------------------------------------|:--------:|:--------:|:----------------------------------------|:---------------------------------|:---------------------------------------------------------|:-----------------------------------------------------|
+| AutoRound + AWQ(激活感知缩放) | 🟢 | 🟢 | [awq_details](awq_details.md) | `--algorithm awq,signround` | 在量化激活(如 W4A4)场景下推荐使用。 | [arXiv:2306.00978](https://arxiv.org/abs/2306.00978) |
+| AutoRound + Hadamard 旋转 | 🟢 | 🔴 | [rotation_details](rotation_details.md) | `--algorithm hadamard,signround` | 尤其适用于 INT4(W4A4)及部分 MXFP4 场景;暂无生产内核。 | [arXiv:2404.00456](https://arxiv.org/abs/2404.00456) |
+| AutoRound + SpinQuant | 🟡 | 🔴 | [rotation_details](rotation_details.md) | 仅 Python API | 需学习旋转矩阵,精度更高但有额外训练开销;暂无生产内核。 | [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) |
+| AutoRound + LFQ(logit 感知的末层块量化) | 🟡 | 🟢 | [lfq_acc](lfq_acc.md) | `--enable_lfq` | 优化末层块以提升低比特生成质量。 | [arXiv:2605.29756](https://arxiv.org/abs/2605.29756) |
+| AutoRound + MX Attention(MXFP4 变体) | 🟡 | 🔴 | [mxnv_acc](mxnv_acc.md) | `--data_type mx_fp4_rceil_v2` | 采用 7.25 作为 scale 计算的分母。 | [arXiv:2607.24377](https://arxiv.org/abs/2607.24377) |
+| AutoRound + SVDQuant(低秩离群值吸收) | 🟡 | 🟡 | [svdquant_details](svdquant_details.md) | `--algorithm svdquant,signround` | 推荐用于扩散模型;目前仅支持 FLUX。 | [arXiv:2411.05007](https://arxiv.org/abs/2411.05007) |
diff --git a/docs/lfq_acc.md b/docs/lfq_acc.md
index 79ffa6343..8216269da 100644
--- a/docs/lfq_acc.md
+++ b/docs/lfq_acc.md
@@ -1,4 +1,23 @@
+Qwen3-8B with signround iters 200
+### W2G64
+| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |
+| ----- | ----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: |
+| V1 | 0.4959 | 0.2596 | 0.4064 | 0.5331 | 0.3720 | 0.7016 | 0.8131 | 0.4437 | 0.4822 | 0.2740 | 0.7051 | 0.3280 | 0.6314 |
+| V1+lfq new | 0.5103 | 0.2547 | 0.4420 | 0.5570 | 0.4036 | 0.7344 | 0.7920 | 0.4556 | 0.5267 | 0.2860 | 0.7155 | 0.3182 | 0.6377 |
+| V2 | 0.5275 | 0.2921 | 0.5807 | 0.5739 | 0.4317 | 0.7529 | 0.7550 | 0.4495 | 0.5251 | 0.2740 | 0.7171 | 0.3280 | 0.6496 |
+| V2+lfq new | 0.5292| 0.2965 | 0.5595 | 0.5695 | 0.4369 | 0.7668 | 0.7636 | 0.4558 | 0.5255 | 0.2680 | 0.7193 | 0.3305 | 0.6590 |
+
+### W3G128
+
+| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |
+| ----- | ----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: |
+| V1 | 0.6198 | 0.4221 | 0.8423 | 0.6971 | 0.5213 | 0.8136 | 0.8648 | 0.5374 | 0.6220 | 0.3180 | 0.7573 | 0.3611 | 0.6803 |
+| V1+lfq new | 0.6151 | 0.4252 | 0.8150 | 0.6967 | 0.5188 | 0.8047 | 0.8676 | 0.5394 | 0.6179 | 0.3120 | 0.7557 | 0.3599 | 0.6685 |
+| V2 | 0.6230 | 0.4301 | 0.8696 | 0.6980 | 0.5119 | 0.8114 | 0.8563 | 0.5338 | 0.6447 | 0.3120 | 0.7601 | 0.3574 | 0.6906 |
+| V2+lfq new | 0.6253 | 0.4279 | 0.8560 | 0.7012 | 0.5299 | 0.8211 | 0.8615 | 0.5364 | 0.6297 | 0.3140 | 0.7639 | 0.3660 | 0.6961 |
+
+### legacy result with AutoRound version<=0.16.1
iters 200 enable_alg_ext is turned on Qwen3-8B
lfq causes large drop, maybe some bugs in our implementation