PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
refine int4 doc (#1720)
Wenhua Cheng5b6f7ea7344f
diff --git a/README.md b/README.mdindex dc7acfc8e..6ae8daf43 100644--- a/README.md+++ b/README.md@@ -36,7 +36,7 @@ See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2]## 🆕 What's New-* [2026/04] We demonstrate that AutoRound achieves SOTA or near SOTA performance under INT4 (W4A4) quantization [accuracy data](./docs/int_acc.md). This capability is currently a **research-only** feature, with no production model export.+* [2026/04] We demonstrate that AutoRound achieves SOTA or near SOTA performance under INT4 (W4A4) quantization [accuracy data](./docs/int4_acc.md). This capability is currently a **research-only** feature, with no production model export.* [2026/03] **Block-wise FP8** quantization is available via `--scheme FP8_BLOCK --iters 0 --disable_opt_rtn`.diff --git a/README_CN.md b/README_CN.mdindex 72e99df18..fc3683cbc 100644--- a/README_CN.md+++ b/README_CN.md@@ -35,7 +35,7 @@ AutoRound 是专为大语言模型(LLMs)和视觉-语言模型(VLMs)设## 🆕 最新进展-* [2026/04] AutoRound可以在INT4上取得非常好的精度[accuracy data](./docs/int_acc.md). 不过目前这个特性还停留在研究性质,我们正在努力将其优化到生产环境中。+* [2026/04] AutoRound可以在INT4上取得非常好的精度 [accuracy data](./docs/int4_acc.md). 不过目前这个特性还停留在研究性质,我们正在努力将其优化到生产环境中。* [2026/03] 支持**Block-wise FP8**,只需要将scheme改成 `--scheme FP8_BLOCK`.为了更快的量化,我们推荐使用RTN模式,即·--iters 0 --disable_opt_rtn·diff --git a/docs/int_acc.md b/docs/int4_acc.mdsimilarity index 68%rename from docs/int_acc.mdrename to docs/int4_acc.mdindex e3f09a39c..3408a42fe 100644--- a/docs/int_acc.md+++ b/docs/int4_acc.md@@ -1,34 +1,37 @@-### Algorithm Details-| Alg | Rotation Type | Activation Clip Ratio | weight quantizer | w/a symmetric |-|-----------|---------------|-----------------------|------------------|--------------|-| Quarot | fixed | 0.9 | GPTQ | yes |-| FlatQuant | learnable | learnable | GPTQ | yes |-| Ours | fixed | 0.9 | SignRound | yes |+| Alg | Rotation Type | Activation Clip Ratio | Weight Quantizer | W&A symmetric |+|-----------|---------------|-----------------------|------------------|---------------|+| Quarot | fixed | 0.9 | GPTQ | yes |+| FlatQuant | learnable | learnable | GPTQ | yes |+| Ours | fixed | 0.9 | SignRoundV2 | yes |-### Eval Details.+### Eval Details-lm-eval 0.4.11 for all the data evaluation.+lm-eval 0.4.11 for all the data evaluation.-We report the average of 10 tasks (AVG10) as well as the individual performance on each task. The 10 tasks are: ARC-Challenge (arc_c), ARC-Easy (arc_e), BoolQ (boolq), HellaSwag (hellaswag), Lambada (lambada), MMLU (mmlu), OpenBookQA (obqa), PiQA (piqa), TriviaQA (tfq), and Winogrande (winog).+We report the average of 10 tasks (AVG10) as well as the individual performance on each task. The 10 tasks are:+ARC-Challenge (arc_c), ARC-Easy (arc_e), BoolQ (boolq), HellaSwag (hellaswag), Lambada-openai (lambada), MMLU (mmlu),+OpenBookQA (obqa), PiQA (piqa), Truthfulqa_mc1 (tfq), and Winogrande (winog).Following Quarot and FlatQuant, we report `acc_norm` instead of accuracy for tasks where acc_norm is available.All results below `exclude` KV-cache quantization, as it is not yet supported in our implementation.### How to Reproduce+~~~ bash-AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadamard --format fake --tasks mmlu+AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadamard --enable_alg_ext --format fake --tasks mmlu~~~+### Qwen3-8B| Method | AVG10 | arc_c | arc_e | boolq | hellaswag | lambada | mmlu | obqa | piqa | tfq | winog ||-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|| 16 bit | 0.6598 | 0.5640 | 0.8068 | 0.8670 | 0.7491 | 0.6420 | 0.7288 | 0.420 | 0.7764 | 0.3647 | 0.6788 || QuaRot | 0.6123 | 0.5060 | 0.7428 | 0.8220 | 0.6836 | 0.5762 | 0.6545 | 0.412 | 0.7383 | 0.3464 | 0.6409 |-| FlatQuant | 0.6511 | 0.5657 | 0.8072 | 0.8609 | 0.7256 | 0.6369 | 0.7032 | 0.410 | **0.7682** | 0.3525 | 0.6803 |-| Ours | **0.6533** | **0.5614** | **0.8035** | **0.8657** | **0.7336** | **0.6336** | **0.7188** | **0.408** | 0.7704 | **0.3562** | **0.6819** |+| FlatQuant | 0.6511 | **0.5657** | **0.8072** | 0.8609 | 0.7256 | **0.6369** | 0.7032 | **0.410** | 0.7682 | 0.3525 | 0.6803 |+| Ours | **0.6533** | 0.5614 | 0.8035 | **0.8657** | **0.7336** | 0.6336 | **0.7188** | 0.408 | **0.7704** | **0.3562** | **0.6819** |### Llama-3-8B@@ -36,8 +39,8 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|| 16 bit | 0.6569 | 0.5392 | 0.7769 | 0.8190 | 0.7919 | 0.7545 | 0.6216 | 0.452 | 0.8069 | 0.2717 | 0.7356 || QuaRot | 0.5924 | 0.4326 | 0.7155 | 0.7526 | 0.7342 | 0.6810 | 0.5007 | 0.426 | 0.7492 | 0.2546 | 0.6772 |-| FlatQuant | 0.6442 | 0.5213 | 0.7706 | 0.8168 | 0.7724 | 0.7363 | 0.5875 | 0.438 | **0.7987** | **0.2754** | 0.7253 |-| Ours | **0.6521** | **0.5452** | **0.7862** | **0.8119** | **0.7832** | **0.7460** | **0.6107** | **0.436** | 0.8079 | 0.2668 | **0.7269** |+| FlatQuant | 0.6442 | 0.5213 | 0.7706 | **0.8168** | 0.7724 | 0.7363 | 0.5875 | **0.438** | 0.7987 | **0.2754** | 0.7253 |+| Ours | **0.6521** | **0.5452** | **0.7862** | 0.8119 | **0.7832** | **0.7460** | **0.6107** | 0.436 | **0.8079** | 0.2668 | **0.7269** |### Llama-2-7B@@ -45,8 +48,8 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|| 16 bit | 0.6071 | 0.4505 | 0.7382 | 0.7939 | 0.7615 | 0.7367 | 0.4177 | 0.442 | 0.7878 | 0.2485 | 0.6946 || QuaRot | 0.5742 | 0.4352 | 0.6907 | 0.7388 | 0.7275 | 0.7093 | 0.3527 | 0.402 | 0.7715 | 0.2509 | 0.6630 |-| FlatQuant | 0.5954 | 0.4454 | 0.7239 | 0.7743 | 0.7452 | 0.7233 | **0.3920** | 0.430 | **0.7840** | 0.2448 | **0.6914** |-| Ours | **0.5991** | **0.4403** | **0.7281** | **0.7902** | **0.7495** | **0.7256** | 0.3915 | **0.450** | 0.7802 | **0.2497** | 0.6859 |+| FlatQuant | 0.5954 | **0.4454** | 0.7239 | 0.7743 | 0.7452 | 0.7233 | **0.3920** | 0.430 | **0.7840** | 0.2448 | **0.6914** |+| Ours | **0.5991** | 0.4403 | **0.7281** | **0.7902** | **0.7495** | **0.7256** | 0.3915 | **0.450** | 0.7802 | **0.2497** | 0.6859 |### Llama-2-13B@@ -54,6 +57,6 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|| 16 bit | 0.6446 | 0.4881 | 0.7652 | 0.8217 | 0.7965 | 0.7652 | 0.5239 | 0.452 | 0.8030 | 0.2681 | 0.7621 || QuaRot | 0.6114 | 0.4573 | 0.7348 | 0.7872 | 0.7644 | 0.7444 | 0.4584 | 0.430 | 0.7818 | 0.2546 | 0.7009 |-| FlatQuant | 0.6308 | 0.4795 | 0.7534 | 0.8180 | 0.7817 | 0.7588 | 0.4985 | 0.446 | **0.7982** | **0.2583** | 0.7151 |-| Ours | **0.6369** | **0.4804** | **0.7626** | **0.8245** | **0.7886** | **0.7720** | **0.5106** | **0.450** | 0.8014 | 0.2497 | **0.7293** |+| FlatQuant | 0.6308 | 0.4795 | 0.7534 | 0.8180 | 0.7817 | 0.7588 | 0.4985 | 0.446 | 0.7982 | **0.2583** | 0.7151 |+| Ours | **0.6369** | **0.4804** | **0.7626** | **0.8245** | **0.7886** | **0.7720** | **0.5106** | **0.450** | **0.8014** | 0.2497 | **0.7293** |