Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

refine int4 doc (#1720)

Wenhua Cheng5b6f7ea7344f
diff --git a/README.md b/README.md
index dc7acfc8e..6ae8daf43 100644
--- a/README.md
+++ b/README.md
@@ -36,7 +36,7 @@ See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2]
## 🆕 What's New
-* [2026/04] We demonstrate that AutoRound achieves SOTA or near SOTA performance under INT4 (W4A4) quantization [accuracy data](./docs/int_acc.md). This capability is currently a **research-only** feature, with no production model export.
+* [2026/04] We demonstrate that AutoRound achieves SOTA or near SOTA performance under INT4 (W4A4) quantization [accuracy data](./docs/int4_acc.md). This capability is currently a **research-only** feature, with no production model export.
* [2026/03] **Block-wise FP8** quantization is available via `--scheme FP8_BLOCK --iters 0 --disable_opt_rtn`.
diff --git a/README_CN.md b/README_CN.md
index 72e99df18..fc3683cbc 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -35,7 +35,7 @@ AutoRound 是专为大语言模型(LLMs)和视觉-语言模型(VLMs)设
## 🆕 最新进展
-* [2026/04] AutoRound可以在INT4上取得非常好的精度[accuracy data](./docs/int_acc.md). 不过目前这个特性还停留在研究性质,我们正在努力将其优化到生产环境中。
+* [2026/04] AutoRound可以在INT4上取得非常好的精度 [accuracy data](./docs/int4_acc.md). 不过目前这个特性还停留在研究性质,我们正在努力将其优化到生产环境中。
* [2026/03] 支持**Block-wise FP8**,只需要将scheme改成 `--scheme FP8_BLOCK`.为了更快的量化,我们推荐使用RTN模式,即·--iters 0 --disable_opt_rtn·
diff --git a/docs/int_acc.md b/docs/int4_acc.md
similarity index 68%
rename from docs/int_acc.md
rename to docs/int4_acc.md
index e3f09a39c..3408a42fe 100644
--- a/docs/int_acc.md
+++ b/docs/int4_acc.md
@@ -1,34 +1,37 @@
-
### Algorithm Details
-| Alg | Rotation Type | Activation Clip Ratio | weight quantizer | w/a symmetric |
-|-----------|---------------|-----------------------|------------------|--------------|
-| Quarot | fixed | 0.9 | GPTQ | yes |
-| FlatQuant | learnable | learnable | GPTQ | yes |
-| Ours | fixed | 0.9 | SignRound | yes |
+| Alg | Rotation Type | Activation Clip Ratio | Weight Quantizer | W&A symmetric |
+|-----------|---------------|-----------------------|------------------|---------------|
+| Quarot | fixed | 0.9 | GPTQ | yes |
+| FlatQuant | learnable | learnable | GPTQ | yes |
+| Ours | fixed | 0.9 | SignRoundV2 | yes |
-### Eval Details.
+### Eval Details
-lm-eval 0.4.11 for all the data evaluation.
+lm-eval 0.4.11 for all the data evaluation.
-We report the average of 10 tasks (AVG10) as well as the individual performance on each task. The 10 tasks are: ARC-Challenge (arc_c), ARC-Easy (arc_e), BoolQ (boolq), HellaSwag (hellaswag), Lambada (lambada), MMLU (mmlu), OpenBookQA (obqa), PiQA (piqa), TriviaQA (tfq), and Winogrande (winog).
+We report the average of 10 tasks (AVG10) as well as the individual performance on each task. The 10 tasks are:
+ARC-Challenge (arc_c), ARC-Easy (arc_e), BoolQ (boolq), HellaSwag (hellaswag), Lambada-openai (lambada), MMLU (mmlu),
+OpenBookQA (obqa), PiQA (piqa), Truthfulqa_mc1 (tfq), and Winogrande (winog).
Following Quarot and FlatQuant, we report `acc_norm` instead of accuracy for tasks where acc_norm is available.
All results below `exclude` KV-cache quantization, as it is not yet supported in our implementation.
### How to Reproduce
+
~~~ bash
-AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadamard --format fake --tasks mmlu
+AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadamard --enable_alg_ext --format fake --tasks mmlu
~~~
+
### Qwen3-8B
| Method | AVG10 | arc_c | arc_e | boolq | hellaswag | lambada | mmlu | obqa | piqa | tfq | winog |
|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|
| 16 bit | 0.6598 | 0.5640 | 0.8068 | 0.8670 | 0.7491 | 0.6420 | 0.7288 | 0.420 | 0.7764 | 0.3647 | 0.6788 |
| QuaRot | 0.6123 | 0.5060 | 0.7428 | 0.8220 | 0.6836 | 0.5762 | 0.6545 | 0.412 | 0.7383 | 0.3464 | 0.6409 |
-| FlatQuant | 0.6511 | 0.5657 | 0.8072 | 0.8609 | 0.7256 | 0.6369 | 0.7032 | 0.410 | **0.7682** | 0.3525 | 0.6803 |
-| Ours | **0.6533** | **0.5614** | **0.8035** | **0.8657** | **0.7336** | **0.6336** | **0.7188** | **0.408** | 0.7704 | **0.3562** | **0.6819** |
+| FlatQuant | 0.6511 | **0.5657** | **0.8072** | 0.8609 | 0.7256 | **0.6369** | 0.7032 | **0.410** | 0.7682 | 0.3525 | 0.6803 |
+| Ours | **0.6533** | 0.5614 | 0.8035 | **0.8657** | **0.7336** | 0.6336 | **0.7188** | 0.408 | **0.7704** | **0.3562** | **0.6819** |
### Llama-3-8B
@@ -36,8 +39,8 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama
|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|
| 16 bit | 0.6569 | 0.5392 | 0.7769 | 0.8190 | 0.7919 | 0.7545 | 0.6216 | 0.452 | 0.8069 | 0.2717 | 0.7356 |
| QuaRot | 0.5924 | 0.4326 | 0.7155 | 0.7526 | 0.7342 | 0.6810 | 0.5007 | 0.426 | 0.7492 | 0.2546 | 0.6772 |
-| FlatQuant | 0.6442 | 0.5213 | 0.7706 | 0.8168 | 0.7724 | 0.7363 | 0.5875 | 0.438 | **0.7987** | **0.2754** | 0.7253 |
-| Ours | **0.6521** | **0.5452** | **0.7862** | **0.8119** | **0.7832** | **0.7460** | **0.6107** | **0.436** | 0.8079 | 0.2668 | **0.7269** |
+| FlatQuant | 0.6442 | 0.5213 | 0.7706 | **0.8168** | 0.7724 | 0.7363 | 0.5875 | **0.438** | 0.7987 | **0.2754** | 0.7253 |
+| Ours | **0.6521** | **0.5452** | **0.7862** | 0.8119 | **0.7832** | **0.7460** | **0.6107** | 0.436 | **0.8079** | 0.2668 | **0.7269** |
### Llama-2-7B
@@ -45,8 +48,8 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama
|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|
| 16 bit | 0.6071 | 0.4505 | 0.7382 | 0.7939 | 0.7615 | 0.7367 | 0.4177 | 0.442 | 0.7878 | 0.2485 | 0.6946 |
| QuaRot | 0.5742 | 0.4352 | 0.6907 | 0.7388 | 0.7275 | 0.7093 | 0.3527 | 0.402 | 0.7715 | 0.2509 | 0.6630 |
-| FlatQuant | 0.5954 | 0.4454 | 0.7239 | 0.7743 | 0.7452 | 0.7233 | **0.3920** | 0.430 | **0.7840** | 0.2448 | **0.6914** |
-| Ours | **0.5991** | **0.4403** | **0.7281** | **0.7902** | **0.7495** | **0.7256** | 0.3915 | **0.450** | 0.7802 | **0.2497** | 0.6859 |
+| FlatQuant | 0.5954 | **0.4454** | 0.7239 | 0.7743 | 0.7452 | 0.7233 | **0.3920** | 0.430 | **0.7840** | 0.2448 | **0.6914** |
+| Ours | **0.5991** | 0.4403 | **0.7281** | **0.7902** | **0.7495** | **0.7256** | 0.3915 | **0.450** | 0.7802 | **0.2497** | 0.6859 |
### Llama-2-13B
@@ -54,6 +57,6 @@ AR_ACT_SCALE=0.9 auto-round Qwen/Qwen3-8B --scheme INT4 --rotation_type hadama
|-----------|------------|------------|------------|------------|------------|------------|------------|-----------|------------|------------|------------|
| 16 bit | 0.6446 | 0.4881 | 0.7652 | 0.8217 | 0.7965 | 0.7652 | 0.5239 | 0.452 | 0.8030 | 0.2681 | 0.7621 |
| QuaRot | 0.6114 | 0.4573 | 0.7348 | 0.7872 | 0.7644 | 0.7444 | 0.4584 | 0.430 | 0.7818 | 0.2546 | 0.7009 |
-| FlatQuant | 0.6308 | 0.4795 | 0.7534 | 0.8180 | 0.7817 | 0.7588 | 0.4985 | 0.446 | **0.7982** | **0.2583** | 0.7151 |
-| Ours | **0.6369** | **0.4804** | **0.7626** | **0.8245** | **0.7886** | **0.7720** | **0.5106** | **0.450** | 0.8014 | 0.2497 | **0.7293** |
+| FlatQuant | 0.6308 | 0.4795 | 0.7534 | 0.8180 | 0.7817 | 0.7588 | 0.4985 | 0.446 | 0.7982 | **0.2583** | 0.7151 |
+| Ours | **0.6369** | **0.4804** | **0.7626** | **0.8245** | **0.7886** | **0.7720** | **0.5106** | **0.450** | **0.8014** | 0.2497 | **0.7293** |