Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

update readme

wenhuach218330e2a083cc
diff --git a/README.md b/README.md
index 6797d44cb..edbbc354d 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,5 @@
-# AutoRound: Advanced Weight-Only Quantization Algorithm for a Broad Range of Models
+# AutoRound: Advanced Weight-Only Quantization Algorithm for a Broad Range of LLM Models
AutoRound is an advanced weight-only quantization algorithm, based on SignRound. It's tailored for a wide range of models and consistently delivers noticeable improvements, often significantly outperforming SignRound. However, it comes at the cost of approximately 2.5 times the tuning runtime.
@@ -44,13 +44,13 @@ CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --nu
CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --num_bits 4 --group_size -1 --low_gpu_mem_usage --train_bs 1 --gradient_accumulate_steps 8
```
- **Utilizing the AdamW Optimizer:**
-Include the flag `--adam`. Note that AdamW may be slightly less effective than Sign gradient descent in certain scenarios.
+Include the flag `--adam`. Note that AdamW may be less effective than Sign gradient descent in many scenarios.
- **Running the Original SignRound:**
```bash
CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --num_bits 4 --group_size -1 --iters 400 --lr 0.0025 --minmax_lr 0.0025
```
-It's recommended to use `--enable_minmax_tuning`.
+ `--enable_minmax_tuning` is strongly recommended
@@ -63,7 +63,18 @@ Auto Rounding may encounter random issues with Qwen models.
ChatGlm-V1 is not supported
+We are working on exporting the quantized model to HF format
+
+Cpu kernel will be supported soon
+
## Validated Models
+
+![](./figs/W4G-1.png)
+
+![](./figs/W4G128.png)
+![](./figs/W3G128.png)
+![](./figs/W2G128.png)
+
Mistral-7b done
LLaMAV1 done
diff --git a/figs/W2G128.png b/figs/W2G128.png
new file mode 100644
index 000000000..1fdb363ad
Binary files /dev/null and b/figs/W2G128.png differ
diff --git a/figs/W3G128.png b/figs/W3G128.png
new file mode 100644
index 000000000..c713fc6f5
Binary files /dev/null and b/figs/W3G128.png differ
diff --git a/figs/W4G-1.png b/figs/W4G-1.png
new file mode 100644
index 000000000..4180a4445
Binary files /dev/null and b/figs/W4G-1.png differ
diff --git a/figs/W4G128.png b/figs/W4G128.png
new file mode 100644
index 000000000..0771853f6
Binary files /dev/null and b/figs/W4G128.png differ