PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update 2bits acc results (#491)
WeiweiZhang1226d2bfc9665
diff --git a/README.md b/README.mdindex 91b4c5f85..3868178fa 100644--- a/README.md+++ b/README.md@@ -6,7 +6,7 @@ AutoRound<h3> Advanced Quantization Algorithm for LLMs</h3>[](https://github.com/intel/auto-round)-[](https://github.com/intel/auto-round)+[](https://github.com/intel/auto-round)[](https://github.com/intel/auto-round/blob/main/LICENSE)<a href="https://huggingface.co/OPEA"><img alt="Model Checkpoints" src="https://img.shields.io/badge/%F0%9F%A4%97%20HF-Models-F57C00">@@ -114,8 +114,8 @@ auto-round-light \```<!-- ```bash-## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128auto-round-fast \+## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128--model facebook/opt-125m \--bits 4 \--group_size 128 \@@ -137,8 +137,17 @@ W4G128 Average Accuracy of 13 tasks and Time Cost Results(Testing was conducted| Light | 0.4052(2m) | 0.5108(3m) | **0.6453**(5m) | 0.6104(6m) | 0.6063(6m) | 0.7243(37m) |--+<details>+ <summary>W2G64 results</summary>+W2G64 Average Accuracy of 13 tasks and Time Cost Results(Testing was conducted on the Nvidia A100 80G using the version of PyTorch 2.6.0 with enable_torch_compile). We recommend using higher precision for the head, tail, and non-expert modules to alleviate the significant accuracy drop.++| Model | Qwen2.5-0.5B-Instruct | Falcon3-3B | Qwen2.5-7B-Instruct | Falcon3-10B | Qwen2.5-72B-Instruct |+ |---------|-----------------------|----------------------|---------------------|----------------------|-----------------------|+ | 16bits | 0.4192 | 0.5203 | 0.6470 | 0.6151 | 0.7229 |+ | Best | **0.2989**(6m) | **0.4267**(24m) | **0.5343**(56m)| **0.5207**(79m) | **0.6715**(564m) |+ | Default | 0.2878(2m) | 0.4219(6m) | 0.5209(13m) | 0.5133(18m) | 0.6713(122m) |+ | Light | 0.2760(2m) | 0.4063(3m) | 0.4764(5m) | 0.4810(7m) | 0.6581(38m) |+</details>### API Usage (Gaudi2/CPU/GPU)@@ -451,3 +460,4 @@ If you find AutoRound useful for your research, please cite our paper:+