PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
rm redundant line break (#475)
WeiweiZhang1050594f1d919
diff --git a/README.md b/README.mdindex 31b4cb4a8..a60a684c2 100644--- a/README.md+++ b/README.md@@ -70,7 +70,6 @@ pip install auto-round-lib```</details>-<br>## Model Quantization@@ -124,11 +123,11 @@ auto-round-fast \``` --></details>-<br>-In conclusion, we recommend using **auto-round for INT4 and auto-round-best for INT2**. However, you may adjust the configuration to suit your specific requirements and available resources.+In conclusion, we recommend using **auto-round for INT4 and auto-round-best for INT2**. However, you may adjust the configuration to suit your specific requirements and available resources.++W4G128 Average Accuracy of 13 tasks and Time Cost Results(Testing was conducted on the Nvidia A100 80G using the version of PyTorch 2.6.0 with enable_torch_compile):-Average Accuracy of 13 tasks(W4G128) and Time Cost(enable_torch_compile) Results| Model | Qwen2.5-0.5B-Instruct | Falcon3-3B | Qwen2.5-7B-Instruct | Falcon3-10B | Qwen2.5-72B-Instruct ||---------|-----------------------|----------------------|---------------------|----------------------|-----------------------|@@ -138,7 +137,8 @@ Average Accuracy of 13 tasks(W4G128) and Time Cost(enable_torch_compile) Results| Light | 0.4052(2m) | 0.5108(3m) | **0.6453**(5m) | 0.6063(6m) | 0.7243(37m) |-<br>++### API Usage (Gaudi2/CPU/GPU)@@ -217,7 +217,6 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)- `device`: The device to be used for tuning. The default is set to 'auto', allowing for automatic detection.</details>-<br>### API Usage for VLMs@@ -256,7 +255,7 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)```</details>-<br>+### Export Formats**AutoRound Format**: This format is well-suited for CPU, HPU devices, 2 bits, as well as mixed-precision@@ -273,7 +272,7 @@ adopted within the community, **only 4-bits quantization is supported**.**GGUF** Format: This format is well-suited for CPU devices and is widely adopted by the community, **only q4_0 andq4_1 (W4G32) is supported in our repo**.-<br>+### Quantization Costs@@ -330,7 +329,6 @@ inputs = tokenizer(text, return_tensors="pt").to(model.device)print(tokenizer.decode(model.generate(**inputs, max_new_tokens=50)[0]))```-<br>#### Evaluation<details>@@ -344,7 +342,7 @@ auto-round --model saved_quantized_model \```</details>-<br>+### AutoGPTQ/AutoAWQ format@@ -420,7 +418,7 @@ release most of the models ourselves.</details>-<br>+## Integration@@ -432,7 +430,7 @@ AutoRound has been integrated into multiple repositories.[pytorch/ao](https://github.com/pytorch/ao)-<br>+## Reference@@ -451,3 +449,4 @@ If you find AutoRound useful for your research, please cite our paper:+