Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

update readme (#242)

n1ck-guo5d92b25ccc0c
diff --git a/README.md b/README.md
index 51b7c09ec..fe41b2ebb 100644
--- a/README.md
+++ b/README.md
@@ -51,18 +51,6 @@ pip install auto-round
## Model Quantization
-### Basic Usage
-AutoRound support Gaudi2, CPU and GPU. A user guide detailing the full list of supported arguments is provided by calling ```auto_round -h``` on the terminal. Alternatively, you can use ```auto-round``` instead of ```auto_round```. (**auto-round version > 0.3.0**)
-
-
-```bash
-auto_round --model facebook/opt-125m \
- --bits 4 \
- --group_size 128 \
- --format auto_round \
- --output_dir ./tmp_autoround
-```
-
### API Usage
```python
@@ -89,6 +77,44 @@ output_dir = "./tmp_autoround"
autoround.save_quantized(output_dir, format='auto_round', inplace=True)
```
+### Basic Usage
+AutoRound support Gaudi2, CPU and GPU. A user guide detailing the full list of supported arguments is provided by calling ```auto_round -h``` on the terminal. Alternatively, you can use ```auto-round``` instead of ```auto_round```. (**auto-round version > 0.3.0**)
+
+
+```bash
+auto_round --model facebook/opt-125m \
+ --bits 4 \
+ --group_size 128 \
+ --format auto_round \
+ --output_dir ./tmp_autoround
+```
+We provide two recipes for best accuracy and fast running speed with low memory. Details as below.
+<details>
+ <summary>Other Recipes</summary>
+
+ ```bash
+## best accuracy, 3X slower, low_gpu_mem_usage could save ~20G but ~30% slower
+ auto_round --model facebook/opt-125m \
+ --bits 4 \
+ --group_size 128 \
+ --nsamples 512 \
+ --iters 1000 \
+ --low_gpu_mem_usage
+ ```
+
+ ```bash
+## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128
+ auto_round --model facebook/opt-125m \
+ --bits 4 \
+ --group_size 128 \
+ --nsamples 128 \
+ --iters 200 \
+ --seqlen 512 \
+ --batch_size 4
+ ```
+</details>
+<br>
+
<details>
<summary>Detailed Hyperparameters</summary>
@@ -141,32 +167,7 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)
</details>
<br>
-We provide two recipes for best accuracy and fast running speed with low memory. Details as below.
-<details>
- <summary>Other Recipes</summary>
- ```bash
-## best accuracy, 3X slower, low_gpu_mem_usage could save ~20G but ~30% slower
- auto_round --model facebook/opt-125m \
- --bits 4 \
- --group_size 128 \
- --nsamples 512 \
- --iters 1000 \
- --low_gpu_mem_usage
- ```
-
- ```bash
-## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128
- auto_round --model facebook/opt-125m \
- --bits 4 \
- --group_size 128 \
- --nsamples 128 \
- --iters 200 \
- --seqlen 512 \
- --batch_size 4
- ```
-
-</details>
#### Formats