PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update readme (#242)
n1ck-guo5d92b25ccc0c
diff --git a/README.md b/README.mdindex 51b7c09ec..fe41b2ebb 100644--- a/README.md+++ b/README.md@@ -51,18 +51,6 @@ pip install auto-round## Model Quantization-### Basic Usage-AutoRound support Gaudi2, CPU and GPU. A user guide detailing the full list of supported arguments is provided by calling ```auto_round -h``` on the terminal. Alternatively, you can use ```auto-round``` instead of ```auto_round```. (**auto-round version > 0.3.0**)---```bash-auto_round --model facebook/opt-125m \- --bits 4 \- --group_size 128 \- --format auto_round \- --output_dir ./tmp_autoround-```-### API Usage```python@@ -89,6 +77,44 @@ output_dir = "./tmp_autoround"autoround.save_quantized(output_dir, format='auto_round', inplace=True)```+### Basic Usage+AutoRound support Gaudi2, CPU and GPU. A user guide detailing the full list of supported arguments is provided by calling ```auto_round -h``` on the terminal. Alternatively, you can use ```auto-round``` instead of ```auto_round```. (**auto-round version > 0.3.0**)+++```bash+auto_round --model facebook/opt-125m \+ --bits 4 \+ --group_size 128 \+ --format auto_round \+ --output_dir ./tmp_autoround+```+We provide two recipes for best accuracy and fast running speed with low memory. Details as below.+<details>+ <summary>Other Recipes</summary>++ ```bash+## best accuracy, 3X slower, low_gpu_mem_usage could save ~20G but ~30% slower+ auto_round --model facebook/opt-125m \+ --bits 4 \+ --group_size 128 \+ --nsamples 512 \+ --iters 1000 \+ --low_gpu_mem_usage+ ```++ ```bash+## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128+ auto_round --model facebook/opt-125m \+ --bits 4 \+ --group_size 128 \+ --nsamples 128 \+ --iters 200 \+ --seqlen 512 \+ --batch_size 4+ ```+</details>+<br>+<details><summary>Detailed Hyperparameters</summary>@@ -141,32 +167,7 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)</details><br>-We provide two recipes for best accuracy and fast running speed with low memory. Details as below.-<details>- <summary>Other Recipes</summary>- ```bash-## best accuracy, 3X slower, low_gpu_mem_usage could save ~20G but ~30% slower- auto_round --model facebook/opt-125m \- --bits 4 \- --group_size 128 \- --nsamples 512 \- --iters 1000 \- --low_gpu_mem_usage- ```-- ```bash-## fast and low memory, 2-3X speedup, slight accuracy drop at W4G128- auto_round --model facebook/opt-125m \- --bits 4 \- --group_size 128 \- --nsamples 128 \- --iters 200 \- --seqlen 512 \- --batch_size 4- ```--</details>#### Formats