PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
refine readme (#1063)
Wenhua Cheng2102257e9256
diff --git a/README.md b/README.mdindex a6289e2f9..b12b62731 100644--- a/README.md+++ b/README.md@@ -20,9 +20,9 @@## 🚀 What is AutoRound?-AutoRound is an advanced quantization library designed for Large Language Models (LLMs) and Vision-Language Models (VLMs).-It delivers high accuracy at ultra-low bit widths (2–4 bits) with minimal tuning by leveraging sign-gradient descent and offering broad hardware compatibility.-See our [paper](https://arxiv.org/pdf/2309.05516) for more details. For usage instructions, please refer to [User Guide](./docs/step_by_step.md).+AutoRound is an advanced quantization toolkit designed for Large Language Models (LLMs) and Vision-Language Models (VLMs).+It achieves high accuracy at ultra-low bit widths (2–4 bits) with minimal tuning by leveraging sign-gradient descent and providing broad hardware compatibility.+See our [paper](https://arxiv.org/pdf/2309.05516) for more details. For usage instructions, please refer to the [User Guide](./docs/step_by_step.md).<p align="center"><img src="docs/imgs/autoround_overview.png" alt="AutoRound Overview" width="80%">@@ -34,7 +34,7 @@ See our [paper](https://arxiv.org/pdf/2309.05516) for more details. For usage in[2025/11] AutoRound now offers preliminary support for an enhanced GGUF quantization algorithm via `--enable_alg_ext`. For detailed accuracy benchmarks, please refer to the [documentation](./docs/gguf_alg_ext_acc.md).-[2025/10] AutoRound has been integrated into **SGLang**. You can now run models in the AutoRound format directly using the latest SGLang later than v0.5.4.+[2025/10] AutoRound has been integrated into **SGLang**. You can now run models in the AutoRound format directly using the SGLang versions newer than v0.5.4.[2025/10] We enhanced the RTN mode (--iters 0) to significantly reduce quantization cost compared to the default tuning mode. Check out [this doc](./docs/opt_rtn.md) for some accuracy results. If you don’t have sufficient resources, you can use this mode for 4-bit quantization.@@ -212,13 +212,13 @@ ar.quantize_and_save(output_dir="./qmodel", format="auto_round")##### Device/Speed Configuration- **`enable_torch_compile` (bool)**: If no exception is raised, typically we recommend setting it to True for faster quantization with lower resource.- **`low_gpu_mem_usage` (bool)**: Whether to offload intermediate features to CPU at the cost of ~20% more tuning time (default is `False`).-- **`low_cpu_mem_usage` (bool)**: [Experimental Feature]Whether to enable saving immediately to save ram usage (default is `False`).+- **`low_cpu_mem_usage` (bool)**: [Experimental Feature]Whether to enable saving immediately to reduce ram usage (default is `False`).- **`device_map` (str|dict|int)**: The device to be used for tuning, e.g., `auto`, "cpu"`, `"cuda"`, `"0,1,2"` (default is `'0'`). When using "auto", it will try to use all available GPUs.</details>### Adaptive Bits/Dtype Usage-AutoScheme provide automatically algorithm to provide mixed bits/data_type quantization recipes. For some accuracy result, please refer to this [doc](https://github.com/intel/auto-round/blob/main/docs/auto_scheme_acc.md).+AutoScheme provides an automatic algorithm to generate adaptive mixed bits/data-type quantization recipes.Please refer to the [user guide](https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme) for more details on AutoScheme.~~~pythonfrom auto_round import AutoRound, AutoScheme@@ -249,7 +249,7 @@ ar.quantize_and_save()### API Usage for VLMs-If you encounter issues during quantization, try setting iters=0 (to enable RTN) and use group_size=32 for better+If you encounter issues during quantization, try setting iters=0 (to enable RTN) and group_size=32 for betterresults.@@ -320,7 +320,7 @@ for prompt, output in zip(prompts, outputs):### Transformers (CPU/Intel GPU/Gaudi/CUDA)-AutoRound support 10+ backends and automatically selects the best available backend based on the installed libraries and prompts the user to+AutoRound supports 10+ backends and automatically selects the best available backend based on the installed libraries and prompts the user toinstall additional libraries when a better backend is found.**Please avoid manually moving the quantized model to a different device** (e.g., model.to('cpu')) during inference, asdiff --git a/docs/step_by_step.md b/docs/step_by_step.mdindex e8c1dca8d..c33ca3701 100644--- a/docs/step_by_step.md+++ b/docs/step_by_step.md@@ -284,7 +284,7 @@ W2G64 Average Accuracy of 13 tasks and Time Cost Results(Testing was conducted o### AutoScheme-AutoScheme provide automatically algorithm to provide mixed bits/data_type quantization recipes. For some accuracy result, please refer this doc [here](./auto_scheme_acc.md)+AutoScheme provides an automatic algorithm to generate adaptive mixed bits/data-type quantization recipes. For some accuracy result, please refer this doc [here](./auto_scheme_acc.md)**Please note that mixed data types are supported during tuning, but cannot be exported to real models at this time..**@@ -559,7 +559,7 @@ autoround.save_quantized(format="auto_awq", output_dir="tmp_autoround")- **Reduced CPU Memory Usage :**- - Enable low_cpu_mem_usage (experimental): Only one export format is supported. The quantized model is saved immediately after each block is packed, reducing peak CPU memory usage.+ - Enable `low_cpu_mem_usage` (experimental): Only one export format is supported. The quantized model is saved immediately after each block is packed, reducing peak CPU memory usage.- Trigger immediate packing: Packing will be triggered immediately when using the command-line interface or thequantize_and_save API, as long as only one export format is specified.