PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
restrict accelerate version to reduce ram usage (#673)
Wenhua Cheng308e5e363b2d
diff --git a/.azure-pipelines/unit-test-hpu.yml b/.azure-pipelines/unit-test-hpu.ymlindex 212699241..8eb4c6999 100644--- a/.azure-pipelines/unit-test-hpu.yml+++ b/.azure-pipelines/unit-test-hpu.yml@@ -11,7 +11,7 @@ pr:- auto_round- test/test*hpu*'- setup.py- - requirements-hpu.txt+ - requirements-lib.txt- .azure-pipelines/scripts/ut- .azure-pipelines/template/docker-template.yml- .azure-pipelines/template/ut-template.ymldiff --git a/.azure-pipelines/unit-test.yml b/.azure-pipelines/unit-test.ymlindex 42ba44c9f..c4cc085cb 100644--- a/.azure-pipelines/unit-test.yml+++ b/.azure-pipelines/unit-test.yml@@ -13,6 +13,7 @@ pr:- test- setup.py- requirements.txt+ - requirements-cpu.txt- .azure-pipelines/scripts/ut- .azure-pipelines/unit-test.yml- .azure-pipelines/template/ut-template.ymldiff --git a/README.md b/README.mdindex 7e520b1f3..27e9e082d 100644--- a/README.md+++ b/README.md@@ -8,7 +8,7 @@ AutoRound[](https://github.com/intel/auto-round)[](https://github.com/intel/auto-round)[](https://github.com/intel/auto-round/blob/main/LICENSE)-<a href="https://huggingface.co/OPEA">+<a href="https://huggingface.co/Intel"><img alt="Model Checkpoints" src="https://img.shields.io/badge/%F0%9F%A4%97%20HF-Models-F57C00"></a>---@@ -23,7 +23,7 @@ AutoRound also offers a variety of useful features, including mixed-bit tuning asupport for exporting to formats like GPTQ/AWQ/GGUF, and flexible tuning recipes. The belowimage presents an overview of AutoRound. Check out our paper on [arxiv](https://arxiv.org/pdf/2309.05516) for moredetails and quantized models in several Hugging Face Spaces,-e.g. [OPEA](https://huggingface.co/OPEA), [Intel](https://huggingface.co/Intel), [Kaitchup](https://huggingface.co/kaitchup)+e.g. [Intel](https://huggingface.co/Intel), [OPEA](https://huggingface.co/OPEA), [Kaitchup](https://huggingface.co/kaitchup)and [fbaldassarri](https://huggingface.co/fbaldassarri).<div align="center">@@ -275,6 +275,9 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)**AutoRound Format**: This format is well-suited for CPU, HPU devices, 2 bits, as well as mixed-precisioninference. **[2,3,4,8] bits are supported**.+**GGUF** Format: Experimental feature. This format is well-suited for CPU devices and is widely adopted by the+community. `q*_k`,`q*_0`,`q*_1` are supported.+**AutoGPTQ Format**: This format is well-suited for symmetric quantization on CUDA devices and is widely adopted by thecommunity, **[2,3,4,8] bits are supported**. However, **theasymmetric kernel has issues** that can cause considerable accuracy drops, particularly at 2-bit quantization and small@@ -286,8 +289,7 @@ adopted within the community, **only 4-bits quantization is supported**.**llmcompressor Format**: This format is for reusing llmcompressor format, **only INT8 W8A8 dynamic quantization issupported**.-**GGUF** Format: Experimental feature. This format is well-suited for CPU devices and is widely adopted by the-community.+### Quantization Costsdiff --git a/requirements-cpu.txt b/requirements-cpu.txtindex 70a8eccec..037967b7e 100644--- a/requirements-cpu.txt+++ b/requirements-cpu.txt@@ -1,2 +1 @@intel-extension-for-pytorch-intel-extension-for-transformersdiff --git a/requirements-lib.txt b/requirements-lib.txtindex 47b287e00..02e39eb9b 100644--- a/requirements-lib.txt+++ b/requirements-lib.txt@@ -1,4 +1,4 @@-accelerate+accelerate <= 1.5.1datasetspy-cpuinfosentencepiece