Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

restrict accelerate version to reduce ram usage (#673)

Wenhua Cheng308e5e363b2d
diff --git a/.azure-pipelines/unit-test-hpu.yml b/.azure-pipelines/unit-test-hpu.yml
index 212699241..8eb4c6999 100644
--- a/.azure-pipelines/unit-test-hpu.yml
+++ b/.azure-pipelines/unit-test-hpu.yml
@@ -11,7 +11,7 @@ pr:
- auto_round
- test/test*hpu*'
- setup.py
- - requirements-hpu.txt
+ - requirements-lib.txt
- .azure-pipelines/scripts/ut
- .azure-pipelines/template/docker-template.yml
- .azure-pipelines/template/ut-template.yml
diff --git a/.azure-pipelines/unit-test.yml b/.azure-pipelines/unit-test.yml
index 42ba44c9f..c4cc085cb 100644
--- a/.azure-pipelines/unit-test.yml
+++ b/.azure-pipelines/unit-test.yml
@@ -13,6 +13,7 @@ pr:
- test
- setup.py
- requirements.txt
+ - requirements-cpu.txt
- .azure-pipelines/scripts/ut
- .azure-pipelines/unit-test.yml
- .azure-pipelines/template/ut-template.yml
diff --git a/README.md b/README.md
index 7e520b1f3..27e9e082d 100644
--- a/README.md
+++ b/README.md
@@ -8,7 +8,7 @@ AutoRound
[![python](https://img.shields.io/badge/python-3.9%2B-blue)](https://github.com/intel/auto-round)
[![version](https://img.shields.io/badge/release-0.5.1-green)](https://github.com/intel/auto-round)
[![license](https://img.shields.io/badge/license-Apache%202-9C27B0)](https://github.com/intel/auto-round/blob/main/LICENSE)
-<a href="https://huggingface.co/OPEA">
+<a href="https://huggingface.co/Intel">
<img alt="Model Checkpoints" src="https://img.shields.io/badge/%F0%9F%A4%97%20HF-Models-F57C00">
</a>
---
@@ -23,7 +23,7 @@ AutoRound also offers a variety of useful features, including mixed-bit tuning a
support for exporting to formats like GPTQ/AWQ/GGUF, and flexible tuning recipes. The below
image presents an overview of AutoRound. Check out our paper on [arxiv](https://arxiv.org/pdf/2309.05516) for more
details and quantized models in several Hugging Face Spaces,
-e.g. [OPEA](https://huggingface.co/OPEA), [Intel](https://huggingface.co/Intel), [Kaitchup](https://huggingface.co/kaitchup)
+e.g. [Intel](https://huggingface.co/Intel), [OPEA](https://huggingface.co/OPEA), [Kaitchup](https://huggingface.co/kaitchup)
and [fbaldassarri](https://huggingface.co/fbaldassarri).
<div align="center">
@@ -275,6 +275,9 @@ autoround.save_quantized(output_dir, format='auto_round', inplace=True)
**AutoRound Format**: This format is well-suited for CPU, HPU devices, 2 bits, as well as mixed-precision
inference. **[2,3,4,8] bits are supported**.
+**GGUF** Format: Experimental feature. This format is well-suited for CPU devices and is widely adopted by the
+community. `q*_k`,`q*_0`,`q*_1` are supported.
+
**AutoGPTQ Format**: This format is well-suited for symmetric quantization on CUDA devices and is widely adopted by the
community, **[2,3,4,8] bits are supported**. However, **the
asymmetric kernel has issues** that can cause considerable accuracy drops, particularly at 2-bit quantization and small
@@ -286,8 +289,7 @@ adopted within the community, **only 4-bits quantization is supported**.
**llmcompressor Format**: This format is for reusing llmcompressor format, **only INT8 W8A8 dynamic quantization is
supported**.
-**GGUF** Format: Experimental feature. This format is well-suited for CPU devices and is widely adopted by the
-community.
+
### Quantization Costs
diff --git a/requirements-cpu.txt b/requirements-cpu.txt
index 70a8eccec..037967b7e 100644
--- a/requirements-cpu.txt
+++ b/requirements-cpu.txt
@@ -1,2 +1 @@
intel-extension-for-pytorch
-intel-extension-for-transformers
diff --git a/requirements-lib.txt b/requirements-lib.txt
index 47b287e00..02e39eb9b 100644
--- a/requirements-lib.txt
+++ b/requirements-lib.txt
@@ -1,4 +1,4 @@
-accelerate
+accelerate <= 1.5.1
datasets
py-cpuinfo
sentencepiece