PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update readme
wenhuach218330e2a083cc
diff --git a/README.md b/README.mdindex 6797d44cb..edbbc354d 100644--- a/README.md+++ b/README.md@@ -1,5 +1,5 @@-# AutoRound: Advanced Weight-Only Quantization Algorithm for a Broad Range of Models+# AutoRound: Advanced Weight-Only Quantization Algorithm for a Broad Range of LLM ModelsAutoRound is an advanced weight-only quantization algorithm, based on SignRound. It's tailored for a wide range of models and consistently delivers noticeable improvements, often significantly outperforming SignRound. However, it comes at the cost of approximately 2.5 times the tuning runtime.@@ -44,13 +44,13 @@ CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --nuCUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --num_bits 4 --group_size -1 --low_gpu_mem_usage --train_bs 1 --gradient_accumulate_steps 8```- **Utilizing the AdamW Optimizer:**-Include the flag `--adam`. Note that AdamW may be slightly less effective than Sign gradient descent in certain scenarios.+Include the flag `--adam`. Note that AdamW may be less effective than Sign gradient descent in many scenarios.- **Running the Original SignRound:**```bashCUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --num_bits 4 --group_size -1 --iters 400 --lr 0.0025 --minmax_lr 0.0025```-It's recommended to use `--enable_minmax_tuning`.+ `--enable_minmax_tuning` is strongly recommended@@ -63,7 +63,18 @@ Auto Rounding may encounter random issues with Qwen models.ChatGlm-V1 is not supported+We are working on exporting the quantized model to HF format++Cpu kernel will be supported soon+## Validated Models+++++++Mistral-7b doneLLaMAV1 donediff --git a/figs/W2G128.png b/figs/W2G128.pngnew file mode 100644index 000000000..1fdb363adBinary files /dev/null and b/figs/W2G128.png differdiff --git a/figs/W3G128.png b/figs/W3G128.pngnew file mode 100644index 000000000..c713fc6f5Binary files /dev/null and b/figs/W3G128.png differdiff --git a/figs/W4G-1.png b/figs/W4G-1.pngnew file mode 100644index 000000000..4180a4445Binary files /dev/null and b/figs/W4G-1.png differdiff --git a/figs/W4G128.png b/figs/W4G128.pngnew file mode 100644index 000000000..0771853f6Binary files /dev/null and b/figs/W4G128.png differ