PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update readme and shell file
wenhuach216c00de1a1840
diff --git a/README.md b/README.mdindex 112d61942..b2aff5184 100644--- a/README.md+++ b/README.md@@ -48,7 +48,7 @@ autoround.export(output_dir)```-### On GPU+### On CPU/GPU```pythonfrom transformers import AutoModelForCausalLM, AutoTokenizerdiff --git a/examples/code-generation/run_autoround.sh b/examples/code-generation/run_autoround.shindex 8993a8665..97880ef7e 100644--- a/examples/code-generation/run_autoround.sh+++ b/examples/code-generation/run_autoround.sh@@ -1,6 +1,6 @@-device=0#!/bin/bashset -x+device=0model_name="Salesforce/codegen25-7b-multi"CUDA_VISIBLE_DEVICES=$device \diff --git a/examples/language-modeling/run_autoround.sh b/examples/language-modeling/run_autoround.shindex ceabc30ca..a5fcb1e94 100644--- a/examples/language-modeling/run_autoround.sh+++ b/examples/language-modeling/run_autoround.sh@@ -1,17 +1,19 @@#!/bin/bashset -x+device=0+model_name="Intel/neural-chat-7b-v3"-model="Intel/neural-chat-7b-v3"--CUDA_VISIBLE_DEVICES=4 python3 main.py \- --model_name /models/${model} \- --bits 4 --group_size 128 \- --enable_minmax_tuning \- --use_quant_input \- --amp \- --iters 200 \- --deployment_device 'cpu' \- --scale_dtype 'fp32' \- --eval_bs 16 \- --output_dir "./tmp_signround"+CUDA_VISIBLE_DEVICES=$device \+python3 main.py \+--model_name $model_name \+--bits 4 \+--group_size 128 \+--enable_minmax_tuning \+--use_quant_input \+--amp \+--iters 200 \+--deployment_device 'cpu' \+--scale_dtype 'fp32' \+--eval_bs 32 \+--output_dir "./tmp_signround"