PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Fix doc (#119)
wenhuach217bc9d8fb77ca
diff --git a/docs/gemma-2b-acc.md b/docs/gemma-2b-acc.mdindex 0b5c5e918..033d37dca 100644--- a/docs/gemma-2b-acc.md+++ b/docs/gemma-2b-acc.md@@ -11,8 +11,8 @@ Please note that there is a discrepancy between the baseline result and the offilm_eval --model hf --model_args pretrained="Intel/gemma-2b-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 16```-| Metric | FP16 | INT4 |-| -------------- | ------ |--------|+| Metric | BF16 | INT4 |+| -------------- |--------|--------|| Avg. | 0.5383 | 0.5338 || mmlu | 0.3337 | 0.3276 || lambada_openai | 0.6398 | 0.6319 |diff --git a/docs/gemma-7b-acc.md b/docs/gemma-7b-acc.mdindex 9f03be4dd..b371eadd9 100644--- a/docs/gemma-7b-acc.md+++ b/docs/gemma-7b-acc.md@@ -6,8 +6,8 @@ Please note that there is a discrepancy between the baseline result and the offilm_eval --model hf --model_args pretrained="Intel/gemma-7b-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32```-| Metric | FP16 | int4 |-| -------------- | ------ | ------ |+| Metric | BF16 | int4 |+| -------------- |--------| ------ || Avg. | 0.6239 | 0.6307 || mmlu | 0.6162 | 0.6147 || lambada_openai | 0.6751 | 0.7204 |diff --git a/docs/gemma-7b-it-acc.md b/docs/gemma-7b-it-acc.mdindex 1d1f1b52f..1cd2395f2 100644--- a/docs/gemma-7b-it-acc.md+++ b/docs/gemma-7b-it-acc.md@@ -6,8 +6,8 @@ Please note that there is a discrepancy between the baseline result and the offilm_eval --model hf --model_args pretrained="Intel/gemma-7b-it-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32```-| Metric | FP16 | int4 |-| -------------- | ------ | ------ |+| Metric | BF16 | int4 |+| -------------- |--------| ------ || Avg. | 0.6022 | 0.6017 || mmlu | 0.5029 | 0.4993 || lambada_openai | 0.6035 | 0.6286 |diff --git a/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh b/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.shindex a64143260..51ef47e3b 100644--- a/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh+++ b/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh@@ -6,4 +6,4 @@ python3 main.py \--iters 1000 \--deployment_device 'gpu' \--output_dir "./tmp_autoround" \---disable_quanted_input \\ No newline at end of file+--disable_quanted_input\ No newline at end of filediff --git a/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh b/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.shindex 3e1f75ffb..84ab60490 100644--- a/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh+++ b/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh@@ -5,5 +5,5 @@ python3 main.py \--bits 4 \--iters 1000 \--deployment_device 'gpu' \---minmax_lr 2e-3 \+--minmax_lr 2e-3diff --git a/examples/language-modeling/scripts/Yi-6B-Chat.sh b/examples/language-modeling/scripts/Yi-6B-Chat.shindex 57fa0727f..ce59f1bbe 100644--- a/examples/language-modeling/scripts/Yi-6B-Chat.sh+++ b/examples/language-modeling/scripts/Yi-6B-Chat.sh@@ -5,4 +5,4 @@ python3 main.py \--bits 4 \--iters 1000 \--deployment_device 'gpu' \---minmax_lr 2e-3 \+--minmax_lr 2e-3diff --git a/examples/language-modeling/scripts/baichuan2-7b-chat.sh b/examples/language-modeling/scripts/baichuan2-7b-chat.shindex 7fb02dcfa..51d674509 100644--- a/examples/language-modeling/scripts/baichuan2-7b-chat.sh+++ b/examples/language-modeling/scripts/baichuan2-7b-chat.sh@@ -5,4 +5,4 @@ python3 main.py \--bits 4 \--iters 1000 \--deployment_device 'gpu' \---minmax_lr 2e-3 \+--minmax_lr 2e-3diff --git a/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh b/examples/language-modeling/scripts/neural-chat-7b-v3-1.shindex d98afe8c4..208694e0b 100644--- a/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh+++ b/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh@@ -8,4 +8,4 @@ python3 main.py \--minmax_lr 0.0002 \--deployment_device 'gpu' \--output_dir "./tmp_autoround" \---disable_quanted_input \\ No newline at end of file+--disable_quanted_input\ No newline at end of filediff --git a/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh b/examples/language-modeling/scripts/neural-chat-7b-v3-3.shindex 99ea7ccfe..c7db8b1f4 100644--- a/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh+++ b/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh@@ -7,4 +7,4 @@ python3 main.py \--minmax_lr 0.0002 \--deployment_device 'gpu' \--output_dir "./tmp_autoround" \---disable_quanted_input \\ No newline at end of file+--disable_quanted_input\ No newline at end of file