Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

Fix doc (#119)

wenhuach217bc9d8fb77ca
diff --git a/docs/gemma-2b-acc.md b/docs/gemma-2b-acc.md
index 0b5c5e918..033d37dca 100644
--- a/docs/gemma-2b-acc.md
+++ b/docs/gemma-2b-acc.md
@@ -11,8 +11,8 @@ Please note that there is a discrepancy between the baseline result and the offi
lm_eval --model hf --model_args pretrained="Intel/gemma-2b-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 16
```
-| Metric | FP16 | INT4 |
-| -------------- | ------ |--------|
+| Metric | BF16 | INT4 |
+| -------------- |--------|--------|
| Avg. | 0.5383 | 0.5338 |
| mmlu | 0.3337 | 0.3276 |
| lambada_openai | 0.6398 | 0.6319 |
diff --git a/docs/gemma-7b-acc.md b/docs/gemma-7b-acc.md
index 9f03be4dd..b371eadd9 100644
--- a/docs/gemma-7b-acc.md
+++ b/docs/gemma-7b-acc.md
@@ -6,8 +6,8 @@ Please note that there is a discrepancy between the baseline result and the offi
lm_eval --model hf --model_args pretrained="Intel/gemma-7b-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
```
-| Metric | FP16 | int4 |
-| -------------- | ------ | ------ |
+| Metric | BF16 | int4 |
+| -------------- |--------| ------ |
| Avg. | 0.6239 | 0.6307 |
| mmlu | 0.6162 | 0.6147 |
| lambada_openai | 0.6751 | 0.7204 |
diff --git a/docs/gemma-7b-it-acc.md b/docs/gemma-7b-it-acc.md
index 1d1f1b52f..1cd2395f2 100644
--- a/docs/gemma-7b-it-acc.md
+++ b/docs/gemma-7b-it-acc.md
@@ -6,8 +6,8 @@ Please note that there is a discrepancy between the baseline result and the offi
lm_eval --model hf --model_args pretrained="Intel/gemma-7b-it-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
```
-| Metric | FP16 | int4 |
-| -------------- | ------ | ------ |
+| Metric | BF16 | int4 |
+| -------------- |--------| ------ |
| Avg. | 0.6022 | 0.6017 |
| mmlu | 0.5029 | 0.4993 |
| lambada_openai | 0.6035 | 0.6286 |
diff --git a/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh b/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh
index a64143260..51ef47e3b 100644
--- a/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh
+++ b/examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh
@@ -6,4 +6,4 @@ python3 main.py \
--iters 1000 \
--deployment_device 'gpu' \
--output_dir "./tmp_autoround" \
---disable_quanted_input \
\ No newline at end of file
+--disable_quanted_input
\ No newline at end of file
diff --git a/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh b/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh
index 3e1f75ffb..84ab60490 100644
--- a/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh
+++ b/examples/language-modeling/scripts/Qwen1.5-7B-Chat-asym.sh
@@ -5,5 +5,5 @@ python3 main.py \
--bits 4 \
--iters 1000 \
--deployment_device 'gpu' \
---minmax_lr 2e-3 \
+--minmax_lr 2e-3
diff --git a/examples/language-modeling/scripts/Yi-6B-Chat.sh b/examples/language-modeling/scripts/Yi-6B-Chat.sh
index 57fa0727f..ce59f1bbe 100644
--- a/examples/language-modeling/scripts/Yi-6B-Chat.sh
+++ b/examples/language-modeling/scripts/Yi-6B-Chat.sh
@@ -5,4 +5,4 @@ python3 main.py \
--bits 4 \
--iters 1000 \
--deployment_device 'gpu' \
---minmax_lr 2e-3 \
+--minmax_lr 2e-3
diff --git a/examples/language-modeling/scripts/baichuan2-7b-chat.sh b/examples/language-modeling/scripts/baichuan2-7b-chat.sh
index 7fb02dcfa..51d674509 100644
--- a/examples/language-modeling/scripts/baichuan2-7b-chat.sh
+++ b/examples/language-modeling/scripts/baichuan2-7b-chat.sh
@@ -5,4 +5,4 @@ python3 main.py \
--bits 4 \
--iters 1000 \
--deployment_device 'gpu' \
---minmax_lr 2e-3 \
+--minmax_lr 2e-3
diff --git a/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh b/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh
index d98afe8c4..208694e0b 100644
--- a/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh
+++ b/examples/language-modeling/scripts/neural-chat-7b-v3-1.sh
@@ -8,4 +8,4 @@ python3 main.py \
--minmax_lr 0.0002 \
--deployment_device 'gpu' \
--output_dir "./tmp_autoround" \
---disable_quanted_input \
\ No newline at end of file
+--disable_quanted_input
\ No newline at end of file
diff --git a/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh b/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh
index 99ea7ccfe..c7db8b1f4 100644
--- a/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh
+++ b/examples/language-modeling/scripts/neural-chat-7b-v3-3.sh
@@ -7,4 +7,4 @@ python3 main.py \
--minmax_lr 0.0002 \
--deployment_device 'gpu' \
--output_dir "./tmp_autoround" \
---disable_quanted_input \
\ No newline at end of file
+--disable_quanted_input
\ No newline at end of file