Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

Fix cuda ut fail caused by used instructions for base model (#1994)

Heng Guo43f443e6a799
diff --git a/test/helpers.py b/test/helpers.py
index 60555cc46..e9309a269 100644
--- a/test/helpers.py
+++ b/test/helpers.py
@@ -71,11 +71,13 @@ def generate_prompt(
def eval_generated_prompt(
model,
tokenizer=None,
- prompt_text='Convert "hello" to uppercase, the answer is:',
- target_text="HELLO",
+ prompt_text="United States of",
+ target_text="America",
max_new_tokens=10,
device=None,
):
+ # Use a completion-style prompt: base models (e.g. OPT-125M) cannot follow
+ # instructions, so an instruction prompt makes the check flaky after quantization.
generated_text = generate_prompt(
model,
tokenizer,