PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Fix cuda ut fail caused by used instructions for base model (#1994)
Heng Guo43f443e6a799
diff --git a/test/helpers.py b/test/helpers.pyindex 60555cc46..e9309a269 100644--- a/test/helpers.py+++ b/test/helpers.py@@ -71,11 +71,13 @@ def generate_prompt(def eval_generated_prompt(model,tokenizer=None,- prompt_text='Convert "hello" to uppercase, the answer is:',- target_text="HELLO",+ prompt_text="United States of",+ target_text="America",max_new_tokens=10,device=None,):+ # Use a completion-style prompt: base models (e.g. OPT-125M) cannot follow+ # instructions, so an instruction prompt makes the check flaky after quantization.generated_text = generate_prompt(model,tokenizer,