Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

fix bloom issue

pursure-Dbd2fcc9f1311
diff --git a/examples/language-modeling/main.py b/examples/language-modeling/main.py
index b0c9c5448..31540bd1e 100644
--- a/examples/language-modeling/main.py
+++ b/examples/language-modeling/main.py
@@ -232,6 +232,8 @@ if __name__ == '__main__':
dtype = 'float32'
excel_name = f"{model_name}_{args.bits}_{args.group_size}"
+ if "bloom" in model_name:
+ args.disable_low_gpu_mem_usage=True
if args.eval_fp16_baseline:
if args.disable_low_gpu_mem_usage:
model = model.to(torch_device)