Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

force auto-scheme low_gpu to True in CLI (#1653)

Wenhua Cheng0f7dba7da309
diff --git a/auto_round/__main__.py b/auto_round/__main__.py
index 9da465810..e28199ddf 100644
--- a/auto_round/__main__.py
+++ b/auto_round/__main__.py
@@ -696,7 +696,7 @@ def tune(args):
avg_bits=args.avg_bits,
shared_layers=args.shared_layers,
ignore_scale_zp_bits=args.ignore_scale_zp_bits,
- low_gpu_mem_usage=args.low_gpu_mem_usage,
+ low_gpu_mem_usage=True, # force it to be True as it uses much smaller vram but similar time cost
low_cpu_mem_usage=low_cpu_mem_usage,
)