PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
fix severe vram leak regression in auto-round format packing
Wenhua Chengdd7811ef6a99
diff --git a/auto_round/export/export_to_autoround/export.py b/auto_round/export/export_to_autoround/export.pyindex 12ed39bd8..1e4036b69 100644--- a/auto_round/export/export_to_autoround/export.py+++ b/auto_round/export/export_to_autoround/export.py@@ -234,7 +234,7 @@ def pack_layer(layer_name, model, backend, device=None):qlayer.pack(layer, scale, device=device)else:qlayer.pack(layer, scale, zp, None, device=device)- qlayer.to(device)+ qlayer.to(orig_device)else:scale = scale.to(torch.float32).t().contiguous()if isinstance(zp, torch.Tensor):