Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

fix severe vram leak regression in auto-round format packing

Wenhua Chengdd7811ef6a99
diff --git a/auto_round/export/export_to_autoround/export.py b/auto_round/export/export_to_autoround/export.py
index 12ed39bd8..1e4036b69 100644
--- a/auto_round/export/export_to_autoround/export.py
+++ b/auto_round/export/export_to_autoround/export.py
@@ -234,7 +234,7 @@ def pack_layer(layer_name, model, backend, device=None):
qlayer.pack(layer, scale, device=device)
else:
qlayer.pack(layer, scale, zp, None, device=device)
- qlayer.to(device)
+ qlayer.to(orig_device)
else:
scale = scale.to(torch.float32).t().contiguous()
if isinstance(zp, torch.Tensor):