Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

fix gguf opt-rtn regression (#1905)

Wenhua Chengdc5e67f343d5
diff --git a/auto_round/export/export_to_gguf/convert.py b/auto_round/export/export_to_gguf/convert.py
index 539c90c00..61ac14bb5 100644
--- a/auto_round/export/export_to_gguf/convert.py
+++ b/auto_round/export/export_to_gguf/convert.py
@@ -237,12 +237,12 @@ def _quant_data(cls, data_torch, data_qtype, name, modify_name, new_name, bid, d
layer_name = name
module = get_module(cls.model, layer_name)
kwargs = {
- "scale": None,
- "zp": None,
- "d_scale": None,
- "d_wmin": None,
- "wmin": None,
- "imatrix": None,
+ "scale": module.scale if hasattr(module, "scale") else None,
+ "zp": module.zp if hasattr(module, "zp") else None,
+ "d_scale": module.w_d_scale if hasattr(module, "w_d_scale") else None,
+ "d_wmin": module.w_d_wmin if hasattr(module, "w_d_wmin") else None,
+ "wmin": module.w_wmin if hasattr(module, "w_wmin") else None,
+ "imatrix": module.imatrix if hasattr(module, "imatrix") else None,
}
# patch for Qwen3_5, Qwen3_5 handles some weights specially,
# but the scale doesn't match; these weights are handled by gguf itself.