Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

skip quantizing mtp.fc since vLLM doesn't support (#1731)

Xin Heb3ef561dc299
diff --git a/auto_round/utils/missing_tensors.py b/auto_round/utils/missing_tensors.py
index dd82c537b..e51f1835c 100644
--- a/auto_round/utils/missing_tensors.py
+++ b/auto_round/utils/missing_tensors.py
@@ -497,7 +497,7 @@ def _woq_quantize_missing_tensors(target_dir: str, missing_tensors_dict: dict) -
"""
import re as _re
- BLOCK_NAME_TO_IGNORE = ["shared_expert_gate.", "mlp.gate.", "g_proj."]
+ BLOCK_NAME_TO_IGNORE = [".shared_expert_gate.", ".mlp.gate.", ".g_proj.", "mtp.fc."]
qconfig = _get_woq_config_from_dir(target_dir)
if qconfig is None:
return missing_tensors_dict