PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
fix mistral moe issue
wenhuach21ee40f588f89f
diff --git a/autoround/README.md b/autoround/README.mdindex 2e3c1f650..c2da30bd3 100644--- a/autoround/README.md+++ b/autoround/README.md@@ -50,9 +50,14 @@ CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --nu```It's recommended to use `--enable_minmax_tuning`.++detailed arguments:+- `--model_name`: the local model path or huggingface format.+## TipsConsider increasing tuning steps and adjusting the learning rate based on a scaling law to achieve better results, albeit with increased tuning time. For instance, at step 800, a learning rate of 0.00125 could be employed.+## Known IssuesAuto Rounding may encounter random issues with Qwen models.diff --git a/autoround/src/autoround.py b/autoround/src/autoround.pyindex fa66a25e0..03c834c9f 100644--- a/autoround/src/autoround.py+++ b/autoround/src/autoround.py@@ -12,17 +12,20 @@# See the License for the specific language governing permissions and# limitations under the License.-try:- from neural_compressor.utils.utility import LazyImport- torch = LazyImport("torch")- from neural_compressor.utils import logger-except: # pragma: no cover- import logging+import logging- import torch+logging.basicConfig(level=logging.INFO,+ format=('%(filename)s: '+ '%(levelname)s: '+ '%(funcName)s(): '+ '%(lineno)d:\t'+ '%(message)s')+ )- logger = logging.getLogger()+import torch++logger = logging.getLogger()import copyimport time@@ -819,6 +822,7 @@ def get_block_names(model):for n, m in model.named_modules():if hasattr(type(m), "__name__") and "ModuleList" in type(m).__name__:target_m = (n, m)+ break ## only find the first modulelist, may be not robustfor n, m in target_m[1].named_children():block_names.append(target_m[0] + "." + n)return block_names