Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

fix mistral moe issue

wenhuach21ee40f588f89f
diff --git a/autoround/README.md b/autoround/README.md
index 2e3c1f650..c2da30bd3 100644
--- a/autoround/README.md
+++ b/autoround/README.md
@@ -50,9 +50,14 @@ CUDA_VISIBLE_DEVICES=0 python3 main.py --model_name facebook/opt-125m --amp --nu
```
It's recommended to use `--enable_minmax_tuning`.
+
+detailed arguments:
+- `--model_name`: the local model path or huggingface format.
+
## Tips
Consider increasing tuning steps and adjusting the learning rate based on a scaling law to achieve better results, albeit with increased tuning time. For instance, at step 800, a learning rate of 0.00125 could be employed.
+
## Known Issues
Auto Rounding may encounter random issues with Qwen models.
diff --git a/autoround/src/autoround.py b/autoround/src/autoround.py
index fa66a25e0..03c834c9f 100644
--- a/autoround/src/autoround.py
+++ b/autoround/src/autoround.py
@@ -12,17 +12,20 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-try:
- from neural_compressor.utils.utility import LazyImport
- torch = LazyImport("torch")
- from neural_compressor.utils import logger
-except: # pragma: no cover
- import logging
+import logging
- import torch
+logging.basicConfig(level=logging.INFO,
+ format=('%(filename)s: '
+ '%(levelname)s: '
+ '%(funcName)s(): '
+ '%(lineno)d:\t'
+ '%(message)s')
+ )
- logger = logging.getLogger()
+import torch
+
+logger = logging.getLogger()
import copy
import time
@@ -819,6 +822,7 @@ def get_block_names(model):
for n, m in model.named_modules():
if hasattr(type(m), "__name__") and "ModuleList" in type(m).__name__:
target_m = (n, m)
+ break ## only find the first modulelist, may be not robust
for n, m in target_m[1].named_children():
block_names.append(target_m[0] + "." + n)
return block_names