PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Qwen3-8B with signround iters 200
W2G64
| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| V1 | 0.4959 | 0.2596 | 0.4064 | 0.5331 | 0.3720 | 0.7016 | 0.8131 | 0.4437 | 0.4822 | 0.2740 | 0.7051 | 0.3280 | 0.6314 |
| V1+lfq new | 0.5103 | 0.2547 | 0.4420 | 0.5570 | 0.4036 | 0.7344 | 0.7920 | 0.4556 | 0.5267 | 0.2860 | 0.7155 | 0.3182 | 0.6377 |
| V2 | 0.5275 | 0.2921 | 0.5807 | 0.5739 | 0.4317 | 0.7529 | 0.7550 | 0.4495 | 0.5251 | 0.2740 | 0.7171 | 0.3280 | 0.6496 |
| V2+lfq new | 0.5292 | 0.2965 | 0.5595 | 0.5695 | 0.4369 | 0.7668 | 0.7636 | 0.4558 | 0.5255 | 0.2680 | 0.7193 | 0.3305 | 0.6590 |
W3G128
| Model | AVG | mmlu-pro | GSM8K | MMLU | ARC-c | ARC-e | BoolQ | Hella | LAMB | OBQA | PIQA | Truth. | Wino |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| V1 | 0.6198 | 0.4221 | 0.8423 | 0.6971 | 0.5213 | 0.8136 | 0.8648 | 0.5374 | 0.6220 | 0.3180 | 0.7573 | 0.3611 | 0.6803 |
| V1+lfq new | 0.6151 | 0.4252 | 0.8150 | 0.6967 | 0.5188 | 0.8047 | 0.8676 | 0.5394 | 0.6179 | 0.3120 | 0.7557 | 0.3599 | 0.6685 |
| V2 | 0.6230 | 0.4301 | 0.8696 | 0.6980 | 0.5119 | 0.8114 | 0.8563 | 0.5338 | 0.6447 | 0.3120 | 0.7601 | 0.3574 | 0.6906 |
| V2+lfq new | 0.6253 | 0.4279 | 0.8560 | 0.7012 | 0.5299 | 0.8211 | 0.8615 | 0.5364 | 0.6297 | 0.3140 | 0.7639 | 0.3660 | 0.6961 |
legacy result with AutoRound version<=0.16.1
iters 200 enable_alg_ext is turned on Qwen3-8B
lfq causes large drop, maybe some bugs in our implementation
| Model | ARC-c | ARC-e | BoolQ | Hella | LAMB | MMLU | OBQA | PIQA | Truth. | Wino | GSM8K | if-eval stirct | mmlu-pro | AVG. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LFQ W2G64 | 0.3046 | 0.5396 | 0.6783 | 0.3856 | 0.2340 | 0.2822 | 0.2000 | 0.6436 | 0.3011 | 0.5564 | 0.3419 | 0.2206+0.1183 | 0.2334 | |
| ours W2G64 | 0.3942 | 0.7412 | 0.7841 | 0.4282 | 0.5242 | 0.5498 | 0.2540 | 0.7051 | 0.3097 | 0.6464 | 0.5239 | 0.2542+0.1201 | 0.2745 | |
| LFQ w3g128 | 0.4949 | 0.7605 | 0.8061 | 0.5276 | 0.4883 | 0.6558 | 0.2860 | 0.7530 | 0.3452 | 0.6717 | 0.8241 | 0.3777+0.2366 | 0.4076 | |
| oursw3g128 | 0.5213 | 0.8093 | 0.8667 | 0.5319 | 0.6445 | 0.6949 | 0.3140 | 0.7546 | 0.3623 | 0.7088 | 0.8597 | 0.3333+0.1904 | 0.4269 | |