Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

2.8 KB
Raw Blame

Qwen3-8B with signround iters 200

W2G64

Model AVG mmlu-pro GSM8K MMLU ARC-c ARC-e BoolQ Hella LAMB OBQA PIQA Truth. Wino
V1 0.4959 0.2596 0.4064 0.5331 0.3720 0.7016 0.8131 0.4437 0.4822 0.2740 0.7051 0.3280 0.6314
V1+lfq new 0.5103 0.2547 0.4420 0.5570 0.4036 0.7344 0.7920 0.4556 0.5267 0.2860 0.7155 0.3182 0.6377
V2 0.5275 0.2921 0.5807 0.5739 0.4317 0.7529 0.7550 0.4495 0.5251 0.2740 0.7171 0.3280 0.6496
V2+lfq new 0.5292 0.2965 0.5595 0.5695 0.4369 0.7668 0.7636 0.4558 0.5255 0.2680 0.7193 0.3305 0.6590

W3G128

Model AVG mmlu-pro GSM8K MMLU ARC-c ARC-e BoolQ Hella LAMB OBQA PIQA Truth. Wino
V1 0.6198 0.4221 0.8423 0.6971 0.5213 0.8136 0.8648 0.5374 0.6220 0.3180 0.7573 0.3611 0.6803
V1+lfq new 0.6151 0.4252 0.8150 0.6967 0.5188 0.8047 0.8676 0.5394 0.6179 0.3120 0.7557 0.3599 0.6685
V2 0.6230 0.4301 0.8696 0.6980 0.5119 0.8114 0.8563 0.5338 0.6447 0.3120 0.7601 0.3574 0.6906
V2+lfq new 0.6253 0.4279 0.8560 0.7012 0.5299 0.8211 0.8615 0.5364 0.6297 0.3140 0.7639 0.3660 0.6961

legacy result with AutoRound version<=0.16.1

iters 200 enable_alg_ext is turned on Qwen3-8B

lfq causes large drop, maybe some bugs in our implementation

Model ARC-c ARC-e BoolQ Hella LAMB MMLU OBQA PIQA Truth. Wino GSM8K if-eval stirct mmlu-pro AVG.
LFQ W2G64 0.3046 0.5396 0.6783 0.3856 0.2340 0.2822 0.2000 0.6436 0.3011 0.5564 0.3419 0.2206+0.1183 0.2334
ours W2G64 0.3942 0.7412 0.7841 0.4282 0.5242 0.5498 0.2540 0.7051 0.3097 0.6464 0.5239 0.2542+0.1201 0.2745
LFQ w3g128 0.4949 0.7605 0.8061 0.5276 0.4883 0.6558 0.2860 0.7530 0.3452 0.6717 0.8241 0.3777+0.2366 0.4076
oursw3g128 0.5213 0.8093 0.8667 0.5319 0.6445 0.6949 0.3140 0.7546 0.3623 0.7088 0.8597 0.3333+0.1904 0.4269