Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

3.3 KB
Raw Blame

MXFP4 variants

Qwen3-8B

Method avg MMLU arc_challenge arc_easy boolq gsm8k hellaswag lambada_openai openbookqa piqa truthfulqa_mc1 winogrande
ocp rtn 0.607072727 0.6613 0.4932 0.7715 0.8593 0.8127 0.5223 0.5851 0.2800 0.7301 0.3293 0.6330
rceil rtn 0.614190909 0.6718 0.4974 0.7656 0.8450 0.8271 0.5299 0.5905 0.2880 0.7427 0.3525 0.6456
rceil 7.25 RTN 0.617018182 0.6780 0.5000 0.7778 0.8498 0.8317 0.5338 0.5991 0.2940 0.7329 0.3476 0.6425
ocp iters 200 0.624609091 0.6771 0.5171 0.8119 0.8456 0.7961 0.5284 0.5979 0.3080 0.7546 0.3513 0.6827
rceil iters 200 0.618427273 0.6794 0.5111 0.7984 0.8550 0.7680 0.5242 0.5993 0.3040 0.7530 0.3378 0.6725
rceil 7.25 iters 200 0.627009091 0.6837 0.5307 0.8136 0.8569 0.8052 0.5317 0.6113 0.2960 0.7514 0.3378 0.6788

Llama3.1-8B-I

Method avg MMLU arc_challenge arc_easy boolq gsm8k hellaswag lambada_openai openbookqa piqa truthfulqa_mc1 winogrande
ocp rtn 0.575163636 0.5734 0.4582 0.7630 0.8080 0.5011 0.5554 0.6216 0.3080 0.7606 0.2766 0.7009
rceil rtn 0.6031 0.6002 0.4710 0.7816 0.8235 0.5792 0.5661 0.6647 0.3260 0.7780 0.3390 0.7048
rceil 7.25 rtn 0.601372727 0.6057 0.4753 0.7803 0.8242 0.5663 0.5657 0.6637 0.3340 0.7769 0.3158 0.7072
ocp iters 200 0.605645455 0.6160 0.4659 0.8009 0.8257 0.6035 0.5577 0.6831 0.3000 0.7709 0.3415 0.6969
rceil iters 200 0.607863636 0.6133 0.4846 0.7959 0.8300 0.6156 0.5628 0.6645 0.3120 0.7835 0.3329 0.6914
rceil 7.25 iters 200 0.614072727 0.6149 0.4804 0.7984 0.8303 0.6262 0.5670 0.6829 0.3300 0.7807 0.3305 0.7135

Average accuracy of hellaswag,lambada_openai,mmlu,piqa,winogrande.

We evaluated using a fake model since we currently have no access to devices for running the real models. However, we have verified that in most cases the fake model closely matches the real model.

mxfp4 g32 llama3.1-8B-Instruct Qwen2-7.5-Instruct Phi4 Qwen3-32B
RTN 0.6212 0.6550 0.7167 0.6901
AutoRound 0.6686 0.6758 0.7247 0.7211
AutoRound+alg_ext 0.6732 0.6809 0.7225 0.7201
nvfp4 g16 llama3.1-8B-Instruct Qwen2-7.5-Instruct Phi4 Qwen3-32B
RTN 0.6876 0.6906 0.7296 0.7164
AutoRound 0.6918 0.6973 0.7306 0.7306
AutoRound+alg_ext 0.6965 0.6989 0.7318 0.7295