PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Code Commits Issues Pull requests Wiki Builds Discussions Projects Insights Search Files 291 branches · 45 tags auto-round / docs / mxnv_acc.md 3.3 KB
Code Raw Blame Permalink MXFP4 variants
Qwen3-8B
Method
avg
MMLU
arc_challenge
arc_easy
boolq
gsm8k
hellaswag
lambada_openai
openbookqa
piqa
truthfulqa_mc1
winogrande
ocp rtn
0.607072727
0.6613
0.4932
0.7715
0.8593
0.8127
0.5223
0.5851
0.2800
0.7301
0.3293
0.6330
rceil rtn
0.614190909
0.6718
0.4974
0.7656
0.8450
0.8271
0.5299
0.5905
0.2880
0.7427
0.3525
0.6456
rceil 7.25 RTN
0.617018182
0.6780
0.5000
0.7778
0.8498
0.8317
0.5338
0.5991
0.2940
0.7329
0.3476
0.6425
ocp iters 200
0.624609091
0.6771
0.5171
0.8119
0.8456
0.7961
0.5284
0.5979
0.3080
0.7546
0.3513
0.6827
rceil iters 200
0.618427273
0.6794
0.5111
0.7984
0.8550
0.7680
0.5242
0.5993
0.3040
0.7530
0.3378
0.6725
rceil 7.25 iters 200
0.627009091
0.6837
0.5307
0.8136
0.8569
0.8052
0.5317
0.6113
0.2960
0.7514
0.3378
0.6788
Llama3.1-8B-I
Method
avg
MMLU
arc_challenge
arc_easy
boolq
gsm8k
hellaswag
lambada_openai
openbookqa
piqa
truthfulqa_mc1
winogrande
ocp rtn
0.575163636
0.5734
0.4582
0.7630
0.8080
0.5011
0.5554
0.6216
0.3080
0.7606
0.2766
0.7009
rceil rtn
0.6031
0.6002
0.4710
0.7816
0.8235
0.5792
0.5661
0.6647
0.3260
0.7780
0.3390
0.7048
rceil 7.25 rtn
0.601372727
0.6057
0.4753
0.7803
0.8242
0.5663
0.5657
0.6637
0.3340
0.7769
0.3158
0.7072
ocp iters 200
0.605645455
0.6160
0.4659
0.8009
0.8257
0.6035
0.5577
0.6831
0.3000
0.7709
0.3415
0.6969
rceil iters 200
0.607863636
0.6133
0.4846
0.7959
0.8300
0.6156
0.5628
0.6645
0.3120
0.7835
0.3329
0.6914
rceil 7.25 iters 200
0.614072727
0.6149
0.4804
0.7984
0.8303
0.6262
0.5670
0.6829
0.3300
0.7807
0.3305
0.7135
Average accuracy of hellaswag,lambada_openai,mmlu,piqa,winogrande.
We evaluated using a fake model since we currently have no access to devices for running the real models. However, we have verified that in most cases the fake model closely matches the real model.
mxfp4 g32
llama3.1-8B-Instruct
Qwen2-7.5-Instruct
Phi4
Qwen3-32B
RTN
0.6212
0.6550
0.7167
0.6901
AutoRound
0.6686
0.6758
0.7247
0.7211
AutoRound+alg_ext
0.6732
0.6809
0.7225
0.7201
nvfp4 g16
llama3.1-8B-Instruct
Qwen2-7.5-Instruct
Phi4
Qwen3-32B
RTN
0.6876
0.6906
0.7296
0.7164
AutoRound
0.6918
0.6973
0.7306
0.7306
AutoRound+alg_ext
0.6965
0.6989
0.7318
0.7295