Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

774 B
Raw Blame

W2G32 nsamples 512,iter 200, average accuracy of 10 tasks

Models gptq_sym asym full_range_sym
Meta-Llama-3.1-8B-Instruct 0.4500 0.52802 0.5381
Qwen2-7B 0.5229 0.5559 0.5486

W4G128 nsamples 128,iter 200, average accuracy of 10 tasks

Models asym full_range_sym
Meta-Llama-3.1-8B-Instruct 0.6342 0.6370
Qwen2-7B 0.6143 0.6167
Mistral-7B-Instruct-v0.2 0.6606 0.6635
Phi-3-mini-4k-instruct 0.6475 0.6432