PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Update README.md
wenhuach2134b22b5bc718
diff --git a/README.md b/README.mdindex 831ac7eaf..ee4dd5ca9 100644--- a/README.md+++ b/README.md@@ -47,7 +47,8 @@ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)from auto_round import AutoRoundbits, group_size, sym = 4, 128, False-autoround = AutoRound(model, tokenizer, bits=bits, group_size=group_size, sym=sym)+## The device will be detected automatically, or you can specify it.+autoround = AutoRound(model, tokenizer, bits=bits, group_size=group_size, sym=sym, device=None)autoround.quantize()output_dir = "./tmp_autoround"autoround.save_quantized(output_dir)