Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

Update README.md

wenhuach2134b22b5bc718
diff --git a/README.md b/README.md
index 831ac7eaf..ee4dd5ca9 100644
--- a/README.md
+++ b/README.md
@@ -47,7 +47,8 @@ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
from auto_round import AutoRound
bits, group_size, sym = 4, 128, False
-autoround = AutoRound(model, tokenizer, bits=bits, group_size=group_size, sym=sym)
+## The device will be detected automatically, or you can specify it.
+autoround = AutoRound(model, tokenizer, bits=bits, group_size=group_size, sym=sym, device=None)
autoround.quantize()
output_dir = "./tmp_autoround"
autoround.save_quantized(output_dir)