Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

add new accuracy test for VLM loading and fix bug (#1829)

Xin Hea577f101d6de
diff --git a/README.md b/README.md
index 11091324c..380704574 100644
--- a/README.md
+++ b/README.md
@@ -36,9 +36,9 @@ See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2]
## 🆕 What's New
-* [2026/05] **Model free** quantization is available, `--iters 0 --disable_opt_rtn` will now default to using the model-free approach: [Doc](https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#model-free-mode).
+* [2026/05] **Model free** quantization is available, `auto-round-rtn` will now default to using the model-free approach: [Doc](https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#model-free-mode).
-* [2026/03] **Block-wise FP8** quantization is available via `--scheme FP8_BLOCK --iters 0 --disable_opt_rtn`.
+* [2026/03] **Block-wise FP8** quantization is available via `auto-round-rtn --scheme FP8_BLOCK`.
* [2026/03] **MTP layer quantization** has been supported in this [PR](https://github.com/intel/auto-round/pull/1526)
diff --git a/README_CN.md b/README_CN.md
index 3b263380f..645d8ca6d 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -35,9 +35,9 @@ AutoRound 是专为大语言模型(LLMs)和视觉-语言模型(VLMs)设
## 🆕 最新进展
-* [2026/05] 现已支持**免模型量化**模式, `--iters 0 --disable_opt_rtn`现在会默认启用免模型方式: [文档](https://github.com/intel/auto-round/blob/main/docs/step_by_step_CN.md#%E5%85%8D%E6%A8%A1%E5%9E%8B%E9%87%8F%E5%8C%96%E6%A8%A1%E5%BC%8F).
+* [2026/05] 现已支持**免模型量化**模式, `auto-round-rtn`现在会默认启用免模型方式: [文档](https://github.com/intel/auto-round/blob/main/docs/step_by_step_CN.md#%E5%85%8D%E6%A8%A1%E5%9E%8B%E9%87%8F%E5%8C%96%E6%A8%A1%E5%BC%8F).
-* [2026/03] 支持**Block-wise FP8**,只需要将scheme改成 `--scheme FP8_BLOCK`.为了更快的量化,我们推荐使用RTN模式,即·--iters 0 --disable_opt_rtn·
+* [2026/03] 支持**Block-wise FP8**,只需要将scheme改成 `--scheme FP8_BLOCK`.为了更快的量化,我们推荐使用RTN模式,即`auto-round-rtn`
* [2026/03] 支持**MTP层量化**
diff --git a/test/test_cuda/export/test_fp8_format.py b/test/test_cuda/export/test_fp8_format.py
index b612cf27d..0d832e372 100644
--- a/test/test_cuda/export/test_fp8_format.py
+++ b/test/test_cuda/export/test_fp8_format.py
@@ -34,7 +34,9 @@ class TestAutoRoundBlockFP:
seqlen=2,
)
quantized_model_path = self.save_dir
- compressed_model, _ = autoround.quantize_and_save(output_dir=quantized_model_path, format="fp8")
+ compressed_model, quantized_model_path = autoround.quantize_and_save(
+ output_dir=quantized_model_path, format="fp8"
+ )
tmp_layer = compressed_model.model.layers[1].self_attn.q_proj
assert hasattr(tmp_layer, "weight_scale_inv")
assert tmp_layer.weight.dtype is torch.float8_e4m3fn
diff --git a/test/test_cuda/integrations/test_huggingface.py b/test/test_cuda/integrations/test_huggingface.py
new file mode 100644
index 000000000..e027977d8
--- /dev/null
+++ b/test/test_cuda/integrations/test_huggingface.py
@@ -0,0 +1,10 @@
+import pytest
+
+from ...helpers import evaluate_accuracy
+
+model_name_or_path = "Intel/Qwen3.5-2B-int4-AutoRound"
+
+
[email protected]_ci(reason="Requires downloading a large model from HuggingFace")
+def test_evaluate_accuracy():
+ evaluate_accuracy(model_name_or_path, threshold=0.5)