Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

add system checker in backend (#1097)

Wenhua Chengae4ac94cb935
diff --git a/README.md b/README.md
index 43feb0538..a83510560 100644
--- a/README.md
+++ b/README.md
@@ -31,6 +31,8 @@ See our [paper](https://arxiv.org/pdf/2309.05516) for more details. For usage in
## 🆕 What's New
+* [2025/12] The [AutoRoundV2 paper](http://arxiv.org/abs/2512.04746) is now publicly available on arXiv. Turn on `enable_alg_ext` and use the **AutoScheme** API for mixed-precision quantization to reproduce the results.
+
* [2025/11] AutoRound has landed in **LLM-Compressor**: [*Usage*](https://github.com/vllm-project/llm-compressor/tree/main/examples/autoround/README.md).
* [2025/11] An **enhanced GGUF** quantization algorithm is available via `--enable_alg_ext`: [*Accuracy*](./docs/gguf_alg_ext_acc.md).
@@ -199,6 +201,7 @@ ar.quantize_and_save(output_dir="./qmodel", format="auto_round")
- **`iters` (int)**: Number of tuning iterations (default is `200`). Common values: 0 (RTN mode), 50 (with lr=5e-3 recommended), 1000. Higher values increase accuracy but slow down tuning.
- **`lr` (float)**: The learning rate for rounding value (default is `None`). When None, it will be set to `1.0/iters` automatically.
- **`batch_size` (int)**: Batch size for training (default is `8`). 4 is also commonly used.
+- ** `enable_deterministic_algorithms` (bool)**: Whether to enable deterministic algorithms for reproducibility (default is `False`).
##### Calibration Dataset
- **`dataset` (str|list|tuple|torch.utils.data.DataLoader)**: The dataset for tuning (default is `"NeelNanda/pile-10k"`). Supports local JSON files and dataset combinations, e.g. `"./tmp.json,NeelNanda/pile-10k:train,mbpp:train+validation+test"`.
@@ -334,11 +337,14 @@ inputs = tokenizer(text, return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=50)[0]))
```
+
+## Publications & Events
+
+[Publication List](./docs/publication_list.md).
+
## Acknowledgement
Special thanks to open-source low precision libraries such as AutoGPTQ, AutoAWQ, GPTQModel, Triton, Marlin, and ExLLaMAV2 for providing low-precision CUDA kernels, which are leveraged in AutoRound.
-> **Note**:
-> For all publications/events, please view [Publication List](./docs/publication_list.md).
## 🌟 Support Us
If you find AutoRound helpful, please ⭐ star the repo and share it with your community!
diff --git a/auto_round/inference/backend.py b/auto_round/inference/backend.py
index 5900207d5..8dbba46f6 100644
--- a/auto_round/inference/backend.py
+++ b/auto_round/inference/backend.py
@@ -13,6 +13,7 @@
# limitations under the License.
import functools
+import platform
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Optional
@@ -79,6 +80,9 @@ class BackendInfo:
an empty list.
alias: An optional list of strings representing alternative names for the
backend. Defaults to None.
+ requirements: An optional list of strings specifying the library dependencies
+ required by the backend (e.g., 'triton>=2.0'). Defaults to None.
+ systems: An optional list of strings specifying the operating systems,(e.g., 'windows', 'linux', 'darwin').
"""
device: list[str] # TODO change to tuple
@@ -97,6 +101,7 @@ class BackendInfo:
checkers: list[Any] = field(default_factory=list)
alias: Optional[list[str]] = None
requirements: Optional[list[str]] = None
+ systems: Optional[list[str]] = None
BACKEND_ACT_ATTRS = [
@@ -340,6 +345,7 @@ BackendInfos["auto_round:tritonv2_zp"] = BackendInfo(
checkers=[feature_multiply_checker_32],
alias=["tritonv2", "tritonv2_zp", "triton"],
requirements=["triton>=2.0", "auto-round>=0.5.0"],
+ # systems=["windows", "linux", "darwin"],
)
BackendInfos["auto_round:torch"] = BackendInfo(
@@ -569,6 +575,11 @@ def check_compatible(
pass
else:
return False
+ if backend.systems is not None:
+ current_system = platform.system()
+ systems = [s.lower() for s in backend.systems]
+ if current_system.lower() not in systems:
+ return False
# Check scheme
for key, value in config.items():
backend_value = getattr(backend, key, None)
diff --git a/docs/publication_list.md b/docs/publication_list.md
index 3b6f88848..58fb75e7b 100644
--- a/docs/publication_list.md
+++ b/docs/publication_list.md
@@ -1,7 +1,9 @@
Full Publications/Events
==========
-## 2025 (3)
+## 2025 (4)
+
+* Paper: [SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs](https://arxiv.org/abs/2512.04746) (Dec 2025)
* Blog in LMSYS: [AutoRound Meets SGLang: Enabling Quantized Model Inference with AutoRound](https://lmsys.org/blog/2025-11-13-AutoRound/) (Nov 2025)