PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
add system checker in backend (#1097)
Wenhua Chengae4ac94cb935
diff --git a/README.md b/README.mdindex 43feb0538..a83510560 100644--- a/README.md+++ b/README.md@@ -31,6 +31,8 @@ See our [paper](https://arxiv.org/pdf/2309.05516) for more details. For usage in## 🆕 What's New+* [2025/12] The [AutoRoundV2 paper](http://arxiv.org/abs/2512.04746) is now publicly available on arXiv. Turn on `enable_alg_ext` and use the **AutoScheme** API for mixed-precision quantization to reproduce the results.+* [2025/11] AutoRound has landed in **LLM-Compressor**: [*Usage*](https://github.com/vllm-project/llm-compressor/tree/main/examples/autoround/README.md).* [2025/11] An **enhanced GGUF** quantization algorithm is available via `--enable_alg_ext`: [*Accuracy*](./docs/gguf_alg_ext_acc.md).@@ -199,6 +201,7 @@ ar.quantize_and_save(output_dir="./qmodel", format="auto_round")- **`iters` (int)**: Number of tuning iterations (default is `200`). Common values: 0 (RTN mode), 50 (with lr=5e-3 recommended), 1000. Higher values increase accuracy but slow down tuning.- **`lr` (float)**: The learning rate for rounding value (default is `None`). When None, it will be set to `1.0/iters` automatically.- **`batch_size` (int)**: Batch size for training (default is `8`). 4 is also commonly used.+- ** `enable_deterministic_algorithms` (bool)**: Whether to enable deterministic algorithms for reproducibility (default is `False`).##### Calibration Dataset- **`dataset` (str|list|tuple|torch.utils.data.DataLoader)**: The dataset for tuning (default is `"NeelNanda/pile-10k"`). Supports local JSON files and dataset combinations, e.g. `"./tmp.json,NeelNanda/pile-10k:train,mbpp:train+validation+test"`.@@ -334,11 +337,14 @@ inputs = tokenizer(text, return_tensors="pt").to(model.device)print(tokenizer.decode(model.generate(**inputs, max_new_tokens=50)[0]))```++## Publications & Events++[Publication List](./docs/publication_list.md).+## AcknowledgementSpecial thanks to open-source low precision libraries such as AutoGPTQ, AutoAWQ, GPTQModel, Triton, Marlin, and ExLLaMAV2 for providing low-precision CUDA kernels, which are leveraged in AutoRound.-> **Note**:-> For all publications/events, please view [Publication List](./docs/publication_list.md).## 🌟 Support UsIf you find AutoRound helpful, please ⭐ star the repo and share it with your community!diff --git a/auto_round/inference/backend.py b/auto_round/inference/backend.pyindex 5900207d5..8dbba46f6 100644--- a/auto_round/inference/backend.py+++ b/auto_round/inference/backend.py@@ -13,6 +13,7 @@# limitations under the License.import functools+import platformfrom dataclasses import dataclass, fieldfrom typing import TYPE_CHECKING, Any, Optional@@ -79,6 +80,9 @@ class BackendInfo:an empty list.alias: An optional list of strings representing alternative names for thebackend. Defaults to None.+ requirements: An optional list of strings specifying the library dependencies+ required by the backend (e.g., 'triton>=2.0'). Defaults to None.+ systems: An optional list of strings specifying the operating systems,(e.g., 'windows', 'linux', 'darwin')."""device: list[str] # TODO change to tuple@@ -97,6 +101,7 @@ class BackendInfo:checkers: list[Any] = field(default_factory=list)alias: Optional[list[str]] = Nonerequirements: Optional[list[str]] = None+ systems: Optional[list[str]] = NoneBACKEND_ACT_ATTRS = [@@ -340,6 +345,7 @@ BackendInfos["auto_round:tritonv2_zp"] = BackendInfo(checkers=[feature_multiply_checker_32],alias=["tritonv2", "tritonv2_zp", "triton"],requirements=["triton>=2.0", "auto-round>=0.5.0"],+ # systems=["windows", "linux", "darwin"],)BackendInfos["auto_round:torch"] = BackendInfo(@@ -569,6 +575,11 @@ def check_compatible(passelse:return False+ if backend.systems is not None:+ current_system = platform.system()+ systems = [s.lower() for s in backend.systems]+ if current_system.lower() not in systems:+ return False# Check schemefor key, value in config.items():backend_value = getattr(backend, key, None)diff --git a/docs/publication_list.md b/docs/publication_list.mdindex 3b6f88848..58fb75e7b 100644--- a/docs/publication_list.md+++ b/docs/publication_list.md@@ -1,7 +1,9 @@Full Publications/Events==========-## 2025 (3)+## 2025 (4)++* Paper: [SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs](https://arxiv.org/abs/2512.04746) (Dec 2025)* Blog in LMSYS: [AutoRound Meets SGLang: Enabling Quantized Model Inference with AutoRound](https://lmsys.org/blog/2025-11-13-AutoRound/) (Nov 2025)