PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
AGENTS.md
This file provides guidance to ANY AGENTS when working with code in this repository.
Project
AutoRound — post-training quantization for LLMs/VLMs using sign-gradient descent. Publishes as auto-round (GPU/CPU) and auto-round-hpu (Intel Gaudi).
Build & Install
# From source (GPU/CPU) — --no-build-isolation is required when PyTorch is already installed
pip install --no-build-isolation -e .
# HPU variant
BUILD_HPU_ONLY=1 pip install --no-build-isolation .
# or: python setup.py hpu install
# XPU variant — install Intel PyTorch first
pip install torch --index-url https://download.pytorch.org/whl/xpu
pip install --no-build-isolation .
Testing
Tests are split into three tiers under test/: unit/ (fast, runs in PR CI),
integration/ (third-party frameworks, runs nightly), and e2e/ (full models /
real inference engines, runs weekly). Each tier is further split by hardware
(test_cpu/, test_cuda/, test_hpu/, test_xpu/, test_ark/, test_mlx/).
# CPU unit tests (most common during development)
pytest test/unit/test_cpu/ -x -q
# Single test
pytest test/unit/test_cpu/ -k "test_name" -x -q
# Hardware-specific unit tests
pytest test/unit/test_cuda/
pytest test/unit/test_hpu/ --mode=lazy # or --mode=compile
pytest test/unit/test_xpu/
# Slower suites (nightly / weekly)
pytest test/integration/test_cpu/
pytest test/e2e/test_cpu/
Test fixtures create tiny models (OPT-125M, Qwen-0.6B) at session scope — first run downloads them.
Code Quality & Pre-commit
All syntax, formatting, licensing, spelling, and linting checks are enforced via .pre-commit-config.yaml.
Execution Rules for Agents
- Always run pre-commit after editing files:
- For focused changes (preferred):
pre-commit run --files <changed_file1> <changed_file2> - For a specific hook/tool (e.g.,
ruff,black,codespell):pre-commit run <hook_id> --files <changed_file> - For repository-wide checks (recommended for extensive changes or initial runs; fast and lightweight):
pre-commit run --all-files
- For focused changes (preferred):
- Rerun until clean: Tools like
ruffandcodespellautomatically fix issues but will exit with non-zero on first modification (--exit-non-zero-on-fix). Keep changes and rerun the command until all checks pass. - Manual fixes: Only make manual changes if a check fails and cannot be auto-fixed (e.g.,
banditsecurity findings,check-json, or complex syntax errors).
Automated Checks Summary
- Auto-fixing hooks:
black(line length 120),isort(imports),ruff(lint fixes),insert-license(Apache 2.0 header),codespell(typos),mixed-line-ending(LF). - Check-only hooks:
bandit(security issues underauto_round/),typos,check-yaml/check-json,markdown-link-check. - Note:
auto_round/export/export_to_gguf/conversion/is intentionally excluded from most hooks.
Commit & PR Conventions
- Conventional commits:
feat:,fix:,chore:,docs:,refactor:,test: - PRs target
main, squash-merged - CN docs rule: any change to a
.mdfile must include a matching update to its_CNcounterpart (e.g.,README.md→README_CN.md)
Key Environment Variables
BUILD_HPU_ONLY=1— build HPU package variantAR_USE_MODELSCOPE=1— use ModelScope instead of HuggingFace for model downloadsFORCE_BF16=1— force BF16 in tests (used in CI)
Source Layout
auto_round/— core library (AutoRound class, sign-SGD, exporters, eval, data types)auto_round_extension/— hardware backends (CUDA, HPU, IPEX/XPU, Triton, ARK, vLLM)test/— tests organized by tier then hardware:unit/(PR CI),integration/(nightly),e2e/(weekly), each withtest_cpu/,test_cuda/, ...examples/— usage examples for different model types
Gotchas
setup.pyforcesCC=CXX=g++at import time- Version is computed dynamically from git tags — untagged commits produce dev versions
- Some test dependencies (AutoAWQ, GPTQModel, llama-cpp) require manual git installs — see comments in
test/unit/test_cuda/requirements.txt