Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

4.1 KB
Raw Blame

AGENTS.md

This file provides guidance to ANY AGENTS when working with code in this repository.

Project

AutoRound — post-training quantization for LLMs/VLMs using sign-gradient descent. Publishes as auto-round (GPU/CPU) and auto-round-hpu (Intel Gaudi).

Build & Install

# From source (GPU/CPU) — --no-build-isolation is required when PyTorch is already installed
pip install --no-build-isolation -e .

# HPU variant
BUILD_HPU_ONLY=1 pip install --no-build-isolation .
# or: python setup.py hpu install

# XPU variant — install Intel PyTorch first
pip install torch --index-url https://download.pytorch.org/whl/xpu
pip install --no-build-isolation .

Testing

Tests are split into three tiers under test/: unit/ (fast, runs in PR CI), integration/ (third-party frameworks, runs nightly), and e2e/ (full models / real inference engines, runs weekly). Each tier is further split by hardware (test_cpu/, test_cuda/, test_hpu/, test_xpu/, test_ark/, test_mlx/).

# CPU unit tests (most common during development)
pytest test/unit/test_cpu/ -x -q

# Single test
pytest test/unit/test_cpu/ -k "test_name" -x -q

# Hardware-specific unit tests
pytest test/unit/test_cuda/
pytest test/unit/test_hpu/ --mode=lazy   # or --mode=compile
pytest test/unit/test_xpu/

# Slower suites (nightly / weekly)
pytest test/integration/test_cpu/
pytest test/e2e/test_cpu/

Test fixtures create tiny models (OPT-125M, Qwen-0.6B) at session scope — first run downloads them.

Code Quality & Pre-commit

All syntax, formatting, licensing, spelling, and linting checks are enforced via .pre-commit-config.yaml.

Execution Rules for Agents

  1. Always run pre-commit after editing files:
    • For focused changes (preferred):
      pre-commit run --files <changed_file1> <changed_file2>
      
    • For a specific hook/tool (e.g., ruff, black, codespell):
      pre-commit run <hook_id> --files <changed_file>
      
    • For repository-wide checks (recommended for extensive changes or initial runs; fast and lightweight):
      pre-commit run --all-files
      
  2. Rerun until clean: Tools like ruff and codespell automatically fix issues but will exit with non-zero on first modification (--exit-non-zero-on-fix). Keep changes and rerun the command until all checks pass.
  3. Manual fixes: Only make manual changes if a check fails and cannot be auto-fixed (e.g., bandit security findings, check-json, or complex syntax errors).

Automated Checks Summary

  • Auto-fixing hooks: black (line length 120), isort (imports), ruff (lint fixes), insert-license (Apache 2.0 header), codespell (typos), mixed-line-ending (LF).
  • Check-only hooks: bandit (security issues under auto_round/), typos, check-yaml/check-json, markdown-link-check.
  • Note: auto_round/export/export_to_gguf/conversion/ is intentionally excluded from most hooks.

Commit & PR Conventions

  • Conventional commits: feat:, fix:, chore:, docs:, refactor:, test:
  • PRs target main, squash-merged
  • CN docs rule: any change to a .md file must include a matching update to its _CN counterpart (e.g., README.md → README_CN.md)

Key Environment Variables

  • BUILD_HPU_ONLY=1 — build HPU package variant
  • AR_USE_MODELSCOPE=1 — use ModelScope instead of HuggingFace for model downloads
  • FORCE_BF16=1 — force BF16 in tests (used in CI)

Source Layout

  • auto_round/ — core library (AutoRound class, sign-SGD, exporters, eval, data types)
  • auto_round_extension/ — hardware backends (CUDA, HPU, IPEX/XPU, Triton, ARK, vLLM)
  • test/ — tests organized by tier then hardware: unit/ (PR CI), integration/ (nightly), e2e/ (weekly), each with test_cpu/, test_cuda/, ...
  • examples/ — usage examples for different model types

Gotchas

  • setup.py forces CC=CXX=g++ at import time
  • Version is computed dynamically from git tags — untagged commits produce dev versions
  • Some test dependencies (AutoAWQ, GPTQModel, llama-cpp) require manual git installs — see comments in test/unit/test_cuda/requirements.txt