Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

5.4 KB
Raw Blame

Quantization recipes

Recipes are the shared recommendation source for the Python API and CLI. Omitting recipe selects auto-round. A recipe chooses a complete algorithm pipeline, calibration defaults, and optional per-layer recommendations. Scheme definitions and algorithm implementations remain independent of recipes.

Built-in plans

Recipe Initial pipeline Iterations Samples
auto-round SignRound 200 128
auto-round-light SignRound, learning rate 0.005 50 128
auto-round-best SignRound 1000 512
rtn Plain RTN — 1
opt-rtn Optimized RTN — 128
awq AWQ smoothing → plain RTN — 128

All initial plans use sequence length 2048 and batch size 8. Names are case-insensitive and accept the following aliases in both Python and the CLI:

Canonical name Aliases
auto-round auto_round, ar, default
auto-round-light auto_round_light, ar-light, ar_light, light
auto-round-best auto_round_best, ar-best, ar_best, best
opt-rtn opt_rtn

Listings and resolved recipe names use the canonical names, matching the existing CLI naming. The three auto-round tiers express cost/accuracy goals: their pipelines may gain validated scheme-specific recommendations in future releases. They do not guarantee an accuracy ordering on every model. Core calibration requirements still apply, including static activation calibration for RTN when required.

Usage and overrides

from auto_round import AutoRound, SignRoundConfig

compressor = AutoRound("Qwen/Qwen3-0.6B", scheme="MXFP4", recipe="auto-round-best")

# Explicit values win even when equal to an older API default.
compressor = AutoRound("Qwen/Qwen3-0.6B", recipe="auto-round-best", nsamples=128)

# Replace the entire algorithm pipeline, retaining auto-round-best calibration defaults.
compressor = AutoRound("Qwen/Qwen3-0.6B", recipe="auto-round-best", alg_configs=SignRoundConfig(iters=20))
auto_round Qwen/Qwen3-0.6B --scheme MXFP4 --recipe auto-round-best
auto_round Qwen/Qwen3-0.6B --recipe auto-round-best --nsamples 128 --iters 20
auto_round Qwen/Qwen3-0.6B --recipe auto-round-best --algorithm awq
auto_round list recipes
auto_round list recipe awq

Explicit algorithm configs replace the recipe pipeline, rather than appending to it. Their algorithm parameters are not overwritten by recipe stage parameters. The deprecated algorithm= Python argument and CLI --algorithm retain their existing algorithm-selection semantics. Explicit iters=0 still selects RTN. Explicit layer entries override recipe entries per pattern and per field; a scalar scheme entry replaces the entire entry. Capability checks still reject unsupported combinations regardless of recipe overrides.

auto_round_best, auto_round_light, auto_round_rtn, and auto_round_opt_rtn select the corresponding recipe. Their hyphenated spellings work identically. An explicit --recipe overrides the shortcut's default selection.

Inspecting and extending recipes

from auto_round.recipes import get_recipe, list_recipes, resolve_recipe

print(list_recipes())
definition = get_recipe("auto-round-best")
plan = resolve_recipe(definition)
print(plan.pipeline, plan.calibration)
configs = plan.build_configs()

resolve_recipe() returns an independent, processed Recipe copy. Each build_configs() call creates fresh algorithm configs. A constructed compressor exposes its recommendation as compressor.resolved_recipe; this is the selected recommendation, not a replacement for the effective config after explicit overrides and validation.

For custom recipes, use Recipe and AlgorithmSpec. A recipe can provide an ordinary Python function to adjust its defaults:

from auto_round.recipes import AlgorithmSpec, Recipe, resolve_recipe


def configure(recipe, *, scheme, model_config):
    if scheme == "MXFP4":
        recipe.calibration["nsamples"] = 256
    if getattr(model_config, "model_type", None) == "some_model":
        recipe.layer_config["lm_head"] = "BF16"


custom = Recipe(
    "custom",
    pipeline=(AlgorithmSpec("quantizer", "auto_round", {"iters": 50}),),
    process=configure,
)
plan = resolve_recipe(custom, scheme="MXFP4")

This example illustrates the API, not a benchmarked recommendation. The function edits an independent copy of the recipe and returns nothing. It can replace recipe.pipeline directly or modify algorithm parameters, calibration, and layer defaults using ordinary Python. There is no conditional patch language or merge order to learn. Calibration keys are dataset, seqlen, nsamples, and batch_size.

Pass a Recipe directly to AutoRound or register it with register_recipe to use its name in Python and an in-process CLI. Registration rejects duplicate names and aliases. The resolver accepts scheme and model_config directly. Entry points provide the model; configuration metadata is looked up for string inputs only when a processing function exists. If lookup is unavailable, model_config is None. No model weights are loaded during recipe processing. Existing mixed-scheme handling remains in auto_round/schemes.py, and model layer defaults remain in auto_round/special_model_handler.py. They run after model loading for both Python and CLI. Shape restrictions and model compatibility patches also remain outside recipes.

AutoScheme remains the independent mixed-bit scheme-selection feature.