PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Add markdown-link-check (#1531)
Sun, Xuehao50c4c7f4e581
diff --git a/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json b/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.jsonnew file mode 100644index 000000000..677b45bbd--- /dev/null+++ b/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json@@ -0,0 +1,7 @@+{+ "ignorePatterns": [+ {+ "pattern": "^https?://"+ }+ ]+}\ No newline at end of filediff --git a/.pre-commit-config.yaml b/.pre-commit-config.yamlindex c32089c4a..9b966be05 100644--- a/.pre-commit-config.yaml+++ b/.pre-commit-config.yaml@@ -66,3 +66,9 @@ repos:hooks:- id: ruffargs: [--fix, --exit-non-zero-on-fix, --no-cache]++ - repo: https://github.com/tcort/markdown-link-check+ rev: v3.14.2+ hooks:+ - id: markdown-link-check+ args: [-q, -c .azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json]\ No newline at end of filediff --git a/README.md b/README.mdindex 81d172a57..d38161bfc 100644--- a/README.md+++ b/README.md@@ -294,7 +294,7 @@ ar.quantize_and_save()By default, AutoRound only quantize the text module of VLMs and uses `NeelNanda/pile-10k` for calibration. Toquantize the entire model, you can enable `quant_nontext_module` by setting it to True, though support for this feature-is limited. For more information, please refer to the AutoRound [readme](./auto_round/mllm/README.md).+is limited. For more information, please refer to the AutoRound [readme](auto_round/compressors/mllm/README.md).```pythonfrom auto_round import AutoRounddiff --git a/README_CN.md b/README_CN.mdindex b950b4dd3..7a98fbb36 100644--- a/README_CN.md+++ b/README_CN.md@@ -287,7 +287,7 @@ ar.quantize_and_save()**该功能仍在实验阶段**-默认情况下,AutoRound 只会量化 VLM 的文本模块,并默认采用 `NeelNanda/pile-10k` 作为标定数据集。若需量化整个模型,可设置 `quant_nontext_module = True` (但目前为止该功能的适用范围仍较为有限)。更多信息请参考 [readme](./auto_round/mllm/README.md)+默认情况下,AutoRound 只会量化 VLM 的文本模块,并默认采用 `NeelNanda/pile-10k` 作为标定数据集。若需量化整个模型,可设置 `quant_nontext_module = True` (但目前为止该功能的适用范围仍较为有限)。更多信息请参考 [readme](auto_round/compressors/mllm/README.md)```pythonfrom auto_round import AutoRounddiff --git a/auto_round/compressors/diffusion/README.md b/auto_round/compressors/diffusion/README.mdindex 16dc87d59..c5b307df5 100644--- a/auto_round/compressors/diffusion/README.md+++ b/auto_round/compressors/diffusion/README.md@@ -43,7 +43,7 @@ autoround.quantize_and_save(output_dir, format="fake", inplace=True)- `generator_seed`: A seed that controls the initial noise from which an image is generated.-for more hyperparameters introduction, please refer [Homepage Detailed Hyperparameters](../../README.md#api-usage-gaudi2cpugpu)+for more hyperparameters introduction, please refer [Homepage Detailed Hyperparameters](../../../README.md#quantization-scheme--configuration)### CLI Usage@@ -73,7 +73,7 @@ For diffusion models, currently we only validate quantizaion on the FLUX.1-dev,<details><summary style="font-size:17px;">Calibration Dataset</summary>-For diffusion models, we used [**coco2014**]("https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv") calibration dataset as our default.+For diffusion models, we used [**coco2014**](https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv) calibration dataset as our default.If users want to use their own dataset, please build the dataset file in ".tsv" format following below structure and use it through argument --dataset (tsv file):```diff --git a/auto_round/compressors/mllm/README.md b/auto_round/compressors/mllm/README.mdindex 7963ef8e0..c28cc0520 100644--- a/auto_round/compressors/mllm/README.md+++ b/auto_round/compressors/mllm/README.md@@ -28,7 +28,7 @@ autoround.quantize_and_save(output_dir, format="auto_round")- `quant_nontext_module`: whether to quantize non-text module, e.g. vision component.for more hyperparameters introduction, please-refer [Homepage Detailed Hyperparameters](../../README.md#api-usage-gaudi2cpugpu)+refer [Homepage Detailed Hyperparameters](../../../README.md#quantization-scheme--configuration)### Basic Usagediff --git a/docs/step_by_step.md b/docs/step_by_step.mdindex 5ee04692b..dde5006f8 100644--- a/docs/step_by_step.md+++ b/docs/step_by_step.md@@ -29,9 +29,8 @@ This document presents step-by-step instructions for auto-round llm quantization+ [OPT RTN mode](#opt-rtn-mode)+ [GGUF format](#gguf-format)+ [Quantization Costs](#quantization-costs)- + [Device/Multi-GPU setting in Quantization](#device-multi-gpu-setting-in-quantization)- - [Enable multiple gpus calibration in lm_head quantization](#enable-multiple-gpus-calibration-in-lm-head-quantization)- - [Enable multiple gpus tuning for extremely large model](#enable-multiple-gpus-tuning-for-extremely-large-model)+ + [Device/Multi-GPU setting in Quantization](#devicemulti-gpu-setting-in-quantization)+ - [Enable multiple gpus calibration in lm_head quantization](#enable-multiple-gpus-calibration-in-lm_head-quantization)+ [Adjust Hyperparameters](#adjust-hyperparameters)* [4 Inference](#4-inference)+ [CPU](#cpu)@@ -39,7 +38,7 @@ This document presents step-by-step instructions for auto-round llm quantization+ [CUDA](#cuda)+ [HPU](#hpu)+ [Specify Inference Backend](#specify-inference-backend)- + [Convert GPTQ/AWQ to AutoRound](#convert-gptq-awq-to-autoround)+ + [Convert GPTQ/AWQ to AutoRound](#convert-gptqawq-to-autoround)* [5 Evaluation](#5-evaluation)+ [Single GPU Evaluation](#single-gpu-evaluation)+ [Multi-GPU Evaluation](#multi-gpu-evaluation)diff --git a/docs/step_by_step_CN.md b/docs/step_by_step_CN.mdindex 845fe594b..3a4801a25 100644--- a/docs/step_by_step_CN.md+++ b/docs/step_by_step_CN.md@@ -16,33 +16,33 @@+ [硬件兼容性](#硬件兼容性)+ [环境参数配置](#环境参数配置)+ [命令行使用方法](#命令行用法)- + [API 使用方法](#API-使用方法)- - [AutoRound API 基础用法](#AutoRound-API-基础用法)+ + [API 使用方法](#api-使用方法)+ - [AutoRound API 基础用法](#autoround-api-基础用法)- [混合精度量化方案](#混合精度量化)- - [AutoRoundBest 配置方案](#AutoRoundBest-高精度配置用法)- - [AutoRoundLight 配置方案](#AutoRoundLight-高速度配置用法)+ - [AutoRoundBest 配置方案](#autoroundbest-高精度配置用法)+ - [AutoRoundLight 配置方案](#autoroundlight-高速度配置用法)- [超参方案推荐](#超参方案推荐)- + [AutoScheme 自动混合精度量化方案](#AutoScheme-自动混合精度量化方案)+ + [AutoScheme 自动混合精度量化方案](#autoscheme-自动混合精度量化方案)- [命令行用法](#命令行用法-1)- - [API 用法](#API-用法)- - [AutoScheme 中的超参数](#AutoScheme-超参数说明)- + [OPT RTN 模式](#OPT-RTN-模式)- + [GGUF 格式](#GGUF-格式量化)+ - [API 用法](#api-用法)+ - [AutoScheme 中的超参数](#autoscheme-超参数说明)+ + [OPT RTN 模式](#opt-rtn-模式)+ + [GGUF 格式](#gguf-格式量化)+ [量化成本](#量化成本)+ [设备及多卡量化设置](#设备及多卡量化设置)- - [lm_head 量化中开启多 GPU 标定](#lm_head-量化中开启多-GPU-标定)+ - [lm_head 量化中开启多 GPU 标定](#lm_head-量化中开启多-gpu-标定)- [手动配置设备映射](#手动配置设备映射)+ [超参数调整](#超参数调整)* [4 推理部署](#4-推理部署)- + [CPU](#CPU)- + [英特尔 GPU](#英特尔-GPU)- + [CUDA](#CUDA)- + [HPU](#HPU)+ + [CPU](#cpu)+ + [英特尔 GPU](#英特尔-gpu)+ + [CUDA](#cuda)+ + [HPU](#hpu)+ [指定推理后端](#指定推理后端)- + [将 GPTQ/AWQ 模型转换为 AutoRound 格式](#将-GPTQ-或-AWQ-模型转换为-AutoRound-格式)+ + [将 GPTQ/AWQ 模型转换为 AutoRound 格式](#将-gptq-或-awq-模型转换为-autoround-格式)* [5 效果评估](#5-效果评估)- + [单卡评估](#单-GPU-评估)- + [多卡评估](#多-GPU-评估)+ + [单卡评估](#单-gpu-评估)+ + [多卡评估](#多-gpu-评估)+ [注意事项](#注意事项)* [6 已知问题](#6-已知问题)@@ -208,7 +208,7 @@ ar.quantize_and_save(output_dir, format="auto_gptq,auto_awq,auto_round")```#### 混合精度量化-自 0.8 版本起,AutoRound 提供了 AutoScheme 功能,可自动生成混合精度方案,详情请参阅 [Auto Scheme自动方案](#autoscheme)章节。+自 0.8 版本起,AutoRound 提供了 AutoScheme 功能,可自动生成混合精度方案,详情请参阅 [Auto Scheme自动方案](#autoscheme-自动混合精度量化方案)章节。Auto-GPTQ 和 Auto-AWQ 仅支持有限的混合精度。如果您不熟悉具体细节,**建议导出 AutoRound 格式**。