Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

Add markdown-link-check (#1531)

Sun, Xuehao50c4c7f4e581
diff --git a/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json b/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json
new file mode 100644
index 000000000..677b45bbd
--- /dev/null
+++ b/.azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json
@@ -0,0 +1,7 @@
+{
+ "ignorePatterns": [
+ {
+ "pattern": "^https?://"
+ }
+ ]
+}
\ No newline at end of file
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index c32089c4a..9b966be05 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -66,3 +66,9 @@ repos:
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix, --no-cache]
+
+ - repo: https://github.com/tcort/markdown-link-check
+ rev: v3.14.2
+ hooks:
+ - id: markdown-link-check
+ args: [-q, -c .azure-pipelines/scripts/codeScan/markdown-link-check/markdown-link-check.json]
\ No newline at end of file
diff --git a/README.md b/README.md
index 81d172a57..d38161bfc 100644
--- a/README.md
+++ b/README.md
@@ -294,7 +294,7 @@ ar.quantize_and_save()
By default, AutoRound only quantize the text module of VLMs and uses `NeelNanda/pile-10k` for calibration. To
quantize the entire model, you can enable `quant_nontext_module` by setting it to True, though support for this feature
-is limited. For more information, please refer to the AutoRound [readme](./auto_round/mllm/README.md).
+is limited. For more information, please refer to the AutoRound [readme](auto_round/compressors/mllm/README.md).
```python
from auto_round import AutoRound
diff --git a/README_CN.md b/README_CN.md
index b950b4dd3..7a98fbb36 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -287,7 +287,7 @@ ar.quantize_and_save()
**该功能仍在实验阶段**
-默认情况下,AutoRound 只会量化 VLM 的文本模块,并默认采用 `NeelNanda/pile-10k`​ 作为标定数据集。若需量化整个模型,可设置 `quant_nontext_module = True` (但目前为止该功能的适用范围仍较为有限)。更多信息请参考 [readme](./auto_round/mllm/README.md)
+默认情况下,AutoRound 只会量化 VLM 的文本模块,并默认采用 `NeelNanda/pile-10k`​ 作为标定数据集。若需量化整个模型,可设置 `quant_nontext_module = True` (但目前为止该功能的适用范围仍较为有限)。更多信息请参考 [readme](auto_round/compressors/mllm/README.md)
```python
from auto_round import AutoRound
diff --git a/auto_round/compressors/diffusion/README.md b/auto_round/compressors/diffusion/README.md
index 16dc87d59..c5b307df5 100644
--- a/auto_round/compressors/diffusion/README.md
+++ b/auto_round/compressors/diffusion/README.md
@@ -43,7 +43,7 @@ autoround.quantize_and_save(output_dir, format="fake", inplace=True)
- `generator_seed`: A seed that controls the initial noise from which an image is generated.
-for more hyperparameters introduction, please refer [Homepage Detailed Hyperparameters](../../README.md#api-usage-gaudi2cpugpu)
+for more hyperparameters introduction, please refer [Homepage Detailed Hyperparameters](../../../README.md#quantization-scheme--configuration)
### CLI Usage
@@ -73,7 +73,7 @@ For diffusion models, currently we only validate quantizaion on the FLUX.1-dev,
<details>
<summary style="font-size:17px;">Calibration Dataset</summary>
-For diffusion models, we used [**coco2014**]("https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv") calibration dataset as our default.
+For diffusion models, we used [**coco2014**](https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv) calibration dataset as our default.
If users want to use their own dataset, please build the dataset file in ".tsv" format following below structure and use it through argument --dataset (tsv file):
```
diff --git a/auto_round/compressors/mllm/README.md b/auto_round/compressors/mllm/README.md
index 7963ef8e0..c28cc0520 100644
--- a/auto_round/compressors/mllm/README.md
+++ b/auto_round/compressors/mllm/README.md
@@ -28,7 +28,7 @@ autoround.quantize_and_save(output_dir, format="auto_round")
- `quant_nontext_module`: whether to quantize non-text module, e.g. vision component.
for more hyperparameters introduction, please
-refer [Homepage Detailed Hyperparameters](../../README.md#api-usage-gaudi2cpugpu)
+refer [Homepage Detailed Hyperparameters](../../../README.md#quantization-scheme--configuration)
### Basic Usage
diff --git a/docs/step_by_step.md b/docs/step_by_step.md
index 5ee04692b..dde5006f8 100644
--- a/docs/step_by_step.md
+++ b/docs/step_by_step.md
@@ -29,9 +29,8 @@ This document presents step-by-step instructions for auto-round llm quantization
+ [OPT RTN mode](#opt-rtn-mode)
+ [GGUF format](#gguf-format)
+ [Quantization Costs](#quantization-costs)
- + [Device/Multi-GPU setting in Quantization](#device-multi-gpu-setting-in-quantization)
- - [Enable multiple gpus calibration in lm_head quantization](#enable-multiple-gpus-calibration-in-lm-head-quantization)
- - [Enable multiple gpus tuning for extremely large model](#enable-multiple-gpus-tuning-for-extremely-large-model)
+ + [Device/Multi-GPU setting in Quantization](#devicemulti-gpu-setting-in-quantization)
+ - [Enable multiple gpus calibration in lm_head quantization](#enable-multiple-gpus-calibration-in-lm_head-quantization)
+ [Adjust Hyperparameters](#adjust-hyperparameters)
* [4 Inference](#4-inference)
+ [CPU](#cpu)
@@ -39,7 +38,7 @@ This document presents step-by-step instructions for auto-round llm quantization
+ [CUDA](#cuda)
+ [HPU](#hpu)
+ [Specify Inference Backend](#specify-inference-backend)
- + [Convert GPTQ/AWQ to AutoRound](#convert-gptq-awq-to-autoround)
+ + [Convert GPTQ/AWQ to AutoRound](#convert-gptqawq-to-autoround)
* [5 Evaluation](#5-evaluation)
+ [Single GPU Evaluation](#single-gpu-evaluation)
+ [Multi-GPU Evaluation](#multi-gpu-evaluation)
diff --git a/docs/step_by_step_CN.md b/docs/step_by_step_CN.md
index 845fe594b..3a4801a25 100644
--- a/docs/step_by_step_CN.md
+++ b/docs/step_by_step_CN.md
@@ -16,33 +16,33 @@
+ [硬件兼容性](#硬件兼容性)
+ [环境参数配置](#环境参数配置)
+ [命令行使用方法](#命令行用法)
- + [API 使用方法](#API-使用方法)
- - [AutoRound API 基础用法](#AutoRound-API-基础用法)
+ + [API 使用方法](#api-使用方法)
+ - [AutoRound API 基础用法](#autoround-api-基础用法)
- [混合精度量化方案](#混合精度量化)
- - [AutoRoundBest 配置方案](#AutoRoundBest-高精度配置用法)
- - [AutoRoundLight 配置方案](#AutoRoundLight-高速度配置用法)
+ - [AutoRoundBest 配置方案](#autoroundbest-高精度配置用法)
+ - [AutoRoundLight 配置方案](#autoroundlight-高速度配置用法)
- [超参方案推荐](#超参方案推荐)
- + [AutoScheme 自动混合精度量化方案](#AutoScheme-自动混合精度量化方案)
+ + [AutoScheme 自动混合精度量化方案](#autoscheme-自动混合精度量化方案)
- [命令行用法](#命令行用法-1)
- - [API 用法](#API-用法)
- - [AutoScheme 中的超参数](#AutoScheme-超参数说明)
- + [OPT RTN 模式](#OPT-RTN-模式)
- + [GGUF 格式](#GGUF-格式量化)
+ - [API 用法](#api-用法)
+ - [AutoScheme 中的超参数](#autoscheme-超参数说明)
+ + [OPT RTN 模式](#opt-rtn-模式)
+ + [GGUF 格式](#gguf-格式量化)
+ [量化成本](#量化成本)
+ [设备及多卡量化设置](#设备及多卡量化设置)
- - [lm_head 量化中开启多 GPU 标定](#lm_head-量化中开启多-GPU-标定)
+ - [lm_head 量化中开启多 GPU 标定](#lm_head-量化中开启多-gpu-标定)
- [手动配置设备映射](#手动配置设备映射)
+ [超参数调整](#超参数调整)
* [4 推理部署](#4-推理部署)
- + [CPU](#CPU)
- + [英特尔 GPU](#英特尔-GPU)
- + [CUDA](#CUDA)
- + [HPU](#HPU)
+ + [CPU](#cpu)
+ + [英特尔 GPU](#英特尔-gpu)
+ + [CUDA](#cuda)
+ + [HPU](#hpu)
+ [指定推理后端](#指定推理后端)
- + [将 GPTQ/AWQ 模型转换为 AutoRound 格式](#将-GPTQ-或-AWQ-模型转换为-AutoRound-格式)
+ + [将 GPTQ/AWQ 模型转换为 AutoRound 格式](#将-gptq-或-awq-模型转换为-autoround-格式)
* [5 效果评估](#5-效果评估)
- + [单卡评估](#单-GPU-评估)
- + [多卡评估](#多-GPU-评估)
+ + [单卡评估](#单-gpu-评估)
+ + [多卡评估](#多-gpu-评估)
+ [注意事项](#注意事项)
* [6 已知问题](#6-已知问题)
@@ -208,7 +208,7 @@ ar.quantize_and_save(output_dir, format="auto_gptq,auto_awq,auto_round")
```
#### 混合精度量化
-自 0.8 版本起,AutoRound 提供了 AutoScheme 功能,可自动生成混合精度方案,详情请参阅 [Auto Scheme自动方案](#autoscheme)章节。
+自 0.8 版本起,AutoRound 提供了 AutoScheme 功能,可自动生成混合精度方案,详情请参阅 [Auto Scheme自动方案](#autoscheme-自动混合精度量化方案)章节。
Auto-GPTQ 和 Auto-AWQ 仅支持有限的混合精度。如果您不熟悉具体细节,**建议导出 AutoRound 格式**。