PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Improve the low_cpu_mem_usage usage in the doc (#2278)
Haihao Shen32369a69cd90
diff --git a/README.md b/README.mdindex cdeaef4cb..8dc397fa2 100644--- a/README.md+++ b/README.md@@ -267,7 +267,7 @@ ar.quantize_and_save(output_dir="./qmodel", format="auto_round")##### Device/Speed Configuration- **`enable_torch_compile` (bool)**: Enabling `torch.compile` may speed up quantization, but compilation overhead can change peak memory and may increase it for some models or quantization schemes. It is enabled by default except on Windows, where it is disabled because TorchInductor requires the MSVC `cl.exe` compiler. On Windows, pass `enable_torch_compile=True` in Python or `--enable_torch_compile` on the CLI to force enable it. On other platforms, use `enable_torch_compile=False` or `--disable_torch_compile` to opt out.- **`low_gpu_mem_usage` (bool)**: Whether to offload intermediate features to CPU at the cost of ~30%-100% more tuning time (default is `False`).-- **`low_cpu_mem_usage` (bool)**: [Experimental Feature]Whether to enable saving immediately to reduce ram usage (default is `True`).+- **`low_cpu_mem_usage` (bool)**: [Experimental Feature] Whether to enable immediate-save mode to reduce RAM usage. It is enabled by default. In CLI, use `--disable_low_cpu_mem_usage` to turn it off (the legacy `--low_cpu_mem_usage` flag is kept only for compatibility).- **`device_map` (str|dict|int)**: The device to be used for tuning, e.g., `auto`, `cpu`, `cuda`, `0,1,2` (default is `0`). When using `auto`, it will try to use all available GPUs.</details>diff --git a/README_CN.md b/README_CN.mdindex f149bec6c..a09734492 100644--- a/README_CN.md+++ b/README_CN.md@@ -263,7 +263,7 @@ ar.quantize_and_save(output_dir="./qmodel", format="auto_round")- **`enable_torch_compile`**(bool):启用 `torch.compile` 可能提升量化速度,但编译开销会改变峰值内存,某些模型或量化方案的峰值内存可能增加。除 Windows 外默认开启;Windows 上默认关闭,因为 TorchInductor 需要 MSVC 的 `cl.exe` 编译器。在 Windows 上可通过 Python 参数 `enable_torch_compile=True` 或命令行参数 `--enable_torch_compile` 强制开启;其他平台可使用 `enable_torch_compile=False` 或 `--disable_torch_compile` 关闭。- **`low_gpu_mem_usage`**(bool):若要节省显存,可以设为 `True` 。它会将中间特征卸载到 CPU,但会增加 30%-100% 的时间(默认 `False`)。-- **`low_cpu_mem_usage`**(bool):[实验性功能] 若要减少内存占用,可以设为 `True` 来启用即时保存(默认 `True`)。+- **`low_cpu_mem_usage`**(bool):[实验性功能] 用于启用即时保存以减少内存占用。该功能默认开启;在 CLI 中可使用 `--disable_low_cpu_mem_usage` 关闭(`--low_cpu_mem_usage` 仅为兼容旧参数保留)。- **`device_map`**(str | dict | int):计算设备指定,如 `auto`、`cpu`、`cuda`、`0,1,2`(默认 `0`)。使用 `auto` 时会尝试利用所有可用 GPU。</details>diff --git a/docs/step_by_step.md b/docs/step_by_step.mdindex 682f2bdb2..d8221a92c 100755--- a/docs/step_by_step.md+++ b/docs/step_by_step.md@@ -933,7 +933,7 @@ autoround.save_quantized(format="auto_awq", output_dir="tmp_autoround")- **Reduced CPU Memory Usage :**- - Enable `low_cpu_mem_usage` (experimental): Only one export format is supported. The quantized model is saved immediately after each block is packed, reducing peak CPU memory usage.+ - `low_cpu_mem_usage` (experimental) is enabled by default. In CLI, use `--disable_low_cpu_mem_usage` to turn it off (`--low_cpu_mem_usage` is retained only for compatibility). Only one export format is supported. The quantized model is saved immediately after each block is packed, reducing peak CPU memory usage.- Trigger immediate packing: Packing will be triggered immediately when using the command-line interface or thequantize_and_save API, as long as only one export format is specified.diff --git a/docs/step_by_step_CN.md b/docs/step_by_step_CN.mdindex 9a0563822..4dced5f01 100755--- a/docs/step_by_step_CN.md+++ b/docs/step_by_step_CN.md@@ -905,7 +905,7 @@ autoround.save_quantized(format="auto_awq", output_dir="tmp_autoround")- 将 `seqlen` 降至 512(**可能会有精度损失**)#### 降低 CPU 内存占用-- 开启 `low_cpu_mem_usage`(实验性功能):仅支持**导出指定一种格式**。每个 block 量化封装完成后会立即保存,从而降低峰值内存占用。+- `low_cpu_mem_usage`(实验性功能)默认开启;CLI 中可使用 `--disable_low_cpu_mem_usage` 关闭(`--low_cpu_mem_usage` 仅为兼容旧参数保留)。仅支持**导出指定一种格式**。每个 block 量化封装完成后会立即保存,从而降低峰值内存占用。- 触发立即封装:使用命令行或 `quantize_and_save` API 时,只要指定**单一导出格式**,就会自动触发即时打包,无需额外配置。#### 提升训练速度