Skip to content

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

update document for eval (#1140)

Xin He76b100c8d99c
diff --git a/docs/step_by_step.md b/docs/step_by_step.md
index 2a842dcc2..edfbdcc93 100644
--- a/docs/step_by_step.md
+++ b/docs/step_by_step.md
@@ -728,7 +728,6 @@ If not explicitly specify '--task', the default value will be used (typically co
~~~
The last format will be used in evaluation if multiple formats have been exported.
-Note: To use the vllm backend, please add `--eval_backend vllm` to the command above.
### Eval the Quantized model
@@ -737,6 +736,8 @@ Note: To use the vllm backend, please add `--eval_backend vllm` to the command
~~~bash
auto-round --model="your_model_path" --eval --tasks lambada_openai --eval_bs 16
~~~
+ > Note: To use the vllm backend, add `--eval_backend vllm` to the command above. Common vllm parameters are already supported, such as `--tensor_parallel_size`.
+
Multiple gpu evaluation
~~~bash
auto-round --model="your_model_path" --eval --device 0,1 --tasks lambada_openai --eval_bs 16
@@ -760,6 +761,7 @@ Note: To use the vllm backend, please add `--eval_backend vllm` to the command
~~~
+
## 6 Known Issues
Randomness in quantization may affect tuning results for some models, set `enable_deterministic_algorithms=True` to ensure reproducibility.