PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
update document for eval (#1140)
Xin He76b100c8d99c
diff --git a/docs/step_by_step.md b/docs/step_by_step.mdindex 2a842dcc2..edfbdcc93 100644--- a/docs/step_by_step.md+++ b/docs/step_by_step.md@@ -728,7 +728,6 @@ If not explicitly specify '--task', the default value will be used (typically co~~~The last format will be used in evaluation if multiple formats have been exported.-Note: To use the vllm backend, please add `--eval_backend vllm` to the command above.### Eval the Quantized model@@ -737,6 +736,8 @@ Note: To use the vllm backend, please add `--eval_backend vllm` to the command~~~bashauto-round --model="your_model_path" --eval --tasks lambada_openai --eval_bs 16~~~+ > Note: To use the vllm backend, add `--eval_backend vllm` to the command above. Common vllm parameters are already supported, such as `--tensor_parallel_size`.+Multiple gpu evaluation~~~bashauto-round --model="your_model_path" --eval --device 0,1 --tasks lambada_openai --eval_bs 16@@ -760,6 +761,7 @@ Note: To use the vllm backend, please add `--eval_backend vllm` to the command~~~+## 6 Known IssuesRandomness in quantization may affect tuning results for some models, set `enable_deterministic_algorithms=True` to ensure reproducibility.