PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Commits on main
vllm/llama-vision llava calibration infinite loop fix (#343)
a7fcb51 WeiweiZhang1 committed 1 year ago
fix eval_bs in fake format and reset auto-gptq exporting max_shard_size (#332)
8fb9552 Wenhua Cheng committed 1 year ago
fix mllm dataset config bug and add gptq cuda backend (#329)
56e3400 Wenhua Cheng committed 1 year ago
fix fp_layers issue and force to FP16 on cuda for autoround format inference (#326)
459aab0 wenhuach21 committed 2 years ago