PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Commits on main
fix missing change for compress_layer_names and add UTs (#1556)
6d17bbf Xin He committed 8 months ago
Fix shard finalization issue to prevent skipping non-LLM tail layers (#1548)
81e79a1 Liang Lv committed 8 months ago
Optimize mtp tensor process by compiling layer names once for all (#1554)
1111bcd Xin He committed 8 months ago
Use AR_WORK_SPACE/offload/ as base dir for offload temp files (#1552)
803912a Yi Liu committed 8 months ago
fix wrong match of ignore_layers and use warning instead of error for mismatch (#1553)
d8ee85c Xin He committed 8 months ago
Fix CI test_transformers issue caused by transformers 5.3.0 (#1535)
7b087fb Liang Lv committed 8 months ago
support MTP module WOQ quantization and step3p5 model quantization (#1526)
42fdb06 Xin He committed 8 months ago
Refactor build process to use 'uv build' instead of 'python setup.py' (#1495)
5024611 Sun, Xuehao committed 8 months ago
Fix Inference tensors do not track version counter error (#1502)
e2b88a5 Wang, Mengni committed 8 months ago
release CUDA memory in WeightConverter and avoid meaningless print (#1498)
cd10eab Xin He committed 8 months ago
[BUG] update moe check logic and make .gate ignore general (#1517)
2eaa3b1 Xin He committed 8 months ago