PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Commits on main
Skip fused_moe replacement when llm-compressor oneshot already linearizes experts (#1966)
09d3dec Wang, Chang committed 4 months ago
Adapt the latest sglang integration test Engine kwargs. (#1964)
4924454 Wang, Chang committed 4 months ago
fix auto_scheme gguf format wrong bits if use tiny model (#1953)
b1a6baf Heng Guo committed 4 months ago
Implement shard parallelism configuration and update documentation (#1942)
7e2b69d Xin He committed 4 months ago
fix saving hadamard rotation model without rotation config. (#1933)
6a2e5ad lkk committed 4 months ago
Add flashinfer-jit-cache installation to cuda unit test setup (#1947)
349b041 Sun, Xuehao committed 5 months ago
Update PyTorch and torchvision versions to 2.12.1 and 0.27.1 in CI (#1943)
354c6a3 Sun, Xuehao committed 5 months ago
Fix SIGSEGV in pack-weight unpack/GEMM on malformed blob (#1928)
495e253 Luo Yu committed 5 months ago
Refine AutoScheme logic for gguf, reduce memory consumption and improve speed (#1916)
0c9c5b1 Wenhua Cheng committed 5 months ago
[Model Free] Add MXFP mix-precision quantization and support RoutedExperts (#1921)
be043f3 Xin He committed 5 months ago