LLM inference in C/C++
C++ ggml
Commits on master
vulkan: add DeepSeek-V4 hyper-connection fused ops (DSV4_HC_COMB/PRE/POST) (#26578)
7a333e7 Kevin Hopper committed 2 months ago
CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546)
0c96345 ravel7524 committed 2 months ago
convert : refactor Hy4-preview conversion - move HC tensor mapping to the global map (#28451)
d23c47f fairydreaming committed 2 months ago
CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (#26705)
73ab759 Pranesh Gonegandla committed 2 months ago
examples : print ggml_version and ggml_commit in test-cmake [no ci] (#28538)
1173700 Daniel Bevenius committed 2 months ago
caps : recheck typed content if template checks for string (#28511)
5202104 Sigbjørn Skjæret committed 2 months ago
convert : write explicit recurrent_layers for Qwen3-Next / Qwen3.5 (#28208)
9a75705 DevVexus committed 2 months ago
ggml-cuda: fix divergent barrier in f16 flash attention (#27870)
b74f590 Siavash Norouzi committed 2 months ago
ggml: allow backend inputs to not create another split (#28387)
992cb50 Aman Gupta committed 2 months ago
ci : add container image checking and tagging (wip) (#28394)
2092353 Daniel Bevenius committed 2 months ago
vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765)
8fe90e1 Anjielon committed 2 months ago
convert : add `--fuse-qkv` flag to fuse Q/K/V into QKV during HF-to-GGUF conversion (#22780)
465e49b PikaPikachu committed 2 months ago
models : fix GDN normalization from `max` to `rsqrt` (#28068)
5fdfa62 Daniel Han committed 2 months ago
[Model] Support for Spark2_5ForCausalLM implementation (#27868)
3ad1ba7 KnightYao committed 2 months ago
opencl: properly choose weights pack for q4_K, q5_K mul_mat (#28402)
d03efa5 lhez committed 2 months ago
ui: Improve Chat Messages rendering performance (#28460)
0afb805 Aleksander Grygier committed 2 months ago
metal : add remaining fa-vec tunings for M2 Max (#28458)
971595d Niklas Wenzel committed 2 months ago
sycl : fix test-backend-ops CI break && restore Kronecker product FWHT support (#28016) (#28254)
4d91760 Jingxin (Philip) Li committed 2 months ago
sycl: attribute device allocations by site (GGML_SYCL_MEMTRACE) (#27631)
cd8cdf3 Nick Farrell committed 2 months ago