LLM inference in C/C++
C++ ggml
Commits on master
llama-bench: respect -fitc if bigger than required benchmark size (#28331)
f39148a Martin Emrich committed 1 month ago
CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807)
50e3e3e Anav Prasad committed 1 month ago
llama: keep the backend sampling graph static across ubatches (#30223)
a518119 Pascal committed 1 month ago
ggml: fix OOB write in ggml_acc with negative offset (#30135)
8ae3867 Jasmine-tim committed 1 month ago
webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219)
ba6439a Masashi Yoshimura committed 1 month ago
convert : support compressed-tensor mixed-precision NVFP4 checkpoint (#28636)
609290b ynankani committed 1 month ago
ci: fix Models Backend webgpu by supporting GGML_OP_DUP (#30216)
e94acad Pascal committed 1 month ago
ci: fix HIP quality check by ignoring the 320/256 FA kernel spill (#30212)
2c564f4 Pascal committed 1 month ago
metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209)
1f8fa52 Pascal committed 1 month ago
CUDA: pass src1 precision to host MMQ config helpers (#30168)
8a1a9b5 ynankani committed 1 month ago
hexagon: fix IM2COL patch-embed DMA ring overflow (#30189)
d4d82d6 Mark Todorovich committed 1 month ago
CUDA : looped PAD kernel for more than 65535 rows or slices (#30147)
c35b667 Prabhsimran Singh committed 1 month ago
CUDA: fix CCCL version guard breaking on major version rollover (#29453)
1167d3f hey-gm committed 1 month ago
ui: apply ui_settings on first visit in router mode (#29668)
4f92965 Simon Sudarushkin committed 1 month ago
server : preserve context checkpoints across slot save/restore (#26004)
033df86 Leebr Data Consulting committed 1 month ago