LLM inference in C/C++
C++ ggml
Commits on master
gguf : harden loader against malformed tensor dims and metadata types (#25596)
d8a8bea HarrisonSec committed 3 months ago
kleidiai: Add runtime feature detection mechanism for aarch64/kleidiai (#26076)
132753b Jonathan Clohessy committed 3 months ago
model : disallow integer dflash sliding_window_pattern (#26900)
ece98b8 Sigbjørn Skjæret committed 3 months ago
server : support slot save/restore with media inputs (#26640)
5d9e5ac Chipmunk committed 3 months ago
opencl: default FA c8 cluster width to 16 on X1E (#26433)
89e0aa6 Hongqiang Wang committed 3 months ago
wavtokenizer-dec : bound posnet/convnext block_count against n_layer_all (#26892)
55f453b Oğuzhan Akkaya committed 3 months ago
convert : handle per_layer_config in Gemma4 (transformers 5.15) (#26882)
6eff593 Wang Zhiyu committed 3 months ago
opencl: use flat mv q5_k when weight exceeds image1d_buffer_t limit (#26880)
cb27fe9 lhez committed 3 months ago
chat : fix muse-glimmer detection of tool calls after EOM (#26879)
0b1bad1 ruanslv committed 3 months ago
CUDA: only disable CUDA graphs when mul_mat_id actually needs a stream sync (#26802)
ebb546b Rafail Giavrimis committed 3 months ago
cuda : add warp-per-row wkv7 kernel for single-token decode (#26111)
5988633 0 committed 3 months ago
chat : tighten bare function parsing for Qwen models (#26793)
ba360ef Aldehir Rojas committed 3 months ago
imatrix.cpp: Move finite check and only check touched experts (#26861)
38406d5 Bartowski committed 3 months ago
requirements: use stable torch packages on s390x (#26864)
2468576 Niklas Wenzel committed 3 months ago
convert : keep quantization scales for nemotron --mtp export (#26903)
5d16e81 ynankani committed 3 months ago
model-conversion : use save_output_data for causual embeddings [no ci] (#26890)
1138b85 Daniel Bevenius committed 3 months ago