LLM inference in C/C++
C++ ggml
Commits on master
conversion : fix Nemotron 3.5 Lightning layers (#27729)
dac869b Daniel Bevenius committed 2 months ago
ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (#26647)
11cd988 Dominik Pantaleoni committed 2 months ago
ggml-meta: propagate buffer usage and call init on the new tensors (#27586)
5d5cb4c Max Krasnyansky committed 2 months ago
kleidiai: Rework KleidiAI Build System/Integration (#26077)
d222767 Jonathan Clohessy committed 2 months ago
ci : update OS used for ROCM to Ubuntu 24.04 (#27681)
eab8ee4 Mario Limonciello committed 2 months ago
server : reject prefilled assistant messages with tool calls (#27626)
1729ed5 Kyozzz committed 2 months ago
ci : store ccache on HF buckets (test with cuda-ubuntu for now) (#27699)
790b571 Sigbjørn Skjæret committed 2 months ago
metal : null-check buffer alloc to fix OOM crash (#25371)
3737e41 Yuri Khrustalev committed 2 months ago
grammar : parse \- in char classes as literal hyphen (#27591)
eb25b72 Saad Ali committed 2 months ago
webgpu : fix handling of infinity values during ARGSORT and TOP_K (#27538)
5ea87dd fairydreaming committed 2 months ago
metal : per-device tuned (Q, NE) for flash-attn vec (#26570)
f280b26 YiChen Lv committed 2 months ago
misc : read repetition_penalty from generation_config.json (#27659)
b3c3b96 Tarek Dakhran committed 2 months ago
tests : disable DOTS3NOTE arch test for WebGPU (#27654)
7584430 fairydreaming committed 2 months ago
ggml : shorten virtual device naming in CUDA and Metal (#27608)
a14dba6 Georgi Gerganov committed 2 months ago
webgpu : reorder includes since V that appears in common_decls.tmpl may be defined as K in flash_attn_decls.tmpl if KV_OVERLAP (#27545)
c1c766d fairydreaming committed 2 months ago
mtmd: video: fix moov atom at the end of file (#27596)
160c6b0 Xuan-Son Nguyen committed 2 months ago
ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602)
985b149 Georgi Gerganov committed 2 months ago