LLM inference in C/C++
C++ ggml
Commits on master
sycl: wide stores for the q4_K/q5_K weight dequant (#29696)
13de11e Titaniumtown committed 1 month ago
tests : catch vanished rows in the fusion baseline (#30618)
88aaf32 Georgi Gerganov committed 1 month ago
graph : add precision handling for MELLUM architecture (#30584)
5e03a1f Mikhail Podvitskii committed 1 month ago
hexagon: tensor repacking rework and support for tiled and flat quantized and fp weights (#30635)
b797b9c Max Krasnyansky committed 1 month ago
llama : share the k-pool input views between layers (#30613)
6afa9a3 Aman Gupta committed 1 month ago
mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629)
2851b3b Xuan-Son Nguyen committed 1 month ago
convert : fix Granite Switch block_count for MultiSwitch checkpoints (#30621)
59d5fad Nigel Jones committed 1 month ago
convert : fix regression in mapping of Granite speech tensors (#30628)
e287bd8 Matt Rutkowski committed 1 month ago
mtmd: lightonocr does not require longest edge to always be 1540, native resolution is the default (#30600)
2f729cf Said Taghadouini committed 1 month ago
pi : expand AI disclosure with hardware and stats (#30615)
b76b133 Georgi Gerganov committed 1 month ago
models : add Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support (#30520)
cd0a5b6 Kartik Gulia committed 1 month ago
scripts : add ci-status.sh to list active CI runs (#30586)
a11b653 Georgi Gerganov committed 1 month ago
model : Mellum - read MoE gating params from GGUF, unify RoPE (#30578)
a6b91d8 Mikhail Podvitskii committed 1 month ago
server: prefer empty slots and keep states overwritten by child tasks (#30567)
b0d2391 Pascal committed 1 month ago
fix(ui): take the agentic turns default from DEFAULT_AGENTIC_CONFIG (#30582)
fd55cb9 Aleksander Grygier committed 1 month ago
jinja : improve context copy constructor performance (#30562)
08698c8 William Tan committed 1 month ago
jinja : write nan and inf in tojson as json.dumps does (#30483)
82e1e2f Yuri Khrustalev committed 1 month ago