LLM inference in C/C++
C++ ggml
Commits on master
ui : key draft sidecar badges by model instead of repo (#30919)
a6b83d4 Aleksander Grygier committed 28 days ago
jinja : fix namespace attributes shadowed by dict methods (#30941)
99fb997 Tarek Dakhran committed 29 days ago
server : forward llama.cpp sampling params on /v1/messages (#30856)
261700b David Garcia committed 29 days ago
vendor: update BoringSSL to 0.20261020.0 (#30889)
f849095 Alessandro de Oliveira Faria (A.K.A.CABELO) committed 29 days ago
llama : place d2t by its set_rows use so it can stay on the GPU (#30912)
eb182d8 Aman Gupta committed 29 days ago
metal : fix memory leaks and cleanup logic in GPU capture path (#30890)
ea9e344 Niklas Wenzel committed 29 days ago
hexagon: neon-vectorized and threaded repack (tiling) (#30901)
0efe8e4 Max Krasnyansky committed 29 days ago
opencl: add dp4a GEMM variants to vector load activation (#30753)
b531e4c lhez committed 29 days ago
sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / D=2048/4096) (#30286)
74b115c Neo Zhang committed 29 days ago
sycl: collapse duplicate need_check launch bodies in mmq (#30739)
96caa80 Titaniumtown committed 29 days ago
ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 table (#30737)
479a527 bosh committed 29 days ago
llama : scale moe down projection per expert token (#30861)
8baabd5 Georgi Gerganov committed 29 days ago
opencl: fix rope program build error with newer Clang (#30893)
339f324 Ben Ashbaugh committed 29 days ago
server : fix usage.completion_tokens undercount with n > 1 (#29486)
9a3b91c Unbug Lee committed 1 month ago
server : add POST /tts endpoint for TTS models (#26603)
5c0b49a Xuan-Son Nguyen committed 1 month ago
refactor : move LFM2 shortconv and FFN blocks into a shared LFM base (#30874)
84e66d0 Tarek Dakhran committed 1 month ago
server: Support logprobs with speculative decoding (#27196)
ead9f32 Nigel Bosch committed 1 month ago