LLM inference in C/C++
C++ ggml
Commits on master
hexagon: fix VTCM size query on images that need layout out-params (#30655)
b684b49 Tarek Dakhran committed 1 month ago
mtmd : lfm2 tiles and layout match transformers (#30261)
4a45de4 Yuri Khrustalev committed 1 month ago
model : add H2O-Lightning-4B decision model (#30361)
7e3eeee PSEUDOTENSOR / Jonathan McKinney committed 1 month ago
server : avoid repeated mmproj memory margin addition on sleep/resume (#28233)
5a9a34d TANGBUDU committed 1 month ago
server: Router wasn't propagating the client disconnects fully to the child, the response wait could cause a delayed cancellation (#29707)
4e20138 Me42019 committed 1 month ago
server: fix missing output_index in Responses API SSE events for compatibility (#30863)
1ba568c Renzis committed 1 month ago
mtmd : read LFM2 tiling params from GGUF metadata (#30840)
36140db Tarek Dakhran committed 1 month ago
model : drop the redundant precision overrides in the Kolibri-1 router (#30865)
9df0b73 Georgi Gerganov committed 1 month ago
opencl: add several gemm variants for small batches and some fixes (#30850)
a01ce9f lhez committed 1 month ago
ggml-cpu : select the NEON fp32->fp16 conversion (#30505)
119f68d Bernard Ladenthin committed 1 month ago
HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by 64 (#30836)
b684498 uvos committed 1 month ago
CUDA: Clear CUDA graphs after legacy memory pool flush (#26771)
13786fb Bálint Bokros committed 1 month ago
server : pass id_slot through the Anthropic /v1/messages conversion (#28554)
d806193 Peter van Velzen committed 1 month ago
metal : fix missing MoE kernel variant for 9 experts per token (#30610) (#30699)
704366d Caladrel committed 1 month ago
vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer regexes (#30845)
d8d9297 tkran2 committed 1 month ago
tests : make MoE router selection independent of the backend noise (#30818)
7879491 Georgi Gerganov committed 1 month ago
hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (#30854)
8466395 Max Krasnyansky committed 1 month ago
sycl: fix compiler warnings - missing prototypes, dead code, extra semicolons (#30271)
dd002b9 Michael Taylor committed 1 month ago
sycl : fix get_rows on reordered quantized weights (#30430)
3bb63cb Sam Murdoch committed 1 month ago