LLM inference in C/C++
C++ ggml
Commits on master
vulkan: disable large matmul tile on Samsung GPUs with 32KB shared memory (#28531)
9e258a6 Jiwoong Song committed 1 month ago
sycl: large register file for D=512 FA vec kernels (#29062)
b933289 Titaniumtown committed 1 month ago
sycl : do not use slow oneDNN reference matmul and fattn (#28985)
c328acc Łukasz Ślusarczyk committed 1 month ago
ggml : add `alloc_buffer_n` to buffer type interface (#23671)
631109b Georgi Gerganov committed 1 month ago
pyproject : add linux platform marker to uv torch source (#29177)
207bdab Kasimir Tanner committed 1 month ago
llama: refer to segment documentation [no ci] (#29074)
78e2964 Johannes Gäßler committed 1 month ago
common,rpc : fix cache dir creation through symlinks on buggy libstdc++ (#29816)
f1cee99 Adrien Gallouët committed 1 month ago
skill: note about model-specific CLI arguments + testings (#29808)
68e79bd Xuan-Son Nguyen committed 1 month ago
ci: fix Fusion / metal by updating the qwen4exp baseline (#29812)
e358d59 Pascal committed 1 month ago
llama : clamp kpool re-pool bound to existing pools (#29805)
81e39ad Georgi Gerganov committed 1 month ago
hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA (#29685)
dcd387a Yiwei Shao committed 1 month ago
server: return HTTP 400 for invalid embedding requests (#29060)
d775ebf Sam Malayek committed 1 month ago
convert : write Gemma embedding scale for DFlash drafts (#29802)
2b36825 Yu Chengye committed 1 month ago
metal : release temporary private transfer buffers (#29777)
13b4d71 Mike van Lammeren committed 1 month ago
webgpu: add bfloat16 support for MUL_MAT/MUL_MAT_ID/GET_ROWS- #29358 (#29358)
4b1622a Masashi Yoshimura committed 1 month ago
llama : fix invalid assert in recurrent memory (#29799)
869034b Georgi Gerganov committed 1 month ago
CUDA: Make CCCL configurable + pin it to 3.4.3 for CI jobs (#29792)
7677678 Oliver Simons committed 1 month ago
mtmd: cap max_image to ubatch for non_causal models (#29773)
552f18f Xuan-Son Nguyen committed 1 month ago
meta: clear inactive AllReduce shards with FILL, not SCALE (#29793)
5503b04 Georgi Gerganov committed 1 month ago