LLM inference in C/C++
C++ ggml
Commits on master
jinja: treat a null left operand of in as a plain lookup (#28620)
b31b71f Pascal committed 2 months ago
vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471)
1945e09 Jiang, Fish committed 2 months ago
tests : use less threads for data initialization (#28325)
30b6a75 Georgi Gerganov committed 2 months ago
llama: disable lazy tensor loading by default on iGPUs (#28326)
f3f1a8f Ruben Ortlam committed 2 months ago
Revert "ggml-cuda : restore prop.integrated on HIP builds (#24233)" (#28604)
d4389a4 uvos committed 2 months ago
server : apply checkpoint min-step eviction only when the checkpoint list is full (#28302)
5d806aa Foad Abo Dahood committed 2 months ago
metal : fix idle threads in mul_mv_iq3_xxs for ne00 < 1024 (#28086)
88ada91 Foad Abo Dahood committed 2 months ago
spec: single device drafter should create meta backend wrapper (#28390)
415e909 Aman Gupta committed 2 months ago
ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603)
1744c6b Daniel Bevenius committed 2 months ago
vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL (#27220)
64e9bce Ankit Khandelwal committed 2 months ago
chat : split specialized parsers into common/parsers (#27764)
895c045 Piotr Wilkin (ilintar) committed 2 months ago
opencl: properly handle non-contiguous inputs to conv2d (#28503)
7d701b5 lhez committed 2 months ago
Revert "CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546)" (#28551)
e71b805 Piotr Wilkin (ilintar) committed 2 months ago
server: fix LRU hang on multiple requests same model (#28539)
160bd03 Xuan-Son Nguyen committed 2 months ago