LLM inference in C/C++
C++ ggml
Commits on master
vulkan: Introduce driver version check for Windows Intel GPU to mitigate crashing (#25192)
eb41d50 Masato Nakasaka committed 3 months ago
llama : load MTP tensors only if they are really used (#26296)
82dbc4f fairydreaming committed 3 months ago
server: correct accepted tokens when need draft token replay (#26320)
0005475 Ruixiang Wang committed 3 months ago
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… (#25025)
9d9a6d2 Ozymandias_EBON committed 3 months ago
llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized (#25871)
69e62fc fairydreaming committed 3 months ago
ggml-zendnn : group matmul direct API for mul_mat_id (#25918)
1e22599 Sachin Sharma committed 3 months ago
sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD (#26234)
1c5b89f Neo Zhang committed 3 months ago
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts (#25956)
8f4646a Masashi Yoshimura committed 3 months ago
mtmd: add lanczos resize method [no release] (#26341)
5f55650 Xuan-Son Nguyen committed 3 months ago
server: support inp embd to generate next token (#26313)
b4ca032 Xuan-Son Nguyen committed 3 months ago
llama-context : sync pending async copies before clearing embd_seq (#25676)
432d7ff o7si committed 3 months ago
tests : avoid building get-model.cpp many times (#26317)
47f686f Georgi Gerganov committed 3 months ago
ggml-cuda: Allow transpose-free gemmv computation (#26171)
e1a1abb Robert Esclapez committed 3 months ago
Remove custom cpu op from the M3 graph, express with stock ops (#26297)
b2f2216 timkhronos committed 3 months ago
metal: fix memory unwire if model is freed without any GPU operations (#26082)
d0bfb19 Niklas Wenzel committed 3 months ago
ui: IndexedDB and Conversations data fixes (#26278)
21a5f5b Aleksander Grygier committed 3 months ago
ggml : Fix issue with kleidiai ci and stringop overflow warning (#26277)
32703b4 Jonathan Clohessy committed 3 months ago