LLM inference in C/C++
C++ ggml
Commits on master
vulkan: allow multiple devices to access gpu-visible host memory (#29741)
e0bcacc Ruben Ortlam committed 1 month ago
cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327)
f851e24 Georgi Gerganov committed 1 month ago
cuda : fix compiler warnings in mmq-load-tiles.cuh (#30443)
bfe65e1 Adrien Gallouët committed 1 month ago
vulkan : split matmul dispatch when it would exceed maxComputeWorkGroupCount (#29533)
411d548 losewayy committed 1 month ago
spec : remove tokens assert from dsv4 graph_mtp (#30444)
296538a Oliver Simons committed 1 month ago
model : allow Qwen3.8 Flash Next to accept image embds for MTP (#30397)
7f51953 Oliver Simons committed 1 month ago
llama : skip logits buffer allocation when graph produces no logits (#30255)
ed019cd Animesh committed 1 month ago
graph : avoid reallocations in the "no cache" path (#30439)
9880157 Georgi Gerganov committed 1 month ago
models : remove redundant ggml_cont calls now supported by all unary kernels (#30198)
071b553 Muhammad Saad committed 1 month ago
hexagon: extend scalar DMA broadcasting support for binary ops (#30245)
4413a19 kurquhar committed 1 month ago
ci : build OpenVINO releases with GGML_NATIVE=OFF and CPU backend variants (#30324)
8f574b0 Pratyush Kumar committed 1 month ago
graph: pad the KV dimension of the no-cache attention to 256 (#30416)
358a420 Pascal committed 1 month ago
metal: fix metal tensor kernel_mul_mm error for large tensor (#28748)
48e9857 SecondSeed committed 1 month ago
hexagon: optimize CPY/CONCAT — round 3, short-rows edition (#30341)
8bb3cf0 Max Krasnyansky committed 1 month ago
vulkan: pack FMA for f16 accumulation, no dot product op (#29877)
0f701c8 Alex Morgan committed 1 month ago
tests : link each test against only the libraries it uses (#30299)
ccdbe29 Georgi Gerganov committed 1 month ago
examples : fix draft sampling and rejection handling (#30095)
77757a3 Daniel Bevenius committed 1 month ago
llama : report compute graph reallocations during inference (#30329)
972b5fc Georgi Gerganov committed 1 month ago