LLM inference in C/C++
C++ ggml
Commits on master
vulkan : fix TOP_K for +inf/NaN inputs and k = 1 on negative values (#30107)
ff5888f gianni-cor committed 1 month ago
cuda : support arbitrary striding for unary ops on f16, f32, and bf16 (#29781)
08246a2 Muhammad Saad committed 1 month ago
hex-mmadd: do not assume aligned read/write when bias-add is fused (#30133)
097f5b5 Max Krasnyansky committed 1 month ago
cuda : Use byte strides for roll to allow non-contiguous ROLL operations (#29547)
d888016 Bertay Eren committed 1 month ago
CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits (#28175)
fda1866 Eden committed 1 month ago
sycl: fuse the delta-net alpha gate (add + unary + mul) (#29687)
ff30363 Titaniumtown committed 1 month ago
sycl: stage bulk uploads (model loading) through a pinned ring buffer (#29608)
000bee5 cwriter committed 1 month ago
sycl: remove duplicate block-size defines from op headers (#29507)
9b4ed0c Titaniumtown committed 1 month ago
model : add LiquidAI/d1-omni-600M decision model (#30114)
a657f7e Tarek Dakhran committed 1 month ago
llama : add a GPU cache for MoE experts kept in host memory (#29887)
d6cf9ac Aman Gupta committed 1 month ago
server : accumulate generated text and tokens as parse input (#29876)
18b5f8b Aldehir Rojas committed 1 month ago
metal : few-row MMA mat-mul for the remaining src0 types (#30065)
9881906 pratiknarola-t committed 1 month ago