LLM inference in C/C++
C++ ggml
Commits on master
opencl: read/write MoE dp4a activation tiles to local memory as 128-bit (vectorized LD/ST perf opt) for Adreno GPUs (#25810)
6bdd77f Hongqiang Wang committed 4 months ago
opencl: transpose q4_K noshuffle scales for coalesced reads (#25805)
86d86ed Hongqiang Wang committed 4 months ago
tests : initialize all tensors in test_dsv4_hc to avoid NaNs in sentinel tensors (#25822)
5d5306b fairydreaming committed 4 months ago
common : auto-download dflash- and eagle3- HF sidecars (#25811)
635cdd5 Georgi Gerganov committed 4 months ago
ggml-blas: default hadamard mul_mat to cpu routine (#25710)
11fd0a6 Aaron Teo committed 4 months ago
sycl: fix row calculation when K_QUANTS_PER_ITERATION is 1 (#25690)
0bd0ec6 Todd Malsbary committed 4 months ago
opencl: loads quants as uint for q4_K and q5_K flat mv (optimization for Adreno A7x GPUs) (#25780)
e8f19cc Hongqiang Wang committed 4 months ago
docs: added a note about using OpenCl with Adreno 810 (#25786)
ac2557c akleine committed 4 months ago
hexagon: L2 cache handling rework (dirty bit tracking with lazy flushing) and more MUL_MAT updates (#25762)
b2dd28a Max Krasnyansky committed 4 months ago
kleidiai: Add SME vs SME2 distinction in kernel dispatch (#25478)
f15bd60 Rajendra Matcha committed 4 months ago
vulkan: when using transfer queue for async copies, sync on event_wait to avoid race (#25229)
b15ca93 Ruben Ortlam committed 4 months ago
conversion: accept BitNetForCausalLM architecture name (#25769)
3278e92 Khashayar Ghafouri committed 4 months ago
vendor: update BoringSSL to 0.20260713.0 (#25624)
86b719b Alessandro de Oliveira Faria (A.K.A.CABELO) committed 4 months ago
tests: actually exercise `test-recurrent-state-rollback` (#25758)
32e789f Aman Gupta committed 4 months ago
server : allow text-only slot save/restore with mtmd (#25076)
a8dc0e3 Chipmunk committed 4 months ago
convert : fix dflash target tokenizer mismatch during conversion (#25733)
a55a8c5 Ruixiang Wang committed 4 months ago
server: Ignore empty / non-existing `Origin` headers (#25756)
8ee54c8 Sebastian Dröge committed 4 months ago
ggml-cuda : restore prop.integrated on HIP builds (#24233)
c7d8722 liminfei-amd committed 4 months ago
CUDA: dedup MoE gate/up activation quantization (#25441)
5839ba3 Pranesh Gonegandla committed 4 months ago
ci : add official website link to release notes (#25728)
a320cbf Georgi Gerganov committed 4 months ago
quant : allow using manual tensor types with --pure (#25716)
56d6e9d Georgi Gerganov committed 4 months ago
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU (#25745)
3dafb58 Hongqiang Wang committed 4 months ago