LLM inference in C/C++
C++ ggml
Commits on master
sycl : replace oneAPI package by DLE+oneDNN package (#30131)
e36df60 Neo Zhang committed 1 month ago
opencl: fix OOB tail access for adreno noshuffle gemv kernels (#30421)
9166cea lhez committed 1 month ago
unicode : reject overlong UTF-8 and out-of-range codepoints (#30485)
2ce20d2 Simone Iacomino committed 1 month ago
llama : split llama_prefetch() into merge and prefetch steps (#30532)
68ac765 Adrien Gallouët committed 1 month ago
vulkan : copy strided f16 KV once on AMD so reads spread across memory channels (#27703)
733f57a Nathan Wilson committed 1 month ago
cpu: add F16 SWIGLU_OAI reference implementation and validation (#29200)
5a638ae cqderek committed 1 month ago
metal : add BF16 variant of the tensor API flash attention kernel (#30491)
6e66cb5 Ethan Guo committed 1 month ago
server: save only restorable slot states to the prompt cache (#30498)
2639a3b Pascal committed 1 month ago
vulkan: Use L2-friendly workgroup scheduling for cm2 mulmat (#30215)
3e390af Ruben Ortlam committed 1 month ago
ggml-cpu : keep the FP16->FP32 lookup table on x86 with F16C (#30342) (#30343)
685f708 Leebr Data Consulting committed 1 month ago
ggml-cpu : size set_rows conversion buffer for all threads (#30454)
5cdadfc Aman Gupta committed 1 month ago
skill: Update `code-review` skill for UI development (#30509)
37d6d65 Aleksander Grygier committed 1 month ago
docs : clarify installation recommendation for Intel oneAPI Toolkit (#30407)
318a471 Francisco Herrera committed 1 month ago
ggml : add note about mul_mat_id + disable mul_mat_id_dup tests (#30501)
8c2134a Georgi Gerganov committed 1 month ago
model: split layers evenly when no device reports free memory (#29677)
df18bee Pascal committed 1 month ago