LLM inference in C/C++
C++ ggml
Commits on master
opencl: add bin kernel `kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin` (#29056)
d5f6649 shaofeiqi committed 1 month ago
server: fix router eviction races with the existing queue (#29217)
9919911 Pascal committed 1 month ago
server: support input_image in function_call_output (#20663) (#22575)
4098fdc Empressia committed 1 month ago
vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) (#24406)
4ceb171 Jiang, Fish committed 1 month ago
metal : gate mul_mm_id src1 rescale behind ggml_prec (#29029)
0f8a414 Michael de Gans committed 1 month ago
Performance tune for gemma4-26b-a4b flash attention shape. (#28450)
c350a40 David M. Rogers committed 1 month ago
jinja: use const for statement::execute and ::visit (#29271)
348f853 Xuan-Son Nguyen committed 1 month ago
server: Add support for binding to multiple addresses (#28690)
217f81c Emanuil Rusev committed 1 month ago
chat : Fix Muse Glimmer tool-call first parser error (#29242)
7ab4ee7 Nicolas Mowen committed 1 month ago
ci : publish snapdragon builds in release workflow (#29007)
0ee9435 Yuri Khrustalev committed 1 month ago
ci : update Level Zero SDK to v1.33.1 and enable the L0/oneDNN CMake flags in the SYCL job (#29230)
c550d2f Asahi-Prv committed 1 month ago
metal : fix mask bounds in flash attention block pre-pass (#29220)
fb34fc2 Foad Abo Dahood committed 1 month ago
test-save-load-state : compare logits with NMSE and feed expected tokens (#29238)
c641dfa Georgi Gerganov committed 1 month ago
llama-context : report graph inputs and input tensors during sched reserve (#26625)
9655061 Georgi Gerganov committed 1 month ago
ggml-cuda : convert contiguous tensors four elements at a time (#29155)
f4e276a Piotr Wilkin (ilintar) committed 1 month ago
ggml : fix dimension and stride truncation in ggml_permute (#29227)
c21284c leejet committed 1 month ago