LLM inference in C/C++
C++ ggml
Commits on master
metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (#30100)
7e8324f Niko Maroulis committed 1 month ago
qwen3tts : guard speaker_encoder_config patch for CustomVoice variant (#29179)
48499d2 SIDDARTHA REDDY committed 1 month ago
sampling : use greedy selection for eligible temperature-zero chains (#29797)
d0b490f Hrishith Thadicherla committed 1 month ago
vendor : update cpp-httplib to 0.60.0 (#30081)
b7dafa0 Alessandro de Oliveira Faria (A.K.A.CABELO) committed 1 month ago
ggml-webgpu: fix flash_attn supports_op check for overlapping KV (#28205)
36a7391 Masashi Yoshimura committed 1 month ago
tests : retain the anchor when testing recurrent rollback (#29923)
fa3c2fa Evan Huus committed 1 month ago
[SYCL] fix the issue in mixed different model GPUs in FA (#29071)
005a1e1 Neo Zhang committed 1 month ago
sycl: accelerate GLM MLA prefill with MKL flash attention (#29171)
d2a79e6 Anant Shrivastava committed 1 month ago
hexagon: CPY/CONCAT/CONT/DUP overhaul to use DMA/HVX for all cases (#30067)
c479922 Max Krasnyansky committed 1 month ago
ggml-cuda: use per-thread stream for buffer-init padding memset (#28782)
51ce9c1 Harkirat Gill committed 1 month ago
vocab : implement PLaMo-3 tokenizer pre-segmentation (#30045)
abeada3 Toki Nasin committed 1 month ago
llama: remove the gather path of the glm5-next sparse attention (#30042)
3109914 Pascal committed 1 month ago
model: support embeddinggemma2 (text+vision+audio) (#30054)
4fbc76d Xuan-Son Nguyen committed 1 month ago
metal : fix excess threadgroup memory in quantized flash attention (#29340)
a043d38 Foad Abo Dahood committed 1 month ago