LLM inference in C/C++
C++ ggml
Commits on master
graph: Fix granite speech model inference by applying embedding scale when deepstack is not used (#24357)
d73cd07 Aarnav Pai committed 5 months ago
vulkan: add `v_dot2_f32_f16` support in matrix-matrix multiplication and Flash Attention (#24123)
b4e3dc6 Ruben Ortlam committed 5 months ago
ui: fix mobile chat form overflow and bust stale bundle cache (#24158)
efbacf8 Pascal committed 5 months ago
models : fix plamo2 attention_key/value_length regression (#24317)
f0152ef Sigbjørn Skjæret committed 5 months ago
ggml-cpu : fix rms_norm_back wrong output under in-place aliasing (#24305)
fd3271e Yash Raj Pandey committed 5 months ago
ggml-webgpu: Improve prefill speeds for k-quants + refactor matmul for Q4/Q5/Q8 and k-quants (#24225)
1e1aca0 Masashi Yoshimura committed 5 months ago
mtp: support for gemma-4 E2B and E4B assistants (#24282)
7d2b45b Max Krasnyansky committed 5 months ago
server : do not parse when flushing http headers (#24281)
42a0afd Aldehir Rojas committed 5 months ago
[ggml-webgpu] Handle buffer overlap / buffer aliasing for concat operator (#24000)
1705d43 Nikhil Jain committed 5 months ago
[ggml-webgpu] Implement 2D workgroups for scale, binary, and unary ops (#24044)
3b3da01 Nikhil Jain committed 5 months ago
docker: install ffmpeg in the released image (#24302)
3ebe862 Xuan-Son Nguyen committed 5 months ago
cli: fix spinner not show during prompt processing (#24283)
715b86a Xuan-Son Nguyen committed 5 months ago
vulkan: Use cm2 decode_vector for mul_mat_id B matrix loads (#23991)
c74759a Jeff Bolz committed 5 months ago
cuda: reset cuda context after reading memory size (#23935)
0f7fada Ruben Ortlam committed 5 months ago