LLM inference in C/C++
C++ ggml
Commits on master
kleidiai : dynamic chunck-based scheduling for hybrid execution (#23819)
3ecfb15 Charles Xu committed 5 months ago
sycl : port multi-column MMVQ from CUDA backend (#21845)
7fe2ae4 Mason Milburn committed 5 months ago
webui: [a11y] fix keyboard navigation issues in chat interface and sidebar (#23132)
42b2d60 viggy committed 5 months ago
Move duplicated imatrix code into single common imatrix-loader.cpp (#22445)
e7bcf1c Bartowski committed 5 months ago
ggml: vectorize ggml_vec_dot_q4_1_q8_1 with WASM SIMD128 (#22209)
4c51309 Kartik Sirohi committed 5 months ago
server: avoid unnecessary checkpoint restore when new tokens are present (#24110)
6f3a9f3 Yongyue Sun committed 5 months ago
webui: fix tool selector toggle/counter, key tools by stable identity (#24065)
4586479 Pascal committed 5 months ago
build : use umbrella Headers directory for XCFramework module map (#23974)
4d74287 Gerard Martinez committed 5 months ago
cmake: skip cvector-generator and export-lora when CPU backend is disabled (#24053)
7ac5a42 Andrea Richiardi committed 5 months ago
fix(mtmd): handle Gemma 4 audio projector embedding size (#24091)
e3ba22d Andrei committed 5 months ago
tests : refactor test-save-load-state to accept token input (#24073)
65ef50a Georgi Gerganov committed 5 months ago
metal : reduce rset heartbeat from 500ms -> 5ms (#24074)
3d19986 Georgi Gerganov committed 5 months ago
ggml-webgpu: FlashAttention refactor + standardize quantization support (#23834)
e8c5489 Reese Levine committed 5 months ago
ggml-cpu: extend RVV quantization vec dot to higher VLENs (#22754)
3c7450c rehan-10xengineer committed 5 months ago