LLM inference in C/C++
C++ ggml
Commits on master
llama: add default load-mode auto, which avoids mmap on iGPUs (#26081)
153d324 Ruben Ortlam committed 3 months ago
tests : clean-up server test, use `tests.sh` in ci (#26886)
b3df572 Georgi Gerganov committed 3 months ago
common/peg : suppress incomplete escape sequences (#26780)
48d22e2 Aldehir Rojas committed 3 months ago
ggml-webgpu: fix CI errors from #25025 and #25262 (#26566)
84f7129 Masashi Yoshimura committed 3 months ago
opencl: transpose the K tile in local memory for FA prefill kernels (#26428)
689e227 Hongqiang Wang committed 3 months ago
ci : target ROCm 7.14 for build and release (#25775)
0666ad2 Mario Limonciello committed 3 months ago
ggml-cpu : fix CPU affinity mask being ignored on Android (#26838)
d2f8305 Hitesh Chopra committed 3 months ago
ggml : require contiguous src for ROLL on CUDA and Metal (#25928)
f8def7f Yash Raj Pandey committed 3 months ago
ci : don't specify python version in server-sanitize for broader runner compatibility (#26840)
e5275f6 Sigbjørn Skjæret committed 3 months ago
server: add more tool isolation support (ssh remote + podman rootless) (#26774)
4ae84de Pascal committed 3 months ago
chat : Align Laguna-S-2.1 chat template to huggingface (#26232)
a52077c Guido Imperiale committed 3 months ago
ci: Add support for CUDA 13.4 ARM64 builds for Windows (#26650)
2e2d99c shivamkumard-ctrl committed 3 months ago
vendor : update cpp-httplib to 0.53.0 (#26821)
e23e944 Alessandro de Oliveira Faria (A.K.A.CABELO) committed 3 months ago
server: gate the docker tools runtime tests on a real container run (#26826)
1e396e7 Pascal committed 3 months ago
model-saver : fix expert shared/chunk FFN length key clobber (#26693)
0377426 Caleb DeLeeuw committed 3 months ago
ggml-webgpu : refactor several wgsl files and simplify flash_attn wgsl. (#26134)
f401bb1 Masashi Yoshimura committed 3 months ago