LLM inference in C/C++
C++ ggml
Commits on master
CUDA/HIP: Flash Attention tuning (gfx1201) (#28102)
16378d9 Piotr Wilkin (ilintar) committed 2 months ago
vulkan: fix data race and OOB access in argsort(large) (#28705)
481c65f Jeff Bolz committed 2 months ago
vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618)
28ff095 Jeff Bolz committed 2 months ago
vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422)
50182a5 Jeff Bolz committed 2 months ago
ci : add self-hosted-gpu-cuda and server-sanitize to hf-jobs (#28693)
52d4268 Sigbjørn Skjæret committed 2 months ago
ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687)
18c17b4 shivamkumard-ctrl committed 2 months ago
spec: fix failed to decode mtmd chunk with DFlash (#28587)
fa67698 Jesus Gulfo committed 2 months ago
scripts : use sed instead of grep for version parsing [no ci] (#28700)
41fc758 Daniel Bevenius committed 2 months ago
models: clean up some dead switch branches in old models (#28669)
d344123 Iggy Jackson committed 2 months ago
tests : drop SYCL special-casing in test-backend-ops.cpp (#28688)
e5a8d43 Georgi Gerganov committed 2 months ago
vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341)
3ff67eb Julian Pscheid committed 2 months ago
memory : avoid allocating V cache for indexer (it's not used) (#28330)
311d421 fairydreaming committed 2 months ago
vulkan : add command-buffer debug labels for GPU profilers (#28101)
72797e8 Cordell Blanchard committed 2 months ago
ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606)
f1b6fbf Aaron Teo committed 2 months ago
vulkan: use spec constant for matrix matrix multiplication A-type (#25773)
91f6a6c Ruben Ortlam committed 2 months ago
vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592)
22397c3 Masato Nakasaka committed 2 months ago
llama : use int32_t for llama_sampler_chain_n return type (#28631)
4850c77 linsen458-spec committed 2 months ago
CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552)
d4abd57 Piotr Wilkin (ilintar) committed 2 months ago
CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079)
5a4d0fe Piotr Wilkin (ilintar) committed 2 months ago