LLM inference in C/C++
C++ ggml
Commits on master
webgpu: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor (#29471)
f00a64c jbooth committed 1 month ago
tests : refactor test-recurrent-state-rollback (#29426)
d77dd08 Georgi Gerganov committed 1 month ago
ggml-cpu: enable tiled flash attention for non-vector-multiple head dims on x86 (#29423)
6f767fe SXX committed 1 month ago
vulkan: fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#29520)
03a667a François-Xavier Gsell committed 1 month ago
metal: support left and circular padding in GGML_OP_PAD (#29561)
4364bf7 Pascal committed 1 month ago
context : do not re-reserve the scheduler when toggling causal_attn (#28751)
ed7ac35 Sihan Yu committed 1 month ago
vulkan: fix wrong results when a mul_mat reads a slice of a larger cache (#28956)
5262471 Pascal committed 1 month ago
server : allow RANK pooling batch splitting for causal LLM rerankers (ie. Qwen3 and Qwen3-VL) (#28876)
4da6337 Tim Wang committed 1 month ago
common : avoid side effects around params parsing (#29537)
a97cce8 Georgi Gerganov committed 1 month ago
common : make string_split<T> throw on invalid values (#29518)
136887b Adrien Gallouët committed 1 month ago
convert : export YaRN scaling parameters for PLaMo-3 (#29528)
9adc7f4 Toki Nasin committed 1 month ago
CUDA: tune fp16 tile FlashAttention configs for head sizes 40-112 (#26289)
36d7b08 Animesh committed 1 month ago
HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes (#28907)
2ebd9ae uvos committed 1 month ago
common : throw instead of abort on grammar without llguidance (#29516)
da6c28e Adrien Gallouët committed 1 month ago
ci : enable GGML_SCHED_DEBUG_REALLOC=1 for ctest workflows (#29514)
7fb2b08 Georgi Gerganov committed 1 month ago
hrm : fix layer placement of `z_l_init` weight (#29512)
85ca3b5 Georgi Gerganov committed 1 month ago