LLM inference in C/C++
C++ ggml
Commits on master
jinja : skip copying loop scope unless a loop filter needs it (#29776)
def4d40 a u s t i n committed 1 month ago
llama-mmap : avoid a second full-size copy of each tensor with direct-io (#29749)
32dd62e Pranesh Gonegandla committed 1 month ago
HIP: avoid treating CDNA as dgx spark for gqa_ratio 20 in fattn_mma dqk 576 (#29572)
f11d642 uvos committed 1 month ago
hex-workqueue: fix race condition in seqn getting out of sync with idx_read/write (#29785)
3aa0ce9 Max Krasnyansky committed 1 month ago
BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS (#29640)
b0aca3c Pradeep Rao committed 1 month ago
opencl: mark vec subgroup bcast as supproted for Adreno E17 compiler (#29698)
3ec4df4 lhez committed 1 month ago
vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 (#29734)
db33d3c Toki Nasin committed 1 month ago
llama-bench : fix verbosity filter to show GGML_LOG_ERROR (#28229)
7dad6db Kushal Garg committed 1 month ago
ggml-opencl : replace alloca() with std::vector (#29765)
f7b384c Adrien Gallouët committed 1 month ago
cuda: guard the iq4_nl dequantize row kernel against short rows (#29683)
f872b59 R0CKSTAR committed 1 month ago
Hexagon: optimize ALLREDUCE with support for safe scatter mode (#29757)
a4d880f Ehsan Bateni committed 1 month ago
llama : preserve original batch order for speculative decoding layer inputs (#29019)
4453b53 Hrishith Thadicherla committed 1 month ago
test-llama-archs : toggle causal_attn to catch graph shape changes (#29724)
4f31296 Sihan Yu committed 1 month ago
convert : fix LoRA conversion crash for Qwen3.5 V-head reorder (#28324)
b016f46 Swigler committed 1 month ago
batch: migrate the rest of examples to llama_batch_ext (#29601)
60e9cf7 Xuan-Son Nguyen committed 1 month ago
glm5-next: give dead indexer slots unique scatter rows (#29745)
05af0d2 Pascal committed 1 month ago
cli: exit on stdin EOF and drop the console wide Ctrl+C broadcast (#29722)
b046420 Pascal committed 1 month ago
mimo : support dflash (convert + feature extraction) (#29650)
22bdcc4 Georgi Gerganov committed 1 month ago