LLM inference in C/C++
C++ ggml
Commits on master
server : fix reasoning budget WebUI precedence over model.ini (#24517)
ebc1077 Georgi Gerganov committed 5 months ago
vulkan: add pipeline barriers for memcpy read operations (#23770)
3e7bd4f Ruben Ortlam committed 5 months ago
ggml: support concat for scalar types at cuda backend (#24011)
85f99dc ZihaoMu committed 5 months ago
opencl: add q5_0/q5_1 gemm and gemv kernels for Adreno (#24319)
ba1df05 shaofeiqi committed 5 months ago
vulkan: add fast path for contiguous buffer transfers (#23973)
fdc3db9 Winston Ma committed 5 months ago
server: skip unused log lines on router mode (#24463)
18ef86e Xuan-Son Nguyen committed 5 months ago
vocab : adopt leading TemplateProcessing special token as BOS (#24428)
1bfbdb1 o7si committed 5 months ago
vocab : refactor normalizer flags into options struct, add strip_accents (#24371)
68f3066 o7si committed 5 months ago