LLM inference in C/C++
C++ ggml
Commits on master
server : move chat-template thinking probe inside the init try/catch (#24093)
82fce65 Jesse LaRose committed 4 months ago
meta: add hard emphasis on agents not writing descriptions/comments (#25480)
c15c5c7 Piotr Wilkin (ilintar) committed 4 months ago
Refactor: Consistently use smart pointers in `test-backend-ops` (#25440)
f84a519 Oliver Simons committed 4 months ago
Only index by compile times + always multiply/add (#25445)
683f0c7 Oliver Simons committed 4 months ago
metal : add CONV_2D_DW (depthwise convolution) support (#21565)
92b187c Sou-ly committed 4 months ago
cuda: align snake fusion matcher with the other backends (#25460)
2021515 Pascal committed 4 months ago
server : respect min-step when splitting prompt batches (#25420)
64c8b7d Aldehir Rojas committed 4 months ago
ggml-webgpu: tune subgroup split (d_split) in flash_attn_vec (#25418)
32e41fa Masashi Yoshimura committed 4 months ago
opencl: Q6_K GEMM/GEMV fix for ne01 of weights that are not multiples of 128. (#25464)
92366df Hongqiang Wang committed 4 months ago
vulkan: disable FA mask_opt on GCN to improve performance (#24362)
a646006 Ruben Ortlam committed 4 months ago
opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433)
167d057 Hongqiang Wang committed 4 months ago
vulkan: for small AMD GPUs, reduce submission threshold based on CU count (#25240)
0bbc87b Ruben Ortlam committed 4 months ago
hexagon: new vtcm layouts and improved pipelines for MUL_MAT, MUL_MAT_ID and FLASH_ATTN_EXT (#25425)
81ff7ab Max Krasnyansky committed 4 months ago
cuda : add support for f16->f16 GGML_OP_SET_ROWS (#25367)
ed8c261 fairydreaming committed 4 months ago
server-stream: follow-up on SSE Replay Buffer (#23226) (#25047)
bbebeec Pascal committed 4 months ago
llama-batch: add n_keep_tail in split_equal for recurrent models (#25278)
230ea9d Aman Gupta committed 4 months ago
common: auto-create prompts-log-dir at argument parsing, so all tools using the flag benefit (#25322)
f296fdf rankaiyx committed 4 months ago
llama-eval : fix crash when answer is None in HTML dump (#25435)
da46e59 Georgi Gerganov committed 4 months ago
fix: OOB reads in UGM tokenizer (precompiled_charsmap handling) (#18750)
4a7ee31 hourhl committed 4 months ago
ggml : fix A indexing in simd_gemm scalar tail-column path (#25390)
57b50e1 tyronecai committed 4 months ago