LLM inference in C/C++
C++ ggml
Commits on master
Extend probabilistic sampling for DFlash2 drafts (#30663)
1ab09d6 Pranesh Gonegandla committed 27 days ago
hexagon: Op Fusion refactoring, optimizations and new fusions like `BINARY_NX` (back-to-back ADDs, etc) (#30843)
643b0cd Max Krasnyansky committed 27 days ago
sycl: Replace host waits in fattn-mkl with explicit dependencies (#30684)
0a9ef00 Todd Malsbary committed 27 days ago
server: Substitute rerank placeholder from the original template only. (#30864)
184f653 pzhu committed 27 days ago
server: realtime API: send complete after committed audio drains (#30994)
d814cbb Xuan-Son Nguyen committed 27 days ago
clef : include <algorithm> for std::find and std::copy (#30981)
19ab5a5 Martin Kleinberger committed 28 days ago
model : fix qwen4exp layer inputs for DFlash drafters (#30927)
d9ee007 Georgi Gerganov committed 28 days ago
vulkan: replace std::aligned_alloc with posix_memalign for termux compatibility (#30828)
d772569 Bill Sideris committed 28 days ago
nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type (#23343)
0693dbf martin-cimmino committed 28 days ago
server : support typed content input for RERANK endpoint (#29625)
a5d2227 Tim Wang committed 28 days ago
ci : be able to reconfigure HTTP port for server pytest workers (#30972)
1f4c2ee Georgi Gerganov committed 28 days ago
sycl: fix CONV_2D wrong results for non-contiguous input (#30363)
7170cad Michael Taylor committed 28 days ago
vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 one (#30960)
e197cef tkran2 committed 28 days ago
models : Record layer inputs for all granite models (#30957)
bf039c8 Gabe Goodhart committed 28 days ago
kleidiai : add Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels (#30808)
dfeea52 Charles Xu committed 28 days ago
ui: Add "Show tool call in progress" setting (#30681)
84746c5 Aleksander Grygier committed 28 days ago