LLM inference in C/C++
C++ ggml
Commits on master
ggml-cpu : conditionally add SpacemiT IME kernel sources (#27961)
2637dfe Alan Tseng committed 2 months ago
opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632)
43d87ff Hongqiang Wang committed 2 months ago
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (#28217)
69320fe Trivikram Reddy committed 2 months ago
metal : add metallib build support for xcframework (#28163)
b96806d Jhen-Jie Hong committed 2 months ago
kv-cells: look up the n-gram history in the sequence position index (#28040)
b356fa2 Pascal committed 2 months ago
context : autoscale n_ctx_train when yarn scaling specified (#28030)
dfc29b6 Sigbjørn Skjæret committed 2 months ago
models : appropriately flag noscan ssm_a tensors (#28121)
f28493c Sigbjørn Skjæret committed 2 months ago
model : load relevant arrays with n_layer_all (#28173)
d11b3cc Sigbjørn Skjæret committed 2 months ago
Revert "sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…" (#28184)
c845263 Titaniumtown committed 2 months ago
sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 (#28016)
1f3d318 Jingxin (Philip) Li committed 2 months ago
metal : add fa-vec tunings for A18 Pro (MacBook Neo) (#28152)
be789c3 Jhen-Jie Hong committed 2 months ago
model : load hparams.n_layer_nextn before n_layer() calls (#28159)
9d81721 Sigbjørn Skjæret committed 2 months ago
metal : fix more leaks due to missing autoreleasepools (#27883)
fe2120b Niklas Wenzel committed 2 months ago
sycl : support limit max alloc memory within 2GB for host-pinned memory (#27559)
5eec3ad Neo Zhang committed 2 months ago
qwen4exp: fix seq_cp, block position keying, mtmd input, cuda abort, add tests (#27941)
36b1015 Daniel Han committed 2 months ago
tests : fix log verbosity for test-llama-archs (#28147)
d086dbb Georgi Gerganov committed 2 months ago
quantize: row-slab stream to avoid thread starvation (#27830)
1b89a43 Xuan-Son Nguyen committed 2 months ago
metal: enable Metal 4.0 tensor API on M5+/A19+ (#27461)
d5d993a James Francis committed 2 months ago
metal : add concat support for quantized types (#28116)
ab0b3bd Georgi Gerganov committed 2 months ago
AVX2: Speed up large batch size prompt processing of IQ models (#27402)
85c5522 Bartowski committed 2 months ago