LLM inference in C/C++
C++ ggml
Commits on master
sycl : support gated DSV4_HC_PRE and optional HC_POST comb matrix (#29132)
bb3c853 cwriter committed 1 month ago
sycl : pinned memory use right device context instead of 0 (#28895)
af91114 Łukasz Ślusarczyk committed 1 month ago
CUDA: Follow up of #25635, refactoring FA shared smem swizzle (#28536)
1884824 ynankani committed 1 month ago
test-llama-archs : make tensor data stdev configurable and improve help (#29133)
161755f Georgi Gerganov committed 1 month ago
tests/test-backend-ops : allow regex entries in the -o filter (#29204)
1d72b05 Georgi Gerganov committed 1 month ago
ci : refactor build-self-hosted into backend-specific workflows (#28991)
542e920 Sigbjørn Skjæret committed 1 month ago
args: add env vars for temperature, top-p, min-p and penalties (#27380)
e0dff58 Mikolaj Kucharski committed 1 month ago
server : do not forward --api-key-file to router-spawned child instances (#28938)
982a332 Nandan Vallamdasu committed 1 month ago
ggml-metal : simplify fusion pattern op list declaration (#29206)
335b21f Georgi Gerganov committed 1 month ago
sycl : coalesce MKL-FA softmax loads instead of one work-item per row (#28918)
1aa2954 Anant Shrivastava committed 1 month ago
ci : Upgrade CUDA to 13.4 for Ubuntu CUDA Release Builds (#29202)
6ad1af5 Samriddha Sinha committed 1 month ago
hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements (#29197)
0c3626e Max Krasnyansky committed 1 month ago
cuda : tune MMVQ to MMQ crossover for SM70 (Volta) (#28912)
68d9053 Yangyu Chen committed 1 month ago
metal : fix deprecation warnings from macOS 27 SDK (#29136)
8aa161b Niklas Wenzel committed 1 month ago
CUDA: tune FA for Gemma 4 on Ampere or newer (#29152)
ce8caa6 Johannes Gäßler committed 2 months ago
common/peg : handle invalid utf-8 sequences in the AST (#29161)
3d82ef6 Aldehir Rojas committed 2 months ago
ui: Fix mobile breakpoint + content overflow issues (#29108)
b23efaa Aleksander Grygier committed 2 months ago
fix(mamba) : make time-step projection input contiguous (#28832)
4260903 Andrei committed 2 months ago