LLM inference in C/C++
C++ ggml
Commits on master
ui: exclude generated build dirs from prettier and eslint so lint errors stop being masked (#23910)
d38d50e Pascal committed 5 months ago
TP: fix granularity for Qwen 3.5/3.6 + 3 GPUs (#23843)
8b0e0db Johannes Gäßler committed 5 months ago
metal : restore im2col implementation for large kernels (#23901)
2d9b7c8 Georgi Gerganov committed 5 months ago
test: (test-llama-archs) log the config name first (#23885)
e674b12 Xuan-Son Nguyen committed 5 months ago
ci : update ios-xcode release job to macos-26 (#23906)
4c4e91b Georgi Gerganov committed 5 months ago
vulkan: add Flash Attention support for BFloat16 KV cache (#23420)
6e093b8 Ruben Ortlam committed 5 months ago
ci : clear cache instead of "no timestamp" keys + fix macos (#23895)
d4204b0 Georgi Gerganov committed 5 months ago
llama : do not skip iGPU when only RPC devices are present (#23868)
1738129 Radoslav Gerganov committed 5 months ago
server: in SSE mode, send HTTP headers when slot starts (#23884)
0821c5f Xuan-Son Nguyen committed 5 months ago
ggml-webgpu: Check earlier for WebGPU required features (#23879)
151f3a9 Reese Levine committed 5 months ago
server-bench : add speed-bench for speculative decoding benchmarking (#23869)
689a9a4 Ruixiang Wang committed 5 months ago
ci : update macos release to use macos-26 runner (#23878)
dc71236 Georgi Gerganov committed 5 months ago
CUDA: Check PTX version on host side to guard PDL dispatch (#23530)
6ed481e Oliver Simons committed 5 months ago
model : support for DeepseekV32ForCausalLM with generic DeepSeek Sparse Attention (DSA) implementation (#23346)
1f0aa2a fairydreaming committed 5 months ago