LLM inference in C/C++
C++ ggml
Commits on master
server: add --connect, allow p2p access to server outside LAN (#28277)
7adf160 Xuan-Son Nguyen committed 1 month ago
ui : fix attachment text preview clipped when a line is longer than the container (#30806)
ab7ed0d A-Raphie committed 1 month ago
opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k (#30741)
1b29621 lhez committed 1 month ago
vendor : update cpp-httplib to 0.62.0 (#30785)
7951f60 Alessandro de Oliveira Faria (A.K.A.CABELO) committed 1 month ago
moe-cache : replace MoE LRU cache with a 50/50 SLRU cache (#30637)
33d6135 Evan Huus committed 1 month ago
metal : fix moe_reduce fusion race when dst aliases its inputs (#30795)
098ce52 Georgi Gerganov committed 1 month ago
cann : refactor `#ifdef` blocks to avoid unreachable code after `return` (#25273)
f3fb2fc Raul Torres committed 1 month ago
CUDA: fix warning for nonexistent extra semicolon (#30774)
4866db3 Johannes Gäßler committed 1 month ago
qwen3 : bidirectional attention and dense_2 output projection (#30607)
412a363 fzowl committed 1 month ago
ui: Fix markdown tables and lists being cut off on phones (#30687)
2b57148 Aleksander Grygier committed 1 month ago
tools : reject truncated GGUF tensor data in split and export tools (#30497)
8f97d82 ppAlexa committed 1 month ago
tests : update Metal fusion baseline for beam (#30764)
27af581 Georgi Gerganov committed 1 month ago
pi : turn .pi/gg into a system prompt addendum (#30762)
7db0d3d Georgi Gerganov committed 1 month ago
CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573)
99db719 Paul Mairhofer committed 1 month ago
hexagon: FLASH_ATTN_EXT improvements for longer context and SWA models (#30731)
31288c9 Max Krasnyansky committed 1 month ago
cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#30602)
ff5a233 Aman Gupta committed 1 month ago
ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) (#28851)
0878f7c Andrea Rossetti committed 1 month ago
ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_fa) (#30572)
5aeac37 thelittlefireman committed 1 month ago
hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and tiled Q1_0 and Q2_0 (#30627)
848bbda Todor Boinovski committed 1 month ago