LLM inference in C/C++
C++ ggml
Commits on master
cuda: Don't crash when querying memory on device with no free memory. (#25157)
3cec3bc cphlipot committed 4 months ago
DeepseekV4: clear cache only for seq rather than full (#25521)
13f2b28 Aman Gupta committed 4 months ago
ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (#24231)
00f5442 fairydreaming committed 4 months ago
Vulkan: route large matmuls to medium tile on Adreno (#24877)
76f2798 Raman Shinde committed 4 months ago
opencl: add int8 dp4 dense and MoE prefill optimization for Adreno GPUs (#25537)
1d1d9a9 Hongqiang Wang committed 4 months ago
llama : make tensor-split regex patterns static (#24710)
c749cb0 eduardopessin committed 4 months ago
hexagon: improve ARGSORT performance for small tensors (#25512)
67776ea Max Krasnyansky committed 4 months ago
ggml : use ggml_vqtbl1q_u8 for 32-bit compat (whisper/0)
ac17f8a Georgi Gerganov committed 4 months ago
ui: prevent tooltip from flickering open and closed on hover (#25503)
c7af942 Pascal committed 4 months ago
ui: export full message tree instead of active path only (#25501)
d46786f Pascal committed 4 months ago
llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370)
2ed3c1a fairydreaming committed 4 months ago
opencl: cluster-parallel decode FA for Adreno (#25473)
049326a Hongqiang Wang committed 4 months ago
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776)
0749449 fairydreaming committed 4 months ago
hexagon: tiling, tracing and optimizations for unary ops (#25474)
fb30ba9 Aparna M P committed 4 months ago