LLM inference in C/C++
C++ ggml
Commits on master
CUDA: Allow concurrent streams per split for multi-GPU (#28198)
0ba6499 Tanner Bruhn committed 2 months ago
sycl : enhance the api to support peer-to-peer copy (#27550)
0df974d Neo Zhang committed 2 months ago
convert : skip bias_vl tensor in DeepSeek-V4 DSpark conversion (#28294)
d646c9d Georgi Gerganov committed 2 months ago
misc : prevent RAM peaking at model loading stage (#27483)
5ec4eab Tarek Dakhran committed 2 months ago
sycl: reduce redundant work in Q4_K multi-column MMVQ (#27062)
4aa6ffb Eurekatic committed 2 months ago
model: add NVIDIA Nemotron-3-Puzzle-75B-A9B (NemotronHPuzzle) support (#25444)
c61b98b Yaniss Amazouz committed 2 months ago
server : accept data: URLs for input_video and input_audio (#27735)
9cffdcc Abhiram committed 2 months ago
metal : fix memory query under low-memory conditions (#27701)
8c0b9cd Mads Marquart committed 2 months ago
model: correctly support input vision for deepseek4 (#28154)
9400c89 Xuan-Son Nguyen committed 2 months ago
common, server : enable preserve_reasoning kwarg by default, log its effective state (#28174)
e750b88 Georgi Gerganov committed 2 months ago
ggml: avoid KleidiAI buffer type init on dispatch (#27891)
b81c99b Aman Chadha(IVIXMMI) committed 2 months ago
hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (#28202)
960dffa Max Krasnyansky committed 2 months ago
vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (#27449)
ba8818c Laurent Zuijdwijk committed 2 months ago
vulkan : only request VK_KHR_shader_bfloat16 extension if supported (#28155)
56dd815 Mads Marquart committed 2 months ago