LLM inference in C/C++
C++ ggml
Commits on master
vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164)
08ffba2 Amadeus dyw committed 1 month ago
ggml: Change default visibility of symbols to hidden (#30014)
a05e58c Christian Kastner committed 1 month ago
vulkan: fix buffer logging on 32-bit platforms after source split (#30326)
b2f036c miyan committed 1 month ago
opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266)
f2cc728 lhez committed 1 month ago
opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310)
c83f305 lhez committed 1 month ago
rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs (#29544)
5c8854f Georgi Gerganov committed 1 month ago
vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998)
966baae Jeff Bolz committed 1 month ago
server: leave a busy slot untouched when a request pins it (#30295)
23b0202 Pascal committed 1 month ago
cmake(s390x): disable z17 target for unsupported compilers (#30297)
abee0c8 Aaron Teo committed 1 month ago
args: add LLAMA_ARG_SLOT_SAVE_PATH env for --slot-save-path (#30272)
aa94f20 Mikolaj Kucharski committed 1 month ago
CUDA/HIP: fix race in flash_attn_ext_f16_process_tile when nbatch_combine != DKQ/2 (#30103)
1bb2b9f uvos committed 1 month ago
opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187)
f2918ca Shawn Gu committed 1 month ago
sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809)
1e6f04a Captain-Tripps committed 1 month ago
vendor: apply deep nested json patch from upstream (#30253)
10a60cf Xuan-Son Nguyen committed 1 month ago
CUDA: fix round issue, under MSVC the CPU and GPU agree (#30229)
79e2e74 Dante committed 1 month ago