LLM inference in C/C++
C++ ggml
Commits on master
json-schema : accept escaped hyphen in regex patterns (#29127)
eb1e1f4 chiheb ben cheikh committed 2 months ago
metal : add MoE and SSM_CONV fusion optimizations (#28948)
5b59b83 Georgi Gerganov committed 2 months ago
test-llama-archs : generate dummy test vocab (#29084)
efa28e9 Georgi Gerganov committed 2 months ago
cuda : fix CUB argsort corruption caused by in-place keys (#28389)
b23701f TheArchitectit committed 2 months ago
opencl: add support for bin kernel `flash_attn_f32_f16_bin` (#29046)
60081bb dsproule committed 2 months ago
hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539)
18a04f0 Todor Boinovski committed 2 months ago
opencl: add bin kernel `kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin` (#28678)
ec92815 shaofeiqi committed 2 months ago
ggml : check for allocation failures to prevent crashes (#28149)
5b335f4 Alexey Kopytko committed 2 months ago
Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042)
542348a Pascal committed 2 months ago
ggml-webgpu: fix supports_op condition for GET_ROWS (#28978)
44be98f Masashi Yoshimura committed 2 months ago
cmake : fix build when GGML_CPU=OFF and GGML_CUDA=ON (#29026)
8ed1a55 Nikita Gordeev committed 2 months ago
gguf-py: fix Q8_1 block size in GGML_QUANT_SIZES (2+2+32) (#29036)
bb11ebb yanghong committed 2 months ago
ci : bump android-actions/setup-android to 4.0.4 (#29065)
bdcbaaf Sigbjørn Skjæret committed 2 months ago
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 512 experts (#28501)
5c53396 drluoto committed 2 months ago
model : skip gate_up_exps if TENSOR_SKIP is set (#29014)
b49650a David Friehs committed 2 months ago