LLM inference in C/C++
C++ ggml
Commits on master
cuda: support Nemotron 3 Puzzle state size 96 for ssm scan (#28717)
9588757 Anav Prasad committed 1 month ago
musa: build the docker images from the PH1 MUSA SDK image (#29481)
694ec23 R0CKSTAR committed 1 month ago
Revert "Change max context length for auto-fitting with unified KV (#28849)" (#29437)
2145525 Gaurav Garg committed 1 month ago
llama : fix K/V and recurrent state cleanup after failed restores (#27530)
08618ff Chipmunk committed 1 month ago
jinja : support noncall test statements with arg (#29443)
a1de614 Sigbjørn Skjæret committed 1 month ago
hexagon: find software divide calls using binary inspection tool (#29449)
4e74811 Trivikram Reddy committed 1 month ago
vendor : update cpp-httplib to 0.58.0 (#29407)
171e884 Alessandro de Oliveira Faria (A.K.A.CABELO) committed 1 month ago
common,rpc : simplify fs_create_directory_with_parents() (#29432)
4b1a27f Adrien Gallouët committed 1 month ago
opencl: add bin kernel `kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin`, `kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin` (#29401)
a25c986 shaofeiqi committed 1 month ago
Fixing the vulkan build issue of legacy GLSLC version that has no cooperativeMatrix API support (https://github.com/ggml-org/llama.cpp/issues/29373) (#29409)
e85e15c sliu39 committed 1 month ago
gguf-py : ByteLevel processing defaults bos/eos to False (#29422)
b248f4a Sigbjørn Skjæret committed 1 month ago
gguf-py : TemplateProcessing has final word on add_special_token (#29417)
d81aef1 Sigbjørn Skjæret committed 1 month ago
common : extract shared unicode path/string helpers (#29415)
27b20ba Adrien Gallouët committed 1 month ago
metal : split fa kernels into per-dtype libraries (#29329)
5a75f14 Georgi Gerganov committed 1 month ago
llama : add `llama_prec_policy` + model-driven W4A4 path (#24364)
e9f824d ynankani committed 1 month ago
HIP: bump HIP_VERSION requried for fp8 to avoid missing __hip_fp8_e4m3 support in 6.2 (#29231)
d028c69 uvos committed 1 month ago
rpc: include nb in the get_alloc_size cache key and floor the result at ggml_nbytes (#29283)
66963a8 Jess Sullivan committed 1 month ago
musa: fix PH1 (MTT S5000) operator failures and build issues (#29193)
f9af9be R0CKSTAR committed 1 month ago
llama : fix tensor split for fused qkv with uneven K/V head sizes (#29294)
f805c57 Aman Gupta committed 1 month ago