LLM inference in C/C++
C++ ggml
Commits on master
cuda : use SIMT implicit GEMM conv2d on AMD without WMMA (#30325)
62bacfa An Long committed 1 month ago
jinja : write floats in tojson as Python does (#30362)
77390ba Yuri Khrustalev committed 1 month ago
opencl: use buf+offset instead of subbuffer for adreno moe expert sort (#30418)
c5a9976 lhez committed 1 month ago
opencl: fix q8_0 dp4a mul_mat_id and mark PREC_F32 mul_mat_id unsupported for adreno (#30365)
21a19cd lhez committed 1 month ago
models: drop the t_embd expand of the embedding graphs (#30461)
9cf967a Pascal committed 1 month ago
HIP: use 64-row MMQ tiles for some quant types on RDNA3.5 (#28195)
4dd6dd4 liangliangchang committed 1 month ago
llama : build the k-pool selection in the causal mask layout (#30480)
e7268bf Georgi Gerganov committed 1 month ago
vulkan: Add workaround for device free memory > total memory on Intel Windows (#29835)
cdce056 Masato Nakasaka committed 1 month ago
vulkan: Define the shader count for Intel Arc Pro B65 + B70 (#30431)
d524bb6 Matt Corallo committed 1 month ago
unicode : add opt-in NFC normalization for Unigram tokenizers (#30390)
10da8c4 Sandro Hanea committed 1 month ago
Enable OpenSSL in FP32/FP16 SYCL builds to restore HF HTTPS downloads (#30330)
091b6a7 Raffaele Di Donna committed 1 month ago
webgpu: (Fix CI) fix mul_mat_id with duplicate expert ids (#30451)
fcac190 Masashi Yoshimura committed 1 month ago
server : escape the special tokens without std::regex (#30243)
04f7226 Yuri Khrustalev committed 1 month ago
ggml : simplify backend search path error handling (#28367)
1b70350 Adrien Gallouët committed 1 month ago
vulkan: Optimize pool_2d when the pooling window is large (#30119)
77c4540 Jeff Bolz committed 1 month ago
meta: recreate all subgraphs when the arena is reallocated (#30453)
ecdd29e Igor Matuszewski committed 1 month ago
hexagon: add configurable HTP DCVS/HMX power mode (#30423)
ec0bfac Pranav Dhinakar committed 1 month ago
llama : do not map foreign ranges into a backend buffer (#30405)
b9f3599 Georgi Gerganov committed 1 month ago
model: load YaRN beta metadata for laguna models (#30331)
f441d2d Adam Caldwell committed 1 month ago
tests : consolidate context state tests into test-llama-context (#30023)
a17e94d Georgi Gerganov committed 1 month ago