LLM inference in C/C++
C++ ggml
Commits on master
CUDA: add a reserve to avoid spurious warning on older GCC builds (#29317)
66fba63 Aman Gupta committed 1 month ago
common : keep HF cache dir as path, expose UTF-8 only for logs (#29320)
bddf826 Adrien Gallouët committed 1 month ago
metal : key the fa-vec tuned table by family instead of SKU (#29075)
ee3ecce YiChen Lv committed 1 month ago
server: accept OpenAI video_url content type and data: video URIs (#27921)
057494f calebrio02 committed 1 month ago
server: Dedup the draft HF model via dedup-cache-models (#27934)
bcbc936 Xie Wenxiang committed 1 month ago
model-conversion : add causal-compare-logits recipe (#29305)
18f9f7b Daniel Bevenius committed 1 month ago
model : support Gemma4 DSpark draft backbone (#29226)
633733d Hrishith Thadicherla committed 1 month ago
vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415)
b1ff4ca Piotr Wilkin (ilintar) committed 1 month ago
sycl : support new UT case for mul_mat_hadamard fp16 (#29218)
384a534 Neo Zhang committed 1 month ago
sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions (#28931)
5e48b31 Anant Shrivastava committed 1 month ago
sycl : support op get_rows_back, only support fp32/fp16 (#25266)
4d7d770 Neo Zhang committed 1 month ago
vulkan: hide internal symbols to prevent duplicate-dlopen state destruction (#29139)
08b1d2a Erik Winter committed 1 month ago
hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling (#29282)
e6ab7c1 Max Krasnyansky committed 1 month ago
HIP : optimize IQ2/IQ3 (`__vsub4` `__vcmpne4`) using SWAR (#27962)
f46bc30 Felix Ye committed 1 month ago
jinja : fix dangling reference warning in for_statement (#29279)
709fe75 Georgi Gerganov committed 1 month ago