LLM inference in C/C++
C++ ggml
Commits on master
hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (#27798)
83d855c Aparna M P committed 2 months ago
ci : build only the ggml-hip backend for windows-rocm release (#27753)
3217633 Harkirat Gill committed 2 months ago
ci : bundle HIP runtime DLLs with Windows ROCm release (#26973)
6fdd0ac Slobodan Josic committed 2 months ago
spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816)
b10f9ca Xuan-Son Nguyen committed 2 months ago
opencl: add bin kernels `kernel_gemm_moe_q4_0_q8_1_dp4a_bin`, `kernel_gemm_moe_mxfp4_q8_1_dp4a_bin` (#27768)
5854625 Shawn Gu committed 2 months ago
quantize: cap working memory size to avoid loading big tensors onto RAM (#27795)
732707d Xuan-Son Nguyen committed 2 months ago
Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (#27453)
cb30059 Shobhit committed 2 months ago
pr2wt : use ssh/https remote in worktree depending on base (#27800)
1a946ec Sigbjørn Skjæret committed 2 months ago
ui: Improve Chat Form Actions UI/UX (models selector, add panel) (#27746)
cae6357 Aleksander Grygier committed 2 months ago
ui: Replace per-conversation MCP overrides with per-conversation tool policy (#27745)
fe235f4 Aleksander Grygier committed 2 months ago
spec: Add benchmark-only synthetic speculative acceptance options (#27711)
2bb9bdd Gaurav Garg committed 2 months ago
metal : fix memory leaks due to missing autoreleasepools (#27758)
915dc6d Niklas Wenzel committed 2 months ago
hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (#26501)
192067b Max Krasnyansky committed 2 months ago
ui: Move Settings and MCP Servers routes to dialog-based views (#27744)
539f245 Aleksander Grygier committed 2 months ago
vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (#27726)
5e6a37c Ruben Ortlam committed 2 months ago
Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (#27216)
bf94216 Pranav Uttarkar committed 2 months ago
rpc : implement event and async backend APIs (#18626)
d0132a6 Radoslav Gerganov committed 2 months ago