LLM inference in C/C++
C++ ggml
Commits on master
convert : more consistent handling of rope_parameters (#24833)
f4043fe Sigbjørn Skjæret committed 5 months ago
ggml-webgpu: add adapter toggles for F16 on Vulkan + NVIDIA
f449e05 Masashi Yoshimura committed 5 months ago
server: refactor child --> router communication (#24821)
2b686a9 Xuan-Son Nguyen committed 5 months ago
server: remove all internal mentions about "webui" (#24817)
175147e Xuan-Son Nguyen committed 5 months ago
arg: Add comment line support to --api-key-file (#23168)
fabde3b Mikolaj Kucharski committed 5 months ago
vendor : update cpp-httplib to 0.48.0 (#24787)
0d2d9cc Alessandro de Oliveira Faria (A.K.A.CABELO) committed 5 months ago
server: add --agent arg, remove redundant webui naming compat (#24801)
8c2d6f6 Xuan-Son Nguyen committed 5 months ago
server: fix non-bound n_discard value (ctx shifting) (#24786)
159d093 Xuan-Son Nguyen committed 5 months ago
server : consolidate slot selection into get_available_slot (#24755)
80452d6 Georgi Gerganov committed 5 months ago
ggml-cpu: support K tails in power10 Q8/Q4 MMA matmul (#24753)
8141e73 shalinib-ibm committed 5 months ago
server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774)
40f3aaf Reguna committed 5 months ago
mtmd: add batching for mtmd-cli, add video tests (#24778)
a6b3260 Xuan-Son Nguyen committed 5 months ago
mtmd: refactor llava-uhd overview image handling (always use ov_img_first) (#24769)
060ce1b Xuan-Son Nguyen committed 5 months ago
hexagon: support for op-trace (fine-grain tracing of HVX/HMX/DMA events) (#24592)
d2c6795 Max Krasnyansky committed 5 months ago
docs: fix export-lora --lora-scaled syntax [no release] (#24703)
7b6c5a2 Kangjia Gao committed 5 months ago
server: (router) fix stopping_thread potentially hang (#24728)
fe7c8b2 Xuan-Son Nguyen committed 5 months ago
server : add last-5-seconds generation speed display (#24291)
0802307 Aarni Koskela committed 5 months ago