LLM inference in C/C++
C++ ggml
Commits on master
mtmd: enable non-causal vision for gemma 4 unified (#24082)
c8d6a00 Xuan-Son Nguyen committed 5 months ago
ui: Mermaid Diagrams in chat + interactive preview (#24032)
ee4cf70 Aleksander Grygier committed 5 months ago
Avoid PDL race conditions by disabling __restrict__ when PDL is used (#24030)
9e58d4d Andreas Kieslinger committed 5 months ago
cuda: reserve space for quantize kv-cache at startup (#23907)
f8f0a47 Aman Gupta committed 5 months ago
update BoringSSL to 0.20260526.0 (#23794)
d545a2a Alessandro de Oliveira Faria (A.K.A.CABELO) committed 5 months ago
ci : disable ccache for msvc windows release jobs (#23911)
4da6370 Georgi Gerganov committed 5 months ago
arg : removed unecesary mmproj download when users pass --no-mmproj (#23425)
e366626 Ryan Mangeno committed 5 months ago
opencl: use flat variants of q4_K and q6_K gemv for very large M (#24006)
63e66fd lhez committed 5 months ago
hexagon: profiler output fix and script updates (#24042)
5c394fd Max Krasnyansky committed 5 months ago
model : support granite multilingual embeddings R2 (ibm-granite/granite-embedding-{97,311}m-multilingual-r2) (#22716)
bfb4308 Hans Florian committed 5 months ago
common : fix state save in common_prompt_batch_decode (#23468)
0b71540 Daniel Bevenius committed 5 months ago
ci : reduce self-hosted server workflow jobs (#24012)
a468b89 Georgi Gerganov committed 5 months ago
ui: Add Thinking mode toggle with reasoning effort levels + improvements for Chat Form Add Action UI (#23434)
f8e67fc Aleksander Grygier committed 5 months ago
kv-cache : SWA checkpoints store only non-masked cells (#23981)
2365315 Georgi Gerganov committed 5 months ago
hexagon: MUL_MAT, MUL_MAT_ID, FLASH_ATTN and GDN cleanup and optimizations for latest models (#23989)
8f7f3bf Max Krasnyansky committed 5 months ago
server: real-time reasoning interruption via control endpoint (#23971)
354ebac Pascal committed 5 months ago