Skip to content

LLM inference in C/C++

C++ ggml

Commits on master

mtmd: add n_embd_head (#26342)
Xuan-Son Nguyen committed 3 months ago
db7d8b2
Support rotated kv cache quant (#26180)
timkhronos committed 3 months ago
a09d8ab
82dbc4f
vulkan: update vulkan sdk to 1.4.357.0 (#26303)
Jeff Bolz committed 3 months ago
6f3c0a7
cuda: extract Q2_0 elements via __byte_perm (#25603)
David Friehs committed 3 months ago
15e755f
[SYCL] support the missed types in cpy (#26005)
Neo Zhang committed 3 months ago
d5d3e05
1e22599
[SYCL] Support q2 mul_mat (#26231)
Neo Zhang committed 3 months ago
a2be61d
sycl: fuse RMS_NORM + MUL (#26015)
Titaniumtown committed 3 months ago
1553725
mtmd: add lanczos resize method [no release] (#26341)
Xuan-Son Nguyen committed 3 months ago
5f55650
server: support inp embd to generate next token (#26313)
Xuan-Son Nguyen committed 3 months ago
b4ca032
vulkan: Support quantized concat (#25684)
Jeff Bolz committed 3 months ago
ea63b4d
958d9c0
tests : avoid building get-model.cpp many times (#26317)
Georgi Gerganov committed 3 months ago
47f686f
ggml-cuda: Allow transpose-free gemmv computation (#26171)
Robert Esclapez committed 3 months ago
e1a1abb
readme : refresh (#26280)
Georgi Gerganov committed 3 months ago
6b36c23
sync : ggml
Georgi Gerganov committed 3 months ago
9ebfc3a
ggml : bump version to 0.18.0 (ggml/1576)
Georgi Gerganov committed 3 months ago
6a4c335
CUDA: add Q2_0 support (#25707)
Pasha Khosravi committed 3 months ago
9b2a088
ui: IndexedDB and Conversations data fixes (#26278)
Aleksander Grygier committed 3 months ago
21a5f5b