LLM inference in C/C++
C++ ggml
Commits on master
opencl: exclude some moe kernels on Adreno a7x (#25698)
505b1ed Hongqiang Wang committed 4 months ago
cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) (#25545)
3b53219 fairydreaming committed 4 months ago
tokenize : drop --stdin mutual-exclusion check (#25672)
aff6eb6 Adrien Gallouët committed 4 months ago
opencl: fix two issues on flash attention for Adreno a7x (#25697)
c3d47e6 Hongqiang Wang committed 4 months ago
CUDA: tighter MMQ src1 buffer size for native fp4 (#25613)
f6f12e4 leonardHONG committed 4 months ago
server: fix read_file append_loc space breaking edit_file match (#25705)
a582222 Pascal committed 4 months ago
ui: fix thinking menu never appearing in single-model mode (#25637)
a05df0a Pascal committed 4 months ago
cuda : relax tensor contiguity requirements for quantized concat (#25678)
a3e5b96 fairydreaming committed 4 months ago
metal: fuse snake activation (mul, sin, sqr, mul, add) (#25459)
b3c9d1b Pascal committed 4 months ago
ggml: add f16 out_prod support for CPU and out_prod op for Vulkan (#23997)
f955e39 Michael Lamothe committed 4 months ago
sycl: Increase minimum buffer size for USM system allocations (#25525)
0e148a5 Francois Dugast committed 4 months ago
opencl: do not use `clCreateBufferWithProperties` when targeting CL 2.x (#25673)
12127de Hongqiang Wang committed 4 months ago
opencl: handle OOB write in noshuffle GEMV kernels (odd ne01) (#25640)
00fa7cb Hongqiang Wang committed 4 months ago
opencl: avoid the vec path in GEMV for unaligned row stride (#25671)
a4ce259 Hongqiang Wang committed 4 months ago
server : refactor prompt cache state ownership (#25649)
bf2c86d Georgi Gerganov committed 4 months ago
webui: parse effective-parameter sizes (E2B, E4B) as params (#25529)
dfba90d Emanuil Rusev committed 4 months ago
opencl: fix a dp4a bug for devices where cl_khr_integer_dot_product is unavailable (#25639)
00e79f6 Hongqiang Wang committed 4 months ago
ui: fix mcp panel for toggle + timeout + proxy + ON/OFF state (#25631)
17a05e4 Pascal committed 4 months ago