LLM inference in C/C++
C++ ggml
Commits on master
ggml : add support for CPU f16->f16 GGML_OP_SET_ROWS (#25344)
68a521b fairydreaming committed 4 months ago
Add Q2_0 quantization: type definition and CPU backend (#24448)
bec4772 Pasha Khosravi committed 4 months ago
server : add timings and progress to /responses API stream (#25348)
5eca4e3 Thomas LECONTE committed 4 months ago
common : add missing <fstream> include in common.h (#25220)
c1a411f zhangrunda committed 4 months ago
ggml-hip : add -fno-finite-math-only alongside -ffast-math (#25373)
33ca0dc asf0 committed 4 months ago
[SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236)
55edb2d Neo Zhang committed 4 months ago
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063)
d209086 Todd Malsbary committed 4 months ago
[SYCL] fix unsupport ACC UT cases for noncontiguous (#25124)
95e5254 Neo Zhang committed 4 months ago
sycl : rename the env vars from "disable" to "enable" (#25042)
26145b3 Neo Zhang committed 4 months ago
speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936)
defa95c o7si committed 4 months ago
vulkan : check src0 type in GGML_OP_SET_ROWS to avoid failures due to unimplemented f16 support (#25351)
a8cfdbb fairydreaming committed 4 months ago
opencl: general flash attention decode performance optimizations (#25366)
6f8895f Hongqiang Wang committed 4 months ago
common: Set optimal default thread count for ppc ( linux as well as AIX) (#25237)
ee445f9 shalinib-ibm committed 4 months ago
server: fix deadlock in load_models() when erasing a finished download (#25358)
9abce74 Pascal committed 4 months ago
CUDA: extend K-type validation to V-types for flash attention (#24403)
cb295bf Alexey Kopytko committed 4 months ago
server: temporary skip model downloading API test (#25355)
bfdf581 Xuan-Son Nguyen committed 4 months ago