LLM inference in C/C++
C++ ggml
Commits on master
ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking (#29927)
9d1878a thelittlefireman committed 1 month ago
ui : Discover & Download Models from HF Hub (#29584)
c0fd786 Aleksander Grygier committed 1 month ago
tests: give the random model generator a per-tensor filler table (#30674)
10b8b75 Pascal committed 1 month ago
cuda : fix -Wtautological-compare in fattn vec dispatch (#30519)
9d1e036 Adrien Gallouët committed 1 month ago
sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder (#30374)
f74e2fa Marcus committed 1 month ago
ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620)
6420cf5 Zack Li committed 1 month ago
opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices (#30691)
2f2763d Shawn Gu committed 1 month ago
common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client (#21752)
7182ef7 texasich committed 1 month ago
convert : respect Mistral format for DeepseekV2Model (#30669)
2a66703 aiqubits committed 1 month ago
convert : speed up model endian conversion and add more datatype support (#30647)
b84ea5c Aaron Teo committed 1 month ago
[SYCL]: fix memory errors in mul_mat, split buffer, host pool (#29889)
a30fb49 Michael Taylor committed 1 month ago
gguf-py : add numpy Q1_0 and Q2_0 de/quantization support (#30630)
495c580 Sh1m13 committed 1 month ago
CUDA: HIP: rms_norm: add variants that load float4 at a time and store x in registers (#29720)
623c163 uvos committed 1 month ago
common : read ngram cache parts by reference instead of copying (#30061)
13b8048 hayder committed 1 month ago
convert : support text-only checkpoints in pplx-decider (#30575)
0f43762 FINAL-Bench committed 1 month ago