LLM inference in C/C++
C++ ggml
Commits on master
cuda : disables launch_fattn PDL enrollment due to compiler bug (#23825)
241cbd4 Andreas Kieslinger committed 5 months ago
meta : Add missing `buffer` set in allreduce fallback !COMPUTE clear (#23480)
33c718d Matt Corallo committed 5 months ago
hexagon: basic/generic op fusion support and RMS_NORM+MUL fusion (#23835)
19e92c3 Max Krasnyansky committed 5 months ago
opencl: move backend info printing into its own function (#23702)
408ae2b lhez committed 5 months ago
ci : releases use Github-hosted builds for the UI (#23823)
445b7ce Georgi Gerganov committed 5 months ago
ci : change Vulkan builds to Release to reduce ccache (#23820)
dd15579 Georgi Gerganov committed 5 months ago
arg: Add LLAMA_ARG_API_KEY_FILE environment variable for --api-key-file (#23167)
7fb1e70 Mikolaj Kucharski committed 5 months ago
test-llama-archs: fix table format [no release] (#23810)
d374e71 Johannes Gäßler committed 5 months ago
ggml: auto apply iGPU flag CUDA/HIP if integrated device (#23007)
30af6e2 fl0rianr committed 5 months ago
mmvq Optim: add MMVQ_PARAMETERS_TURING(mmvq_parameter_table_id) for … (#23729)
d7be461 redfox committed 5 months ago
CUDA: route batch>=4 quantized matmul to MMQ on AMD MFMA hardware (#23227)
bc81d47 Jaden_Mach committed 5 months ago
server: minor tweaks to use more cpp features (#23785)
0b24686 Funtowicz Morgan committed 5 months ago
server, ui : Add support for HTTP ETags in llama-server (#23701)
d205df6 Markus Tavenrath committed 5 months ago
cuda : fix KQ mask offset integer overflow in fattn MMA kernel (#23610)
09e7b76 fairydreaming committed 5 months ago
perplexity : fix format specifier in LOG_ERR (#23788)
48e7eae Adrien Gallouët committed 5 months ago