LLM inference in C/C++
C++ ggml
Commits on master
ggml : fix c++ odr by properly using GGML_COMMON_DECL_CPP (#29504)
da89bb3 XZiar committed 1 month ago
ggml : accumulate f16 dot products in f32 on AVX512-FP16 (#29545)
284153e Adrien Gallouët committed 1 month ago
ggml : require input tensors to be GGML_OP_NONE (#29647)
b5cf8ce Georgi Gerganov committed 1 month ago
server : remove the built-in UI's service worker when the UI is not served (#29565)
ba0ba54 Emanuil Rusev committed 1 month ago
tests : adjust server string regex to also match m2 utlra results (#29648)
c85b92c Georgi Gerganov committed 1 month ago
ggml : collect all input tensors into graph_inputs (#29634)
8019dc5 Georgi Gerganov committed 1 month ago
musa: build the docker image and CI container from the MUSA SDK images (#29624)
18b74ff R0CKSTAR committed 1 month ago
vulkan : reuse descriptor sets when bindings are constant (#29280)
0bc845d Andrew Lee committed 1 month ago
chat : fix Muse Glimmer ignoring response_format json_schema with --jinja (#29615)
139997d vaibhavdedhia committed 1 month ago
tests : skip pytest workers when PYTEST_WORKERS=1 (#29610)
46e17a6 Georgi Gerganov committed 1 month ago
mtmd: fix GCC 15 stringop-overflow in decode_embd_batch (#29607)
526c43b Pascal committed 1 month ago
hex-scripts: show trace events smaller than 100nsec in perfetto (#29614)
1c47294 Trivikram Reddy committed 1 month ago
server : support typed content (vision/audio/video) input for /v1/embeddings endpoint (#29556)
680a036 Tim Wang committed 1 month ago
ggml-openvino: mark unaligned batch-stride views unsupported (#29603)
14ebbd5 Ravi Panchumarthy committed 1 month ago
batch: migrate speculative, mtmd and server to batch_ext (#29385)
f1ea206 Xuan-Son Nguyen committed 1 month ago