LLM inference in C/C++
C++ ggml
Commits on master
sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568)
31558db Neo Zhang committed 3 months ago
sycl : update guide Q&A and script for device setting (#26442)
c1f4109 Neo Zhang committed 3 months ago
sycl : fix error Error OP FLASH_ATTN_EXT on arc770 (#26441)
eef5f3e Neo Zhang committed 3 months ago
sycl : enhance OP set_rows to support all missed data types (#26515)
c074cb3 Neo Zhang committed 3 months ago
cuda: fix warnings for unused variable/function (#26688)
5b87ed3 David Friehs committed 3 months ago
metal : avoid `threadgroup` matrix array instantiation in kernel_lightning_indexer (#26646)
e40bf88 JamePeng committed 3 months ago
convert : fix DeepseekV4 rope parameters with transformers 5.x (#26673)
eb5667a Sigbjørn Skjæret committed 3 months ago
model-loader : fix quantized reshaped tensor strides (#26672)
3db4ff8 Georgi Gerganov committed 3 months ago
convert : accept "ExaoneMoeForCausalLM" arch spelling (#26660)
e700bfb Csaba Kecskemeti committed 3 months ago
model-conversion : add --model-name to conversion scripts (#26665)
9de0fcf Daniel Bevenius committed 3 months ago
vulkan: fix submission batching size, add debug tools for diagnosing causes of DeviceLost drivers errors (#26371)
803b7fc Ruben Ortlam committed 3 months ago
tests: re-enable MiniMax M3 in `test-llama-archs` (#26633)
360e134 Niklas Wenzel committed 3 months ago
mtmd: Unlimited-OCR fix max_tiles, setting in converter (#25614)
b06aa77 Saba Fallah committed 3 months ago
grammar : degrade max repetition >= 2000 to unbounded (#26613)
cd0fa60 Aldehir Rojas committed 3 months ago
mtmd: support multi-row batching for deepseek-ocr (#26154)
717dad5 Xuan-Son Nguyen committed 3 months ago
security : clarify about AI-generated reports (#26579)
9303cdd Xuan-Son Nguyen committed 3 months ago
server: Adding spec-decode counters to /metrics endpoint (#26389)
a035a88 Bhavik Sharda committed 3 months ago
convert: Add endianness conversion for Q1 and TQ2 quantizations (#26618)
020760a Andreas Krebbel committed 3 months ago
ui: show generation statistics by default in chat settings (#26624)
3e3a7a4 Aleksander Grygier committed 3 months ago
build : remove GGML_METAL_USE_BF16 from all build scripts (#26604)
d52ec04 Niklas Wenzel committed 3 months ago
ui: Update vulnerable packages + cleanup Storybook config (#26607)
e031d95 Aleksander Grygier committed 3 months ago