LLM inference in C/C++
C++ ggml
Commits on master
ggml-cpu: add AVX2 optimization for nvfp4 dot product and use UE4M3 LUT (#23961)
6dbc117 ragz4125 committed 4 months ago
ui Prevent tool messages from incorrectly appending to other conversations (#25177)
9d88e7c Aleksander Grygier committed 4 months ago
ui: Remove PWA navigate fallback to prevent caching API endpoint requests (#25174)
7af4279 Aleksander Grygier committed 4 months ago
cuda : prevent integer truncation and overflow errors when using KQ mask strides in flash_attn_mask_to_KV_max kernel (#24945)
0eca4d4 fairydreaming committed 4 months ago
common,server: handle bracketed IPv6 literals in URL authority (#25140)
799fcc0 Pascal committed 4 months ago
CUDA: fix get_rows_back for tables with more than 65535 rows (grid-y clamp + stride) (#25103)
931eb37 Matt Jallo committed 4 months ago
HIP: use hipBLAS for dense prefill on gfx900, keep MMQ for MoE (#24588)
d9df110 zduford committed 4 months ago
Revert "sched : reintroduce less synchronizations during split compute (#20793)" (#25138)
86b9470 Oliver Simons committed 4 months ago
common : dedup preset and cached model entries in /v1/models (#25131)
6f4f53f Adrien Gallouët committed 4 months ago
vulkan: use flops instead of weight tensor size for submission heuristic (#25005)
25a1d63 Ruben Ortlam committed 4 months ago
tools/ui: restore Tailwind scanning in ignored worktrees (#24879)
6cb18b2 seryogakovalyov committed 4 months ago
Revert "ui: fix accessibility for hover-gated interactive elements assisted by claude(in debugging and tests) (#24727)" (#25098)
dbdaece Aleksander Grygier committed 4 months ago
[CUDA] Added a cudaMemcpy2DAsync fast path to ggml_cuda_cpy (#25057)
0ed235e Gaurav Garg committed 4 months ago