Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1209 from JustVugg/perf/tune-ram-cache-cap
8696e2a Vincenzo Fornaro committed 2 months ago
Merge pull request #1208 from JustVugg/perf/k3-adaptive-vk-fill
15e4b4e Vincenzo Fornaro committed 2 months ago
Merge pull request #1155 from ZacharyZcR/feat/kv8-flash-aware
ae67e8d Vincenzo Fornaro committed 2 months ago
Merge pull request #1201 from JustVugg/perf/tune-placement-sweep
ad53e35 Vincenzo Fornaro committed 2 months ago
Merge pull request #1207 from ZacharyZcR/feat/kv8-scale128
21c2ec6 Vincenzo Fornaro committed 2 months ago
Merge pull request #1205 from ZacharyZcR/tools/594-rans-census
eec2a43 Vincenzo Fornaro committed 2 months ago
Merge pull request #1206 from ZacharyZcR/fix/1042-win11-label
38daecf Vincenzo Fornaro committed 2 months ago
perf(k3): measured adaptive expert-tier fill (K3_VK_UP=auto)
d8ed374 JustVugg committed 2 months ago
KV8: grouped latent scales (KV8_GS), FlashMLA's fp8 cache geometry
aa273d3 ZacharyZcR committed 2 months ago
fix(tools): label Windows 11 correctly in datapoint machine_info
ffa4c3b ZacharyZcR committed 2 months ago
tools: per-record layout census for rANS containers (#594)
e9455df ZacharyZcR committed 3 months ago
attn: KV8/KV_TQ awareness for the flash and gather kvb arms
80b11dc ZacharyZcR committed 3 months ago
Merge pull request #1203 from JustVugg/perf/dsv4-hybrid-overlap
6896e5f Vincenzo Fornaro committed 3 months ago
Merge pull request #1202 from JustVugg/perf/dsv4-hybrid-expert-split
929e99b Vincenzo Fornaro committed 3 months ago
perf(dsv4): overlap hybrid fill DMA with host compute; pin slabs on Linux
605d98b JustVugg committed 3 months ago
perf(dsv4): hybrid GPU/CPU expert split on decode misses (opt-in)
bc44ff0 JustVugg committed 3 months ago
Merge pull request #1198 from Mr-Neutr0n/fix/clamp-ram-gb-to-mem-available
4032232 Vincenzo Fornaro committed 3 months ago
Merge pull request #1197 from JustVugg/perf/inkling-shared-prefill-batch
44d1970 Vincenzo Fornaro committed 3 months ago
Merge pull request #1193 from JustVugg/perf/expert-cache-index-models-v2
1d37123 Vincenzo Fornaro committed 3 months ago
docs(degrade-zero): document warm-cache assumption as known limitation
9a39018 Kritika Garg committed 3 months ago
Merge pull request #1200 from JustVugg/perf/qwen-shared-prefill-batch
3d0bab1 Vincenzo Fornaro committed 3 months ago
Merge pull request #1182 from Unknown-Findout/fix/datapoint-evict-space-check
0e1a14f Vincenzo Fornaro committed 3 months ago
Merge pull request #1196 from JustVugg/fix/tune-refuse-non-glm
a3d11c2 Vincenzo Fornaro committed 3 months ago
Merge pull request #1195 from JustVugg/fix/tune-cap-resolution
b0c6805 Vincenzo Fornaro committed 3 months ago
Merge pull request #1194 from JustVugg/feat/k3-ckpt-disk
8107f53 Vincenzo Fornaro committed 3 months ago