Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
kimi_k3: Vulkan tier (K3_VK) β shared experts resident + fill-once MXFP4 routed-expert tier
b61089a Steve Markgraf committed 3 months ago
vulkan: fmt=7 MXFP4 decode (e2m1 nibbles + per-32 group scales) for Kimi K3 experts
d4bf041 Steve Markgraf committed 3 months ago
Merge pull request #650 from CodePrometheus/fix/deprecate-convert-olmoe
ecade07 Vincenzo Fornaro committed 3 months ago
Merge pull request #542 from opxyc/feat/olmoe-chat-mode
74112da Vincenzo Fornaro committed 3 months ago
docs(readme): show that one command line runs any of the three models
3937c00 JustVugg committed 3 months ago
Merge pull request #704 from JustVugg/feat/inkling-topp
ff16fed Vincenzo Fornaro committed 3 months ago
feat(inkling): TOPP β adaptive routed-expert trimming, and report what it trims
8dbadc0 JustVugg committed 3 months ago
vulkan: skip the cross-layer WILLNEED hint for VK-tier-served experts
d60ce64 Steve Markgraf committed 4 months ago
docs: vulkan ReBAR requirement, shader resolution, cross-backend benchmarking notes
ca89f11 Steve Markgraf committed 4 months ago
vulkan: warn when ReBAR leaves VRAM host-invisible; find shaders next to the binary
809a847 Steve Markgraf committed 4 months ago
vulkan: VK_PROF phase split for dev2 issue + worker/join timers
3cba6ce Steve Markgraf committed 4 months ago
vulkan: second-device expert tier (COLI_VK_DEV2) β e.g. an RX 580 beside the primary
f4290b2 Steve Markgraf committed 4 months ago
pilot: multi-worker PILOT_REAL prefetch (PILOT_WORKERS) + #474 guard, rebased on current dev
b5af8a9 cdhdt committed 4 months ago
mirror: stripe O_DIRECT demand reads across all replicas (COLI_MIR_STRIPE)
c048ad8 Steve Markgraf committed 4 months ago
vulkan: preload the dense working set before the tier fill
75175ef Steve Markgraf committed 4 months ago
mirror: N-way multi-SSD read replicas (COLI_MODEL_MIRROR dir list)
2737ee6 Steve Markgraf committed 4 months ago
vulkan: single-submit q-prep chain β q_a+kv_a -> on-GPU rmsnorm -> q_b
b0b7f7b Steve Markgraf committed 4 months ago
vulkan: fmt=4 (grouped int4) decode with #298 per-group scale semantics
4d21d8a Steve Markgraf committed 4 months ago
vulkan: global submit/wait VK_PROF totals across all sync paths
2d13ab4 Steve Markgraf committed 4 months ago
vulkan: VK_PROF phase split for the dense matmul (localize the tier-linear submit tax)
4004392 Steve Markgraf committed 4 months ago
vulkan: suballocate weight tensors from 256 MB arenas β the tier-size submit tax
b055ca9 Steve Markgraf committed 4 months ago
vulkan: VRAM pressure-proofing β priority-classed allocations + budget-capped tier fill
048ccbc Steve Markgraf committed 4 months ago
vulkan: PROF anatomy of the expert block (VK-BLOCK line)
22b26aa Steve Markgraf committed 4 months ago
vulkan: readiness-ordered CPU expert share + kernel-only t_ecpu
da8149d Steve Markgraf committed 4 months ago
fix: expert_load grew a demand parameter in #438 (DISK-CLASS) β classify the VK-path call sites
0341174 Steve Markgraf committed 4 months ago