Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
vk_tier: vkt_begin_forward, for a model with a single MoE layer
87a9391 JustVugg committed 1 month ago
vk_tier: an engine's own cap on the experts it keeps (VktConfig.max_experts)
c53218e JustVugg committed 1 month ago
Merge pull request #1840 from JustVugg/feat/vulkan-expert-tier
ef51b80 Vincenzo Fornaro committed 1 month ago
ci: the qwen engines' expert tier under ASan and UBSan on Lavapipe
f258cff JustVugg committed 1 month ago
Merge remote-tracking branch 'origin/dev' into feat/vulkan-expert-tier
ffa2bd2 JustVugg committed 1 month ago
docs: the dense matrices' default place, and COLI_VK_TIER_SYNC
0e1bc76 JustVugg committed 1 month ago
tests: the qwen family with the trunk on the device and on the CPU
193d232 JustVugg committed 1 month ago
vk_tier: COLI_VK_TIER_SYNC; a small model's pool is the size of its experts
b960df9 JustVugg committed 1 month ago
vulkan: one rule for where the dense matrices go; on a shared-RAM device with the tier, the CPU
be77bf3 JustVugg committed 1 month ago
docs: the Vulkan routed-expert tier, measured on a Radeon 780M
3ebbd20 JustVugg committed 1 month ago
tests: the qwen engines under ASan and UBSan with the tier on
2574038 JustVugg committed 1 month ago
qwen36, qwen38: the Vulkan device goes down at exit, after the tier
05d2422 JustVugg committed 1 month ago
vk_tier: look for a victim before staging a promotion, take it after
642ff0a JustVugg committed 1 month ago
Merge pull request #1839 from JustVugg/fix/qwenimage-cli-windows-teardown
d20a3f0 Vincenzo Fornaro committed 1 month ago
vk_tier: a forward is a step back in the layer index; the uploader waits for room
aebd65b JustVugg committed 1 month ago
tests: the qwen family on Lavapipe with the routed-expert tier on
502c92e JustVugg committed 1 month ago
expert_ffn: the layer's expert outputs apart from their rank-order sum
1f479ab JustVugg committed 1 month ago
vulkan: weight memory that is given back, and an async expert batch
971d0b6 JustVugg committed 1 month ago
tests: stop the qwen-image gateway's whole process tree on Windows
bba4338 JustVugg committed 1 month ago
Merge pull request #1837 from JustVugg/feat/prefill-batch-matmuls
8001d05 Vincenzo Fornaro committed 1 month ago