Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #901 from JustVugg/fix/tune-arch-guard
47a749d Vincenzo Fornaro committed 3 months ago
Merge pull request #896 from JustVugg/feat/web-markdown
6ff5765 Vincenzo Fornaro committed 3 months ago
Merge pull request #895 from JustVugg/ci/vulkan-lavapipe
febdba8 Vincenzo Fornaro committed 3 months ago
Merge pull request #899 from JustVugg/diag/pin-budget
549f9f1 Vincenzo Fornaro committed 3 months ago
feat(moe): add DEGRADE_ZERO miss-slot zero-fill policy (issue #865)
dc2f633 Kritika Garg committed 3 months ago
tune: dispatch per architecture, and give every engine a throughput line (#898)
bac9bd7 JustVugg committed 3 months ago
plan: print every placement term, and a switch to A/B the slab shrink (#885)
591eace JustVugg committed 3 months ago
web: render markdown in the chat pane, and stop clamping it to 1024 tokens
6e90242 JustVugg committed 3 months ago
Merge pull request #587 from RDouglasSharp/metal-attn-fmt4
98cab3b Vincenzo Fornaro committed 3 months ago
Merge pull request #886 from shujaanazhar/olmoe-serve-mode
77619b0 Vincenzo Fornaro committed 3 months ago
test: reconcile test_backend_metal with post-rebase dev (qgs=0 at fp8/e8 call sites, rename local fp8_nblk to avoid quant.h clash)
184b3a5 Doug Sharp committed 3 months ago
metal: COLI_METAL_MOE_EXACT covers all fmt=4 GPU paths (attn/layer decode + dense)
36a555a Doug Sharp committed 3 months ago
metal: COLI_METAL_MOE_EXACT — opt-in CPU-exact fmt=4 experts, fast batched default (#587 gate-2)
4427dab Doug Sharp committed 3 months ago
probe: env-gated top-2 logit-gap dump at greedy selection (COLI_LOGIT_GAP)
32dc3f9 monotophic committed 3 months ago
engine: kv_b fmt-gate notice, re-derived for post-#587 two-format gate (r3)
3be4753 monotophic committed 4 months ago
moe: gs-consistency guard for fmt=4 GPU batches (review F2)
95fdfda monotophic committed 4 months ago
metal: adapt #587's fmt=4 kv_b + moe_block coverage onto our #457 base
19d4004 monotophic committed 4 months ago
feat(olmoe): implement the SERVE protocol so coli chat/web/serve work
5684d3f shujaanazhar committed 3 months ago
Merge pull request #880 from JustVugg/test/release-dispatch
8d06bd1 Vincenzo Fornaro committed 3 months ago
Merge remote-tracking branch 'origin/dev' into test/release-dispatch
f02a335 JustVugg committed 3 months ago
Merge pull request #881 from terrizoaguimor/fix/olmoe-run-dispatch
207532f Vincenzo Fornaro committed 3 months ago
Merge pull request #875 from waizuichougou/fix/deepseek-v4-dspark-cache-profile
08a499e Vincenzo Fornaro committed 3 months ago