Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
convert: default to group-scaled int4 (gs64) — close the #455 tracking item and stop steering new users into the per-row trap
21cbc29 ZacharyZcR committed 4 months ago
web: ship the Expert Atlas data — Brain hover has fetched /experts.json since #126, but the file never shipped
c5f7c91 ZacharyZcR committed 4 months ago
install: ship version.py to libexec and make the launcher's version import survive its absence (#575)
9491770 ZacharyZcR committed 4 months ago
serve: emit finish_reason=length on real cap-hits, not just at admission
db4ed61 recviking committed 4 months ago
Merge pull request #572 from JustVugg/fix/serve-all-stops
5724dae Vincenzo Fornaro committed 4 months ago
review: pass gs=0 in #488's large-batch gemm calls; document the falloc->qalloc trade-off
649b552 monotophic committed 4 months ago
metal-test: fix two blind spots in run_attn_grouped (review round 1)
991eba4 monotophic committed 4 months ago
colibri: fix fmt=4 scale-buffer allocator (falloc -> qalloc)
98eb881 monotophic committed 4 months ago
metal: fmt=4 grouped-int4 GEMV support (mm_gemv + host plumbing) + tests
5407494 monotophic committed 4 months ago
Merge pull request #568 from winklemad/fix/anthropic-thinking-blocks-c2
75a7374 Vincenzo Fornaro committed 4 months ago
Merge pull request #569 from NeuralNotwerk/draft-default-1
02984a8 Vincenzo Fornaro committed 4 months ago
serve: COLI_SERVE_ALL_STOPS re-arms the full stop set (#549)
a391eee JustVugg committed 4 months ago
Merge pull request #556 from bherald/perf/554-causal-attention-static-1
8182858 Vincenzo Fornaro committed 4 months ago
docs/tools: document PILOT_EVICT_GUARD and add CLI overrides to test_olmoe_real
fbdb109 Egon Ruiter committed 4 months ago
Merge pull request #548 from ZacharyZcR/feat/dsa-gather-cuda
defd424 Vincenzo Fornaro committed 4 months ago
Merge pull request #546 from ZacharyZcR/feat/prefill-chunk
bdfa744 Vincenzo Fornaro committed 4 months ago
Merge pull request #545 from ZacharyZcR/feat/kv-cross-slot-adopt
8ffdf96 Vincenzo Fornaro committed 4 months ago
Merge pull request #547 from ZacharyZcR/perf/silu-fused-epilogue
a6c2cc4 Vincenzo Fornaro committed 4 months ago
Merge pull request #565 from mxkev1n/fix/expert-budget-zero-and-weights
7d3c1d0 Vincenzo Fornaro committed 4 months ago
Merge pull request #562 from mxkev1n/fix/pipe2-stale-residual
c0e5b43 Vincenzo Fornaro committed 4 months ago
Merge pull request #566 from mxkev1n/fix/group-async-double-work
db4b820 Vincenzo Fornaro committed 4 months ago
Merge pull request #563 from mxkev1n/fix/router-nan-oob
b2d56b3 Vincenzo Fornaro committed 4 months ago
Merge pull request #564 from mxkev1n/fix/moe-unchecked-alloc
2363917 Vincenzo Fornaro committed 4 months ago
Merge pull request #539 from nbeerbower/feat/inkling-serve-upstream
5fefde0 Vincenzo Fornaro committed 4 months ago
Merge pull request #540 from dqiaok/fix/cuda-env-test-colibri-rename
f17c9e4 Vincenzo Fornaro committed 4 months ago
Merge pull request #538 from mcollinswisc/rm_google_redirects
f6b1904 Vincenzo Fornaro committed 4 months ago