Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1320 from Avicennasis/fix/darwin-libomp-prefix-fallback
a68ded5 Vincenzo Fornaro committed 2 months ago
Merge pull request #1322 from configurowebmax/fix/doctor-cpu-only-plan
1fa47ff Vincenzo Fornaro committed 2 months ago
Merge pull request #1346 from SyedYousufFaizan/fix/metal-fp8-exact-decode
bba8aea Vincenzo Fornaro committed 2 months ago
fix(metal): bit-exact fp8-e4m3 decode via ieee754 exponent bitcast
abe3ca8 SyedYousufFaizan committed 2 months ago
fix(qwen36): keep int8 expert weights the tier still points at (#1341)
d937ad2 Courtney Andrew Richardson committed 2 months ago
fix(qwen36): wake cv_take waiters on tier shutdown (#1340)
246ce2c Courtney Andrew Richardson committed 2 months ago
fix(qwen36): size the tier's input replicas per device (#1339)
b5f6f1a Courtney Andrew Richardson committed 2 months ago
Merge pull request #1336 from JustVugg/fix/serve-cancel-midturn
f58a267 Vincenzo Fornaro committed 2 months ago
Merge remote-tracking branch 'origin/dev' into fix/serve-cancel-midturn
9afe79b JustVugg committed 2 months ago
perf(kimi): stop paying for a DSA indexer nothing reads (#1335)
9b13d3a Vincenzo Fornaro committed 2 months ago
fix(qwen36): the VRAM tier now promotes int8 experts instead of reserving for nothing (#1334)
002bbc9 Vincenzo Fornaro committed 2 months ago
fix(serve): on Windows a broken pipe is ready, as select() already says
6736c61 JustVugg committed 2 months ago
test(serve): build the poll test on MinGW so Windows exercises its own branch
83155be JustVugg committed 2 months ago
test(serve): build the disconnect scenario instead of hoping for it (#1329)
e1efc68 Vincenzo Fornaro committed 2 months ago
fix(glm53): --no-think is the template's lowest level, not a shape of ours (#1327)
53c57f1 Vincenzo Fornaro committed 2 months ago
st.h: shared per-shard mapping primitive, opt-in via COLI_MAP_EXPERTS=1
eabeb9a Ronald Klomp committed 2 months ago
fix(doctor): omit GPU plan for CPU-only engine
700151d Mauricio Sevilla Britto committed 2 months ago
fix(qwen36): read tokenizer.json merges in string and pair spellings
cfd5106 Léon Avic Simmons committed 2 months ago
fix(build): fall back to the Homebrew prefixes when brew is off the PATH
f01621f Léon Avic Simmons committed 2 months ago
improved c/deepseek_v4.c to allow the run of deepseek_v4_flash-0731-reap-150b (#1310)
387653f Matteo Deliperi committed 2 months ago
perf(dsv4): compute four output rows per pass in coli_fp4_matvec_rows16_order (#1314)
f59199e Brian committed 2 months ago
fix(dsv4): report real CPU and memory in HWINFO on macOS (#1308)
5247f14 Brian committed 2 months ago
fix(loader): a missing core tensor explains itself on every fatal path (#1318)
eccfb69 Vincenzo Fornaro committed 2 months ago
docs(benchmarks): node1 GLM-5.2 datapoints and a paired M4 Pro cap comparison (#1272)
2d1e2aa Peter Groom committed 2 months ago
fix(convert): refuse a checkpoint this converter cannot serve (#1305)
a642a8b Vincenzo Fornaro committed 2 months ago