Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
qwen38: int8 trunk with integer dot products, vector FP8 expert kernel
0137b8f JustVugg committed 2 months ago
fix(qwen38): merge dev while preserving prefix and dot-product dependencies
e958c0e bokiko committed 2 months ago
Merge pull request #1665 from JustVugg/release/1.12.1-notes
87eb980 Vincenzo Fornaro committed 2 months ago
Merge branch 'dev' into feat/windows-xdna-shared-expert-lane
b2e72ef Kenneth-Javier Ortigoza committed 2 months ago
changelog: 1.12.1 counts 32 pull requests, dated the day of the tag
3c164f6 JustVugg committed 2 months ago
Merge pull request #1664 from JustVugg/perf/qwen36-dense-idot
391168c Vincenzo Fornaro committed 2 months ago
Merge pull request #1662 from JustVugg/feat/systemone-jev-api
24dad80 Vincenzo Fornaro committed 2 months ago
qwen36: integer dot products for the dense trunk and the routed experts
dfec3a4 JustVugg committed 2 months ago
fix(olmoe): use the shared RAM probe, add a plausibility floor, warn on degenerate cache
cde727a Cometbuster4969 committed 2 months ago
Merge remote-tracking branch 'origin/dev' into feat/systemone-jev-api
2a61e58 JustVugg committed 2 months ago
Merge pull request #1660 from JustVugg/release/1.12.1
66ffa1d Vincenzo Fornaro committed 2 months ago
gateway: POST /v1/systemone, the request and reply of Jev, served by the brio channel
48c974d JustVugg committed 2 months ago
Merge pull request #1657 from JustVugg/feat/qwen36-trunk-vram
3b2ebb3 Vincenzo Fornaro committed 2 months ago
fix(comment): stale "qdw entry" reference after the g_qdw removal
f9674b6 Jaapjan Tinbergen committed 2 months ago
qwen36: measure the trunk placement at startup, withdraw it when the GPU loses
0e84b37 JustVugg committed 2 months ago
Merge pull request #1659 from JustVugg/fix/serve-max-tokens-ceiling
12e7101 Vincenzo Fornaro committed 2 months ago
qwen36, qwen38: max_tokens is a ceiling, not a target (#1641)
5c6a318 JustVugg committed 2 months ago
Merge pull request #1656 from JustVugg/fix/tok-gpt2-family
4d96d7c Vincenzo Fornaro committed 2 months ago
Merge pull request #1658 from JustVugg/fix/test-mem-available-two-reads
472450f Vincenzo Fornaro committed 2 months ago
tests: two reads of available memory may differ, compare within a quarter of a GB
6719b2c JustVugg committed 2 months ago
qwen36: offer the rest of the dense trunk to the VRAM placer (dnout, attnproj, shexp)
6203a14 JustVugg committed 2 months ago
Merge pull request #1655 from JustVugg/feat/deepseek-v4-brio
2edf748 Vincenzo Fornaro committed 2 months ago
tok.h: the GPT-2 pre-tokenizer family, so OLMoE tokenizes like HF
26e000b JustVugg committed 2 months ago
Merge pull request #1654 from JustVugg/fix/qwen36-special-after-punct
d99eb3f Vincenzo Fornaro committed 2 months ago
deepseek_v4: the numeric channel (SUBMIT logprobs=k, pin=1), so brio works on the ninth engine (#1648)
6f152cd JustVugg committed 2 months ago
qwen36: split added tokens before the pre-tokenizer, and the three whitespace rules (#1653)
682f62d JustVugg committed 2 months ago
fix: resolve static analysis bugs in coli, autotune, family_registry, and convert_qwen36
be958c1 Suraj committed 2 months ago