Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
fix(serve): a tool whose function is not an object answered 500
c64c8d6 kevin9327 committed 1 month ago
fix(glm53): close the Vulkan status fprintf after the ternary
1165bcd Thomas Wittchen committed 1 month ago
fix(v4): rebuild the unit objects when the build flags change
4e8ef20 Marcello Costagliola committed 1 month ago
Merge pull request #1706 from JustVugg/release/1.12.1-notes-1705
30c70ad Vincenzo Fornaro committed 1 month ago
Merge pull request #1705 from ZacharyZcR/feat/three-engine-serving-baseline
926be8e Vincenzo Fornaro committed 1 month ago
Merge pull request #1704 from JustVugg/release/1.12.1-notes-wave4
5d6671f Vincenzo Fornaro committed 1 month ago
Merge pull request #1693 from namespaceMarcello/fix/olmoe-inflight-dedup
aa2d295 Vincenzo Fornaro committed 1 month ago
Merge pull request #1697 from kevin9327/fix/web-reasoning-stream-dropped
4399eef Vincenzo Fornaro committed 1 month ago
Merge pull request #1695 from namespaceMarcello/fix/qwen36-echo-guard
bf7c767 Vincenzo Fornaro committed 1 month ago
fix(ablate): derive the per-item token cap from CTX, bound all three allocations
4925883 monotophic committed 2 months ago
ci(release): package check_ablate_evidence.py, then prove it actually imports
31d7227 monotophic committed 2 months ago
perf(ablate): stop re-scanning for an already-full top-k slot buffer
95b70e5 monotophic committed 2 months ago
feat(engine): add a logit-row status layer, evidence digests, and a checked ablation scoring mode
c3d0805 monotophic committed 2 months ago
exact verify (#689): document the quantised-KV fallback and the untested near-tie flip
a13b3cd Ry Bruscoe committed 1 month ago
Opt-in exact verify mode for speculative batches (COLI_EXACT_VERIFY=1, #689)
049d444 Ry Bruscoe committed 2 months ago
docs(fp8): FORMATS.md names symbols, so stop promising line numbers
4159a23 monotophic committed 1 month ago
docs(fp8): correct stale anchors, an unreachable test rule and a wrong test comment
b95b942 monotophic committed 1 month ago
feat(server): logprobs, echo and top_logprobs on the two completion endpoints
42f8d80 monotophic committed 1 month ago
refactor(server): report what each text stage deleted as a span map
68cb5e6 monotophic committed 1 month ago
Merge pull request #1402 from enitimeago/feat/glm53-assistant-continuation
b02b801 Vincenzo Fornaro committed 1 month ago
Merge pull request #1694 from JustVugg/release/1.12.1-notes-1402
6497713 Vincenzo Fornaro committed 1 month ago
qwen36: keep the prefill echo out of the segment build
8212a39 Marcello Costagliola committed 1 month ago