Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Commits on main

fix: scope implicit stops to GLM chat
bill committed 4 months ago
74326b9
fix: stop chat at implicit role boundaries
bill committed 4 months ago
83fc0a7
feat: support patient native stop handling
Bill Herald committed 4 months ago
37c8fac
feat: support OpenAI stop sequences
Bill Herald committed 4 months ago
cc5df23
Merge pull request #572 from JustVugg/fix/serve-all-stops
Vincenzo Fornaro committed 4 months ago
5724dae
98eb881
Merge pull request #569 from NeuralNotwerk/draft-default-1
Vincenzo Fornaro committed 4 months ago
02984a8
Merge pull request #548 from ZacharyZcR/feat/dsa-gather-cuda
Vincenzo Fornaro committed 4 months ago
defd424
Merge pull request #546 from ZacharyZcR/feat/prefill-chunk
Vincenzo Fornaro committed 4 months ago
bdfa744
8ffdf96
a6c2cc4
c0e5b43
db4b820
Merge pull request #563 from mxkev1n/fix/router-nan-oob
Vincenzo Fornaro committed 4 months ago
b2d56b3
Merge pull request #564 from mxkev1n/fix/moe-unchecked-alloc
Vincenzo Fornaro committed 4 months ago
2363917
5fefde0
f17c9e4
f6b1904