Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
test(cuda): verify attention prefill continuation and KV state
48795d4 ZacharyZcR committed 1 month ago
build: avoid conflicts between CUDA benchmark and test targets
44150b6 ZacharyZcR committed 1 month ago
feat(bench): separate warmup from measured serving requests
282b743 ZacharyZcR committed 1 month ago
fix(serve): check cancellation before admitting an available slot
24788ac ZacharyZcR committed 1 month ago
test(qwen36): verify split prefill state and continued decode
35c39ca ZacharyZcR committed 1 month ago
test(qwen36): verify each attention projection failure with poisoned output
f17038d ZacharyZcR committed 1 month ago
fix(bench): reject malformed choices and unsuccessful stream endings
2c442da ZacharyZcR committed 1 month ago
fix(cuda): preserve active contexts on repeated initialization
f0f8dfc ZacharyZcR committed 1 month ago
feat(bench): schedule fixed-rate arrivals and account for client backlog
df6cf53 ZacharyZcR committed 1 month ago
feat(serve): measure first output and engine call durations
6567e76 ZacharyZcR committed 1 month ago