Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
Merge pull request #1094 from JustVugg/perf/k1b-gs-idot
ad79236 Vincenzo Fornaro committed 3 months ago
perf(colibri): K1b β grouped planar IDOT for gs64 containers, opt-in (IDOT_GS=1)
88a4a82 JustVugg committed 3 months ago
Merge pull request #713 from kreuzzelg/qwen36-cuda-tier
7d1d19e Vincenzo Fornaro committed 3 months ago
Merge pull request #1090 from terrizoaguimor/refactor/serve-codec-kimi
8c0aa8c Vincenzo Fornaro committed 3 months ago
feat(qwen36): CUDA VRAM expert tier β heat-based placement across GPUs via the shared CUDA backend
34deee3 kreuzzelg committed 3 months ago
Merge pull request #712 from kreuzzelg/qwen36-engine
6784aa8 Vincenzo Fornaro committed 3 months ago
perf(colibri): K3-lite β parallel silu and the down-side half of the #1071 hoist
921ce46 JustVugg committed 3 months ago
ci(qwen36-tiny-check): the sanitizer step asserts memory safety, not tokens
046842e kreuzzelg committed 3 months ago
qwen36: drop the packed int4 buffers this engine never reads
67c7490 kreuzzelg committed 3 months ago
Merge pull request #1056 from dcutugno/pr/v4-deepgemm
cb8524b Vincenzo Fornaro committed 3 months ago
fix(qwen36): refuse a container whose tensors do not match the config (A3+A4)
ca96e12 kreuzzelg committed 3 months ago
fix(qwen36): validate every config dimension before the forward pass uses it (A1+A2)
4e0dc88 kreuzzelg committed 3 months ago
ci(qwen36-tiny-check): make the gate deterministic and actually exact
e84005e kreuzzelg committed 3 months ago
ci: build and ship qwen36, and gate it against a tiny oracle (B5 + B5b)
9ae0c8a kreuzzelg committed 3 months ago
fix(qwen36): expert_get must not evict an in-flight slot (A5)
3252675 kreuzzelg committed 3 months ago
fix(qwen36): heap per-thread attention score rows β 8k context smashed the stack
c30c545 kreuzzelg committed 3 months ago
fix(qwen36): serve EOS ids from the tokenizer, not the 151k-vocab constant
342f104 kreuzzelg committed 3 months ago
fix(qwen36): serve mode must not require an argv prompt file
2ff73e0 kreuzzelg committed 3 months ago
docs(qwen36): user-facing page β prebuilt containers as the easy path
24d3c89 kreuzzelg committed 3 months ago
feat(qwen36): group-scaled int4 (gs64) containers β converter flag + CPU read path
5a0715e kreuzzelg committed 3 months ago
feat(qwen36): make engine drivable by coli + SERVE=1 gateway protocol
9c901b5 minne100 committed 3 months ago
feat(qwen36): add Qwen3.6-35B-A3B engine (CPU): hybrid Gated Attention + Gated DeltaNet + streaming MoE
fc52d73 kreuzzelg committed 3 months ago