Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
docs: document full-family fp8 repack output and container declaration
40dd6ec monotophic committed 3 months ago
feat(tools): extend fp8 repack tool to full-family emission
886eacf monotophic committed 3 months ago
test(cuda): host-testable weight_at format predicate + device-side trap coverage
8560724 monotophic committed 3 months ago
Merge pull request #1106 from monotophic/fix/duplicate-tensor-name-refusal
f2f8253 Vincenzo Fornaro committed 3 months ago
Merge pull request #1101 from monotophic/fix/798-loader-guards
c54b28d Vincenzo Fornaro committed 3 months ago
Merge pull request #1096 from terrizoaguimor/refactor/serve-codec-v4
78ea979 Vincenzo Fornaro committed 3 months ago
Merge pull request #1095 from benmaster82/feat/olmoe-planner-1066
c09fb52 Vincenzo Fornaro committed 3 months ago
test(cuda): wire test_fp8_cuda into cuda-test and add independent footprint oracle
9480d88 monotophic committed 3 months ago
Merge pull request #1113 from JustVugg/metal-k3-rebase
ee5610b Vincenzo Fornaro committed 3 months ago
fix(cuda): stop coli_cuda_tensor_bytes from overcounting fmt=8/fmt=6 scale bytes
5823580 monotophic committed 3 months ago
fix(cuda): stop fmt=8 tensor_free from overcounting freed scale bytes
3bad5a2 monotophic committed 3 months ago
Merge pull request #1098 from monotophic/fix/cuda-absorb-syncthreads
f130c4e Vincenzo Fornaro committed 3 months ago
docs: Kimi K3 Metal tuning (M5 Max) + ignore build junk
a851e5b RDouglasSharp committed 3 months ago
Metal Kimi K3: aligned KDA state/window bufs, wrap-once buffer cache, CPU-side MLA KV cache, one-shot banners + throttled stats
4459b61 RDouglasSharp committed 3 months ago
Merge pull request #1111 from JustVugg/fix/cuda-syncwarp
2b5ae0e Vincenzo Fornaro committed 3 months ago
Merge pull request #1109 from SebaWag/fix/arm64-dotprod-probe
93271fd Vincenzo Fornaro committed 3 months ago
fix(cuda): add the __syncwarp() grouped_s4_wmma's siblings carry (closes #1099)
10af5e9 JustVugg committed 3 months ago
fix(arm64): probe dotprod by compiling, prefer armv8.2-a base (#1104)
48e4574 SebaWag committed 3 months ago
fix(loader): refuse duplicate tensor names across indexed shards
93226cf monotophic committed 3 months ago
Merge pull request #1097 from JustVugg/perf/v4-lanes-default
31debb7 Vincenzo Fornaro committed 3 months ago
perf(deepseek_v4): expert-loader pool default 3 -> 9 lanes (measured 1.41x decode)
ebc68da JustVugg committed 3 months ago
fix(tests): gap-check counts non-finite logprobs as present, flagged
8bc77e7 monotophic committed 3 months ago
fix(engine): reject glued, duplicate, and overflowing SUBMIT extension fields
7066288 monotophic committed 3 months ago
fix(c): check #798's alloc/snprintf results on the checkpoint-load path
80fbea4 monotophic committed 3 months ago
test(openai): golden-fixture battery harness and speculative-gap frame audit
8a80182 monotophic committed 3 months ago
test(cuda): cover run-to-run reproducibility of the batch and ragged absorb kernels
07e48a7 monotophic committed 3 months ago
fix(cuda): barrier the absorb softmax reduction before shared red[] is reused
2b7a892 monotophic committed 3 months ago
feat(engine): per-token logprob channel, ECHO prefill read-out, token-ID intake
075ca6b monotophic committed 3 months ago