Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1192 from JustVugg/feat/k3-recurrent-checkpoints
b851427 Vincenzo Fornaro committed 3 months ago
fix: clamp explicit RAM_GB to the memory actually available (#759)
ee79841 Mr-Neutr0n committed 3 months ago
feat(k3): disk-parked recurrent-state checkpoints (COLI_K3_CKPT_DIR)
3b2f1e1 JustVugg committed 3 months ago
feat(tune): measure every engine, not just GLM, via the serve protocol
df5f8f6 JustVugg committed 3 months ago
fix(tune): resolve the cap from the plan when --cap is not given
32cdef0 JustVugg committed 3 months ago
feat(k3): opt-in recurrent-state checkpoints for divergent prompts
70c948c JustVugg committed 3 months ago
Merge pull request #1180 from JustVugg/perf/dsv4-prefill-pool-v2
5129f83 Vincenzo Fornaro committed 3 months ago
Merge pull request #1188 from JustVugg/test/kimi-k3-tiny-oracle
f3698b2 Vincenzo Fornaro committed 3 months ago
Merge pull request #1181 from 8PotatoChip8/feat/dsv4-cuda-pre-ampere
7412b94 Vincenzo Fornaro committed 3 months ago
Merge pull request #1185 from JustVugg/fix/glm-banner-compute-width
b53130e Vincenzo Fornaro committed 3 months ago
Merge pull request #1186 from waizuichougou/bug/qwen36-cuda-default-device
1a861ef Vincenzo Fornaro committed 3 months ago
Merge pull request #1187 from waizuichougou/bug/qwen36-final-token-step
035fbcc Vincenzo Fornaro committed 3 months ago
Merge pull request #1178 from TheOptik/pr/fp8-simd-decode
a2aba48 Vincenzo Fornaro committed 3 months ago
Merge pull request #1184 from JustVugg/fix/datapoint-teardown-preserves-results
a7e2e38 Vincenzo Fornaro committed 3 months ago
test(k3): tiny fixture + independent vendor oracle for Kimi K3
03df9d8 JustVugg committed 3 months ago
fix(v4/cuda): the runtime arch gate still refused pre-Ampere cards
7e359b5 8PotatoChip8 committed 3 months ago
fix(qwen36): discover visible CUDA devices for default tier
f663ce5 waizuichougou committed 3 months ago
fix(tools): keep datapoint results when engine teardown outlasts the grace period
21e80f3 JustVugg committed 3 months ago
bench: report execution and allocation latency in us and ms per call
65c1bed Thanniru Sai Teja committed 3 months ago
docs(v4): add reentrancy and single-threaded assumption comment on indexer scratch_buf
3cd5e47 Thanniru Sai Teja committed 3 months ago
fix(bench): fix realloc leak and add ALIGN32 32-byte SIMD alignment matching deepseek_v4.c
8a27730 Thanniru Sai Teja committed 3 months ago
fix(v4): align base scratch_ptr to 32-byte boundary before sub-allocations
8e3e2e1 Thanniru Sai Teja committed 3 months ago
fix(v4): enforce 32-byte SIMD alignment on indexer scratch sub-allocations
85818a6 Thanniru Sai Teja committed 3 months ago
perf(v4): eliminate dynamic heap allocations in indexer batch selection path
e29ae2a Thanniru Sai Teja committed 3 months ago