Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
feat: worker init parity — st_init_multi + COLI_MODEL_DIRS, clean unresolvable-expert error
a380a04 eli committed 3 months ago
Merge pull request #1067 from JustVugg/fix/olmoe-converter-test-numpy-mock
1cc0be2 Vincenzo Fornaro committed 3 months ago
fix(test): stub numpy in the OLMoE converter import mock — dev CI is red
b84154d JustVugg committed 3 months ago
Merge pull request #965 from bherald/feat/k3-mmap-prepared-weights
ed75a25 Vincenzo Fornaro committed 3 months ago
Merge pull request #1048 from sami7969/fix/olmoe-numpy-and-size
36e8397 Vincenzo Fornaro committed 3 months ago
Merge pull request #1064 from jerome-benoit/fix/doctor-deep-dtypes
4d51576 Vincenzo Fornaro committed 3 months ago
Merge pull request #1065 from jerome-benoit/feat/flake-all-engines
eeaa6b6 Vincenzo Fornaro committed 3 months ago
Merge pull request #1063 from terrizoaguimor/refactor/family-registry
12368b7 Vincenzo Fornaro committed 3 months ago
fix(doctor): accept engine-supported I64/F8 dtypes in --deep validator
8c5f046 Jérôme Benoit committed 3 months ago
Merge pull request #1035 from lineape/pr551-prefactor-shared-ffn
0578bba Vincenzo Fornaro committed 3 months ago
Merge pull request #1047 from sami7969/fix/windows-build-blockers
85951ef Vincenzo Fornaro committed 3 months ago
Merge pull request #1061 from nadiadatepe-eng/fix/olmoe-idot-default-off
2410c6e Vincenzo Fornaro committed 3 months ago
Merge pull request #1059 from JustVugg/fix/windows-stop-and-orphan-engine
9e10ebd Vincenzo Fornaro committed 3 months ago
Merge pull request #1055 from dcutugno/pr/v4-cuda-tier
dd39c4e Vincenzo Fornaro committed 3 months ago
test: fix the Windows job test — never put the runner in the job
ca657ce JustVugg committed 3 months ago
docs(v4): CUDA tier — build (Windows DLLs, Linux CUDA=1), serve line, VRAM budget + per-card settings, performance, GPU coverage, environment reference, validation
5eb9b32 dcutugno committed 3 months ago
feat(v4): the CUDA tier wired into the engine — every stage CPU-canonical with per-stage fallback
66fca1e dcutugno committed 3 months ago
feat(v4/cuda): the DeepSeek V4 CUDA kernels the engine tier needs — generic sm_80+ + opt-in DeepGEMM, Windows DLL export/loader
4eddc62 dcutugno committed 3 months ago
fix(windows): make coli stop actually find the serve, and stop orphaning engines
35349e1 JustVugg committed 3 months ago
fix(plan): match DeepSeek V4 expert tensor naming (layers.N.ffn.experts.M)
d44b096 dcutugno committed 3 months ago
docs(v4): prefill segments/chunks/checkpoints, env additions, what CPU-path output identity means
623f6c1 dcutugno committed 3 months ago
feat(v4/serve): prefix hint in SUBMIT, hold admission until the engine acks a CANCEL
62be01b dcutugno committed 3 months ago
feat(v4): prefix checkpoints (memory + disk), resumable 4096-token prefill segments, cancellable prefill, batched indexer selection
51638e8 dcutugno committed 3 months ago
olmoe: make IDOT opt-in on x86 and NEON, and add the test that was missing
af48fe8 nadi committed 3 months ago
Merge pull request #827 from monotophic/kvb/fmt-gate-notice-r3
03e8677 Vincenzo Fornaro committed 3 months ago