Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
security(win): load coli_cuda.dll by absolute path, never from the CWD (DLL hijack)
ff04b32 JustVugg committed 4 months ago
security: harden safetensors/JSON parsers against malicious model files
4268e00 JustVugg committed 4 months ago
experiment: group-scaled int4 (fmt=4) β one scale per 128 elements, not per row
498ab0c woolcoxm committed 4 months ago
glm: honest short-read reporting in expert pread; Makefile: -pthread for *BSD
c71e5d1 JustVugg committed 4 months ago
glm: PIN_GB=all clamps to the --ram budget instead of pinning every expert (fixes #229)
ddf9212 JustVugg committed 4 months ago
Merge pull request #228 from bokiko/feature/score-prefix-default
22c9140 Vincenzo committed 4 months ago
Merge pull request #224 from michael-denyer/fix-kv-alloc-double-free
00902a2 Vincenzo committed 4 months ago
Merge pull request #194 from bokiko/feature/glm-prefix-default
5025f00 Vincenzo committed 4 months ago
tools: expert_atlas β confound-controlled probe harness for the GLM-5.2 expert atlas (#175) (#218)
a28f31f Dennis Paul committed 4 months ago
tests: remove accidentally committed Linux test binaries + gitignore them β fixes make test-c on non-Linux (#226)
4165703 Michael Denyer committed 4 months ago
glm: fix kv_alloc double-free on KV re-allocation (stale pre-Metal free block)
2bdf054 Michael Denyer committed 4 months ago
routing: opt-in CACHE_ROUTE max-rank prefer (pinβͺLRU), default off β 2.4->3.33 tok/s on GB10 (#199)
62419af Marquez committed 4 months ago
FreeBSD compatibility: platform guards in glm.c/olmoe.c (#206)
4a936c8 yuri@FreeBSD committed 4 months ago
Makefile: select $(PYTHON) from the host, not the target triple (#205)
bfd49c4 bopof committed 4 months ago
openai_server: fix path traversal in web static serving (relative_to, not startswith) + empty-prompt validation, with tests (#212)
8c18b7a Kushida committed 4 months ago
Makefile: opt-in NVCC_CCBIN=<compiler> to override nvcc's host compiler (fixes #211)
bad64d1 JustVugg committed 4 months ago
pilot: two-step shared-expert router prediction, +3.1% recall, opt-in PILOT_TWO=1 (#204, #200)
6ca9565 woolcoxm committed 4 months ago
bench: pass the resolved GLM engine path to eval_glm.py β cwd-relative ./glm default broke coli bench from other dirs (#203)
ab09dc5 Benny Powers committed 4 months ago
README: dashboard screenshots + contents index + Web dashboard section (#202)
e21318e ZacharyZcR committed 4 months ago
cuda(win): export + load the #111 pipeline ABI β 24 missing DLLEXPORT decorations + loader wrappers, unbreaks CUDA_DLL=1 (#201)
21cbf14 Tom Olorin committed 4 months ago
bench: default --data to the XDG/LocalAppData cache dir instead of the repo tree (#190)
3045b35 Benny Powers committed 4 months ago
GPU resident pipeline: batch CUDA attention, head-sharded kv_b, prefill expert groups, W4A16 mixed dispatch (#111)
ec89136 ZacharyZcR committed 4 months ago
serve mux: fix Windows request dispatch β BINARY-mode stdin/stdout + PeekNamedPipe-only polling (fixes #195)
2ead86a JustVugg committed 4 months ago
glm.c: silence unused-variable 'c' in hwinfo_emit (reported in #148 on Darwin, present on Linux too)
e12a429 JustVugg committed 4 months ago
run_tests: normpath test binaries β forward-slash relative paths fail CreateProcess on Windows (#196)
372f8fd Tom Olorin committed 4 months ago
glm.c: remove the LMHEAD_EXACT probe accidentally merged with #152 β functional no-op strip (#197)
4a0435c Dennis Paul committed 4 months ago
engine: default the [gMASK]<sop> prefix ON in SCORE mode for GLM snapshots (#194, #108)
c4d1401 bokiko committed 4 months ago
ci: run make check on ubuntu / windows (MSYS2 UCRT64) / macos (#140)
f7c7ab6 lEWFkRAD committed 4 months ago