Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
eval: default the [gMASK]<sop> prefix ON for GLM snapshots (#108)
5230717 bokiko committed 4 months ago
stats: DISK_SPLIT=1 opt-in disk-load split by speculation context and layer kind
b611304 bokiko committed 4 months ago
placement: full-resident experts on large-memory hosts β CUDA_EXPERT_GB=auto, PIN_GB=all, adaptive GPU slots, RoPE cache (#80)
f1fa5bf ZacharyZcR committed 4 months ago
SCHEMA=<file.json>: JSON-Schema -> GBNF compiler for grammar-forced drafts (#148)
504fcdd Fabio Rovai committed 4 months ago
cuda-dll: make the Windows CUDA build work as shipped β MSVC host flags, CUDA_PATH defaults, POSIX shim in kernel test (#158, #157)
9b08cbc LewfKrad committed 4 months ago
prefill: batch the attention input projections β bit-identical, -4.5% prefill (#152)
12350a4 Dennis Paul committed 4 months ago
routing: cross-layer coupling trace dump + pair-table builder + opt-in coupled prefetch source (#176)
37ffb61 Fabio Rovai committed 4 months ago
web dashboard: live expert-tier panel (VRAM/RAM/disk), one-process hosting, coli web (#126)
2878c60 ZacharyZcR committed 4 months ago
Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)
d47b095 Tom Olorin committed 4 months ago
olmoe: int8 container support (fixes SIGBUS), NEON int8 dot kernels (2.7x on ARM), macOS RSS fix, converter fallback (#187)
ec0aadf Louie committed 4 months ago
serve: WaitForSingleObject/PeekNamedPipe stdin polling for run_serve_mux on native Windows (#177, fixes #189/#139)
32629ea woolcoxm committed 4 months ago
glm.c: harden 5 crash paths β qk_rope buffer guard, OOM-checked mallocs, null derefs (#183)
d872dd5 woolcoxm committed 4 months ago
loaders: fix silent short-read (uninitialized memory) in config/oracle readers (#181)
0f5d5c8 woolcoxm committed 4 months ago
config: fix UTF-8 decode crash + null config-value TypeError on Windows (doctor/resource_plan) (#184)
35ed9bc woolcoxm committed 4 months ago
openai_server: constant-time API key check (hmac.compare_digest) + resource_plan: csv-parse GPU names (fixes comma-in-name drop) (#186)
39dcebf woolcoxm committed 4 months ago
convert: NVFP4 (modelopt e2m1) dequant path for convert_fp8_to_int4.py (#83)
8395435 Rodolfo Hansen committed 4 months ago
convert: fix converter crashes on Windows (fcntl import, null config) (#185)
789169f woolcoxm committed 4 months ago
coli: raise RLIMIT_NOFILE at startup β fixes 'Too many open files' (#147)
bc688a6 Harvad Li committed 4 months ago
test: cover native-Windows platform detection without uname (#145)
7216992 Ronit Baldaniya committed 4 months ago
quant_ablation: condition GLM contexts on [gMASK]<sop> for in-distribution scoring (#155)
5407815 Dennis Paul committed 4 months ago
openai_server: COLI_DEBUG levels (1=output, 2=both sides) (#149)
101b8b7 RDouglas committed 4 months ago
docs: canonical grammar-forced-drafts reference page + README link (#170, fixes #146)
748787c Fabio Rovai committed 4 months ago
flake.nix: add tokenizers + datasets for the quality bench (#173)
c0fe3b7 Jon Bosque committed 4 months ago
resource_plan: macOS memory_available() via vm_stat β fixes 0-on-macOS budget (#150)
bf3e890 RDouglas committed 4 months ago
Makefile: add install/uninstall targets (PREFIX/bin + libexec) (#167, fixes #164)
6e9affb Sai Saharsh Chittimilla committed 4 months ago
Makefile: portable clean/test-c via python helpers β works without sh.exe on native Windows (#179, fixes #172)
c333840 woolcoxm committed 4 months ago
doctor: platform-aware engine.binary check β Windows has no X_OK bit (#178, fixes #141)
a0e9c8d woolcoxm committed 4 months ago