Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
docs(env): refresh ENVIRONMENT.md β document all 34 drifted env vars
099e0c1 woolcoxm committed 4 months ago
fix(test): make test_stops build on Windows (mkdtemp compat shim)
5e2be61 woolcoxm committed 4 months ago
fix(warnings): silence two -Wall/-Wextra warnings on the MinGW build
411f237 woolcoxm committed 4 months ago
fix(flake): create flake.lock, use --set-default for COLI_ENGINE
03e643f LordMZTE committed 4 months ago
omp: cap LLVM libomp idle spin β a parked engine must not burn 3000% CPU (#341)
d5327e2 JustVugg committed 4 months ago
Merge pull request #348 from woolcoxm/fix/nix-flake-packaging-345
617ae51 Vincenzo committed 4 months ago
coli: serve pidfile + coli stop β one command to shut engine and server down
7eb2393 JustVugg committed 4 months ago
coli: chat attaches to a running serve β the engine survives the chat (local)
8bf4cb9 JustVugg committed 4 months ago
fix(flake): package engine, support modules, and python for Nix builds (#345)
528b3cf woolcoxm committed 4 months ago
tools: rate-scale the E8 lattice ball by bit-width β int3-e8 is now a real int3 codebook, not a fixed 2-bit ball
3f239db ZacharyZcR committed 4 months ago
pipe: blocking pipe_wait (COLI_PIPE_BLOCK=1) + PIPE_WORKERS implies PIPE=1
dc19663 ebootheee committed 4 months ago
glm.c: detect 9p by statfs f_type, not the "/mnt/" path prefix
81a5677 Tonu Samuel committed 4 months ago
feat: AMD GPU support via HIP/ROCm β single-source backend through backend_gpu_compat.h
d52e761 noobdev-ph committed 4 months ago
test_tok_o200k: fgets instead of getline for the windows job
5ad4d54 Nicholas Beerbower committed 4 months ago
test_st_pread: Windows-compatible β relative tmpdir, fork subtest gated POSIX
ed8dab4 Nicholas Beerbower committed 4 months ago
st: chunked pread with EINTR retry and honest short-read errors
6507817 Nicholas Beerbower committed 4 months ago
tests: o200k tokenizer coverage, no model download
9e2d415 Nicholas Beerbower committed 4 months ago
tok: o200k pre-tokenizer support, auto-detected from tokenizer.json
364b974 Nicholas Beerbower committed 4 months ago
docs: Windows 11 native install walkthrough - toolchain, SAC, CUDA DLL, failure index (#198)
69284fa Mohamed Mastouri committed 4 months ago
convert_olmoe.py: wire --ebits through to quantization (#323)
80f886c Mohamed Mastouri committed 4 months ago
Merge pull request #324 from dawnfield-institute/feat/olmoe-ppl-mode
cead3d8 Vincenzo committed 4 months ago
ci: build the Windows CUDA path (nvcc + MSVC host) β the one toolchain no job covers
b75422d lEWFkRAD committed 4 months ago
tests: test_stops must not assume /tmp exists (fixes the windows job)
e7188df JustVugg committed 4 months ago
Merge pull request #313 from ZacharyZcR/feat/numa-expert-interleave
3cec90c Vincenzo committed 4 months ago