Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
quant: FWHT primitive for the fmt=6 rotation, with the placement rule (#452)
377cc35 ZacharyZcR committed 4 months ago
Merge #421: dual-SSD streaming (COLI_MODEL_MIRROR) β read the model from two drives at once
68ac9ff Vincenzo Fornaro committed 4 months ago
engine: fmt=6 E8/IQ3 lattice container β CPU decode kernel and dispatch (#452 step 3)
c45a882 ZacharyZcR committed 4 months ago
Merge remote-tracking branch 'upstream/dev' into feat/fmt5-engine
57fe1a3 ZacharyZcR committed 4 months ago
Merge dev into dual-ssd-mirror: resolve #298/#192-era conflicts
87cd0aa JustVugg committed 4 months ago
cuda: reject fmt=4 at the per-row-only entry points (#334 follow-up to #298)
e942f62 ZacharyZcR committed 4 months ago
Merge #192: serve stage 2 β response_format, per-request grammars, grammar-forced drafts
880cfb4 Vincenzo Fornaro committed 4 months ago
Merge #458: fmt=5 index codec tools β deployable bytes for the E8/IQ3 container
133d271 Vincenzo Fornaro committed 4 months ago
Merge #298: per-group scales in CUDA dense/attention kernels (fmt=4 correctness)
c98e5f8 Vincenzo Fornaro committed 4 months ago
fix(rebase): adapt to dev's split tensor_upload/_g API + fault-injection hook
8b69c89 woolcoxm committed 4 months ago
fix(rebase): adapt fmt=4 to dev's quant.h refactor + colibri.c rename
f4409fd woolcoxm committed 4 months ago
fix: apply per-group scales in CUDA attention kernels for fmt=4 (kv_b crash)
d96e425 woolcoxm committed 4 months ago
fix: layer_cuda_shard_kvb scale offset for fmt=4 (multi-GPU, per JustVugg review)
bfb8000 woolcoxm committed 4 months ago
cuda+engine: full fmt=4 (grouped int4 gs=64) support + diagnostic harness
6e29260 woolcoxm committed 4 months ago
eval_glm: stream results line-by-line + incremental CSV (no more lost runs)
8698715 woolcoxm committed 4 months ago
cuda: COLI_CUDA_RESID=1 β expert-group results stay on device (#431 PR-C0)
0f8cbe2 ZacharyZcR committed 4 months ago
Merge #168: int3-g64 (fmt=5) per-group-scale 3-bit expert quant
f42adfa Vincenzo Fornaro committed 4 months ago
convert: --up-bits/--gate-bits/--down-bits per-projection expert quant
932f367 FABIOTESS committed 4 months ago
int3-g64 (fmt=5): per-group-scale 3-bit weight format β engine, converter, tests
5e42e70 FABIOTESS committed 4 months ago
review: docs section, 1 MiB grammar pre-check, negative tests, measured compile overhead
84514a5 FABIOTESS committed 4 months ago
serve stage 2: per-request grammars end-to-end β response_format -> SUBMIT -> grammar-forced drafts in the multiplexed server
d7b855f FABIOTESS committed 4 months ago