Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
Merge #483: windows build-config stamp + scoop/VS-prompt sh.exe doc (#478)
d338217 Vincenzo Fornaro committed 4 months ago
Merge #485: silence -Wformat-truncation in expert tensor-name snprintfs (#484)
928cf17 Vincenzo Fornaro committed 4 months ago
Merge #495: honor explicit CUDA_EXPERT_GB under CUDA_DENSE=1 (#491)
6dc16c5 Vincenzo Fornaro committed 4 months ago
Merge #464: reject fmt=4 without a group size at the per-row-only CUDA entry points (#334/#298)
0cd1bb7 Vincenzo Fornaro committed 4 months ago
Merge #465: fmt=6 E8/IQ3 lattice container β CPU decode kernel + dispatch (#452 step 3)
a0f294b Vincenzo Fornaro committed 4 months ago
Merge #481: SIMD MLA-absorb score + value-mix reductions (+4.7x, byte-identical)
2910e44 Vincenzo Fornaro committed 4 months ago
fix(cuda): honor explicit CUDA_EXPERT_GB under CUDA_DENSE=1 (#491)
3d854e7 woolcoxm committed 4 months ago
Fix GPU detection on unified-memory chips; wire HF_TOKEN into downloader
ee9b11d ideaclaw committed 4 months ago
fix: silence -Wformat-truncation in expert tensor-name snprintfs (#484)
ea45ee9 woolcoxm committed 4 months ago
windows: use $(file) for the build-config stamp + document the scoop/VS-prompt sh.exe gap (#478)
44c8153 woolcoxm committed 4 months ago
merge: dev (#298 fmt=4 per-group scales) β HIP verified, test call sites repaired
fe6c164 noobdev-ph committed 4 months ago
test: add CUDA env integration tests (COLI_CUDA modes + expert budget auto-size)
173cda0 Hermes Quant Auditor committed 4 months ago
Merge remote-tracking branch 'upstream/dev' into fix/dense-fmt4-scales
d8277d6 ZacharyZcR committed 4 months ago
Merge remote-tracking branch 'upstream/dev' into feat/fmt5-engine
56e4ed7 ZacharyZcR committed 4 months ago
attn: vectorize the MLA absorption helpers (qt_addrow / qt_matvec_rows int4)
a66c99a veerareddyvishal144 committed 4 months ago
Merge remote-tracking branch 'upstream/main' into fix/dense-fmt4-scales
981a82e ZacharyZcR committed 4 months ago
Merge remote-tracking branch 'upstream/main' into feat/fmt5-engine
04ef848 ZacharyZcR committed 4 months ago
omp: reset CPU affinity before the tuning re-exec (fixes #471)
4dfe581 veerareddyvishal144 committed 4 months ago
moe: XEXP=1 opt-in cross-expert OpenMP block for S=1 all-resident int4 decode
e0f12c2 veerareddyvishal144 committed 4 months ago
Merge #474: LFRU-aware pilot eviction guard β a speculation never evicts a warm demand expert
4aca059 Vincenzo Fornaro committed 4 months ago
pilot: LFRU-aware eviction guard β never evict a warm demand expert for a speculation (#441)
eaa68f4 cdhdt committed 4 months ago
quant: int4 IDOT at S=1 on AVX-512 VNNI (g_i4s=1), like ARM SDOT
80cf3d4 veerareddyvishal144 committed 4 months ago
Merge #470: respect explicit COLI_CUDA_MTP=1 β planner skips DRAFT=0 export
1b8f307 Vincenzo Fornaro committed 4 months ago
fix(plan): respect explicit COLI_CUDA_MTP=1 - skip the DRAFT=0 export so the engine's auto path can engage
ac04b37 Mohamed Mastouri committed 4 months ago
Merge #469: COLI_MODEL_DIRS β split model shards across N drives (capacity aggregation)
e16ec68 Vincenzo Fornaro committed 4 months ago
Merge #468: honour explicit COLI_CUDA=0 over Windows auto-enable + drive-dependent DIRECT docs
4cc9885 Vincenzo Fornaro committed 4 months ago
fix(coli): honour explicit COLI_CUDA=0 over the Windows bare-run auto-enable
f939191 Mohamed Mastouri committed 4 months ago
feat(st): COLI_MODEL_DIRS - split model shards across N drives, no duplication
0d2fb6f Mohamed Mastouri committed 4 months ago
cuda: fix two loader stubs missed in the resident-rename (#434 Windows CI)
32eac69 ZacharyZcR committed 4 months ago