Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
Merge #434: COLI_CUDA_RESID=1 β expert-group results stay on device (#431 PR-C0)
31d644c Vincenzo Fornaro committed 4 months ago
Merge #504: fix broken anchor links in the Docker READMEs
645d6fb Vincenzo Fornaro committed 4 months ago
Merge #501: convert --xbits e8 β produce and load the fmt=6 E8/IQ3 container (#452 step 4)
38763b2 Vincenzo Fornaro committed 4 months ago
Merge #500: remove 12 stray test binaries from dev + gitignore the pattern
3be2336 Vincenzo Fornaro committed 4 months ago
tools: bytes-true 'iq3-pack' ablation scheme + inverse rotation (#452 step 2)
f7f6826 ZacharyZcR committed 4 months ago
convert: --xbits e8 β produce and load the fmt=6 E8/IQ3 container (#452 step 4)
3634001 ZacharyZcR committed 4 months ago
tests: drop 12 stray test binaries committed in #368's conflict resolution
fb92fa5 ZacharyZcR committed 4 months ago
hip: map the peer-copy/event-wait surface the resident path uses
ed66f5c ZacharyZcR committed 4 months ago
Merge #368: security hardening β server, parsers, build (untrusted-mirror threat model)
e90a4a5 Vincenzo Fornaro committed 4 months ago
Merge dev into security branch: resolve #391/#413-era conflicts + fix fmt=5 scale cap
cc7461f JustVugg committed 4 months ago
cuda: carry per-tensor group sizes into the issue/resident GroupDesc (post-merge)
7577f94 ZacharyZcR committed 4 months ago
Merge remote-tracking branch 'upstream/dev' into feat/cuda-resident-accum
6a958f8 ZacharyZcR committed 4 months ago
sec(olmoe): validate expert weight/scale sizes before the raw read
9b1d87d JustVugg committed 4 months ago
Merge #497: narrow the LFRU eviction guard so speculations don't over-drop (fixes #490)
700532d Vincenzo Fornaro committed 4 months ago
Merge #339: AMD GPU support β HIP/ROCm via single-source backend_gpu_compat.h
7c54ad0 Vincenzo Fornaro committed 4 months ago
Merge remote-tracking branch 'upstream/dev' into feat/hip-single-source
65be434 noobdev-ph committed 4 months ago
fix(pilot): narrow LFRU eviction guard so speculations don't over-drop (#490)
38d1777 woolcoxm committed 4 months ago
fix(ci): guard #471 CPU-affinity reset with #ifdef CPU_SETSIZE
5e02064 JustVugg committed 4 months ago
Merge #477: vectorize MLA absorption helpers (qt_addrow/qt_matvec_rows) β +13% decode AVX-512, byte-identical
b86cc1d Vincenzo Fornaro committed 4 months ago
Merge #475: opt-in XEXP=1 β one OpenMP region per expert block at S=1 full residency (+11.6%)
e05ae97 Vincenzo Fornaro committed 4 months ago
Merge #473: int4 IDOT at S=1 on AVX-512 VNNI (g_i4s=1)
52fee21 Vincenzo Fornaro committed 4 months ago
test: add CUDA env integration tests (COLI_CUDA modes + expert budget auto-size)
2b098cf Hermes Quant Auditor committed 4 months ago
Fix GPU detection on unified-memory chips; wire HF_TOKEN into downloader
a920336 ideaclaw committed 4 months ago
Merge #479: CUDA env integration tests (COLI_CUDA modes + expert budget auto-size)
44e489b Vincenzo Fornaro committed 4 months ago
Merge #487: GPU detection on unified-memory chips (GB10) + wire HF_TOKEN into downloader
6798bfe Vincenzo Fornaro committed 4 months ago
Merge #476: reset CPU affinity before the OMP tuning re-exec (fixes #471)
766dd07 Vincenzo Fornaro committed 4 months ago