Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
Makefile(win): link -lpsapi so the MinGW build resolves GetProcessMemoryInfo
a239158 JustVugg committed 4 months ago
run_text: TOKENS=1 env to dump generated token ids for path A/B comparison
e56d483 woolcoxm committed 4 months ago
Merge pull request #289 from woolcoxm/fix/grouped-quant-detect-arbitrary-gs
a37dda9 Vincenzo committed 4 months ago
quant: generalize fmt=4 group-size detection (g64/g128/g256, not just 128)
5557694 woolcoxm committed 4 months ago
cuda: gate pipe2 S-threshold on device count (single-GPU S=1, multi-GPU S>=8)
a55cdfd woolcoxm committed 4 months ago
feat(prefetch): implement persistent hot pinning and pre-warmup wait loop to break 94% hit rate barrier
ca78883 Egon Ruiter committed 4 months ago
feat(pinning): implement dynamic asymmetric expert pinning and layer 0 EMA update to reach 90.9% hit rate and 3.25 tok/s
24058d3 Egon Ruiter committed 4 months ago
feat(prefetch): add opt-in lookahead-3 prefetching option for PILOT=3
b3fdb14 Egon Ruiter committed 4 months ago
feat(prefetch): implement stale request pruning and queue de-duplication to break 90% hit rate barrier
4ae4f61 Egon Ruiter committed 4 months ago
feat(io): implement Consolidated Expert I/O to reduce expert disk reads by 3x and accelerate prefetching
c462427 Egon Ruiter committed 4 months ago
feat(prefetch): implement prefetcher v2.1 with lookahead-2, hot pinning, adaptive cache, RMSNorm scaling, and routing EMA
ac1f7a8 Egon Ruiter committed 4 months ago
docs: CACHE_ROUTE breakthrough β 1.41 tok/s (4.3x over stock), misses 27%->17%
0430145 woolcoxm committed 4 months ago
cache: fix ws_b overcounting under EXPERT_BUDGET (cap 3->4, hit 57%->73%)
d0971ff woolcoxm committed 4 months ago
glm: hwinfo_emit Windows support β CPU brand, cores, RAM were hardcoded-zero
939b689 KingIcyCreamProjects committed 4 months ago
coli: measured Windows launcher defaults β OMP tuning parity + DIRECT/PIPE/PILOT_REAL
1a243cf KingIcyCreamProjects committed 4 months ago
docs(readme): mechanical fixes β expert count, IO_THREADS, line counts, desktop/
4059e10 KingIcyCreamProjects committed 4 months ago
benchmark_cuda_fixture: accept the current PROFILE service/wait format
51dd480 KingIcyCreamProjects committed 4 months ago
resource_plan: count physical cores on Windows (GetLogicalProcessorInformationEx)
f72ee65 KingIcyCreamProjects committed 4 months ago
cuda: enable pipe_layer_sparse resident pipeline at decode (S>=1 gate)
c5b1d14 woolcoxm committed 4 months ago
ARM i8mm: SMMLA fast path for the int8/int4 IDOT drivers
d0c9941 Michael Denyer committed 4 months ago
Merge pull request #267 from michael-denyer/fix-profile-disk-wait
52550ff Vincenzo committed 4 months ago
Merge pull request #261 from FECO-Admin/fix/randomuuid-fallback
39a0777 Vincenzo committed 4 months ago
Merge pull request #265 from skeldoor/feat/interrupt-generation
0fc18ab Vincenzo committed 4 months ago
Merge pull request #264 from skeldoor/perf/neon-multiacc-sdot
87ea13c Vincenzo committed 4 months ago