Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Commits on main

fix: commit capped final token to stateful KV
winklemad committed 4 months ago
0588021
fix(plan): warn when VRAM is already in use
ThefloorMiner committed 3 months ago
3e2a261
docs(v4): describe target-only engine split
DrewZ committed 3 months ago
aa1ec16
test(v4): generate target-only oracle in CI
DrewZ committed 3 months ago
f8888ae
43c5db8
refactor(v4): preserve target-only runtime
DrewZ committed 3 months ago
8245eea
feat(st): support V4 checkpoint metadata
DrewZ committed 3 months ago
9869094
fddb786
c215880
fix(tune): retain legacy throughput parsing
dyKiU committed 3 months ago
8430c38
fe308c9
a570e86
Merge pull request #756 from rgbkrk/docs-inkling-audio
Vincenzo Fornaro committed 3 months ago
bec1546
b6f4dfc
b9ee0ed
cda2e4e
46da5ce
ee0c668
Merge pull request #750 from rgbkrk/shrinkling
Vincenzo Fornaro committed 3 months ago
5b8ae12
113bc0f
inkling: Apple defaults from the measured sweep
Kyle Kelley committed 3 months ago
e57083f
inkling: keep Metal off the decode path
Kyle Kelley committed 3 months ago
d7b18c0