Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
inkling perf: packed-int4 cache, parallel expert fills, usage pinning, auto cap
1ee59cc Nicholas Beerbower committed 4 months ago
convert_inkling: completion check counts only model-* shards
de12586 Nicholas Beerbower committed 4 months ago
convert_inkling: thread the per-expert quantization (12 workers)
593bc3d Nicholas Beerbower committed 4 months ago
inkling: bf16-resident weights + AVX-512 BF16/VNNI kernels
92bdb87 Nicholas Beerbower committed 4 months ago
tools: Inkling bf16 -> colibri int4 converter, e2e-validated
b721a1e Nicholas Beerbower committed 4 months ago
inkling: Stage-A C engine for Thinking Machines Inkling (975B MoE), oracle-validated
b7d4835 Nicholas Beerbower committed 4 months ago
Merge pull request #488 from RDouglasSharp/fix/metal-gemm-grid-truncation
f1b0f7e Vincenzo Fornaro committed 4 months ago
Merge pull request #533 from JustVugg/release/v1.1.1
cd278c2 Vincenzo Fornaro committed 4 months ago
release: 1.1.1 — ship the .data fix before more Windows users hit it
004f7ae JustVugg committed 4 months ago
Merge pull request #532 from JustVugg/fix/grdraft-bss-blob
6164f9f Vincenzo Fornaro committed 4 months ago
engine: keep the 107 KB GrDraft struct out of .data (binary -22%)
388747a JustVugg committed 4 months ago
gemm-test: correct stale threshold comments (trigger is the dispatch grid NT=S*O, not S; S=4376 is clean)
42b3f13 Doug Sharp committed 4 months ago
Merge pull request #530 from JustVugg/fix/release-sha256sums
7f77d39 Vincenzo Fornaro committed 4 months ago
metal: clamp chunk size in 64-bit before narrowing (UB in the COLI_GEMM_CHUNK=0 path)
70f06cd Doug Sharp committed 4 months ago
Merge pull request #526 from JustVugg/fix/gateway-engine-default
9baae9b Vincenzo Fornaro committed 4 months ago
Merge pull request #525 from JustVugg/feat/343-anthropic-messages
a04721e Vincenzo Fornaro committed 4 months ago
serve: speak the Anthropic Messages API on /v1/messages (#343)
1d31c7e JustVugg committed 4 months ago
feat(olmoe): port LFRU prefetch eviction guard to OLMoE testbed
7e707c7 Egon Ruiter committed 4 months ago