Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
Commits on main
build(v4/cuda): fetch the pinned DeepGEMM sm120 headers on first build β cuda-dsv4-dg-dll / DEEPGEMM=1 need no external checkout, and nothing is vendored
467bec4 dcutugno committed 3 months ago
Merge pull request #1087 from terrizoaguimor/refactor/serve-codec-olmoe
a0d7cfb Vincenzo Fornaro committed 3 months ago
Merge pull request #1088 from JustVugg/perf/k2-union-tile
256bbe9 Vincenzo Fornaro committed 3 months ago
Merge pull request #1089 from JustVugg/fix/exactness-avx512-gate
4da41bf Vincenzo Fornaro committed 3 months ago
fix(test): the f32 planar assertion must mirror planar_on() exactly
19afc84 JustVugg committed 3 months ago
perf(colibri): K2 β 1x4 union tile in the planar IDOT matmul (bit-identical)
3cb832f JustVugg committed 3 months ago
Merge pull request #1086 from JustVugg/fix/k1-followup
0ad0ca7 Vincenzo Fornaro committed 3 months ago
Merge pull request #1085 from JustVugg/docs/changelog-summary
1de2939 Vincenzo Fornaro committed 3 months ago
Merge pull request #1084 from JustVugg/fix/web-triplet
b04d602 Vincenzo Fornaro committed 3 months ago
fix(k1): planar off on non-AVX2 builds; drop WIP loop; f32 planar in the exactness gate
feb9dba JustVugg committed 3 months ago
Merge pull request #1083 from JustVugg/ci/arm-oracle
2d2d92d Vincenzo Fornaro committed 3 months ago
docs: CHANGELOG catch-up v1.2.0βv1.6.2; SUMMARY.md was an experiment report
a8e0a5f JustVugg committed 3 months ago
fix(web): auto-connect in useEffect, atlas from engine origin, drop inert ternary
6c63091 JustVugg committed 3 months ago
Merge pull request #1082 from JustVugg/perf/fused3-olmoe
438d09a Vincenzo Fornaro committed 3 months ago
Merge pull request #1080 from JustVugg/fix/inkling-idot-optin
62b8d63 Vincenzo Fornaro committed 3 months ago
ci: ARM oracle job + integer-kernel bit-exactness gate (closes #1081)
a2ce84e JustVugg committed 3 months ago
feat(moe): FUSED3=1 opt-in AVX2 expert matmul β vectorized activation quant + gate/up pair
d2d9fd6 outtodata committed 3 months ago
Merge pull request #1079 from JustVugg/perf/k1-planar
fae9553 Vincenzo Fornaro committed 3 months ago
fix(inkling): IDOT opt-in β the fast path is x86-only and not token-equivalent
01d59ae JustVugg committed 3 months ago
Merge pull request #1073 from bherald/fix/k3-prefill-cancellation
6132a67 Vincenzo Fornaro committed 3 months ago
Merge pull request #1074 from bherald/fix/server-cancel-accounting
aa05364 Vincenzo Fornaro committed 3 months ago
Merge pull request #1072 from bherald/docs/k3-mmap-rss-accounting
73cc7e1 Vincenzo Fornaro committed 3 months ago
perf(colibri): K1 β plane-nibble int4 layout + unsigned-VNNI dot (bit-identical)
d8ea56f JustVugg committed 3 months ago
wip(k1): planar int4 kernels β dot_i4p_u, matmul_i4p_idot, matmul_i4p + planarize_i4
17be735 JustVugg committed 3 months ago
Merge pull request #1077 from JustVugg/perf/qrow-hoist-v4b
377a6eb Vincenzo Fornaro committed 3 months ago
Merge pull request #1076 from JustVugg/perf/qrow-hoist-v4
f70b1ed Vincenzo Fornaro committed 3 months ago