Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
fix(datapoint): fresh-process mode talked to legacy engines in the locale's code page
f15360f kevin9327 committed 2 months ago
fix(autotune): coli tune on GLM died decoding the engine's UTF-8 output
9e7d3b4 kevin9327 committed 2 months ago
fix(coli): coli run on OLMoE sent the prompt in the locale's code page
ec51ce9 kevin9327 committed 2 months ago
Merge pull request #1558 from JustVugg/feat/v41-kv-prefix-reuse
b5968eb Vincenzo Fornaro committed 2 months ago
deepseek_v41: opt-in KV prefix reuse, and a resumed prefill that is exact
fcf0aae JustVugg committed 2 months ago
fix(qwen36): a truncated multibyte tail made the tokenizer read past the prompt
3b459d0 Qutibah Ananzeh committed 2 months ago
Merge pull request #1553 from JustVugg/feat/kv-prefix-reuse
1330bfd Vincenzo Fornaro committed 2 months ago
colibri: add CONSIST=1 prefill/decode self-consistency check
de83684 phreakocious committed 2 months ago
qwen36, olmoe: reuse the previous turn's KV instead of re-prefilling the transcript
16bfe58 JustVugg committed 2 months ago
fix(st): publish the lazy shard-mapping table without a data race
7b46843 basil-k-aji-dev committed 2 months ago
metal: fall back to MTLCopyAllDevices when there is no system default device
5b9d011 phreakocious committed 2 months ago
test(olmoe): add routing test + relax converter_accepts assertion
9dd5ec7 yeduwuren2026 committed 2 months ago
fix(olmoe): rename --out to --outdir to match cmd_convert convention
57f34a6 yeduwuren2026 committed 2 months ago
fix(olmoe): route coli convert to convert_olmoe_merged.py
b6d7297 yeduwuren2026 committed 2 months ago
glm53: size the expert cache around the model, not around MemAvailable
cf7288a Ronald Klomp committed 2 months ago
Merge pull request #1543 from JustVugg/fix/v4-mirror-cache-unified
87c54a9 Vincenzo Fornaro committed 2 months ago
Merge pull request #1521 from rcklomp/perf/hc-parallel-mix
0264c5a Vincenzo Fornaro committed 2 months ago
Merge pull request #1524 from rcklomp/perf/kda-parallel
48df0fa Vincenzo Fornaro committed 2 months ago
Merge pull request #1517 from LiRunGuo/fix/omp-team-sizing
2c917b2 Vincenzo Fornaro committed 2 months ago
Merge pull request #1522 from LiRunGuo/feat/dsv41-expert-io
6c68af9 Vincenzo Fornaro committed 2 months ago
deepseek_v4: mirror cache on unified memory and the frozen-cache recycle (#1538)
78a9ee9 JustVugg committed 2 months ago
Merge branch 'dev' into fix/hip-test-cuda-runtime-include
955974b Vincenzo Fornaro committed 2 months ago
Merge pull request #1513 from huppiflupp/vk-integrated
12a1104 Vincenzo Fornaro committed 2 months ago
delta_attention: reword the comments without private labels or fork paths
fdedf17 Ronald Klomp committed 2 months ago
hyper_connections: reword the comments without private labels
4cdc736 Ronald Klomp committed 2 months ago
docs(experiments): correct the V4.1 expert-read record for the regime it ran in
e761776 Gawr Gura 0721 committed 2 months ago
docs(experiments): the V4.1 expert-read A/B, with the phase ratio that bounds it
4b9627b Gawr Gura 0721 committed 2 months ago