Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Handle SIGTERM, and drop the SA_RESTART that would have made it useless
6509ae2 ZacharyZcR committed 3 months ago
Merge pull request #847 from JustVugg/fix/doctor-names-shard
a54b4da Vincenzo Fornaro committed 3 months ago
Merge pull request #844 from JustVugg/fix/release-idempotent
c7ae787 Vincenzo Fornaro committed 3 months ago
Merge pull request #846 from JustVugg/fix/reasoning-effort
9bdb0d4 Vincenzo Fornaro committed 3 months ago
Merge pull request #845 from JustVugg/fix/metal-test-fp8-nblk
552c2eb Vincenzo Fornaro committed 3 months ago
fix(api): reasoning_effort rendered Max for every level except high
3c6788a JustVugg committed 3 months ago
fix(metal-test): rename test-local fp8_nblk, which quant.h now also declares
c7aaaf7 JustVugg committed 3 months ago
ci(release): attach to an existing release instead of failing on it
256b8fb JustVugg committed 3 months ago
Merge pull request #842 from JustVugg/docs/readme-roster
e323575 Vincenzo Fornaro committed 3 months ago
docs: DeepSeek V4 joins the roster, and the repo layout matches the repo
388fdf7 JustVugg committed 3 months ago
security: six advisories — loader bounds, router NaN, serve limits
edcdd4e JustVugg committed 3 months ago
Merge pull request #839 from JustVugg/feat/deepseek-v4-perf
133925e Vincenzo Fornaro committed 3 months ago
Merge pull request #835 from ZacharyZcR/fix/tracked-bench-binary
2e45ad3 Vincenzo Fornaro committed 3 months ago
Decode MXFP4 (fmt=7) on CUDA, so Kimi K3 has a CUDA expert tier
db0c80f ZacharyZcR committed 3 months ago
deepseek_v4: release aligned slabs with compat_aligned_free, not free()
63a2c8d JustVugg committed 3 months ago
deepseek_v4: AVX2 kernels, direct expert I/O, warm pins, verified drafting
b7117bf JustVugg committed 3 months ago
inkling: make kv_alloc grow-with-copy ring-aware (#786 collision)
bccdcb5 Davide Panelli committed 3 months ago
inkling: ring-buffer KV cache for sliding-window layers
7fefa8c Davide Panelli committed 3 months ago
docs(fmt2): M1 Ultra fmt=2 report + benchmarks row + fmt4->fmt2 converter
f2f745c boblaublaw committed 3 months ago
Run the test suite under ASan+UBSan, and fix the clean that hid it
ca13cce ZacharyZcR committed 3 months ago
wip(fmt2): snapshot fmt=2 benchmark workspace — logs, plans, harness, converter
1ccdd58 boblaublaw committed 3 months ago