Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1493 from JustVugg/fix/image-url-local-paths
1efb7c8 Vincenzo Fornaro committed 2 months ago
Merge pull request #1494 from JustVugg/feat/v4-prof-phases
9c7644c Vincenzo Fornaro committed 2 months ago
test_chat_images: do not assume the drive letter is part of the matched path
2dbd627 JustVugg committed 2 months ago
deepseek_v4: PROF reports block time, head time and forwards
f14ca10 JustVugg committed 2 months ago
server: local image paths need COLI_IMAGE_ROOT; coli chat sends data URIs
b1a7b8d JustVugg committed 2 months ago
Merge pull request #1488 from JustVugg/fix/convert-refuses-checkpoint-dir
1679a2f Vincenzo Fornaro committed 2 months ago
Merge pull request #1487 from JustVugg/fix/st-missing-diagnosis
a459928 Vincenzo Fornaro committed 2 months ago
Merge pull request #1486 from JustVugg/fix/no-config-no-glm
3874d71 Vincenzo Fornaro committed 2 months ago
coli convert: refuse an output directory that already holds a checkpoint
41b4544 JustVugg committed 2 months ago
st.h: a missing core tensor is diagnosed through the checkpoint's index
88082cc JustVugg committed 2 months ago
Merge pull request #1482 from JustVugg/feat/chat-stats
e155c73 Vincenzo Fornaro committed 2 months ago
Merge pull request #1481 from JustVugg/fix/glm53-stop-ids
035699f Vincenzo Fornaro committed 2 months ago
Merge pull request #1480 from JustVugg/fix/st-overlay-index
b22d83b Vincenzo Fornaro committed 2 months ago
fix(rans_verify): a stdout that cannot encode U+2014 loses the verdict
53748e1 kevin9327 committed 2 months ago
coli chat: configurable generation statistics with exact token throughput
d12d449 JustVugg committed 2 months ago
fix(serve): olmoe and inkling refuse an over-long prompt with CONTEXT_EXCEEDED
83d47b1 kevin9327 committed 2 months ago
glm53: stop ids from config.json too, and the converter keeps them
d799354 JustVugg committed 2 months ago
st.h: resolve duplicate tensor names through model.safetensors.index.json
57e5115 JustVugg committed 2 months ago
qwen38: trunk offload measured, CPU int8 reference, trunk on the fake tier, planner prices the trunk
1b4d19a kreuzzelg committed 2 months ago
backend: the resident dense matvec gets its own staging buffers
ad487ef kreuzzelg committed 2 months ago
qwen38: dense trunk matrices as int8 residents on the GPU (stage 1), generic tier dense API
19ab934 kreuzzelg committed 2 months ago
docs: qwen38 GPU section -- measured decode figures, honest verdict
eeff443 kreuzzelg committed 2 months ago
docs: Qwen3.6 accounting was already exact (512 KiB matrices)
1db800c kreuzzelg committed 2 months ago