Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1160 from JustVugg/ci/metal-compile-gate
2c45694 Vincenzo Fornaro committed 3 months ago
Merge pull request #1156 from JustVugg/fix/web-reasoning-callback
d3960da Vincenzo Fornaro committed 3 months ago
Merge pull request #1151 from ZacharyZcR/feat/kv8-rebase
a7fb0ca Vincenzo Fornaro committed 3 months ago
KV_TQ: refuse row widths >512 (consumer staging buffers); kv8_sanitize_row: refuse negative scale
c240492 recviking committed 3 months ago
KV_TQ: refuse non-power-of-two row widths; KV_TQ<2 means the 4-bit tier
b7534d9 recviking committed 3 months ago
CPU: KV cache quantization — KV8 (fp8 e4m3) + KV_TQ (rotated-int4 / PolarQuant)
1119782 recviking committed 4 months ago
Merge pull request #1153 from JustVugg/fix/web-reasoning-stream
ffd3eb7 Vincenzo Fornaro committed 3 months ago
Merge pull request #1152 from JustVugg/fix/inkling-metal-qgs
eb7c946 Vincenzo Fornaro committed 3 months ago
fix(web): render the reasoning stream instead of dropping it (#1148)
001fcfe JustVugg committed 3 months ago
Merge pull request #1134 from tonnthuir/fix/rss-guard-trim
95079d9 Vincenzo Fornaro committed 3 months ago
Merge pull request #1107 from monotophic/feat/logprobs-engine-channel
701bc98 Vincenzo Fornaro committed 3 months ago
Linux current memory usage measurement falls back to peak when current is unavailable.
fdbccab James Whiting committed 3 months ago
Merge pull request #1144 from ZacharyZcR/feat/k3-tool-calling
e573b53 Vincenzo Fornaro committed 3 months ago
Merge pull request #1146 from ZacharyZcR/fix/1136-kernel-convergence
2856210 Vincenzo Fornaro committed 3 months ago
Merge pull request #1145 from ZacharyZcR/docs/1136-rows16-env-entry
6ce6858 Vincenzo Fornaro committed 3 months ago
fix(dsv4): reference matvec adopts the rows16 accumulation order (#1136)
13b8f16 ZacharyZcR committed 3 months ago
docs: promote COLI_V4_ROWS16 to an ENVIRONMENT.md table entry (#1136)
7b925a7 ZacharyZcR committed 3 months ago
Merge pull request #1127 from view321/perf/iobench-subMB-blocks-and-per-thread-fd
de6fe7e Vincenzo Fornaro committed 3 months ago
Merge pull request #1139 from SyedYousufFaizan/fix/datapoint-v4-parsing
47cb456 Vincenzo Fornaro committed 3 months ago
Merge pull request #1133 from QQSHI13/fix/dsv4-compress-ratios-length
833087b Vincenzo Fornaro committed 3 months ago
Merge branch 'dev' into perf/iobench-subMB-blocks-and-per-thread-fd
61d7727 Vincenzo Fornaro committed 3 months ago
Merge pull request #1142 from ZacharyZcR/fix/768-kvb-flash
0321b9e Vincenzo Fornaro committed 3 months ago