Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
Merge pull request #1676 from ZacharyZcR/feat/kimi-cuda-situ-pipeline
a926d1b Vincenzo Fornaro committed 1 month ago
Merge pull request #1691 from JustVugg/fix/launcher-realpath
d1c2bca Vincenzo Fornaro committed 1 month ago
ci(qwen36), docs: the mixed expert layout on the tiny fixture, and the #1370 numbers in "Which container?"
e1d51be kreuzzelg committed 2 months ago
qwen36: read the mixed expert layout (int4 gate/up, int8 down) on the CPU path
87d4139 kreuzzelg committed 2 months ago
tools(qwen36): --down-bits, the mixed expert layout (int4 gate/up, int8 down in one slab)
5fbf3a6 kreuzzelg committed 2 months ago
Merge pull request #1681 from XBold/docs/qwen38-remove-stale-gpu-backend-claims
3961574 Vincenzo Fornaro committed 1 month ago
Merge pull request #1688 from ZacharyZcR/feat/http-serving-benchmark
ed7f535 Vincenzo Fornaro committed 1 month ago
Merge pull request #1687 from ZacharyZcR/feat/serve-prometheus-metrics
a4e18f9 Vincenzo Fornaro committed 1 month ago
Merge pull request #1686 from DebugSultan/prefill-batching
52be8e1 Vincenzo Fornaro committed 1 month ago
coli: resolve the launcher's own symlink before deriving libexec (#1689)
b197c56 JustVugg committed 1 month ago
Merge dev into feat/kimi-cuda-situ-pipeline: keep both MXFP4 scratch sets, both cuda-test additions
7edf6eb JustVugg committed 1 month ago
Merge pull request #1677 from ZacharyZcR/perf/qwen36-dnproj-prefill
94cef8c Vincenzo Fornaro committed 1 month ago
Merge pull request #1674 from ZacharyZcR/perf/qwen36-batch-dense
ae48037 Vincenzo Fornaro committed 1 month ago
Merge pull request #1678 from ZacharyZcR/fix/qwen-cuda-tier-shutdown
f969118 Vincenzo Fornaro committed 1 month ago
Merge pull request #1684 from ZacharyZcR/fix/cuda-context-init-unwind
bc54327 Vincenzo Fornaro committed 1 month ago
Merge pull request #1683 from ZacharyZcR/fix/qwen-cuda-init-cleanup
6a25042 Vincenzo Fornaro committed 1 month ago
Merge pull request #1682 from ZacharyZcR/fix/qwen-cuda-take-error
62f6a47 Vincenzo Fornaro committed 1 month ago
Merge pull request #1680 from ZacharyZcR/fix/qwen-cuda-scale-budget
86f4567 Vincenzo Fornaro committed 1 month ago
Merge pull request #1679 from ZacharyZcR/fix/qwen-cuda-upload-rollback
ccf32ec Vincenzo Fornaro committed 1 month ago
Merge pull request #1675 from ZacharyZcR/fix/cuda-mxfp4-scale-layout
8ca90cd Vincenzo Fornaro committed 1 month ago
Merge pull request #1673 from ZacharyZcR/perf/cuda-mxfp4-scratch
3c0b387 Vincenzo Fornaro committed 1 month ago
Merge pull request #1671 from ZacharyZcR/fix/v41-output-budget
f24abbc Vincenzo Fornaro committed 1 month ago
Merge pull request #1670 from ZacharyZcR/fix/1666-v41-ram-budget
9f49a80 Vincenzo Fornaro committed 1 month ago
Merge pull request #1669 from ZacharyZcR/fix/systemone-test-discovery
73f6a6b Vincenzo Fornaro committed 1 month ago
Stop stream keepalive threads on generation errors and cancellation
0d91cdc ZacharyZcR committed 1 month ago