Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
serve: fix HTTP/1.1 keep-alive desync and SSE framing (#597 item 3)
595c7db ZacharyZcR committed 3 months ago
Merge pull request #543 from opxyc/feat/olmoe-safe-converter
cd6a342 Vincenzo Fornaro committed 3 months ago
Merge pull request #599 from lucasgrecco/feature/amd-rx-7900-xtx
7b03ce0 Vincenzo Fornaro committed 3 months ago
Merge pull request #600 from lBroth/fix/596-metal-sdk-floor
43c13ac Vincenzo Fornaro committed 3 months ago
Merge pull request #541 from opxyc/feat/olmoe-avx2-int8-dot
0d1f658 Vincenzo Fornaro committed 3 months ago
Merge pull request #608 from ZacharyZcR/fix/595-nix-flake-binary-name
aa6a279 Vincenzo Fornaro committed 3 months ago
Merge pull request #609 from JustVugg/fix/597-context-accept-frame
c5aac27 Vincenzo Fornaro committed 3 months ago
serve: ACCEPT frame so an oversized prompt is a clean 400, not a broken 200 stream (#597 item 6)
b4244f1 JustVugg committed 3 months ago
Merge pull request #607 from JustVugg/fix/597-glm-reasoning-content
cf56052 Vincenzo Fornaro committed 3 months ago
serve: stream GLM reasoning through reasoning_content on the OpenAI endpoint (#597 item 4)
b2b50e9 JustVugg committed 3 months ago
Merge pull request #606 from JustVugg/fix/597-serve-allowlist-keepalive
cc0fd12 Vincenzo Fornaro committed 3 months ago
serve: reverse-proxy Host allowlist + quiet prefill keepalive (#597)
594c87e JustVugg committed 3 months ago
Merge pull request #570 from ghostnumber7/build/portable-cuda-bat
26629e7 Vincenzo Fornaro committed 4 months ago
Merge pull request #593 from bherald/fix/windows-mirror-direct-io
f381d4c Vincenzo Fornaro committed 4 months ago
Merge pull request #535 from bherald/contrib/openai-stop-sequences
11cd1c6 Vincenzo Fornaro committed 4 months ago
Merge pull request #571 from EgonRuiter/feature/olmoe-lfru-eviction-guard
ea13b5c Vincenzo Fornaro committed 4 months ago
Merge pull request #576 from NeuralNotwerk/fix/serve-finish-reason-cap-hit
047968a Vincenzo Fornaro committed 4 months ago
Merge pull request #581 from ZacharyZcR/feat/serve-single-slot-mtp
de62a18 Vincenzo Fornaro committed 4 months ago
Merge pull request #579 from ZacharyZcR/fix/converter-default-gs64
e24995a Vincenzo Fornaro committed 4 months ago
Merge pull request #577 from ZacharyZcR/fix/575-version-install
e04bbdc Vincenzo Fornaro committed 4 months ago
Merge pull request #582 from ZacharyZcR/feat/docker-multistage
6352a4e Vincenzo Fornaro committed 4 months ago
Merge pull request #578 from ZacharyZcR/feat/ship-expert-atlas
5dc736a Vincenzo Fornaro committed 4 months ago
docker: add slim multi-stage image (Dockerfile.slim) + compose — 115MB non-root portable build alongside the beginner Dockerfile
aa65e21 ZacharyZcR committed 4 months ago
serve: single-slot MTP speculation in run_serve_mux (KV_SLOTS=1) — #492/#358
7ceb3a0 ZacharyZcR committed 4 months ago