Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
Commits on main
Merge pull request #155 from Edge0-AI/release/android-gguf-import
d4a2fa0 Jiahao committed 21 days ago
Merge pull request #151 from agourakis82/feat/bench-submission-json-20261022
e9cc00c Jiahao committed 28 days ago
Merge pull request #150 from Edge0-AI/release/ios-documents-models
dc5e1bf Jiahao committed 28 days ago
feat(bench): add community submission and JSON output
6e67349 Demetrios Agourakis committed 28 days ago
Merge pull request #149 from Anu27n/fix/server-finish-reason-length
7a203f2 Jiahao committed 29 days ago
fix(server): report finish_reason "length" when max_tokens cuts a reply off
3707551 Anubhav Jain committed 1 month ago
Merge pull request #148 from kevin9327/fix/stream-trailing-replacement-char
305b459 Jiahao committed 1 month ago
Merge pull request #147 from kevin9327/fix/server-max-completion-tokens
70f3b29 Jiahao committed 1 month ago
Merge pull request #144 from ArdaEkiz0/windows-bench-submission
73f5a26 Jiahao committed 1 month ago
Merge pull request #145 from Raphaeal19/feat/android-benchmark-submission-block
412116a Jiahao committed 1 month ago
fix(server): flush a trailing U+FFFD at the end of a chat stream
7319438 kevin9327 committed 1 month ago
windows: parse the ggml_vulkan device line; clarify peak_working_set semantics
ed627fc Arda M. Ekiz committed 1 month ago
Merge pull request #143 from agourakis82/test/cli-routing-contract-local
8c2d422 Jiahao committed 1 month ago
Merge pull request #1 from shuk777/cursor/readme-discord-link-6553
8631c5e Jiahao committed 1 month ago
android: add an on-device benchmark that prints the community submission block
40f2bf1 Ayush Pathak committed 1 month ago
windows: one-command benchmark submission block (fixes #139)
258ce7b Arda M. Ekiz committed 1 month ago
test(cli): cover prerouter defaults at engine dispatch
85407d4 Demetrios Agourakis committed 1 month ago
Merge pull request #108 from VKVstudio/fix/hygiene-mlx-lm-regex
5b32e63 Jiahao committed 1 month ago
tests: make the MLX boundary guard catch mlx_lm imports
82e9f7a Valerii Karpov committed 1 month ago
Merge pull request #105 from ahrazzle/fix/sse-incremental-decode
d94af8a Jiahao committed 1 month ago
Merge pull request #117 from agourakis82/fix/strip-vision-total-size
5a12107 Jiahao committed 1 month ago