Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
Commits on main
docs(streaming): document warm_willneed as inert under prod_k8()
08fcb46 Demetrios Agourakis committed 1 month ago
Merge pull request #112 from Edge0-AI/feat/prefill-ondemand-cli
fb4cd2c linyubupa committed 2 months ago
Merge pull request #111 from Edge0-AI/fix/8b-prefill-full-layer-noop
afaf9d0 linyubupa committed 2 months ago
fix(prefill): honor full_layer_prefill=False on the streaming engines
62291e1 linyubupa committed 2 months ago
docs(readme): add the ModelScope mirrors next to the Hugging Face links
f8c4cb2 linyubupa committed 2 months ago
docs(paper): add Edge0 technical report (The Other Half of the Memory Wall)
ca40e7d linyubupa committed 2 months ago
Merge pull request #102 from linyubupa/feat/streaming-willneed-readahead
c18eb62 linyubupa committed 2 months ago
perf(streaming): WILLNEED readahead + in-place staged stack for edge0-35b
62dcab2 bupalinyu committed 2 months ago
Merge pull request #99 from Edge0-AI/fix/35b-nan-hidden-clip
ba4597b linyubupa committed 2 months ago
Merge pull request #98 from Edge0-AI/fix/mlx-0.30.6-nax-a18
9a83488 linyubupa committed 2 months ago
Merge pull request #97 from Edge0-AI/fix/prerouter-switch-and-ling-sync
217725b linyubupa committed 2 months ago
Merge pull request #95 from Edge0-AI/fix/prerouter-switch-and-ling-sync
62f4bed linyubupa committed 2 months ago
fix(prerouter): zero-drop layer scoping, working --no-prerouter, 8b sync
b57d83a linyubupa committed 2 months ago
Merge pull request #14 from agourakis82/fix/hot-stack-overflow-row
ae1ee2d linyubupa committed 2 months ago
fix(streaming): give the hot stack its zero overflow row
1e6f8c6 Demetrios Agourakis committed 2 months ago
Merge pull request #12 from Edge0-AI/fix/35b-nan-hidden-clip
03a5a15 linyubupa committed 2 months ago
fix(server): keep the rendered chat template in ChatSession.prompt_ids
a8392a8 linyubupa committed 2 months ago
fix(35b): clip per-layer hidden states so one fp16 overflow cannot NaN-poison the net
51b8697 linyubupa committed 2 months ago