Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

docs(readme): polish EN/ZH READMEs and fix roadmap anchors

Bine76499258410
diff --git a/README.md b/README.md
index 110e96c..0f6ba4f 100644
--- a/README.md
+++ b/README.md
@@ -20,14 +20,14 @@ English | [中文](README_zh.md)
</div>
-## 📰 News
+## News
-- **[2026-09-30]** We released the **edge0 inference engines for three platforms — iOS, macOS and Android** — so users get the best inference experience across architectures and platforms. The source is open-sourced in this repo ([`ios/`](ios/) · [`macos/`](macos/) · [`android/`](android/)) — see each directory's README for details. The **unified inference framework** follows by the **end of October 2026**; see the [Roadmap](#-roadmap).
+- **[2026-09-30]** We released the **edge0 inference engines for three platforms — iOS, macOS and Android** — so users get the best inference experience across architectures and platforms. The source is open-sourced in this repo ([`ios/`](ios/) · [`macos/`](macos/) · [`android/`](android/)) — see each directory's README for details. The **unified inference framework** follows by the **end of October 2026**; see the [Roadmap](#roadmap).
- **[2026-09-20]** `--prefill-ondemand` lands for `edge0 demo / chat / serve` ([#112](https://github.com/Edge0-AI/Edge0/pull/112)).
- **[2026-09-16]** Our technical report is on arXiv: [The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction](https://arxiv.org/abs/2609.18063).
- **[2026-09-08]** Initial open-source release of **edge0**, together with both model tiers — [`Edge0-35B-A3B-preview`](https://huggingface.co/Edge0/Edge0-35B-A3B-preview) and [`Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview) — on Hugging Face and ModelScope.
-## 🤔 About
+## About
**edge0** is an open-source streaming MoE inference framework. It
generalizes the production-proven recipe — **SSD expert offload +
@@ -90,7 +90,7 @@ are co-located with each checkpoint and load automatically, so
zero changes to core code. The iOS / macOS / Android engines ship
platform-native stacks today — bringing every platform under one
access layer is exactly what the unified inference framework
- (see [Roadmap](#-roadmap)) will deliver;
+ (see [Roadmap](#roadmap)) will deliver;
- **Adapters as safetensors**: LoRA and prerouter weights are
`.safetensors` files with provenance metadata (source, version, owner
layers), resolved from the model directory or `artifacts/`;
@@ -138,7 +138,7 @@ python examples/bench.py edge0-35b # via $EDGE0_35B_MODEL
python examples/bench.py edge0-8b # via $EDGE0_8B_MODEL
```
-## 🚀 Getting Started
+## Getting Started
### Python (macOS · Apple Silicon)
@@ -226,7 +226,7 @@ edge0 serve edge0-35b
```bash
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
- -d '{"messages":[{"role":"user","content":"Hello!"}],"max_tokens":32}'
+ -d '{"model":"edge0-35b","messages":[{"role":"user","content":"Hello!"}],"max_tokens":32}'
# 5) One-shot chat (pass --max-new to cap length; add --show-thinking to
# print the model's reasoning block too)
@@ -294,9 +294,9 @@ in each directory's README:
The **unified inference framework** — one access layer, runtime
auto-adapting to iOS / macOS / Android / Windows / Python — arrives by
-the **end of October 2026**; see the [Roadmap](#-roadmap).
+the **end of October 2026**; see the [Roadmap](#roadmap).
-## 🗺️ Roadmap
+## Roadmap
- **[End of Oct 2026] edge0 unified inference framework** — we will
open-source a unified inference framework: **one unified access
@@ -310,7 +310,7 @@ the **end of October 2026**; see the [Roadmap](#-roadmap).
`python/src/edge0/backends/cuda/`, core code needs zero changes.
- More model tiers and adapter releases on the existing pipeline.
-## 🤝 Contributing
+## Contributing
Contributions are welcome — issues, PRs, benchmark reports and model
ports all count.
@@ -341,7 +341,7 @@ with the unified framework code.
Workflow: fork → feature branch → PR against `main`. Please keep the
hygiene suite green and add tests for new behavior.
-## 📖 Citation
+## Citation
If you find edge0 useful, please cite our technical report:
@@ -357,16 +357,14 @@ If you find edge0 useful, please cite our technical report:
}
```
-## 📧 Contact Us
+## Contact Us
<!-- TODO(edge0 team): fill in the official contact channels before publish. -->
Community and support channels are coming soon — this section will list
the official ways to reach us:
-- **Email**: _to be added_
-- **Discord**: _to be added_
-- **WeChat group**: _to be added_
+- **Email**: [email protected]
For bugs and feature requests, please use
[GitHub Issues](https://github.com/Edge0-AI/Edge0/issues).
diff --git a/README_zh.md b/README_zh.md
index 6631ef3..aeb9ce6 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -20,13 +20,13 @@
</div>
-## 📰 新闻
+## 新闻
- **【2026-09-30】** 我们发布了**三端推理引擎**,适配 iOS、macOS、Android
平台,让用户在不同架构、不同平台上都有最佳的推理体验。三端源码已
开源至本仓库([`ios/`](ios/) · [`macos/`](macos/) ·
[`android/`](android/)),更多细节见各目录 README;**统一推理框架**
- 将于 **2026 年 10 月底**发布,详见[路线图](#-路线图)。
+ 将于 **2026 年 10 月底**发布,详见[路线图](#路线图)。
- **【2026-09-20】** `edge0 demo / chat / serve` 新增 `--prefill-ondemand`
([#112](https://github.com/Edge0-AI/Edge0/pull/112))。
- **【2026-09-16】** 技术报告上线 arXiv:[The Other Half of the Memory
@@ -37,7 +37,7 @@
与 [`Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview)
同步登陆 Hugging Face 与 ModelScope。
-## 🤔 关于 edge0
+## 关于 edge0
**edge0** 是一个开源的流式 MoE 推理框架:把「SSD 专家 offload +
Recover-LoRA + prerouter 路由预判」抽象成可扩展的通用框架,让大型稀疏
@@ -91,7 +91,7 @@ MoE 模型跑在消费级硬件上——峰值内存由**激活**专家集而非
`nn` 门面),新增后端实现同一门面即可平级接入(`backends/cuda/`
预留插槽),核心代码零改动。iOS / macOS / Android 引擎目前是各自的
平台原生技术栈——把所有平台收进统一接入层,正是统一推理框架
- (见[路线图](#-路线图))要交付的内容;
+ (见[路线图](#路线图))要交付的内容;
- **适配器统一为 safetensors**:LoRA 与 prerouter 权重均为带元数据
(来源、版本、owner 层)的 `.safetensors`,放模型目录或 `artifacts/`
均可自动解析;
@@ -135,7 +135,7 @@ python examples/bench.py edge0-35b # 经 $EDGE0_35B_MODEL
python examples/bench.py edge0-8b # 经 $EDGE0_8B_MODEL
```
-## 🚀 快速开始
+## 快速开始
### Python(macOS · Apple Silicon)
@@ -222,7 +222,7 @@ edge0 serve edge0-35b
```bash
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
- -d '{"messages":[{"role":"user","content":"Hello!"}],"max_tokens":32}'
+ -d '{"model":"edge0-35b","messages":[{"role":"user","content":"Hello!"}],"max_tokens":32}'
# 5) 单轮对话(--max-new 限制生成长度;加 --show-thinking 会一并打印思考块)
edge0 chat edge0-35b --prompt "用一句话解释流式推理。"
@@ -285,9 +285,9 @@ engine.close() # 释放 mmap / 专家缓存
[`android/README.md`](android/README.md)
**统一推理框架**(接入层统一,runtime 自动适配 iOS / macOS / Android /
-Windows / Python)将于 **2026 年 10 月底**发布,见[路线图](#-路线图)。
+Windows / Python)将于 **2026 年 10 月底**发布,见[路线图](#路线图)。
-## 🗺️ 路线图
+## 路线图
- **【2026 年 10 月底】edge0 统一推理框架开源** —— 发布统一推理框架:
**接入层统一**为一套 API(chat / serve / 端侧共用),**runtime 自动
@@ -298,7 +298,7 @@ Windows / Python)将于 **2026 年 10 月底**发布,见[路线图](#-路线
`python/src/edge0/backends/cuda/`,核心代码零改动。
- 更多模型档位与适配器版本在现有管线上持续发布。
-## 🤝 贡献
+## 贡献
欢迎各种形式的贡献——issue、PR、性能实测报告、新模型接入都算。
@@ -327,7 +327,7 @@ examples/demo.py # 最小 API walkthrough(edge0 demo 的等价代
流程:fork → 功能分支 → 向 `main` 发 PR。请保持卫生检查通过,并为新
行为补测试。
-## 📖 引用
+## 引用
如果 edge0 对你有帮助,请引用我们的技术报告:
@@ -343,15 +343,13 @@ examples/demo.py # 最小 API walkthrough(edge0 demo 的等价代
}
```
-## 📧 联系我们
+## 联系我们
-<!-- TODO(edge0 团队):发布前填写官方联系渠道。 -->
+<!-- TODO(edge0 团队):如需补充更多渠道(Discord / 微信群等),在此添加。 -->
社区与支持渠道即将上线,本节将列出联系我们的官方方式:
-- **邮箱**:_待补充_
-- **Discord**:_待补充_
-- **微信群**:_待补充_
+- **邮箱**:[email protected]
Bug 与功能建议请直接提
[GitHub Issues](https://github.com/Edge0-AI/Edge0/issues)。
diff --git a/android/README.md b/android/README.md
index 6068ea4..9be16bd 100644
--- a/android/README.md
+++ b/android/README.md
@@ -11,7 +11,7 @@ Recover-LoRA, and prerouter routing prediction.
The next milestone is the **unified inference framework** — one access
layer, runtime auto-adapting to the hardware platform (iOS / macOS /
Android / Windows / Python) — targeted for the **end of October 2026**;
-see the [roadmap](../README.md#-roadmap).
+see the [roadmap](../README.md#roadmap).
See the [repository README](../README.md) for the multi-platform picture.
diff --git a/ios/README.md b/ios/README.md
index 389fa70..7fc05a3 100644
--- a/ios/README.md
+++ b/ios/README.md
@@ -11,7 +11,7 @@ Recover-LoRA, and prerouter routing prediction.
The next milestone is the **unified inference framework** — one access
layer, runtime auto-adapting to the hardware platform (iOS / macOS /
Android / Windows / Python) — targeted for the **end of October 2026**;
-see the [roadmap](../README.md#-roadmap).
+see the [roadmap](../README.md#roadmap).
See the [repository README](../README.md) for the multi-platform picture.
diff --git a/macos/README.md b/macos/README.md
index 35e7ddd..e4efb5c 100644
--- a/macos/README.md
+++ b/macos/README.md
@@ -12,7 +12,7 @@ and serves them locally behind an OpenAI-compatible API.
The next milestone is the **unified inference framework** — one access
layer, runtime auto-adapting to the hardware platform (iOS / macOS /
Android / Windows / Python) — targeted for the **end of October 2026**;
-see the [roadmap](../README.md#-roadmap).
+see the [roadmap](../README.md#roadmap).
See the [repository README](../README.md) for the multi-platform picture.