Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

refactor(repo): adopt the multi-platform layout

Bincad29a1e1d32
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9c3a3ad..78cfea2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -9,6 +9,9 @@ jobs:
test-macos:
name: Unit tests (macOS + MLX)
runs-on: macos-14
+ defaults:
+ run:
+ working-directory: python
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
@@ -22,6 +25,9 @@ jobs:
hygiene:
name: Repo hygiene (pure Python, no MLX)
runs-on: ubuntu-latest
+ defaults:
+ run:
+ working-directory: python
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
diff --git a/.gitignore b/.gitignore
index c3b7432..b41702c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -18,4 +18,4 @@ artifacts/
# Bench / runtime logs
*.log
-scripts/alignment_results.json
+python/scripts/alignment_results.json
diff --git a/android/README.md b/android/README.md
new file mode 100644
index 0000000..6068ea4
--- /dev/null
+++ b/android/README.md
@@ -0,0 +1,23 @@
+# edge0 — Android runtime
+
+> ✅ Open-sourced on **2026-09-30**. This README will be updated with
+> full build & usage docs together with the source drop.
+
+The Android runtime runs the edge0 models fully on-device on Android
+(Kotlin app + native inference engine): local streaming MoE inference
+with the same recipe as the rest of the framework — SSD expert offload,
+Recover-LoRA, and prerouter routing prediction.
+
+The next milestone is the **unified inference framework** — one access
+layer, runtime auto-adapting to the hardware platform (iOS / macOS /
+Android / Windows / Python) — targeted for the **end of October 2026**;
+see the [roadmap](../README.md#-roadmap).
+
+See the [repository README](../README.md) for the multi-platform picture.
+
+---
+
+> ✅ 源码已于 **2026-09-30** 开源,本 README 将随源码落库补全完整的构建
+> 与使用文档。Android runtime:Kotlin App + 原生推理引擎的端侧推理。
+> **统一推理框架**将于 **2026 年 10 月底**发布,详见
+> [根 README 路线图](../README_zh.md)。
diff --git a/docs/adding-a-model.md b/docs/adding-a-model.md
index 6af69b4..f14ac8c 100644
--- a/docs/adding-a-model.md
+++ b/docs/adding-a-model.md
@@ -42,7 +42,7 @@ The subclass only needs to implement the class method `_defaults(model_dir) -> C
Key points:
-- The repository root ships `scripts/convert_adapters_legacy.py`, which one-shot converts the training npz exports into safetensors artifacts under `artifacts/`; `ModelConfig.artifact(name)` (`models/base.py:28–30`) returns the absolute paths of those artifacts, and the adapter uses them to fill in the LoRA / prerouter weight paths.
+- The Python project root (`python/`) ships `scripts/convert_adapters_legacy.py`, which one-shot converts the training npz exports into safetensors artifacts under `artifacts/`; `ModelConfig.artifact(name)` (`models/base.py:28–30`) returns the absolute paths of those artifacts, and the adapter uses them to fill in the LoRA / prerouter weight paths.
- LoRA overrides go through `resolve_lora` (`models/base.py:88–95`): a bare model name resolves to that tier's artifacts, `"model_dir"` means "keep the training weights in place", and an empty string disables it.
## Step 3: `build_model` / `build_engine`
diff --git a/docs/prerouter.md b/docs/prerouter.md
index 460a1da..8744fa4 100644
--- a/docs/prerouter.md
+++ b/docs/prerouter.md
@@ -73,7 +73,7 @@ per `start_layer`/`owners`. Family differences are expressed through three hooks
| weight file | `prerouter_edge0_35b.safetensors` | `prerouter_edge0_8b.safetensors` |
Both weight files are resolved from the model directory by default, with
-`artifacts/` (repo root, gitignored) as the fallback — see the README's
+`artifacts/` (Python project root, gitignored) as the fallback — see the README's
"Models and adapters" section.
## Tests and regression
diff --git a/ios/README.md b/ios/README.md
new file mode 100644
index 0000000..389fa70
--- /dev/null
+++ b/ios/README.md
@@ -0,0 +1,23 @@
+# edge0 — iOS runtime
+
+> ✅ Open-sourced on **2026-09-30**. This README will be updated with
+> full build & usage docs together with the source drop.
+
+The iOS runtime runs the edge0 models fully on-device on iPhone
+(Swift / SwiftUI + MLX Swift): local streaming MoE inference with the
+same recipe as the rest of the framework — SSD expert offload,
+Recover-LoRA, and prerouter routing prediction.
+
+The next milestone is the **unified inference framework** — one access
+layer, runtime auto-adapting to the hardware platform (iOS / macOS /
+Android / Windows / Python) — targeted for the **end of October 2026**;
+see the [roadmap](../README.md#-roadmap).
+
+See the [repository README](../README.md) for the multi-platform picture.
+
+---
+
+> ✅ 源码已于 **2026-09-30** 开源,本 README 将随源码落库补全完整的构建
+> 与使用文档。iOS runtime:Swift / SwiftUI + MLX Swift 的 iPhone 端侧
+> 推理。**统一推理框架**将于 **2026 年 10 月底**发布,详见
+> [根 README 路线图](../README_zh.md)。
diff --git a/macos/README.md b/macos/README.md
new file mode 100644
index 0000000..35e7ddd
--- /dev/null
+++ b/macos/README.md
@@ -0,0 +1,24 @@
+# edge0 — macOS runtime
+
+> ✅ Open-sourced on **2026-09-30**. This README will be updated with
+> full build & usage docs together with the source drop.
+
+The macOS runtime is the local CLI / daemon / desktop app for edge0 on
+Apple Silicon (Rust + Tauri): an Ollama-style UX that pulls the edge0
+models ([Edge0-35B-A3B-preview](https://huggingface.co/Edge0/Edge0-35B-A3B-preview),
+[Edge0-8B-A1B-preview](https://huggingface.co/Edge0/Edge0-8B-A1B-preview))
+and serves them locally behind an OpenAI-compatible API.
+
+The next milestone is the **unified inference framework** — one access
+layer, runtime auto-adapting to the hardware platform (iOS / macOS /
+Android / Windows / Python) — targeted for the **end of October 2026**;
+see the [roadmap](../README.md#-roadmap).
+
+See the [repository README](../README.md) for the multi-platform picture.
+
+---
+
+> ✅ 源码已于 **2026-09-30** 开源,本 README 将随源码落库补全完整的构建
+> 与使用文档。macOS runtime:Rust + Tauri 的本地 CLI / daemon / 桌面
+> App,Ollama 式体验 + OpenAI 兼容 API。**统一推理框架**将于 **2026 年
+> 10 月底**发布,详见[根 README 路线图](../README_zh.md)。
diff --git a/python/README.md b/python/README.md
new file mode 100644
index 0000000..2bee612
--- /dev/null
+++ b/python/README.md
@@ -0,0 +1,24 @@
+# edge0 — Python framework
+
+The Python implementation of the edge0 streaming MoE inference framework
+(SSD expert offload + Recover-LoRA + prerouter routing prediction),
+shipping the MLX backend for Apple Silicon.
+
+This directory is one platform of the edge0 multi-platform repo — see the
+[repository README](../README.md) for the full picture (macOS / iOS /
+Android runtimes and the unified-framework roadmap).
+
+## Quick start
+
+```bash
+python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'
+edge0 demo edge0-8b # after downloading a model; see ../README.md
+```
+
+- Docs: [`docs/`](../docs/) at the repo root (architecture, attention, MoE, streaming, prerouter)
+- Examples: [`examples/`](examples/) · Scripts: [`scripts/`](scripts/)
+- Tests: `pytest` (unit) · `pytest -m slow` (real weights)
+
+## License
+
+Apache-2.0, including vendored third-party code (see [NOTICE](../NOTICE)).
diff --git a/examples/bench.py b/python/examples/bench.py
similarity index 100%
rename from examples/bench.py
rename to python/examples/bench.py
diff --git a/examples/demo.py b/python/examples/demo.py
similarity index 100%
rename from examples/demo.py
rename to python/examples/demo.py
diff --git a/pyproject.toml b/python/pyproject.toml
similarity index 100%
rename from pyproject.toml
rename to python/pyproject.toml
diff --git a/scripts/convert_adapters_legacy.py b/python/scripts/convert_adapters_legacy.py
similarity index 100%
rename from scripts/convert_adapters_legacy.py
rename to python/scripts/convert_adapters_legacy.py
diff --git a/scripts/e2e_smoke.py b/python/scripts/e2e_smoke.py
similarity index 100%
rename from scripts/e2e_smoke.py
rename to python/scripts/e2e_smoke.py
diff --git a/scripts/fetch_models.py b/python/scripts/fetch_models.py
similarity index 100%
rename from scripts/fetch_models.py
rename to python/scripts/fetch_models.py
diff --git a/scripts/generate_example.py b/python/scripts/generate_example.py
similarity index 100%
rename from scripts/generate_example.py
rename to python/scripts/generate_example.py
diff --git a/scripts/strip_vision_weights.py b/python/scripts/strip_vision_weights.py
similarity index 100%
rename from scripts/strip_vision_weights.py
rename to python/scripts/strip_vision_weights.py
diff --git a/scripts/upload_hf.py b/python/scripts/upload_hf.py
similarity index 100%
rename from scripts/upload_hf.py
rename to python/scripts/upload_hf.py
diff --git a/scripts/verify_alignment.py b/python/scripts/verify_alignment.py
similarity index 100%
rename from scripts/verify_alignment.py
rename to python/scripts/verify_alignment.py
diff --git a/src/edge0/__init__.py b/python/src/edge0/__init__.py
similarity index 100%
rename from src/edge0/__init__.py
rename to python/src/edge0/__init__.py
diff --git a/src/edge0/__main__.py b/python/src/edge0/__main__.py
similarity index 100%
rename from src/edge0/__main__.py
rename to python/src/edge0/__main__.py
diff --git a/src/edge0/adapters/__init__.py b/python/src/edge0/adapters/__init__.py
similarity index 100%
rename from src/edge0/adapters/__init__.py
rename to python/src/edge0/adapters/__init__.py
diff --git a/src/edge0/adapters/lora.py b/python/src/edge0/adapters/lora.py
similarity index 100%
rename from src/edge0/adapters/lora.py
rename to python/src/edge0/adapters/lora.py
diff --git a/src/edge0/attention/spec.py b/python/src/edge0/attention/spec.py
similarity index 100%
rename from src/edge0/attention/spec.py
rename to python/src/edge0/attention/spec.py
diff --git a/src/edge0/backends/__init__.py b/python/src/edge0/backends/__init__.py
similarity index 100%
rename from src/edge0/backends/__init__.py
rename to python/src/edge0/backends/__init__.py
diff --git a/src/edge0/backends/base.py b/python/src/edge0/backends/base.py
similarity index 100%
rename from src/edge0/backends/base.py
rename to python/src/edge0/backends/base.py
diff --git a/src/edge0/backends/mlx/__init__.py b/python/src/edge0/backends/mlx/__init__.py
similarity index 100%
rename from src/edge0/backends/mlx/__init__.py
rename to python/src/edge0/backends/mlx/__init__.py
diff --git a/src/edge0/backends/mlx/_impl/__init__.py b/python/src/edge0/backends/mlx/_impl/__init__.py
similarity index 100%
rename from src/edge0/backends/mlx/_impl/__init__.py
rename to python/src/edge0/backends/mlx/_impl/__init__.py
diff --git a/src/edge0/backends/mlx/_impl/bailing_hybrid.py b/python/src/edge0/backends/mlx/_impl/bailing_hybrid.py
similarity index 100%
rename from src/edge0/backends/mlx/_impl/bailing_hybrid.py
rename to python/src/edge0/backends/mlx/_impl/bailing_hybrid.py
diff --git a/src/edge0/backends/mlx/_impl/qwen3_5.py b/python/src/edge0/backends/mlx/_impl/qwen3_5.py
similarity index 100%
rename from src/edge0/backends/mlx/_impl/qwen3_5.py
rename to python/src/edge0/backends/mlx/_impl/qwen3_5.py
diff --git a/src/edge0/backends/mlx/_impl/qwen3_5_moe.py b/python/src/edge0/backends/mlx/_impl/qwen3_5_moe.py
similarity index 100%
rename from src/edge0/backends/mlx/_impl/qwen3_5_moe.py
rename to python/src/edge0/backends/mlx/_impl/qwen3_5_moe.py
diff --git a/src/edge0/backends/mlx/_impl/qwen3_next.py b/python/src/edge0/backends/mlx/_impl/qwen3_next.py
similarity index 100%
rename from src/edge0/backends/mlx/_impl/qwen3_next.py
rename to python/src/edge0/backends/mlx/_impl/qwen3_next.py
diff --git a/src/edge0/backends/mlx/backend.py b/python/src/edge0/backends/mlx/backend.py
similarity index 100%
rename from src/edge0/backends/mlx/backend.py
rename to python/src/edge0/backends/mlx/backend.py
diff --git a/src/edge0/backends/mlx/io.py b/python/src/edge0/backends/mlx/io.py
similarity index 100%
rename from src/edge0/backends/mlx/io.py
rename to python/src/edge0/backends/mlx/io.py
diff --git a/src/edge0/backends/mlx/quant.py b/python/src/edge0/backends/mlx/quant.py
similarity index 100%
rename from src/edge0/backends/mlx/quant.py
rename to python/src/edge0/backends/mlx/quant.py
diff --git a/src/edge0/cli.py b/python/src/edge0/cli.py
similarity index 99%
rename from src/edge0/cli.py
rename to python/src/edge0/cli.py
index 97555d8..d2cbb73 100644
--- a/src/edge0/cli.py
+++ b/python/src/edge0/cli.py
@@ -211,9 +211,9 @@ def cmd_serve(args) -> int:
def cmd_convert(args) -> int:
import runpy
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve()
- .parents[1] / "scripts"))
+ .parents[2] / "scripts"))
runpy.run_path(
- str(__import__("pathlib").Path(__file__).resolve().parents[1]
+ str(__import__("pathlib").Path(__file__).resolve().parents[2]
/ "scripts" / "convert_adapters_legacy.py"),
run_name="__main__",
)
diff --git a/src/edge0/config.py b/python/src/edge0/config.py
similarity index 100%
rename from src/edge0/config.py
rename to python/src/edge0/config.py
diff --git a/src/edge0/engine/__init__.py b/python/src/edge0/engine/__init__.py
similarity index 100%
rename from src/edge0/engine/__init__.py
rename to python/src/edge0/engine/__init__.py
diff --git a/src/edge0/engine/base.py b/python/src/edge0/engine/base.py
similarity index 100%
rename from src/edge0/engine/base.py
rename to python/src/edge0/engine/base.py
diff --git a/src/edge0/engine/hooks.py b/python/src/edge0/engine/hooks.py
similarity index 100%
rename from src/edge0/engine/hooks.py
rename to python/src/edge0/engine/hooks.py
diff --git a/src/edge0/engine/ling.py b/python/src/edge0/engine/ling.py
similarity index 100%
rename from src/edge0/engine/ling.py
rename to python/src/edge0/engine/ling.py
diff --git a/src/edge0/engine/qwen.py b/python/src/edge0/engine/qwen.py
similarity index 100%
rename from src/edge0/engine/qwen.py
rename to python/src/edge0/engine/qwen.py
diff --git a/src/edge0/models/__init__.py b/python/src/edge0/models/__init__.py
similarity index 100%
rename from src/edge0/models/__init__.py
rename to python/src/edge0/models/__init__.py
diff --git a/src/edge0/models/base.py b/python/src/edge0/models/base.py
similarity index 99%
rename from src/edge0/models/base.py
rename to python/src/edge0/models/base.py
index d3dead5..bb1d73d 100644
--- a/src/edge0/models/base.py
+++ b/python/src/edge0/models/base.py
@@ -19,7 +19,7 @@ from edge0.moe.spec import MoESpec
from edge0.prerouter.spec import PrerouterSpec
from edge0.streaming.options import LayerOptions
-# Repository-root artifacts (gitignored; produced once by
+# Project-root artifacts (gitignored; produced once by
# scripts/convert_adapters_legacy.py from the training npz exports, then
# the source npz are discarded).
ARTIFACTS_DIR = Path(__file__).resolve().parents[3] / "artifacts"
diff --git a/src/edge0/models/edge0_35b/__init__.py b/python/src/edge0/models/edge0_35b/__init__.py
similarity index 100%
rename from src/edge0/models/edge0_35b/__init__.py
rename to python/src/edge0/models/edge0_35b/__init__.py
diff --git a/src/edge0/models/edge0_8b/__init__.py b/python/src/edge0/models/edge0_8b/__init__.py
similarity index 100%
rename from src/edge0/models/edge0_8b/__init__.py
rename to python/src/edge0/models/edge0_8b/__init__.py
diff --git a/src/edge0/moe/__init__.py b/python/src/edge0/moe/__init__.py
similarity index 100%
rename from src/edge0/moe/__init__.py
rename to python/src/edge0/moe/__init__.py
diff --git a/src/edge0/moe/routing.py b/python/src/edge0/moe/routing.py
similarity index 100%
rename from src/edge0/moe/routing.py
rename to python/src/edge0/moe/routing.py
diff --git a/src/edge0/moe/spec.py b/python/src/edge0/moe/spec.py
similarity index 100%
rename from src/edge0/moe/spec.py
rename to python/src/edge0/moe/spec.py
diff --git a/src/edge0/prerouter/__init__.py b/python/src/edge0/prerouter/__init__.py
similarity index 100%
rename from src/edge0/prerouter/__init__.py
rename to python/src/edge0/prerouter/__init__.py
diff --git a/src/edge0/prerouter/heads.py b/python/src/edge0/prerouter/heads.py
similarity index 100%
rename from src/edge0/prerouter/heads.py
rename to python/src/edge0/prerouter/heads.py
diff --git a/src/edge0/prerouter/install.py b/python/src/edge0/prerouter/install.py
similarity index 100%
rename from src/edge0/prerouter/install.py
rename to python/src/edge0/prerouter/install.py
diff --git a/src/edge0/prerouter/spec.py b/python/src/edge0/prerouter/spec.py
similarity index 100%
rename from src/edge0/prerouter/spec.py
rename to python/src/edge0/prerouter/spec.py
diff --git a/src/edge0/prerouter/stager.py b/python/src/edge0/prerouter/stager.py
similarity index 100%
rename from src/edge0/prerouter/stager.py
rename to python/src/edge0/prerouter/stager.py
diff --git a/src/edge0/prerouter/state.py b/python/src/edge0/prerouter/state.py
similarity index 100%
rename from src/edge0/prerouter/state.py
rename to python/src/edge0/prerouter/state.py
diff --git a/src/edge0/registry.py b/python/src/edge0/registry.py
similarity index 100%
rename from src/edge0/registry.py
rename to python/src/edge0/registry.py
diff --git a/src/edge0/sampling.py b/python/src/edge0/sampling.py
similarity index 100%
rename from src/edge0/sampling.py
rename to python/src/edge0/sampling.py
diff --git a/src/edge0/server/__init__.py b/python/src/edge0/server/__init__.py
similarity index 100%
rename from src/edge0/server/__init__.py
rename to python/src/edge0/server/__init__.py
diff --git a/src/edge0/server/app.py b/python/src/edge0/server/app.py
similarity index 100%
rename from src/edge0/server/app.py
rename to python/src/edge0/server/app.py
diff --git a/src/edge0/server/chat.py b/python/src/edge0/server/chat.py
similarity index 100%
rename from src/edge0/server/chat.py
rename to python/src/edge0/server/chat.py
diff --git a/src/edge0/streaming/__init__.py b/python/src/edge0/streaming/__init__.py
similarity index 100%
rename from src/edge0/streaming/__init__.py
rename to python/src/edge0/streaming/__init__.py
diff --git a/src/edge0/streaming/cache.py b/python/src/edge0/streaming/cache.py
similarity index 100%
rename from src/edge0/streaming/cache.py
rename to python/src/edge0/streaming/cache.py
diff --git a/src/edge0/streaming/install.py b/python/src/edge0/streaming/install.py
similarity index 100%
rename from src/edge0/streaming/install.py
rename to python/src/edge0/streaming/install.py
diff --git a/src/edge0/streaming/layer.py b/python/src/edge0/streaming/layer.py
similarity index 100%
rename from src/edge0/streaming/layer.py
rename to python/src/edge0/streaming/layer.py
diff --git a/src/edge0/streaming/mmap.py b/python/src/edge0/streaming/mmap.py
similarity index 100%
rename from src/edge0/streaming/mmap.py
rename to python/src/edge0/streaming/mmap.py
diff --git a/src/edge0/streaming/options.py b/python/src/edge0/streaming/options.py
similarity index 100%
rename from src/edge0/streaming/options.py
rename to python/src/edge0/streaming/options.py
diff --git a/tests/test_cli_flags.py b/python/tests/test_cli_flags.py
similarity index 100%
rename from tests/test_cli_flags.py
rename to python/tests/test_cli_flags.py
diff --git a/tests/test_e2e_slow.py b/python/tests/test_e2e_slow.py
similarity index 100%
rename from tests/test_e2e_slow.py
rename to python/tests/test_e2e_slow.py
diff --git a/tests/test_moe_spec.py b/python/tests/test_moe_spec.py
similarity index 100%
rename from tests/test_moe_spec.py
rename to python/tests/test_moe_spec.py
diff --git a/tests/test_prefill_hook.py b/python/tests/test_prefill_hook.py
similarity index 100%
rename from tests/test_prefill_hook.py
rename to python/tests/test_prefill_hook.py
diff --git a/tests/test_registry.py b/python/tests/test_registry.py
similarity index 100%
rename from tests/test_registry.py
rename to python/tests/test_registry.py
diff --git a/tests/test_repo_hygiene.py b/python/tests/test_repo_hygiene.py
similarity index 91%
rename from tests/test_repo_hygiene.py
rename to python/tests/test_repo_hygiene.py
index c8ef1d0..a6ed8ef 100644
--- a/tests/test_repo_hygiene.py
+++ b/python/tests/test_repo_hygiene.py
@@ -16,13 +16,15 @@ from __future__ import annotations
import pathlib
import re
-ROOT = pathlib.Path(__file__).resolve().parents[1]
+# Repo root (this file lives at <root>/python/tests/): the hygiene guards
+# scan the whole multi-platform repo, not just the Python subproject.
+ROOT = pathlib.Path(__file__).resolve().parents[2]
_EXCLUDE_DIRS = {".git", ".venv", ".pytest_cache", "__pycache__", ".agents",
".codex"}
_EXCLUDE_PARTS = {"egg-info"}
_EXCLUDE_SUFFIXES = {".pyc", ".so", ".dylib", ".bin", ".safetensors", ".npz",
- ".png", ".jpg", ".jpeg", ".gif", ".ico"}
+ ".png", ".jpg", ".jpeg", ".gif", ".ico", ".pdf"}
# NOTE: patterns are assembled with + so this file never contains the
# literal string it is looking for.
@@ -85,7 +87,7 @@ def test_no_hardcoded_local_paths():
def test_mlx_imports_stay_inside_the_backend():
bad = []
- for path in (ROOT / "src" / "edge0").rglob("*.py"):
+ for path in (ROOT / "python" / "src" / "edge0").rglob("*.py"):
rel = path.relative_to(ROOT)
text = path.read_text(encoding="utf-8")
if _MLX_IMPORT.search(text) and "backends/mlx" not in rel.as_posix():
diff --git a/tests/test_sampling.py b/python/tests/test_sampling.py
similarity index 100%
rename from tests/test_sampling.py
rename to python/tests/test_sampling.py
diff --git a/tests/test_server.py b/python/tests/test_server.py
similarity index 100%
rename from tests/test_server.py
rename to python/tests/test_server.py
diff --git a/tests/test_streaming_math.py b/python/tests/test_streaming_math.py
similarity index 100%
rename from tests/test_streaming_math.py
rename to python/tests/test_streaming_math.py