Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Contributing to edge0
Thanks for your interest in contributing! edge0 is an open-source streaming MoE inference framework, and contributions of all kinds are welcome — bug reports, performance measurements, documentation fixes, new model support, and new platform/backend work.
This is a multi-platform monorepo. Please read the section for the part you're touching.
Ways to contribute
- Report a bug / performance issue / request a feature or model — open an
issue using one of the issue forms. The forms ask
for the platform, chip, memory, and (for the Python runtime) the
mlxversion — these fields resolve the large majority of reports, so please fill them in. - Ask a question or share results — please use GitHub Discussions rather than an issue.
- Send a pull request — see the workflow below.
Repository layout
| Path | What | Stack |
|---|---|---|
python/ |
Reference framework + CLI + OpenAI-compatible server | Python + MLX (Apple Silicon) |
macos/ |
Desktop app / CLI / daemon | Rust |
ios/ |
iPhone on-device app | Swift + MLX Swift |
android/ |
On-device app + native engine | Kotlin + native |
windows/ |
Desktop app + native engine | C++ + Vulkan |
docs/ |
Architecture & design docs | — |
Each platform directory has its own README with build and test instructions. The sections below focus on the Python framework, which is the easiest place to start.
Development setup (Python framework)
Requires macOS on Apple Silicon (the MLX backend has no wheels on other
platforms yet — a CUDA backend slot is reserved under
python/src/edge0/backends/cuda/).
cd python
python3.12 -m venv .venv
.venv/bin/pip install -e '.[dev,fetch]'
Do not bump
mlxpast the pinned0.30.6.0.30.4and older mis-gate the NAX matmul kernels on Apple A18 / A18 Pro (silently wrong numbers → garbled output, #8), andmlx>=0.31.2breaks the streaming pool threads.mlx-lmstays0.31.0. See the note inpython/pyproject.toml. Only touch these pins in a PR that is specifically about the upgrade.
Running the tests
cd python
pytest -m 'not slow' # unit tests (no real weights) — what CI runs
pytest tests/test_repo_hygiene.py # repo hygiene (pure Python, runs anywhere)
# Real-weight tests (need a downloaded checkpoint; skipped if absent):
EDGE0_8B_MODEL=/path/to/edge0-8b pytest -m slow -q
.venv/bin/python scripts/e2e_smoke.py \
--qwen-dir /path/to/edge0-35b --ling-dir /path/to/edge0-8b
What CI checks (and how to pass it)
Every PR runs two jobs (.github/workflows/ci.yml):
- Unit tests (macOS + MLX) —
pytest -m 'not slow'. - Repo hygiene (Linux, pure Python) —
tests/test_repo_hygiene.py, which enforces the "no dependency on the author's machine" rules across the entire repo:- No hardcoded local paths — nothing that points at a specific machine's filesystem (absolute home directories, network mounts, cluster node names, etc.). Use env vars, fixtures, or relative paths.
- Backend boundary —
import mlx/from mlx ...may appear only underpython/src/edge0/backends/mlx/. Framework code must go through theedge0.backendsfacade (core/nn/io/quant). This keeps the core backend-agnostic so other backends can plug in. - No secrets — no API keys or private keys in the tree.
- Docs links resolve — e.g. the README
NOTICElink must point to a real file.
You can run both locally before pushing (see above).
Adding a new model
edge0 is designed to take new MoE checkpoints through a documented path. Start
with docs/adding-a-model.md: implement the
three-part adapter contract (Config / build_model / build_engine),
register it, and map the checkpoint's model_type via TYPE_ALIASES. Please
include a short note on the architecture (layers / experts / routing width /
attention type) in your PR.
Pull request workflow
- Fork the repo and create a feature branch off
main(e.g.fix/streaming-oob,docs/clarify-memory,feat/tool-calling). - Make your change; add or update tests for new behavior.
- Run
pytest -m 'not slow'andpytest tests/test_repo_hygiene.pylocally. - Open a PR against
Edge0-AI/Edge0:mainand fill in the PR template. Link the issue it closes (Fixes #NN). - Keep PRs focused — one logical change per PR is easier to review and merge.
Commit / PR style
This repo uses conventional-commit-style prefixes, e.g.
fix(streaming): ..., feat(server): ..., docs(readme): ...,
perf(streaming): ..., chore: .... Please follow the same convention.
Reporting security issues
Do not open a public issue for security problems. Email [email protected] instead.
License
By contributing, you agree that your contributions will be licensed under the project's Apache-2.0 license.