Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
README: official checkpoint names (Edge0/Edge0-35b-a3b, Edge0-10b-a1b)
linyubupaec6a7e3a39d5
diff --git a/README.md b/README.mdindex 8ad5cf1..c4f612a 100644--- a/README.md+++ b/README.md@@ -9,12 +9,20 @@ framework. The backend is isolated by design: the current MLX backendruns on Apple Silicon, and additional platforms (CUDA, …) plug into thesame core abstractions.-Two model tiers work out of the box:+Two model tiers ship with the framework. Each tier is an end-to-end+release: the released checkpoint, the trained LoRA adapters, and the+trained prerouter heads work together as one unit.-| Tier | Model | Inference profile |+| Tier | Released checkpoint | Inference profile ||---|---|---|-| `edge0-35b` | Qwen3.6-35B-A3B 4-bit (40 layers, 256 experts) | prerouter K=4 |-| `edge0-10b` | Ling-10B 4-bit bailing hybrid (24 layers, 128 experts) | prerouter K=8 |+| `edge0-35b` | `Edge0/Edge0-35b-a3b` | 4-bit, 40 layers, 256 experts, prerouter K=4 |+| `edge0-10b` | `Edge0/Edge0-10b-a1b` | 4-bit, 24 layers, 128 experts, prerouter K=8 |++Both checkpoints are built on open sparse-MoE base models (Qwen3.5-MoE+35B-A3B and the Ling 3.0 bailing hybrid respectively) and ship with the+LoRA and prerouter training done for this framework — the adapter files+are co-located with each checkpoint and load automatically, so+`edge0 serve <tier>` runs the trained pipeline out of the box.## Requirementsdiff --git a/README_zh.md b/README_zh.mdindex 2fba259..dc087f3 100644--- a/README_zh.md+++ b/README_zh.md@@ -6,12 +6,18 @@并行 LoRA + prerouter 路由预判」抽象成可扩展的通用框架。后端隔离设计,当前实现 MLX 后端(Apple Silicon),更多平台(CUDA 等)即将接入。-开箱支持两个模型 tier:+框架随附两个模型档位。每个档位是一个端到端发布:发布的 checkpoint、+训练好的 LoRA 适配器与训练好的 prerouter 头作为整体协同工作。-| 模型 | 说明 | 推理档 |+| 档位 | 发布 checkpoint | 推理档 ||---|---|---|-| `edge0-35b` | Qwen3.6-35B-A3B 4bit(40 层,256 专家) | prerouter K=4 |-| `edge0-10b` | Ling-10B 4bit(24 层,128 专家) | prerouter K=8 |+| `edge0-35b` | `Edge0/Edge0-35b-a3b` | 4bit,40 层,256 专家,prerouter K=4 |+| `edge0-10b` | `Edge0/Edge0-10b-a1b` | 4bit,24 层,128 专家,prerouter K=8 |++两个 checkpoint 均基于开源稀疏 MoE 基座(分别为 Qwen3.5-MoE 35B-A3B+与 Ling 3.0 混合架构),并携带为本框架训练的 LoRA 与 prerouter 权重——+适配器文件与 checkpoint 同目录、自动加载,`edge0 serve <tier>` 开箱即跑+训练好的完整管线。## 环境要求diff --git a/scripts/fetch_models.py b/scripts/fetch_models.pyindex e8c35a4..fe31f0e 100755--- a/scripts/fetch_models.py+++ b/scripts/fetch_models.py@@ -9,10 +9,11 @@ download produces a ready-to-run model directory that ``edge0 demo`` /Requirements:pip install 'edge0[fetch]' # or: pip install huggingface_hub-Repo ids come from the environment (set them once, e.g. in ~/.zshrc):+Repo ids default to the official releases (override via the environment+if you mirror them):- export EDGE0_35B_REPO=<your-hf-org>/edge0-35b- export EDGE0_10B_REPO=<your-hf-org>/edge0-10b+ export EDGE0_35B_REPO=Edge0/Edge0-35b-a3b # default+ export EDGE0_10B_REPO=Edge0/Edge0-10b-a1b # defaultUsage:python scripts/fetch_models.py --tier edge0-35b --target-dir models@@ -32,20 +33,14 @@ import sysfrom pathlib import PathTIER_REPOS = {- "edge0-35b": "EDGE0_35B_REPO",- "edge0-10b": "EDGE0_10B_REPO",+ "edge0-35b": ("EDGE0_35B_REPO", "Edge0/Edge0-35b-a3b"),+ "edge0-10b": ("EDGE0_10B_REPO", "Edge0/Edge0-10b-a1b"),}def _repo_for(tier: str) -> str:- env = TIER_REPOS[tier]- repo = os.environ.get(env, "")- if not repo:- raise SystemExit(- f"[fetch] {tier} needs a repo id.\n"- f"Set {env} to your Hugging Face repo, e.g.\n"- f" export {env}=<your-hf-org>/{tier}\n"- f"then re-run this script.")+ env, default = TIER_REPOS[tier]+ repo = os.environ.get(env, default)return repo