Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
docs: rename the 35b base model to Qwen3.6-35B-A3B
linyubupa63d6efb28a53
diff --git a/README.md b/README.mdindex fb8168a..01c20ca 100644--- a/README.md+++ b/README.md@@ -31,8 +31,8 @@ trained prerouter heads work together as one unit.| `edge0-35b` | [`Edge0/Edge0-35B-A3B-preview`](https://huggingface.co/Edge0/Edge0-35B-A3B-preview) | 4-bit, 40 layers, 256 experts, prerouter K=4 || `edge0-8b` | [`Edge0/Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview) | 4-bit, 24 layers, 128 experts, prerouter K=8 |-Both checkpoints are built on open sparse-MoE base models (Qwen3.5-MoE-35B-A3B and the Ling 3.0 bailing hybrid respectively) and ship with the+Both checkpoints are built on open sparse-MoE base models (Qwen3.6-35B-A3B+and the Ling 3.0 bailing hybrid respectively) and ship with theLoRA and prerouter training done for this framework — the adapter filesare co-located with each checkpoint and load automatically, so`edge0 serve <tier>` runs the trained pipeline out of the box.@@ -209,7 +209,7 @@ trained adapters + prerouter routing) and the original fp16 base models.The loss of the edge0 pipeline is small: **3.9 points on average foredge0-35b, 2.8 for edge0-8b** (MMLU-Pro is even above the base). Max 100:-| Benchmark | edge0-35b (int4) | Qwen3.5-MoE 35B-A3B (fp16) | edge0-8b (int4) | Ling 3.0 tiny (fp16) |+| Benchmark | edge0-35b (int4) | Qwen3.6-35B-A3B (fp16) | edge0-8b (int4) | Ling 3.0 tiny (fp16) ||---|---:|---:|---:|---:|| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 || HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |diff --git a/README_zh.md b/README_zh.mdindex 649023d..46987e1 100644--- a/README_zh.md+++ b/README_zh.md@@ -27,7 +27,7 @@ Recover-LoRA + prerouter 路由预判」抽象成可扩展的通用框架。后| `edge0-35b` | [`Edge0/Edge0-35B-A3B-preview`](https://huggingface.co/Edge0/Edge0-35B-A3B-preview) | 4bit,40 层,256 专家,prerouter K=4 || `edge0-8b` | [`Edge0/Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview) | 4bit,24 层,128 专家,prerouter K=8 |-两个 checkpoint 均基于开源稀疏 MoE 基座(分别为 Qwen3.5-MoE 35B-A3B+两个 checkpoint 均基于开源稀疏 MoE 基座(分别为 Qwen3.6-35B-A3B与 Ling 3.0 混合架构),并携带为本框架训练的 LoRA 与 prerouter 权重——适配器文件与 checkpoint 同目录、自动加载,`edge0 serve <tier>` 开箱即跑训练好的完整管线。@@ -191,7 +191,7 @@ engine.close() # 释放 mmap / 专家缓存与原 fp16 基座模型测得。edge0 管线的损失很小:**edge0-35b 平均仅落后3.9 分、edge0-8b 落后 2.8 分**(MMLU-Pro 甚至反超基座)。满分 100:-| 评测集 | edge0-35b(int4) | Qwen3.5-MoE 35B-A3B(fp16) | edge0-8b(int4) | Ling 3.0 tiny(fp16) |+| 评测集 | edge0-35b(int4) | Qwen3.6-35B-A3B(fp16) | edge0-8b(int4) | Ling 3.0 tiny(fp16) ||---|---:|---:|---:|---:|| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 || HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |diff --git a/docs/architecture.md b/docs/architecture.mdindex 67ac6d3..3a07c42 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -6,7 +6,7 @@ validated in production deployments — into an extensible, general-purposeframework. Backends are isolated by design: the current release implements theMLX backend (Apple Silicon), core logic is decoupled from the backend, and otherplatforms plug in through the same facade. Two models are supported out of the-box: `edge0-35b` (Qwen3.5-MoE, K=4) and `edge0-8b` (Ling 3.0 hybrid, K=8).+box: `edge0-35b` (Qwen3.6-35B-A3B, K=4) and `edge0-8b` (Ling 3.0 hybrid, K=8).## Core Design Goals@@ -153,6 +153,6 @@ across backends.## License-Apache-2.0, including vendored third-party model code (Qwen3.5-MoE from mlx-lm+Apache-2.0, including vendored third-party model code (Qwen3.6-35B-A3B from mlx-lmand the Ling backbone from the reference deployment); see the root[NOTICE](../NOTICE) for details.diff --git a/docs/models/edge0-35b.md b/docs/models/edge0-35b.mdindex b95a24a..99d611b 100644--- a/docs/models/edge0-35b.md+++ b/docs/models/edge0-35b.md@@ -1,6 +1,6 @@# edge0-35b-The flagship first tier of the edge0 platform: a 35B-class sparse mixture-of-experts model built on Qwen3.5-MoE (K=4 tier). Through streaming expert loading, a trained prerouter, and LoRA adaptation, the full quantized weight set resides on disk and is loaded on demand, so the model can be served on a single device.+The flagship first tier of the edge0 platform: a 35B-class sparse mixture-of-experts model built on Qwen3.6-35B-A3B (K=4 tier). Through streaming expert loading, a trained prerouter, and LoRA adaptation, the full quantized weight set resides on disk and is loaded on demand, so the model can be served on a single device.The performance profile is based on benchmarks of the current release adapter version and is pinned by default in `Qwen35Config` (`src/edge0/models/edge0_35b/__init__.py`).diff --git a/docs/moe.md b/docs/moe.mdindex 2021bc1..3697213 100644--- a/docs/moe.md+++ b/docs/moe.md@@ -15,7 +15,7 @@ subsystem consumes that spec (layout + key templates), so **one generic| Member | Value | Semantics || --- | --- | --- |-| `SOFTMAX_TOPK` | `"softmax_topk"` | exact softmax → top-k → re-normalization (Qwen3.5-MoE style, `norm_topk_prob=True`) |+| `SOFTMAX_TOPK` | `"softmax_topk"` | exact softmax → top-k → re-normalization (Qwen3.6-35B-A3B style, `norm_topk_prob=True`) || `SIGMOID_GROUP` | `"sigmoid_group"` | sigmoid scores + group-limited top-k + routed scaling (DeepSeek-V3 / Bailing style) |The two families correspond to the two models: edge0-35b uses `SOFTMAX_TOPK`, edge0-8b@@ -163,7 +163,7 @@ stay identical to them — pinned by parity tests.### `select_from_logits(logits, top_k, norm=True)`-Softmax-topk routing (Qwen3.5-MoE / `norm_topk_prob=True` semantics):+Softmax-topk routing (Qwen3.6-35B-A3B / `norm_topk_prob=True` semantics):```pythongates = core.softmax(logits, axis=-1, precise=True)diff --git a/docs/streaming.md b/docs/streaming.mdindex 149502f..7ba4647 100644--- a/docs/streaming.md+++ b/docs/streaming.md@@ -2,7 +2,7 @@## The Problem-Qwen3.5-MoE has 40 layers × 256 experts, with each layer's MoE weights around 310MB after 4-bit quantization — keeping them all resident far exceeds the Apple Silicon unified-memory budget. edge0's approach: **keep the weights on SSD, mmap them into the GPU on demand, and use LRU + prefetch + fixed slots to turn "tens of MB moved per step" into "almost nothing moved per step"**.+Qwen3.6-35B-A3B has 40 layers × 256 experts, with each layer's MoE weights around 310MB after 4-bit quantization — keeping them all resident far exceeds the Apple Silicon unified-memory budget. edge0's approach: **keep the weights on SSD, mmap them into the GPU on demand, and use LRU + prefetch + fixed slots to turn "tens of MB moved per step" into "almost nothing moved per step"**.Core components (`edge0/streaming/`):diff --git a/src/edge0/backends/mlx/_impl/__init__.py b/src/edge0/backends/mlx/_impl/__init__.pyindex b9c0548..9c68213 100644--- a/src/edge0/backends/mlx/_impl/__init__.py+++ b/src/edge0/backends/mlx/_impl/__init__.py@@ -1,6 +1,6 @@"""Vendored base model implementations (licensing in NOTICE).-* ``qwen3_next.py`` — mlx-lm 0.31.0's Qwen3.5-MoE model, byte-identical+* ``qwen3_next.py`` — mlx-lm 0.31.0's Qwen3.6-35B-A3B model, byte-identicalto the upstream file except the import block, which was rewritten toabsolute ``mlx_lm.models`` paths so the file can live outside themlx_lm package. Pinned to mlx-lm==0.31.0 by pyproject.diff --git a/src/edge0/engine/qwen.py b/src/edge0/engine/qwen.pyindex 96ce7f2..cdf88c4 100644--- a/src/edge0/engine/qwen.py+++ b/src/edge0/engine/qwen.py@@ -1,4 +1,4 @@-"""edge0-35b engine: Qwen3.5-MoE (K=4 tier).+"""edge0-35b engine: Qwen3.6-35B-A3B (K=4 tier).Port of the deployment's ``engine_qwen.py`` trained-prerouter path:@@ -109,7 +109,7 @@ def load_installed(model_dir: str, cfg):class Qwen35Engine(Edge0Engine):- """Streaming Qwen3.5-MoE engine (staged decode + trained prerouter)."""+ """Streaming Qwen3.6-35B-A3B engine (staged decode + trained prerouter)."""name = "edge0-35b"diff --git a/src/edge0/models/edge0_35b/__init__.py b/src/edge0/models/edge0_35b/__init__.pyindex ca5305d..3f27819 100644--- a/src/edge0/models/edge0_35b/__init__.py+++ b/src/edge0/models/edge0_35b/__init__.py@@ -1,4 +1,4 @@-"""edge0-35b adapter: Qwen3.5-MoE (K=4 tier, 35B-class).+"""edge0-35b adapter: Qwen3.6-35B-A3B (K=4 tier, 35B-class).Family facts (production profile, verified against the currentcheckpoint):@@ -26,7 +26,7 @@ from edge0.streaming.options import LayerOptionsclass Qwen35Config(ModelConfig):- """edge0-35b family config (Qwen3.5-MoE, K=4)."""+ """edge0-35b family config (Qwen3.6-35B-A3B, K=4)."""@classmethoddef _defaults(cls, model_dir: str) -> "Qwen35Config":diff --git a/src/edge0/moe/routing.py b/src/edge0/moe/routing.pyindex 0a9d2e4..4c292bb 100644--- a/src/edge0/moe/routing.py+++ b/src/edge0/moe/routing.py@@ -13,7 +13,7 @@ from edge0.backends import coredef select_from_logits(logits, top_k: int, norm: bool = True):"""Softmax-topk router: precise softmax -> top-k -> renormalize.- Mirrors the Qwen3.5-MoE HF router semantics (norm_topk_prob=True).+ Mirrors the Qwen3.6-35B-A3B HF router semantics (norm_topk_prob=True).Returns ``(inds [..., k], scores [..., k])``."""gates = core.softmax(logits, axis=-1, precise=True)diff --git a/src/edge0/moe/spec.py b/src/edge0/moe/spec.pyindex 176c43c..ced4ac4 100644--- a/src/edge0/moe/spec.py+++ b/src/edge0/moe/spec.py@@ -17,7 +17,7 @@ class RouterKind(Enum):"""Router math families supported by edge0."""SOFTMAX_TOPK = "softmax_topk"- """precise softmax -> top-k -> renormalize (Qwen3.5-MoE style,+ """precise softmax -> top-k -> renormalize (Qwen3.6-35B-A3B style,norm_topk_prob=True)."""SIGMOID_GROUP = "sigmoid_group"