Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

rename edge0-10b tier to edge0-8b (8B-class model, Ling 3.0 tiny base); HF repo Edge0/Edge0-8b-a1b-preview

linyubupa4823516cec7f
diff --git a/README.md b/README.md
index ec18cd1..8052d58 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@
**An open-source streaming MoE inference framework — SSD expert offload + parallel LoRA + prerouter routing prediction.**
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--35b--a3b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-35b-a3b-preview)
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--10b--a1b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-10b-a1b-preview)
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--8b--a1b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-8b-a1b-preview)
[![GitHub](https://img.shields.io/badge/GitHub-Edge0--AI%2Fedge0-black?style=for-the-badge&logo=github)](https://github.com/Edge0-AI/edge0)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue?style=for-the-badge)](LICENSE)
@@ -29,7 +29,7 @@ trained prerouter heads work together as one unit.
| Tier | Released checkpoint | Inference profile |
|---|---|---|
| `edge0-35b` | [`Edge0/Edge0-35b-a3b-preview`](https://huggingface.co/Edge0/Edge0-35b-a3b-preview) | 4-bit, 40 layers, 256 experts, prerouter K=4 |
-| `edge0-10b` | [`Edge0/Edge0-10b-a1b-preview`](https://huggingface.co/Edge0/Edge0-10b-a1b-preview) | 4-bit, 24 layers, 128 experts, prerouter K=8 |
+| `edge0-8b` | [`Edge0/Edge0-8b-a1b-preview`](https://huggingface.co/Edge0/Edge0-8b-a1b-preview) | 4-bit, 24 layers, 128 experts, prerouter K=8 |
Both checkpoints are built on open sparse-MoE base models (Qwen3.5-MoE
35B-A3B and the Ling 3.0 bailing hybrid respectively) and ship with the
@@ -44,10 +44,10 @@ are co-located with each checkpoint and load automatically, so
platforms are supported yet.
- **Python**: 3.10+ (3.12 recommended).
- **Memory**: ~2.9 GB peak active memory for `edge0-35b`, ~1.0 GB for
- `edge0-10b` (short contexts; see [Benchmark](#benchmark)). Add
+ `edge0-8b` (short contexts; see [Benchmark](#benchmark)). Add
headroom for the OS, tokenizer, and long-context KV growth.
- **Disk**: the 4-bit checkpoints are ~23 GB (`edge0-35b`) and ~4.2 GB
- (`edge0-10b`); expert weights are mmapped and read on demand, they are
+ (`edge0-8b`); expert weights are mmapped and read on demand, they are
not loaded into RAM up front.
## Design
@@ -107,16 +107,16 @@ base checkpoint and the trained LoRA + prerouter adapters in **one
directory**, so a single download is a ready-to-run model:
- [`Edge0/Edge0-35b-a3b-preview`](https://huggingface.co/Edge0/Edge0-35b-a3b-preview) (~23 GB)
-- [`Edge0/Edge0-10b-a1b-preview`](https://huggingface.co/Edge0/Edge0-10b-a1b-preview) (~4.2 GB)
+- [`Edge0/Edge0-8b-a1b-preview`](https://huggingface.co/Edge0/Edge0-8b-a1b-preview) (~4.2 GB)
```bash
# with the repo's helper (defaults to the two repos above):
.venv/bin/python scripts/fetch_models.py --tier edge0-35b --target-dir models
-.venv/bin/python scripts/fetch_models.py --tier edge0-10b --target-dir models
+.venv/bin/python scripts/fetch_models.py --tier edge0-8b --target-dir models
# or directly with the CLI:
.venv/bin/huggingface-cli download Edge0/Edge0-35b-a3b-preview --local-dir models/edge0-35b
-.venv/bin/huggingface-cli download Edge0/Edge0-10b-a1b-preview --local-dir models/edge0-10b
+.venv/bin/huggingface-cli download Edge0/Edge0-8b-a1b-preview --local-dir models/edge0-8b
```
Either way you end up with a directory like:
@@ -135,7 +135,7 @@ Tier names resolve to local directories via environment variables
```bash
export EDGE0_35B_MODEL=$PWD/models/edge0-35b
-export EDGE0_10B_MODEL=$PWD/models/edge0-10b
+export EDGE0_8B_MODEL=$PWD/models/edge0-8b
```
Or skip the env vars entirely and pass the directory directly — the
@@ -143,7 +143,7 @@ tier is auto-detected from the checkpoint's `config.json`:
```bash
edge0 demo models/edge0-35b
-edge0 serve models/edge0-10b
+edge0 serve models/edge0-8b
```
### 4) Run
@@ -212,10 +212,10 @@ this exact path).
Internal self-evaluation, included only to quantify the loss of the edge0
pipeline (int4 + trained adapters + prerouter routing) relative to the
original fp16 base models — the loss is small: **3.9 points on average for
-edge0-35b, 2.8 for edge0-10b** (MMLU-Pro is even above the base). All scores
+edge0-35b, 2.8 for edge0-8b** (MMLU-Pro is even above the base). All scores
self-run, max 100:
-| Benchmark | edge0-35b (int4) | Base fp16 | edge0-10b (int4) | Base fp16 |
+| Benchmark | edge0-35b (int4) | Base fp16 | edge0-8b (int4) | Base fp16 |
|---|---:|---:|---:|---:|
| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |
| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |
@@ -232,7 +232,7 @@ warmup steps → 200 timed sampled decode tokens, 2 runs per tier):
| Tier | Decode speed | Prefill throughput (cold / warm)* | Peak active memory** | Test machine |
|---|---|---|---|---|
| `edge0-35b` | 14.9–17.7 tok/s | 113 / 140 tok/s | 2.9 GiB | Mac mini M4 Pro, 24 GB |
-| `edge0-10b` | 23.9–25.3 tok/s | 500 / 1428 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+| `edge0-8b` | 23.9–25.3 tok/s | 500 / 1428 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
*Cold = first request after process start (expert weights fault in from
SSD); warm = subsequent requests (page cache resident). Prefill numbers
@@ -240,13 +240,13 @@ are throughput over a ~3.3k-token prompt (`BENCH_LONG=1`).*
**Peak active memory at short contexts (MLX allocator peak; expert weights
stream from SSD via mmap and are not resident). Long contexts add KV
-cache: ~3.3 GiB on `edge0-10b` at 3.3k tokens.*
+cache: ~3.3 GiB on `edge0-8b` at 3.3k tokens.*
Reproduce:
```bash
python examples/bench.py edge0-35b # via $EDGE0_35B_MODEL
-python examples/bench.py edge0-10b # via $EDGE0_10B_MODEL
+python examples/bench.py edge0-8b # via $EDGE0_8B_MODEL
```
## Tests
@@ -264,7 +264,7 @@ examples/demo.py # minimal API walkthrough
- [Architecture](docs/architecture.md)
- [Attention](docs/attention.md) / [MoE](docs/moe.md) / [SSD streaming](docs/streaming.md) / [prerouter](docs/prerouter.md)
- [Adding a model](docs/adding-a-model.md)
-- [edge0-35b](docs/models/edge0-35b.md) / [edge0-10b](docs/models/edge0-10b.md)
+- [edge0-35b](docs/models/edge0-35b.md) / [edge0-8b](docs/models/edge0-8b.md)
## License
diff --git a/README_zh.md b/README_zh.md
index 1ff94d7..02d98a1 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -7,7 +7,7 @@
**开源流式 MoE 推理框架 —— SSD 专家 offload + 并行 LoRA + prerouter 路由预判**
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--35b--a3b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-35b-a3b-preview)
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--10b--a1b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-10b-a1b-preview)
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--8b--a1b--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-8b-a1b-preview)
[![GitHub](https://img.shields.io/badge/GitHub-Edge0--AI%2Fedge0-black?style=for-the-badge&logo=github)](https://github.com/Edge0-AI/edge0)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue?style=for-the-badge)](LICENSE)
@@ -25,7 +25,7 @@
| 档位 | 发布 checkpoint | 推理档 |
|---|---|---|
| `edge0-35b` | [`Edge0/Edge0-35b-a3b-preview`](https://huggingface.co/Edge0/Edge0-35b-a3b-preview) | 4bit,40 层,256 专家,prerouter K=4 |
-| `edge0-10b` | [`Edge0/Edge0-10b-a1b-preview`](https://huggingface.co/Edge0/Edge0-10b-a1b-preview) | 4bit,24 层,128 专家,prerouter K=8 |
+| `edge0-8b` | [`Edge0/Edge0-8b-a1b-preview`](https://huggingface.co/Edge0/Edge0-8b-a1b-preview) | 4bit,24 层,128 专家,prerouter K=8 |
两个 checkpoint 均基于开源稀疏 MoE 基座(分别为 Qwen3.5-MoE 35B-A3B
与 Ling 3.0 混合架构),并携带为本框架训练的 LoRA 与 prerouter 权重——
@@ -37,11 +37,11 @@
- **系统 / 硬件**:MLX 后端目前仅支持 Apple Silicon 的 macOS
(M1/M2/M3/M4);CUDA 后端在路线图中,其余平台暂不支持。
- **Python**:3.10+(推荐 3.12)。
-- **内存**:短上下文下 `edge0-35b` ≈2.9 GB、`edge0-10b` ≈1.0 GB
+- **内存**:短上下文下 `edge0-35b` ≈2.9 GB、`edge0-8b` ≈1.0 GB
峰值激活内存(见[性能实测](#性能实测));另为系统、tokenizer 与
长上下文 KV 增长预留余量。
- **磁盘**:4bit checkpoint 约 23 GB(`edge0-35b`)/ 4.2 GB
- (`edge0-10b`);专家权重 mmap 按需读取,不一次性载入内存。
+ (`edge0-8b`);专家权重 mmap 按需读取,不一次性载入内存。
## 设计
@@ -88,18 +88,18 @@ python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'
LoRA + prerouter 适配器打包在**同一目录**,一次下载即为可运行的模型:
- [`Edge0/Edge0-35b-a3b-preview`](https://huggingface.co/Edge0/Edge0-35b-a3b-preview)(约 23 GB)
-- [`Edge0/Edge0-10b-a1b-preview`](https://huggingface.co/Edge0/Edge0-10b-a1b-preview)(约 4.2 GB)
+- [`Edge0/Edge0-8b-a1b-preview`](https://huggingface.co/Edge0/Edge0-8b-a1b-preview)(约 4.2 GB)
```bash
# 用仓库自带脚本(默认即上述两个仓库):
.venv/bin/python scripts/fetch_models.py --tier edge0-35b --target-dir models
-.venv/bin/python scripts/fetch_models.py --tier edge0-10b --target-dir models
+.venv/bin/python scripts/fetch_models.py --tier edge0-8b --target-dir models
# 或直接用 CLI:
.venv/bin/huggingface-cli download Edge0/Edge0-35b-a3b-preview \
--local-dir models/edge0-35b
-.venv/bin/huggingface-cli download Edge0/Edge0-10b-a1b-preview \
- --local-dir models/edge0-10b
+.venv/bin/huggingface-cli download Edge0/Edge0-8b-a1b-preview \
+ --local-dir models/edge0-8b
```
下载完成后目录结构:
@@ -117,7 +117,7 @@ models/edge0-35b/
```bash
export EDGE0_35B_MODEL=$PWD/models/edge0-35b
-export EDGE0_10B_MODEL=$PWD/models/edge0-10b
+export EDGE0_8B_MODEL=$PWD/models/edge0-8b
```
也可以不用环境变量,直接传目录——框架从 checkpoint 的 `config.json`
@@ -125,7 +125,7 @@ export EDGE0_10B_MODEL=$PWD/models/edge0-10b
```bash
edge0 demo models/edge0-35b
-edge0 serve models/edge0-10b
+edge0 serve models/edge0-8b
```
### 4) 运行
@@ -177,10 +177,10 @@ engine.close() # 释放 mmap / 专家缓存
内部自测分数,仅用于对比 edge0 管线(int4 量化 + 训练适配器 + prerouter
路由)相对原 fp16 基座模型的**损失——损失很小:edge0-35b 平均仅落后
-3.9 分、edge0-10b 落后 2.8 分**(MMLU-Pro 甚至反超基座)。均为自测,
+3.9 分、edge0-8b 落后 2.8 分**(MMLU-Pro 甚至反超基座)。均为自测,
满分 100:
-| 评测项 | edge0-35b (int4) | 基座 fp16 | edge0-10b (int4) | 基座 fp16 |
+| 评测项 | edge0-35b (int4) | 基座 fp16 | edge0-8b (int4) | 基座 fp16 |
|---|---:|---:|---:|---:|
| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |
| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |
@@ -197,17 +197,17 @@ engine.close() # 释放 mmap / 专家缓存
| 档位 | 解码速度 | Prefill 吞吐(冷/热)* | 峰值 active 内存** | 测试机器 |
|---|---|---|---|---|
| `edge0-35b` | 14.9–17.7 tok/s | 113 / 140 tok/s | 2.9 GiB | Mac mini M4 Pro, 24 GB |
-| `edge0-10b` | 23.9–25.3 tok/s | 500 / 1428 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+| `edge0-8b` | 23.9–25.3 tok/s | 500 / 1428 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
*冷 = 进程启动后首请求(专家权重从 SSD 逐页换入);热 = 后续请求(页缓存常驻)。Prefill 为 ≈3.3k token 长 prompt 的吞吐(`BENCH_LONG=1`)。
-**短上下文下的峰值 active 内存(MLX allocator 峰值;专家权重经 mmap 流式读取、不常驻内存)。长上下文增加 KV cache:3.3k token 下 `edge0-10b` ≈3.3 GiB。**
+**短上下文下的峰值 active 内存(MLX allocator 峰值;专家权重经 mmap 流式读取、不常驻内存)。长上下文增加 KV cache:3.3k token 下 `edge0-8b` ≈3.3 GiB。**
复现:
```bash
python examples/bench.py edge0-35b # 经 $EDGE0_35B_MODEL
-python examples/bench.py edge0-10b # 经 $EDGE0_10B_MODEL
+python examples/bench.py edge0-8b # 经 $EDGE0_8B_MODEL
```
## 验证
@@ -225,7 +225,7 @@ examples/demo.py # 最小 API walkthrough(edge0 demo 的等价代码)
- [架构总览](docs/architecture.md)
- [注意力抽象](docs/attention.md) / [MoE 抽象](docs/moe.md) / [SSD 流式](docs/streaming.md) / [prerouter](docs/prerouter.md)
- [如何接入新模型](docs/adding-a-model.md)
-- [edge0-35b](docs/models/edge0-35b.md) / [edge0-10b](docs/models/edge0-10b.md)
+- [edge0-35b](docs/models/edge0-35b.md) / [edge0-8b](docs/models/edge0-8b.md)
## License
diff --git a/docs/adding-a-model.md b/docs/adding-a-model.md
index eb9df68..f56a746 100644
--- a/docs/adding-a-model.md
+++ b/docs/adding-a-model.md
@@ -1,6 +1,6 @@
# Adding a New Model
-This document is a step-by-step guide for adding a new model family to edge0, using `edge0-35b` (`src/edge0/models/edge0_35b/__init__.py`) as the running example, with real code line numbers noted throughout. `edge0-10b` (`src/edge0/models/edge0_10b/__init__.py`) is a second live example; its path resolution and contract are identical.
+This document is a step-by-step guide for adding a new model family to edge0, using `edge0-35b` (`src/edge0/models/edge0_35b/__init__.py`) as the running example, with real code line numbers noted throughout. `edge0-8b` (`src/edge0/models/edge0_8b/__init__.py`) is a second live example; its path resolution and contract are identical.
Integration goal: let `AutoConfig` / `AutoModel` / `AutoEngine` resolve your model by name (or by the `model_type` in the checkpoint's `config.json`), and have the streaming MoE layers driven by a common `StreamingSwitchGLU`.
@@ -68,7 +68,7 @@ def build_engine(model_dir=None, **overrides):
return Qwen35Engine(cfg.model_dir, cfg)
```
-If your model family's math differs (e.g. edge0-10b's `SIGMOID_GROUP` + hybrid MLA), write a matching engine under `src/edge0/engine/` (model it on `engine/qwen.py` / `engine/ling.py`) and reference it from `build_model` / `build_engine`. `load_installed` assembles the streaming twins / LoRA / prerouter; it is the construction path the framework provides to engines.
+If your model family's math differs (e.g. edge0-8b's `SIGMOID_GROUP` + hybrid MLA), write a matching engine under `src/edge0/engine/` (model it on `engine/qwen.py` / `engine/ling.py`) and reference it from `build_model` / `build_engine`. `load_installed` assembles the streaming twins / LoRA / prerouter; it is the construction path the framework provides to engines.
## Step 4: Register with the registry
@@ -92,8 +92,8 @@ Config = Qwen35Config # registry contract: adapter.Config
TYPE_ALIASES = {
"qwen3_5_moe_text": "edge0-35b",
"qwen3_5_moe": "edge0-35b",
- "bailing_hybrid": "edge0-10b",
- "bailing_moe_linear": "edge0-10b",
+ "bailing_hybrid": "edge0-8b",
+ "bailing_moe_linear": "edge0-8b",
}
```
@@ -165,5 +165,5 @@ Config = MyConfig
- `src/edge0/registry.py` — the registry + the Auto trio
- `src/edge0/models/base.py` — `ModelConfig` / `from_pretrained` / artifact resolution
-- `src/edge0/models/edge0_35b/__init__.py`, `edge0_10b/__init__.py` — the two live examples
+- `src/edge0/models/edge0_35b/__init__.py`, `edge0_8b/__init__.py` — the two live examples
- `tests/test_registry.py`, `tests/test_moe_spec.py` — behavior examples
diff --git a/docs/architecture.md b/docs/architecture.md
index b1a6df3..4b8cc01 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -6,7 +6,7 @@ validated in production deployments — into an extensible, general-purpose
framework. Backends are isolated by design: the current release implements the
MLX backend (Apple Silicon), core logic is decoupled from the backend, and other
platforms plug in through the same facade. Two models are supported out of the
-box: `edge0-35b` (Qwen3.5-MoE, K=4) and `edge0-10b` (Ling 3.0 hybrid, K=8).
+box: `edge0-35b` (Qwen3.5-MoE, K=4) and `edge0-8b` (Ling 3.0 hybrid, K=8).
## Core Design Goals
@@ -53,12 +53,12 @@ src/edge0/
├── engine/ # inference orchestration (prefill/decode loop, subclassed per model)
│ ├── base.py # shared loop (via backends.core)
│ ├── qwen.py # Qwen35Engine (K=4 tier)
-│ └── ling.py # Ling10BEngine (K=8 tier)
+│ └── ling.py # Ling8BEngine (K=8 tier)
├── adapters/lora.py # parallel LoRA installation (backends.nn)
├── models/ # model adapter layer (transformers-style)
│ ├── base.py # ModelConfig base class + artifact path resolution
│ ├── edge0_35b/ # edge0-35b tier (registered name "edge0-35b")
-│ └── edge0_10b/ # edge0-10b tier (registered name "edge0-10b")
+│ └── edge0_8b/ # edge0-8b tier (registered name "edge0-8b")
└── server/ # OpenAI-compatible HTTP service (pure stdlib)
├── app.py # ThreadingHTTPServer + routing
└── chat.py # /v1/chat/completions session state
diff --git a/docs/attention.md b/docs/attention.md
index 0a29371..2a0eaa9 100644
--- a/docs/attention.md
+++ b/docs/attention.md
@@ -1,6 +1,6 @@
# Attention Specification (`AttentionSpec`)
-`edge0` does not re-implement attention kernels: the concrete sequence-mixing computation is carried by the **vendored base model** (GQA / GatedDeltaNet in `edge0.backends.mlx._impl` for edge0-35b; MLA / DeltaNet for edge0-10b). Attention kernels are **backend model assets**, not part of the framework.
+`edge0` does not re-implement attention kernels: the concrete sequence-mixing computation is carried by the **vendored base model** (GQA / GatedDeltaNet in `edge0.backends.mlx._impl` for edge0-35b; MLA / DeltaNet for edge0-8b). Attention kernels are **backend model assets**, not part of the framework.
So why have an `AttentionSpec` at all? Because the framework needs to **introspect** a model **without depending on a concrete implementation**: cache sizing, layer roles, documentation, and future kernel adaptation. `AttentionSpec` is the carrier for this "attention taxonomy" layer. This document explains what it is, the field semantics, how the engine uses it, and why it is a separate abstraction (the MLA/MHA differences).
@@ -13,7 +13,7 @@ The docstring at the top of the module draws the boundary explicitly:
```
edge0 does not re-implement attention kernels: the vendored base model
implementations (edge0.backends.mlx._impl) carry the kernels (GQA /
-GatedDeltaNet for edge0-35b, MLA / DeltaNet for edge0-10b). This module
+GatedDeltaNet for edge0-35b, MLA / DeltaNet for edge0-8b). This module
describes attention so the framework can introspect a model — cache
sizing, layer roles, documentation, and future kernels — without knowing
the concrete implementation.
@@ -33,7 +33,7 @@ Adding a new attention type = a new `AttentionKind` member + an `AttentionSpec`
| `GATED_DELTANET` | `"gated_deltanet"` | GatedDeltaNet (gated delta / linear attention with decay) |
| `DENSE_MLP` | `"dense_mlp"` | No sequence mixing (pure MLP layer) |
-The existence of `DENSE_MLP` says that some layers of a model may have no attention block at all (for example, layer 0 of edge0-10b is a dense layer), and the taxonomy must be able to state explicitly that "this layer does no sequence mixing".
+The existence of `DENSE_MLP` says that some layers of a model may have no attention block at all (for example, layer 0 of edge0-8b is a dense layer), and the taxonomy must be able to state explicitly that "this layer does no sequence mixing".
## The Spec: `AttentionSpec`
@@ -92,7 +92,7 @@ Rather than "the engine runs attention", it is more accurate to say the engine o
| Head geometry | `num_heads` / `num_kv_heads` / `head_dim` are meaningful | Usually not applicable (`None`) |
| Mixing layer coverage | Nearly every layer | Possibly only some layers (incl. dense layers) |
-`AttentionSpec` expresses both families with a single, uniform field set: `kind` carries the family, `cache` carries cache presence, the head-geometry fields are populated only for GQA/MLA, and `layer_indices` carries the mixing layer coverage and scheduling. The framework can then run the same introspection logic for edge0-35b's GatedDeltaNet and edge0-10b's MLA, without hardcoding each of them.
+`AttentionSpec` expresses both families with a single, uniform field set: `kind` carries the family, `cache` carries cache presence, the head-geometry fields are populated only for GQA/MLA, and `layer_indices` carries the mixing layer coverage and scheduling. The framework can then run the same introspection logic for edge0-35b's GatedDeltaNet and edge0-8b's MLA, without hardcoding each of them.
## Steps to Add a New Attention Type
diff --git a/docs/models/edge0-10b.md b/docs/models/edge0-8b.md
similarity index 87%
rename from docs/models/edge0-10b.md
rename to docs/models/edge0-8b.md
index d86719d..2a4097b 100644
--- a/docs/models/edge0-10b.md
+++ b/docs/models/edge0-8b.md
@@ -1,8 +1,8 @@
-# edge0-10b
+# edge0-8b
The lightweight, high-throughput tier of the edge0 platform: an 8B-class sparse mixture-of-experts model built on the Ling 3.0 hybrid architecture (MLA + MoE). It trades some parameter count versus the 35b tier for lower peak memory and a higher generation rate, making it a good fit for latency- and memory-sensitive scenarios.
-The performance profile is based on benchmarks of the current release adapter version and is pinned by default in `Ling10BConfig` (`src/edge0/models/edge0_10b/__init__.py`).
+The performance profile is based on benchmarks of the current release adapter version and is pinned by default in `Ling8BConfig` (`src/edge0/models/edge0_8b/__init__.py`).
## Model profile
@@ -26,7 +26,7 @@ The performance profile is based on benchmarks of the current release adapter ve
| Measured throughput | 23.9–25.3 tok/s (M4 Pro) |
| Measured peak activation memory | ≈ 1.0 GB (short context) / 3.1 GB (3.3k-token context) |
-> Note: all values are taken from `Ling10BConfig._defaults()` and `LayerOptions.prod_k8()`. The head count comes from the explicit `owners=range(7, 23)` — 16 heads in total (the current release head distribution: prediction is consumed from L7 onward, and L1–6 use the original router); `feature_topk="executed"`.
+> Note: all values are taken from `Ling8BConfig._defaults()` and `LayerOptions.prod_k8()`. The head count comes from the explicit `owners=range(7, 23)` — 16 heads in total (the current release head distribution: prediction is consumed from L7 onward, and L1–6 use the original router); `feature_topk="executed"`.
## Staged decode
@@ -53,7 +53,7 @@ Optional arguments:
For single-turn chat in the terminal, use `chat` instead:
```bash
-edge0 chat --name edge0-10b --model-dir /path/to/checkpoint --prompt "Hello"
+edge0 chat --name edge0-8b --model-dir /path/to/checkpoint --prompt "Hello"
```
To view this tier's default profile:
@@ -69,17 +69,17 @@ from edge0 import AutoEngine, AutoModel, AutoConfig
# an engine ready to generate
engine = AutoEngine.from_pretrained(
- "/path/to/checkpoint", name="edge0-10b",
+ "/path/to/checkpoint", name="edge0-8b",
)
ids = engine.generate([156895]) # uses the default sampling from the config
text = engine._tok.decode(ids)
engine.close()
# weights only (streaming experts + prerouter + LoRA installed)
-model = AutoModel.from_pretrained("/path/to/checkpoint", name="edge0-10b")
+model = AutoModel.from_pretrained("/path/to/checkpoint", name="edge0-8b")
# config only
-cfg = AutoConfig.from_pretrained("/path/to/checkpoint", name="edge0-10b")
+cfg = AutoConfig.from_pretrained("/path/to/checkpoint", name="edge0-8b")
```
`AutoEngine` / `AutoModel` / `AutoConfig` can all omit `name` and resolve automatically from the `model_type` in the checkpoint's `config.json` or from the directory basename (see `src/edge0/registry.py`).
@@ -101,7 +101,7 @@ cfg = AutoConfig.from_pretrained("/path/to/checkpoint", name="edge0-10b")
curl -s http://127.0.0.1:8083/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
- "model": "edge0-10b",
+ "model": "edge0-8b",
"messages": [{"role": "user", "content": "Introduce Ling"}],
"temperature": 0.7,
"max_tokens": 256
@@ -115,7 +115,7 @@ Response fields (non-streaming):
"id": "chatcmpl-...",
"object": "chat.completion",
"created": 1750000000,
- "model": "edge0-10b",
+ "model": "edge0-8b",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "..."},
@@ -133,7 +133,7 @@ Optional request fields: `model`, `messages` (with `role`/`content`; content sup
curl -N http://127.0.0.1:8083/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
- "model": "edge0-10b",
+ "model": "edge0-8b",
"messages": [{"role": "user", "content": "Count to five"}],
"stream": true
}'
@@ -150,7 +150,7 @@ from edge0 import AutoEngine
engine = AutoEngine.from_pretrained(
"/path/to/checkpoint",
- name="edge0-10b",
+ name="edge0-8b",
port=9083, # override the default port 8083
target_tok_s=35.0, # override the acceptance throughput target
prerouter=None, # disable the prerouter
@@ -159,4 +159,4 @@ engine = AutoEngine.from_pretrained(
)
```
-The corresponding CLI overrides are `--no-prerouter` / `--no-lora` (see `_engine_kwargs` in `src/edge0/cli.py`). To override engine parameters, call `Ling10BConfig.from_pretrained(model_dir, **overrides)` directly.
+The corresponding CLI overrides are `--no-prerouter` / `--no-lora` (see `_engine_kwargs` in `src/edge0/cli.py`). To override engine parameters, call `Ling8BConfig.from_pretrained(model_dir, **overrides)` directly.
diff --git a/docs/moe.md b/docs/moe.md
index fc88f97..2021bc1 100644
--- a/docs/moe.md
+++ b/docs/moe.md
@@ -18,7 +18,7 @@ subsystem consumes that spec (layout + key templates), so **one generic
| `SOFTMAX_TOPK` | `"softmax_topk"` | exact softmax → top-k → re-normalization (Qwen3.5-MoE style, `norm_topk_prob=True`) |
| `SIGMOID_GROUP` | `"sigmoid_group"` | sigmoid scores + group-limited top-k + routed scaling (DeepSeek-V3 / Bailing style) |
-The two families correspond to the two models: edge0-35b uses `SOFTMAX_TOPK`, edge0-10b
+The two families correspond to the two models: edge0-35b uses `SOFTMAX_TOPK`, edge0-8b
uses `SIGMOID_GROUP`. See the "Routing functions" section for the matching math
implementations.
@@ -199,7 +199,7 @@ Returns `(inds [..., k], scores [..., k])`. Note that "who is selected" uses the
selection score (bias included) while the "weights" use the raw sigmoid (bias excluded) —
the key distinction of DeepSeek-style routing.
-The two models each use one: edge0-35b uses `select_from_logits`, edge0-10b uses
+The two models each use one: edge0-35b uses `select_from_logits`, edge0-8b uses
`group_select_from_logits`; the prerouter stager also reuses these two functions for
cross-token teacher routing.
diff --git a/docs/prerouter.md b/docs/prerouter.md
index cba0e2f..6bf2d84 100644
--- a/docs/prerouter.md
+++ b/docs/prerouter.md
@@ -62,7 +62,7 @@ per `start_layer`/`owners`. Family differences are expressed through three hooks
## Model tiers
-| | edge0-35b | edge0-10b |
+| | edge0-35b | edge0-8b |
|---|---|---|
| Routing family | SOFTMAX_TOPK (precise softmax → top-k → normalization) | SIGMOID_GROUP (sigmoid + group-limited top-k, n_group 8 / topk_group 4, routed_scaling 2.5) |
| start_layer | 7 (first consumer layer) | 1 |
@@ -70,7 +70,7 @@ per `start_layer`/`owners`. Family differences are expressed through three hooks
| hidden | 512 (fp16) | 512 (fp16) |
| feature_topk | executed | executed |
| patch_call | True (qwen3_next needs the patch) | False (built-in model hook) |
-| weight file | `artifacts/prerouter_edge0_35b_k4.safetensors` | `artifacts/prerouter_edge0_10b.safetensors` |
+| weight file | `artifacts/prerouter_edge0_35b_k4.safetensors` | `artifacts/prerouter_edge0_8b.safetensors` |
## Tests and regression
diff --git a/docs/streaming.md b/docs/streaming.md
index 94ea65c..8ab41a7 100644
--- a/docs/streaming.md
+++ b/docs/streaming.md
@@ -75,7 +75,7 @@ With `use_compile`, the staged/exact paths are wrapped in `mx.compile`:
| `prefill_hot` | hot stack size during prefill | 0 |
| `use_compile` / `top_k` | compile wrapping / routing top-k override | True / None |
-Presets: `staged_k4()` (edge0-35b: staged with 4 slots + 32 hot + whole-layer prefill over 12 layers), `staged_k8()` (edge0-10b: staged with 8 slots, no hot residency). Both tiers share `cache_slots=64`.
+Presets: `staged_k4()` (edge0-35b: staged with 4 slots + 32 hot + whole-layer prefill over 12 layers), `staged_k8()` (edge0-8b: staged with 8 slots, no hot residency). Both tiers share `cache_slots=64`.
## Why Whole-Layer Loading Is Also Fast
diff --git a/examples/bench.py b/examples/bench.py
index 2fcec36..971c459 100644
--- a/examples/bench.py
+++ b/examples/bench.py
@@ -31,7 +31,7 @@ from edge0.sampling import sample
PROMPTS = {
"edge0-35b": "什么是混合专家模型(MoE)?它和普通 Transformer 有什么区别?",
- "edge0-10b": "9.11 和 9.8 哪个大?请仔细比较。",
+ "edge0-8b": "9.11 和 9.8 哪个大?请仔细比较。",
}
DEFAULT_PROMPT = "什么是混合专家模型(MoE)?简单介绍一下。"
diff --git a/examples/demo.py b/examples/demo.py
index 0a4efb0..e4e0641 100644
--- a/examples/demo.py
+++ b/examples/demo.py
@@ -5,7 +5,7 @@ Install:
Run (checkpoint path or registered tier name):
.venv/bin/python examples/demo.py --model-dir /path/to/qwen35/model
- .venv/bin/python examples/demo.py --model edge0-10b
+ .venv/bin/python examples/demo.py --model edge0-8b
Three lines do everything: build the engine (prerouter + LoRA + SSD
offload installed automatically from the tier config), template the
diff --git a/scripts/convert_adapters_legacy.py b/scripts/convert_adapters_legacy.py
index 34c301d..6483c33 100644
--- a/scripts/convert_adapters_legacy.py
+++ b/scripts/convert_adapters_legacy.py
@@ -39,8 +39,8 @@ NPZ_DIRS = [
SOURCES = [
("pregate_<family>.npz", "edge0-35b", "prerouter"),
("lora_<family>.npz", "edge0-35b", "lora"),
- ("pregate_<family>.npz", "edge0-10b", "prerouter"),
- ("lora_<family>.npz", "edge0-10b", "lora"),
+ ("pregate_<family>.npz", "edge0-8b", "prerouter"),
+ ("lora_<family>.npz", "edge0-8b", "lora"),
]
@@ -82,8 +82,8 @@ def convert(source: Path, tier: str, kind: str, force: bool = False) -> Path:
name = {
("edge0-35b", "prerouter"): "prerouter_edge0_35b_k4.safetensors",
("edge0-35b", "lora"): "lora_edge0_35b_k4.safetensors",
- ("edge0-10b", "prerouter"): "prerouter_edge0_10b.safetensors",
- ("edge0-10b", "lora"): "lora_edge0_10b.safetensors",
+ ("edge0-8b", "prerouter"): "prerouter_edge0_8b.safetensors",
+ ("edge0-8b", "lora"): "lora_edge0_8b.safetensors",
}[(tier, kind)]
out = ARTIFACTS / name
if out.exists() and not force:
diff --git a/scripts/e2e_smoke.py b/scripts/e2e_smoke.py
index 65a0074..1052a85 100644
--- a/scripts/e2e_smoke.py
+++ b/scripts/e2e_smoke.py
@@ -21,7 +21,7 @@ from edge0.config import GenerationConfig
from edge0.streaming.options import LayerOptions
QWEN_DEFAULT = os.environ.get("EDGE0_35B_MODEL", "/path/to/qwen35/model")
-LING_DEFAULT = os.environ.get("EDGE0_10B_MODEL", "/path/to/ling/model")
+LING_DEFAULT = os.environ.get("EDGE0_8B_MODEL", "/path/to/ling/model")
def _prompt_ids(model_dir: str, text: str) -> list[int]:
@@ -74,7 +74,7 @@ def main() -> int:
return 1
print("qwen staged == exact greedy: OK")
- run("edge0-10b", args.ling_dir, ling_prompt, args.max_new, label="[staged]")
+ run("edge0-8b", args.ling_dir, ling_prompt, args.max_new, label="[staged]")
print("edge0 e2e smoke OK")
return 0
diff --git a/scripts/fetch_models.py b/scripts/fetch_models.py
index eb109b0..d43d93d 100755
--- a/scripts/fetch_models.py
+++ b/scripts/fetch_models.py
@@ -13,7 +13,7 @@ Repo ids default to the official releases (override via the environment
if you mirror them):
export EDGE0_35B_REPO=Edge0/Edge0-35b-a3b-preview # default
- export EDGE0_10B_REPO=Edge0/Edge0-10b-a1b-preview # default
+ export EDGE0_8B_REPO=Edge0/Edge0-8b-a1b-preview # default
Usage:
python scripts/fetch_models.py --tier edge0-35b --target-dir models
@@ -34,7 +34,7 @@ from pathlib import Path
TIER_REPOS = {
"edge0-35b": ("EDGE0_35B_REPO", "Edge0/Edge0-35b-a3b-preview"),
- "edge0-10b": ("EDGE0_10B_REPO", "Edge0/Edge0-10b-a1b-preview"),
+ "edge0-8b": ("EDGE0_8B_REPO", "Edge0/Edge0-8b-a1b-preview"),
}
diff --git a/scripts/verify_alignment.py b/scripts/verify_alignment.py
index 8742bd5..300a8ec 100644
--- a/scripts/verify_alignment.py
+++ b/scripts/verify_alignment.py
@@ -26,7 +26,7 @@ PROMPTS = {
"ling": "你好,用一句话介绍海滨城市。",
}
QMODEL = os.environ.get("EDGE0_35B_MODEL", "/path/to/qwen35/model")
-LMODEL = os.environ.get("EDGE0_10B_MODEL", "/path/to/ling/model")
+LMODEL = os.environ.get("EDGE0_8B_MODEL", "/path/to/ling/model")
QDIR = os.path.dirname(QMODEL)
LDIR = os.path.dirname(LMODEL)
RESULT = os.path.join(os.path.dirname(os.path.abspath(__file__)),
diff --git a/src/edge0/attention/spec.py b/src/edge0/attention/spec.py
index 88928e3..0d535d8 100644
--- a/src/edge0/attention/spec.py
+++ b/src/edge0/attention/spec.py
@@ -2,7 +2,7 @@
edge0 does not re-implement attention kernels: the vendored base model
implementations (``edge0.backends.mlx._impl``) carry the kernels (GQA /
-GatedDeltaNet for edge0-35b, MLA / DeltaNet for edge0-10b). This module
+GatedDeltaNet for edge0-35b, MLA / DeltaNet for edge0-8b). This module
describes attention so the framework can introspect a model — cache
sizing, layer roles, documentation, and future kernels — without knowing
the concrete implementation.
diff --git a/src/edge0/cli.py b/src/edge0/cli.py
index 580b223..cefa632 100644
--- a/src/edge0/cli.py
+++ b/src/edge0/cli.py
@@ -23,12 +23,12 @@ from edge0.registry import MODEL_REGISTRY
# (no built-in paths: every machine resolves its own checkpoints).
TIER_ENV = {
"edge0-35b": "EDGE0_35B_MODEL",
- "edge0-10b": "EDGE0_10B_MODEL",
+ "edge0-8b": "EDGE0_8B_MODEL",
}
DEMO_PROMPTS = {
"edge0-35b": "Hello! Write one short sentence about the seaside.",
- "edge0-10b": "你好,用一句话介绍海滨城市。",
+ "edge0-8b": "你好,用一句话介绍海滨城市。",
}
@@ -73,7 +73,7 @@ def _resolve_model(args) -> tuple[str | None, str | None]:
"""Resolve the positional ``model`` argument.
``edge0 serve <model>`` accepts either a registered tier name
- (``edge0-35b`` / ``edge0-10b`` — checkpoint located via the matching
+ (``edge0-35b`` / ``edge0-8b`` — checkpoint located via the matching
``EDGE0_<TIER>_MODEL`` environment variable) or a checkpoint path
(tier auto-detected from the checkpoint's ``config.json``).
diff --git a/src/edge0/engine/__init__.py b/src/edge0/engine/__init__.py
index 56a6161..b14a5d2 100644
--- a/src/edge0/engine/__init__.py
+++ b/src/edge0/engine/__init__.py
@@ -2,7 +2,7 @@
``edge0.engine.base.Edge0Engine`` owns the shared prefill / step /
generate loops; ``edge0.engine.qwen.Qwen35Engine`` and
-``edge0.engine.ling.Ling10BEngine`` implement the family hooks
+``edge0.engine.ling.Ling8BEngine`` implement the family hooks
(streaming install, prerouter wiring, step-boundary staging).
"""
@@ -12,13 +12,13 @@ from edge0.engine.hooks import (
make_intra_after_layer,
make_prefill_before_layer,
)
-from edge0.engine.ling import Ling10BEngine
+from edge0.engine.ling import Ling8BEngine
from edge0.engine.qwen import Qwen35Engine
__all__ = [
"Edge0Engine",
"Qwen35Engine",
- "Ling10BEngine",
+ "Ling8BEngine",
"make_history_prefetch",
"make_intra_after_layer",
"make_prefill_before_layer",
diff --git a/src/edge0/engine/ling.py b/src/edge0/engine/ling.py
index 949e5c0..b8a0344 100644
--- a/src/edge0/engine/ling.py
+++ b/src/edge0/engine/ling.py
@@ -1,4 +1,4 @@
-"""edge0-10b engine: Ling 3.0 hybrid (MLA + MoE, 10B tier).
+"""edge0-8b engine: Ling 3.0 hybrid (MLA + MoE, 8B tier).
Port of the deployment's ling ``engine.py`` hybrid-prerouter path:
@@ -78,7 +78,7 @@ def load_installed(model_dir: str, cfg):
model=model, spec=spec, pspec=cfg.prerouter,
state=pg_state, stream_layers=stream_layers,
top_k=cfg.prerouter_top_k)
- print(f"[edge0-10b] prerouter installed: {len(heads)} heads, "
+ print(f"[edge0-8b] prerouter installed: {len(heads)} heads, "
f"start={cfg.prerouter.start_layer}, "
f"K={cfg.prerouter_top_k}", flush=True)
installs = dict(all_stream_layers=all_stream,
@@ -87,10 +87,10 @@ def load_installed(model_dir: str, cfg):
return model, model_config, shards, installs
-class Ling10BEngine(Edge0Engine):
+class Ling8BEngine(Edge0Engine):
"""Streaming Ling-3.0 engine (staged decode + hybrid prerouter)."""
- name = "edge0-10b"
+ name = "edge0-8b"
def __init__(self, model_dir: str, cfg, tokenizer=None,
think: bool = False):
@@ -166,7 +166,7 @@ class Ling10BEngine(Edge0Engine):
self.reset()
except Exception: # noqa: BLE001 — advisory only
pass
- print(f"[edge0-10b] prewarm done in "
+ print(f"[edge0-8b] prewarm done in "
f"{_t.perf_counter() - t0:.1f}s", flush=True)
# ---- forward ----------------------------------------------------------
diff --git a/src/edge0/models/__init__.py b/src/edge0/models/__init__.py
index 9804d40..d89e01a 100644
--- a/src/edge0/models/__init__.py
+++ b/src/edge0/models/__init__.py
@@ -1,7 +1,7 @@
"""Model adapters: one package per tier, registered on import.
``edge0.models.base`` carries the shared ``ModelConfig``; the tier
-packages (``edge0.models.edge0_35b``, ``edge0.models.edge0_10b``)
+packages (``edge0.models.edge0_35b``, ``edge0.models.edge0_8b``)
define their family configs and the ``build_model`` / ``build_engine``
entry points, then call ``register_model`` so ``AutoConfig`` /
``AutoModel`` / ``AutoEngine`` can resolve them by name.
@@ -12,6 +12,6 @@ from __future__ import annotations
from edge0.models.base import ModelConfig
# Importing the tier packages populates edge0.registry.MODEL_REGISTRY.
-from edge0.models import edge0_10b, edge0_35b # noqa: F401
+from edge0.models import edge0_8b, edge0_35b # noqa: F401
-__all__ = ["ModelConfig", "edge0_10b", "edge0_35b"]
+__all__ = ["ModelConfig", "edge0_8b", "edge0_35b"]
diff --git a/src/edge0/models/edge0_10b/__init__.py b/src/edge0/models/edge0_8b/__init__.py
similarity index 80%
rename from src/edge0/models/edge0_10b/__init__.py
rename to src/edge0/models/edge0_8b/__init__.py
index 5102fc1..7c8f4db 100644
--- a/src/edge0/models/edge0_10b/__init__.py
+++ b/src/edge0/models/edge0_8b/__init__.py
@@ -1,4 +1,4 @@
-"""edge0-10b adapter: Ling 3.0 hybrid (MLA + MoE, 10B tier).
+"""edge0-8b adapter: Ling 3.0 hybrid (MLA + MoE, 8B tier).
Family facts (production profile, verified against the reference checkpoint):
@@ -26,13 +26,13 @@ from edge0.registry import register_model
from edge0.streaming.options import LayerOptions
-class Ling10BConfig(ModelConfig):
- """edge0-10b family config (Ling 3.0 hybrid, K=8)."""
+class Ling8BConfig(ModelConfig):
+ """edge0-8b family config (Ling 3.0 hybrid, K=8)."""
@classmethod
- def _defaults(cls, model_dir: str) -> "Ling10BConfig":
+ def _defaults(cls, model_dir: str) -> "Ling8BConfig":
return cls(
- name="edge0-10b",
+ name="edge0-8b",
model_dir=model_dir,
moe_spec=MoESpec(
num_experts=128, top_k=8, intermediate_size=512,
@@ -52,11 +52,11 @@ class Ling10BConfig(ModelConfig):
feature_topk="executed",
owners=tuple(range(7, 23)),
weights_file=artifact(
- "prerouter_edge0_10b.safetensors", model_dir),
+ "prerouter_edge0_8b.safetensors", model_dir),
patch_call=False,
),
prerouter_top_k=8,
- lora=artifact("lora_edge0_10b.safetensors", model_dir),
+ lora=artifact("lora_edge0_8b.safetensors", model_dir),
lora_r=16, lora_alpha=32.0,
gen=GenerationConfig(
temperature=0.7, top_p=0.95, top_k=64,
@@ -77,25 +77,25 @@ def build_model(model_dir: str | None = None, **overrides):
installed (the engine's own build path, DRY-shared)."""
from edge0.engine.ling import load_installed
- cfg = Ling10BConfig.from_pretrained(model_dir, **overrides)
+ cfg = Ling8BConfig.from_pretrained(model_dir, **overrides)
model, _mcfg, _shards, _installs = load_installed(cfg.model_dir, cfg)
return model
def build_engine(model_dir: str | None = None, **overrides):
- """Build a ready-to-generate Ling10BEngine.
+ """Build a ready-to-generate Ling8BEngine.
``think`` (THINK_MODE parity, default False) selects the
chat-template thinking mode for ``encode_chat``.
"""
- from edge0.engine.ling import Ling10BEngine
+ from edge0.engine.ling import Ling8BEngine
think = overrides.pop("think", False)
- cfg = Ling10BConfig.from_pretrained(model_dir, **overrides)
- return Ling10BEngine(cfg.model_dir, cfg, think=think)
+ cfg = Ling8BConfig.from_pretrained(model_dir, **overrides)
+ return Ling8BEngine(cfg.model_dir, cfg, think=think)
-register_model("edge0-10b", sys.modules[__name__])
+register_model("edge0-8b", sys.modules[__name__])
-Config = Ling10BConfig # registry contract: adapter.Config
+Config = Ling8BConfig # registry contract: adapter.Config
diff --git a/src/edge0/registry.py b/src/edge0/registry.py
index 3827c40..1585051 100644
--- a/src/edge0/registry.py
+++ b/src/edge0/registry.py
@@ -17,9 +17,9 @@ TYPE_ALIASES: dict[str, str] = {
"qwen3_5_moe_text": "edge0-35b",
"qwen3_5_moe": "edge0-35b",
# ling family (config.json carries architectures, not model_type)
- "bailing_hybrid": "edge0-10b",
- "bailing_moe_linear": "edge0-10b",
- "BailingMoeV3ForCausalLM": "edge0-10b",
+ "bailing_hybrid": "edge0-8b",
+ "bailing_moe_linear": "edge0-8b",
+ "BailingMoeV3ForCausalLM": "edge0-8b",
}
diff --git a/src/edge0/streaming/options.py b/src/edge0/streaming/options.py
index cf8f83b..34a27a3 100644
--- a/src/edge0/streaming/options.py
+++ b/src/edge0/streaming/options.py
@@ -99,7 +99,7 @@ class LayerOptions:
@classmethod
def staged_k8(cls, **overrides) -> "LayerOptions":
- """edge0-10b tier: staged decode with K=8 (native routing width),
+ """edge0-8b tier: staged decode with K=8 (native routing width),
no hot pins, E3b whole-layer prefill."""
return cls(
staged=True, staged_replace=False, staged_n=8,
@@ -110,7 +110,7 @@ class LayerOptions:
@classmethod
def prod_k8(cls, **overrides) -> "LayerOptions":
- """edge0-10b tier: deployment production profile.
+ """edge0-8b tier: deployment production profile.
Mirrors the reference deployment profile: STAGED_DECODE=0
(staged decode is OFF — the deployment verified staged decode on
diff --git a/tests/test_registry.py b/tests/test_registry.py
index 6cdcb47..a50e2c2 100644
--- a/tests/test_registry.py
+++ b/tests/test_registry.py
@@ -10,17 +10,17 @@ from edge0.registry import MODEL_REGISTRY, TYPE_ALIASES
def test_both_tiers_registered():
- assert set(MODEL_REGISTRY) == {"edge0-35b", "edge0-10b"}
+ assert set(MODEL_REGISTRY) == {"edge0-35b", "edge0-8b"}
def test_type_aliases_resolve():
assert TYPE_ALIASES["qwen3_5_moe"] == "edge0-35b"
assert TYPE_ALIASES["qwen3_5_moe_text"] == "edge0-35b"
- assert TYPE_ALIASES["bailing_hybrid"] == "edge0-10b"
- assert TYPE_ALIASES["bailing_moe_linear"] == "edge0-10b"
+ assert TYPE_ALIASES["bailing_hybrid"] == "edge0-8b"
+ assert TYPE_ALIASES["bailing_moe_linear"] == "edge0-8b"
[email protected]("name", ["edge0-35b", "edge0-10b"])
[email protected]("name", ["edge0-35b", "edge0-8b"])
def test_auto_config_defaults(name):
cfg = AutoConfig.from_pretrained(name=name)
assert cfg.name == name
@@ -62,8 +62,8 @@ def test_qwen35_profile():
assert cfg.port == 8085
-def test_ling10b_profile():
- cfg = AutoConfig.from_pretrained(name="edge0-10b")
+def test_ling8b_profile():
+ cfg = AutoConfig.from_pretrained(name="edge0-8b")
assert cfg.moe_spec.num_experts == 128
assert cfg.moe_spec.top_k == 8
assert cfg.moe_spec.router.value == "sigmoid_group"
@@ -80,13 +80,13 @@ def test_ling10b_profile():
def test_override_and_reject():
- cfg = AutoConfig.from_pretrained(name="edge0-10b", prerouter=None,
+ cfg = AutoConfig.from_pretrained(name="edge0-8b", prerouter=None,
lora="", prerouter_top_k=0)
assert cfg.prerouter is None
assert cfg.lora == ""
assert cfg.prerouter_top_k == 0
with pytest.raises(TypeError, match="unknown"):
- AutoConfig.from_pretrained(name="edge0-10b", bogus_field=1)
+ AutoConfig.from_pretrained(name="edge0-8b", bogus_field=1)
def test_unknown_name_lists_registry():