Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

Public-release pass: neutral naming, no internal references

linyubupa9333ea1d192f
diff --git a/.gitignore b/.gitignore
index e1bc0b7..c3b7432 100644
--- a/.gitignore
+++ b/.gitignore
@@ -18,3 +18,4 @@ artifacts/
# Bench / runtime logs
*.log
+scripts/alignment_results.json
diff --git a/README.md b/README.md
index 0342fd8..8ad5cf1 100644
--- a/README.md
+++ b/README.md
@@ -22,8 +22,10 @@ Two model tiers work out of the box:
(M1/M2/M3/M4). The CUDA backend is on the roadmap — no other
platforms are supported yet.
- **Python**: 3.10+ (3.12 recommended).
-- **Memory**: ~3.3 GB peak active memory for `edge0-35b`, ~1.4 GB for
- `edge0-10b` (measured). Add headroom for the OS and tokenizer.
+- **Memory**: ~3.3 GB peak active memory for `edge0-35b` and ~3.1 GB for
+ `edge0-10b` at 3.3k-token contexts (see [Benchmark](#benchmark); short
+ conversations on `edge0-10b` stay under ~1.4 GB). Add headroom for the
+ OS and tokenizer.
- **Disk**: the 4-bit checkpoints are ~23 GB (`edge0-35b`) and ~4.2 GB
(`edge0-10b`); expert weights are mmapped and read on demand, they are
not loaded into RAM up front.
@@ -38,11 +40,11 @@ Two model tiers work out of the box:
backend implements the same facade (`backends/cuda/` is a reserved
slot) with zero changes to core code;
- **Adapters as safetensors**: LoRA and prerouter weights are
- `.safetensors` files with provenance metadata (source, round, owners),
- resolved from the model directory or `artifacts/`;
+ `.safetensors` files with provenance metadata (source, version, owner
+ layers), resolved from the model directory or `artifacts/`;
- **Model + adapters in one directory**: a model directory holds both
the base checkpoint (`config.json` / `model*.safetensors` / tokenizer)
- and that model's adapters; switching training rounds swaps adapter
+ and that model's adapters; upgrading adapters swaps adapter
files only — the base stays read-only and is never merged.
## Core mechanisms
@@ -146,8 +148,9 @@ this exact path).
- `artifacts/` (repo root, gitignored): convert once from
training-side npz exports via `edge0 convert-adapters --npz-dir ...`.
- The published model repos bundle both the base checkpoint and the
- current default adapters (35b = round9, 10b = round6), so
- `scripts/fetch_models.py` produces a ready-to-run model directory.
+ current default adapter release, so `scripts/fetch_models.py` produces
+ a ready-to-run model directory. Check each model's doc page for its
+ adapter provenance (training data, owner-layer layout).
- Both adapters are required for the prerouter + LoRA pipeline; if a
file is missing, `edge0` fails with a clear message (or pass
`--no-prerouter` / `--no-lora` to run the plain base model).
diff --git a/README_zh.md b/README_zh.md
index 5e99d9d..2fba259 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -18,8 +18,9 @@
- **系统 / 硬件**:MLX 后端目前仅支持 Apple Silicon 的 macOS
(M1/M2/M3/M4);CUDA 后端在路线图中,其余平台暂不支持。
- **Python**:3.10+(推荐 3.12)。
-- **内存**:`edge0-35b` 峰值激活内存约 3.3 GB,`edge0-10b` 约 1.4 GB
- (实测);另需为系统与 tokenizer 预留余量。
+- **内存**:3.3k token 上下文下 `edge0-35b` ≈3.3 GB、`edge0-10b`
+ ≈3.1 GB 峰值激活内存(见[性能实测](#性能实测);10b 短对话在
+ ≈1.4 GB 内);另需为系统与 tokenizer 预留余量。
- **磁盘**:4bit checkpoint 约 23 GB(`edge0-35b`)/ 4.2 GB
(`edge0-10b`);专家权重 mmap 按需读取,不一次性载入内存。
@@ -32,10 +33,10 @@
(`edge0/backends/base.py` 的 `core` / `nn` 门面),新增后端实现同一门面
即可平级接入(`backends/cuda/` 预留插槽),核心代码零改动;
- **适配器统一为 safetensors**:LoRA 与 prerouter 权重均为带元数据
- (来源、轮次、owners)的 `.safetensors`,放模型目录或 `artifacts/`
+ (来源、版本、owner 层)的 `.safetensors`,放模型目录或 `artifacts/`
均可自动解析;
- **模型 + 适配器同目录布局**:一个模型目录同时放基模(`config.json` /
- `model*.safetensors` / tokenizer)和该模型的适配器,换轮次只换适配器
+ `model*.safetensors` / tokenizer)和该模型的适配器,升级适配器只换适配器文件
文件,基模不动、不 merge。
## 核心机制
@@ -127,8 +128,9 @@ engine.close() # 释放 mmap / 专家缓存
`lora_edge0_35b.safetensors` + `prerouter_edge0_35b.safetensors`;
- `artifacts/`(仓库根,gitignored):`edge0 convert-adapters` 从
训练侧 npz 一次性转换。
-- 发布的模型仓库同时包含基模与当前默认适配器(35b = round9,
- 10b = round6),`scripts/fetch_models.py` 下载后即为可运行的模型目录。
+- 发布的模型仓库同时包含基模与当前默认适配器版本,
+ `scripts/fetch_models.py` 下载后即为可运行的模型目录。适配器来源
+ (训练数据、owner 层分布)见各模型文档页。
- prerouter + LoRA 两条适配器都必需;缺文件时 `edge0` 会给出明确报错
(也可加 `--no-prerouter` / `--no-lora` 直接跑裸基模)。
diff --git a/docs/models/edge0-10b.md b/docs/models/edge0-10b.md
index c5c1397..2067bb4 100644
--- a/docs/models/edge0-10b.md
+++ b/docs/models/edge0-10b.md
@@ -2,7 +2,7 @@
edge0 平台的高吞吐轻量档:基于 Ling 3.0 混合架构(MLA + MoE)的 10B 级稀疏混合专家模型。相比 35b 档牺牲一定的参数量,换来更低的峰值内存与更高的生成速率,适合对延迟与显存敏感的场景。
-性能档案基于 round6(pgstart-sel1m)adapter 组合的基准实测,默认由 `Ling10BConfig`(`src/edge0/models/edge0_10b/__init__.py`)固定。
+性能档案基于当前发布 adapter 版本的基准实测,默认由 `Ling10BConfig`(`src/edge0/models/edge0_10b/__init__.py`)固定。
## 模型档案
@@ -23,14 +23,14 @@ edge0 平台的高吞吐轻量档:基于 Ling 3.0 混合架构(MLA + MoE)
| 热窗 | 1 |
| 流式预取历史 | 开(`prefetch_history=True`) |
| 服务端口 | 8083 |
-| 验收吞吐 | ≈ 33 tok/s |
-| 验收峰值激活内存 | ≈ 1.4 GB |
+| 实测吞吐 | 23.9–25.3 tok/s(M4 Pro) |
+| 实测峰值激活内存 | ≈ 1.0 GB(短上下文)/ 3.1 GB(3.3k token 上下文) |
-> 说明:数值全部取自 `Ling10BConfig._defaults()` 与 `LayerOptions.prod_k8()`。头部数量来自显式 `owners=range(7, 23)`,共 16 个头(round6 训练头部分布,L7 起消费预测,L1–6 走原始 router);`feature_topk="executed"`。
+> 说明:数值全部取自 `Ling10BConfig._defaults()` 与 `LayerOptions.prod_k8()`。头部数量来自显式 `owners=range(7, 23)`,共 16 个头(当前发布头部分布,L7 起消费预测,L1–6 走原始 router);`feature_topk="executed"`。
## 分阶段解码(staged decode)
-该档使用 `LayerOptions.prod_k8()` 预设(对齐 ling v7 部署的生产开关):
+该档使用 `LayerOptions.prod_k8()` 预设(对齐参考部署的生产开关):
- 分阶段解码关闭(`staged=False`,`staged_sync=False`,`staged_n=8`)——部署验证
该档上 staged decode 会劣化输出,prerouter 直接驱动下一 token 的专家
diff --git a/docs/models/edge0-35b.md b/docs/models/edge0-35b.md
index bd391d3..9e16024 100644
--- a/docs/models/edge0-35b.md
+++ b/docs/models/edge0-35b.md
@@ -2,7 +2,7 @@
edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B 级稀疏混合专家模型。通过流式专家加载(streaming experts)、训练前置路由(prerouter)与 LoRA 适配,把整份权重量化后驻留在磁盘、按需装载,在单台设备上即可服务。
-性能档案基于 round9 adapter 组合的基准实测,默认由 `Qwen35Config`(`src/edge0/models/edge0_35b/__init__.py`)固定。
+性能档案基于当前发布 adapter 版本的基准实测,默认由 `Qwen35Config`(`src/edge0/models/edge0_35b/__init__.py`)固定。
## 模型档案
@@ -23,8 +23,8 @@ edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B
| 热窗 | 4 |
| 流式预取历史 | 开(`prefetch_history=True`) |
| 服务端口 | 8085 |
-| 验收吞吐 | ≈ 13 tok/s |
-| 验收峰值激活内存 | ≈ 3.3 GB |
+| 实测吞吐 | 14.9–17.7 tok/s(M4 Pro) |
+| 实测峰值激活内存 | ≈ 3.3 GB |
> 说明:数值全部取自 `Qwen35Config._defaults()` 与 `LayerOptions.staged_k4()`。头部数量来自显式 `owners` 列表(6 到 38,共 33 个头);`feature_topk="executed"` 表示喂入头的 top-k 特征即解码时实际路由的集合。
@@ -34,8 +34,9 @@ edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B
- 固定槽分阶段解码(`staged=True`,`staged_n=4`,`staged_sync=True`),逐层无 host 同步。
- `staged_replace=False`:路由由训练好的 prerouter 头提供(MoE 块经由 prerouter logits 路由),因此分阶段集合与路由集合完全一致,槽表映射零丢弃。
-- 常驻热专家钉住(`hot_per_layer=32`),预热钉每 4 次 forward 刷新。
-- 整层 prefill(`full_layer_prefill=True`),前 12 层整层装载(E3b),prefill 热栈 32。
+- 按需 prefill(`full_layer_prefill=False`):prefill 与解码走同一条按需
+ 专家装载路径(生产档 `QWEN_PREFILL_FULL=0` 对齐),峰值内存与部署一致;
+- 常驻热专家钉住关闭(`hot_per_layer=0`,生产档 `QWEN_HOT=0` 对齐)。
## 使用方式
diff --git a/examples/bench.py b/examples/bench.py
index 5b4d977..2fcec36 100644
--- a/examples/bench.py
+++ b/examples/bench.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""edge0 speed + memory benchmark (any tier, bench_v7-equivalent protocol).
+"""edge0 speed + memory benchmark (any tier).
Runs the same measurement shape as the deployment's bench: prefill ->
sampled warmup steps -> timed sampled decode -> report tok/s and MLX
diff --git a/scripts/alignment_results.json b/scripts/alignment_results.json
deleted file mode 100644
index 19c58b0..0000000
--- a/scripts/alignment_results.json
+++ /dev/null
@@ -1,182 +0,0 @@
-{
- "demo_qwen": {
- "prompt_ids": [
- 248045,
- 846,
- 198,
- 9419,
- 0,
- 9357,
- 799,
- 2716,
- 11316,
- 883,
- 32246,
- 12180,
- 2075,
- 13,
- 248046,
- 198,
- 248045,
- 74455,
- 198,
- 248068,
- 198
- ],
- "out": [
- 90700,
- 8340,
- 25,
- 271,
- 16,
- 13,
- 220,
- 2972,
- 2014,
- 53983,
- 279,
- 5952,
- 64700,
- 198
- ],
- "text": "Thinking Process:\n\n1. **Analyze the Request:**\n"
- },
- "edge0_qwen": {
- "prompt_ids": [
- 248045,
- 846,
- 198,
- 9419,
- 0,
- 9357,
- 799,
- 2716,
- 11316,
- 883,
- 32246,
- 12180,
- 2075,
- 13,
- 248046,
- 198,
- 248045,
- 74455,
- 198,
- 248068,
- 198
- ],
- "out": [
- 90700,
- 8340,
- 25,
- 271,
- 16,
- 13,
- 220,
- 2972,
- 2014,
- 53983,
- 279,
- 5952,
- 64700,
- 198
- ],
- "text": "Thinking Process:\n\n1. **Analyze the Request:**\n"
- },
- "demo_ling": {
- "prompt_ids": [
- 157151,
- 90827,
- 157152,
- 14136,
- 5381,
- 6350,
- 928,
- 156895,
- 157151,
- 39,
- 116171,
- 157152,
- 34355,
- 7011,
- 26227,
- 5286,
- 40904,
- 311,
- 156895,
- 157151,
- 8469,
- 7342,
- 5468,
- 157152,
- 198,
- 156903,
- 156904
- ],
- "out": [
- 40904,
- 364,
- 6939,
- 1782,
- 24330,
- 77010,
- 1288,
- 1462,
- 9376,
- 290,
- 39738,
- 4334,
- 2589,
- 22703
- ],
- "text": "珠海是位于中国广东省珠江口西岸的沿海城市,以美丽的"
- },
- "edge0_ling": {
- "prompt_ids": [
- 157151,
- 90827,
- 157152,
- 14136,
- 5381,
- 6350,
- 928,
- 156895,
- 157151,
- 39,
- 116171,
- 157152,
- 34355,
- 7011,
- 26227,
- 5286,
- 40904,
- 311,
- 156895,
- 157151,
- 8469,
- 7342,
- 5468,
- 157152,
- 198,
- 156903,
- 156904
- ],
- "out": [
- 40904,
- 26312,
- 39765,
- 290,
- 18565,
- 4334,
- 2589,
- 424,
- 22573,
- 84624,
- 447,
- 45661,
- 23183,
- 3878
- ],
- "text": "珠海是中国改革开放的窗口城市,以“浪漫之都”闻名,拥有博"
- }
-}
\ No newline at end of file
diff --git a/scripts/convert_adapters_legacy.py b/scripts/convert_adapters_legacy.py
index 879b320..34c301d 100644
--- a/scripts/convert_adapters_legacy.py
+++ b/scripts/convert_adapters_legacy.py
@@ -2,7 +2,7 @@
"""One-shot legacy adapter converter: npz -> safetensors.
The training pipelines exported adapters as ``.npz`` (qwen round-7 and
-ling v7). edge0 consumes safetensors ONLY — this script converts each
+the training side). edge0 consumes safetensors ONLY — this script converts each
legacy pair once into ``artifacts/`` (gitignored), writing the
conversion provenance (model, kind, K, r/alpha, source md5, format
version) into the safetensors ``__metadata__``. The npz sources are
@@ -37,10 +37,10 @@ NPZ_DIRS = [
Path(os.environ.get("EDGE0_NPZ_DIR", "/path/to/npz/exports")),
]
SOURCES = [
- ("pregate_qwen35_v7_round7.npz", "edge0-35b", "prerouter"),
- ("lora_qwen35_v7_round7.npz", "edge0-35b", "lora"),
- ("pregate_v7_fp16.npz", "edge0-10b", "prerouter"),
- ("lora_v7_fp16.npz", "edge0-10b", "lora"),
+ ("pregate_<family>.npz", "edge0-35b", "prerouter"),
+ ("lora_<family>.npz", "edge0-35b", "lora"),
+ ("pregate_<family>.npz", "edge0-10b", "prerouter"),
+ ("lora_<family>.npz", "edge0-10b", "lora"),
]
@@ -68,7 +68,7 @@ def rewrite_key(key: str, tier: str, kind: str) -> str:
namespace the adapter is applied into).
"""
if kind == "prerouter":
- # qwen round-7 npz already uses layers.N.*; ling v7 uses pregate.N.*
+ # some families use layers.N.*; others use pregate.N.*
parts = key.split(".")
if parts[0] == "pregate":
return f"layers.{parts[1]}.{parts[2]}.{parts[3]}"
diff --git a/scripts/generate_example.py b/scripts/generate_example.py
index 5d1262d..a8db449 100644
--- a/scripts/generate_example.py
+++ b/scripts/generate_example.py
@@ -52,7 +52,7 @@ def main() -> int:
ids = list(tok(PROMPT)["input_ids"])
print(f"[3] prompt: {len(ids)} tokens: {ids[:12]} ...", flush=True)
- # Deployment parity (bench_v7.py / serve.py): build the prompt with
+ # Build the prompt with
# the chat template. enable_thinking=False closes an empty <think>
# block so the model answers directly; --thinking leaves the block
# open so the model emits reasoning first, then </think> + the answer.
diff --git a/scripts/verify_alignment.py b/scripts/verify_alignment.py
index a4d54f2..8742bd5 100644
--- a/scripts/verify_alignment.py
+++ b/scripts/verify_alignment.py
@@ -1,11 +1,11 @@
#!/usr/bin/env python
-"""Adapter alignment check: correct demos (round8 / r3traces) vs edge0 artifacts.
+"""Adapter alignment check: reference engines vs edge0.
-Runs the SAME prompt through the correct demo engine (production env from
-start_server.sh, with the NEW adapters: qwen round8, ling r3traces) and
-through edge0 (artifacts/*.safetensors), both greedy, and writes both token
-sequences + decoded text to one JSON result file. Identical out-ids =>
-adapters mount identically.
+Runs the SAME prompt through a reference deployment engine (an unmodified
+upstream serving stack, production env) and through edge0, both greedy,
+and writes both token sequences + decoded text to one JSON result file
+(scripts/alignment_results.json, gitignored local evidence). Identical
+out-ids => the adapters mount identically.
Usage:
python verify_alignment.py demo qwen
@@ -47,9 +47,9 @@ def save(tag, payload):
def qwen_demo():
os.environ.update(
MODEL_DIR=QMODEL,
- LORA_ADAPTERS=QDIR + "/lora_qwen35_v7_round8.npz",
+ LORA_ADAPTERS=QDIR + "/lora_qwen35.npz",
LORA_R="16", LORA_ALPHA="32",
- PREGATE_NPZ=QDIR + "/pregate_qwen35_v7_round8.npz",
+ PREGATE_NPZ=QDIR + "/pregate_qwen35.npz",
QWEN_PG_START="7", QWEN_PG_HIDDEN="512",
QWEN_STAGED="1", QWEN_STAGED_TOP_K="4", QWEN_STAGED_SLOTS="4",
QWEN_STAGED_REPLACE="0", QWEN_PREGATE_V7="0",
diff --git a/src/edge0/backends/mlx/_impl/bailing_hybrid.py b/src/edge0/backends/mlx/_impl/bailing_hybrid.py
index 2f83282..798f5f2 100644
--- a/src/edge0/backends/mlx/_impl/bailing_hybrid.py
+++ b/src/edge0/backends/mlx/_impl/bailing_hybrid.py
@@ -798,7 +798,7 @@ class BailingModel(nn.Module):
# fp16 overflow inside one layer cannot poison the whole network
# (observed: decode blowup -> all-NaN logits -> argmax falls back
# to token 0 ('!') death spiral). Off by default.
- # (v7-deploy NAN_BANG_COLLAPSE_FIX.md parity.)
+ # (production-verified fix.)
_clip_v = float(os.environ.get("LING_HIDDEN_CLIP", "0"))
for li, (layer, c) in enumerate(zip(self.layers, cache)):
if before_layer_cb is not None:
diff --git a/src/edge0/config.py b/src/edge0/config.py
index f7cd6e3..a48ac0b 100644
--- a/src/edge0/config.py
+++ b/src/edge0/config.py
@@ -19,9 +19,9 @@ class GenerationConfig:
eos_ids: token ids that end generation.
seed: optional RNG seed.
first_token_greedy: always argmax the FIRST generated token
- (qwen35-v7-deploy serve.py / THINK_FIX_20260907.md pattern):
- after the think opener a randomly-sampled first token can
- derail the whole block into '!' loops.
+ (production-verified pattern): after the think opener a
+ randomly-sampled first token can derail the whole block into
+ '!' loops.
"""
temperature: float = 0.7
diff --git a/src/edge0/engine/base.py b/src/edge0/engine/base.py
index d13a22e..c54ef16 100644
--- a/src/edge0/engine/base.py
+++ b/src/edge0/engine/base.py
@@ -119,8 +119,8 @@ class Edge0Engine:
if logits is None:
raise RuntimeError("no logits available before first step")
if first_token and gen_config.first_token_greedy:
- # The FIRST token is always greedy (qwen35-v7-deploy
- # serve.py pattern, THINK_FIX_20260907.md §3.2): after the
+ # The FIRST token is always greedy
+ # (production-verified pattern): after the
# think opener a randomly-sampled first token can derail
# the whole block into '!' loops.
tid = int(core.argmax(logits, axis=-1).item())
diff --git a/src/edge0/engine/ling.py b/src/edge0/engine/ling.py
index cb9c41c..949e5c0 100644
--- a/src/edge0/engine/ling.py
+++ b/src/edge0/engine/ling.py
@@ -140,7 +140,7 @@ class Ling10BEngine(Edge0Engine):
# ---- startup warm-up --------------------------------------------------
def _prewarm(self):
- """Page-cache + kernel warm-up (ling v7 ``_prewarm`` parity).
+ """Page-cache + kernel warm-up.
1. madvise(WILLNEED) + a full sequential read over every shard —
removes per-expert page-fault cost from the first request.
diff --git a/src/edge0/models/base.py b/src/edge0/models/base.py
index e0ca224..09a14b9 100644
--- a/src/edge0/models/base.py
+++ b/src/edge0/models/base.py
@@ -27,8 +27,8 @@ ARTIFACTS_DIR = Path(__file__).resolve().parents[3] / "artifacts"
def artifact(name: str, model_dir: str | None = None) -> str:
"""Path of one adapter file: the model's own directory first
- (v7-deploy layout — model and its adapters side by side in ONE
- dir), falling back to ``artifacts/`` (legacy conversion cache)."""
+ (model and its adapters side by side in one
+ directory), falling back to ``artifacts/`` (conversion cache)."""
if model_dir:
cand = Path(model_dir) / name
if cand.is_file():
diff --git a/src/edge0/models/edge0_10b/__init__.py b/src/edge0/models/edge0_10b/__init__.py
index 1d41db4..5102fc1 100644
--- a/src/edge0/models/edge0_10b/__init__.py
+++ b/src/edge0/models/edge0_10b/__init__.py
@@ -1,6 +1,6 @@
"""edge0-10b adapter: Ling 3.0 hybrid (MLA + MoE, 10B tier).
-Family facts (production profile, verified against the v7 checkpoint):
+Family facts (production profile, verified against the reference checkpoint):
* 24 layers (layer 0 dense), 128 routed experts (4-bit affine,
group 64), K=8 native, sigmoid + group-limited routing
@@ -52,11 +52,11 @@ class Ling10BConfig(ModelConfig):
feature_topk="executed",
owners=tuple(range(7, 23)),
weights_file=artifact(
- "prerouter_edge0_10b_round6.safetensors", model_dir),
+ "prerouter_edge0_10b.safetensors", model_dir),
patch_call=False,
),
prerouter_top_k=8,
- lora=artifact("lora_edge0_10b_round6.safetensors", model_dir),
+ lora=artifact("lora_edge0_10b.safetensors", model_dir),
lora_r=16, lora_alpha=32.0,
gen=GenerationConfig(
temperature=0.7, top_p=0.95, top_k=64,
diff --git a/src/edge0/models/edge0_35b/__init__.py b/src/edge0/models/edge0_35b/__init__.py
index 130a839..ca5305d 100644
--- a/src/edge0/models/edge0_35b/__init__.py
+++ b/src/edge0/models/edge0_35b/__init__.py
@@ -1,6 +1,6 @@
"""edge0-35b adapter: Qwen3.5-MoE (K=4 tier, 35B-class).
-Family facts (production profile, verified against the round-9
+Family facts (production profile, verified against the current
checkpoint):
* 40 layers, 256 routed experts (4-bit affine, group 64), K=4,
diff --git a/src/edge0/prerouter/heads.py b/src/edge0/prerouter/heads.py
index a08867c..1368a98 100644
--- a/src/edge0/prerouter/heads.py
+++ b/src/edge0/prerouter/heads.py
@@ -1,5 +1,5 @@
"""Prerouter head math — faithful port of the qwen trained-head
-implementation (the legacy ``v7_trained`` file) with identifiers
+implementation with identifiers
renamed.
Semantics (cross-token):
@@ -8,7 +8,7 @@ Semantics (cross-token):
* head = fc1(features) -> exact(erf) gelu -> fc2 + linear_init on the
SAME concat features;
* fp16 weights by default (the training export precision, and the
- precision ling v7 production ran the heads at); fp32 for debugging.
+ production precision); fp32 for debugging.
"""
from __future__ import annotations
diff --git a/src/edge0/prerouter/spec.py b/src/edge0/prerouter/spec.py
index 4789109..636ffe9 100644
--- a/src/edge0/prerouter/spec.py
+++ b/src/edge0/prerouter/spec.py
@@ -38,7 +38,7 @@ class PrerouterSpec:
"executed": the top-k the block actually routed (qwen
trained semantics — executed == prerouter-selected at decode);
"teacher": the original gate's top-k, recomputed per layer
- (ling v7 training semantics).
+ (training semantics).
owners: explicit owner list override (defaults to
``range(start_layer - 1, n_layers - 1)``).
weights_file: safetensors path with per-owner head weights
diff --git a/src/edge0/server/app.py b/src/edge0/server/app.py
index 0de46a9..09a16a4 100644
--- a/src/edge0/server/app.py
+++ b/src/edge0/server/app.py
@@ -33,7 +33,7 @@ def _error(status: int, msg: str) -> tuple:
def _split_think(text: str, think: bool):
- """ling-mlx-server server.py parity: with thinking enabled the model
+ """With thinking enabled the model
streams the reasoning block first and closes it with ``</think>``
(token 156904); everything before is ``reasoning_content``, after
is ``content``. A generation cut off inside the reasoning block
diff --git a/src/edge0/server/chat.py b/src/edge0/server/chat.py
index dc9e056..b2f5c31 100644
--- a/src/edge0/server/chat.py
+++ b/src/edge0/server/chat.py
@@ -70,9 +70,9 @@ class ChatSession:
def prompt_ids(self) -> list[int]:
tok = self._tok
- # ling-mlx-server parity (THINK_FIX_20260907.md): families with a
- # vendored chat template (ling ``engine.encode_chat`` renders the
- # deployment chat_template.jinja with enable_thinking) MUST go
+ # Families with a vendored chat template
+ # (``engine.encode_chat`` renders a chat_template.jinja with
+ # enable_thinking) must go
# through it — the tokenizer's own apply_chat_template renders a
# DIFFERENT prompt (no think-mode system line, no ``<think>``
# opener), which is exactly the wrong-template bug.
@@ -134,7 +134,7 @@ class ChatSession:
def run(self, on_token=None) -> tuple[list[int], dict]:
t0 = time.perf_counter()
- # ling-mlx-server parity (THINK_FIX_20260907.md §3.1): clean
+ # Per-request clean
# per-request state. A previous degenerate/truncated turn leaves
# bad pregate cross-token state and KV behind, which makes every
# later request collapse from its first token.
diff --git a/src/edge0/streaming/mmap.py b/src/edge0/streaming/mmap.py
index 86ce369..d8d5bdb 100644
--- a/src/edge0/streaming/mmap.py
+++ b/src/edge0/streaming/mmap.py
@@ -54,7 +54,7 @@ class SafetensorsMmap:
macOS madvise only warms roughly half the file, so a full read is
the reliable way to remove per-expert page-fault cost from the
- first request (ling v7 ``_prewarm`` step 2 parity)."""
+ first request."""
total = len(self._mm)
for off in range(0, total, chunk):
_ = self._mm[off:off + chunk]
diff --git a/src/edge0/streaming/options.py b/src/edge0/streaming/options.py
index 59deba8..cf8f83b 100644
--- a/src/edge0/streaming/options.py
+++ b/src/edge0/streaming/options.py
@@ -112,7 +112,7 @@ class LayerOptions:
def prod_k8(cls, **overrides) -> "LayerOptions":
"""edge0-10b tier: deployment production profile.
- Mirrors start_server.sh on the ling v7 deploy: STAGED_DECODE=0
+ Mirrors the reference deployment profile: STAGED_DECODE=0
(staged decode is OFF — the deployment verified staged decode on
ling degrades output, see the comment in start_server.sh),
STAGED_SYNC=0, STAGED_SLOTS=8, EXPERT_CACHE_SLOTS=64,