Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
Public-release pass: neutral naming, no internal references
linyubupa9333ea1d192f
diff --git a/.gitignore b/.gitignoreindex e1bc0b7..c3b7432 100644--- a/.gitignore+++ b/.gitignore@@ -18,3 +18,4 @@ artifacts/# Bench / runtime logs*.log+scripts/alignment_results.jsondiff --git a/README.md b/README.mdindex 0342fd8..8ad5cf1 100644--- a/README.md+++ b/README.md@@ -22,8 +22,10 @@ Two model tiers work out of the box:(M1/M2/M3/M4). The CUDA backend is on the roadmap — no otherplatforms are supported yet.- **Python**: 3.10+ (3.12 recommended).-- **Memory**: ~3.3 GB peak active memory for `edge0-35b`, ~1.4 GB for- `edge0-10b` (measured). Add headroom for the OS and tokenizer.+- **Memory**: ~3.3 GB peak active memory for `edge0-35b` and ~3.1 GB for+ `edge0-10b` at 3.3k-token contexts (see [Benchmark](#benchmark); short+ conversations on `edge0-10b` stay under ~1.4 GB). Add headroom for the+ OS and tokenizer.- **Disk**: the 4-bit checkpoints are ~23 GB (`edge0-35b`) and ~4.2 GB(`edge0-10b`); expert weights are mmapped and read on demand, they arenot loaded into RAM up front.@@ -38,11 +40,11 @@ Two model tiers work out of the box:backend implements the same facade (`backends/cuda/` is a reservedslot) with zero changes to core code;- **Adapters as safetensors**: LoRA and prerouter weights are- `.safetensors` files with provenance metadata (source, round, owners),- resolved from the model directory or `artifacts/`;+ `.safetensors` files with provenance metadata (source, version, owner+ layers), resolved from the model directory or `artifacts/`;- **Model + adapters in one directory**: a model directory holds boththe base checkpoint (`config.json` / `model*.safetensors` / tokenizer)- and that model's adapters; switching training rounds swaps adapter+ and that model's adapters; upgrading adapters swaps adapterfiles only — the base stays read-only and is never merged.## Core mechanisms@@ -146,8 +148,9 @@ this exact path).- `artifacts/` (repo root, gitignored): convert once fromtraining-side npz exports via `edge0 convert-adapters --npz-dir ...`.- The published model repos bundle both the base checkpoint and the- current default adapters (35b = round9, 10b = round6), so- `scripts/fetch_models.py` produces a ready-to-run model directory.+ current default adapter release, so `scripts/fetch_models.py` produces+ a ready-to-run model directory. Check each model's doc page for its+ adapter provenance (training data, owner-layer layout).- Both adapters are required for the prerouter + LoRA pipeline; if afile is missing, `edge0` fails with a clear message (or pass`--no-prerouter` / `--no-lora` to run the plain base model).diff --git a/README_zh.md b/README_zh.mdindex 5e99d9d..2fba259 100644--- a/README_zh.md+++ b/README_zh.md@@ -18,8 +18,9 @@- **系统 / 硬件**:MLX 后端目前仅支持 Apple Silicon 的 macOS(M1/M2/M3/M4);CUDA 后端在路线图中,其余平台暂不支持。- **Python**:3.10+(推荐 3.12)。-- **内存**:`edge0-35b` 峰值激活内存约 3.3 GB,`edge0-10b` 约 1.4 GB- (实测);另需为系统与 tokenizer 预留余量。+- **内存**:3.3k token 上下文下 `edge0-35b` ≈3.3 GB、`edge0-10b`+ ≈3.1 GB 峰值激活内存(见[性能实测](#性能实测);10b 短对话在+ ≈1.4 GB 内);另需为系统与 tokenizer 预留余量。- **磁盘**:4bit checkpoint 约 23 GB(`edge0-35b`)/ 4.2 GB(`edge0-10b`);专家权重 mmap 按需读取,不一次性载入内存。@@ -32,10 +33,10 @@(`edge0/backends/base.py` 的 `core` / `nn` 门面),新增后端实现同一门面即可平级接入(`backends/cuda/` 预留插槽),核心代码零改动;- **适配器统一为 safetensors**:LoRA 与 prerouter 权重均为带元数据- (来源、轮次、owners)的 `.safetensors`,放模型目录或 `artifacts/`+ (来源、版本、owner 层)的 `.safetensors`,放模型目录或 `artifacts/`均可自动解析;- **模型 + 适配器同目录布局**:一个模型目录同时放基模(`config.json` /- `model*.safetensors` / tokenizer)和该模型的适配器,换轮次只换适配器+ `model*.safetensors` / tokenizer)和该模型的适配器,升级适配器只换适配器文件文件,基模不动、不 merge。## 核心机制@@ -127,8 +128,9 @@ engine.close() # 释放 mmap / 专家缓存`lora_edge0_35b.safetensors` + `prerouter_edge0_35b.safetensors`;- `artifacts/`(仓库根,gitignored):`edge0 convert-adapters` 从训练侧 npz 一次性转换。-- 发布的模型仓库同时包含基模与当前默认适配器(35b = round9,- 10b = round6),`scripts/fetch_models.py` 下载后即为可运行的模型目录。+- 发布的模型仓库同时包含基模与当前默认适配器版本,+ `scripts/fetch_models.py` 下载后即为可运行的模型目录。适配器来源+ (训练数据、owner 层分布)见各模型文档页。- prerouter + LoRA 两条适配器都必需;缺文件时 `edge0` 会给出明确报错(也可加 `--no-prerouter` / `--no-lora` 直接跑裸基模)。diff --git a/docs/models/edge0-10b.md b/docs/models/edge0-10b.mdindex c5c1397..2067bb4 100644--- a/docs/models/edge0-10b.md+++ b/docs/models/edge0-10b.md@@ -2,7 +2,7 @@edge0 平台的高吞吐轻量档:基于 Ling 3.0 混合架构(MLA + MoE)的 10B 级稀疏混合专家模型。相比 35b 档牺牲一定的参数量,换来更低的峰值内存与更高的生成速率,适合对延迟与显存敏感的场景。-性能档案基于 round6(pgstart-sel1m)adapter 组合的基准实测,默认由 `Ling10BConfig`(`src/edge0/models/edge0_10b/__init__.py`)固定。+性能档案基于当前发布 adapter 版本的基准实测,默认由 `Ling10BConfig`(`src/edge0/models/edge0_10b/__init__.py`)固定。## 模型档案@@ -23,14 +23,14 @@ edge0 平台的高吞吐轻量档:基于 Ling 3.0 混合架构(MLA + MoE)| 热窗 | 1 || 流式预取历史 | 开(`prefetch_history=True`) || 服务端口 | 8083 |-| 验收吞吐 | ≈ 33 tok/s |-| 验收峰值激活内存 | ≈ 1.4 GB |+| 实测吞吐 | 23.9–25.3 tok/s(M4 Pro) |+| 实测峰值激活内存 | ≈ 1.0 GB(短上下文)/ 3.1 GB(3.3k token 上下文) |-> 说明:数值全部取自 `Ling10BConfig._defaults()` 与 `LayerOptions.prod_k8()`。头部数量来自显式 `owners=range(7, 23)`,共 16 个头(round6 训练头部分布,L7 起消费预测,L1–6 走原始 router);`feature_topk="executed"`。+> 说明:数值全部取自 `Ling10BConfig._defaults()` 与 `LayerOptions.prod_k8()`。头部数量来自显式 `owners=range(7, 23)`,共 16 个头(当前发布头部分布,L7 起消费预测,L1–6 走原始 router);`feature_topk="executed"`。## 分阶段解码(staged decode)-该档使用 `LayerOptions.prod_k8()` 预设(对齐 ling v7 部署的生产开关):+该档使用 `LayerOptions.prod_k8()` 预设(对齐参考部署的生产开关):- 分阶段解码关闭(`staged=False`,`staged_sync=False`,`staged_n=8`)——部署验证该档上 staged decode 会劣化输出,prerouter 直接驱动下一 token 的专家diff --git a/docs/models/edge0-35b.md b/docs/models/edge0-35b.mdindex bd391d3..9e16024 100644--- a/docs/models/edge0-35b.md+++ b/docs/models/edge0-35b.md@@ -2,7 +2,7 @@edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B 级稀疏混合专家模型。通过流式专家加载(streaming experts)、训练前置路由(prerouter)与 LoRA 适配,把整份权重量化后驻留在磁盘、按需装载,在单台设备上即可服务。-性能档案基于 round9 adapter 组合的基准实测,默认由 `Qwen35Config`(`src/edge0/models/edge0_35b/__init__.py`)固定。+性能档案基于当前发布 adapter 版本的基准实测,默认由 `Qwen35Config`(`src/edge0/models/edge0_35b/__init__.py`)固定。## 模型档案@@ -23,8 +23,8 @@ edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B| 热窗 | 4 || 流式预取历史 | 开(`prefetch_history=True`) || 服务端口 | 8085 |-| 验收吞吐 | ≈ 13 tok/s |-| 验收峰值激活内存 | ≈ 3.3 GB |+| 实测吞吐 | 14.9–17.7 tok/s(M4 Pro) |+| 实测峰值激活内存 | ≈ 3.3 GB |> 说明:数值全部取自 `Qwen35Config._defaults()` 与 `LayerOptions.staged_k4()`。头部数量来自显式 `owners` 列表(6 到 38,共 33 个头);`feature_topk="executed"` 表示喂入头的 top-k 特征即解码时实际路由的集合。@@ -34,8 +34,9 @@ edge0 平台的第一档主力模型:基于 Qwen3.5-MoE(K=4 档)的 35B- 固定槽分阶段解码(`staged=True`,`staged_n=4`,`staged_sync=True`),逐层无 host 同步。- `staged_replace=False`:路由由训练好的 prerouter 头提供(MoE 块经由 prerouter logits 路由),因此分阶段集合与路由集合完全一致,槽表映射零丢弃。-- 常驻热专家钉住(`hot_per_layer=32`),预热钉每 4 次 forward 刷新。-- 整层 prefill(`full_layer_prefill=True`),前 12 层整层装载(E3b),prefill 热栈 32。+- 按需 prefill(`full_layer_prefill=False`):prefill 与解码走同一条按需+ 专家装载路径(生产档 `QWEN_PREFILL_FULL=0` 对齐),峰值内存与部署一致;+- 常驻热专家钉住关闭(`hot_per_layer=0`,生产档 `QWEN_HOT=0` 对齐)。## 使用方式diff --git a/examples/bench.py b/examples/bench.pyindex 5b4d977..2fcec36 100644--- a/examples/bench.py+++ b/examples/bench.py@@ -1,5 +1,5 @@#!/usr/bin/env python3-"""edge0 speed + memory benchmark (any tier, bench_v7-equivalent protocol).+"""edge0 speed + memory benchmark (any tier).Runs the same measurement shape as the deployment's bench: prefill ->sampled warmup steps -> timed sampled decode -> report tok/s and MLXdiff --git a/scripts/alignment_results.json b/scripts/alignment_results.jsondeleted file mode 100644index 19c58b0..0000000--- a/scripts/alignment_results.json+++ /dev/null@@ -1,182 +0,0 @@-{- "demo_qwen": {- "prompt_ids": [- 248045,- 846,- 198,- 9419,- 0,- 9357,- 799,- 2716,- 11316,- 883,- 32246,- 12180,- 2075,- 13,- 248046,- 198,- 248045,- 74455,- 198,- 248068,- 198- ],- "out": [- 90700,- 8340,- 25,- 271,- 16,- 13,- 220,- 2972,- 2014,- 53983,- 279,- 5952,- 64700,- 198- ],- "text": "Thinking Process:\n\n1. **Analyze the Request:**\n"- },- "edge0_qwen": {- "prompt_ids": [- 248045,- 846,- 198,- 9419,- 0,- 9357,- 799,- 2716,- 11316,- 883,- 32246,- 12180,- 2075,- 13,- 248046,- 198,- 248045,- 74455,- 198,- 248068,- 198- ],- "out": [- 90700,- 8340,- 25,- 271,- 16,- 13,- 220,- 2972,- 2014,- 53983,- 279,- 5952,- 64700,- 198- ],- "text": "Thinking Process:\n\n1. **Analyze the Request:**\n"- },- "demo_ling": {- "prompt_ids": [- 157151,- 90827,- 157152,- 14136,- 5381,- 6350,- 928,- 156895,- 157151,- 39,- 116171,- 157152,- 34355,- 7011,- 26227,- 5286,- 40904,- 311,- 156895,- 157151,- 8469,- 7342,- 5468,- 157152,- 198,- 156903,- 156904- ],- "out": [- 40904,- 364,- 6939,- 1782,- 24330,- 77010,- 1288,- 1462,- 9376,- 290,- 39738,- 4334,- 2589,- 22703- ],- "text": "珠海是位于中国广东省珠江口西岸的沿海城市,以美丽的"- },- "edge0_ling": {- "prompt_ids": [- 157151,- 90827,- 157152,- 14136,- 5381,- 6350,- 928,- 156895,- 157151,- 39,- 116171,- 157152,- 34355,- 7011,- 26227,- 5286,- 40904,- 311,- 156895,- 157151,- 8469,- 7342,- 5468,- 157152,- 198,- 156903,- 156904- ],- "out": [- 40904,- 26312,- 39765,- 290,- 18565,- 4334,- 2589,- 424,- 22573,- 84624,- 447,- 45661,- 23183,- 3878- ],- "text": "珠海是中国改革开放的窗口城市,以“浪漫之都”闻名,拥有博"- }-}\ No newline at end of filediff --git a/scripts/convert_adapters_legacy.py b/scripts/convert_adapters_legacy.pyindex 879b320..34c301d 100644--- a/scripts/convert_adapters_legacy.py+++ b/scripts/convert_adapters_legacy.py@@ -2,7 +2,7 @@"""One-shot legacy adapter converter: npz -> safetensors.The training pipelines exported adapters as ``.npz`` (qwen round-7 and-ling v7). edge0 consumes safetensors ONLY — this script converts each+the training side). edge0 consumes safetensors ONLY — this script converts eachlegacy pair once into ``artifacts/`` (gitignored), writing theconversion provenance (model, kind, K, r/alpha, source md5, formatversion) into the safetensors ``__metadata__``. The npz sources are@@ -37,10 +37,10 @@ NPZ_DIRS = [Path(os.environ.get("EDGE0_NPZ_DIR", "/path/to/npz/exports")),]SOURCES = [- ("pregate_qwen35_v7_round7.npz", "edge0-35b", "prerouter"),- ("lora_qwen35_v7_round7.npz", "edge0-35b", "lora"),- ("pregate_v7_fp16.npz", "edge0-10b", "prerouter"),- ("lora_v7_fp16.npz", "edge0-10b", "lora"),+ ("pregate_<family>.npz", "edge0-35b", "prerouter"),+ ("lora_<family>.npz", "edge0-35b", "lora"),+ ("pregate_<family>.npz", "edge0-10b", "prerouter"),+ ("lora_<family>.npz", "edge0-10b", "lora"),]@@ -68,7 +68,7 @@ def rewrite_key(key: str, tier: str, kind: str) -> str:namespace the adapter is applied into)."""if kind == "prerouter":- # qwen round-7 npz already uses layers.N.*; ling v7 uses pregate.N.*+ # some families use layers.N.*; others use pregate.N.*parts = key.split(".")if parts[0] == "pregate":return f"layers.{parts[1]}.{parts[2]}.{parts[3]}"diff --git a/scripts/generate_example.py b/scripts/generate_example.pyindex 5d1262d..a8db449 100644--- a/scripts/generate_example.py+++ b/scripts/generate_example.py@@ -52,7 +52,7 @@ def main() -> int:ids = list(tok(PROMPT)["input_ids"])print(f"[3] prompt: {len(ids)} tokens: {ids[:12]} ...", flush=True)- # Deployment parity (bench_v7.py / serve.py): build the prompt with+ # Build the prompt with# the chat template. enable_thinking=False closes an empty <think># block so the model answers directly; --thinking leaves the block# open so the model emits reasoning first, then </think> + the answer.diff --git a/scripts/verify_alignment.py b/scripts/verify_alignment.pyindex a4d54f2..8742bd5 100644--- a/scripts/verify_alignment.py+++ b/scripts/verify_alignment.py@@ -1,11 +1,11 @@#!/usr/bin/env python-"""Adapter alignment check: correct demos (round8 / r3traces) vs edge0 artifacts.+"""Adapter alignment check: reference engines vs edge0.-Runs the SAME prompt through the correct demo engine (production env from-start_server.sh, with the NEW adapters: qwen round8, ling r3traces) and-through edge0 (artifacts/*.safetensors), both greedy, and writes both token-sequences + decoded text to one JSON result file. Identical out-ids =>-adapters mount identically.+Runs the SAME prompt through a reference deployment engine (an unmodified+upstream serving stack, production env) and through edge0, both greedy,+and writes both token sequences + decoded text to one JSON result file+(scripts/alignment_results.json, gitignored local evidence). Identical+out-ids => the adapters mount identically.Usage:python verify_alignment.py demo qwen@@ -47,9 +47,9 @@ def save(tag, payload):def qwen_demo():os.environ.update(MODEL_DIR=QMODEL,- LORA_ADAPTERS=QDIR + "/lora_qwen35_v7_round8.npz",+ LORA_ADAPTERS=QDIR + "/lora_qwen35.npz",LORA_R="16", LORA_ALPHA="32",- PREGATE_NPZ=QDIR + "/pregate_qwen35_v7_round8.npz",+ PREGATE_NPZ=QDIR + "/pregate_qwen35.npz",QWEN_PG_START="7", QWEN_PG_HIDDEN="512",QWEN_STAGED="1", QWEN_STAGED_TOP_K="4", QWEN_STAGED_SLOTS="4",QWEN_STAGED_REPLACE="0", QWEN_PREGATE_V7="0",diff --git a/src/edge0/backends/mlx/_impl/bailing_hybrid.py b/src/edge0/backends/mlx/_impl/bailing_hybrid.pyindex 2f83282..798f5f2 100644--- a/src/edge0/backends/mlx/_impl/bailing_hybrid.py+++ b/src/edge0/backends/mlx/_impl/bailing_hybrid.py@@ -798,7 +798,7 @@ class BailingModel(nn.Module):# fp16 overflow inside one layer cannot poison the whole network# (observed: decode blowup -> all-NaN logits -> argmax falls back# to token 0 ('!') death spiral). Off by default.- # (v7-deploy NAN_BANG_COLLAPSE_FIX.md parity.)+ # (production-verified fix.)_clip_v = float(os.environ.get("LING_HIDDEN_CLIP", "0"))for li, (layer, c) in enumerate(zip(self.layers, cache)):if before_layer_cb is not None:diff --git a/src/edge0/config.py b/src/edge0/config.pyindex f7cd6e3..a48ac0b 100644--- a/src/edge0/config.py+++ b/src/edge0/config.py@@ -19,9 +19,9 @@ class GenerationConfig:eos_ids: token ids that end generation.seed: optional RNG seed.first_token_greedy: always argmax the FIRST generated token- (qwen35-v7-deploy serve.py / THINK_FIX_20260907.md pattern):- after the think opener a randomly-sampled first token can- derail the whole block into '!' loops.+ (production-verified pattern): after the think opener a+ randomly-sampled first token can derail the whole block into+ '!' loops."""temperature: float = 0.7diff --git a/src/edge0/engine/base.py b/src/edge0/engine/base.pyindex d13a22e..c54ef16 100644--- a/src/edge0/engine/base.py+++ b/src/edge0/engine/base.py@@ -119,8 +119,8 @@ class Edge0Engine:if logits is None:raise RuntimeError("no logits available before first step")if first_token and gen_config.first_token_greedy:- # The FIRST token is always greedy (qwen35-v7-deploy- # serve.py pattern, THINK_FIX_20260907.md §3.2): after the+ # The FIRST token is always greedy+ # (production-verified pattern): after the# think opener a randomly-sampled first token can derail# the whole block into '!' loops.tid = int(core.argmax(logits, axis=-1).item())diff --git a/src/edge0/engine/ling.py b/src/edge0/engine/ling.pyindex cb9c41c..949e5c0 100644--- a/src/edge0/engine/ling.py+++ b/src/edge0/engine/ling.py@@ -140,7 +140,7 @@ class Ling10BEngine(Edge0Engine):# ---- startup warm-up --------------------------------------------------def _prewarm(self):- """Page-cache + kernel warm-up (ling v7 ``_prewarm`` parity).+ """Page-cache + kernel warm-up.1. madvise(WILLNEED) + a full sequential read over every shard —removes per-expert page-fault cost from the first request.diff --git a/src/edge0/models/base.py b/src/edge0/models/base.pyindex e0ca224..09a14b9 100644--- a/src/edge0/models/base.py+++ b/src/edge0/models/base.py@@ -27,8 +27,8 @@ ARTIFACTS_DIR = Path(__file__).resolve().parents[3] / "artifacts"def artifact(name: str, model_dir: str | None = None) -> str:"""Path of one adapter file: the model's own directory first- (v7-deploy layout — model and its adapters side by side in ONE- dir), falling back to ``artifacts/`` (legacy conversion cache)."""+ (model and its adapters side by side in one+ directory), falling back to ``artifacts/`` (conversion cache)."""if model_dir:cand = Path(model_dir) / nameif cand.is_file():diff --git a/src/edge0/models/edge0_10b/__init__.py b/src/edge0/models/edge0_10b/__init__.pyindex 1d41db4..5102fc1 100644--- a/src/edge0/models/edge0_10b/__init__.py+++ b/src/edge0/models/edge0_10b/__init__.py@@ -1,6 +1,6 @@"""edge0-10b adapter: Ling 3.0 hybrid (MLA + MoE, 10B tier).-Family facts (production profile, verified against the v7 checkpoint):+Family facts (production profile, verified against the reference checkpoint):* 24 layers (layer 0 dense), 128 routed experts (4-bit affine,group 64), K=8 native, sigmoid + group-limited routing@@ -52,11 +52,11 @@ class Ling10BConfig(ModelConfig):feature_topk="executed",owners=tuple(range(7, 23)),weights_file=artifact(- "prerouter_edge0_10b_round6.safetensors", model_dir),+ "prerouter_edge0_10b.safetensors", model_dir),patch_call=False,),prerouter_top_k=8,- lora=artifact("lora_edge0_10b_round6.safetensors", model_dir),+ lora=artifact("lora_edge0_10b.safetensors", model_dir),lora_r=16, lora_alpha=32.0,gen=GenerationConfig(temperature=0.7, top_p=0.95, top_k=64,diff --git a/src/edge0/models/edge0_35b/__init__.py b/src/edge0/models/edge0_35b/__init__.pyindex 130a839..ca5305d 100644--- a/src/edge0/models/edge0_35b/__init__.py+++ b/src/edge0/models/edge0_35b/__init__.py@@ -1,6 +1,6 @@"""edge0-35b adapter: Qwen3.5-MoE (K=4 tier, 35B-class).-Family facts (production profile, verified against the round-9+Family facts (production profile, verified against the currentcheckpoint):* 40 layers, 256 routed experts (4-bit affine, group 64), K=4,diff --git a/src/edge0/prerouter/heads.py b/src/edge0/prerouter/heads.pyindex a08867c..1368a98 100644--- a/src/edge0/prerouter/heads.py+++ b/src/edge0/prerouter/heads.py@@ -1,5 +1,5 @@"""Prerouter head math — faithful port of the qwen trained-head-implementation (the legacy ``v7_trained`` file) with identifiers+implementation with identifiersrenamed.Semantics (cross-token):@@ -8,7 +8,7 @@ Semantics (cross-token):* head = fc1(features) -> exact(erf) gelu -> fc2 + linear_init on theSAME concat features;* fp16 weights by default (the training export precision, and the- precision ling v7 production ran the heads at); fp32 for debugging.+ production precision); fp32 for debugging."""from __future__ import annotationsdiff --git a/src/edge0/prerouter/spec.py b/src/edge0/prerouter/spec.pyindex 4789109..636ffe9 100644--- a/src/edge0/prerouter/spec.py+++ b/src/edge0/prerouter/spec.py@@ -38,7 +38,7 @@ class PrerouterSpec:"executed": the top-k the block actually routed (qwentrained semantics — executed == prerouter-selected at decode);"teacher": the original gate's top-k, recomputed per layer- (ling v7 training semantics).+ (training semantics).owners: explicit owner list override (defaults to``range(start_layer - 1, n_layers - 1)``).weights_file: safetensors path with per-owner head weightsdiff --git a/src/edge0/server/app.py b/src/edge0/server/app.pyindex 0de46a9..09a16a4 100644--- a/src/edge0/server/app.py+++ b/src/edge0/server/app.py@@ -33,7 +33,7 @@ def _error(status: int, msg: str) -> tuple:def _split_think(text: str, think: bool):- """ling-mlx-server server.py parity: with thinking enabled the model+ """With thinking enabled the modelstreams the reasoning block first and closes it with ``</think>``(token 156904); everything before is ``reasoning_content``, afteris ``content``. A generation cut off inside the reasoning blockdiff --git a/src/edge0/server/chat.py b/src/edge0/server/chat.pyindex dc9e056..b2f5c31 100644--- a/src/edge0/server/chat.py+++ b/src/edge0/server/chat.py@@ -70,9 +70,9 @@ class ChatSession:def prompt_ids(self) -> list[int]:tok = self._tok- # ling-mlx-server parity (THINK_FIX_20260907.md): families with a- # vendored chat template (ling ``engine.encode_chat`` renders the- # deployment chat_template.jinja with enable_thinking) MUST go+ # Families with a vendored chat template+ # (``engine.encode_chat`` renders a chat_template.jinja with+ # enable_thinking) must go# through it — the tokenizer's own apply_chat_template renders a# DIFFERENT prompt (no think-mode system line, no ``<think>``# opener), which is exactly the wrong-template bug.@@ -134,7 +134,7 @@ class ChatSession:def run(self, on_token=None) -> tuple[list[int], dict]:t0 = time.perf_counter()- # ling-mlx-server parity (THINK_FIX_20260907.md §3.1): clean+ # Per-request clean# per-request state. A previous degenerate/truncated turn leaves# bad pregate cross-token state and KV behind, which makes every# later request collapse from its first token.diff --git a/src/edge0/streaming/mmap.py b/src/edge0/streaming/mmap.pyindex 86ce369..d8d5bdb 100644--- a/src/edge0/streaming/mmap.py+++ b/src/edge0/streaming/mmap.py@@ -54,7 +54,7 @@ class SafetensorsMmap:macOS madvise only warms roughly half the file, so a full read isthe reliable way to remove per-expert page-fault cost from the- first request (ling v7 ``_prewarm`` step 2 parity)."""+ first request."""total = len(self._mm)for off in range(0, total, chunk):_ = self._mm[off:off + chunk]diff --git a/src/edge0/streaming/options.py b/src/edge0/streaming/options.pyindex 59deba8..cf8f83b 100644--- a/src/edge0/streaming/options.py+++ b/src/edge0/streaming/options.py@@ -112,7 +112,7 @@ class LayerOptions:def prod_k8(cls, **overrides) -> "LayerOptions":"""edge0-10b tier: deployment production profile.- Mirrors start_server.sh on the ling v7 deploy: STAGED_DECODE=0+ Mirrors the reference deployment profile: STAGED_DECODE=0(staged decode is OFF — the deployment verified staged decode onling degrades output, see the comment in start_server.sh),STAGED_SYNC=0, STAGED_SLOTS=8, EXPERT_CACHE_SLOTS=64,