Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
docs(readme): fold the Q4 2026 model & algorithm roadmap into Roadmap
Bin27866bd11974
diff --git a/README.md b/README.mdindex 0f6ba4f..f1cb73c 100644--- a/README.md+++ b/README.md@@ -298,6 +298,8 @@ the **end of October 2026**; see the [Roadmap](#roadmap).## Roadmap+**Platforms & systems**+- **[End of Oct 2026] edge0 unified inference framework** — we willopen-source a unified inference framework: **one unified accesslayer** (a single API across chat / serve / on-device use), with the@@ -308,6 +310,28 @@ the **end of October 2026**; see the [Roadmap](#roadmap).- **Windows** platform support (via the unified framework).- **CUDA backend** for the Python framework — reserved slot at`python/src/edge0/backends/cuda/`, core code needs zero changes.++**Models & algorithms — Q4 2026**++Q4 works two fronts: bringing a next-generation architecture into the+framework, and turning latent reasoning into a real latency saving rather+than just an arithmetic one.++- **Next-gen architecture support (Qwen3.8-Flash class)** — run hybrid+ linear attention (GDN + QSA), gated multi-branch residual, and N-gram+ embedding topologies on edge0. These designs suit SSD streaming offload+ naturally: O(1)-state attention keeps long thinking from becoming a+ KV-cache problem, and lookup-only N-gram tables stream on demand. Goal:+ the tier runs on a single device and benchmarks within an acceptable+ gap of the fp16 base.+- **Latent thinking + batched expert pre-prediction** — make latent+ reasoning a *latency* saving, not only an arithmetic one. The core+ engineering problem: move expert routing from per-position to **once+ per block**, so one prediction covers every position and round of a+ block and **expert load volume decouples from the reasoning loop+ count** — plus cross-block prefetch that loads the next block's experts+ inside the current block's compute window. Progress is measured as+ **end-to-end thinking-phase time at matched accuracy** (never tokens/s).- More model tiers and adapter releases on the existing pipeline.## Contributingdiff --git a/README_zh.md b/README_zh.mdindex aeb9ce6..37f615f 100644--- a/README_zh.md+++ b/README_zh.md@@ -289,6 +289,8 @@ Windows / Python)将于 **2026 年 10 月底**发布,见[路线图](#路线## 路线图+**平台与系统**+- **【2026 年 10 月底】edge0 统一推理框架开源** —— 发布统一推理框架:**接入层统一**为一套 API(chat / serve / 端侧共用),**runtime 自动适配不同硬件平台**——iOS、macOS、Android、Windows、Python;在本仓库@@ -296,6 +298,23 @@ Windows / Python)将于 **2026 年 10 月底**发布,见[路线图](#路线- **Windows** 平台支持(经统一框架)。- **CUDA 后端**(Python 框架)—— 插槽已预留在`python/src/edge0/backends/cuda/`,核心代码零改动。++**模型与算法 —— 2026 Q4**++Q4 推进两条主线:把下一代架构引入框架,以及把潜在推理(latent+thinking)变成真正的延迟收益、而不只是算力节省。++- **下一代架构支持(Qwen3.8-Flash 级)** —— 在 edge0 上跑通混合线性+ 注意力(GDN + QSA)、门控多分支残差、N-gram embedding 拓扑。这些+ 设计天然契合 SSD 流式 offload:O(1) 状态的注意力层让长思考不再是+ KV cache 问题,纯查表的 N-gram 表按需流式加载。目标:该档位在单+ 设备跑通,且基准与 fp16 基座的差距在可接受范围内。+- **潜在思考 + 批量专家预预测** —— 让潜在推理成为*延迟*收益,而不+ 只是算力收益。核心工程问题:把专家路由从 per-position 改为**每 block+ 一次**,一次预测覆盖一个 block 内所有位置与所有轮次,使**专家加载量+ 与推理循环次数解耦**;配合跨 block 预取,在当前 block 的计算窗口内+ 加载下一个 block 的专家集。进展以**等准确率下的端到端思考阶段耗时**+ 衡量(而非 tokens/s)。- 更多模型档位与适配器版本在现有管线上持续发布。## 贡献