Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

docs: drop the long-context KV-cache figure for edge0-8b

linyubupa48bd22456a57
diff --git a/README.md b/README.md
index 01c20ca..66bba35 100644
--- a/README.md
+++ b/README.md
@@ -233,8 +233,7 @@ SSD); warm = subsequent requests (page cache resident). Prefill numbers
are throughput over a ~3.3k-token prompt (`BENCH_LONG=1`).*
**Peak active memory at short contexts (MLX allocator peak; expert weights
-stream from SSD via mmap and are not resident). Long contexts add KV
-cache: ~3.3 GiB on `edge0-8b` at 3.3k tokens.*
+stream from SSD via mmap and are not resident).*
Reproduce:
diff --git a/README_zh.md b/README_zh.md
index 46987e1..4413371 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -212,7 +212,7 @@ engine.close() # 释放 mmap / 专家缓存
*冷 = 进程启动后首请求(专家权重从 SSD 逐页换入);热 = 后续请求(页缓存常驻)。Prefill 为 ≈3.3k token 长 prompt 的吞吐(`BENCH_LONG=1`)。
-**短上下文下的峰值 active 内存(MLX allocator 峰值;专家权重经 mmap 流式读取、不常驻内存)。长上下文增加 KV cache:3.3k token 下 `edge0-8b` ≈3.3 GiB。**
+**短上下文下的峰值 active 内存(MLX allocator 峰值;专家权重经 mmap 流式读取、不常驻内存)。**
复现:
diff --git a/docs/models/edge0-8b.md b/docs/models/edge0-8b.md
index 470e2cc..5b83a06 100644
--- a/docs/models/edge0-8b.md
+++ b/docs/models/edge0-8b.md
@@ -24,7 +24,7 @@ The performance profile is based on benchmarks of the current release adapter ve
| Streaming prefetch history | on (`prefetch_history=True`) |
| Serving port | 8083 |
| Measured throughput | 23.9–25.3 tok/s (M4 Pro) |
-| Measured peak activation memory | ≈ 1.0 GiB (short context) / 3.3 GiB (3.3k-token context) |
+| Measured peak activation memory | ≈ 1.0 GiB (short context) |
> Note: all values are taken from `Ling8BConfig._defaults()` and `LayerOptions.prod_k8()`. The head count comes from the explicit `owners=range(7, 23)` — 16 heads in total (the current release head distribution: prediction is consumed from L7 onward, and L1–6 use the original router); `feature_topk="executed"`.