Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

README: add prefill timing (cold/warm) to the benchmark table

linyubupa918d61c932e7
diff --git a/README.md b/README.md
index 67a046b..2dc9a6d 100644
--- a/README.md
+++ b/README.md
@@ -168,10 +168,14 @@ multiple prompts):
Measured with `examples/bench.py` (prefill → 10 sampled warmup steps → 200
timed sampled decode tokens, 2 runs per tier):
-| Tier | Decode speed | Peak active memory | Test machine |
-|---|---|---|---|
-| `edge0-35b` | 14.9–17.7 tok/s | 2.9–3.1 GiB | Mac mini M4 Pro, 24 GB |
-| `edge0-10b` | 25.3 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+| Tier | Decode speed | Prefill (cold / warm)* | Peak active memory | Test machine |
+|---|---|---|---|---|
+| `edge0-35b` | 14.9–17.7 tok/s | 4.4 s / 0.8 s (27 tok) | 2.9–3.1 GiB | Mac mini M4 Pro, 24 GB |
+| `edge0-10b` | 25.3 tok/s | 0.15 s / 0.25 s (37 tok) | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+
+*Cold = first request after process start (expert weights fault in from
+SSD); warm = subsequent requests (page cache resident). Prompt length in
+parentheses.*
*Peak active memory is the MLX allocator's peak (model weights + KV cache +
expert working set), not RSS: expert weights stream from SSD via mmap and the
diff --git a/README_zh.md b/README_zh.md
index b4095d0..91e0d58 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -144,10 +144,13 @@ engine.close() # 释放 mmap / 专家缓存
`examples/bench.py` 实测(prefill → 10 步采样 warmup → 200 token 计时段,每档 2 轮):
-| 档位 | 解码速度 | 峰值 active 内存 | 测试机器 |
-|---|---|---|---|
-| `edge0-35b` | 14.9–17.7 tok/s | 2.9–3.1 GiB | Mac mini M4 Pro, 24 GB |
-| `edge0-10b` | 25.3 tok/s | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+| 档位 | 解码速度 | Prefill(冷/热)* | 峰值 active 内存 | 测试机器 |
+|---|---|---|---|---|
+| `edge0-35b` | 14.9–17.7 tok/s | 4.4 s / 0.8 s(27 tok) | 2.9–3.1 GiB | Mac mini M4 Pro, 24 GB |
+| `edge0-10b` | 25.3 tok/s | 0.15 s / 0.25 s(37 tok) | 1.0 GiB | Mac mini M4 Pro, 24 GB |
+
+*冷 = 进程启动后首请求(专家权重从 SSD 逐页换入);热 = 后续请求(页缓存常驻)。
+括号内为 prompt 长度。*
*峰值 active 内存为 MLX allocator 的峰值(权重 + KV cache + 专家工作集),
不含 RSS:专家权重经 mmap 从 SSD 流式读取,OS 页缓存不计入。*