Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Python
README: self-evaluation scores vs fp16 base models
linyubupa61aa744d374a
diff --git a/README.md b/README.mdindex 44150b7..ec18cd1 100644--- a/README.md+++ b/README.md@@ -207,8 +207,24 @@ this exact path).- Both adapters are required for the prerouter + LoRA pipeline; if afile is missing, `edge0` fails with a clear message (or pass`--no-prerouter` / `--no-lora` to run the plain base model).--+## Quality (self-evaluation)++Internal self-evaluation, included only to quantify the loss of the edge0+pipeline (int4 + trained adapters + prerouter routing) relative to the+original fp16 base models — the loss is small: **3.9 points on average for+edge0-35b, 2.8 for edge0-10b** (MMLU-Pro is even above the base). All scores+self-run, max 100:++| Benchmark | edge0-35b (int4) | Base fp16 | edge0-10b (int4) | Base fp16 |+|---|---:|---:|---:|---:|+| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |+| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |+| GPQA-Diamond | 79.8 | 81.8 | 70.7 | 71.2 |+| MMLU-Pro | 81.0 | 84.6 | 70.1 | 65.8 |+| IFBench | 57.9 | 61.7 | 53.9 | 60.6 |+| **Average** | **79.2** | **83.2** | **69.9** | **72.7** |++## BenchmarkMeasured with `examples/bench.py` (3.3k-token prompt prefill → 10 sampledwarmup steps → 200 timed sampled decode tokens, 2 runs per tier):diff --git a/README_zh.md b/README_zh.mdindex ea33a25..1ff94d7 100644--- a/README_zh.md+++ b/README_zh.md@@ -173,6 +173,22 @@ engine.close() # 释放 mmap / 专家缓存- prerouter + LoRA 两条适配器都必需;缺文件时 `edge0` 会给出明确报错(也可加 `--no-prerouter` / `--no-lora` 直接跑裸基模)。+## 质量自测++内部自测分数,仅用于对比 edge0 管线(int4 量化 + 训练适配器 + prerouter+路由)相对原 fp16 基座模型的**损失——损失很小:edge0-35b 平均仅落后+3.9 分、edge0-10b 落后 2.8 分**(MMLU-Pro 甚至反超基座)。均为自测,+满分 100:++| 评测项 | edge0-35b (int4) | 基座 fp16 | edge0-10b (int4) | 基座 fp16 |+|---|---:|---:|---:|---:|+| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |+| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |+| GPQA-Diamond | 79.8 | 81.8 | 70.7 | 71.2 |+| MMLU-Pro | 81.0 | 84.6 | 70.1 | 65.8 |+| IFBench | 57.9 | 61.7 | 53.9 | 60.6 |+| **平均** | **79.2** | **83.2** | **69.9** | **72.7** |+## 性能实测`examples/bench.py` 实测(3.3k token prompt prefill → 10 步采样 warmup →