Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

README: self-evaluation scores vs fp16 base models

linyubupa61aa744d374a
diff --git a/README.md b/README.md
index 44150b7..ec18cd1 100644
--- a/README.md
+++ b/README.md
@@ -207,8 +207,24 @@ this exact path).
- Both adapters are required for the prerouter + LoRA pipeline; if a
file is missing, `edge0` fails with a clear message (or pass
`--no-prerouter` / `--no-lora` to run the plain base model).
-
-
+## Quality (self-evaluation)
+
+Internal self-evaluation, included only to quantify the loss of the edge0
+pipeline (int4 + trained adapters + prerouter routing) relative to the
+original fp16 base models — the loss is small: **3.9 points on average for
+edge0-35b, 2.8 for edge0-10b** (MMLU-Pro is even above the base). All scores
+self-run, max 100:
+
+| Benchmark | edge0-35b (int4) | Base fp16 | edge0-10b (int4) | Base fp16 |
+|---|---:|---:|---:|---:|
+| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |
+| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |
+| GPQA-Diamond | 79.8 | 81.8 | 70.7 | 71.2 |
+| MMLU-Pro | 81.0 | 84.6 | 70.1 | 65.8 |
+| IFBench | 57.9 | 61.7 | 53.9 | 60.6 |
+| **Average** | **79.2** | **83.2** | **69.9** | **72.7** |
+
+## Benchmark
Measured with `examples/bench.py` (3.3k-token prompt prefill → 10 sampled
warmup steps → 200 timed sampled decode tokens, 2 runs per tier):
diff --git a/README_zh.md b/README_zh.md
index ea33a25..1ff94d7 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -173,6 +173,22 @@ engine.close() # 释放 mmap / 专家缓存
- prerouter + LoRA 两条适配器都必需;缺文件时 `edge0` 会给出明确报错
(也可加 `--no-prerouter` / `--no-lora` 直接跑裸基模)。
+## 质量自测
+
+内部自测分数,仅用于对比 edge0 管线(int4 量化 + 训练适配器 + prerouter
+路由)相对原 fp16 基座模型的**损失——损失很小:edge0-35b 平均仅落后
+3.9 分、edge0-10b 落后 2.8 分**(MMLU-Pro 甚至反超基座)。均为自测,
+满分 100:
+
+| 评测项 | edge0-35b (int4) | 基座 fp16 | edge0-10b (int4) | 基座 fp16 |
+|---|---:|---:|---:|---:|
+| AIME 2026 | 86.6 | 92.7 | 63.3 | 73.3 |
+| HumanEval | 90.9 | 95.1 | 91.5 | 92.7 |
+| GPQA-Diamond | 79.8 | 81.8 | 70.7 | 71.2 |
+| MMLU-Pro | 81.0 | 84.6 | 70.1 | 65.8 |
+| IFBench | 57.9 | 61.7 | 53.9 | 60.6 |
+| **平均** | **79.2** | **83.2** | **69.9** | **72.7** |
+
## 性能实测
`examples/bench.py` 实测(3.3k token prompt prefill → 10 步采样 warmup →