Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

fix(deps): pin mlx 0.30.6 and document the A18 NAX mis-gating

linyubupa2e168c5c7280
diff --git a/README.md b/README.md
index 8234c6e..fb8168a 100644
--- a/README.md
+++ b/README.md
@@ -43,6 +43,10 @@ are co-located with each checkpoint and load automatically, so
(M1/M2/M3/M4). The CUDA backend is on the roadmap — no other
platforms are supported yet.
- **Python**: 3.10+ (3.12 recommended).
+- **MLX**: `mlx==0.30.6` / `mlx-metal==0.30.6` with `mlx-lm==0.31.0` (see
+ `pyproject.toml`). Garbled, mixed-language output on Apple A18 / A18 Pro
+ means an older `mlx`: `pip install 'mlx==0.30.6' 'mlx-metal==0.30.6'`
+ ([#8](https://github.com/Edge0-AI/Edge0/issues/8)).
- **Memory**: ~2.9 GB peak active memory for `edge0-35b`, ~1.0 GB for
`edge0-8b` (short contexts; see [Benchmark](#benchmark)). Add
headroom for the OS, tokenizer, and long-context KV growth.
diff --git a/README_zh.md b/README_zh.md
index 89aef98..649023d 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -37,6 +37,10 @@ Recover-LoRA + prerouter 路由预判」抽象成可扩展的通用框架。后
- **系统 / 硬件**:MLX 后端目前仅支持 Apple Silicon 的 macOS
(M1/M2/M3/M4);CUDA 后端在路线图中,其余平台暂不支持。
- **Python**:3.10+(推荐 3.12)。
+- **MLX**:`mlx==0.30.6` / `mlx-metal==0.30.6`(`mlx-lm==0.31.0`,见
+ `pyproject.toml`)。Apple A18 / A18 Pro 上输出乱码 = mlx 版本旧:
+ `pip install 'mlx==0.30.6' 'mlx-metal==0.30.6'`
+ ([#8](https://github.com/Edge0-AI/Edge0/issues/8))。
- **内存**:短上下文下 `edge0-35b` ≈2.9 GB、`edge0-8b` ≈1.0 GB
峰值激活内存(见[性能实测](#性能实测));另为系统、tokenizer 与
长上下文 KV 增长预留余量。
diff --git a/pyproject.toml b/pyproject.toml
index 5186b0b..7d93e06 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -22,11 +22,15 @@ classifiers = [
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
- # NOTE: mlx-lm is pinned to 0.31.0 on purpose. Newer releases crash in
- # decode with a `tolist()` on lazily-evaluated arrays. Do not upgrade
- # without re-running the full test suite.
- "mlx==0.30.4",
- "mlx-metal==0.30.4; platform_system == 'Darwin'",
+ # NOTE: mlx/mlx-metal pinned to 0.30.6 on purpose. 0.30.4 and older
+ # mis-gate the NAX matmul kernels on Apple A18/A18 Pro (silently wrong
+ # numbers -> garbled output, issue #8); fixed upstream in 0.30.5
+ # (ml-explore/mlx#3083, #3092). Do not go to mlx>=0.31.2 yet: default
+ # streams become thread-local and the pool-thread builds in
+ # edge0/streaming/layer.py fail with "no Stream(gpu, N) in current
+ # thread". mlx-lm stays 0.31.0 (newer releases crash in decode).
+ "mlx==0.30.6",
+ "mlx-metal==0.30.6; platform_system == 'Darwin'",
"mlx-lm==0.31.0",
"numpy>=1.24",
"safetensors>=0.4",