Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

clean up convert_adapters command

Bind464ceb737f1
diff --git a/README.md b/README.md
index f1cb73c..1d5b87b 100644
--- a/README.md
+++ b/README.md
@@ -265,8 +265,8 @@ runs this exact path).
location automatically:
- the model directory (recommended): side by side with the base, e.g.
`lora_edge0_35b.safetensors` + `prerouter_edge0_35b.safetensors`;
- - `artifacts/` (gitignored): convert once from training-side npz
- exports via `edge0 convert-adapters --npz-dir ...`.
+ - `artifacts/` at the Python project root (gitignored): an optional
+ fallback cache for adapter safetensors not co-located with the model.
- The published model repos bundle both the base checkpoint and the
current default adapter release, so `scripts/fetch_models.py` produces
a ready-to-run model directory. Check each model's doc page for its
diff --git a/README_zh.md b/README_zh.md
index 37f615f..6b76609 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -257,8 +257,8 @@ engine.close() # 释放 mmap / 专家缓存
- **适配器**(LoRA + prerouter,safetensors)放两处任一,自动解析:
- **模型目录内**(推荐):与基模同目录,如
`lora_edge0_35b.safetensors` + `prerouter_edge0_35b.safetensors`;
- - `artifacts/`(gitignored):`edge0 convert-adapters` 从训练侧 npz
- 一次性转换。
+ - `artifacts/`(Python 项目根,gitignored):可选的回退缓存,放置未
+ 与模型同目录的适配器 safetensors。
- 发布的模型仓库同时包含基模与当前默认适配器版本,
`scripts/fetch_models.py` 下载后即为可运行的模型目录。适配器来源
(训练数据、owner 层分布)见各模型文档页。
diff --git a/docs/adding-a-model.md b/docs/adding-a-model.md
index f14ac8c..be304bb 100644
--- a/docs/adding-a-model.md
+++ b/docs/adding-a-model.md
@@ -42,7 +42,7 @@ The subclass only needs to implement the class method `_defaults(model_dir) -> C
Key points:
-- The Python project root (`python/`) ships `scripts/convert_adapters_legacy.py`, which one-shot converts the training npz exports into safetensors artifacts under `artifacts/`; `ModelConfig.artifact(name)` (`models/base.py:28–30`) returns the absolute paths of those artifacts, and the adapter uses them to fill in the LoRA / prerouter weight paths.
+- `ModelConfig.artifact(name)` (`models/base.py`) resolves an adapter's absolute path — from the model directory first, falling back to the gitignored `artifacts/` cache — and the adapter uses it to fill in the LoRA / prerouter weight paths.
- LoRA overrides go through `resolve_lora` (`models/base.py:88–95`): a bare model name resolves to that tier's artifacts, `"model_dir"` means "keep the training weights in place", and an empty string disables it.
## Step 3: `build_model` / `build_engine`
@@ -105,7 +105,7 @@ The lookup order of `_resolve_name` (registry.py:42–56): explicit `name` → t
- `config.json` exists, and its `model_type` is registered (or covered by your `TYPE_ALIASES`);
- the weights are in safetensors format, with key prefixes matching `moe_spec.key_template` (e.g. `language_model.model.layers.N.mlp.switch_mlp`);
-- the LoRA / prerouter weights are `.safetensors` files carrying metadata (see the comment at the top of `models/base.py`: they are converted one-shot from the training npz by `convert_adapters_legacy.py`).
+- the LoRA / prerouter weights are `.safetensors` files carrying metadata (see the comment at the top of `models/base.py`).
The directory basename is the last-resort resolution fallback (`_model_type_from_dir`), so aligning the directory name with the registered name is recommended, but not required.
diff --git a/docs/architecture.md b/docs/architecture.md
index 3a07c42..c5adc22 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -19,12 +19,10 @@ box: `edge0-35b` (Qwen3.6-35B-A3B, K=4) and `edge0-8b` (Ling 3.0 hybrid, K=8).
future CUDA backend (`backends/cuda/`, selected via the `EDGE0_BACKEND`
environment variable);
3. **Adapters unified on safetensors**: LoRA and prerouter weights are both
- `.safetensors` files carrying metadata; legacy npz training exports are
- converted by a one-off migration script and then deprecated;
+ `.safetensors` files carrying metadata;
4. **Unified terminology**: the pre-routing head is always called the
- **prerouter** — zero legacy-term residue in code and docs. The legacy npz
- exports are migrated by `scripts/convert_adapters_legacy.py`, which
- normalizes their key namespace to `layers.<N>.<part>.weight`.
+ **prerouter** — zero legacy-term residue in code and docs. Prerouter
+ weight keys are normalized to `layers.<N>.<part>.weight`.
## Layered Structure
diff --git a/python/scripts/convert_adapters_legacy.py b/python/scripts/convert_adapters_legacy.py
deleted file mode 100644
index 09c1b96..0000000
--- a/python/scripts/convert_adapters_legacy.py
+++ /dev/null
@@ -1,159 +0,0 @@
-#!/usr/bin/env python3
-"""One-shot legacy adapter converter: npz -> safetensors.
-
-The training pipelines exported adapters as ``.npz`` (qwen round-7 and
-the training side). edge0 consumes safetensors ONLY — this script converts each
-legacy pair once into ``artifacts/`` (gitignored), writing the
-conversion provenance (model, kind, K, r/alpha, source md5, format
-version) into the safetensors ``__metadata__``. The npz sources are
-NOT deleted (see the conversion note in README); they are only needed
-to regenerate the artifacts.
-
-Usage:
- python scripts/convert_adapters_legacy.py [--force]
-
-Idempotent: existing artifacts are skipped unless ``--force``.
-"""
-
-from __future__ import annotations
-
-import argparse
-import hashlib
-import os
-import json
-import sys
-from datetime import datetime, timezone
-from pathlib import Path
-
-import numpy as np
-
-ROOT = Path(__file__).resolve().parents[1]
-ARTIFACTS = ROOT / "artifacts"
-
-# (npz path, tier, kind) — every legacy source this script understands.
-# Training-side npz exports discovered per machine; pass --npz-dir (or
-# edit this list) to point at your own exports.
-NPZ_DIRS = [
- Path(os.environ.get("EDGE0_NPZ_DIR", "/path/to/npz/exports")),
-]
-SOURCES = [
- ("prerouter_<family>.npz", "edge0-35b", "prerouter"),
- ("lora_<family>.npz", "edge0-35b", "lora"),
- ("prerouter_<family>.npz", "edge0-8b", "prerouter"),
- ("lora_<family>.npz", "edge0-8b", "lora"),
-]
-
-
-def _find_source(name: str) -> Path | None:
- for d in NPZ_DIRS:
- p = d / name
- if p.is_file():
- return p
- return None
-
-
-def md5(path: Path) -> str:
- h = hashlib.md5()
- with open(path, "rb") as f:
- for chunk in iter(lambda: f.read(1 << 20), b""):
- h.update(chunk)
- return h.hexdigest()
-
-
-def rewrite_key(key: str, tier: str, kind: str) -> str:
- """Map a legacy npz key to the edge0 artifact key.
-
- prerouter heads are normalized to ``layers.<N>.<part>.weight``;
- qwen lora gains the ``language_model.`` prefix (the checkpoint
- namespace the adapter is applied into).
- """
- if kind == "prerouter":
- # Some families use layers.N.*; others namespace the layer index
- # with a prefix segment (<ns>.<N>.<part>.weight). Both are
- # normalized to layers.<N>.<part>.weight.
- parts = key.split(".")
- if (len(parts) >= 4 and parts[0] != "layers"
- and parts[1].isdigit()):
- return f"layers.{parts[1]}.{parts[2]}.{parts[3]}"
- return key
- if kind == "lora" and tier == "edge0-35b" and key.startswith("model.layers."):
- return "language_model." + key
- return key
-
-
-def convert(source: Path, tier: str, kind: str, force: bool = False) -> Path:
- name = {
- ("edge0-35b", "prerouter"): "prerouter_edge0_35b_k4.safetensors",
- ("edge0-35b", "lora"): "lora_edge0_35b_k4.safetensors",
- ("edge0-8b", "prerouter"): "prerouter_edge0_8b.safetensors",
- ("edge0-8b", "lora"): "lora_edge0_8b.safetensors",
- }[(tier, kind)]
- out = ARTIFACTS / name
- if out.exists() and not force:
- print(f"skip {out.name} (exists)")
- return out
-
- data = np.load(source)
- tensors = {}
- for key in data.files:
- tensors[rewrite_key(key, tier, kind)] = data[key]
-
- # provenance: rank from the A matrix, alpha from the training run
- # (32 for both shipped pairs), K from the tier profile.
- meta = {
- "model": tier,
- "kind": kind,
- "K": "4" if tier == "edge0-35b" else "8",
- "r": "16", "alpha": "32",
- "source": str(source),
- "source_md5": md5(source),
- "converted": datetime.now(timezone.utc).isoformat(timespec="seconds"),
- "format_version": "1",
- }
- if kind == "prerouter":
- owners = sorted({int(k.split(".")[1])
- for k in tensors if k.startswith("layers.")})
- meta["owners"] = json.dumps(owners)
-
- from safetensors.numpy import save_file as save_np
- save_np(tensors, str(out), metadata={"__metadata__": json.dumps(meta)})
- print(f"write {out.name} ({len(tensors)} tensors, "
- f"{out.stat().st_size >> 20} MiB)")
- return out
-
-
-def main(argv: list[str] | None = None) -> int:
- ap = argparse.ArgumentParser(description=__doc__)
- ap.add_argument("--force", action="store_true",
- help="reconvert even if the artifact exists")
- ap.add_argument("--npz-dir", default=None,
- help="directory holding the legacy npz exports "
- "(default: $EDGE0_NPZ_DIR)")
- args = ap.parse_args(argv)
-
- if args.npz_dir:
- NPZ_DIRS.insert(0, Path(args.npz_dir))
- ARTIFACTS.mkdir(parents=True, exist_ok=True)
- missing = []
- resolved = []
- for name, tier, kind in SOURCES:
- p = _find_source(name)
- if p is None:
- missing.append(name)
- else:
- resolved.append((p, tier, kind))
- if missing:
- print("missing legacy sources (skip their conversion):")
- for m in missing:
- print(" -", m)
- for path, tier, kind in resolved:
- try:
- convert(path, tier, kind, force=args.force)
- except Exception as exc: # noqa: BLE001 — report and continue
- print(f"FAIL {path.name}: {exc}", file=sys.stderr)
- return 1
- return 0
-
-
-if __name__ == "__main__":
- raise SystemExit(main())
diff --git a/python/src/edge0/adapters/lora.py b/python/src/edge0/adapters/lora.py
index b13bf32..1103dd8 100644
--- a/python/src/edge0/adapters/lora.py
+++ b/python/src/edge0/adapters/lora.py
@@ -16,9 +16,7 @@ Safetensors key naming (framework contract)::
where ``<target>`` is the module path inside the loaded model (e.g.
``model.layers.3.attention.q_proj`` or
-``language_model.model.layers.0.linear_attn.in_proj_qkv``). The legacy
-npz files used the same A/B naming, so conversion is a pure format
-migration (see ``scripts/convert_adapters_legacy.py``).
+``language_model.model.layers.0.linear_attn.in_proj_qkv``).
"""
from __future__ import annotations
diff --git a/python/src/edge0/cli.py b/python/src/edge0/cli.py
index 5b3ec54..097442c 100644
--- a/python/src/edge0/cli.py
+++ b/python/src/edge0/cli.py
@@ -6,7 +6,6 @@ Commands:
edge0 serve start the HTTP server (one model, queued generations)
edge0 chat one-shot prompt -> answer on the terminal
edge0 models list registered tiers and their default profiles
- edge0 convert-adapters one-shot legacy npz -> safetensors migration
"""
from __future__ import annotations
@@ -208,25 +207,6 @@ def cmd_serve(args) -> int:
return 0
-def cmd_convert(args) -> int:
- from pathlib import Path
-
- scripts_dir = Path(__file__).resolve().parents[2] / "scripts"
- if str(scripts_dir) not in sys.path:
- sys.path.insert(0, str(scripts_dir))
- # Delegate to the legacy converter's own main(argv) with a clean argv,
- # so the subcommand name never leaks into its argparse (runpy would
- # re-parse sys.argv and choke on "convert-adapters").
- from convert_adapters_legacy import main as _convert_main
-
- argv: list[str] = []
- if getattr(args, "force", False):
- argv.append("--force")
- if getattr(args, "npz_dir", None):
- argv += ["--npz-dir", args.npz_dir]
- return _convert_main(argv)
-
-
def _add_engine_flags(p) -> None:
"""Engine-construction flags shared by demo / chat / serve."""
p.add_argument("--no-prerouter", action="store_true")
@@ -295,15 +275,6 @@ def _build_parser() -> argparse.ArgumentParser:
_add_engine_flags(p)
p.set_defaults(fn=cmd_serve)
- p = sub.add_parser("convert-adapters",
- help="one-shot legacy npz -> safetensors migration")
- p.add_argument("--force", action="store_true",
- help="reconvert even if the artifact exists")
- p.add_argument("--npz-dir", default=None,
- help="directory holding the legacy npz exports "
- "(default: $EDGE0_NPZ_DIR)")
- p.set_defaults(fn=cmd_convert)
-
return ap
diff --git a/python/src/edge0/models/base.py b/python/src/edge0/models/base.py
index bb1d73d..d6ede85 100644
--- a/python/src/edge0/models/base.py
+++ b/python/src/edge0/models/base.py
@@ -19,16 +19,16 @@ from edge0.moe.spec import MoESpec
from edge0.prerouter.spec import PrerouterSpec
from edge0.streaming.options import LayerOptions
-# Project-root artifacts (gitignored; produced once by
-# scripts/convert_adapters_legacy.py from the training npz exports, then
-# the source npz are discarded).
+# Project-root artifacts (gitignored): an optional local cache of adapter
+# safetensors. Adapters normally ship inside the model directory; this is
+# only the fallback location artifact() returns when they are not there.
ARTIFACTS_DIR = Path(__file__).resolve().parents[3] / "artifacts"
def artifact(name: str, model_dir: str | None = None) -> str:
"""Path of one adapter file: the model's own directory first
(model and its adapters side by side in one
- directory), falling back to ``artifacts/`` (conversion cache)."""
+ directory), falling back to ``artifacts/`` (optional local cache)."""
if model_dir:
cand = Path(model_dir) / name
if cand.is_file():