Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python

chore: add issue forms and PR template

JiahaoWang87efe800e2c2
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
new file mode 100644
index 0000000..deb83cb
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -0,0 +1,133 @@
+name: 🐞 Bug report
+description: Something is broken, crashes, or produces wrong/garbled output.
+labels: ["bug"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Thanks for reporting a bug! Please fill in the environment fields β€” most issues we see
+ (garbled output, install errors, collapse-to-`!` loops) are pinned down by the platform,
+ chip, mlx version, and whether the prerouter/LoRA adapters are loaded.
+
+ **Before filing:** search [existing issues](https://github.com/Edge0-AI/Edge0/issues?q=is%3Aissue)
+ and check the [known mlx version pin](https://github.com/Edge0-AI/Edge0/blob/main/python/pyproject.toml)
+ (A18 / A18 Pro garbled output β†’ `mlx==0.30.6`, see #8).
+
+ - type: dropdown
+ id: platform
+ attributes:
+ label: Platform / runtime
+ description: Which runtime are you using?
+ options:
+ - Python (MLX, Apple Silicon macOS)
+ - macOS app / CLI (Rust)
+ - iOS app
+ - Android app / engine
+ - Windows app / engine
+ validations:
+ required: true
+
+ - type: dropdown
+ id: tier
+ attributes:
+ label: Model tier
+ options:
+ - edge0-35b
+ - edge0-8b
+ - both
+ - other / custom checkpoint
+ validations:
+ required: true
+
+ - type: input
+ id: hardware
+ attributes:
+ label: Hardware / chip
+ description: e.g. "Mac mini M4 Pro", "MacBook Air M1", "iPhone 16 Pro (A18 Pro)", "RTX 4090", "Pixel 10 Pro".
+ placeholder: Apple M1 Pro
+ validations:
+ required: true
+
+ - type: input
+ id: ram
+ attributes:
+ label: RAM / unified memory
+ description: Total memory of the device. Many "slow" / OOM reports come from low-memory machines.
+ placeholder: 16 GB
+ validations:
+ required: true
+
+ - type: input
+ id: os
+ attributes:
+ label: OS version
+ placeholder: macOS 15.1 / Windows 11 23H2 / Android 15 / iOS 18.1
+ validations:
+ required: true
+
+ - type: input
+ id: mlx_version
+ attributes:
+ label: mlx / mlx-metal / mlx-lm version (Python runtime only)
+ description: "Output of: pip show mlx mlx-metal mlx-lm | grep -i version β€” pin is mlx==0.30.6, mlx-lm==0.31.0."
+ placeholder: mlx 0.30.6, mlx-metal 0.30.6, mlx-lm 0.31.0
+ validations:
+ required: false
+
+ - type: dropdown
+ id: install_method
+ attributes:
+ label: How did you install / get the model?
+ options:
+ - pip install -e '.[dev,fetch]' from source
+ - scripts/fetch_models.py
+ - huggingface-cli download
+ - ModelScope
+ - Prebuilt app (macOS/iOS/Android/Windows)
+ - Other (describe below)
+ validations:
+ required: false
+
+ - type: dropdown
+ id: adapters
+ attributes:
+ label: Adapters in use
+ description: Does the bug still happen with the prerouter disabled? (`--no-prerouter`) This isolates a whole class of bugs (see #16).
+ options:
+ - Default (prerouter + LoRA enabled)
+ - "--no-prerouter"
+ - "--no-lora"
+ - Both disabled (bare base model)
+ validations:
+ required: false
+
+ - type: textarea
+ id: what_happened
+ attributes:
+ label: What happened?
+ description: What went wrong, and what did you expect instead?
+ placeholder: After the 3rd request in the same process, every output collapses to "!!!!".
+ validations:
+ required: true
+
+ - type: textarea
+ id: repro
+ attributes:
+ label: Steps to reproduce
+ description: Exact commands / code. Include the prompt if output is wrong.
+ placeholder: |
+ 1. edge0 serve edge0-35b
+ 2. curl -X POST http://127.0.0.1:8000/v1/chat/completions -d '...'
+ 3. Repeat 3 times
+ render: shell
+ validations:
+ required: true
+
+ - type: textarea
+ id: logs
+ attributes:
+ label: Logs / stack trace / output
+ description: Paste any error output or the garbled text. This will be rendered as code, no need for backticks.
+ render: shell
+ validations:
+ required: false
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 0000000..1f2c7a0
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+ - name: πŸ’¬ Questions & Discussions
+ url: https://github.com/Edge0-AI/Edge0/discussions
+ about: Usage questions, "how do I run it on X", show-and-tell, and general Q&A. Please ask here instead of opening an issue.
+ - name: πŸ“– Documentation
+ url: https://github.com/Edge0-AI/Edge0#readme
+ about: Setup, quickstart, benchmarks, and the per-platform READMEs (python/ Β· macos/ Β· ios/ Β· android/ Β· windows/).
+ - name: πŸ“§ Security / private contact
+ url: mailto:[email protected]
+ about: Report security issues or anything you cannot share publicly.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
new file mode 100644
index 0000000..a037dcc
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,48 @@
+name: πŸ’‘ Feature request
+description: Suggest a capability, backend, platform, or API improvement.
+labels: ["enhancement"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Before filing, please check the [roadmap in the README](https://github.com/Edge0-AI/Edge0#roadmap)
+ and existing issues β€” several popular asks are already tracked (CUDA backend #107/#20/#19,
+ non-macOS support #18, tool/function calling #115/#119).
+
+ - type: dropdown
+ id: area
+ attributes:
+ label: Area
+ options:
+ - New backend (CUDA / other accelerator)
+ - New platform support
+ - Server / OpenAI-compatible API
+ - CLI / Python API
+ - Quantization / adapters (LoRA / prerouter)
+ - Documentation
+ - Other
+ validations:
+ required: true
+
+ - type: textarea
+ id: problem
+ attributes:
+ label: What problem are you trying to solve?
+ description: Describe the use case and who it helps. "I have an NVIDIA GPU and want to run edge0-8b for ..."
+ validations:
+ required: true
+
+ - type: textarea
+ id: proposal
+ attributes:
+ label: Proposed solution
+ description: What would you like to see? If you're willing to contribute it, say so β€” we love PRs (see docs/adding-a-model.md for model work).
+ validations:
+ required: false
+
+ - type: textarea
+ id: alternatives
+ attributes:
+ label: Alternatives considered
+ validations:
+ required: false
diff --git a/.github/ISSUE_TEMPLATE/model_request.yml b/.github/ISSUE_TEMPLATE/model_request.yml
new file mode 100644
index 0000000..a17a204
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/model_request.yml
@@ -0,0 +1,56 @@
+name: 🧩 New model support request
+description: Request (or offer to add) support for another MoE model / checkpoint.
+labels: ["model-support"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ edge0 is built to take new models through a documented path β€” see
+ [`docs/adding-a-model.md`](https://github.com/Edge0-AI/Edge0/blob/main/docs/adding-a-model.md).
+ The best model requests come with the config below. **Contributions welcome** β€” if you
+ want to add it yourself, say so and we'll help review.
+
+ - type: input
+ id: model_name
+ attributes:
+ label: Model name
+ placeholder: Qwen3.6-Next-80B-A3B
+ validations:
+ required: true
+
+ - type: input
+ id: hf_link
+ attributes:
+ label: Hugging Face / ModelScope link
+ placeholder: https://huggingface.co/...
+ validations:
+ required: true
+
+ - type: input
+ id: model_type
+ attributes:
+ label: config.json `model_type`
+ description: The architecture id from the checkpoint's config.json β€” edge0 resolves tiers via this (see TYPE_ALIASES).
+ placeholder: qwen3_5_moe / bailing_hybrid / ...
+ validations:
+ required: false
+
+ - type: textarea
+ id: arch
+ attributes:
+ label: Architecture summary
+ description: Sparse MoE? Number of layers / experts / top-k routing width? Attention type (MHA / MLA / linear)? Anything unusual.
+ placeholder: Sparse MoE, 48 layers, 128 experts, top-8 routing, MLA attention.
+ validations:
+ required: true
+
+ - type: dropdown
+ id: contribute
+ attributes:
+ label: Are you willing to contribute this?
+ options:
+ - "Yes β€” I'd like to open a PR (happy to get guidance)"
+ - "Maybe, with help"
+ - "No, this is a request"
+ validations:
+ required: false
diff --git a/.github/ISSUE_TEMPLATE/performance_report.yml b/.github/ISSUE_TEMPLATE/performance_report.yml
new file mode 100644
index 0000000..8d5e30c
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/performance_report.yml
@@ -0,0 +1,89 @@
+name: 🐒 Performance report
+description: Decode/prefill speed, time-to-first-token, or memory usage differs from the documented benchmarks.
+labels: ["performance"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Use this for "slower than the README numbers" or "high memory / OOM" reports
+ (e.g. #17, #110). The documented benchmarks were measured on a **Mac mini M4 Pro, 24 GB**.
+ Low-memory machines (8–16 GB) can be dramatically slower because expert pages are
+ continuously evicted and re-read from SSD β€” please include your total RAM.
+
+ - type: dropdown
+ id: tier
+ attributes:
+ label: Model tier
+ options:
+ - edge0-35b
+ - edge0-8b
+ validations:
+ required: true
+
+ - type: input
+ id: hardware
+ attributes:
+ label: Hardware / chip
+ placeholder: Apple M1 Pro
+ validations:
+ required: true
+
+ - type: input
+ id: ram
+ attributes:
+ label: Total RAM / unified memory
+ placeholder: 16 GB
+ validations:
+ required: true
+
+ - type: input
+ id: disk
+ attributes:
+ label: Storage type
+ description: Internal SSD vs external/USB matters a lot for streaming expert reads.
+ placeholder: Internal NVMe SSD
+ validations:
+ required: false
+
+ - type: input
+ id: decode_observed
+ attributes:
+ label: Observed decode speed (tok/s)
+ placeholder: "0.6 tok/s (doc says ~15)"
+ validations:
+ required: false
+
+ - type: input
+ id: ttft
+ attributes:
+ label: Observed time-to-first-token
+ placeholder: 11 s
+ validations:
+ required: false
+
+ - type: input
+ id: memory
+ attributes:
+ label: Observed peak memory
+ description: Note whether this is MLX allocator peak or process RSS (they differ β€” see #106).
+ placeholder: 4.2 GB (process RSS)
+ validations:
+ required: false
+
+ - type: textarea
+ id: flags
+ attributes:
+ label: Flags / settings tried
+ description: e.g. --prefill-ondemand, full_layer_prefill, warm_willneed, context length, max_tokens.
+ placeholder: Tried warm_willneed and full_layer_prefill=False, no change.
+ validations:
+ required: false
+
+ - type: textarea
+ id: command
+ attributes:
+ label: Exact command used to measure
+ description: Ideally the output of examples/bench.py so numbers are comparable.
+ render: shell
+ validations:
+ required: true
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000..3de612b
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,28 @@
+<!-- Thanks for contributing to edge0! Keep this short but complete. -->
+
+## What & why
+
+<!-- What does this PR change, and what problem does it solve? Link issues: "Fixes #NN". -->
+
+Fixes #
+
+## Area
+
+- [ ] Python framework (`python/`)
+- [ ] macOS (`macos/`)
+- [ ] iOS (`ios/`)
+- [ ] Android (`android/`)
+- [ ] Windows (`windows/`)
+- [ ] Docs / other
+
+## How was it tested?
+
+<!-- Commands you ran, hardware/chip, model tier. Paste bench.py numbers for perf changes. -->
+
+## Checklist
+
+- [ ] `pytest -m 'not slow'` passes locally (Python changes; CI runs this on macOS + MLX)
+- [ ] Repo hygiene passes: `pytest tests/test_repo_hygiene.py` (no hardcoded paths, no secrets, **no MLX imports outside `backends/mlx/`**)
+- [ ] Added/updated tests for new behavior
+- [ ] Did **not** bump `mlx` past the pinned `0.30.6` (see the note in `python/pyproject.toml` / issue #8) unless this PR is specifically about that
+- [ ] Updated relevant docs / READMEs (incl. translations if user-facing)