Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs(fmt2): M1 Ultra fmt=2 benchmark context β€” issue/PR summaries, plans, scripts

boblaublaw1743054e489f
diff --git a/c/convert_fmt4_to_fmt2.sh b/c/convert_fmt4_to_fmt2.sh
new file mode 100755
index 00000000..f6d402e3
--- /dev/null
+++ b/c/convert_fmt4_to_fmt2.sh
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+rm -rf ~/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp
+
+python3 ./coli convert \
+ --model /mnt/zfs1/noprot/mlx-lm/models/GLM-5.2-colibri-int4-g64-with-int8-mtp \
+ --outdir ~/mlx-models/GLM-5.2-colibri-int4-perrow \
+ --group-size 0
diff --git a/c/venv.sh b/c/venv.sh
new file mode 100755
index 00000000..37393177
--- /dev/null
+++ b/c/venv.sh
@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# colibri virtual environment helper
+# Usage: source venv.sh
+
+if [ -z "$COLI_VENV_ACTIVE" ]; then
+ if [ -d ".venv" ]; then
+ source .venv/bin/activate
+ export COLI_VENV_ACTIVE=1
+ echo "βœ“ colibri venv activated (numpy, huggingface_hub, safetensors)"
+ else
+ echo "Creating venv..."
+ python3 -m venv .venv
+ source .venv/bin/activate
+ python -m pip install --quiet --upgrade pip
+ python -m pip install --quiet numpy huggingface_hub safetensors
+ export COLI_VENV_ACTIVE=1
+ echo "βœ“ venv created and activated"
+ fi
+fi
diff --git a/c/x.sh b/c/x.sh
new file mode 100755
index 00000000..b245ab69
--- /dev/null
+++ b/c/x.sh
@@ -0,0 +1,24 @@
+#!/bin/bash -x
+
+# ============================================================
+# M1 Ultra Metal benchmark β€” "Compare the myths of Lucifer and Prometheus"
+#
+# The original run (below, Config B from METAL-M5MAX-PERF-REPORT.md)
+# produces ~0.14 tok/s because PIPE defaults to 0 on macOS and OMP
+# active-spin throttles the GPU. See docs/METAL-M5MAX-PERF-REPORT.md
+# for the full analysis.
+#
+# The winning config (E) from the M5 Max report achieves 2.24 tok/s
+# by adding PIPE=1 PIPE_WORKERS=8 (async expert loading) and
+# COLI_NO_OMP_TUNE=1 (passive OMP waits, preserves GPU clocks).
+# M1 Ultra (24 GPU cores vs M5 Max's 40) should target >=2.0 tok/s.
+# ============================================================
+
+# OLD β€” missing PIPE + NO_OMP_TUNE; produces ~1.25 tok/s or worse
+# COLI_METAL=1 DIRECT=1 MTP=0 ./coli run --model /Users/joe/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp/ --ram 110 "Compare the myths of Lucifer and Prometheus"
+
+# SHORT RUN β€” verify PIPE+NO_OMP_TUNE fix with 128 tokens (fast check, ~60-120s at 0.16 tok/s or ~60s at 2 tok/s)
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 ./coli run --model /Users/joe/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp/ --ram 110 --ngen 128 "Compare the myths of Lucifer and Prometheus"
+
+# FULL RUN β€” 1024 tokens once short run confirms correct throughput
+# COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 ./coli run --model /Users/joe/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp/ --ram 110 "Compare the myths of Lucifer and Prometheus"
diff --git a/docs/issue_103.md b/docs/issue_103.md
new file mode 100644
index 00000000..3cbd96fe
--- /dev/null
+++ b/docs/issue_103.md
@@ -0,0 +1,164 @@
+[Performance]: tok/s of 2.06 tok/s !!! Β· Issue #103 Β· JustVugg/colibri Β· GitHub
+
+# [Performance]: tok/s of 2.06 tok/s !!! #103
+
+**RDouglasSharp** opened on Jul 12, 2026
+
+## Commit
+pr-72-metal
+
+## Hardware and storage
+Apple M5 Max 128Mb 14" 1
+8 core (6 Super and 12 Performance)
+40 Core Metal 4 GPU
+
+## Software environment
+sudo sysctl iogpu.wired_limit_mb=120832
+
+## Build and benchmark commands
+COLI_METAL=1 DIRECT=1 MTP=0 ./coli run --model /Users/doug/glm52_i4 "Compare the myths of Lucifer and Prometheus" --ram 110
+
+## Results
+```
+COLI_METAL=1 DIRECT=1 MTP=0 ./coli run --model /Users/doug/glm52_i4 "Compare the myths of Lucifer and Prometheus" --ram 110
+
+ ▄▀▀▀▄ ▄ colibrì v1.0
+ β–„β–„β–„β–„β–€β–€β–€β–€β–„β–€β–€ piccolo motore, modello immenso
+ β–€β–€β–€β–€β–€β–€β–€ GLM-5.2 Β· 744B MoE Β· int4 Β· streaming CPU
+ β–€β–€β–€β–€ run
+ β–€
+ ──────────────────────────────────────────────────────────
+[METAL] mode: batched routed experts on GPU (unified-memory zero-copy)
+== Motore C GLM (glm_moe_dsa), cache=8 expert/layer | expert@8-bit densa@8-bit | idot: neon ==
+caricato in 0.81s | densa residente: 9647.10 MB | layers=78 experts=256 | MTP assente (draft=0)
+[MTP] assente (draft=0)
+[USAGE] storia expert: 2936400 selezioni (/Users/doug/glm52_i4/.coli_usage)
+[PIN] hot-store: 2478 expert in RAM (46.9 GB) in 3s da /Users/doug/glm52_i4/.coli_usage
+[PIN] mlock: 46.9 GB inchiodati in RAM fisica / wired in physical RAM (niente compressione/no compression) in 1s
+[RAM_GB=110.0] cap ALZATO 8->33: il budget lo consente (proiezione picco 109.9 GB; CAP_RAISE=0 per disattivare)
+[stop] 3 token di stop: 154820 154827 154829
+prompt: 13 token | genero fino a 1024 (stop EOS=154820) | draft n-gram=0
+[gMASK]<sop><|user|>Compare the myths of Lucifer and Prometheus<|assistant|></think>[prefill] layer 1/78 Β· 13 token
+[prefill] layer 5/78 Β· 13 token
+[prefill] layer 9/78 Β· 13 token
+[prefill] layer 13/78 Β· 13 token
+[prefill] layer 17/78 Β· 13 token
+[prefill] layer 21/78 Β· 13 token
+[prefill] layer 25/78 Β· 13 token
+[prefill] layer 29/78 Β· 13 token
+[prefill] layer 33/78 Β· 13 token
+[prefill] layer 37/78 Β· 13 token
+[prefill] layer 41/78 Β· 13 token
+[prefill] layer 45/78 Β· 13 token
+[prefill] layer 49/78 Β· 13 token
+[prefill] layer 53/78 Β· 13 token
+[prefill] layer 57/78 Β· 13 token
+[prefill] layer 61/78 Β· 13 token
+[prefill] layer 65/78 Β· 13 token
+[prefill] layer 69/78 Β· 13 token
+[prefill] layer 73/78 Β· 13 token
+[prefill] layer 77/78 Β· 13 token
+[prefill] layer 78/78 Β· 13 token
+The myths of Lucifer (in Christian tradition) and Prometheus (in Greek tradition)
+[t=16 RSS 97.83 GB hit 61% 1.23 tok/s 1.00 tok/fw] represent two of the most profound and complex archetypes in human mythology: the "
+[t=32 RSS 97.89 GB hit 67% 1.60 tok/s 1.00 tok/fw] rebel" or the "trickster" who defies the heavens.
+[t=48 RSS 97.89 GB hit 70% 1.81 tok/s 1.00 tok/fw] Though they originate from entirely different cultural and theological contexts, their narratives share striking structural
+[t=64 RSS 97.89 GB hit 70% 1.85 tok/s 1.00 tok/fw] and thematic similarities. as well as distinct differences.
+
+Here is a detailed comparison of
+[t=80 RSS 97.89 GB hit 71% 1.90 tok/s 1.00 tok/fw] the two figures:
+
+## **1. Core Identity and Sphere of Influence**
+
+**
+[t=96 RSS 97.89 GB hit 72% 1.96 tok/s 1.00 tok/fw] Lucifer**
+* **Origin:**** The name "Lucifer" translates
+[t=112 RSS 97.89 GB hit 73% 2.02 tok/s 1.00 tok/fw] from Latin *lux* (light) + *ferre* (to carry
+[t=128 RSS 97.89 GB hit 73% 2.05 tok/s 1.00 tok/fw] or to bring), meaning "Light-Bringer" or "Morning Star."
+[t=144 RSS 97.89 GB hit 74% 2.08 tok/s 1.00 tok/fw] The title originally referred to the planet Venus in its morning appearance, signifying the
+[t=160 RSS 97.89 GB hit 74% 2.09 tok/s 1.00 tok/fw] dawn and bringing of light to dispel the darkness of night.
+* **
+[t=176 RSS 97.89 GB hit 74% 2.10 tok/s 1.00 tok/fw] Christian Tradition:**** In Christian theology, Lucifer is often identified with Satan, the
+[t=192 RSS 97.89 GB hit 74% 2.11 tok/s 1.00 tok/fw] adversary. However, strictly speaking, Lucifer is a Latin translation of the Hebrew phrase
+[t=208 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] *helel ben-shahar* (meaning "Morning Star, son
+[t=224 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] of the Dawn") found in Isaiah 14:12. This passage was a
+[t=240 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] prophecy against the King of Babylon, mocking his human arrogance and ambition.
+*
+[t=256 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] **Mythological Evolution:**** Over time, through the influence of literature like
+[t=272 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] John Milton's *Paradise Lost* (1667) and Dante's
+[t=288 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] *Inferno* (1320), Lucifer became firmly established in the popular
+[t=304 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] consciousness as the name of the archangel who, driven by pride, rebelled
+[t=320 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] against God. He was cast down to hell...
+* **Role:****
+[t=336 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] In Christian theology, Lucifer/Satan plays the role of the "adversary
+[t=352 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] " or the "accuser." He is the great antagonist in the Christian cosmic
+[t=368 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] drama: a fallen angel who defies the divine order and brings evil and suffering
+[t=384 RSS 97.89 GB hit 74% 2.13 tok/s 1.00 tok/fw] into the world through his deception of Adam and Eve in the Garden of Eden.
+[t=400 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] His "crime" is primarily one of *hubris*β€”overweening
+[t=416 RSS 97.89 GB hit 74% 2.12 tok/s 1.00 tok/fw] ambition and prideβ€”which leads to a rebellion against the divine hierarchy.
+* **
+[t=432 RSS 97.89 GB hit 73% 2.12 tok/s 1.00 tok/fw] Symbolic Meaning:**** Symbolically, Lucifer represents the sin of pride ("*
+[t=448 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] invidia*" in Latin, meaning "envy"). He is the embodiment of
+[t=464 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] the rebellious intellect who seeks to become equal to the gods..
+
+**Prom
+[t=480 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] ometheus**
+* **Origin:**** In Greek myth, Prometheus ("Fores
+[t=496 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] ightful") is a second-generation Titan, a race of powerful beings who ruled
+[t=512 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] the cosmos during the Golden Age.* This is a key difference: while Lucifer
+[t=528 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] is an angel (a servant of God, created solely to obey and worship),
+[t=544 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] Prometheus is a Titan, a primordial cosmic entity, born of the Earth (
+[t=560 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] Gaia) and Sky (Uranus).
+* **Mythological
+[t=576 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] Role:**** Prometheus is the quintessential "trickster" and culture-hero
+[t=592 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] of Greek myth. He is the creator of humanity, fashioning the first humans
+[t=608 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] from clay. He is the benefactor of humanity, whose profound love for his
+[t=624 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] human creations drives him to steal fire from the gods to give to humans, enabling
+[t=640 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] human progress and the dawn of civilization. Because of this theft, he is chained
+[t=656 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] to a rock in the Caucasus Mountains, where an eagle (the symbol of
+[t=672 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] Zeus) eats his liver daily in perpetuity.
+* **Symbolic Meaning
+[t=688 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] :**** Prometheus represents human aspiration and intellect. He embodies "Promethean"
+[t=704 RSS 97.89 GB hit 73% 2.10 tok/s 1.00 tok/fw] ambitionβ€”the drive to achieve greatness through technological and intellectual mastery over nature. He is
+[t=720 RSS 97.89 GB hit 73% 2.10 tok/s 1.00 tok/fw] the patron of human striving and civilization. His myth explores the relationship between *mind
+[t=736 RSS 97.89 GB hit 73% 2.10 tok/s 1.00 tok/fw] * (human intellect and craft) and *matter* (the physical world).
+
+
+[t=752 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] ## **2. Motivations: Rebellion, Pride, and Human Agency**
+
+
+[t=768 RSS 97.89 GB hit 73% 2.11 tok/s 1.00 tok/fw] **Lucifer: Rebellion as Sin**
+In the Christian tradition, Lucifer's rebellion
+[t=784 RSS 97.89 GB hit 73% 2.10 tok/s 1.00 tok/fw] is a sin born of pride and ambition. In Christian theology, angels are strictly
+[t=800 RSS 97.89 GB hit 73% 2.10 tok/s 1.00 tok/fw] created beings, whose purpose is to serve and praise God.* When Lucifer says
+[t=816 RSS 97.89 GB hit 73% 2.09 tok/s 1.00 tok/fw] , "I will ascend into heaven... I will make myself like the Most High
+[t=832 RSS 97.89 GB hit 73% 2.09 tok/s 1.00 tok/fw] " (Isaiah :13-14), he is committing the sin
+[t=848 RSS 97.89 GB hit 73% 2.09 tok/s 1.00 tok/fw] of *hubris* (overwhelming ambition) and *invidia* (
+[t=864 RSS 97.89 GB hit 73% 2.09 tok/s 1.00 tok/fw] envy). He refuses to accept his divinely ordained place in the cosmos.
+
+[t=880 RSS 97.89 GB hit 73% 2.09 tok/s 1.00 tok/fw] **Prometheus: Rebellion as Human Agency**
+In contrast, Prometheus's defiance of
+[t=896 RSS 97.89 GB hit 73% 2.08 tok/s 1.00 tok/fw] Zeus is not a sin, but an act of profound compassion and human agency.
+[t=912 RSS 97.89 GB hit 73% 2.08 tok/s 1.00 tok/fw] His "crime" was motivated by pity for human beings' physical weakness and ignorance
+[t=928 RSS 97.89 GB hit 73% 2.08 tok/s 1.00 tok/fw] . Unlike Lucifer, who is jealous of God, Prometheus is driven by *ag
+[t=944 RSS 97.89 GB hit 73% 2.07 tok/s 1.00 tok/fw] ape* (unconditional love) for his creations, humans. By stealing fire
+[t=960 RSS 97.89 GB hit 73% 2.07 tok/s 1.00 tok/fw] , he acts as a culture-bringer, giving humans the tools of civilizationβ€”
+[t=976 RSS 97.89 GB hit 73% 2.07 tok/s 1.00 tok/fw] language, science,, and the artsβ€”enabling them to become masters of
+[t=992 RSS 97.89 GB hit 73% 2.07 tok/s 1.00 tok/fw] their own fate and to progress.
+**Lucifer and Prometheus as "Light-
+[t=1008 RSS 97.89 GB hit 73% 2.06 tok/s 1.00 tok/fw] Bringers"**
+Despite their different roles, both Lucifer and Prometheus are "light
+
+---
+1024 token in 496.34s (2.06 tok/s) | hit-rate expert 72.5% | RSS 97.89 GB
+expert caricati/token: 607.0 (per-layer 8.09 su 75; baseline topk=8) | TOPK=0 TOPP=0.00
+speculazione: 1.00 token/forward (1023 fw per 1024 tok) | MTP acceptance 0% (0/0)
+PROFILO: expert-disk 266.259s | expert-matmul 109.471s | attention 100.133s (di cui kvb 0.038s) | lm_head 0.004s | altro 20.475s
+METAL-ATTN: layer GPU 79794 | gpu-wall 96.35s (kernel 76.36s | cpu-sched 0.62s gpu-sched 11.63s)
+METAL: blocchi GPU 142391 | fallback CPU 0 | expert su GPU 618242 | setup 15.35s gpu-wall 86.02s (kernel 35.24s) scatter 1.46s
+```
+
+## Baseline comparison
+
+Note that this Metal PR is based on an older base, missing features like interleaved expert loading.
+So once properly merged, the tok/sec may be slightly higher!
diff --git a/docs/issue_107.md b/docs/issue_107.md
new file mode 100644
index 00000000..858b8d06
--- /dev/null
+++ b/docs/issue_107.md
@@ -0,0 +1,22 @@
+# issue_107: Benchmark: Mac Mini M4 Pro (48 GB) β€” 0.30 tok/s with Metal backend (PR #72), 0.18 CPU-only
+
+- **URL:** https://github.com/JustVugg/colibri/issues/107
+- **State:** CLOSED (opened 2026-07-13 by bloxi-bot, closed 2026-07-13)
+
+## Summary
+
+First field report of the Metal backend (PR #72 branch) on a small-RAM Mac. Mac Mini M4 Pro (10P+4E CPU, 16-core GPU, 48 GB unified, ~273 GB/s), internal 1 TB SSD. Model: pre-converted int4 from `jlnsrk/GLM-5.2-colibri-int4` with **MTP heads swapped for the int8 ones** from `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` (the main repo then still shipped the broken int4 sizes from #8).
+
+- Disk: buffered 8.38 GB/s, F_NOCACHE 6.59 GB/s
+- **Metal (PR #72), defaults, 128 tok: 0.30 tok/s** vs CPU-only 0.18 tok/s (all runs `--ram 38`)
+- Metal run detail: `METAL: blocchi GPU 9018 | fallback CPU 0 | expert su GPU 100933`; expert-matmul 78.7s vs ~204s CPU-equivalent (~2.6x); METAL-ATTN 4368 layers on GPU; RSS 27.4 GB; profile 59% disk
+- Throughput warmed from 0.23 at t=16 to steady 0.30-0.31 by t=64
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Shows the Metal dispatch counters (`blocchi GPU`, `fallback CPU`, `expert su GPU`, `METAL-ATTN`) of a healthy run on a constrained box β€” the same counters the M1 Ultra fmt=4 run showed as all-zero (see [metal_dispatch_gap.md](metal_dispatch_gap.md)).
+- **Goal b (fmt_4_m1u):** Context β€” what healthy Metal dispatch output looks like.
+
+## See also
+
+- [pr_72](pr_72.md), [issue_103](issue_103.md)
diff --git a/docs/issue_180.md b/docs/issue_180.md
new file mode 100644
index 00000000..aa221bb2
--- /dev/null
+++ b/docs/issue_180.md
@@ -0,0 +1,28 @@
+# issue_180: Datapoint: M3 Ultra (Mac Studio, 256 GB, 80-core GPU) β€” 1.45 tok/s @ 82% hit, Metal + tuned flags
+
+- **URL:** https://github.com/JustVugg/colibri/issues/180
+- **State:** CLOSED (opened 2026-07-14 by NullNaveen, closed 2026-07-14)
+
+## Summary
+
+Only Ultra-class Apple Silicon datapoint in the tracker. Mac Studio M3 Ultra, 256 GB unified, 80-core GPU, `main` built `make glm METAL=1`, model `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp`. **Shared box** (concurrent 24-user Ollama workload, ~46 GB resident on the same GPU) β€” numbers are a lower bound.
+
+| Config | tok/s | expert hit | experts/token | RSS |
+|---|---|---|---|---|
+| CPU, `--ram 40`, defaults | 0.07 | 19% | 1021 | 37 GB |
+| Metal, `--ram 40`, defaults | 0.39 | 21% | 1142 | 37 GB |
+| **Metal + tuned, `--ram 160`** | **1.45** (peak 1.75) | **82%** | **381** | 131 GB |
+
+- Tuned config: `COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 IO_THREADS=8 ./coli run --ram 160 --topp 0.7`
+- Profile of the 1.45 run: expert-disk 75.3s service / 0.0s wait, expert-matmul 49.3s, attention 45.0s, other 82.1s
+- Confirms `MTP=0` is a large win: 1142 -> 381 experts/token; speculation was working (53-67% acceptance) but still not worth it
+- Note: 82% hit rate helped here (256 GB box, `--ram 160`) β€” contrast with #379's minimal-cache finding on 128 GB M5 Max
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** The closest hardware analog to the M1 Ultra target β€” Ultra-class, unified memory, same tuned flag set. Template for the M1 Ultra Config E and `--ram` exploration.
+- **Goal b (fmt_4_m1u):** Context only.
+
+## See also
+
+- [issue_103](issue_103.md), [pr_116](pr_116.md), [issue_379](issue_379.md)
diff --git a/docs/issue_379.md b/docs/issue_379.md
new file mode 100644
index 00000000..c2eaa9cb
--- /dev/null
+++ b/docs/issue_379.md
@@ -0,0 +1,25 @@
+# issue_379: Metal β€” growing the expert cache makes decode SLOWER
+
+- **URL:** https://github.com/JustVugg/colibri/issues/379
+- **State:** OPEN (opened 2026-07-18 by monotophic)
+- **Labels:** bug, metal
+
+## Summary
+
+On Metal (M5 Max 128 GB, macOS 26.5.2, `main @ 72d3d37`), **every expert-cache mechanism makes decode slower** β€” raising the LRU cap, adding a pin, or enabling AUTOPIN all reduce tok/s even as hit rate rises. Fastest config found: *minimal* cache at **~2.0 tok/s** vs ~0.4 at auto defaults (5Γ— from config alone).
+
+- Champion config: `MTP=0 CAP_RAISE=0 AUTOPIN=0 --cap 1`
+- Suspected mechanism: GPU-side buffer residency / page-mapping cost on large, sparsely-reused host buffers registered via `newBufferWithBytesNoCopy` β€” on Metal, cache hits cost more than the page-cache misses they replace
+- Cross-platform contrast: DGX Spark GB10 gets 2.39 tok/s at 82% hit with a large LRU + AUTOPIN; the M5 Max gets its best at **2% hit, no cache**
+- Four alternative mechanisms falsified via controlled A/Bs (data in #387)
+- Led directly to PR #386 (platform-aware cache defaults via storage probe)
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Critical tuning context β€” on a 128 GB Apple Silicon box, a *big* expert cache can be anti-productive; M1 Ultra benchmark configs must A/B cache size (`--cap`, CAP_RAISE, AUTOPIN), not assume more cache = faster.
+- **Goal b (fmt_4_m1u):** Shared β€” same cache tuning applies to the fmt=4 benchmark runs after backend fixes land.
+
+## See also
+
+- [issue_387](issue_387.md) β€” the A/B data behind this diagnosis
+- [pr_386](pr_386.md) β€” the merged defaults change
diff --git a/docs/issue_387.md b/docs/issue_387.md
new file mode 100644
index 00000000..3067ecfb
--- /dev/null
+++ b/docs/issue_387.md
@@ -0,0 +1,27 @@
+# issue_387: Datapoint: M5 Max 128 GB β€” decode-tuning A/Bs (MTP x cache 2x2): 0.4 -> 2.0 tok/s at identical output
+
+- **URL:** https://github.com/JustVugg/colibri/issues/387
+- **State:** OPEN (opened 2026-07-18 by monotophic)
+- **Labels:** metal, benchmark
+
+## Summary
+
+Controlled, identical-output A/B campaign (greedy temp-0, md5-checked token streams, pristine `.coli_usage` snapshot restored per run) on M5 Max 128 GB, measured 2026-07-15..17, `main @ 72d3d37`, model `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` (144 shards / 384 GB).
+
+- **Champion config:** `MTP=0 CAP_RAISE=0 AUTOPIN=0 coli run --ram 90 --cap 1` β€” takes the box from ~0.4 tok/s (defaults) to ~2.0 tok/s
+- **`MTP=0`:** +79% at a cached config, +10-22% at cap1; -20-45% bytes moved
+- **`--cap 1` (no LRU cache):** smaller cache = faster, always; expert-matmul 20.6s -> 35.4s -> 78.4s as the cache grows
+- **`--ram`:** flat on speed across 20-40 GB; choose on heat/energy, not speed
+- **`COLI_MMAP=1`:** do not use on Metal β€” 39x slower (GPU demand-faults a 358 GB mapping)
+- First two measurement rounds were invalid and discarded; the methodology section documents what survived (temp 0, usage-snapshot restore, etc.)
+
+This is the data behind the #379 diagnosis and the PR #386 defaults change.
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Direct tuning playbook for the M1 Ultra benchmark β€” MTP off, cache minimal, ram sizing, and a rigorous identical-output methodology worth mirroring.
+- **Goal b (fmt_4_m1u):** Shared β€” same tuning applies to fmt=4 benchmark runs.
+
+## See also
+
+- [issue_379](issue_379.md), [pr_386](pr_386.md)
diff --git a/docs/issue_47.md b/docs/issue_47.md
new file mode 100644
index 00000000..9d3fec56
--- /dev/null
+++ b/docs/issue_47.md
@@ -0,0 +1,30 @@
+# issue_47: Benchmark: Apple M3 Max Β· macOS Β· 128 GB Β· internal SSD (12.5 GB/s) β€” 0.35–0.45 tok/s; PILOT neutral; confirms #8
+
+- **URL:** https://github.com/JustVugg/colibri/issues/47
+- **State:** CLOSED (opened 2026-07-11 by fabio-rovai, closed 2026-07-13)
+
+## Summary
+
+Pre-Metal CPU-era benchmark datapoint on Apple M3 Max (12P+4E), 128 GB unified, internal SSD measured at 12.49 GB/s O_DIRECT. Container: `jlnsrk/GLM-5.2-colibri-int4` (144 shards, 379 GB). Greedy `--temp 0`, `--ngen 400`, ~1k-token prompt.
+
+| config | tok/s | expert hit | RSS |
+|---|---|---|---|
+| baseline, `MTP=0` | 0.35 | 68.5% | 82.2 GB |
+| + `PIN PIN_GB=90` | 0.36 | 70.6% | 95.8 GB |
+| + `PILOT=1` | 0.36 (neutral) | 70.6% | 95.7 GB |
+| + `DRAFT=4` | 0.34 | 70.4% | 97.5 GB |
+| + `--topp 0.7` | **0.45 (+29%)** | 70.0% | 96.1 GB |
+
+- Baseline profile per 400 tok: expert-disk 400s, expert-matmul 311s, attention 351s, altro 89s β€” attention ~30% of token cost on CPU
+- `--topp 0.7` cut expert loads 28.7 -> 17.9/layer
+- Confirmed #8: int4 MTP head gives 0% acceptance (0/24) in all configs
+- Also reported a setup.sh libomp false-positive
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Historical baseline β€” the CPU-only world before Metal (#72). Documents the `--topp 0.7` lever and the MTP acceptance problem (#8) that motivated `MTP=0` in later configs.
+- **Goal b (fmt_4_m1u):** Not directly relevant.
+
+## See also
+
+- [issue_87](issue_87.md), [issue_103](issue_103.md), [pr_72](pr_72.md)
diff --git a/docs/issue_622.md b/docs/issue_622.md
new file mode 100644
index 00000000..e9d6ae4e
--- /dev/null
+++ b/docs/issue_622.md
@@ -0,0 +1,23 @@
+# issue_622: [Bug]: Metal prefill GEMM is not token-exact vs CPU on near-tie logits (S >= GEMM_MIN); decode unaffected
+
+- **URL:** https://github.com/JustVugg/colibri/issues/622
+- **State:** OPEN (opened 2026-07-25 by lBroth)
+- **Labels:** bug, metal
+
+## Summary
+
+On tiny fixture containers, teacher-forced prefill disagrees CPU vs Metal at 2 of 32 positions, deterministically, on stock `dev @ 7b03ce0` (M5 Pro, macOS 26.5.2). Greedy **decode is token-identical** on the same containers. Surfaced during the PR #587 hardware runs; filed separately since it reproduces on stock dev where that PR's code does not exist.
+
+- Reproduces on both a grouped int4 g64 container and an int3-experts/per-row-int4-dense container β€” **format-agnostic**
+- `COLI_METAL_GEMM_MIN=1000` (keep every `matmul_qt` GEMM on CPU; default threshold 16 rows) makes both containers bit-identical to the CPU reference β€” localizes the culprit to the Metal prefill GEMM at S >= 16 rows
+- Explains why decode stays exact: the harness prefills a 12-token prompt, below the 16-row threshold
+- Near-tie logits can flip token choice under different FP reduction order β€” benign for throughput, relevant for bit-exact CPU-vs-GPU validation
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** When validating the fmt=2 reproduction against CPU output, prefill (S >= GEMM_MIN) may diverge on near-ties β€” judge on coherence, not bit-identity, or set `COLI_METAL_GEMM_MIN=1000` for an exact A/B.
+- **Goal b (fmt_4_m1u):** Fix B modifies `coli_metal_gemm` / `bind_gemv` β€” the same path this bug lives in. PR #763 (GPU prefill attention) is gated off by default because of this family of divergence.
+
+## See also
+
+- [pr_763](pr_763.md), [pr_587](pr_587.md)
diff --git a/docs/issue_707.md b/docs/issue_707.md
new file mode 100644
index 00000000..530d9d25
--- /dev/null
+++ b/docs/issue_707.md
@@ -0,0 +1,23 @@
+# issue_707: [Bug]: OpenMP spin-wait tuning is inert on macOS β€” and applying it costs 2.2x decode on a 32 GB host
+
+- **URL:** https://github.com/JustVugg/colibri/issues/707
+- **State:** OPEN (opened 2026-07-30 by fredchu)
+- **Labels:** bug, help wanted, performance, metal
+
+## Summary
+
+The OpenMP hot-thread tuning block in `main()` never reaches the OpenMP runtime on macOS, because the self-re-exec that makes it effective is compiled only for Linux and FreeBSD. On the reporter's M1 Max 32 GB host that is **load-bearing in the good direction**: exporting the same four variables externally (`OMP_WAIT_POLICY=active KMP_BLOCKTIME=200 OMP_DYNAMIC=FALSE GOMP_SPINCOUNT=200000`) makes decode **2.2x slower**.
+
+- Item 1: the tuning is inert on macOS β€” and that is currently a good thing
+- Item 2: also about how the OpenMP team is configured on macOS (see issue body)
+- The report is less "fix the missing branch" and more "don't fix it without gating β€” here is the measurement"
+- Environment: macOS 26.5.2, M1 Max (8P+2E), 32 GB, `make colibri METAL=1`, model `mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp`
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Directly justifies `COLI_NO_OMP_TUNE=1` in benchmark Configs D/E (from the M5 Max report): active-spin costs GPU clocks on Apple's shared SoC power budget. M1 Ultra configs must keep OMP waits passive.
+- **Goal b (fmt_4_m1u):** Shared β€” same flag discipline for fmt=4 runs.
+
+## See also
+
+- [pr_116](pr_116.md) β€” the M5 Max report that identified the OMP-spin regression
diff --git a/docs/issue_87.md b/docs/issue_87.md
new file mode 100644
index 00000000..c41b617d
--- /dev/null
+++ b/docs/issue_87.md
@@ -0,0 +1,23 @@
+# issue_87: [Performance]: M5 MAX 128GB with METAL - 1.83 tok/s
+
+- **URL:** https://github.com/JustVugg/colibri/issues/87
+- **State:** CLOSED (opened 2026-07-12 by baha2046, closed 2026-07-13)
+
+## Summary
+
+First M5 Max Metal datapoint, using the pre-merge `codearranger/colibri` `metal-backend` branch (what became PR #72) with early Fable5 tuning: `DIRECT=1 IO_THREADS=8`, MTP off.
+
+- Hardware: MacBook M5 Max 128 GB, 2 TB
+- iobench on an expert shard: buffered 17.35 GB/s; O_DIRECT 64.68 GB/s (8 threads, 19 MB blocks)
+- Result: **1.83 tok/s** with `COLI_MODEL=... ./coli run "Compare the myths of Lucifer and Prometheus" --ram 96`
+- Log shows the auto macOS banner: "MTP ... su macOS il decode liscio e' piu' veloce (A/B misurato 1.68 vs 0.85 tok/s): draft OFF"
+- Same Lucifer/Prometheus prompt later used by #103 and the M5 Max report
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Part of the result lineage: 1.83 (#87) -> 2.06 (#103) -> 2.24 (PR #116 report). Establishes the prompt and the `DIRECT=1 IO_THREADS=8 MTP=0` lever set.
+- **Goal b (fmt_4_m1u):** Context only.
+
+## See also
+
+- [issue_103](issue_103.md), [pr_72](pr_72.md), [pr_116](pr_116.md)
diff --git a/docs/metal_dispatch_gap.md b/docs/metal_dispatch_gap.md
new file mode 100644
index 00000000..096470da
--- /dev/null
+++ b/docs/metal_dispatch_gap.md
@@ -0,0 +1,412 @@
+# Metal Dispatch Gap: fmt=4 Grouped Int4 Not Supported Anywhere in the Metal Backend
+
+> **Scope:** This document is a **problem statement** only. It identifies the fmt=4 Metal backend gap and provides evidence. Implementation plans are in:
+> - [`docs/metal_fmt2.md`](metal_fmt2.md) β€” fmt=2 (per-row int4) conversion and benchmark plan
+> - [`docs/metal_fmt4.md`](metal_fmt4.md) β€” fmt=4 (grouped int4) Metal backend fix plan
+
+---
+
+## Symptom
+
+M1 Ultra (128 GB unified memory) benchmark of GLM-5.2 744B MoE achieves **0.15 tok/s**
+despite `COLI_METAL=1` being set. From `c/3_with_pipe_no_omp_tune128ngen.log`:
+
+```
+METAL: blocchi GPU 0 | fallback CPU 0 | expert su GPU 0 | setup 0.00s gpu-wall 0.00s (kernel 0.00s) scatter 0.00s
+```
+
+Zero GPU blocks dispatched, zero fallback blocks, zero experts on GPU β€” **zero dispatch
+attempts** of any kind.
+
+**Second, equally important signal: there is no `METAL-ATTN:` line in the log at all.**
+That line is printed at `colibri.c:4968–4970` only when `aok > 0` (i.e. only when the fused
+attention/layer kernel actually ran at least once). Its absence proves the fused attention
+path never engaged either. Attention ran entirely on the CPU.
+
+Decode profile from the same log (128 tokens, 877.98s, 127 forwards):
+
+| phase | total | per token | where it ran |
+|---|---|---|---|
+| expert-matmul | 466.662s | 3.67s | **CPU** (should be GPU) |
+| attention | 186.485s | 1.47s | **CPU** (should be GPU) |
+| expert-disk wait | 154.126s | 1.21s | CPU/disk (correct) |
+| other | 70.708s | 0.56s | CPU |
+
+For comparison, the M5 Max report (`docs/METAL-M5MAX-PERF-REPORT.md`) achieves **2.24 tok/s**
+(Config E). **Critical context:** that benchmark was run on **fmt=2 (per-row int4)** weights,
+NOT the fmt=4 grouped int4 container used here. The M5 Max report explicitly shows
+`fallback CPU 0` and non-zero `attn GPU kernel` times (79s in Config E), proving Metal was
+active. Issue #585 documents the exact symptom on fmt=4: **2.39 tok/s on fmt=2 vs 0.81 tok/s
+on fmt=4** (3Γ— slowdown) on the same M5 Max hardware.
+
+**Bottom line:** The 2.24 tok/s M5 Max result is valid Metal performance, but only for fmt=2
+models. The fmt=4 grouped int4 GLM-5.2 container (gs=64) has **no Metal support anywhere** in
+the current backend β€” this is why M1 Ultra shows zero Metal dispatch despite `COLI_METAL=1`.
+
+## Issue #103 Evidence β€” How the M5 Max 2.06–2.24 tok/s Benchmark Was Achieved
+
+Direct inspection of the original benchmark report ([issue_103.md](issue_103.md), Issue #103) reveals the M5 Max
+result was achieved on a **fmt=2 (per-row int4)** model, NOT the official GLM-5.2 fmt=4 container:
+
+- **Model path**: `/Users/doug/glm52_i4` β€” a custom/local conversion, not the official
+ `GLM-5.2-colibri-int4-g64` container
+- **Banner confirms fmt=2**: The log shows `expert@8-bit densa@8-bit` without the
+ "cosmetically wrong" mismatch that appears for fmt=4 models (see line 82–88 below)
+- **Metal fully engaged**: `fallback CPU 0`, `METAL-ATTN: layer GPU 79794`, and
+ `attn GPU kernel 76.36s` prove all three Metal paths were active
+
+**Conclusion**: To reproduce the 2.06–2.24 tok/s results, you must either:
+1. Convert the model with **fmt=2 (per-row int4)** settings instead of fmt=4, OR
+2. Apply the Metal backend fixes (A+B+C) to support fmt=4 grouped int4
+
+The official GLM-5.2 colibri container uses fmt=4 by design β€” this is not a conversion
+error, but a format choice that currently lacks Metal support.
+
+## Verified: the model container is CORRECT β€” conversion did NOT fail
+
+This was checked directly against the safetensors headers in
+`/Users/joe/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp/` (3551 tensors scanned).
+
+For every quantized tensor, the `.qs` scale count is **exactly `O Γ— ceil(I/64)` floats**,
+which is the definition of grouped int4 at gs=64. Examples (layer 0 / layer 77, identical):
+
+| tensor | weight bytes (U8) | `.qs` floats | implies |
+|---|---|---|---|
+| `self_attn.q_a_proj` | 6,291,456 | 196,608 | OΒ·I=12,582,912 β†’ OΒ·I/64 βœ“ fmt=4 g64 |
+| `self_attn.q_b_proj` | 16,777,216 | 524,288 | OΒ·I=33,554,432 β†’ /64 βœ“ fmt=4 g64 |
+| `self_attn.kv_a_proj_with_mqa` | 1,769,472 | 55,296 | OΒ·I=3,538,944 β†’ /64 βœ“ fmt=4 g64 |
+| `self_attn.kv_b_proj` | 7,340,032 | **229,376** | 28,672 rows Γ— **8 groups** βœ“ fmt=4 g64 |
+| `self_attn.o_proj` | 50,331,648 | 1,572,864 | OΒ·I=100,663,296 β†’ /64 βœ“ fmt=4 g64 |
+| `mlp.experts.N.{gate,up,down}_proj` | 6,291,456 | 196,608 | 2048Γ—6144 β†’ /64 βœ“ fmt=4 g64 |
+| `mlp.shared_experts.*` | 6,291,456 | 196,608 | βœ“ fmt=4 g64 |
+| `mlp.gate.weight` (router) | F32 [256,6144] | β€” | f32, not quantized |
+
+Config confirms the shapes: `hidden_size=6144`, `moe_intermediate_size=2048`,
+`n_routed_experts=256`, `num_experts_per_tok=8`, `n_shared_experts=1`,
+`num_hidden_layers=78`, `first_k_dense_replace=3`, `q_lora_rank=2048`, `kv_lora_rank=512`,
+`qk_nope_head_dim=192`, `qk_rope_head_dim=64`, `v_head_dim=256`, `num_attention_heads=64`.
+
+MTP head: `out-mtp-00000.safetensors` is 9,959,321,520 bytes β€” matches the sum of the three
+int8 sizes the top-level `README.md` documents as correct (3,527,131,672 + 5,366,238,584 +
+1,065,950,496 = 9,959,320,752, the ~768-byte delta being one merged safetensors header
+instead of three). This is the int8 MTP container, not the broken int4 one.
+
+Plus the strongest evidence of all: **the run produces coherent, well-formed output.** The
+CPU `matmul_i4_grouped` path is decoding these weights correctly.
+
+**Conclusion: this is a backend gap, not a bad conversion. Do not re-download or re-convert
+the model.**
+
+**Note on the banner as a format diagnostic:** the log line
+`== GLM C engine (glm_moe_dsa), cache=8 experts/layer | experts@8-bit densa@8-bit | idot: neon ==`
+says "8-bit" for both experts and dense. For fmt=4 models, this is cosmetically wrong:
+`qt_from_disk` (`colibri.c:1055`) sees a `name.qs` sidecar and uses the container's
+pre-quantized fmt=4 data, ignoring the bits setting. The banner correctly reflects fmt=2
+(per-row int4) models (as seen in Issue #103), but misrepresents fmt=4 grouped int4.
+This banner can serve as a quick visual check: if it says "8-bit" but you're loading a
+grouped-int4 container, the format mismatch is confirmed.
+
+## Root Cause: fmt=4 is rejected at every Metal entry point
+
+GLM-5.2 stores **everything** quantized as `fmt=4` grouped int4 with `gs=64`.
+`qt_resolve_fmt` (`colibri.c:1027`) resolves the container: it first classifies the weight
+bytes as int4 (`fmt=2`), then `detect_group_size` (`colibri.c:1005`) sees
+`ns == O*ng*4` and promotes it to `fmt=4, gs=64` (`colibri.c:1043`). Every path below then
+sees `fmt=4`.
+
+### Gate 1 β€” MoE experts (`moe_submit`, `backend_metal.mm:963`)
+
+```cpp
+if (!g_dev || (fmt != 1 && fmt != 2)) return nil; // <-- rejects fmt=4
+```
+
+`mfmt` is set from the first collected expert at `colibri.c:3210` (`if(mfmt<0) mfmt=e->g.fmt;`)
+= 4, then passed to `coli_metal_moe_block_begin` (`colibri.c:3242`) β†’
+`moe_submit` (`backend_metal.mm:1079`) β†’ immediate `nil`.
+
+This returns **before** the `g_moe_fb++` fallback counter at `backend_metal.mm:973–978`,
+which is exactly why all three counters read zero instead of showing fallbacks:
+
+- `blocchi GPU 0` = `g_moe_ok == 0`
+- `fallback CPU 0` = `g_moe_fb == 0` ← would be non-zero if the gate were passed and `resolve()` failed
+- `expert su GPU 0` = `g_moe_experts == 0`
+
+With `mh == NULL`, `colibri.c:3245` leaves `cpu_res=1`, the missed-expert submit at
+`colibri.c:3299–3301` hits the same gate, so `metal_done = 0` at `colibri.c:3308` and the
+CPU expert loop runs everything.
+
+The `moe_gemv` kernel itself (`backend_metal.mm:70–99`) has no fmt=4 case: it handles
+`fmt==2` (per-row int4) and `else` (int8), and applies a single per-row scale *after* the
+simd reduction (`yout[row]=acc*sc[o]`, line 98) β€” structurally incompatible with a scale
+that varies along the input dimension.
+
+### Gate 2 β€” prefill GEMM (`coli_metal_gemm`, `backend_metal.mm:860`)
+
+```cpp
+if (!g_dev || (fmt!=1 && fmt!=2)) return 0;
+```
+
+plus the caller-side gate in `matmul_qt_ex` (`colibri.c:575`):
+
+```c
+if(g_metal_enabled && S>=g_metal_gemm_min && !spec_pinned() && (w->fmt==1||w->fmt==2) && !omp_in_parallel()){
+```
+
+So every batched (prefill) projection β€” `q_a`, `q_b`, `kv_a`, `kv_b`, `o_proj`, and the
+dense-layer MLPs of layers 0–2 β€” skips Metal and runs `matmul_i4_grouped` on the CPU
+(`colibri.c:591`).
+
+### Gate 3 β€” fused decode attention (`colibri.c:2392` and `colibri.c:4231`)
+
+Both fused paths require `l->kv_b.fmt==2`:
+
+```c
+&& c->qk_rope==64 && vh==256 && l->kv_b.fmt==2){ // attention_rows, colibri.c:2392
+&& c->qk_nope==192 && c->qk_rope==64 && c->v_head==256 && l->kv_b.fmt==2 // layer_forward, colibri.c:4231
+```
+
+`kv_b` is fmt=4 (229,376 scales = 28,672 rows Γ— 8 groups), so **both gates fail on every
+layer of every token** β†’ `coli_metal_attn_decode` / `coli_metal_layer_decode` are never
+called β†’ no `METAL-ATTN:` line β†’ 186s of decode attention on the CPU.
+
+Note this gate is currently *load-bearing for correctness*, not just performance: the
+absorb-core dequant helper is per-row only (`backend_metal.mm:134–135`):
+
+```cpp
+inline float a_deqrow(device const uchar* base, int row, int i, device const float* sc){
+ device const uchar* w=base+(long)row*((A_KVL+1)/2); uchar b=w[i>>1];
+ int val=(i&1)?(b>>4):(b&0xF); return float(val-8)*sc[row]; } // <-- sc[row], per-row only
+```
+
+If the gate were simply relaxed to admit fmt=4 without fixing `a_deqrow`, `a_qabs`
+(`backend_metal.mm:136`) and `a_ctx` (`backend_metal.mm:168`) would read `sc[row]` out of a
+grouped scale array and produce silently wrong attention. **The gate must be relaxed and
+`a_deqrow` made group-aware in the same change.**
+
+### Latent hazard β€” `bind_gemv` does not validate fmt (`backend_metal.mm:637`)
+
+`bind_gemv` forwards whatever `fmt` it is handed straight into `mm_gemv` without checking it.
+`mm_gemv` (`backend_metal.mm:16–64`) handles fmt 1/2/3 and treats **everything else as f32**
+(`else` branch, line 56). Today nothing reaches it with fmt=4 because Gate 3 blocks the whole
+attention path on `kv_b` alone β€” but the other five projections' fmts are passed through
+*unchecked*. A model with fmt=2 `kv_b` and fmt=4 `o_proj` would pass the gate and then read
+packed int4 bytes as f32 floats: garbage output, no error, no fallback. This should be
+hardened (`return false` on unsupported fmt) as part of the fix regardless of the fmt=4 work.
+
+## Why fmt=4 Is Special β€” Scale Layout
+
+- **fmt=1 (int8):** one scale per output row β†’ `scale[o]`.
+- **fmt=2 (int4 per-row):** one scale per output row β†’ `scale[o]`.
+- **fmt=4 (grouped int4):** `ng = ceil(I / gs)` scales **per output row**, `gs=64` here.
+
+The authoritative layout is the CPU reference `matmul_i4_grouped` (`quant.h:168–202`):
+
+```c
+int rb=(I+1)/2; int ng=(I+gs-1)/gs; // ng is over the INPUT dim I
+const uint8_t *w = q4 + (int64_t)o*rb;
+const float *scl= scale + (int64_t)o*ng; // per output row
+for(int g=0; g*gs<I; g++){ float sc=scl[g]; ... a += (partial dot over group g) * sc; }
+```
+
+and confirmed by CUDA (`backend_cuda.cu:195–205`, `absorb_scale` at `:163–168`):
+
+```cuda
+const float *scl = scales + (size_t)o * ng;
+int g = i / gs; if (g >= ng) g = ng - 1; // clamp tail of last partial group
+sum += xs[i] * weight_at(weights, fmt, row, i) * scl[g];
+```
+
+So for a GEMV with input dim `K`: **`ng = ceil(K/gs)`**, and the scale for output row `o`,
+input element `i` is **`sc[o*ng + i/gs]`**, where `sc` is already the per-expert/per-tensor
+base pointer. Per-gemv `ng` values in this model: gate/up `K=6144 β†’ ng=96`;
+down `K=2048 β†’ ng=32`; kv_b absorb `K=512 β†’ ng=8`.
+
+Nibble decode convention: Metal's existing fmt=2 code uses **offset binary**
+(`float(int(b&0xF)-8)`, `backend_metal.mm:88`), same as the CPU (`quant.h:195`). fmt=4 uses
+the identical nibble encoding β€” **only the scale lookup changes.** (CUDA looks different
+because it pre-XORs weights with `0x88` at upload β€” `backend_cuda.cu:170`,`:778` β€” and then
+sign-extends; net effect is the same values. Metal needs no such conversion.)
+
+Alignment luck worth exploiting: `detect_group_size` (`colibri.c:1010`) only ever returns
+`{16,32,48,64,96,128,192,256}` β€” **all multiples of 8**. The Metal gemv kernels load 8
+weights per iteration (`uchar4` β†’ 8 nibbles), so **an 8-element load never straddles a group
+boundary**, and the group scale can be applied once per 8-load instead of per element:
+`g = (c*8)/gs`. The strided lane loop (`c += 32`, i.e. 256 elements) also preserves this.
+
+## GitHub Issue Evidence β€” This Is a Known, Active Gap
+
+Direct inspection of the issue tracker reveals this is not a newly-discovered bug but a
+**documented, actively-worked gap** in Metal fmt=4 support. The following issues provide
+context, precedent, and partial implementations:
+
+### Active Metal fmt=4 Work
+
+**#585** β€” "[Feature]: METAL Attention support for fmt:4" (OPEN)
+- Documents the exact symptom: `PROF=1 COLI_METAL=1` achieves **2.39 tok/s on fmt=2** vs
+ **0.81 tok/s on fmt=4** (3Γ— slowdown) on M5 Max
+- Root cause identified: attention path gated to `l->kv_b.fmt==2` only
+- Proposed solution matches Fix C in this document
+
+**#587** β€” "Metal fmt=4 grouped-int4 decode: attention + routed experts" (OPEN, needs-rebase)
+- PR implementing the fix for #585
+- **Verified performance gain**: 0.81 β†’ **2.13 tok/s** after fixing both attention and
+ expert paths on a real GLM-5.2 744B checkpoint
+- **Scope limitation**: prefill GEMM path (`coli_metal_gemm`) remains unaddressed β€”
+ this is a decode-only fix
+- Test plan includes kernel unit tests vs CPU oracle for both `moe_gemv` and attention paths
+
+**#457** β€” "metal: grouped-int4 (fmt=4) GEMV support" (MERGED 2026-07-30)
+- Fixed the `mm_gemv` kernel for fmt=4 (dense projections, attention `q_a/q_b/kv_a/o`)
+- Added proper `gs` parameter threading through `AttnW`, `coli_metal_attn_decode`,
+ `coli_metal_layer_decode`, and `bind_gemv`
+- **Critical discovery**: traced that `qt_from_disk` allocated fmt=4 scale buffers with
+ `falloc` (bare `malloc`, never registered), which **accidentally masked the shader bug**
+ β€” the scale buffer never resolved to GPU, so the broken shader code never executed
+- Added `fmt_scale_bytes()` to correctly size fmt=4 scale buffers (`O*ng` floats)
+- **Test coverage**: 12 new fmt=4 cases in `metal-test`, including grouped `coli_metal_gemm`
+ and fused-decode-path (`run_attn_grouped`) with `qraw` metric to detect uniform-scale bugs
+
+### CUDA Precedent β€” Same Bugs Hit First
+
+**#298** β€” "cuda+engine: full fmt=4 support" (MERGED)
+- CUDA's complete fmt=4 implementation across all paths
+- **1.08 tok/s** on RTX 5070 Ti after fix (vs broken 0.05 tok/s before)
+- Established the scale layout convention: `scale[o*ng + g]` with tail clamp `g >= ng β†’ ng-1`
+- Added fused `matmul_i4_grouped_pair` for gate+up (expert path optimization)
+
+**#334** β€” "fmt=4 CUDA attention crash" (CLOSED)
+- **Hard crash** (GPU memory fault β†’ system reboot) when attention absorb kernels used
+ per-row scales on grouped data
+- Root cause: `attention_absorb_kernel` indexed scales as `wscale[row]` instead of
+ `wscale[row*ng + k/gs]` for fmt=4
+- **Prevention note**: "When adding a new quant format or editing a GPU kernel: any kernel
+ that multiplies a quantized weight by a scale must index the scale per-group for fmt=4"
+
+**#464** β€” "cuda: reject fmt=4 at per-row-only entry points" (MERGED)
+- Hardens CUDA to explicitly reject fmt=4 at unsafe entry points (`coli_cuda_matmul`,
+ `coli_cuda_expert_mlp`)
+- Ensures fallback to CPU rather than silent garbage output
+- Precedent for the `bind_gemv` hardening recommended in this document
+
+### Key Insight: Accidental Safety via Allocator Bug
+
+Issue #457's investigation revealed a **critical interaction** that explains why the
+system didn't produce garbage output despite the shader bugs:
+
+1. `qt_from_disk` allocated fmt=4 scale buffers with `falloc` (not `qalloc`)
+2. `falloc` does not page-align and never calls `coli_metal_register`
+3. `resolve()` in `backend_metal.mm` only finds pointers inside registered slabs
+4. Therefore `resolve(t->s, ...)` **always fails** for fmt=4 scale buffers
+5. `bind_gemv`/`coli_metal_gemm` return `false`/`0` β†’ **CPU fallback**
+
+**Consequence**: The allocator bug kept the broken shader code from ever executing.
+Fixing the allocator (as #457 does) without fixing the shader would have **armed the bug**
+β€” scale buffers would resolve, but the shader would read them incorrectly.
+
+This is why #457's commit order matters:
+- **First**: `f0fa5a9` β€” shader fmt=4 branch + host plumbing (capability lands **inert**)
+- **Second**: `2ba12d8` β€” allocator fix (activates the already-correct capability)
+- **Third**: `fc0f5a5` β€” attn-test blind-spot fix (strengthens test coverage)
+
+### Current Status Summary
+
+| Path | Status | Notes |
+|---|---|---|
+| `mm_gemv` (dense projections) | βœ… Fixed (#457) | Merged 2026-07-30, includes `gs` parameter and fmt=4 branch |
+| `moe_gemv` (routed experts) | 🟑 In Progress (#587) | PR open, verified 0.81β†’2.13 tok/s gain on decode |
+| Fused attention (`a_qabs`/`a_ctx`) | 🟑 In Progress (#587) | PR open, requires `a_deqrow` to be group-aware |
+| Prefill GEMM (`coli_metal_gemm`) | ❌ Not Addressed | Gate still rejects fmt=4, out of scope of #587 |
+| `bind_gemv` return value wiring | ❌ Not Fixed | `encode_attention` ignores return at `:683,684,686,699` |
+
+**Net effect**: Even with #587 merged, the M1 Ultra run in this document would still
+experience CPU fallback on prefill (layers 0–2 dense MLPs, batched attention projections).
+The 0.15 tok/s β†’ ~0.7–0.9 tok/s improvement requires **all three paths** (A+B+C) to be fixed.
+
+## Impact β€” Recalibrated, Honest
+
+Per-token decode budget today: 3.67s expert-matmul + 1.47s attention + 1.21s disk wait +
+0.56s other β‰ˆ 6.9s β†’ 0.15 tok/s.
+
+- **Fix A only (MoE experts on GPU):** removes ~3.67s/token of CPU matmul, but attention
+ (1.47s) stays on CPU β†’ roughly **~0.3 tok/s**. This is why the original MoE-only scope was
+ insufficient.
+- **Fixes A+B+C (all three Metal paths):** CPU keeps only disk streaming, routing and
+ scatter; GPU compute is small (expert gemv is bandwidth-bound: ~11.3 GB/token of expert
+ weights at unified-memory speeds β‰ˆ tens of ms) β†’ the wall clock should collapse toward the
+ **~1.2s/token disk-wait floor β‰ˆ 0.7–0.9 tok/s**.
+- **Beating 2.06–2.24 tok/s additionally requires cutting disk traffic**, not more GPU: the
+ run shows 69.2% expert hit rate (pin 7.4% + LRU 61.9%) with RSS 90.96 GB against a 110 GB
+ budget on a 128 GB machine. Levers, in `ultra_benchmark_plan.md` Phase 5 order: higher
+ `--ram` (115/120) for more LRU/pin residency, more `PIPE_WORKERS` (disk service 3.8s/token
+ vs 1.21s/token blocked suggests imperfect I/O parallelism), and β€” since this box has a ZFS
+ pool alongside the internal SSD β€” a `COLI_MODEL_MIRROR` second copy for aggregate read
+ bandwidth (see top-level `README.md`, "Dual-SSD").
+- **Fix D (NEON)** does not change the Metal-on path materially; it raises the floor for
+ fallbacks and for the dense/indexer matmuls that never go to GPU. Expected speedup:
+ 3–5Γ— over the scalar implementation, based on `matmul_i4`'s existing NEON body.
+
+**Risk note:** all of the above assumes expert slabs resolve to registered Metal buffers
+once the format gate passes. Slabs *are* registered (`coli_metal_register` at
+`colibri.c:1364`, `:1581`, `:1609`, `:1795`, `:1809`), but this has never actually executed on
+this machine, so `g_moe_fb` (the `resolve()` failure counter) may well be non-zero on the
+first successful build. If it is, the next thing to investigate is slab registration
+lifetime, not the kernels.
+
+## Implementation Plans
+
+**This document is a problem statement only.** Implementation plans are in:
+
+- [`docs/metal_fmt2.md`](metal_fmt2.md) β€” fmt=2 (per-row int4) conversion and benchmark plan
+- [`docs/metal_fmt4.md`](metal_fmt4.md) β€” fmt=4 (grouped int4) Metal backend fix plan
+
+**Files involved** (for reference in implementation plans):
+
+**Metal backend β€” `c/backend_metal.mm`**
+- `:16–64` `mm_gemv` kernel β€” needs fmt=4 branch + `gs`; final `else` must become f32-only
+- `:70–99` `moe_gemv` kernel β€” needs fmt=4 branch + `gs`; `yout=acc*sc[o]` (`:98`) must be skipped for fmt=4
+- `:134–135` `a_deqrow` β€” per-row `sc[row]`, needs grouped lookup
+- `:136`,`:168` `a_qabs`, `a_ctx` β€” callers of `a_deqrow`, need `fmt`/`gs` args
+- `:590–593` `coli_metal_matmul` β€” gate `fmt>3`; also wraps scales as `O` floats (wrong size for fmt=4)
+- `:637–651` `bind_gemv` β€” no fmt validation (latent garbage hazard), needs `gs`
+- `:665–701` `encode_attention` β€” ignores `bind_gemv` return values; needs `gs` plumbing
+- `:716`,`:753` `coli_metal_attn_decode`, `coli_metal_layer_decode` β€” need `gs` params
+- `:858–860` `coli_metal_gemm` β€” gate rejects fmt=4
+- `:963` `moe_submit` β€” the primary gate; `:973–978` fallback counters; `:1000–1007` gemv dispatch lambda
+- `:1041`,`:1067` `coli_metal_moe_block`, `coli_metal_moe_block_begin` β€” need `gs` params
+
+**Metal header β€” `c/backend_metal.h`**
+- `:36` `coli_metal_matmul`, `:69` `coli_metal_layer_decode`, `:85` `coli_metal_gemm`,
+ `:106` `coli_metal_attn_decode`, `:138`/`:153` `coli_metal_moe_block`/`_begin` β€” signatures
+
+**Engine β€” `c/colibri.c`**
+- `:575–577` `matmul_qt_ex` Metal GEMM gate β€” excludes fmt=4
+- `:591` CPU fmt=4 dispatch to `matmul_i4_grouped` (the path everything currently takes)
+- `:593` IDOT gate β€” fmt=4 excluded by design
+- `:1005` `detect_group_size` (candidates all multiples of 16), `:1027` `qt_resolve_fmt`, `:1043` fmt=2β†’4 promotion, `:1067` `t->gs` assignment
+- `:2390–2412` `attention_rows` fused-attention gate (`kv_b.fmt==2` at `:2392`)
+- `:3203–3234` `MB_BUILD` macro β€” `mfmt` at `:3210`, needs `mgs` + uniformity validation
+- `:3242`,`:3301` MoE submit call sites β€” need `gs` argument
+- `:3308` `metal_done` resolution
+- `:4228–4279` `layer_forward` fused-layer gate (`kv_b.fmt==2` at `:4231`)
+- `:4966–4974` Metal stats printing (`METAL-ATTN` only when `aok>0` β€” the missing-line clue)
+
+**CPU kernels β€” `c/quant.h`**
+- `:168–202` `matmul_i4_grouped` β€” AVX2 only, **no NEON** (Fix D)
+- `:145–157` `matmul_i4` NEON body β€” the template to copy
+- `:228–239` `matmul_i4_pair` NEON body β€” second reference
+- `:513` `matmul_i4_grouped_pair` β€” check whether the fused gate+up grouped variant also lacks NEON (it is used at `colibri.c:279–280` when `S==1`, i.e. on the decode path)
+
+**CUDA reference β€” `c/backend_cuda.cu`**
+- `:101–103` `row_bytes`, `:145–156` `weight_at`, `:163–168` `absorb_scale`,
+ `:195–205`/`:219` `quant_matmul`, `:844`/`:868–870` upload-time gs validation
+
+**Tests**
+- `c/tests/test_backend_metal.mm` (`:44` matmul, `:90` moe_block, `:163` attn_decode)
+- `c/tests/test_gemm_largebatch.mm` (`:62`)
+- `c/tests/test_i4_grouped.c` (CPU grouped reference, incl. the fused-pair variant)
+- `c/tests/test_grouped_g4_cuda.cu` (CUDA fmt=4 semantics test β€” useful cross-check)
+
+**Logs / plans**
+- `c/3_with_pipe_no_omp_tune128ngen.log` (the analyzed run), `c/baseline.log`, `c/2_with_pipe_no_omp_tune.log`
+- `ultra_benchmark_plan.md` (benchmark execution plan; Phase 1 complete but **blocked** on
+ fmt=4 Metal support β€” see this document for the root cause)
diff --git a/docs/metal_fmt2.md b/docs/metal_fmt2.md
new file mode 100644
index 00000000..a97c1ba9
--- /dev/null
+++ b/docs/metal_fmt2.md
@@ -0,0 +1,224 @@
+# Metal fmt=2 Benchmark Plan β€” M1 Ultra Per-Row Int4
+
+## Objective
+
+Convert the official GLM-5.2 fmt=4 (grouped int4) container to fmt=2 (per-row int4) and benchmark on M1 Ultra to establish a speed baseline with full Metal support.
+
+**Rationale:** The M5 Max 2.24 tok/s record was achieved on fmt=2 weights. This plan measures what M1 Ultra's 80 GPU cores can achieve with the same format.
+
+---
+
+## Prerequisites
+
+- fmt=4 model already present: `~/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp`
+- Colibri built with Metal: `make clean && make METAL=1`
+- Metal backend verified working: `make metal-test` passes
+
+---
+
+## Phase 1: fmt=2 Conversion
+
+### 1.1 Source Model Format
+
+**Important:** The source model `GLM-5.2-colibri-int4-g64-with-int8-mtp` is **already int4** (fmt=4 grouped, group_size=64), NOT FP8. The pathname `int4-g64` confirms this.
+
+**No existing converter:** There is NO tool to convert fmt=4 β†’ fmt=2 directly. The `coli convert` command only converts FP8β†’int4.
+
+### 1.2 Required: fmt=4 β†’ fmt=2 Converter
+
+A new converter script must be created to:
+1. Read existing packed int4 weights (unchanged between formats)
+2. Read grouped scales from `.qs` files
+3. Dequantize to approximate FP32
+4. Recompute per-row scales
+5. Write new `.qs` files with per-row scaling
+
+**Size analysis:**
+- Source (fmt=4 grouped): 406 GB
+- Target (fmt=2 per-row): ~372 GB
+- Savings: ~34 GB (from reduced scale file sizes)
+
+**Workflow (disk-safe):**
+```bash
+# Option A: Convert in-place using fmt4_to_fmt2.py (recommended)
+# 1. Create converter script (see c/tools/convert_fmt4_to_fmt2.py)
+# 2. Run conversion from NFS source to SSD destination
+python3 c/tools/convert_fmt4_to_fmt2.py \
+ --indir /mnt/zfs1/noprot/mlx-lm/models/GLM-5.2-colibri-int4-g64-with-int8-mtp \
+ --outdir ~/mlx-models/GLM-5.2-colibri-int4-perrow
+
+# Option B: Delete fmt=4 from SSD first, then convert
+rm -rf ~/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp
+# Then run converter above
+```
+
+**Output:** New directory `~/mlx-models/GLM-5.2-colibri-int4-perrow` (~372 GB)
+
+### 1.2 Verify Conversion
+
+Check the banner on a short run:
+```bash
+COLI_METAL=1 ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow \
+ --ngen 16 'test'
+```
+
+**Expected banner:** `expert@8-bit densa@8-bit` (no "cosmetic mismatch" warning)
+
+**Expected Metal lines:**
+```
+METAL: blocchi GPU N | fallback CPU 0 | expert su GPU E
+METAL-ATTN: layer GPU N
+```
+
+---
+
+## Phase 2: Baseline Measurement
+
+### 2.1 Config A β€” Metal Baseline
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=0 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+**Record:**
+- tok/s
+- wall time (s)
+- expert-disk (s)
+- expert-matmul (s)
+- attention (s)
+- attn GPU kernel (s)
+- expert GPU overhead (s)
+- expert hit rate (%)
+- RSS (GB)
+
+### 2.2 Config B β€” Default (OMP Tuning)
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=0 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+**Check:** If OMP active-spin is present, expect ~39% regression (M5 Max experience)
+
+---
+
+## Phase 3: Tuning Path
+
+### 3.1 Add PIPE (Config C)
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=0 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+### 3.2 Kill OMP Spin (Config D)
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+### 3.3 Combined Optimal (Config E)
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+**Target:** β‰₯2.24 tok/s (match or beat M5 Max by leveraging M1 Ultra's 80 GPU cores)
+
+---
+
+## Phase 4: M1 Ultra Specific Tests
+
+### 4.1 Vary PIPE_WORKERS
+
+```bash
+# Test higher worker counts for M1 Ultra's 32 CPU cores
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=12 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=16 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+### 4.2 Test MTP
+
+```bash
+# M5 Max had MTP=0; test if MTP=1 helps on M1 Ultra
+COLI_METAL=1 DIRECT=1 MTP=1 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 110 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+### 4.3 Memory Pressure Test
+
+```bash
+# Test --ram 115, --ram 120 to find M1 Ultra's compression knee
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 115 \
+ "Compare the myths of Lucifer and Prometheus"
+
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ~/mlx-models/GLM-5.2-colibri-int4-perrow --ram 120 \
+ "Compare the myths of Lucifer and Prometheus"
+```
+
+---
+
+## Phase 5: Documentation
+
+### 5.1 Create METAL-M1ULTRA-FMT2-REPORT.md
+
+Mirror the M5 Max report structure:
+- Setup section (hardware, model, workload, flags)
+- Results table (configs A-E + M1-specific variants)
+- Phase-by-phase analysis
+- Comparison with M5 Max fmt=2 results
+- Recommendations
+- Caveats/untested levers
+
+### 5.2 Update benchmarks.md
+
+Add row to community benchmarks table:
+```markdown
+| Apple M1 Ultra (32 cores) Β· macOS Β· 128 GB unified Β· internal SSD | ~5-6 GB/s cold | Metal on (fmt=2) Β· `--ram 110` Β· COLI_NO_OMP_TUNE=1 Β· PIPE=1 | **X.XX tok/s** Β· hit YY% Β· [PR #NNN] |
+```
+
+---
+
+## Success Criteria
+
+1. **Primary:** Achieve β‰₯2.06 tok/s (match M5 Max baseline)
+2. **Stretch:** Achieve β‰₯2.24 tok/s (beat M5 Max)
+3. **Documentation:** Complete METAL-M1ULTRA-FMT2-REPORT.md with full methodology
+4. **PR:** Submit report + benchmarks.md update
+
+---
+
+## Notes
+
+- **Workload:** Use identical prompt ("Compare the myths of Lucifer and Prometheus") and 1024 tokens
+- **Consistency:** Run each config multiple times, report best/warm-cache numbers
+- **RSS:** Track RSS throughout, note when compression kicks in
+- **Expert hit rate:** Record hit rate for each config
+- **Thermals:** Monitor if M1 Ultra throttles during long runs (1024 tokens @ ~2 tok/s = ~8-10 min)
+- **Quality caveat:** fmt=2 (per-row) has ~9pp lower quality than fmt=4 (grouped scales) β€” this is expected and documented in quant_ablation.py
+- **Source format:** The source model is fmt=4 (grouped int4, group_size=64), NOT FP8. Conversion requires a fmt=4β†’fmt=2 converter, not `coli convert`.
+- **Disk space:** fmt=4 source is 406 GB; fmt=2 output is ~372 GB. Delete fmt=4 from SSD before conversion to avoid 778 GB peak requirement.
+
+---
+
+## References
+
+- [M5 Max Performance Report](METAL-M5MAX-PERF-REPORT.md)
+- [Metal Dispatch Gap](metal_dispatch_gap.md) β€” fmt=4 backend gap context
+- [ultra_benchmark_plan.md](ultra_benchmark_plan.md) β€” overall M1 Ultra benchmark plan
diff --git a/docs/metal_issues.txt b/docs/metal_issues.txt
new file mode 100644
index 00000000..3ff3c466
--- /dev/null
+++ b/docs/metal_issues.txt
@@ -0,0 +1,29 @@
+# contributing:
+https://github.com/JustVugg/colibri/blob/main/CONTRIBUTING.md
+
+# open issues:
+https://github.com/JustVugg/colibri/issues/379 # Metal β€” growing the expert cache makes decode SLOWER
+https://github.com/JustVugg/colibri/issues/387 # Datapoint: M5 Max 128 GB β€” decode-tuning A/Bs (MTP Γ— cache 2Γ—2): 0.4 β†’ 2.0 tok/s at identical output
+https://github.com/JustVugg/colibri/issues/585 # [Feature]: METAL Attention support for fmt:4
+https://github.com/JustVugg/colibri/issues/622 # [Bug]: Metal prefill GEMM is not token-exact vs CPU on near-tie logits (S >= GEMM_MIN); decode unaffected
+https://github.com/JustVugg/colibri/issues/707 # [Bug]: OpenMP spin-wait tuning is inert on macOS β€” and applying it costs 2.2x decode on a 32 GB host
+
+# Closed issues:
+https://github.com/JustVugg/colibri/issues/47 # Benchmark: Apple M3 Max Β· macOS Β· 128 GB Β· internal SSD (12.5 GB/s) β€” 0.35–0.45 tok/s; PILOT neutral; confirms #8; setup.sh libomp false-positive
+https://github.com/JustVugg/colibri/issues/87 # [Performance]: M5 MAX 128GB with METAL - 1.83 tok/s
+https://github.com/JustVugg/colibri/issues/103 # [Performance]: tok/s of 2.06 tok/s !!!
+https://github.com/JustVugg/colibri/issues/180 # Datapoint: M3 Ultra (Mac Studio, 256 GB, 80-core GPU) β€” 1.45 tok/s @ 82% hit, Metal + tuned flags
+https://github.com/JustVugg/colibri/issues/107 # Benchmark: Mac Mini M4 Pro (48 GB) β€” 0.30 tok/s with Metal backend (PR #72), 0.18 CPU-only
+
+# Merged PRs:
+https://github.com/JustVugg/colibri/pull/72 # Metal backend (Apple Silicon): experts + fused attention + full-layer GPU, token-exact - #72
+https://github.com/JustVugg/colibri/pull/116 # docs: Metal backend (#72) β€” M5 Max performance report, OMP-spin regression + fix
+https://github.com/JustVugg/colibri/pull/386 # Metal: platform-aware expert-cache defaults via honest storage probe
+https://github.com/JustVugg/colibri/pull/750 # inkling: audio input, Metal expert MoE, and tuned Apple defaults
+
+# Open PRs:
+https://github.com/JustVugg/colibri/pull/587 # Metal fmt=4 grouped-int4 decode: attention + routed experts (#585)
+https://github.com/JustVugg/colibri/pull/757 # inkling: shared experts to the GPU, 1.75 to 2.46 tok/s on Apple Silicon
+https://github.com/JustVugg/colibri/pull/763 # metal: GPU attention for S>4 prefill (single command buffer)
+
+
diff --git a/docs/pr_116.md b/docs/pr_116.md
new file mode 100644
index 00000000..bb0f73ed
--- /dev/null
+++ b/docs/pr_116.md
@@ -0,0 +1,23 @@
+# pr_116: docs: Metal backend (#72) β€” M5 Max performance report, OMP-spin regression + fix
+
+- **URL:** https://github.com/JustVugg/colibri/pull/116
+- **State:** MERGED 2026-07-13 (opened 2026-07-13 by RDouglasSharp)
+
+## Summary
+
+Adds `docs/METAL-M5MAX-PERF-REPORT.md` β€” the independent M5 Max reproduction of the rebased Metal backend (#72), and the source of the Config A-E methodology the M1 Ultra plan mirrors.
+
+- Tuned, the rebased Metal path reaches **2.24 tok/s β€” ~8.5% faster than the pre-rebase branch (2.06, issue #103)**, thanks to `PIPE` (new since the old base)
+- The **default** rebased config regresses to **1.25 tok/s (-39%)**: the OMP hot-team active-spin (#77) pegs the CPU while it waits on the GPU, and on Apple's shared SoC power budget that throttles the Metal kernels (attention GPU kernel 76s -> 223s for identical work)
+- Fix is two complementary flags: `COLI_NO_OMP_TUNE=1` restores GPU clocks; `PIPE=1 PIPE_WORKERS=8` hides the CPU->GPU dispatch latency the passive wait introduces
+- Documents the 128 GB memory ceiling: `--ram 110` safe / `120` compresses
+- Token-exactness note: output is non-deterministic run-to-run under the parallel config β€” benign FP-reduction ordering, throughput unaffected (see also #622)
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** **Methodology source.** Configs A-E, the Lucifer/Prometheus workload, the metrics list, and the 2.24 tok/s target all come from this report. The M1 Ultra fmt=2 plan ([metal_fmt2.md](metal_fmt2.md)) mirrors it directly.
+- **Goal b (fmt_4_m1u):** Shared β€” same methodology for the fmt=4 benchmark after fixes.
+
+## See also
+
+- [issue_103](issue_103.md), [pr_72](pr_72.md), [issue_707](issue_707.md), [ultra_benchmark_plan.md](ultra_benchmark_plan.md)
diff --git a/docs/pr_386.md b/docs/pr_386.md
new file mode 100644
index 00000000..cc1329bb
--- /dev/null
+++ b/docs/pr_386.md
@@ -0,0 +1,23 @@
+# pr_386: Metal: platform-aware expert-cache defaults via honest storage probe (#379)
+
+- **URL:** https://github.com/JustVugg/colibri/pull/386
+- **State:** MERGED 2026-07-31 (opened 2026-07-18 by monotophic)
+- **Labels:** metal, needs-rebase
+
+## Summary
+
+Turns the #379/#387 findings into platform-aware defaults. On Apple Silicon with fast NVMe, stock defaults leave a large win on the table: 0.4 tok/s stock vs ~2.0 tuned (`MTP=0 CAP_RAISE=0 --ram 90 --cap 1`).
+
+- Changes the defaults of exactly **two** knobs: `cap` and `CAP_RAISE`. MTP and RAM sizing remain manual β€” stock Metal runs land somewhere between 0.4 and 2.0 tok/s, not at 2.0
+- Adds an **honest storage probe**: whether a volume is "fast" is measured, not assumed β€” buffered reads lie (page-cache hits read 23-97 GB/s; a real `F_NOCACHE` read measured 14 GB/s on the same file)
+- Mechanism (from #379): Metal's per-buffer residency cost makes the engine's expert LRU anti-productive on fast NVMe; the OS page cache streams cold experts back cheaper than Metal can re-pin them
+- Corroborated by #180 (M3 Ultra) and #107 (M4 Pro); explicitly invited by the maintainer (no Metal hardware in-repo)
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** The M1 Ultra benchmark runs on a tree that includes this PR β€” stock cache defaults are now platform-aware. When mirroring Configs A-E, know which knobs are already defaulted and which (`MTP`, `--ram`) remain manual.
+- **Goal b (fmt_4_m1u):** Shared β€” same defaults context for fmt=4 runs.
+
+## See also
+
+- [issue_379](issue_379.md), [issue_387](issue_387.md)
diff --git a/docs/pr_72.md b/docs/pr_72.md
new file mode 100644
index 00000000..c98c6cda
--- /dev/null
+++ b/docs/pr_72.md
@@ -0,0 +1,27 @@
+# pr_72: Metal backend (Apple Silicon): experts + fused attention + full-layer GPU, token-exact
+
+- **URL:** https://github.com/JustVugg/colibri/pull/72
+- **State:** MERGED 2026-07-13 (opened 2026-07-11 by codearranger)
+
+## Summary
+
+The foundational PR: colibri's first GPU tier for Apple Silicon, opt-in via `make glm METAL=1` + `COLI_METAL=1`. Unified memory removes the PCIe copy tax; default build untouched (`#ifdef COLI_METAL`); every GPU path falls back to CPU per-block on any fault or unsupported shape.
+
+What runs on the GPU:
+
+- **Routed-expert SwiGLU**, batched per block, read zero-copy from registered RAM slabs (bindless `gpuAddress`); resident experts' block submitted before missed experts' preads so disk I/O and GPU compute overlap
+- **Fused decode attention** (S<=4, covers MTP verify): full MLA layer β€” q/kv projections, RoPE, absorption core, o_proj β€” in one command buffer
+- **Full-layer command buffer**: in_ln -> attention -> residual -> post_ln -> shared expert -> router (f32 matvec + sigmoid) -> exact phase-A top-K on GPU
+- **Prefill**: large row-batch `matmul_qt` GEMMs route to GPU above `COLI_METAL_GEMM_MIN` (default 16 rows)
+
+Measured (M4 Max 128 GB, warm cache ~58% hit, MTP on, greedy): 0.38-0.43 tok/s Metal vs 0.30 CPU. **Token-exact**: greedy outputs byte-identical to CPU in every configuration. `make metal-test` = 16 standalone correctness tests vs CPU references.
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** The backend being benchmarked. Its gates (fmt=1/2 only) are why the fmt=2 track reproduces #103 without code changes.
+- **Goal b (fmt_4_m1u):** The codebase being fixed β€” all fmt=4 work (fixes A/B/C) extends these kernels and gates.
+
+## See also
+
+- [pr_116](pr_116.md) β€” M5 Max performance report on the rebased backend
+- [metal_dispatch_gap.md](metal_dispatch_gap.md) β€” where this backend rejects fmt=4
diff --git a/docs/pr_750.md b/docs/pr_750.md
new file mode 100644
index 00000000..b042ea17
--- /dev/null
+++ b/docs/pr_750.md
@@ -0,0 +1,22 @@
+# pr_750: inkling: audio input, Metal expert MoE, and tuned Apple defaults
+
+- **URL:** https://github.com/JustVugg/colibri/pull/750
+- **State:** MERGED 2026-08-01 (opened 2026-08-01 by rgbkrk)
+- **Labels:** enhancement, metal, model-support
+
+## Summary
+
+Adds audio input to Inkling (the small multimodal model) and speeds it up on Apple Silicon. Everything is `__APPLE__`/`COLI_METAL` guarded or opt-in; text-only Inkling and non-Apple builds unchanged.
+
+- **Audio input (DMel):** frames embed via a small `[mel_bins*mel_vocab, D]` table + RMSNorm and *replace* `<|audio|>` placeholder embeddings; loads from a `--keep-audio` container or an `audio.safetensors` sidecar (~10 MB via HTTP range on lazy mounts). Entry points: `--audio file.dmel` CLI, 7th serve-protocol field, `"dmel"` in the oracle harness
+- **Metal expert MoE:** routed-expert matmul on the Metal GPU (`COLI_METAL=1`, opt-in) for inkling
+- **Three `__APPLE__` default changes** from a measured knob sweep (not a guess)
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Marginal β€” inkling-specific, but its tuned Apple defaults touch shared `__APPLE__` build defaults that the GLM benchmark binary also sees. Worth knowing it landed before the M1 Ultra runs.
+- **Goal b (fmt_4_m1u):** Not directly relevant; its follow-up [pr_757](pr_757.md) is more interesting (adds an fmt=5 branch to `moe_gemv` β€” a template for the fmt=4 branch).
+
+## See also
+
+- [pr_757](pr_757.md) β€” follow-up: shared experts on GPU
diff --git a/docs/pr_757.md b/docs/pr_757.md
new file mode 100644
index 00000000..9a148c84
--- /dev/null
+++ b/docs/pr_757.md
@@ -0,0 +1,22 @@
+# pr_757: inkling: shared experts to the GPU, 1.75 to 2.46 tok/s on Apple Silicon
+
+- **URL:** https://github.com/JustVugg/colibri/pull/757
+- **State:** OPEN (opened 2026-08-01 by rgbkrk)
+- **Labels:** performance, metal
+
+## Summary
+
+Follow-up to #750 (inkling), but with two techniques worth knowing for GLM Metal work. Same validation discipline: greedy output token-identical through eos, tiny multimodal oracle exact at every step. Inkling-Small on Apple Silicon: **1.75 -> 2.46 tok/s decode, 10.3 -> 6.4 s prefill**.
+
+- **Commit 1 β€” overlap shared experts with the last GPU round:** routed and shared experts are independent given the layer input; the last routed round submits with `moe_block_begin`/`_end` and the CPU computes shared experts while the GPU is in flight. `shared_done` flag guards double-compute on GPU fault. No new math, no new kernel. 1.75 -> 1.96 tok/s.
+- **Commit 2 β€” shared experts on GPU as a bf16 block:** `moe_gemv` gains **fmt=5** (raw bf16 rows, no scales, dequant is a 16-bit shift into the upper half of an f32). Shared-expert weights move into one page-aligned slab per sparse layer; `Wt` handles repoint into the slab so the CPU fallback reads the same bytes. One fmt=5 block submitted per MoE layer before the routed rounds. Shared experts drop 8.0s -> 0.1s on the probe. 1.96 -> 2.46 tok/s.
+- The fmt=5 branch is additive; glm/kimi call sites never pass fmt=5 (dead code for them). `INK_METAL_SHARED=0` opts out.
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Perf technique reference β€” CPU/GPU overlap of shared experts and GPU-resident shared experts are candidate levers for beating 2.24 tok/s on M1 Ultra (GLM has 1 shared expert/layer currently on CPU).
+- **Goal b (fmt_4_m1u):** **Code template** β€” the fmt=5 branch shows exactly how to add a new format branch to `moe_gemv` without touching existing paths, which is what Fix A (fmt=4 branch) needs.
+
+## See also
+
+- [pr_750](pr_750.md), [metal_fmt4.md](metal_fmt4.md)
diff --git a/docs/pr_763.md b/docs/pr_763.md
new file mode 100644
index 00000000..c55506db
--- /dev/null
+++ b/docs/pr_763.md
@@ -0,0 +1,23 @@
+# pr_763: metal: GPU attention for S>4 prefill (single command buffer)
+
+- **URL:** https://github.com/JustVugg/colibri/pull/763
+- **State:** OPEN (opened 2026-08-01 by RDouglasSharp)
+- **Labels:** performance, metal, quality
+
+## Summary
+
+Lifts the `S<=4` cap on the Metal fused-attention path (`coli_metal_attn_decode`) so large-S **prefill** attention runs on the GPU in a single command buffer, cutting prefill attention time ~4x with output consistent with the CPU reference.
+
+- **Off by default** β€” `COLI_METAL_PREFILL=1` opts in β€” because GPU prefill can pick a different token on near-tie logits (the #622 family), so a greedy stream is not guaranteed bit-identical to CPU
+- `c/colibri.c`: fused-attention gate no longer hard-caps at S<=4; engageable for any S when Metal absorption is active; adds a cumulative timer to the `[prefill]` progress line
+- `c/backend_metal.mm`: projections -> attention core -> output GEMV in one command buffer ordered by memory barriers (same structure as the token-exact S<=4 path); single `a_score` dispatch is `S*AHEADS*T` threads, guarded under ~2^30, CPU fallback for very large prompts
+- `docs/metal.md` documents the flag. CPU and CUDA paths unchanged
+
+## Relevance
+
+- **Goal a (fmt_2_m1u):** Optional prefill speedup for the benchmark (~4x prefill attention) β€” with the documented near-tie caveat; keep it off for an exact-reproduction claim, or on for a max-speed claim.
+- **Goal b (fmt_4_m1u):** Interacts with the same fused-attention gate that Fix C relaxes for fmt=4 β€” if both land, the fmt=4 gate change and the S>4 change must be reconciled in the same `if` conditions.
+
+## See also
+
+- [issue_622](issue_622.md), [pr_72](pr_72.md), [metal_fmt4.md](metal_fmt4.md)
diff --git a/docs/ultra_benchmark_plan.md b/docs/ultra_benchmark_plan.md
new file mode 100644
index 00000000..a5e19006
--- /dev/null
+++ b/docs/ultra_benchmark_plan.md
@@ -0,0 +1,235 @@
+# M1 Ultra Metal Benchmark Plan
+
+## Objective
+
+Benchmark colibri on M1 Ultra (128GB unified memory) using the same methodology as the M5 Max report, with the goal of meeting or exceeding the M5 Max's 2.06 tok/s baseline.
+
+**Critical Context:** Two separate benchmark tracks exist due to the fmt=4 Metal backend gap:
+- **fmt=2 (per-row int4):** Full Metal support available β€” see [`metal_fmt2.md`](metal_fmt2.md)
+- **fmt=4 (grouped int4):** Metal backend fixes required β€” see [`metal_fmt4.md`](metal_fmt4.md)
+
+---
+
+## Hardware Context
+
+- **Machine:** Apple M1 Ultra (2Γ— M1 Max dies, 20+10 CPU cores, 48+32 GPU cores)
+- **Memory:** 128GB unified (fabric-connected, ~512GB/s bandwidth per die)
+- **SSD:** Internal (typically 7GB/s sequential, ~3-4GB/s random 4K)
+- **Target:** β‰₯2.06 tok/s (M5 Max baseline), ideally 2.2+ tok/s
+
+---
+
+## Track Selection
+
+### fmt=2 Track (Per-Row Int4)
+
+**Status:** Ready to benchmark immediately
+
+**Model:** Converted from official fmt=4 container using `--group-size 0`
+
+**Advantage:** Full Metal support (no code changes needed)
+
+**Tradeoff:** ~9pp lower quality than fmt=4 (grouped scales)
+
+**Plan:** [`docs/metal_fmt2.md`](metal_fmt2.md)
+
+---
+
+### fmt=4 Track (Grouped Int4)
+
+**Status:** Requires Metal backend implementation
+
+**Model:** Official GLM-5.2-colibri-int4-g64-with-int8-mtp
+
+**Advantage:** Higher quality (official container format)
+
+**Requirement:** Implement fixes A+B+C from [`metal_dispatch_gap.md`](metal_dispatch_gap.md)
+
+**Plan:** [`docs/metal_fmt4.md`](metal_fmt4.md)
+
+---
+
+## Prerequisites (Both Tracks)
+
+### 1. Model Availability
+
+- **fmt=4:** `~/mlx-models/GLM-5.2-colibri-int4-g64-with-int8-mtp` (already present)
+- **fmt=2:** Convert from fmt=4 using plan in [`metal_fmt2.md`](metal_fmt2.md#phase-1)
+
+### 2. Build with Metal
+
+```bash
+make clean && make METAL=1
+```
+
+### 3. Verify Metal Backend
+
+```bash
+make metal-test
+```
+
+All 30+ kernel tests should pass (int8/int4/int2 matmul, MoE block, large-batch GEMM, fused attention).
+
+**Note:** Current tests cover fmt=1/2/3 only. fmt=4 tests added as part of [`metal_fmt4.md`](metal_fmt4.md#phase-4).
+
+---
+
+## Benchmark Methodology (Both Tracks)
+
+### Standard Configurations
+
+Mirror M5 Max Configs A–E from [`METAL-M5MAX-PERF-REPORT.md`](METAL-M5MAX-PERF-REPORT.md):
+
+| Config | Flags | Purpose |
+|--------|-------|---------|
+| **A** | `COLI_METAL=1 DIRECT=1 MTP=0` | Metal baseline |
+| **B** | Default (no extra flags) | Check for OMP spin regression |
+| **C** | `+ PIPE=1 PIPE_WORKERS=8` | Add I/O overlap |
+| **D** | `+ COLI_NO_OMP_TUNE=1` | Kill OMP active-spin |
+| **E** | `D + C` | Combined optimal |
+
+### Workload
+
+- **Prompt:** "Compare the myths of Lucifer and Prometheus"
+- **Tokens:** 1024
+- **Rationale:** Matches M5 Max methodology for comparability
+
+### Metrics to Record
+
+For each configuration:
+- tok/s
+- wall time (s)
+- expert-disk (s)
+- expert-matmul (s)
+- attention (s)
+- attn GPU kernel (s)
+- expert GPU overhead (s)
+- expert hit rate (%)
+- RSS (GB)
+
+---
+
+## M1 Ultra Specific Tests
+
+### PIPE_WORKERS Scaling
+
+M1 Ultra has 32 CPU cores (vs M5 Max's 18). Test higher worker counts:
+
+```bash
+# Beyond PIPE_WORKERS=8
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=12 ./coli run ...
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=16 ./coli run ...
+```
+
+### MTP (Multi-Token Prediction)
+
+M5 Max report used MTP=0. Test if MTP=1 helps on M1 Ultra:
+
+```bash
+COLI_METAL=1 DIRECT=1 MTP=1 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 ./coli run ...
+```
+
+### Memory Pressure
+
+Find M1 Ultra's compression knee:
+
+```bash
+# Test --ram 115, --ram 120
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ... --ram 115 ...
+COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 \
+ ./coli run --model ... --ram 120 ...
+```
+
+---
+
+## Documentation
+
+### 6.1 Create Performance Report
+
+**fmt=2 track:** [`METAL-M1ULTRA-FMT2-REPORT.md`](metal_fmt2.md#phase-5)
+
+**fmt=4 track:** [`METAL-M1ULTRA-FMT4-REPORT.md`](metal_fmt4.md#phase-6)
+
+Mirror M5 Max report structure:
+- Setup section (hardware, model, workload, flags)
+- Results table (configs A-E + M1-specific variants)
+- Phase-by-phase analysis
+- Recommendations
+- Caveats/untested levers
+
+### 6.2 Update benchmarks.md
+
+Add row to community benchmarks table:
+
+```markdown
+| Apple M1 Ultra (32 cores) Β· macOS Β· 128 GB unified Β· internal SSD | ~5-6 GB/s cold | Metal on Β· `--ram 110` Β· COLI_NO_OMP_TUNE=1 Β· PIPE=1 | **X.XX tok/s** Β· hit YY% Β· [PR #NNN] |
+```
+
+**Note:** Create separate rows for fmt=2 and fmt=4 results if both tracks complete.
+
+---
+
+## Success Criteria
+
+### fmt=2 Track
+
+1. **Primary:** Achieve β‰₯2.06 tok/s (match M5 Max baseline)
+2. **Stretch:** Achieve β‰₯2.24 tok/s (beat M5 Max)
+3. **Documentation:** Complete METAL-M1ULTRA-FMT2-REPORT.md
+4. **PR:** Submit report + benchmarks.md update
+
+### fmt=4 Track
+
+1. **Primary:** Metal dispatch confirmed (METAL-ATTN and METAL lines present)
+2. **Stretch:** Achieve β‰₯2.06 tok/s (match M5 Max with fmt=4)
+3. **Documentation:** Complete METAL-M1ULTRA-FMT4-REPORT.md
+4. **PR:** Submit report + benchmarks.md update
+
+---
+
+## Execution Order
+
+### Recommended Sequence
+
+1. **fmt=2 track first** β€” Immediate benchmark, establishes speed baseline
+2. **fmt=4 track second** β€” After backend fixes, measures quality-correct performance
+
+**Rationale:** fmt=2 requires no code changes and can be benchmarked immediately. fmt=4 requires implementing fixes A+B+C from [`metal_dispatch_gap.md`](metal_dispatch_gap.md).
+
+### Parallel Execution
+
+If resources allow, both tracks can run in parallel:
+- fmt=2: Benchmarking while fmt=4: Implementation in progress
+
+---
+
+## References
+
+### Core Documents
+
+- [`metal_dispatch_gap.md`](metal_dispatch_gap.md) β€” fmt=4 backend gap problem statement
+- [`metal_fmt2.md`](metal_fmt2.md) β€” fmt=2 conversion and benchmark plan
+- [`metal_fmt4.md`](metal_fmt4.md) β€” fmt=4 backend fix implementation plan
+
+### Methodology Reference
+
+- [`METAL-M5MAX-PERF-REPORT.md`](METAL-M5MAX-PERF-REPORT.md) β€” M5 Max benchmark methodology
+- [`README.md`](../README.md) β€” Dual-SSD, PIPE, COLI_NO_OMP_TUNE tuning guidance
+
+### Related Issues
+
+- **#457** β€” Metal fmt=4 GEMV support (MERGED)
+- **#585** β€” METAL Attention support for fmt:4 (OPEN)
+- **#587** β€” Metal fmt=4 decode: attention + routed experts (OPEN, needs-rebase)
+
+---
+
+## Next Steps
+
+**Choose your track:**
+
+1. **fmt=2 (immediate):** Follow [`metal_fmt2.md`](metal_fmt2.md) β€” convert model and benchmark
+2. **fmt=4 (backend fix):** Follow [`metal_fmt4.md`](metal_fmt4.md) β€” implement fixes A+B+C, then benchmark
+
+**Both tracks contribute to the same goal:** Understanding M1 Ultra's performance potential with colibri, with different tradeoffs between speed (fmt=2) and quality (fmt=4).